Repository navigation
QVAC-27588 fix[audiogen-ggml]: return raw ACE-Step output when normalizeLoudness is off #5603
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
20ec5ca
QVAC-27588 fix: return raw ACE-Step output when normalizeLoudness is …
RamazTs c744f64
QVAC-27588 ci: run the audiogen-ggml C++ unit tests in the cpp-tests …
RamazTs 3c310c6
Merge branch 'main' into fix/QVAC-27588-audiogen-normalize-loudness
RamazTs d41c3c7
Merge branch 'main' into fix/QVAC-27588-audiogen-normalize-loudness
GustavoA1604 174af52
Merge branch 'main' into fix/QVAC-27588-audiogen-normalize-loudness
RamazTs File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
52 changes: 52 additions & 0 deletions
52
packages/audiogen-ggml/addon/src/model-interface/acestep/OutputGain.hpp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| #pragma once | ||
|
|
||
| #include <cmath> | ||
| #include <vector> | ||
|
|
||
| namespace qvac::audiogenggml::acestep { | ||
|
|
||
| // Peak the addon scales a normalized generation to before the int16 | ||
| // quantisation, and the peak below which a result counts as silence. | ||
| // MIN_NORMALIZATION_PEAK ~= -60 dBFS. | ||
| constexpr float TARGET_OUTPUT_PEAK = 0.9F; | ||
| constexpr float MIN_NORMALIZATION_PEAK = 1e-3F; | ||
|
|
||
| // What the addon knows about a run when it decides the output gain. | ||
| struct OutputGainRequest { | ||
| // `normalizeLoudness` on the job (default true). | ||
| bool normalizeLoudness = true; | ||
| // The run executed an edit plan (FlowEdit / Repaint). | ||
| bool isEdit = false; | ||
| // The run generated a lego stem. | ||
| bool isLegoStem = false; | ||
| }; | ||
|
|
||
| // The engine only normalizes loudness when the caller asked for it, and never | ||
| // for an edit (Repaint preserves untouched source regions bit-for-bit) or a | ||
| // lego stem (it keeps its mix gain relative to the source). The addon follows | ||
| // the same rule, so those three outputs reach the caller at the engine's gain. | ||
| inline bool normalizesOutput(const OutputGainRequest& request) { | ||
| return request.normalizeLoudness && !request.isEdit && !request.isLegoStem; | ||
| } | ||
|
|
||
| // Gain applied to the engine's float PCM before the int16 quantisation. | ||
| // | ||
| // A normalized generation is scaled to a fixed TARGET_OUTPUT_PEAK | ||
| // (gain = 0.9 / peak): the engine's percentile normalization leaves the peak | ||
| // at full scale, and the headroom keeps the int16 conversion from clipping. A | ||
| // silent or near-silent result is left untouched, since dividing by a ~0 peak | ||
| // would turn low-level noise into a full-scale blast. | ||
| // | ||
| // Every other output keeps gain 1.0; samples the VAE left outside [-1, 1] then | ||
| // saturate at int16 full scale. | ||
| inline float | ||
| outputGain(const std::vector<float>& pcm, const OutputGainRequest& request) { | ||
| if (!normalizesOutput(request)) | ||
| return 1.0F; | ||
| float peak = 0.0F; | ||
| for (const float sample : pcm) | ||
| peak = std::fmax(peak, std::fabs(sample)); | ||
| return peak > MIN_NORMALIZATION_PEAK ? TARGET_OUTPUT_PEAK / peak : 1.0F; | ||
| } | ||
|
|
||
| } // namespace qvac::audiogenggml::acestep |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| #include <vector> | ||
|
|
||
| #include <gtest/gtest.h> | ||
|
|
||
| #include "model-interface/acestep/OutputGain.hpp" | ||
|
|
||
| using qvac::audiogenggml::acestep::outputGain; | ||
| using qvac::audiogenggml::acestep::OutputGainRequest; | ||
|
|
||
| namespace { | ||
| // Peaks at 1.25: outside [-1, 1], as the VAE's raw output routinely is. | ||
| const std::vector<float> LOUD_PCM{0.5F, -1.25F, 0.25F, 1.0F}; | ||
| } // namespace | ||
|
|
||
| TEST(OutputGainTest, ScalesANormalizedGenerationToTheTargetPeak) { | ||
| EXPECT_FLOAT_EQ(outputGain(LOUD_PCM, OutputGainRequest{}), 0.9F / 1.25F); | ||
| } | ||
|
|
||
| TEST(OutputGainTest, LeavesRawOutputAloneWhenNormalizationIsOff) { | ||
| EXPECT_EQ( | ||
| outputGain(LOUD_PCM, OutputGainRequest{.normalizeLoudness = false}), | ||
| 1.0F); | ||
| } | ||
|
|
||
| TEST(OutputGainTest, KeepsALegoStemAtItsMixGain) { | ||
| EXPECT_EQ(outputGain(LOUD_PCM, OutputGainRequest{.isLegoStem = true}), 1.0F); | ||
| } | ||
|
|
||
| TEST(OutputGainTest, PreservesEditOutput) { | ||
| EXPECT_EQ(outputGain(LOUD_PCM, OutputGainRequest{.isEdit = true}), 1.0F); | ||
| } | ||
|
|
||
| TEST(OutputGainTest, DoesNotAmplifySilence) { | ||
| const std::vector<float> nearSilent{0.0F, 5e-4F, -5e-4F}; | ||
| EXPECT_EQ(outputGain(nearSilent, OutputGainRequest{}), 1.0F); | ||
| EXPECT_EQ(outputGain({}, OutputGainRequest{}), 1.0F); | ||
| } | ||
|
|
||
| TEST(OutputGainTest, AttenuatesAQuietGenerationUpToTheTargetPeak) { | ||
| const std::vector<float> quiet{0.1F, -0.3F}; | ||
| EXPECT_FLOAT_EQ(outputGain(quiet, OutputGainRequest{}), 0.9F / 0.3F); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.