Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .github/scripts/test/cpp-build-cache.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -153,13 +153,16 @@ test('AudioGen native build restores caches, limits builds and propagates failur
assert.match(source, /configure-cpp-build.mjs/)
assert.match(source, /npm run build:native/)
assert.match(source, /npm install --ignore-scripts/)
assert.match(source, /No addon-level C\+\+ tests yet/)
assert.match(source, /npm run test:cpp/)
assert.ok(manifest.scripts['test:cpp'])
assert.match(source, /npm run fuzz:build/)
assert.match(source, /npm run fuzz:run/)
assert.doesNotMatch(source, /continue-on-error/)
const saveCcache = source.indexOf('name: Save ccache (trusted contexts)')
const unitTests = source.indexOf('name: Run addon C++ unit tests')
const fuzzBuild = source.indexOf('name: Build fuzz targets')
assert.ok(saveCcache > 0 && fuzzBuild > saveCcache, 'fuzz reconfigure must follow the native cache save')
assert.ok(saveCcache > 0 && unitTests > saveCcache, 'test reconfigure must follow the native cache save')
assert.ok(fuzzBuild > unitTests, 'fuzz reconfigure must follow the unit tests')
const result = spawnSync('bash', ['--noprofile', '--norc', '-e', '-o', 'pipefail', '-c',
'npm() { return 134; }\nnpm run build:native'], { encoding: 'utf8' })
assert.equal(result.status, 134, result.stderr)
Expand Down
18 changes: 10 additions & 8 deletions .github/workflows/cpp-test-coverage-audiogen-ggml.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name: CPP Tests (AudioGen GGML)

# Cache the native build. Addon-level C++ unit tests are still a stub.
# After those caches are saved, this lane runs the bounded config-parse fuzz target.
# Cache the native build. After those caches are saved, this lane runs the
# addon-level C++ unit tests and then the bounded config-parse fuzz target.
on:
push:
branches: [main]
Expand Down Expand Up @@ -220,19 +220,21 @@
key: ${{ steps.ccache-cache.outputs.cache-primary-key }}
path: ${{ env.CCACHE_DIR }}

- name: No addon-level C++ tests yet
run: |
echo 'AudioGen native build completed; no addon-level C++ unit tests yet.' >> "$GITHUB_STEP_SUMMARY"
echo 'ACE-Step engine unit tests run in qvac-fabric-speech.cpp audiogen CI.' >> "$GITHUB_STEP_SUMMARY"

- name: Show ccache statistics
if: always() && !cancelled() && steps.cpp-build.outcome == 'success'
run: ccache -s

# fuzz:build reconfigures the same tree, so it runs after the native
# test:cpp reconfigures the tree with BUILD_TESTING=ON, so it runs after
# the native build's caches are saved. These are the addon's own unit
# tests; the ACE-Step engine's run in qvac-fabric-speech.cpp audiogen CI.
- name: Run addon C++ unit tests
working-directory: ${{ matrix.workdir }}
run: npm run test:cpp

# fuzz:build reconfigures the same tree again, so it runs after the native
# build's caches are saved. Bounded unit-test mode. A failure fails the
# job that publishes qvac/cpp-tests-audiogen-ggml.
- name: Build fuzz targets

Check failure

Code scanning / CodeQL

Cache Poisoning via execution of untrusted code High

Potential cache poisoning in the context of the default branch due to privilege checkout of untrusted code from
inputs.ref || github.ref
. (
schedule
).
Potential cache poisoning in the context of the default branch due to privilege checkout of untrusted code from
inputs.ref || github.ref
. (
workflow_dispatch
).
Potential cache poisoning in the context of the default branch due to privilege checkout of untrusted code from
inputs.ref || github.ref
. (
schedule
).
Potential cache poisoning in the context of the default branch due to privilege checkout of untrusted code from
inputs.ref || github.ref
. (
workflow_dispatch
).
Potential cache poisoning in the context of the default branch due to privilege checkout of untrusted code from
inputs.ref || github.ref
. (
workflow_dispatch
).
working-directory: ${{ matrix.workdir }}
run: npm run fuzz:build

Expand Down
6 changes: 6 additions & 0 deletions packages/audiogen-ggml/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

- ACE-Step `normalizeLoudness: false` returns the engine's raw output. The
addon rescaled every non-edit result to a 0.9 peak whatever the option said,
so the flag only switched off the engine's percentile pass. Samples the VAE
leaves outside `[-1, 1]` now saturate at int16 full scale on that path.
- ACE-Step lego stems keep their mix gain relative to the source; the same
rescale raised or lowered each stem to a 0.9 peak.
- Include independent LM device pools in `assessFit()` when the primary
pipeline runs on CPU, counting the host peak once.

Expand Down
1 change: 1 addition & 0 deletions packages/audiogen-ggml/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,7 @@ if(BUILD_TESTING)

add_executable(audiogen_ggml_tests
addon/tests/test_app_memory_budget.cpp
addon/tests/test_output_gain.cpp
Comment thread
greptile-apps[bot] marked this conversation as resolved.
Resolved
addon/tests/test_runtime_options.cpp
addon/tests/test_fit_result.cpp)
target_compile_definitions(audiogen_ggml_tests PRIVATE
Expand Down
2 changes: 1 addition & 1 deletion packages/audiogen-ggml/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -739,7 +739,7 @@ wrapped by a level-gated `QvacLogger`.
| `lmPhase1` | Allow the LM to infer missing metadata before generating semantic codes. |
| `simpleMode` | Expand the caption query into a full request (caption, lyrics, unset metadata) before synthesis. |
| `rewriteQuery` | LM FORMAT pass: rewrite the caption into a detailed description, preserving the lyric content. Requires real `lyrics` and `taskType: 'text2music'`; mutually exclusive with `simpleMode`. |
| `normalizeLoudness` | Percentile loudness normalization of generated audio (default `true`); edits are never normalized. |
| `normalizeLoudness` | Percentile loudness normalization of generated audio, then a 0.9 peak (default `true`); `false` returns the raw engine output. Edits and lego stems are never normalized. |
| `generateLrc` | Synchronized lyric timestamps: `stats.lrc` (LRC text) + `stats.lyricsScore`. Requires lyrics and `taskType: 'text2music'`. |
| `computeQualityScore` | Teacher-forced LM quality score of the generated codes; `stats.qualityScore` in `[0, 1]`. Requires `taskType: 'text2music'`. |
| `dcwEnabled` / `dcwScaler` / `dcwHighScaler` | Haar DCW correction controls. |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
#include "audiogen-cpp/acestep/engine.h"
#include "model-interface/BackendUtils.hpp"
#include "model-interface/EngineOptions.hpp"
#include "model-interface/acestep/OutputGain.hpp"

namespace qvac::audiogenggml::acestep {

Expand Down Expand Up @@ -329,28 +330,16 @@ AcestepModel::Output AcestepModel::generate(const AnyInput& in) {
out.metadata = std::move(metadata);
}

// Peak-normalise before the int16 quantisation, exactly like the music CLI's
// wav_write (gain = 0.9 / peak). The Oobleck VAE routinely outputs float
// samples slightly outside [-1, 1]; converting those straight to int16 hard-
// clips at full scale and produces the harsh, distorted "horrible" audio the
// addon path had (vs. the clean CLI output). Normalising to -0.9 dBFS keeps
// headroom and removes the clipping. Single-shot output, so a 2-pass over the
// full track is trivial.
//
// Always normalise non-silent output to a fixed -0.9 dBFS peak
// (gain = 0.9 / peak whenever peak > kMinNormPeak): this removes clipping and
// gives every track consistent headroom vs. the raw engine output. A
// silent/near-silent result (peak below the threshold) is left untouched:
// seeding the peak at ~0 and dividing would turn low-level noise into a
// ~-0.9 dBFS blast. kMinNormPeak ~= -60 dBFS.
constexpr float kMinNormPeak = 1e-3F;
float peak = 0.0F;
for (float s : result.pcm)
peak = std::fmax(peak, std::fabs(s));
// Edit plans promise exact preservation outside Repaint regions. Applying
// whole-track peak normalization here would modify every preserved sample.
const float gain =
in.editOperations.empty() && peak > kMinNormPeak ? 0.9F / peak : 1.0F;
// Edit plans promise exact preservation outside Repaint regions, a lego stem
// keeps its mix gain relative to the source, and `normalizeLoudness: false`
// asks for the engine's raw output, so only a normalized generation is
// rescaled here (see OutputGain.hpp).
const float gain = outputGain(
result.pcm,
OutputGainRequest{
.normalizeLoudness = in.normalizeLoudness,
.isEdit = !in.editOperations.empty(),
.isLegoStem = in.taskType == "lego"});

out.pcm.reserve(result.pcm.size());
for (float s : result.pcm)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
#pragma once

#include <cmath>
#include <vector>

namespace qvac::audiogenggml::acestep {

// Peak the addon scales a normalized generation to before the int16
// quantisation, and the peak below which a result counts as silence.
// MIN_NORMALIZATION_PEAK ~= -60 dBFS.
constexpr float TARGET_OUTPUT_PEAK = 0.9F;
constexpr float MIN_NORMALIZATION_PEAK = 1e-3F;

// What the addon knows about a run when it decides the output gain.
struct OutputGainRequest {
// `normalizeLoudness` on the job (default true).
bool normalizeLoudness = true;
// The run executed an edit plan (FlowEdit / Repaint).
bool isEdit = false;
// The run generated a lego stem.
bool isLegoStem = false;
};

// The engine only normalizes loudness when the caller asked for it, and never
// for an edit (Repaint preserves untouched source regions bit-for-bit) or a
// lego stem (it keeps its mix gain relative to the source). The addon follows
// the same rule, so those three outputs reach the caller at the engine's gain.
inline bool normalizesOutput(const OutputGainRequest& request) {
return request.normalizeLoudness && !request.isEdit && !request.isLegoStem;
}

// Gain applied to the engine's float PCM before the int16 quantisation.
//
// A normalized generation is scaled to a fixed TARGET_OUTPUT_PEAK
// (gain = 0.9 / peak): the engine's percentile normalization leaves the peak
// at full scale, and the headroom keeps the int16 conversion from clipping. A
// silent or near-silent result is left untouched, since dividing by a ~0 peak
// would turn low-level noise into a full-scale blast.
//
// Every other output keeps gain 1.0; samples the VAE left outside [-1, 1] then
// saturate at int16 full scale.
inline float
outputGain(const std::vector<float>& pcm, const OutputGainRequest& request) {
if (!normalizesOutput(request))
return 1.0F;
float peak = 0.0F;
for (const float sample : pcm)
peak = std::fmax(peak, std::fabs(sample));
return peak > MIN_NORMALIZATION_PEAK ? TARGET_OUTPUT_PEAK / peak : 1.0F;
}

} // namespace qvac::audiogenggml::acestep
42 changes: 42 additions & 0 deletions packages/audiogen-ggml/addon/tests/test_output_gain.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
#include <vector>

#include <gtest/gtest.h>

#include "model-interface/acestep/OutputGain.hpp"

using qvac::audiogenggml::acestep::outputGain;
using qvac::audiogenggml::acestep::OutputGainRequest;

namespace {
// Peaks at 1.25: outside [-1, 1], as the VAE's raw output routinely is.
const std::vector<float> LOUD_PCM{0.5F, -1.25F, 0.25F, 1.0F};
} // namespace

TEST(OutputGainTest, ScalesANormalizedGenerationToTheTargetPeak) {
EXPECT_FLOAT_EQ(outputGain(LOUD_PCM, OutputGainRequest{}), 0.9F / 1.25F);
}

TEST(OutputGainTest, LeavesRawOutputAloneWhenNormalizationIsOff) {
EXPECT_EQ(
outputGain(LOUD_PCM, OutputGainRequest{.normalizeLoudness = false}),
1.0F);
}

TEST(OutputGainTest, KeepsALegoStemAtItsMixGain) {
EXPECT_EQ(outputGain(LOUD_PCM, OutputGainRequest{.isLegoStem = true}), 1.0F);
}

TEST(OutputGainTest, PreservesEditOutput) {
EXPECT_EQ(outputGain(LOUD_PCM, OutputGainRequest{.isEdit = true}), 1.0F);
}

TEST(OutputGainTest, DoesNotAmplifySilence) {
const std::vector<float> nearSilent{0.0F, 5e-4F, -5e-4F};
EXPECT_EQ(outputGain(nearSilent, OutputGainRequest{}), 1.0F);
EXPECT_EQ(outputGain({}, OutputGainRequest{}), 1.0F);
}

TEST(OutputGainTest, AttenuatesAQuietGenerationUpToTheTargetPeak) {
const std::vector<float> quiet{0.1F, -0.3F};
EXPECT_FLOAT_EQ(outputGain(quiet, OutputGainRequest{}), 0.9F / 0.3F);
}
7 changes: 5 additions & 2 deletions packages/audiogen-ggml/index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -144,8 +144,11 @@ export interface GenerateOptions {
/**
* Percentile loudness normalization on the generated audio (default true):
* the 99.999th-percentile sample scales to full scale and the tiny tail
* above it clips, matching the reference loudness. Set false for the raw
* engine output. Audio edits are never normalized.
* above it clips, matching the reference loudness; the result is then
* scaled to a 0.9 peak for int16 headroom. Set false for the raw engine
* output at its own gain: samples the VAE leaves outside [-1, 1] saturate
* at int16 full scale. Audio edits and lego stems are never normalized, so
* a stem keeps its mix gain relative to the source.
*/
normalizeLoudness?: boolean;
/**
Expand Down
7 changes: 5 additions & 2 deletions packages/audiogen-ggml/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -202,8 +202,11 @@ export interface GenerateOptions {
/**
* Percentile loudness normalization on the generated audio (default true):
* the 99.999th-percentile sample scales to full scale and the tiny tail
* above it clips, matching the reference loudness. Set false for the raw
* engine output. Audio edits are never normalized.
* above it clips, matching the reference loudness; the result is then
* scaled to a 0.9 peak for int16 headroom. Set false for the raw engine
* output at its own gain: samples the VAE leaves outside [-1, 1] saturate
* at int16 full scale. Audio edits and lego stems are never normalized, so
* a stem keeps its mix gain relative to the source.
*/
normalizeLoudness?: boolean
/**
Expand Down
Loading