diff --git a/docs/website/content/_unpublished/sdk/next/ai-capabilities/text-to-speech.mdx b/docs/website/content/_unpublished/sdk/next/ai-capabilities/text-to-speech.mdx index 8ada14106ef..79bdcd40bd6 100644 --- a/docs/website/content/_unpublished/sdk/next/ai-capabilities/text-to-speech.mdx +++ b/docs/website/content/_unpublished/sdk/next/ai-capabilities/text-to-speech.mdx @@ -9,7 +9,7 @@ schemaType: HowTo Text-to-Speech uses [`@qvac/tts-ggml`](https://github.com/tetherto/qvac/tree/main/packages/tts-ggml) (GGML) as the inference engine. Load any supported model using `modelType: "tts"`. Then, provide `text` as input (with `inputType: "text"`) to generate speech audio. MOSS-SoundEffect also generates sound effects from descriptions, and MOSS-Speech accepts text or audio questions to generate a reply. -`textToSpeech()` returns synchronously with `buffer`, `bufferStream`, `done`, `requestId`, `text`, `sampleRate`, `stats` and `stopReason`; see [Audio output](#audio-output), [Cancelling a run](#cancelling-a-run) and [Runtime statistics](#runtime-statistics) below. +`textToSpeech()` returns synchronously with `buffer`, `bufferStream`, `chunkStream`, `done`, `requestId`, `text`, `sampleRate`, `stats`, `stopReason` and `speechTokenIds`; see [Audio output](#audio-output), [Cancelling a run](#cancelling-a-run) and [Runtime statistics](#runtime-statistics) below. ## Functions @@ -25,6 +25,19 @@ For how to use each function, see [SDK — API reference](/sdk/reference/api/). The SDK returns raw, mono, signed 16-bit PCM samples as a plain `number[]`. With `stream: false`, the `buffer` promise resolves to the complete sample array and `bufferStream` is empty. With `stream: true`, `buffer` resolves to `[]` and `bufferStream` yields individual samples as they become available. The data has no WAV header or other container metadata; use the [example utility](#utils) to write it as a WAV file. +`chunkStream` is the same audio one frame at a time: an `AsyncGenerator` that yields one typed array of signed 16-bit samples per response frame (per sentence with `sentenceStream: true`), so a player can hand each chunk to an audio device without rebuilding it from single samples. The samples are integers in the int16 range, not floats; scale by `1 / 32768` for an API that expects `[-1, 1]`. `bufferStream` and `chunkStream` share one run in plain streaming mode, so iterate one of the two; with `stream: false` both are empty and `buffer` holds the audio. + +```ts +const result = textToSpeech({ modelId, text: 'Hello.' }) +for await (const chunk of result.chunkStream) { + // The run starts when chunkStream is first iterated, so sampleRate + // resolves once the first chunk is in; await it inside the loop. + const sampleRate = await result.sampleRate + // chunk: Int16Array, one per engine frame + player.write(chunk, sampleRate) +} +``` + `textToSpeech()` also returns a `sampleRate` promise, resolved from the first audio frame the engine emits. Prefer it over inferring the rate from the load config — `outputSampleRate` and the LavaSR enhancer both change it. It resolves @@ -85,13 +98,13 @@ statistics from the final frame (`undefined` if the run reported none). The chunked paths — `stream: true` (the default), `sentenceStream: true`, - and `textToSpeechStream()` — run one native job per sentence. Their `stats` - aggregate the timing and sample counts across sentences, while the backend - fields (`backendDevice`, `backendId`, `gpuUnsupported`, and the LavaSR - `enhancerBackend*` and `denoiserBackend*` pairs) and Audio8's two Core ML - fields keep the value of the last sentence that reported them, so a chunked - run reports where it ran just as batch synthesis (`stream: false`) does. A - field no sentence reported stays absent. + and `textToSpeechStream()` — run one native job per sentence. Their + aggregate sums the timing and sample counts, keeps `firstAudioMs` from the + first sentence and the backend and Core ML fields from the last sentence + that reported them, and carries no per-stage timings or counters: those + describe one native synthesis, so they are reported by batch synthesis + (`stream: false`) only. MOSS is the exception for `stream: true`: it runs + the whole text as one native job, so it reports everything there too. | Field | Meaning | @@ -103,20 +116,52 @@ statistics from the final frame (`undefined` if the run reported none). | `totalSamples` | PCM sample count. | | `generatedFrames` | Audio8 and MOSS only: codec frames generated (Audio8 on a fixed 46 ms grid, MOSS at 12.5 per second). | | `backendDevice` | Compute device chosen at load: `0` CPU, `1` GPU. | -| `backendId` | Backend family: `0` CPU, `1` Metal, `2` CUDA, `3` Vulkan, `4` OpenCL, `99` other GPU. | +| `backendId` | Backend family: `0` CPU, `1` Metal, `2` CUDA, `3` Vulkan, `4` OpenCL, `5` WebGPU, `99` other GPU. | | `gpuUnsupported` | `1` when a GPU is present but unusable by engine policy, so the run fell back to CPU. | | `enhancerBackendDevice` | LavaSR enhancer device: `-1` not loaded, `0` CPU, `1` GPU, `2` Core ML-managed. | -| `enhancerBackendId` | LavaSR enhancer backend family, same codes as `backendId`. | +| `enhancerBackendId` | LavaSR enhancer backend family, same codes as `backendId` (`99` for a Core ML-managed stage). | | `denoiserBackendDevice` | LavaSR denoiser device: `-1` not loaded, `0` CPU, `1` GPU, `2` Core ML-managed. | -| `denoiserBackendId` | LavaSR denoiser backend family, same codes as `backendId`. | +| `denoiserBackendId` | LavaSR denoiser backend family, same codes as `backendId` (`99` for a Core ML-managed stage). | | `codecSidecarLoaded` | Audio8 on macOS/iOS: `1` while the Core ML codec sidecar is attached, `0` without one or once a failing sidecar was retired. | | `codecOnCoreml` | Audio8: `1` when the synthesis ran its codec on the Core ML sidecar, `0` when it ran on the ggml backend `backendId` names. | +| `decoderSidecarLoaded`, `dacSidecarLoaded` | Parler on macOS/iOS: `1` while the Core ML decoder / DAC sidecar is attached, `0` once a fallback retired it. | +| `decoderOnCoreml`, `dacOnCoreml` | Parler: `1` when the synthesis ran that stage on Core ML, `0` for ggml or a mixed run; a sentence stream reports `1` only when every sentence did. None of the four says which hardware Core ML scheduled the work on. | +| `firstAudioMs` | Milliseconds from the start of the run to the first audio, where the engine reports it. | +| `t3Ms`, `s3genMs`, `t3Tokens` | Chatterbox: T3 decode and S3Gen + HiFT wall times in milliseconds, and the speech tokens T3 emitted. | +| `stageTotalMs` | CosyVoice3 and Audio8: the engine's own wall time in milliseconds, the sum of its stages, excluding the addon's post-processing. | +| `lmPrefillMs`, `lmDecodeMs`, `flowFrontendMs`, `ditEulerMs`, `hiftF0Ms`, `hiftSourceMs`, `hiftStftMs`, `hiftDecodeMs` | CosyVoice3 stage wall times, in milliseconds. | +| `decodeSteps`, `speechTokens`, `flowFrames`, `melFrames`, `textIds`, `promptSpeechTokens` | CosyVoice3 work counters: LM decode steps, speech tokens generated, flow frames in (prompt + generated), mel frames out, text ids in the LM prefill, and the prompt speech tokens the LM continued from. | +| `voiceEncodeMs`, `promptMs`, `prefillMs`, `sampleMs`, `fastDecodeMs`, `slowDecodeMs`, `codecLatentMs`, `codecSynthMs`, `resampleMs` | Audio8 stage wall times, in milliseconds. | ```ts const stats = await result.stats; console.log(stats?.realTimeFactor, stats?.backendId, stats?.gpuUnsupported); ``` +## Backend selection + +Every engine takes `useGPU` (and `nGpuLayers`) to ask for the host's GPU +backend by policy. `modelConfig.backend` names the backend explicitly instead: +`auto`, `cpu`, `opencl`, `webgpu`, or an exact registered device name such as +`MTL0`, `CUDA0` or `Vulkan0` (device names depend on the host). An explicit +backend that is not available fails the load rather than falling back, and +unset keeps the `useGPU` policy. `assessModelFit()` projects the load on the +same device. + +```ts +await loadModel({ + modelSrc: TTS_MULTILINGUAL_SUPERTONIC3_Q8_0, + modelType: "tts", + modelConfig: { ttsEngine: "supertonic", language: "en", backend: "CUDA0" }, +}); +``` + + + WebGPU is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only; + Chatterbox and the MOSS engines reject a `webgpu` selector at load. The + engine reports where it ran in `stats.backendId` (`5` is WebGPU). + + ## Models ### Chatterbox @@ -152,6 +197,31 @@ as on the other engines; `openclCacheDir` (the Android OpenCL program cache) is accepted by Chatterbox, Supertonic and CosyVoice3 only. Chatterbox supports [LavaSR](#lavasr) post-processing. +Instead of (or alongside) `referenceAudioSrc`, `voiceDir` points at a +directory of pre-baked voice-conditioning tensors — the upstream CLI's +`--save-voice` output — which skips the reference preprocessing; when both are +set, tensors missing from the directory are backfilled from the recording. + +T3 sampling is tunable: `temperature` (engine default `0.8`; `0` decodes +greedily), `topK` (`1000`; `0` disables the cutoff), `topP` (`0.95`) and +`repeatPenalty` (`1.2`; `1` turns it off). The multilingual GGUFs also take +`exaggeration` (prosody intensity, default `0.5`; above `0` more expressive, +below `0` flatter), `cfgWeight` (T3 classifier-free-guidance scale, default +`0.5`; `0` disables it, and it is separate from `cfgRate`, which steers S3Gen) +and `minP` (min-p threshold, `0` = off); the Turbo variant ignores the three. + +Length and streaming cost: `nPredict` caps the speech tokens T3 decodes per +synthesis (default `1000`, about 40 s); longer single calls also need `nCtx` +to cover the prompt plus the generated tokens, or set `maxSentenceChars` so +T3 + S3Gen run per segment of at most that many bytes (the reference CLI uses +`180`), with `crossfadeMs` (default `30`) blending the segments in batch +synthesis. `batchCfmSteps` sets the CFM Euler steps for batch synthesis (`0` += the library default: 2-step meanflow on Turbo, the model's full schedule on +multilingual) while `cfmSteps` keeps driving native streaming, and +`streamLeftContextTokens` bounds the per-chunk cost of long native streams by +keeping only that many already-emitted speech tokens as S3Gen left context +(typically 25–50; `0` = the whole emitted prefix). + ### Supertonic Supertonic uses a **single GGUF** via top-level `modelSrc`. Set `voice`, `ttsSpeed`, and `ttsNumInferenceSteps` in `modelConfig` as needed. Multilingual output is selected by the GGUF (e.g. `TTS_MULTILINGUAL_SUPERTONIC2_Q8_0`) plus `language`. @@ -178,7 +248,45 @@ per request — set it in `modelConfig`. `seed`, `threads`, `nGpuLayers`, `outputSampleRate` and `backendsDir` work as on the other engines; `openclCacheDir` is accepted by Chatterbox, Supertonic and CosyVoice3 only, and `vulkanCacheDir` additionally persists -the compiled Vulkan pipeline cache when `useGPU` is set. +the compiled Vulkan pipeline cache when `useGPU` is set. `vulkanDevice` picks +the Vulkan adapter (`0` = the first, `N` = the Nth, `-1` = auto-pick by free +VRAM preferring a discrete GPU) and is also used by the LavaSR enhancer; +`prewarmText` is synthesized once at load so GPU pipelines (and a Core ML +vocoder sidecar) are ready before the first request. + +Voices: `voice` selects a preset baked into the GGUF (`F1`, `M1`, …), and +`voiceJsonSrc` loads an external voice JSON (`{ style_ttl, style_dp }`, e.g. +a cloned voice) that overrides it; the engine checks the tensor sizes against +the loaded model. A voice can also be passed in memory: `voiceStyleTtl` and +`voiceStyleDp` take the two flattened tensors as number arrays (both or +neither), and win over `voiceJsonSrc` and the baked preset. `noiseNpySrc` +supplies a fixed CFM initial-noise `.npy` tensor for reproducible output. + +Precision and environment: `precision` picks the weight precision the engine +loads at (`auto` keeps the GGUF's own; `f32`, `f16` or `q8_0` convert on +load and change the memory footprint, which `assessModelFit()` accounts for). +`f16Attn` and `f16Weights` override the attention and weight precision (`-1` +automatic, `0` off, `1` f16), `kvAttnType` the native KV attention type code, +and `f16WeightsDenyList` lists tensor-name patterns kept at their original +precision. `vulkanEnvOverrides` is a map of string-valued Vulkan environment +overrides handed to the engine; an explicit entry wins over the +`vulkanCacheDir` convenience setting. The addon's `warmUp()` method has no SDK +counterpart: `prewarmText` warms the engine up at load instead. + +Supertonic also streams natively: set `streamChunkTokens` above zero to emit +audio per chunk of that many text tokens (Unicode code points — about 50 for +English, 25–30 for CJK). `streamFirstChunkTokens` shrinks the first chunk for +lower first-audio latency, `streamChunkTolerancePct` (default `20`) is the +window, as a percentage of the chunk target, within which a boundary snaps to +a sentence end, clause or space, and `streamMinChunkTokens` (default `30`) +merges a shorter trailing chunk into the previous one. + + + Supertonic native chunk streaming cannot be combined with the LavaSR + enhancer or denoiser; the pair is rejected at load. Sentence streaming + (`stream: true` without `streamChunkTokens`, `sentenceStream: true`, or + `textToSpeechStream()`) works with either stage loaded. + ### Parler-TTS @@ -278,6 +386,17 @@ async for response in text_to_speech(transport, request): samples.extend(response.buffer) ``` +On macOS and iOS, Parler runs its DAC on a compiled Core ML sidecar +(`-dac.mlmodelc`) found beside the GGUF, and optionally its decoder +steps on `-decoder.mlmodelc` when `coremlDecoder: true`; the sidecars +are not part of the published model set yet, so supply your own beside a local +GGUF. `coremlComputeUnits` (`all`, `cpu_and_ane`, `cpu_and_gpu`, `cpu_only`) +sets the compute policy and `coremlStrict` decides whether an unusable sidecar +fails the load (`true`) or falls back to ggml (`false`); leaving either unset +inherits the engine's environment policy. The `decoderSidecarLoaded`, +`dacSidecarLoaded`, `decoderOnCoreml` and `dacOnCoreml` stats report which +path ran. + ### CosyVoice3 CosyVoice3 uses the **LLM GGUF** as the top-level `modelSrc`; its registry @@ -323,20 +442,63 @@ Conditioning uses one of three controls: CosyVoice3 accepts **one conditioning control per synthesis** — an `emotion`, a non-`moderate` `pace`, or an `instruct`. Combining them is - rejected at load and request validation. `emotion` and `pace` can also be - set per request on `textToSpeech()`; `instruct` is load-time only. + rejected at load and request validation. All three can also be set per + request on `textToSpeech()` and `textToSpeechStream()`, where the request's + control replaces the loaded one for that synthesis. +```ts +const result = textToSpeech({ + modelId, + text: "今天天气真好。", + stream: false, + // Per-request control; a raw string is passed to the engine verbatim. + instruct: { dialect: "cantonese" }, +}); +``` + Set `streamChunkTokens` above zero for native chunk streaming (`streamFirstChunkTokens` tunes the first chunk only). Native streaming emits at 24 kHz, so omit `outputSampleRate` or set it to `24000` unless the LavaSR enhancer is loaded. `useGPU` offloads to Metal on macOS/iOS, Vulkan on -desktop Linux/Windows, and OpenCL on Adreno Android devices; `threads`, -`nGpuLayers`, `seed`, and `outputSampleRate` (8,000–192,000 Hz) work as on -the other engines. CosyVoice3 also supports [LavaSR](#lavasr) -post-processing via `lavasrEnhancerModelSrc` (48 kHz bandwidth extension) and +desktop Linux/Windows, and OpenCL on Adreno Android devices; `vulkanDevice` +picks the Vulkan adapter (`0` = the first, `N` = the Nth, `-1` = auto-pick by +free VRAM preferring a discrete GPU) and is also used by the LavaSR enhancer. +`flowCutPrompt` treats the baked-voice prompt frames as attention-only +conditioning in the flow — a faster flow whose output deviates slightly from +the reference (off by default). `threads`, `nGpuLayers`, `seed`, and +`outputSampleRate` (8,000–192,000 Hz) work as on the other engines. +CosyVoice3 also supports [LavaSR](#lavasr) post-processing via +`lavasrEnhancerModelSrc` (48 kHz bandwidth extension) and `lavasrDenoiserModelSrc` (batch-only; runs before the enhancer). +Reproducible generation: `seed` fixes the sampler, `greedy: true` takes the +argmax speech token instead of sampling, and the run hands its speech-token +trajectory back as `speechTokenIds` (on the result, and on each frame that +carried it: with the PCM in batch synthesis, on a final `isLast` frame with no +audio under native chunk streaming, once per sentence on `chunkUpdates`). +Pass the array of **one native synthesis** back as `forceSpeechTokens` in +`modelConfig` to replay it through the flow and vocoder: a batch run, a native +chunk stream, or a single sentence's array from `chunkUpdates`. An empty array +restores generation. `forceSpeechTokens` is fixed at load and every synthesis +the model runs replays that same array, so a sentence stream's result +(`speechTokenIds` concatenated across its sentences) cannot be replayed as one: +each sentence would start its own synthesis from the whole joined array and the +original segmentation is lost. Replay a sentence stream sentence by sentence +instead, loading the model once per array. + +```ts +const take = textToSpeech({ modelId, text: "Hello.", stream: false }); +await take.buffer; +const tokens = await take.speechTokenIds; // number[] | undefined + +const replay = await loadModel({ + modelSrc: TTS_COSYVOICE3_LLM_COSYVOICE_Q8_0, + modelType: "tts", + modelConfig: { ttsEngine: "cosyvoice3", forceSpeechTokens: tokens }, +}); +``` + #### CosyVoice3 voice cloning Supply `referenceAudioSrc` — a WAV of 0.5–30 s (5–15 s of clean speech is the @@ -417,6 +579,35 @@ supply `referenceAudioSrc` (a reference recording) together with together — a reference without its transcript or without the encoder is rejected at load. +With the encoder loaded, the voice can also change per request: +`textToSpeech()` and `textToSpeechStream()` take `referenceAudioSrc` and +`referenceText` too. A request-level recording replaces the loaded voice for +that synthesis and must bring its own `referenceText` (the loaded transcript +describes a different recording, so it is not inherited); a request-level +`referenceText` on its own corrects the transcript of the loaded recording. +The engine caches the codes of the most recent reference, so repeating one +across requests skips the encoder. + +```ts +const result = textToSpeech({ + modelId, + text: "Spoken in another voice.", + stream: false, + referenceAudioSrc: "/path/to/someone-else.wav", + referenceText: "Exactly what that recording says.", +}); +``` + + + A request-level `referenceAudioSrc` is resolved inside the request. A local + path or an already cached source costs nothing, but a remote source that is + not cached yet downloads first, with no progress events, while the request + holds the model: every other speech request for that model waits behind it. + Pre-fetch remote recordings with `downloadAsset({ assetSrc })`, which fills + the same cache, and pass the same source to the request afterwards. + `cancel()` stops a download that is already in flight. + + Generation is sampled by default: `temperature` (default `0.7`), `topK` (`50`), and `topP` (`0.9`) tune it, `greedy: true` takes the argmax instead and ignores all three, and `maxFrames` caps generation in codec frames @@ -440,7 +631,9 @@ synthesis fails on it, Audio8 runs the codec on ggml instead; the Audio8 has no `emotion`, `pace`, or description fields (at load or per request), no LavaSR post-processing, and no native chunk streaming. Sentence streaming (`sentenceStream: true`) and duplex - `textToSpeechStream()` work as with every engine. + `textToSpeechStream()` work as with every engine. The request-level + `referenceAudioSrc` / `referenceText` pair is Audio8-only; other engines + reject it. For model constants, see [SDK — Models](/sdk#models). @@ -482,8 +675,17 @@ The speech is directable from the text itself: `[pause 2.0s]` markers insert a silence of roughly that length, and inline Pinyin (`ni3 hao3`) or IPA (`/həloʊ/`) steers pronunciation. `durationTokens` asks for a target length in codec frames (12.5 per second, so `38` is about 3 s; `0` or unset keeps the -length free; at most `2015`). `language` is written into the model prompt as a -hint (default `en`). +length free). `language` is written into the model prompt as a hint (default +`en`). + +Budgets and sampling: `maxNewTokens` is the generation budget in rows (engine +default `2048`), of which 33 are delay and end-of-sequence rows, so +`durationTokens` can be at most `maxNewTokens − 33` (`2015` by default; the +pair is checked at load). `nCtx` sets the backbone's context capacity. The +text and audio channels sample independently: `textTemperature`, `textTopP` +and `textTopK` steer the text channel, `audioTemperature`, `audioTopP`, +`audioTopK` and `audioRepetitionPenalty` the audio channel; each unset knob +keeps the engine default. The same controls apply to MOSS-TTSD. MOSS streams natively. With `streamChunkTokens` set, `stream: true` (the default) delivers audio every that-many codec frames while generation @@ -624,7 +826,9 @@ With Chatterbox, Supertonic, Parler, or CosyVoice3, you can choose to perform po The denoiser needs the whole utterance, so it cannot be combined with native chunk streaming (`streamChunkTokens > 0`) on Chatterbox, Parler, or CosyVoice3. The combination is rejected at load. The enhancer has no such - restriction — it resamples seam-free across chunk boundaries. + restriction there — it resamples seam-free across chunk boundaries. + Supertonic native chunk streaming rejects both stages; use sentence + streaming for enhanced or denoised Supertonic output. ```ts @@ -643,6 +847,31 @@ await loadModel({ }); ``` +On macOS and iOS either stage can run its neural core on Core ML. Give the +stage a compiled model directory on the inference host and a compute policy +together: `lavasrEnhancerCoremlModelPath` with +`lavasrEnhancerCoremlComputeUnits`, or `lavasrDenoiserCoremlModelPath` with +`lavasrDenoiserCoremlComputeUnits` (`CPU_ONLY`, `CPU_AND_GPU`, `CPU_AND_NE`, +`ALL`), beside that stage's GGUF source. A Core ML model that cannot load fails +the load instead of falling back to ggml, and the stage then reports device `2` +(Core ML-managed) and backend `99` in its stats. The models are exported with +the native `export-lavasr-coreml.py` script; they are not published in the +registry. + +```ts +await loadModel({ + modelSrc: TTS_MULTILINGUAL_SUPERTONIC3_Q8_0, + modelType: "tts", + modelConfig: { + ttsEngine: "supertonic", + language: "en", + lavasrEnhancerModelSrc: TTS_ENHANCER_LAVASR_FP16.src, + lavasrEnhancerCoremlModelPath: "/models/lavasr-enhancer.mlmodelc", + lavasrEnhancerCoremlComputeUnits: "CPU_AND_NE", + }, +}); +``` + ## Examples ### Chatterbox diff --git a/packages/cli/src/serve/extensions/openai/audio.ts b/packages/cli/src/serve/extensions/openai/audio.ts index ab7d5672234..0ccd3524327 100644 --- a/packages/cli/src/serve/extensions/openai/audio.ts +++ b/packages/cli/src/serve/extensions/openai/audio.ts @@ -95,14 +95,19 @@ export function pcmContentType(sampleRate: number): string { return `audio/L16; rate=${sampleRate}; channels=1` } -// Engine → native sample rate. Mirrors the constants used in the SDK -// examples (packages/sdk/examples/tts/{chatterbox,supertonic}.ts). -// TODO(QVAC-18522): add the GGML engine key here when the TTS-GGML migration -// lands, otherwise the engine falls through to DEFAULT_SAMPLE_RATE and audio -// plays back at the wrong speed. +// Engine → native sample rate, the fallback for a run that produced no audio +// (the route prefers the rate the engine reports). One row per @qvac/tts-ggml +// engine that resamples; MOSS-SoundEffect emits 48 kHz, the other MOSS +// engines their native 24 kHz. const ENGINE_SAMPLE_RATE: Record = { chatterbox: 24000, - supertonic: 44100 + supertonic: 44100, + parler: 44100, + cosyvoice3: 24000, + audio8: 44100, + moss: 24000, + 'moss-sfx': 48000, + 'moss-speech': 24000 } // 24 kHz matches OpenAI's documented pcm output (mono, 16-bit signed LE) diff --git a/packages/cli/test/unit/extensions/openai/audio.test.ts b/packages/cli/test/unit/extensions/openai/audio.test.ts index 8f4d03b8b66..5000eaf4064 100644 --- a/packages/cli/test/unit/extensions/openai/audio.test.ts +++ b/packages/cli/test/unit/extensions/openai/audio.test.ts @@ -118,6 +118,19 @@ describe('resolveSampleRate', () => { assert.equal(resolveSampleRate({ ttsEngine: 'supertonic' }), 44100) }) + it('knows the native rate of every @qvac/tts-ggml engine', () => { + for (const [ttsEngine, sampleRate] of [ + ['parler', 44100], + ['cosyvoice3', 24000], + ['audio8', 44100], + ['moss', 24000], + ['moss-sfx', 48000], + ['moss-speech', 24000] + ] as const) { + assert.equal(resolveSampleRate({ ttsEngine }), sampleRate, ttsEngine) + } + }) + it('is case-insensitive on engine name', () => { assert.equal(resolveSampleRate({ ttsEngine: 'CHATTERBOX' }), 24000) }) diff --git a/packages/inference/src/api/text-to-speech.ts b/packages/inference/src/api/text-to-speech.ts index 96c6625b398..a7c92dc752c 100644 --- a/packages/inference/src/api/text-to-speech.ts +++ b/packages/inference/src/api/text-to-speech.ts @@ -285,6 +285,7 @@ export type TtsSideChannel = { sampleRate: Promise stats: Promise stopReason: Promise<'completed' | 'cancelled' | undefined> + speechTokenIds: Promise observe(response: TtsResponse): void settle(): void } @@ -299,6 +300,7 @@ export function createTtsSideChannel(): TtsSideChannel { let resolveSampleRate!: (value: number | undefined) => void let resolveStats!: (value: TtsStats | undefined) => void let resolveStopReason!: (value: 'completed' | 'cancelled' | undefined) => void + let resolveSpeechTokenIds!: (value: number[] | undefined) => void const sampleRate = new Promise((resolve) => { resolveSampleRate = resolve }) @@ -308,10 +310,16 @@ export function createTtsSideChannel(): TtsSideChannel { const stopReason = new Promise<'completed' | 'cancelled' | undefined>((resolve) => { resolveStopReason = resolve }) + const speechTokenIds = new Promise((resolve) => { + resolveSpeechTokenIds = resolve + }) let seenSampleRate: number | undefined let latestStats: TtsStats | undefined let seenStopReason: 'completed' | 'cancelled' | undefined + // CosyVoice3 hands back one array per native synthesis; the chunked paths + // run one per sentence, so the run's trajectory is their concatenation. + let seenSpeechTokenIds: number[] | undefined let settled = false return { @@ -319,6 +327,7 @@ export function createTtsSideChannel(): TtsSideChannel { sampleRate, stats, stopReason, + speechTokenIds, observe(response) { if (response.text !== undefined) seenText = response.text // Resolve on the FIRST frame that carries a rate, not at end of stream: @@ -331,6 +340,12 @@ export function createTtsSideChannel(): TtsSideChannel { } if (response.stats !== undefined) latestStats = response.stats if (response.stopReason !== undefined) seenStopReason = response.stopReason + if (response.speechTokenIds !== undefined) { + // Append into one owned array: a spread per frame would copy every + // token collected so far on each sentence of a long stream. + seenSpeechTokenIds ??= [] + for (const id of response.speechTokenIds) seenSpeechTokenIds.push(id) + } }, settle() { if (settled) return @@ -339,6 +354,7 @@ export function createTtsSideChannel(): TtsSideChannel { resolveSampleRate(seenSampleRate) resolveStats(latestStats) resolveStopReason(seenStopReason) + resolveSpeechTokenIds(seenSpeechTokenIds) } } } @@ -389,7 +405,8 @@ function sentenceStreamTts( text: side.text, sampleRate: side.sampleRate, stats: side.stats, - stopReason: side.stopReason + stopReason: side.stopReason, + speechTokenIds: side.speechTokenIds } } @@ -427,7 +444,8 @@ async function* sentenceChunkUpdates( const hasAudio = m.buffer.length > 0 const hasMeta = m.chunkIndex !== undefined || - (typeof m.sentenceChunk === 'string' && m.sentenceChunk.length > 0) + (typeof m.sentenceChunk === 'string' && m.sentenceChunk.length > 0) || + m.speechTokenIds !== undefined if (hasAudio || hasMeta) { yield { buffer: hasAudio ? [...m.buffer] : [], @@ -436,7 +454,8 @@ async function* sentenceChunkUpdates( ...(typeof m.sentenceChunk === 'string' && m.sentenceChunk.length > 0 ? { sentenceChunk: m.sentenceChunk } : {}), - ...(m.isLast !== undefined ? { isLast: m.isLast } : {}) + ...(m.isLast !== undefined ? { isLast: m.isLast } : {}), + ...(m.speechTokenIds !== undefined ? { speechTokenIds: m.speechTokenIds } : {}) } } if (m.done) break @@ -467,7 +486,8 @@ function plainStreamTts( text: side.text, sampleRate: side.sampleRate, stats: side.stats, - stopReason: side.stopReason + stopReason: side.stopReason, + speechTokenIds: side.speechTokenIds } } @@ -534,7 +554,8 @@ function collectTts( text: side.text, sampleRate: side.sampleRate, stats: side.stats, - stopReason: side.stopReason + stopReason: side.stopReason, + speechTokenIds: side.speechTokenIds } } diff --git a/packages/inference/src/plugins/builtin/tts-ggml/fit.ts b/packages/inference/src/plugins/builtin/tts-ggml/fit.ts index b71167f69be..7b5a645411d 100644 --- a/packages/inference/src/plugins/builtin/tts-ggml/fit.ts +++ b/packages/inference/src/plugins/builtin/tts-ggml/fit.ts @@ -8,6 +8,7 @@ import type { FitLoad, FitRequestPlan } from '@/schemas/plugin' interface TtsCommon { nGpuLayers: number + backend?: string backendsDir?: string } @@ -67,26 +68,36 @@ export function ttsFitRequest(load: FitLoad): FitRequestPlan { return { unknown: 'the LavaSR stages have no fitter' } } + // The explicit backend selector resolves the same device the load would, so + // the projection lands on that device rather than on the `useGPU` policy's. const common: TtsCommon = { nGpuLayers: gpuLayersFromCount(config.useGPU, config.nGpuLayers), + ...(config.backend !== undefined && { backend: config.backend }), ...(config.backendsDir !== undefined && { backendsDir: config.backendsDir }) } const request = (fit: TtsFitRequest): FitRequestPlan => ({ request: fit }) switch (config.ttsEngine) { case 'supertonic': + // Precision and the f16 weight override change the weight footprint the + // fitter sizes, so they travel with the request. return request({ engineType: 'supertonic', supertonicModelPath: load.modelPath, ...common, + ...(config.precision !== undefined && { precision: config.precision }), + ...(config.f16Weights !== undefined && { f16Weights: config.f16Weights }), ...(config.ttsNumInferenceSteps !== undefined && { steps: config.ttsNumInferenceSteps }) }) case 'parler': + // The decoder sidecar policy decides whether the decoder weights load on + // ggml at all; the fitter matches the load's choice. return request({ engineType: 'parler', parlerModelPath: load.modelPath, ...common, + ...(config.coremlDecoder !== undefined && { coremlDecoder: config.coremlDecoder }), ...(config.maxFrames !== undefined && { maxFrames: config.maxFrames }) }) @@ -133,7 +144,11 @@ export function ttsFitRequest(load: FitLoad): FitRequestPlan { ...(config.streamChunkTokens !== undefined && { streamChunkTokens: config.streamChunkTokens }), - ...(config.durationTokens !== undefined && { durationTokens: config.durationTokens }) + ...(config.durationTokens !== undefined && { durationTokens: config.durationTokens }), + // The context and generation budgets size the KV cache and the + // generation buffers the fitter projects. + ...(config.nCtx !== undefined && { nCtx: config.nCtx }), + ...(config.maxNewTokens !== undefined && { maxNewTokens: config.maxNewTokens }) }) } diff --git a/packages/inference/src/plugins/builtin/tts-ggml/ops/text-to-speech-stream.ts b/packages/inference/src/plugins/builtin/tts-ggml/ops/text-to-speech-stream.ts index 6d9e27ad6ae..2ff8a40d291 100644 --- a/packages/inference/src/plugins/builtin/tts-ggml/ops/text-to-speech-stream.ts +++ b/packages/inference/src/plugins/builtin/tts-ggml/ops/text-to-speech-stream.ts @@ -9,6 +9,7 @@ import type { TtsStats as AddonTtsStats } from '@/utils/addon-responses' import { type TtsStreamChunk, type TtsOpYield, + carriesSpeechTokenIds, collectTtsStats, pcmFrame, chunkMetadata @@ -196,7 +197,9 @@ export async function* textToSpeechStream( if (data.outputArray == null) { continue } - if (data.outputArray.length === 0) { + // An empty chunk is a native end marker, unless it carries the sentence's + // speech tokens (CosyVoice3 native streaming), which the caller wants. + if (data.outputArray.length === 0 && !carriesSpeechTokenIds(data)) { continue } yield { ...pcmFrame(data.outputArray), ...chunkMetadata(data) } diff --git a/packages/inference/src/plugins/builtin/tts-ggml/ops/text-to-speech.ts b/packages/inference/src/plugins/builtin/tts-ggml/ops/text-to-speech.ts index 09e319dee97..07430f75e0c 100644 --- a/packages/inference/src/plugins/builtin/tts-ggml/ops/text-to-speech.ts +++ b/packages/inference/src/plugins/builtin/tts-ggml/ops/text-to-speech.ts @@ -9,6 +9,7 @@ import { PluginRequestValidationFailedError, TextToSpeechFailedError } from '@/e import { type TtsStreamChunk, type TtsOpYield, + carriesSpeechTokenIds, collectTtsStats, chunkMetadata, concatPcm, @@ -144,6 +145,8 @@ export async function* textToSpeech(params: TtsRequest): AsyncGenerator[] = [] let sampleRate: number | undefined + const speechTokenIds: number[] = [] + let sawSpeechTokenIds = false try { for await (const data of response.iterate()) { if (await cancelIfAborted(model, ctx.signal)) continue @@ -152,6 +155,10 @@ export async function* textToSpeech(params: TtsRequest): AsyncGenerator[] = [] let sampleRate: number | undefined let replyText: string | undefined + // CosyVoice3 batch output carries the run's speech tokens with the PCM; + // under native chunk streaming they arrive on the final empty chunk. + const speechTokenIds: number[] = [] + let sawSpeechTokenIds = false try { for await (const data of response.iterate()) { if (await cancelIfAborted(model, ctx.signal)) continue sampleRate ??= data.sampleRate if (data.text !== undefined) replyText = data.text + if (data.speechTokenIds !== undefined) { + sawSpeechTokenIds = true + speechTokenIds.push(...data.speechTokenIds) + } parts.push(data.outputArray) } } catch (error) { @@ -224,7 +242,8 @@ export async function* textToSpeech(params: TtsRequest): AsyncGenerator 0 ? { sentenceChunk: value.sentenceChunk } : {}), - ...(value.isLast !== undefined ? { isLast: value.isLast } : {}) + ...(value.isLast !== undefined ? { isLast: value.isLast } : {}), + ...(value.speechTokenIds !== undefined ? { speechTokenIds: value.speechTokenIds } : {}) } } @@ -75,12 +76,32 @@ function dirname(path: string) { return path.slice(0, index) } +// The load-only LavaSR fields: the two GGUF sources and, per stage, the Core +// ML model directory and compute policy. All four engines that run LavaSR +// take them, so one pick keeps the runtime configs free of them. +type TtsLavasrLoadFields = Pick< + TtsChatterboxLoadConfig, + | 'lavasrEnhancerModelSrc' + | 'lavasrDenoiserModelSrc' + | 'lavasrEnhancerCoremlModelPath' + | 'lavasrEnhancerCoremlComputeUnits' + | 'lavasrDenoiserCoremlModelPath' + | 'lavasrDenoiserCoremlComputeUnits' +> + // Resolve the optional LavaSR enhancer/denoiser GGUFs shared by both engines. -async function resolveLavasrArtifacts( - lavasrEnhancerModelSrc: TtsChatterboxLoadConfig['lavasrEnhancerModelSrc'], - lavasrDenoiserModelSrc: TtsChatterboxLoadConfig['lavasrDenoiserModelSrc'], - ctx: ResolveContext -) { +// The Core ML model directories are host paths, not sources: they ride along +// as artifacts so createModel builds the addon's `enhancer` / `denoiser` +// options from one place. +async function resolveLavasrArtifacts(fields: TtsLavasrLoadFields, ctx: ResolveContext) { + const { + lavasrEnhancerModelSrc, + lavasrDenoiserModelSrc, + lavasrEnhancerCoremlModelPath, + lavasrEnhancerCoremlComputeUnits, + lavasrDenoiserCoremlModelPath, + lavasrDenoiserCoremlComputeUnits + } = fields const resolve = ctx.resolveModelPath const [lavasrEnhancerPath, lavasrDenoiserPath] = await Promise.all([ lavasrEnhancerModelSrc ? resolve(lavasrEnhancerModelSrc) : Promise.resolve(undefined), @@ -89,7 +110,11 @@ async function resolveLavasrArtifacts( return { ...(lavasrEnhancerPath ? { lavasrEnhancerPath } : {}), - ...(lavasrDenoiserPath ? { lavasrDenoiserPath } : {}) + ...(lavasrDenoiserPath ? { lavasrDenoiserPath } : {}), + ...(lavasrEnhancerCoremlModelPath !== undefined ? { lavasrEnhancerCoremlModelPath } : {}), + ...(lavasrEnhancerCoremlComputeUnits !== undefined ? { lavasrEnhancerCoremlComputeUnits } : {}), + ...(lavasrDenoiserCoremlModelPath !== undefined ? { lavasrDenoiserCoremlModelPath } : {}), + ...(lavasrDenoiserCoremlComputeUnits !== undefined ? { lavasrDenoiserCoremlComputeUnits } : {}) } } @@ -106,6 +131,10 @@ async function resolveChatterboxConfig( cangjieTsvSrc, lavasrEnhancerModelSrc, lavasrDenoiserModelSrc, + lavasrEnhancerCoremlModelPath, + lavasrEnhancerCoremlComputeUnits, + lavasrDenoiserCoremlModelPath, + lavasrDenoiserCoremlComputeUnits, ...runtime } = config if (!s3genModelSrc) { @@ -119,7 +148,17 @@ async function resolveChatterboxConfig( referenceAudioSrc ? resolve(referenceAudioSrc) : Promise.resolve(undefined), mecabDictSrc ? resolve(mecabDictSrc) : Promise.resolve(undefined), cangjieTsvSrc ? resolve(cangjieTsvSrc) : Promise.resolve(undefined), - resolveLavasrArtifacts(lavasrEnhancerModelSrc, lavasrDenoiserModelSrc, ctx) + resolveLavasrArtifacts( + { + lavasrEnhancerModelSrc, + lavasrDenoiserModelSrc, + lavasrEnhancerCoremlModelPath, + lavasrEnhancerCoremlComputeUnits, + lavasrDenoiserCoremlModelPath, + lavasrDenoiserCoremlComputeUnits + }, + ctx + ) ]) return { @@ -140,11 +179,30 @@ async function resolveSupertonicConfig( ): Promise> { rejectLegacyOnnxFields(config) - const { lavasrEnhancerModelSrc, lavasrDenoiserModelSrc, voiceJsonSrc, noiseNpySrc, ...runtime } = - config + const { + lavasrEnhancerModelSrc, + lavasrDenoiserModelSrc, + lavasrEnhancerCoremlModelPath, + lavasrEnhancerCoremlComputeUnits, + lavasrDenoiserCoremlModelPath, + lavasrDenoiserCoremlComputeUnits, + voiceJsonSrc, + noiseNpySrc, + ...runtime + } = config const resolve = ctx.resolveModelPath const [lavasrArtifacts, voiceJsonPath, noiseNpyPath] = await Promise.all([ - resolveLavasrArtifacts(lavasrEnhancerModelSrc, lavasrDenoiserModelSrc, ctx), + resolveLavasrArtifacts( + { + lavasrEnhancerModelSrc, + lavasrDenoiserModelSrc, + lavasrEnhancerCoremlModelPath, + lavasrEnhancerCoremlComputeUnits, + lavasrDenoiserCoremlModelPath, + lavasrDenoiserCoremlComputeUnits + }, + ctx + ), voiceJsonSrc ? resolve(voiceJsonSrc) : Promise.resolve(undefined), noiseNpySrc ? resolve(noiseNpySrc) : Promise.resolve(undefined) ]) @@ -163,10 +221,24 @@ async function resolveParlerConfig( config: TtsParlerLoadConfig, ctx: ResolveContext ): Promise> { - const { lavasrEnhancerModelSrc, lavasrDenoiserModelSrc, ...runtime } = config - const lavasrArtifacts = await resolveLavasrArtifacts( + const { lavasrEnhancerModelSrc, lavasrDenoiserModelSrc, + lavasrEnhancerCoremlModelPath, + lavasrEnhancerCoremlComputeUnits, + lavasrDenoiserCoremlModelPath, + lavasrDenoiserCoremlComputeUnits, + ...runtime + } = config + const lavasrArtifacts = await resolveLavasrArtifacts( + { + lavasrEnhancerModelSrc, + lavasrDenoiserModelSrc, + lavasrEnhancerCoremlModelPath, + lavasrEnhancerCoremlComputeUnits, + lavasrDenoiserCoremlModelPath, + lavasrDenoiserCoremlComputeUnits + }, ctx ) @@ -184,6 +256,10 @@ async function resolveCosyvoice3Config( const { lavasrEnhancerModelSrc, lavasrDenoiserModelSrc, + lavasrEnhancerCoremlModelPath, + lavasrEnhancerCoremlComputeUnits, + lavasrDenoiserCoremlModelPath, + lavasrDenoiserCoremlComputeUnits, referenceAudioSrc, cosyvoice3S3tokModelSrc, cosyvoice3CampplusModelSrc, @@ -193,7 +269,17 @@ async function resolveCosyvoice3Config( const resolve = ctx.resolveModelPath const [lavasrArtifacts, referenceAudioPath, cosyvoiceS3tokPath, cosyvoiceCampplusPath] = await Promise.all([ - resolveLavasrArtifacts(lavasrEnhancerModelSrc, lavasrDenoiserModelSrc, ctx), + resolveLavasrArtifacts( + { + lavasrEnhancerModelSrc, + lavasrDenoiserModelSrc, + lavasrEnhancerCoremlModelPath, + lavasrEnhancerCoremlComputeUnits, + lavasrDenoiserCoremlModelPath, + lavasrDenoiserCoremlComputeUnits + }, + ctx + ), referenceAudioSrc ? resolve(referenceAudioSrc) : Promise.resolve(undefined), cosyvoice3S3tokModelSrc ? resolve(cosyvoice3S3tokModelSrc) : Promise.resolve(undefined), cosyvoice3CampplusModelSrc ? resolve(cosyvoice3CampplusModelSrc) : Promise.resolve(undefined) @@ -293,12 +379,15 @@ async function resolveMossConfig( // The generic ggml backend-loading knobs, forwarded on `config` the same way // Supertonic's vulkanCacheDir already is. Parler, Audio8 and MOSS configs carry // no `openclCacheDir` (their native builders do not read it), so it is simply -// absent for them. +// absent for them. `backend` is the explicit native backend / device selector +// every engine reads from `config`. function ggmlBackendConfig(config: { + backend?: string | undefined backendsDir?: string | undefined openclCacheDir?: string | undefined }) { return { + ...(config.backend !== undefined ? { backend: config.backend } : {}), ...(config.backendsDir !== undefined ? { backendsDir: config.backendsDir } : {}), ...(config.openclCacheDir !== undefined ? { openclCacheDir: config.openclCacheDir } : {}) } @@ -315,6 +404,38 @@ function lavasrFiles(artifacts: Record) { } } +// The addon's `enhancer` / `denoiser` option objects, built only when a stage +// runs on Core ML: the GGUF itself travels in `files`, so the object carries +// just the compiled model directory and the compute policy the schema paired. +function lavasrCoremlOptions(artifacts: Record) { + const enhancerPath = artifacts['lavasrEnhancerCoremlModelPath'] + const denoiserPath = artifacts['lavasrDenoiserCoremlModelPath'] + return { + ...(enhancerPath + ? { + enhancer: { + type: 'lavasr' as const, + coremlModelPath: enhancerPath, + coremlComputeUnits: artifacts[ + 'lavasrEnhancerCoremlComputeUnits' + ] as TTSGgml.LavaSRCoreMLComputeUnits + } + } + : {}), + ...(denoiserPath + ? { + denoiser: { + type: 'lavasr' as const, + coremlModelPath: denoiserPath, + coremlComputeUnits: artifacts[ + 'lavasrDenoiserCoremlComputeUnits' + ] as TTSGgml.LavaSRCoreMLComputeUnits + } + } + : {}) + } +} + function createChatterboxModel( modelId: string, config: TtsChatterboxRuntimeConfig, @@ -342,6 +463,7 @@ function createChatterboxModel( ...(cangjieTsvPath ? { cangjieTsvPath } : {}), ...lavasrFiles(artifacts) }, + ...lavasrCoremlOptions(artifacts), ...(referenceAudioPath ? { referenceAudio: referenceAudioPath } : {}), ...(config.streamChunkTokens !== undefined ? { streamChunkTokens: config.streamChunkTokens } @@ -409,12 +531,27 @@ function createSupertonicModel( const model = new TTSGgml({ engine: TTSGgml.ENGINE_SUPERTONIC, files: { supertonicModel, ...lavasrFiles(artifacts) }, + ...lavasrCoremlOptions(artifacts), voice: config.voice ?? 'F1', - // An external voice JSON overrides the baked preset above. + // An external voice JSON overrides the baked preset above, and in-memory + // style tensors override both. ...(voiceJsonPath ? { voiceJsonPath } : {}), + ...(config.voiceStyleTtl !== undefined ? { voiceStyleTtl: config.voiceStyleTtl } : {}), + ...(config.voiceStyleDp !== undefined ? { voiceStyleDp: config.voiceStyleDp } : {}), ...(noiseNpyPath ? { noiseNpyPath } : {}), ...(config.prewarmText !== undefined ? { prewarmText: config.prewarmText } : {}), ...(config.vulkanDevice !== undefined ? { vulkanDevice: config.vulkanDevice } : {}), + // Native precision and environment controls, forwarded as given. + ...(config.precision !== undefined ? { precision: config.precision } : {}), + ...(config.f16Attn !== undefined ? { f16Attn: config.f16Attn } : {}), + ...(config.f16Weights !== undefined ? { f16Weights: config.f16Weights } : {}), + ...(config.kvAttnType !== undefined ? { kvAttnType: config.kvAttnType } : {}), + ...(config.f16WeightsDenyList !== undefined + ? { f16WeightsDenyList: config.f16WeightsDenyList } + : {}), + ...(config.vulkanEnvOverrides !== undefined + ? { vulkanEnvOverrides: config.vulkanEnvOverrides } + : {}), ...(config.streamChunkTokens !== undefined ? { streamChunkTokens: config.streamChunkTokens } : {}), @@ -474,6 +611,13 @@ function createParlerModel( const model = new TTSGgml({ engine: TTSGgml.ENGINE_PARLER, files: { parlerModel, ...lavasrFiles(artifacts) }, + ...lavasrCoremlOptions(artifacts), + // Parler-only Core ML sidecar controls (load / reload, never per call). + ...(config.coremlDecoder !== undefined ? { coremlDecoder: config.coremlDecoder } : {}), + ...(config.coremlComputeUnits !== undefined + ? { coremlComputeUnits: config.coremlComputeUnits } + : {}), + ...(config.coremlStrict !== undefined ? { coremlStrict: config.coremlStrict } : {}), ...(config.description !== undefined ? { description: config.description } : {}), ...(config.voiceDescription !== undefined ? { voiceDescription: config.voiceDescription } : {}), ...(config.voice !== undefined ? { voice: config.voice } : {}), @@ -546,6 +690,7 @@ function createCosyvoice3Model( ...(cosyvoiceCampplusModel ? { cosyvoiceCampplusModel } : {}), ...lavasrFiles(artifacts) }, + ...lavasrCoremlOptions(artifacts), ...(referenceAudioPath ? { referenceAudio: referenceAudioPath } : {}), ...(config.promptText !== undefined ? { promptText: config.promptText } : {}), ...(config.emotion !== undefined ? { emotion: config.emotion } : {}), @@ -553,6 +698,11 @@ function createCosyvoice3Model( ...(config.instruct !== undefined ? { instruct: toAddonInstruct(config.instruct) } : {}), ...(config.flowCutPrompt !== undefined ? { flowCutPrompt: config.flowCutPrompt } : {}), ...(config.vulkanDevice !== undefined ? { vulkanDevice: config.vulkanDevice } : {}), + // Greedy LM decoding and a forced speech-token trajectory (replay). + ...(config.greedy !== undefined ? { greedy: config.greedy } : {}), + ...(config.forceSpeechTokens !== undefined + ? { forceSpeechTokens: config.forceSpeechTokens } + : {}), ...(config.streamChunkTokens !== undefined ? { streamChunkTokens: config.streamChunkTokens } : {}), @@ -659,6 +809,18 @@ function createMossModel( ...(config.streamChunkTokens !== undefined ? { streamChunkTokens: config.streamChunkTokens } : {}), + // Context and generation budgets, then the text- and audio-channel samplers. + ...(config.nCtx !== undefined ? { nCtx: config.nCtx } : {}), + ...(config.maxNewTokens !== undefined ? { maxNewTokens: config.maxNewTokens } : {}), + ...(config.textTemperature !== undefined ? { textTemperature: config.textTemperature } : {}), + ...(config.textTopP !== undefined ? { textTopP: config.textTopP } : {}), + ...(config.textTopK !== undefined ? { textTopK: config.textTopK } : {}), + ...(config.audioTemperature !== undefined ? { audioTemperature: config.audioTemperature } : {}), + ...(config.audioTopP !== undefined ? { audioTopP: config.audioTopP } : {}), + ...(config.audioTopK !== undefined ? { audioTopK: config.audioTopK } : {}), + ...(config.audioRepetitionPenalty !== undefined + ? { audioRepetitionPenalty: config.audioRepetitionPenalty } + : {}), ...(config.threads !== undefined ? { threads: config.threads } : {}), ...(config.nGpuLayers !== undefined ? { nGpuLayers: config.nGpuLayers } : {}), ...(config.seed !== undefined ? { seed: config.seed } : {}), diff --git a/packages/inference/src/schemas/text-to-speech.ts b/packages/inference/src/schemas/text-to-speech.ts index 6c4912be79f..28ec8405d7f 100644 --- a/packages/inference/src/schemas/text-to-speech.ts +++ b/packages/inference/src/schemas/text-to-speech.ts @@ -134,6 +134,27 @@ export const TTS_COSYVOICE3_INSTRUCT_VOLUMES = ['loud', 'soft'] as const export const TTS_COSYVOICE3_INSTRUCT_STYLES = ['peppa', 'robot'] as const +// Parler's Core ML compute policy (@qvac/tts-ggml `PARLER_COREML_COMPUTE_UNITS`). +export const TTS_PARLER_COREML_COMPUTE_UNITS = [ + 'all', + 'cpu_and_ane', + 'cpu_and_gpu', + 'cpu_only' +] as const + +// The LavaSR stages' Core ML compute policy (@qvac/tts-ggml +// `LavaSRCoreMLComputeUnits`); a different spelling from Parler's because each +// mirrors its own native enum. +export const TTS_LAVASR_COREML_COMPUTE_UNITS = [ + 'CPU_ONLY', + 'CPU_AND_GPU', + 'CPU_AND_NE', + 'ALL' +] as const + +// Supertonic weight precision (@qvac/tts-ggml `precision`). +export const TTS_SUPERTONIC_PRECISIONS = ['auto', 'f32', 'f16', 'q8_0'] as const + // Supertonic languages not already present in the Chatterbox set, used to keep // TTS_LANGUAGES a true union across engines without duplicates. const TTS_SUPERTONIC_ONLY_LANGUAGES = [ @@ -170,6 +191,11 @@ const ttsIntegerSchema = z.number().int() const ttsInt32Schema = ttsIntegerSchema.min(-2147483648).max(2147483647) const ttsNonNegativeInt32Schema = ttsInt32Schema.nonnegative() const ttsPositiveInt32Schema = ttsInt32Schema.positive() +// A CosyVoice3 speech token travels both ways: the response hands them out as +// `speechTokenIds`, the load config takes them back as `forceSpeechTokens`. +// One titled scalar keeps the two interchangeable in every generated client +// (one shared Python class instead of a differently named wrapper per field). +const ttsSpeechTokenIdSchema = ttsInt32Schema.meta({ title: 'TtsSpeechTokenId' }) // Describe text shared by fields repeated across engine arms. const TTS_USE_GPU_DESC = @@ -198,14 +224,33 @@ const TTS_BACKENDS_DIR_DESC = 'Directory scanned as given for dynamically-loaded ggml backend `.so` files. Unset scans the directory the native module was loaded from.' const TTS_OPENCL_CACHE_DIR_DESC = "Persistent directory for ggml-opencl's compiled-program cache (Android only)." +const TTS_BACKEND_DESC = + 'Explicit native backend: `auto`, `cpu`, `opencl`, `webgpu`, or an exact registered device name such as `MTL0`, `CUDA0` or `Vulkan0` (device names depend on the host). An unavailable explicit backend fails the load; unset keeps the `useGPU` policy. WebGPU is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only.' // The ggml backend knobs every engine honours. `openclCacheDir` is added only // on the three engines whose native config reads it (Parler and Audio8 build // their params through the addon's _assignBackendParams, which does not). const ttsBackendDirFieldsShape = { + backend: z.string().trim().min(1).optional().describe(TTS_BACKEND_DESC), backendsDir: z.string().min(1).optional().describe(TTS_BACKENDS_DIR_DESC) } +// Mirrors the addon's validateBackendOption: a WebGPU selector is accepted only +// on the engines whose native library has established WebGPU support +// (Supertonic 3, Parler, CosyVoice3, Audio8); the others reject it before any +// weight is loaded, and so does this schema, naming the field. +function refineNoWebgpuBackend( + config: { ttsEngine: string; backend?: string | undefined }, + ctx: z.RefinementCtx +) { + if (config.backend === undefined || !/^webgpu/i.test(config.backend)) return + ctx.addIssue({ + code: 'custom', + path: ['backend'], + message: `WebGPU has no established support for ${config.ttsEngine}; it is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only.` + }) +} + const ttsOpenclCacheFieldShape = { openclCacheDir: z.string().min(1).optional().describe(TTS_OPENCL_CACHE_DIR_DESC) } @@ -498,7 +543,37 @@ export const ttsChatterboxRuntimeConfigSchema = z ...ttsBackendDirFieldsShape, ...ttsOpenclCacheFieldShape }) - .superRefine(refineGpuIntent) + .superRefine(refineChatterboxRuntimeConfig) + +function refineChatterboxRuntimeConfig( + config: { + ttsEngine: 'chatterbox' + backend?: string | undefined + useGPU?: boolean | undefined + nGpuLayers?: number | undefined + }, + ctx: z.RefinementCtx +) { + refineGpuIntent(config, ctx) + refineNoWebgpuBackend(config, ctx) +} + +// Supertonic's in-memory voice tensors come as a pair: the addon rejects one +// without the other, so the schema does too, naming the missing half. +function refineSupertonicVoiceStyle( + config: { + voiceStyleTtl?: number[] | undefined + voiceStyleDp?: number[] | undefined + }, + ctx: z.RefinementCtx +) { + if ((config.voiceStyleTtl === undefined) === (config.voiceStyleDp === undefined)) return + ctx.addIssue({ + code: 'custom', + path: [config.voiceStyleTtl === undefined ? 'voiceStyleTtl' : 'voiceStyleDp'], + message: 'voiceStyleTtl and voiceStyleDp must be supplied together.' + }) +} // Supertonic drives its duration predictor from either an exact multiplier // (`ttsSpeed`) or the canonical pace vocabulary, never both — the engine @@ -509,10 +584,13 @@ function refineSupertonicRateControls( pace?: string | undefined useGPU?: boolean | undefined nGpuLayers?: number | undefined + voiceStyleTtl?: number[] | undefined + voiceStyleDp?: number[] | undefined }, ctx: z.RefinementCtx ) { refineGpuIntent(config, ctx) + refineSupertonicVoiceStyle(config, ctx) if (config.ttsSpeed !== undefined && config.pace !== undefined) { ctx.addIssue({ code: 'custom', @@ -588,6 +666,45 @@ const ttsSupertonicRuntimeConfigShape = { .describe( 'Floor on every streamed chunk’s size in text tokens (engine default 30); shorter trailing chunks merge into the previous one.' ), + // Native precision and environment controls, forwarded to the engine as + // given; -1 on the integer overrides leaves the engine's automatic choice. + precision: z + .enum(TTS_SUPERTONIC_PRECISIONS) + .optional() + .describe( + 'Weight precision the engine loads at: `auto` keeps the GGUF’s own, `f32` / `f16` / `q8_0` convert on load and change the weight footprint.' + ), + f16Attn: ttsInt32Schema + .optional() + .describe('Attention precision override: `-1` automatic (default), `0` f32, `1` f16.'), + f16Weights: ttsInt32Schema + .optional() + .describe('Weight precision override: `-1` automatic (default), `0` off, `1` f16.'), + kvAttnType: ttsInt32Schema + .optional() + .describe('Native KV attention type code; `-1` leaves the automatic selection.'), + f16WeightsDenyList: z + .array(z.string().min(1)) + .optional() + .describe('Tensor-name patterns kept at their original precision when `f16Weights` is on.'), + vulkanEnvOverrides: z + .record(z.string(), z.string()) + .optional() + .describe( + 'Vulkan environment overrides (string values) passed to the engine; explicit entries win over the `vulkanCacheDir` convenience setting.' + ), + voiceStyleTtl: z + .array(z.number()) + .min(1) + .optional() + .describe( + 'In-memory voice: the flattened `style_ttl` tensor of a cloned voice. Paired with `voiceStyleDp`; the engine checks the sizes against the model, and the pair wins over `voiceJsonSrc` and the baked `voice`.' + ), + voiceStyleDp: z + .array(z.number()) + .min(1) + .optional() + .describe('In-memory voice: the flattened `style_dp` tensor, paired with `voiceStyleTtl`.'), ...ttsBackendDirFieldsShape, ...ttsOpenclCacheFieldShape } @@ -626,6 +743,26 @@ const ttsParlerRuntimeConfigShape = { .boolean() .optional() .describe('Parler prompt digit expansion (engine default: enabled).'), + // Apple Core ML sidecars (macOS / iOS). The DAC sidecar is presence-driven; + // these three steer the optional decoder sidecar and the compute policy. + coremlDecoder: z + .boolean() + .optional() + .describe( + 'macOS / iOS: run the decoder steps on the `-decoder.mlmodelc` sidecar beside the GGUF (the DAC sidecar is picked up by presence); unset keeps the engine default.' + ), + coremlComputeUnits: z + .enum(TTS_PARLER_COREML_COMPUTE_UNITS) + .optional() + .describe( + 'macOS / iOS: Core ML compute policy for the Parler sidecars; unset inherits the engine’s environment default.' + ), + coremlStrict: z + .boolean() + .optional() + .describe( + 'macOS / iOS: `true` fails the load when a sidecar is unusable instead of falling back to ggml, `false` permits the fallback; unset inherits the engine’s strict mode.' + ), ...ttsBackendDirFieldsShape } @@ -756,6 +893,18 @@ const ttsCosyvoice3RuntimeConfigShape = { 'Treat the baked-voice prompt frames as attention-only conditioning in the flow: a faster flow whose output deviates slightly from the reference (engine default off).' ), vulkanDevice: ttsInt32Schema.min(-1).optional().describe(TTS_VULKAN_DEVICE_DESC), + greedy: z + .boolean() + .optional() + .describe( + 'Greedy LM decoding: take the argmax speech token instead of sampling (engine default off).' + ), + forceSpeechTokens: z + .array(ttsSpeechTokenIdSchema) + .optional() + .describe( + 'Speech-token trajectory of ONE native synthesis to replay instead of generating, as a previous batch run, native chunk stream or single sentence returned it in `speechTokenIds`. Fixed at load: every synthesis of the model replays this same array, so a joined sentence stream cannot be replayed sentence by sentence. An empty array restores generation.' + ), useGPU: z.boolean().optional().describe(TTS_USE_GPU_DESC), outputSampleRate: ttsOutputSampleRateSchema.optional(), streamChunkTokens: ttsNonNegativeInt32Schema.optional().describe(TTS_STREAM_CHUNK_TOKENS_DESC), @@ -803,12 +952,86 @@ export const ttsAudio8RuntimeConfigSchema = z .object(ttsAudio8RuntimeConfigShape) .superRefine(refineGpuIntent) -// @qvac/tts-ggml's MOSS_MAX_DURATION_TOKENS: the 2048-row generation budget -// less the 31 delay-pattern drain rows and the two termination rows. -const TTS_MOSS_MAX_DURATION_TOKENS = 2015 +// @qvac/tts-ggml's MOSS generation budget: 2048 rows when `maxNewTokens` is +// unset, of which 31 delay-pattern drain rows and two termination rows are +// never audio, so `durationTokens` fits at most `maxNewTokens - 33` (2015 by +// default). +const TTS_MOSS_DEFAULT_MAX_NEW_TOKENS = 2048 +const TTS_MOSS_RESERVED_ROWS = 33 const TTS_MOSS_MAX_DIALOGUE_SPEAKERS = 5 +// Mirrors the addon's assertMossDurationTokens with the configured budget: a +// target that does not fit the generation budget is refused at load rather +// than by the engine. +function refineMossDurationBudget( + config: { durationTokens?: number | undefined; maxNewTokens?: number | undefined }, + ctx: z.RefinementCtx +) { + if (config.durationTokens === undefined || config.durationTokens === 0) return + const budget = (config.maxNewTokens ?? TTS_MOSS_DEFAULT_MAX_NEW_TOKENS) - TTS_MOSS_RESERVED_ROWS + if (config.durationTokens <= budget) return + ctx.addIssue({ + code: 'custom', + path: ['durationTokens'], + message: `durationTokens must be an integer from 0 to ${Math.max(0, budget)} (maxNewTokens less the 33 delay/EOS rows; 0 = free length).` + }) +} + +function refineMossRuntimeConfig( + config: { + ttsEngine: 'moss' + backend?: string | undefined + useGPU?: boolean | undefined + nGpuLayers?: number | undefined + durationTokens?: number | undefined + maxNewTokens?: number | undefined + }, + ctx: z.RefinementCtx +) { + refineGpuIntent(config, ctx) + refineNoWebgpuBackend(config, ctx) + refineMossDurationBudget(config, ctx) +} + +// MOSS samples its text and audio channels separately; the defaults are the +// engine's, so these do not share the Parler / Audio8 sampling descriptions. +const ttsMossSamplingFieldsShape = { + textTemperature: z + .number() + .nonnegative() + .optional() + .describe('Text-channel sampling temperature; unset keeps the engine default.'), + textTopP: z + .number() + .positive() + .max(1) + .optional() + .describe('Text-channel top-p (nucleus) cutoff, 0 < p ≤ 1; unset keeps the engine default.'), + textTopK: ttsNonNegativeInt32Schema + .optional() + .describe('Text-channel top-k cutoff; unset keeps the engine default.'), + audioTemperature: z + .number() + .nonnegative() + .optional() + .describe('Audio-channel sampling temperature; unset keeps the engine default.'), + audioTopP: z + .number() + .positive() + .max(1) + .optional() + .describe('Audio-channel top-p (nucleus) cutoff, 0 < p ≤ 1; unset keeps the engine default.'), + audioTopK: ttsNonNegativeInt32Schema + .optional() + .describe('Audio-channel top-k cutoff; unset keeps the engine default.'), + audioRepetitionPenalty: z + .number() + .positive() + .optional() + .describe('Audio-channel repetition penalty; 1 = off, unset keeps the engine default.') +} + // MOSS emits its native 24 kHz only (the addon rejects any other // `outputSampleRate`) and supports neither the LavaSR stages nor a smaller // first streaming chunk, so none of those fields exist on this arm. @@ -826,18 +1049,26 @@ const ttsMossRuntimeConfigShape = { .min(1) .optional() .describe('Language hint written into the model prompt, e.g. `en` or `zh`. Default `en`.'), - durationTokens: ttsIntegerSchema + durationTokens: ttsInt32Schema .min(0) - .max(TTS_MOSS_MAX_DURATION_TOKENS) .optional() .describe( - 'Target length of each synthesis in codec frames (12.5 per second, so 38 ≈ 3 s), 0–2015; 0 or unset keeps the length free.' + 'Target length of each synthesis in codec frames (12.5 per second, so 38 ≈ 3 s), at most `maxNewTokens` − 33 (2015 by default); 0 or unset keeps the length free.' ), streamChunkTokens: ttsNonNegativeInt32Schema .optional() .describe( 'Codec frames per native streaming chunk (12.5 per second, so 25 ≈ 2 s); 0 or unset synthesizes the whole text before emitting.' ), + nCtx: ttsPositiveInt32Schema + .optional() + .describe('Context capacity of the backbone in rows; unset keeps the native value.'), + maxNewTokens: ttsPositiveInt32Schema + .optional() + .describe( + 'Generation budget in rows (engine default 2048), of which 33 are delay/EOS rows; bounds `durationTokens`.' + ), + ...ttsMossSamplingFieldsShape, useGPU: z.boolean().optional().describe(TTS_USE_GPU_DESC), threads: ttsPositiveInt32Schema.optional().describe(TTS_THREADS_DESC), // Not TTS_NGPU_LAYERS_DESC: MOSS does not offload per layer, so a layer @@ -853,7 +1084,7 @@ const ttsMossRuntimeConfigShape = { export const ttsMossRuntimeConfigSchema = z .object(ttsMossRuntimeConfigShape) - .superRefine(refineGpuIntent) + .superRefine(refineMossRuntimeConfig) // JSON PCM keeps the client and wire portable; converted to Float32Array at the addon boundary. const ttsSpeechPcmSchema = z.array(z.number().min(-1).max(1)).min(1) @@ -950,20 +1181,33 @@ const ttsBatchBackendShape = { ...ttsBackendDirFieldsShape } +function refineMossBatchRuntimeConfig( + config: { + ttsEngine: 'moss-sfx' | 'moss-speech' + backend?: string | undefined + useGPU?: boolean | undefined + nGpuLayers?: number | undefined + }, + ctx: z.RefinementCtx +) { + refineGpuIntent(config, ctx) + refineNoWebgpuBackend(config, ctx) +} + export const ttsMossSoundEffectRuntimeConfigSchema = z .object({ ttsEngine: z.literal('moss-sfx').describe('TTS engine: MOSS-SoundEffect-v2.'), ...ttsBatchBackendShape, ...ttsSoundEffectFieldsShape }) - .superRefine(refineGpuIntent) + .superRefine(refineMossBatchRuntimeConfig) export const ttsMossSpeechRuntimeConfigSchema = z .object({ ttsEngine: z.literal('moss-speech').describe('TTS engine: MOSS-Speech.'), ...ttsBatchBackendShape }) - .superRefine(refineGpuIntent) + .superRefine(refineMossBatchRuntimeConfig) export const ttsMossSoundEffectLoadConfigSchema = ttsMossSoundEffectRuntimeConfigSchema export const ttsMossSpeechLoadConfigSchema = ttsMossSpeechRuntimeConfigSchema.extend({ @@ -995,9 +1239,74 @@ const ttsLavasrLoadFieldsShape = { .optional() .describe( 'LavaSR denoiser model source; runs before the enhancer, rate-preserving (batch synthesis only).' + ), + // The Core ML models are compiled `.mlmodelc` directories, not single files, + // so they are host paths rather than model sources. Each stage takes its + // path and compute policy together, beside its GGUF source. + lavasrEnhancerCoremlModelPath: z + .string() + .min(1) + .optional() + .describe( + 'macOS / iOS: compiled Core ML model directory (`.mlmodelc`) on the inference host for the LavaSR enhancer’s neural core; requires `lavasrEnhancerModelSrc` and `lavasrEnhancerCoremlComputeUnits`. A model that cannot load fails the load rather than falling back to ggml.' + ), + lavasrEnhancerCoremlComputeUnits: z + .enum(TTS_LAVASR_COREML_COMPUTE_UNITS) + .optional() + .describe( + 'macOS / iOS: Core ML compute policy for the LavaSR enhancer; required with `lavasrEnhancerCoremlModelPath`. Stats then report the stage as device `2` (Core ML-managed), backend `99`.' + ), + lavasrDenoiserCoremlModelPath: z + .string() + .min(1) + .optional() + .describe( + 'macOS / iOS: compiled Core ML model directory (`.mlmodelc`) on the inference host for the LavaSR denoiser’s neural core; requires `lavasrDenoiserModelSrc` and `lavasrDenoiserCoremlComputeUnits`. A model that cannot load fails the load rather than falling back to ggml.' + ), + lavasrDenoiserCoremlComputeUnits: z + .enum(TTS_LAVASR_COREML_COMPUTE_UNITS) + .optional() + .describe( + 'macOS / iOS: Core ML compute policy for the LavaSR denoiser; required with `lavasrDenoiserCoremlModelPath`. Stats then report the stage as device `2` (Core ML-managed), backend `99`.' ) } +type TtsLavasrCoremlRefinementInput = { + lavasrEnhancerModelSrc?: ModelSrcInput | undefined + lavasrDenoiserModelSrc?: ModelSrcInput | undefined + lavasrEnhancerCoremlModelPath?: string | undefined + lavasrEnhancerCoremlComputeUnits?: string | undefined + lavasrDenoiserCoremlModelPath?: string | undefined + lavasrDenoiserCoremlComputeUnits?: string | undefined +} + +// Mirrors the addon's validateLavaSRCoreML: the path and the policy come as a +// pair, and a pair without the stage's GGUF has no stage to attach to. +function refineLavasrCoreml(config: TtsLavasrCoremlRefinementInput, ctx: z.RefinementCtx) { + for (const stage of ['Enhancer', 'Denoiser'] as const) { + const pathField = `lavasr${stage}CoremlModelPath` as const + const unitsField = `lavasr${stage}CoremlComputeUnits` as const + const srcField = `lavasr${stage}ModelSrc` as const + const path = config[pathField] + const units = config[unitsField] + if (path === undefined && units === undefined) continue + if (path === undefined || units === undefined) { + ctx.addIssue({ + code: 'custom', + path: [path === undefined ? pathField : unitsField], + message: `${pathField} and ${unitsField} must be supplied together.` + }) + } + if (config[srcField] === undefined) { + ctx.addIssue({ + code: 'custom', + path: [srcField], + message: `${srcField} is required when the LavaSR ${stage.toLowerCase()} runs on Core ML (${pathField}).` + }) + } + } +} + // The denoiser needs the whole utterance, so every engine with native chunk // streaming rejects the pair at load. Shared so Chatterbox, Parler and // CosyVoice3 fail the same way with the same message. @@ -1040,16 +1349,24 @@ export const ttsChatterboxLoadConfigSchema = ttsChatterboxRuntimeConfigSchema ), ...ttsLavasrLoadFieldsShape }) - .superRefine(refineLavasrDenoiserStreaming) + .superRefine(refineChatterboxLoadConfig) -type TtsSupertonicLoadRefinementInput = { +function refineChatterboxLoadConfig( + config: TtsLavasrCoremlRefinementInput & { streamChunkTokens?: number | undefined }, + ctx: z.RefinementCtx +) { + refineLavasrDenoiserStreaming(config, ctx) + refineLavasrCoreml(config, ctx) +} + +type TtsSupertonicLoadRefinementInput = TtsLavasrCoremlRefinementInput & { ttsSpeed?: number | undefined pace?: string | undefined useGPU?: boolean | undefined nGpuLayers?: number | undefined + voiceStyleTtl?: number[] | undefined + voiceStyleDp?: number[] | undefined streamChunkTokens?: number | undefined - lavasrEnhancerModelSrc?: ModelSrcInput | undefined - lavasrDenoiserModelSrc?: ModelSrcInput | undefined } // Supertonic native chunk streaming rejects BOTH LavaSR stages, not just the @@ -1061,6 +1378,7 @@ function refineSupertonicLoadConfig( ctx: z.RefinementCtx ) { refineSupertonicRateControls(config, ctx) + refineLavasrCoreml(config, ctx) if ((config.streamChunkTokens ?? 0) <= 0) return for (const field of ['lavasrEnhancerModelSrc', 'lavasrDenoiserModelSrc'] as const) { if (config[field] === undefined) continue @@ -1092,10 +1410,7 @@ export const ttsSupertonicLoadConfigSchema = z }) .superRefine(refineSupertonicLoadConfig) -type TtsParlerLoadRefinementInput = TtsParlerRuntimeRefinementInput & { - lavasrEnhancerModelSrc?: ModelSrcInput | undefined - lavasrDenoiserModelSrc?: ModelSrcInput | undefined -} +type TtsParlerLoadRefinementInput = TtsParlerRuntimeRefinementInput & TtsLavasrCoremlRefinementInput function refineParlerLoadConfig(config: TtsParlerLoadRefinementInput, ctx: z.RefinementCtx) { refineParlerDescriptionFields(config, ctx) @@ -1106,6 +1421,7 @@ function refineParlerLoadConfig(config: TtsParlerLoadRefinementInput, ctx: z.Ref refineParlerNativeStreamingRate(config, ctx) } refineLavasrDenoiserStreaming(config, ctx) + refineLavasrCoreml(config, ctx) } export const ttsParlerLoadConfigSchema = z @@ -1182,10 +1498,8 @@ function refineCosyvoice3Cloning( } type TtsCosyvoice3LoadRefinementInput = TtsCosyvoice3RefinementInput & - TtsCosyvoice3CloningRefinementInput & { - lavasrEnhancerModelSrc?: ModelSrcInput | undefined - lavasrDenoiserModelSrc?: ModelSrcInput | undefined - } + TtsCosyvoice3CloningRefinementInput & + TtsLavasrCoremlRefinementInput function refineCosyvoice3LoadConfig( config: TtsCosyvoice3LoadRefinementInput, @@ -1201,6 +1515,7 @@ function refineCosyvoice3LoadConfig( } refineLavasrDenoiserStreaming(config, ctx) + refineLavasrCoreml(config, ctx) refineCosyvoice3Cloning(config, ctx) } @@ -1277,17 +1592,22 @@ export const ttsAudio8LoadConfigSchema = z .superRefine(refineAudio8LoadConfig) type TtsMossLoadRefinementInput = { + ttsEngine: 'moss' + backend?: string | undefined mossCodecEncoderModelSrc?: ModelSrcInput | undefined referenceAudioSrc?: ModelSrcInput | undefined dialogueReferenceSrcs?: ModelSrcInput[] | undefined useGPU?: boolean | undefined nGpuLayers?: number | undefined + durationTokens?: number | undefined + maxNewTokens?: number | undefined } // The addon enforces the same three rules at construction; catching them here -// names the SDK field instead of the addon's `files.*` option. +// names the SDK field instead of the addon's `files.*` option. Built from the +// runtime *shape*, so the runtime refinements are re-run here too. function refineMossLoadConfig(config: TtsMossLoadRefinementInput, ctx: z.RefinementCtx) { - refineGpuIntent(config, ctx) + refineMossRuntimeConfig(config, ctx) if (config.referenceAudioSrc !== undefined && config.dialogueReferenceSrcs !== undefined) { ctx.addIssue({ @@ -1591,10 +1911,15 @@ export const ttsStatsSchema = z.object({ generatedFrames: z.number().optional(), // Backend selection captured once at model load. `0` CPU / `1` GPU; // `backendId` codes the family (0 CPU, 1 Metal, 2 CUDA, 3 Vulkan, 4 OpenCL, - // 99 other GPU). The enhancer and denoiser fields report each LavaSR stage - // separately and are `-1` when that stage is not loaded. + // 5 WebGPU, 99 other GPU). The enhancer and denoiser fields report each + // LavaSR stage separately and are `-1` when that stage is not loaded. backendDevice: z.number().optional(), - backendId: z.number().optional(), + backendId: z + .number() + .optional() + .describe( + 'Backend family the engine ran on: `0` CPU, `1` Metal, `2` CUDA, `3` Vulkan, `4` OpenCL, `5` WebGPU, `99` other.' + ), gpuUnsupported: z.number().optional(), enhancerBackendDevice: z.number().optional(), enhancerBackendId: z.number().optional(), @@ -1618,6 +1943,33 @@ export const ttsStatsSchema = z.object({ .describe( 'Audio8: `1` when this synthesis ran its codec on the Core ML sidecar, `0` when it ran on the ggml backend `backendId` reports.' ), + // Parler Core ML sidecars (macOS / iOS): availability flags keep the last + // report, the "used" flags are 1 only when every reported sentence used the + // stage. None of them says which hardware Core ML scheduled the work on. + decoderSidecarLoaded: z + .number() + .optional() + .describe( + 'Parler on macOS / iOS: `1` while the Core ML decoder sidecar is attached, `0` once a fallback retired it.' + ), + dacSidecarLoaded: z + .number() + .optional() + .describe( + 'Parler on macOS / iOS: `1` while the Core ML DAC sidecar is attached, `0` once a fallback retired it.' + ), + decoderOnCoreml: z + .number() + .optional() + .describe( + 'Parler: `1` when this synthesis ran its decoder steps on Core ML, `0` for ggml or a mixed run.' + ), + dacOnCoreml: z + .number() + .optional() + .describe( + 'Parler: `1` when this synthesis ran its DAC on Core ML, `0` for ggml or a mixed run.' + ), // Chatterbox stage timings and counters. t3Ms: ttsStageMsSchema('Chatterbox T3 decode'), s3genMs: ttsStageMsSchema('Chatterbox S3Gen + HiFT'), @@ -1682,6 +2034,18 @@ const ttsResponseSampleRateSchema = z .optional() .describe('Sample rate in Hz of the signed 16-bit mono PCM in `buffer`.') +// CosyVoice3 owns its speech-token trajectory and hands it back: with the PCM +// in batch synthesis, on a final `isLast` frame with no audio under native +// chunk streaming. One frame's array replays that one native synthesis through +// the load config's `forceSpeechTokens`; the sentence paths run one synthesis +// per sentence, so their arrays replay sentence by sentence, not joined. +const ttsResponseSpeechTokenIdsSchema = z + .array(ttsSpeechTokenIdSchema) + .optional() + .describe( + 'CosyVoice3: the speech tokens of this native synthesis. Batch synthesis carries them with the PCM; native chunk streaming on a final `isLast` frame with no audio; the sentence paths once per sentence. Pass one array back as `forceSpeechTokens` to replay that synthesis (a joined sentence stream cannot be replayed as one).' + ) + export const ttsResponseSchema = z.object({ type: z.literal('textToSpeech'), text: z.string().optional().describe('MOSS-Speech reply text, including text-only replies.'), @@ -1703,6 +2067,7 @@ export const ttsResponseSchema = z.object({ .describe( 'True on the final audio-bearing chunk of a pre-chunked synthesis. Absent when the chunk count is not known up front (streamed text in).' ), + speechTokenIds: ttsResponseSpeechTokenIdsSchema, // On the terminal frame only: how the run ended. A run stopped by // `cancel()` ends cleanly with `cancelled` rather than as an error. stopReason: z.enum(['completed', 'cancelled']).optional() @@ -1758,6 +2123,7 @@ export const textToSpeechStreamResponseSchema = z.object({ .describe( 'True on the final audio-bearing chunk of a pre-chunked synthesis. Absent when the chunk count is not known up front (streamed text in).' ), + speechTokenIds: ttsResponseSpeechTokenIdsSchema, // On the terminal frame only: how the run ended. A run stopped by // `cancel()` ends cleanly with `cancelled` rather than as an error. stopReason: z.enum(['completed', 'cancelled']).optional() @@ -1807,6 +2173,11 @@ export type TtsSentenceChunkUpdate = { chunkIndex?: number sentenceChunk?: string isLast?: boolean + /** + * CosyVoice3: the speech tokens of this sentence's synthesis, when the frame + * carried them. Each array replays its own sentence through `forceSpeechTokens`. + */ + speechTokenIds?: number[] } export type TextToSpeechStreamRequest = z.infer @@ -1854,6 +2225,18 @@ export interface TextToSpeechStreamResult { * `'completed'` otherwise. `undefined` if no terminal frame was seen. */ stopReason: Promise<'completed' | 'cancelled' | undefined> + /** + * CosyVoice3: every speech token the run emitted, in order (one native + * synthesis per batch run; one array per sentence, concatenated, on the + * chunked paths). `undefined` when no frame carried any. + * + * Replay through the load config's `forceSpeechTokens` works for ONE native + * synthesis: a batch run, a native chunk stream, or a single sentence's + * array from `chunkUpdates`. `forceSpeechTokens` is fixed at load and every + * sentence of a chunked run starts its own synthesis with that same array, + * so replaying this concatenation does not restore the sentences it joined. + */ + speechTokenIds: Promise } export interface TextToSpeechStreamSession { diff --git a/packages/inference/src/utils/addon-responses.ts b/packages/inference/src/utils/addon-responses.ts index 6e83e73bedf..b3b18096f82 100644 --- a/packages/inference/src/utils/addon-responses.ts +++ b/packages/inference/src/utils/addon-responses.ts @@ -53,6 +53,12 @@ export interface TtsStats { codecSidecarLoaded?: number /** Audio8 only: 1 when the synthesis ran its codec on that sidecar, 0 on ggml. */ codecOnCoreml?: number + /** Parler only, macOS / iOS: 1 while the decoder / DAC Core ML sidecar is attached. */ + decoderSidecarLoaded?: number + dacSidecarLoaded?: number + /** Parler only: 1 when the synthesis ran that stage on Core ML, 0 for ggml or a mixed run. */ + decoderOnCoreml?: number + dacOnCoreml?: number /** Milliseconds to the first audio, where the engine reports it. */ firstAudioMs?: number /** Chatterbox: T3 decode and S3Gen + HiFT wall times (ms), speech tokens T3 emitted. */ @@ -103,6 +109,11 @@ export interface TtsOutputChunk { sentenceChunk?: string /** True on the final pre-chunked output; undefined when it is not known up front. */ isLast?: boolean + /** + * CosyVoice3: the speech tokens of this synthesis. Batch output carries them + * with the PCM; native chunk streaming on a final `isLast` chunk with no PCM. + */ + speechTokenIds?: number[] } export interface TtsResponse { diff --git a/packages/inference/src/utils/tts-stats.ts b/packages/inference/src/utils/tts-stats.ts index fa24bb06d83..b645bcd9e66 100644 --- a/packages/inference/src/utils/tts-stats.ts +++ b/packages/inference/src/utils/tts-stats.ts @@ -18,6 +18,8 @@ export type TtsOpYield = { chunkIndex?: number sentenceChunk?: string isLast?: boolean + /** CosyVoice3: the speech tokens of the synthesis this frame belongs to. */ + speechTokenIds?: number[] } /** @@ -33,10 +35,21 @@ export function chunkMetadata(chunk: TtsStreamChunk) { ...(typeof chunk.sentenceChunk === 'string' && chunk.sentenceChunk.length > 0 ? { sentenceChunk: chunk.sentenceChunk } : {}), - ...(chunk.isLast !== undefined ? { isLast: chunk.isLast } : {}) + ...(chunk.isLast !== undefined ? { isLast: chunk.isLast } : {}), + ...(chunk.speechTokenIds !== undefined ? { speechTokenIds: chunk.speechTokenIds } : {}) } } +/** + * Whether a chunk has anything to forward besides PCM. CosyVoice3's native + * chunk streaming ends with an empty-PCM `isLast` chunk that carries the + * run's `speechTokenIds`; Audio8's native end markers are empty too but carry + * nothing, so the ops skip those and forward these. + */ +export function carriesSpeechTokenIds(chunk: TtsStreamChunk): boolean { + return chunk.speechTokenIds !== undefined +} + /** * Appends a chunk's samples to `target` in place, which keeps collecting an * utterance linear in its length — a natively chunked MOSS run emits dozens of @@ -104,6 +117,11 @@ const TTS_FORWARDED_STATS = [ 'denoiserBackendId', 'codecSidecarLoaded', 'codecOnCoreml', + // Parler Core ML sidecars + 'decoderSidecarLoaded', + 'dacSidecarLoaded', + 'decoderOnCoreml', + 'dacOnCoreml', // Chatterbox 't3Ms', 's3genMs', diff --git a/packages/inference/test/tts-fit-request.test.ts b/packages/inference/test/tts-fit-request.test.ts index a1b2ec028ff..a8970b9a68c 100644 --- a/packages/inference/test/tts-fit-request.test.ts +++ b/packages/inference/test/tts-fit-request.test.ts @@ -146,3 +146,75 @@ test('CosyVoice: finds its companions beside the checkpoint', (t) => { fs.rmSync(path.join(dir, 'voice.gguf')) t.ok('unknown' in ttsFitRequest(load(llm, { ttsEngine: 'cosyvoice3' })), 'a missing companion') }) + +test('the explicit backend selector reaches every fit request', (t) => { + // The fitter resolves the same device the load would, so the projection + // lands there rather than on the useGPU policy's pick. + const supertonic = ttsFitRequest( + load('/models/supertonic.gguf', { ttsEngine: 'supertonic', backend: 'MTL0' }) + ) + t.is(requestOf(supertonic)?.['backend'], 'MTL0') + const parler = ttsFitRequest(load('/models/parler.gguf', { ttsEngine: 'parler', backend: 'cpu' })) + t.is(requestOf(parler)?.['backend'], 'cpu') + const chatterbox = ttsFitRequest( + load( + '/models/t3.gguf', + { ttsEngine: 'chatterbox', backend: 'Vulkan0' }, + { artifacts: { s3genPath: '/models/s3gen.gguf' } } + ) + ) + t.is(requestOf(chatterbox)?.['backend'], 'Vulkan0') + t.is( + 'backend' in + (requestOf(ttsFitRequest(load('/models/parler.gguf', { ttsEngine: 'parler' }))) ?? {}), + false, + 'absent when unset' + ) +}) + +test('supertonic: carries the precision and f16 weight override the fitter sizes by', (t) => { + const plan = ttsFitRequest( + load('/models/supertonic3-q8_0.gguf', { + ttsEngine: 'supertonic', + precision: 'f16', + f16Weights: 1, + ttsNumInferenceSteps: 4 + }) + ) + t.alike(requestOf(plan), { + engineType: 'supertonic', + supertonicModelPath: '/models/supertonic3-q8_0.gguf', + nGpuLayers: 0, + precision: 'f16', + f16Weights: 1, + steps: 4 + }) +}) + +test('parler: carries the decoder sidecar policy', (t) => { + const plan = ttsFitRequest( + load('/models/parler-mini-v1-q8_0.gguf', { ttsEngine: 'parler', coremlDecoder: true }) + ) + t.alike(requestOf(plan), { + engineType: 'parler', + parlerModelPath: '/models/parler-mini-v1-q8_0.gguf', + nGpuLayers: 0, + coremlDecoder: true + }) +}) + +test('MOSS: carries the context and generation budgets', (t) => { + const plan = ttsFitRequest( + load( + '/models/moss.gguf', + { ttsEngine: 'moss', nCtx: 8192, maxNewTokens: 4096 }, + { + artifacts: { mossCodecDecoderPath: '/models/decoder.gguf' }, + ttsWorkload: { promptRows: 128, referenceSamples: 0, streaming: false } + } + ) + ) + const request = requestOf(plan) + t.is(request?.['nCtx'], 8192) + t.is(request?.['maxNewTokens'], 4096) +}) diff --git a/packages/inference/test/tts-resolve-config.test.ts b/packages/inference/test/tts-resolve-config.test.ts index 2c825e54158..dde49ffa32b 100644 --- a/packages/inference/test/tts-resolve-config.test.ts +++ b/packages/inference/test/tts-resolve-config.test.ts @@ -26,8 +26,32 @@ type TtsGgmlDebugModel = { backend: { enhancerGgufPath?: string denoiserGgufPath?: string + enhancerCoremlModelPath?: string + enhancerCoremlComputeUnits?: string + denoiserCoremlModelPath?: string + denoiserCoremlComputeUnits?: string } settings: { + precision?: string + f16Attn?: number + f16Weights?: number + kvAttnType?: number + f16WeightsDenyList?: string[] + vulkanEnvOverrides?: Record + voiceStyleTtl?: number[] + voiceStyleDp?: number[] + coremlDecoder?: boolean + coremlComputeUnits?: string + coremlStrict?: boolean + forceSpeechTokens?: number[] + maxNewTokens?: number + textTemperature?: number + textTopP?: number + textTopK?: number + audioTemperature?: number + audioTopP?: number + audioTopK?: number + audioRepetitionPenalty?: number streamChunkTokens?: number streamFirstChunkTokens?: number cfmSteps?: number @@ -1004,3 +1028,222 @@ test('ttsPlugin createModel: forwards CosyVoice3 flowCutPrompt and vulkanDevice' t.is(model._setup.settings.flowCutPrompt, true) t.is(model._setup.settings.vulkanDevice, -1) }) + +// === @qvac/tts-ggml 0.15: backend selector, Fabric controls, Core ML === + +test('ttsPlugin createModel: forwards the explicit backend selector on every engine', async (t) => { + const { ttsPlugin } = await import('@/plugins/builtin/tts-ggml/plugin') + + const cases: Array<{ config: Record; artifacts: Record }> = [ + { + config: { ttsEngine: 'chatterbox', language: 'en' }, + artifacts: { s3genPath: '/tmp/s3gen.gguf' } + }, + { config: { ttsEngine: 'supertonic', language: 'en' }, artifacts: {} }, + { config: { ttsEngine: 'parler' }, artifacts: {} }, + { config: { ttsEngine: 'cosyvoice3' }, artifacts: {} }, + { config: { ttsEngine: 'audio8' }, artifacts: { audio8CodecDecoderPath: '/tmp/dec.gguf' } }, + { config: { ttsEngine: 'moss' }, artifacts: { mossCodecDecoderPath: '/tmp/dec.gguf' } }, + { config: { ttsEngine: 'moss-sfx' }, artifacts: {} }, + { config: { ttsEngine: 'moss-speech' }, artifacts: { mossSpeechCodecPath: '/tmp/codec.gguf' } } + ] + for (const { config, artifacts } of cases) { + const result = ttsPlugin.createModel({ + modelId: `tts-backend-${String(config['ttsEngine'])}`, + modelPath: '/tmp/model.gguf', + artifacts, + modelConfig: { ...config, backend: 'MTL0' } + }) + const model = result.model as unknown as TtsGgmlDebugModel + // The addon reads it from `config`, beside useGPU; it is not a settings knob. + t.is(model._setup.config.backend, 'MTL0', String(config['ttsEngine'])) + } +}) + +test('ttsPlugin createModel: forwards the Supertonic Fabric controls', async (t) => { + const { ttsPlugin } = await import('@/plugins/builtin/tts-ggml/plugin') + + const result = ttsPlugin.createModel({ + modelId: 'tts-supertonic-fabric', + modelPath: '/tmp/supertonic3-q8_0.gguf', + artifacts: {}, + modelConfig: { + ttsEngine: 'supertonic', + language: 'en', + precision: 'f16', + f16Attn: 1, + f16Weights: -1, + kvAttnType: 0, + f16WeightsDenyList: ['vocoder.*'], + vulkanEnvOverrides: { GGML_VK_PIPELINE_CACHE_DIR: '/cache/vk' }, + voiceStyleTtl: [0.1, 0.2], + voiceStyleDp: [0.3] + } + }) + + const model = result.model as unknown as TtsGgmlDebugModel + t.is(model._setup.settings.precision, 'f16') + t.is(model._setup.settings.f16Attn, 1) + t.is(model._setup.settings.f16Weights, -1) + t.is(model._setup.settings.kvAttnType, 0) + t.alike(model._setup.settings.f16WeightsDenyList, ['vocoder.*']) + t.alike(model._setup.settings.vulkanEnvOverrides, { GGML_VK_PIPELINE_CACHE_DIR: '/cache/vk' }) + t.alike(model._setup.settings.voiceStyleTtl, [0.1, 0.2]) + t.alike(model._setup.settings.voiceStyleDp, [0.3]) +}) + +test('ttsPlugin createModel: forwards the Parler Core ML sidecar controls', async (t) => { + const { ttsPlugin } = await import('@/plugins/builtin/tts-ggml/plugin') + + const result = ttsPlugin.createModel({ + modelId: 'tts-parler-coreml', + modelPath: '/tmp/parler-mini-v1-q8_0.gguf', + artifacts: {}, + modelConfig: { + ttsEngine: 'parler', + coremlDecoder: true, + coremlComputeUnits: 'cpu_and_ane', + coremlStrict: false + } + }) + + const model = result.model as unknown as TtsGgmlDebugModel + t.is(model._setup.settings.coremlDecoder, true) + t.is(model._setup.settings.coremlComputeUnits, 'cpu_and_ane') + t.is(model._setup.settings.coremlStrict, false) +}) + +test('ttsPlugin createModel: forwards CosyVoice3 greedy decoding and a forced trajectory', async (t) => { + const { ttsPlugin } = await import('@/plugins/builtin/tts-ggml/plugin') + + const result = ttsPlugin.createModel({ + modelId: 'tts-cosyvoice3-replay', + modelPath: '/tmp/qvac/sets/cosyvoice3/cosyvoice3-llm-q8_0.gguf', + artifacts: {}, + modelConfig: { ttsEngine: 'cosyvoice3', greedy: true, forceSpeechTokens: [12, 480, 3] } + }) + + const model = result.model as unknown as TtsGgmlDebugModel + t.is(model._setup.settings.greedy, true) + t.alike(model._setup.settings.forceSpeechTokens, [12, 480, 3]) +}) + +test('ttsPlugin createModel: forwards the MOSS budgets and channel samplers', async (t) => { + const { ttsPlugin } = await import('@/plugins/builtin/tts-ggml/plugin') + + const result = ttsPlugin.createModel({ + modelId: 'tts-moss-fabric', + modelPath: '/tmp/moss-tts-delay-f16.gguf', + artifacts: { mossCodecDecoderPath: '/tmp/moss-codec-decoder-f16.gguf' }, + modelConfig: { + ttsEngine: 'moss', + nCtx: 8192, + maxNewTokens: 4096, + durationTokens: 3000, + textTemperature: 0.7, + textTopP: 0.9, + textTopK: 40, + audioTemperature: 0.8, + audioTopP: 0.95, + audioTopK: 50, + audioRepetitionPenalty: 1.1 + } + }) + + const model = result.model as unknown as TtsGgmlDebugModel + t.is(model._setup.settings.nCtx, 8192) + t.is(model._setup.settings.maxNewTokens, 4096) + // Accepted by the addon only because the budget was raised above 2048. + t.is(model._setup.settings.durationTokens, 3000) + t.is(model._setup.settings.textTemperature, 0.7) + t.is(model._setup.settings.textTopP, 0.9) + t.is(model._setup.settings.textTopK, 40) + t.is(model._setup.settings.audioTemperature, 0.8) + t.is(model._setup.settings.audioTopP, 0.95) + t.is(model._setup.settings.audioTopK, 50) + t.is(model._setup.settings.audioRepetitionPenalty, 1.1) +}) + +test('ttsPlugin resolveConfig: moves the LavaSR Core ML fields into artifacts and strips them', async (t) => { + const { ttsPlugin } = await import('@/plugins/builtin/tts-ggml/plugin') + + for (const base of [ + { + ttsEngine: 'chatterbox', + language: 'en', + s3genModelSrc: 'registry://s3/chatterbox/s3gen.gguf' + }, + { ttsEngine: 'supertonic', language: 'en' }, + { ttsEngine: 'parler' }, + { ttsEngine: 'cosyvoice3' } + ]) { + const resolved = await ttsPlugin.resolveConfig!( + { + ...base, + lavasrEnhancerModelSrc: 'registry://s3/lavasr/enhancer.gguf', + lavasrEnhancerCoremlModelPath: '/models/enhancer.mlmodelc', + lavasrEnhancerCoremlComputeUnits: 'CPU_AND_NE', + lavasrDenoiserModelSrc: 'registry://s3/lavasr/denoiser.gguf', + lavasrDenoiserCoremlModelPath: '/models/denoiser.mlmodelc', + lavasrDenoiserCoremlComputeUnits: 'ALL' + }, + { + resolveModelPath: async (src: unknown) => `/cache/${String(src)}`, + modelSrc: 'registry://s3/model.gguf', + modelType: 'tts-ggml' + } + ) + const engine = base.ttsEngine + const config = resolved.config as Record + for (const field of [ + 'lavasrEnhancerCoremlModelPath', + 'lavasrEnhancerCoremlComputeUnits', + 'lavasrDenoiserCoremlModelPath', + 'lavasrDenoiserCoremlComputeUnits' + ]) { + t.is(field in config, false, `${engine}: ${field} must not leak into the runtime config`) + } + const artifacts = resolved.artifacts ?? {} + t.is(artifacts['lavasrEnhancerPath'], '/cache/registry://s3/lavasr/enhancer.gguf', engine) + t.is(artifacts['lavasrEnhancerCoremlModelPath'], '/models/enhancer.mlmodelc', engine) + t.is(artifacts['lavasrEnhancerCoremlComputeUnits'], 'CPU_AND_NE', engine) + t.is(artifacts['lavasrDenoiserCoremlModelPath'], '/models/denoiser.mlmodelc', engine) + t.is(artifacts['lavasrDenoiserCoremlComputeUnits'], 'ALL', engine) + } +}) + +test('ttsPlugin createModel: builds the LavaSR Core ML options beside the GGUF files', async (t) => { + const { ttsPlugin } = await import('@/plugins/builtin/tts-ggml/plugin') + + const result = ttsPlugin.createModel({ + modelId: 'tts-supertonic-lavasr-coreml', + modelPath: '/tmp/supertonic3-q8_0.gguf', + artifacts: { + lavasrEnhancerPath: '/tmp/lavasr-enhancer.gguf', + lavasrEnhancerCoremlModelPath: '/models/enhancer.mlmodelc', + lavasrEnhancerCoremlComputeUnits: 'CPU_AND_NE', + lavasrDenoiserPath: '/tmp/lavasr-denoiser.gguf', + lavasrDenoiserCoremlModelPath: '/models/denoiser.mlmodelc', + lavasrDenoiserCoremlComputeUnits: 'ALL' + }, + modelConfig: { ttsEngine: 'supertonic', language: 'en' } + }) + + const model = result.model as unknown as TtsGgmlDebugModel + t.is(model._setup.backend.enhancerGgufPath, '/tmp/lavasr-enhancer.gguf') + t.is(model._setup.backend.enhancerCoremlModelPath, '/models/enhancer.mlmodelc') + t.is(model._setup.backend.enhancerCoremlComputeUnits, 'CPU_AND_NE') + t.is(model._setup.backend.denoiserGgufPath, '/tmp/lavasr-denoiser.gguf') + t.is(model._setup.backend.denoiserCoremlModelPath, '/models/denoiser.mlmodelc') + t.is(model._setup.backend.denoiserCoremlComputeUnits, 'ALL') + + // Without the Core ML artifacts the stage stays on ggml, as before. + const plain = ttsPlugin.createModel({ + modelId: 'tts-supertonic-lavasr-ggml', + modelPath: '/tmp/supertonic3-q8_0.gguf', + artifacts: { lavasrEnhancerPath: '/tmp/lavasr-enhancer.gguf' }, + modelConfig: { ttsEngine: 'supertonic', language: 'en' } + }) + const plainModel = plain.model as unknown as TtsGgmlDebugModel + t.is(plainModel._setup.backend.enhancerCoremlModelPath, undefined) +}) diff --git a/packages/inference/test/tts-schemas.test.ts b/packages/inference/test/tts-schemas.test.ts index adef87326c9..a4ba9d5b81c 100644 --- a/packages/inference/test/tts-schemas.test.ts +++ b/packages/inference/test/tts-schemas.test.ts @@ -17,6 +17,9 @@ import { TTS_SUPERTONIC_LANGUAGES, TTS_ENGINES, TTS_SENTENCE_DELIMITER_PRESETS, + TTS_PARLER_COREML_COMPUTE_UNITS, + TTS_LAVASR_COREML_COMPUTE_UNITS, + TTS_SUPERTONIC_PRECISIONS, LEGACY_TTS_ONNX_MODEL_CONFIG_FIELDS } from '@/schemas/text-to-speech' @@ -1823,3 +1826,351 @@ test('ttsStatsSchema: carries the engines’ stage timings and counters', (t) => if (r.success) t.alike(r.data.stats, stats) } }) + +// === @qvac/tts-ggml 0.15 surface: backend selector, Fabric controls, Core ML === + +// Minimal valid load config per engine, for the shared `backend` field. +const ENGINE_BASES: Record> = { + chatterbox: { ttsEngine: 'chatterbox', language: 'en', s3genModelSrc: 's3:///a/s3gen.gguf' }, + supertonic: { ttsEngine: 'supertonic', language: 'en' }, + parler: { ttsEngine: 'parler' }, + cosyvoice3: { ttsEngine: 'cosyvoice3' }, + audio8: { ttsEngine: 'audio8', audio8CodecDecoderModelSrc: 's3:///a/dec.gguf' }, + moss: { ttsEngine: 'moss', mossCodecDecoderModelSrc: 's3:///a/dec.gguf' }, + 'moss-sfx': { ttsEngine: 'moss-sfx' }, + 'moss-speech': { ttsEngine: 'moss-speech', mossSpeechCodecModelSrc: 's3:///a/codec.gguf' } +} + +test('ttsConfigSchema: accepts the explicit backend selector on every engine', (t) => { + for (const [engine, base] of Object.entries(ENGINE_BASES)) { + for (const backend of ['auto', 'cpu', 'MTL0', 'CUDA0', 'Vulkan0']) { + const r = ttsConfigSchema.safeParse({ ...base, backend }) + t.is(r.success, true, `${engine} backend ${backend}`) + if (r.success) t.is((r.data as Record)['backend'], backend) + } + // A blank selector would reach the addon as an unknown backend name. + t.is(ttsConfigSchema.safeParse({ ...base, backend: ' ' }).success, false, `${engine} blank`) + } +}) + +test('ttsConfigSchema: accepts WebGPU only on the engines the native library supports it for', (t) => { + // Mirrors the addon's validateBackendOption / WEBGPU_ENGINES. + for (const engine of ['supertonic', 'parler', 'cosyvoice3', 'audio8']) { + t.is( + ttsConfigSchema.safeParse({ ...ENGINE_BASES[engine], backend: 'webgpu' }).success, + true, + engine + ) + } + for (const engine of ['chatterbox', 'moss', 'moss-sfx', 'moss-speech']) { + for (const backend of ['webgpu', 'WebGPU0']) { + const r = ttsConfigSchema.safeParse({ ...ENGINE_BASES[engine], backend }) + t.is(r.success, false, `${engine} ${backend}`) + if (!r.success) { + t.is(r.error.issues[0]?.path.join('.'), 'backend') + t.ok(r.error.issues[0]?.message.includes(engine)) + } + } + } +}) + +test('ttsConfigSchema: accepts the Supertonic precision, environment and in-memory voice controls', (t) => { + const r = supertonic({ + precision: 'q8_0', + f16Attn: 1, + f16Weights: -1, + kvAttnType: 0, + f16WeightsDenyList: ['vocoder.*', 'dp.*'], + vulkanEnvOverrides: { GGML_VK_PIPELINE_CACHE_DIR: '/cache/vk' }, + voiceStyleTtl: [0.1, 0.2, 0.3], + voiceStyleDp: [0.4, 0.5] + }) + t.is(r.success, true) + if (r.success) { + const data = r.data as Record + t.is(data['precision'], 'q8_0') + t.is(data['f16Attn'], 1) + t.is(data['f16Weights'], -1) + t.is(data['kvAttnType'], 0) + t.alike(data['f16WeightsDenyList'], ['vocoder.*', 'dp.*']) + t.alike(data['vulkanEnvOverrides'], { GGML_VK_PIPELINE_CACHE_DIR: '/cache/vk' }) + t.alike(data['voiceStyleTtl'], [0.1, 0.2, 0.3]) + t.alike(data['voiceStyleDp'], [0.4, 0.5]) + } + for (const precision of TTS_SUPERTONIC_PRECISIONS) { + t.is(supertonic({ precision }).success, true, precision) + } + t.alike([...TTS_SUPERTONIC_PRECISIONS], ['auto', 'f32', 'f16', 'q8_0']) +}) + +test('ttsConfigSchema: bounds the Supertonic Fabric controls like the addon', (t) => { + t.is(supertonic({ precision: 'q4_0' }).success, false, 'not a precision the engine loads at') + t.is(supertonic({ f16Attn: 1.5 }).success, false, 'int32') + t.is(supertonic({ kvAttnType: 2147483648 }).success, false, 'int32') + t.is(supertonic({ f16WeightsDenyList: ['', 'x'] }).success, false, 'empty pattern') + t.is(supertonic({ vulkanEnvOverrides: { A: 1 } }).success, false, 'string values only') + t.is(supertonic({ vulkanEnvOverrides: ['A=1'] }).success, false, 'an object, not a list') + t.is(supertonic({ voiceStyleTtl: [] }).success, false, 'an empty tensor is not a voice') +}) + +test('ttsConfigSchema: requires the Supertonic voice tensors as a pair', (t) => { + for (const [half, other] of [ + ['voiceStyleTtl', 'voiceStyleDp'], + ['voiceStyleDp', 'voiceStyleTtl'] + ] as const) { + for (const schema of [ttsConfigSchema, ttsSupertonicRuntimeConfigSchema]) { + const r = schema.safeParse({ ttsEngine: 'supertonic', language: 'en', [half]: [0.1] }) + t.is(r.success, false, `${half} alone`) + if (!r.success) t.is(r.error.issues[0]?.path.join('.'), other) + } + } +}) + +test('ttsConfigSchema: rejects the Supertonic Fabric controls on other engines', (t) => { + for (const bad of [ + { ...ENGINE_BASES['chatterbox'], precision: 'f16' }, + { ...ENGINE_BASES['parler'], f16Weights: 1 }, + { ...ENGINE_BASES['cosyvoice3'], vulkanEnvOverrides: { A: 'b' } }, + { ...ENGINE_BASES['audio8'], voiceStyleTtl: [0.1], voiceStyleDp: [0.2] } + ]) { + t.is(ttsConfigSchema.safeParse(bad).success, false, JSON.stringify(bad)) + } +}) + +test('ttsConfigSchema: accepts the Parler Core ML sidecar controls', (t) => { + const r = ttsConfigSchema.safeParse({ + ttsEngine: 'parler', + coremlDecoder: true, + coremlComputeUnits: 'cpu_and_ane', + coremlStrict: false + }) + t.is(r.success, true) + if (r.success) { + const data = r.data as Record + t.is(data['coremlDecoder'], true) + t.is(data['coremlComputeUnits'], 'cpu_and_ane') + t.is(data['coremlStrict'], false) + } + for (const coremlComputeUnits of TTS_PARLER_COREML_COMPUTE_UNITS) { + t.is( + ttsConfigSchema.safeParse({ ttsEngine: 'parler', coremlComputeUnits }).success, + true, + coremlComputeUnits + ) + } + t.alike([...TTS_PARLER_COREML_COMPUTE_UNITS], ['all', 'cpu_and_ane', 'cpu_and_gpu', 'cpu_only']) + t.is(ttsConfigSchema.safeParse({ ttsEngine: 'parler', coremlComputeUnits: 'ane' }).success, false) + t.is(ttsConfigSchema.safeParse({ ttsEngine: 'parler', coremlDecoder: 'yes' }).success, false) + // Parler-only: the addon rejects them by name on every other engine. + t.is(supertonic({ coremlDecoder: true }).success, false) + t.is(ttsConfigSchema.safeParse({ ...ENGINE_BASES['audio8'], coremlStrict: true }).success, false) +}) + +test('ttsConfigSchema: accepts CosyVoice3 greedy decoding and a forced speech-token trajectory', (t) => { + const r = ttsConfigSchema.safeParse({ + ttsEngine: 'cosyvoice3', + greedy: true, + forceSpeechTokens: [12, 480, 3, 0] + }) + t.is(r.success, true) + if (r.success) { + const data = r.data as Record + t.is(data['greedy'], true) + t.alike(data['forceSpeechTokens'], [12, 480, 3, 0]) + } + // An empty trajectory restores generation, so it is a valid value. + t.is(ttsConfigSchema.safeParse({ ttsEngine: 'cosyvoice3', forceSpeechTokens: [] }).success, true) + t.is( + ttsConfigSchema.safeParse({ ttsEngine: 'cosyvoice3', forceSpeechTokens: [1.5] }).success, + false, + 'token ids are integers' + ) + t.is( + ttsConfigSchema.safeParse({ ttsEngine: 'cosyvoice3', forceSpeechTokens: [2147483648] }).success, + false, + 'int32' + ) + t.is(supertonic({ forceSpeechTokens: [1] }).success, false, 'CosyVoice3-only') + t.is(supertonic({ greedy: true }).success, false, 'Audio8 / CosyVoice3 only') +}) + +test('ttsConfigSchema: accepts the MOSS text and audio sampling controls and budgets', (t) => { + const r = moss({ + nCtx: 8192, + maxNewTokens: 4096, + textTemperature: 0.7, + textTopP: 0.9, + textTopK: 40, + audioTemperature: 0.8, + audioTopP: 0.95, + audioTopK: 50, + audioRepetitionPenalty: 1.1 + }) + t.is(r.success, true) + if (r.success) { + const data = r.data as Record + t.is(data['nCtx'], 8192) + t.is(data['maxNewTokens'], 4096) + t.is(data['textTemperature'], 0.7) + t.is(data['textTopP'], 0.9) + t.is(data['textTopK'], 40) + t.is(data['audioTemperature'], 0.8) + t.is(data['audioTopP'], 0.95) + t.is(data['audioTopK'], 50) + t.is(data['audioRepetitionPenalty'], 1.1) + } + // The addon requires positive 32-bit counts and finite samplers. + t.is(moss({ nCtx: 0 }).success, false) + t.is(moss({ maxNewTokens: 0 }).success, false) + t.is(moss({ maxNewTokens: 2147483648 }).success, false) + t.is(moss({ textTopK: -1 }).success, false) + t.is(moss({ textTopK: 1.5 }).success, false) + t.is(moss({ audioTopP: 0 }).success, false) + t.is(moss({ audioTopP: 1.5 }).success, false) + t.is(moss({ textTemperature: -0.1 }).success, false) + t.is(moss({ audioRepetitionPenalty: 0 }).success, false) + // MOSS-only: the addon rejects them by name elsewhere. + t.is(supertonic({ textTemperature: 0.7 }).success, false) + t.is(ttsConfigSchema.safeParse({ ...ENGINE_BASES['moss-sfx'], audioTopK: 5 }).success, false) + t.is(ttsConfigSchema.safeParse({ ...ENGINE_BASES['audio8'], maxNewTokens: 10 }).success, false) +}) + +test('ttsConfigSchema: bounds MOSS durationTokens by the configured generation budget', (t) => { + // maxNewTokens less the 33 delay/EOS rows, 2048 - 33 = 2015 by default. + t.is(moss({ maxNewTokens: 1024, durationTokens: 991 }).success, true) + const over = moss({ maxNewTokens: 1024, durationTokens: 992 }) + t.is(over.success, false) + if (!over.success) { + t.is(over.error.issues[0]?.path.join('.'), 'durationTokens') + t.ok(over.error.issues[0]?.message.includes('991')) + } + t.is(moss({ maxNewTokens: 4096, durationTokens: 4063 }).success, true, 'a larger budget') + t.is(moss({ maxNewTokens: 4096, durationTokens: 4064 }).success, false) + t.is(moss({ maxNewTokens: 1024, durationTokens: 0 }).success, true, '0 keeps the length free') + // A budget too small for any audio leaves only 0. + t.is(moss({ maxNewTokens: 10, durationTokens: 1 }).success, false) +}) + +test('ttsConfigSchema: accepts the LavaSR Core ML models on the engines that run LavaSR', (t) => { + const coreml = { + lavasrEnhancerModelSrc: 's3:///lavasr/enhancer.gguf', + lavasrEnhancerCoremlModelPath: '/models/enhancer.mlmodelc', + lavasrEnhancerCoremlComputeUnits: 'CPU_AND_NE', + lavasrDenoiserModelSrc: 's3:///lavasr/denoiser.gguf', + lavasrDenoiserCoremlModelPath: '/models/denoiser.mlmodelc', + lavasrDenoiserCoremlComputeUnits: 'ALL' + } + for (const engine of ['chatterbox', 'supertonic', 'parler', 'cosyvoice3']) { + const r = ttsConfigSchema.safeParse({ ...ENGINE_BASES[engine], ...coreml }) + t.is(r.success, true, engine) + if (r.success) { + const data = r.data as Record + t.is(data['lavasrEnhancerCoremlModelPath'], '/models/enhancer.mlmodelc') + t.is(data['lavasrEnhancerCoremlComputeUnits'], 'CPU_AND_NE') + t.is(data['lavasrDenoiserCoremlModelPath'], '/models/denoiser.mlmodelc') + t.is(data['lavasrDenoiserCoremlComputeUnits'], 'ALL') + } + } + for (const units of TTS_LAVASR_COREML_COMPUTE_UNITS) { + t.is( + ttsConfigSchema.safeParse({ + ttsEngine: 'parler', + lavasrEnhancerModelSrc: 's3:///lavasr/enhancer.gguf', + lavasrEnhancerCoremlModelPath: '/models/enhancer.mlmodelc', + lavasrEnhancerCoremlComputeUnits: units + }).success, + true, + units + ) + } + t.alike([...TTS_LAVASR_COREML_COMPUTE_UNITS], ['CPU_ONLY', 'CPU_AND_GPU', 'CPU_AND_NE', 'ALL']) + // Audio8 and MOSS take no LavaSR stage, so no Core ML model for one either. + for (const engine of ['audio8', 'moss']) { + t.is(ttsConfigSchema.safeParse({ ...ENGINE_BASES[engine], ...coreml }).success, false, engine) + } +}) + +test('ttsConfigSchema: enforces the LavaSR Core ML pairing rules', (t) => { + // Mirrors the addon's validateLavaSRCoreML: path and policy together, and a + // stage to attach to. + const base = ENGINE_BASES['cosyvoice3'] + const noUnits = ttsConfigSchema.safeParse({ + ...base, + lavasrEnhancerModelSrc: 's3:///lavasr/enhancer.gguf', + lavasrEnhancerCoremlModelPath: '/models/enhancer.mlmodelc' + }) + t.is(noUnits.success, false) + if (!noUnits.success) { + t.is(noUnits.error.issues[0]?.path.join('.'), 'lavasrEnhancerCoremlComputeUnits') + } + const noPath = ttsConfigSchema.safeParse({ + ...base, + lavasrDenoiserModelSrc: 's3:///lavasr/denoiser.gguf', + lavasrDenoiserCoremlComputeUnits: 'CPU_ONLY' + }) + t.is(noPath.success, false) + if (!noPath.success) { + t.is(noPath.error.issues[0]?.path.join('.'), 'lavasrDenoiserCoremlModelPath') + } + const noGguf = ttsConfigSchema.safeParse({ + ...base, + lavasrEnhancerCoremlModelPath: '/models/enhancer.mlmodelc', + lavasrEnhancerCoremlComputeUnits: 'ALL' + }) + t.is(noGguf.success, false) + if (!noGguf.success) { + t.is(noGguf.error.issues[0]?.path.join('.'), 'lavasrEnhancerModelSrc') + } + t.is( + ttsConfigSchema.safeParse({ + ...base, + lavasrEnhancerModelSrc: 's3:///lavasr/enhancer.gguf', + lavasrEnhancerCoremlModelPath: '/models/enhancer.mlmodelc', + lavasrEnhancerCoremlComputeUnits: 'cpu_only' + }).success, + false, + 'the LavaSR policy spelling is upper case' + ) +}) + +test('ttsResponseSchema / textToSpeechStreamResponseSchema: carry CosyVoice3 speechTokenIds', (t) => { + for (const [type, schema] of [ + ['textToSpeech', ttsResponseSchema], + ['textToSpeechStream', textToSpeechStreamResponseSchema] + ] as const) { + const withPcm = schema.safeParse({ + type, + buffer: [1, 2], + done: false, + speechTokenIds: [12, 480, 3] + }) + t.is(withPcm.success, true, `${type} with PCM`) + if (withPcm.success) t.alike(withPcm.data.speechTokenIds, [12, 480, 3]) + // Native chunk streaming hands the trajectory over on an empty final chunk. + const final = schema.safeParse({ + type, + buffer: [], + done: false, + isLast: true, + speechTokenIds: [12, 480, 3] + }) + t.is(final.success, true, `${type} final chunk`) + t.is( + schema.safeParse({ type, buffer: [], done: false, speechTokenIds: [1.5] }).success, + false, + `${type} token ids are integers` + ) + } +}) + +test('ttsStatsSchema: keeps the Parler Core ML sidecar flags', (t) => { + const stats = { decoderSidecarLoaded: 1, dacSidecarLoaded: 1, decoderOnCoreml: 0, dacOnCoreml: 1 } + for (const [type, schema] of [ + ['textToSpeech', ttsResponseSchema], + ['textToSpeechStream', textToSpeechStreamResponseSchema] + ] as const) { + const r = schema.safeParse({ type, buffer: [], done: true, stats }) + t.is(r.success, true, type) + if (r.success) t.alike(r.data.stats, stats) + } +}) diff --git a/packages/inference/test/tts-side-channel.test.ts b/packages/inference/test/tts-side-channel.test.ts index 7da2758d094..60834937741 100644 --- a/packages/inference/test/tts-side-channel.test.ts +++ b/packages/inference/test/tts-side-channel.test.ts @@ -60,6 +60,49 @@ test('settle() is idempotent and a late observe() cannot change a settled value' t.is(await side.stats, undefined) }) +test('speechTokenIds concatenates every array the frames carried, in order, at settle()', async (t) => { + // CosyVoice3 batch synthesis hands one array over with the PCM; the chunked + // paths run one native synthesis per sentence, so the run's trajectory is + // their concatenation. A frame without ids (the audio frames of a native + // stream) adds nothing. + const side = createTtsSideChannel() + side.observe(frame({ buffer: [1, 2] })) + side.observe(frame({ buffer: [3], speechTokenIds: [12, 480] })) + side.observe(frame({ buffer: [], isLast: true, speechTokenIds: [3] })) + t.is(await isPending(side.speechTokenIds), true, 'the trajectory waits for settle()') + side.settle() + t.alike(await side.speechTokenIds, [12, 480, 3]) + + const none = createTtsSideChannel() + none.observe(frame({ buffer: [1] })) + none.settle() + t.is(await none.speechTokenIds, undefined, 'engines that emit no tokens resolve undefined') +}) + +test('speechTokenIds appends into one owned array without touching the frames', async (t) => { + // Long sentence streams carry one array per sentence; the trajectory grows + // in place rather than being re-copied per frame, and the frames' own arrays + // stay as the addon sent them. + const first = [12, 480] + const second = [3] + const side = createTtsSideChannel() + side.observe(frame({ buffer: [3], speechTokenIds: first })) + side.observe(frame({ buffer: [], isLast: true, speechTokenIds: second })) + side.settle() + const result = await side.speechTokenIds + t.alike(result, [12, 480, 3]) + t.alike(first, [12, 480], 'the first frame keeps its own tokens') + t.alike(second, [3], 'the second frame keeps its own tokens') + t.not(result, first, 'the result is not an alias of a frame array') + + const single = createTtsSideChannel() + const only = [7, 8] + single.observe(frame({ buffer: [1], speechTokenIds: only })) + single.settle() + t.alike(await single.speechTokenIds, [7, 8]) + t.not(await single.speechTokenIds, only, 'a lone frame is copied too') +}) + test('MOSS-Speech reply text settles even when no PCM was emitted', async (t) => { const side = createTtsSideChannel() side.observe(frame({ text: 'Hello.' })) diff --git a/packages/inference/test/tts-speech-token-ids.test.ts b/packages/inference/test/tts-speech-token-ids.test.ts new file mode 100644 index 00000000000..a7af7a9bf6b --- /dev/null +++ b/packages/inference/test/tts-speech-token-ids.test.ts @@ -0,0 +1,193 @@ +import test from 'brittle' +import Buffer from 'bare-buffer' +import { registerModel, unregisterModel } from '@/runtime/model-registry' +import { textToSpeech } from '@/plugins/builtin/tts-ggml/ops/text-to-speech' +import { textToSpeechStream } from '@/plugins/builtin/tts-ggml/ops/text-to-speech-stream' +import type { TtsOpResult } from '@/plugins/builtin/tts-ggml/ops/text-to-speech' +import type { TtsOpYield } from '@/utils/tts-stats' +import { withDecodedTtsPcm } from '@/schemas/tts-pcm' + +// CosyVoice3 0.15 owns its speech-token trajectory and hands it back on the +// output: with the PCM in batch synthesis, and on a final empty `isLast` +// chunk under native chunk streaming. The ops used to drop the field, and the +// streaming paths skipped every empty chunk as a native end marker, so the +// trajectory never reached a caller who wanted to replay it. + +type Chunk = { + outputArray: Int16Array + sampleRate?: number + chunkIndex?: number + isLast?: boolean + speechTokenIds?: number[] +} + +function register(modelId: string, engine: string, chunks: Chunk[]) { + const response = { + stats: { totalSamples: 4 }, + async *iterate() { + yield* chunks + } + } + registerModel(modelId, { + model: { + getEngineType: () => engine, + async cancel() {}, + async run() { + return response + }, + async runStream() { + return response + }, + async runStreaming(source: AsyncIterable) { + for await (const _ of source) { + // drain + } + return response + } + } as never, + path: `/tmp/${engine}.gguf`, + config: { ttsEngine: engine }, + modelType: 'tts-ggml' + }) +} + +async function collect(op: AsyncGenerator) { + const frames: TtsOpYield[] = [] + let result = await op.next() + while (!result.done) { + frames.push(withDecodedTtsPcm(result.value)) + result = await op.next() + } + return { frames, result: result.value } +} + +const BATCH: Chunk[] = [ + { outputArray: new Int16Array([10, 20, 30, 40]), sampleRate: 24000, speechTokenIds: [12, 480, 3] } +] + +const NATIVE_STREAM: Chunk[] = [ + { outputArray: new Int16Array([10, 20]), sampleRate: 24000, chunkIndex: 0 }, + { outputArray: new Int16Array([30, 40]), sampleRate: 24000, chunkIndex: 1 }, + { outputArray: new Int16Array(0), isLast: true, speechTokenIds: [12, 480, 3] } +] + +test('batch synthesis forwards the speech tokens with the collected PCM', async (t) => { + const modelId = 'cosyvoice3-batch-tokens' + register(modelId, 'cosyvoice3', BATCH) + try { + const { frames, result } = await collect( + textToSpeech({ + type: 'textToSpeech', + modelId, + inputType: 'text', + text: 'Hello.', + stream: false, + sentenceStream: false + }) + ) + t.is(frames.length, 1) + t.alike(frames[0], { + buffer: [10, 20, 30, 40], + sampleRate: 24000, + speechTokenIds: [12, 480, 3] + }) + t.alike(result.stats, { totalSamples: 4 }) + } finally { + unregisterModel(modelId) + } +}) + +test('a batch collect over a native stream concatenates the chunks and keeps the tokens', async (t) => { + // `stream: false` on a model loaded with streamChunkTokens: the addon still + // emits chunks, and the tokens ride the final empty one. + const modelId = 'cosyvoice3-batch-over-stream' + register(modelId, 'cosyvoice3', NATIVE_STREAM) + try { + const { frames } = await collect( + textToSpeech({ + type: 'textToSpeech', + modelId, + inputType: 'text', + text: 'Hello.', + stream: false, + sentenceStream: false + }) + ) + t.is(frames.length, 1) + t.alike(frames[0], { + buffer: [10, 20, 30, 40], + sampleRate: 24000, + speechTokenIds: [12, 480, 3] + }) + } finally { + unregisterModel(modelId) + } +}) + +for (const mode of ['stream', 'sentence', 'duplex'] as const) { + test(`${mode}: the final empty chunk that carries the tokens reaches the caller`, async (t) => { + const modelId = `cosyvoice3-${mode}-tokens` + register(modelId, 'cosyvoice3', NATIVE_STREAM) + try { + const op = + mode === 'duplex' + ? textToSpeechStream( + { type: 'textToSpeechStream', modelId, inputType: 'text' }, + (async function* () { + yield Buffer.from('Hello.') + })() + ) + : textToSpeech({ + type: 'textToSpeech', + modelId, + inputType: 'text', + text: 'Hello.', + stream: true, + sentenceStream: mode === 'sentence' + }) + const { frames } = await collect(op) + t.is(frames.length, 3, 'two audio frames and the token frame') + t.alike(frames[0], { buffer: [10, 20], sampleRate: 24000, chunkIndex: 0 }) + t.alike(frames[1], { buffer: [30, 40], sampleRate: 24000, chunkIndex: 1 }) + t.alike(frames[2], { buffer: [], isLast: true, speechTokenIds: [12, 480, 3] }) + } finally { + unregisterModel(modelId) + } + }) + + test(`${mode}: an empty end marker without tokens is still skipped`, async (t) => { + // Audio8's native end markers are empty chunks carrying nothing; they + // must not surface as empty frames now that token frames pass. + const modelId = `audio8-${mode}-end-marker` + register(modelId, 'audio8', [ + { outputArray: new Int16Array([10, 20]), sampleRate: 44100, chunkIndex: 0 }, + { outputArray: new Int16Array(0), isLast: true } + ]) + try { + const op = + mode === 'duplex' + ? textToSpeechStream( + { type: 'textToSpeechStream', modelId, inputType: 'text' }, + (async function* () { + yield Buffer.from('Hello.') + })() + ) + : textToSpeech({ + type: 'textToSpeech', + modelId, + inputType: 'text', + text: 'Hello.', + stream: true, + sentenceStream: mode === 'sentence' + }) + const { frames } = await collect(op) + // The plain stream path never filtered empty chunks; the chunked paths do. + const expected = mode === 'stream' ? 2 : 1 + t.is(frames.length, expected) + t.alike(frames[0], { buffer: [10, 20], sampleRate: 44100, chunkIndex: 0 }) + for (const frame of frames) t.is('speechTokenIds' in frame, false) + } finally { + unregisterModel(modelId) + } + }) +} diff --git a/packages/inference/test/tts-stats.test.ts b/packages/inference/test/tts-stats.test.ts index 23e4109e596..36616a6bede 100644 --- a/packages/inference/test/tts-stats.test.ts +++ b/packages/inference/test/tts-stats.test.ts @@ -1,5 +1,5 @@ import test from 'brittle' -import { appendPcm, collectTtsStats, chunkMetadata } from '@/utils/tts-stats' +import { appendPcm, carriesSpeechTokenIds, collectTtsStats, chunkMetadata } from '@/utils/tts-stats' test('collectTtsStats: maps LavaSR enhancer and denoiser backend stats', (t) => { const stats = collectTtsStats({ @@ -173,3 +173,22 @@ test('collectTtsStats: forwards the engines’ stage timings and counters, zero test('collectTtsStats: forwards only the stage fields the addon reported', (t) => { t.alike(collectTtsStats({ stats: { t3Ms: 410 } }), { t3Ms: 410 }) }) + +test('collectTtsStats: forwards the Parler Core ML sidecar flags, zero included', (t) => { + // 0 is a real report: a retired sidecar, or a synthesis that ran on ggml. + t.alike( + collectTtsStats({ + stats: { decoderSidecarLoaded: 1, dacSidecarLoaded: 1, decoderOnCoreml: 0, dacOnCoreml: 1 } + }), + { decoderSidecarLoaded: 1, dacSidecarLoaded: 1, decoderOnCoreml: 0, dacOnCoreml: 1 } + ) +}) + +test('chunkMetadata: forwards CosyVoice3 speechTokenIds and nothing else on an end marker', (t) => { + t.alike( + chunkMetadata({ outputArray: new Int16Array(0), isLast: true, speechTokenIds: [12, 480, 3] }), + { isLast: true, speechTokenIds: [12, 480, 3] } + ) + t.is(carriesSpeechTokenIds({ outputArray: new Int16Array(0), speechTokenIds: [] }), true) + t.is(carriesSpeechTokenIds({ outputArray: new Int16Array(0), isLast: true }), false) +}) diff --git a/packages/sdk-kotlin/src/commonMain/kotlin/io/tether/qvac/sdk/generated/schema/SchemaTypes.kt b/packages/sdk-kotlin/src/commonMain/kotlin/io/tether/qvac/sdk/generated/schema/SchemaTypes.kt index a3acb497b68..27cb3d272b1 100644 --- a/packages/sdk-kotlin/src/commonMain/kotlin/io/tether/qvac/sdk/generated/schema/SchemaTypes.kt +++ b/packages/sdk-kotlin/src/commonMain/kotlin/io/tether/qvac/sdk/generated/schema/SchemaTypes.kt @@ -84,7 +84,7 @@ internal object AnyRequestSerializer : KSerializer { Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"name\",\"type\"],\"properties\":{\"name\":{\"type\":\"string\"},\"type\":{\"type\":\"string\",\"const\":\"getModelInfo\"}}}").jsonObject, Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\"],\"properties\":{\"sample\":{\"type\":\"boolean\"},\"type\":{\"type\":\"string\",\"const\":\"getSystemResources\"}}}").jsonObject, Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"heartbeat\"}}}").jsonObject, - Json.parseToJsonElement("{\"anyOf\":[{\"anyOf\":[{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"llamacpp-completion\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"backend\":{\"type\":\"array\",\"items\":{\"type\":\"string\",\"enum\":[\"cuda\",\"vulkan\",\"metal\",\"opencl\",\"auto\"]}},\"ctx_size\":{\"type\":\"number\"},\"temp\":{\"type\":\"number\"},\"top_p\":{\"type\":\"number\"},\"top_k\":{\"type\":\"integer\"},\"seed\":{\"type\":\"number\"},\"gpu_layers\":{\"type\":\"number\"},\"lora\":{\"type\":\"string\"},\"device\":{\"type\":\"string\"},\"predict\":{\"anyOf\":[{\"type\":\"number\",\"const\":-1},{\"type\":\"number\",\"const\":-2},{\"type\":\"integer\"}]},\"system_prompt\":{\"type\":\"string\"},\"load_mode\":{\"type\":\"string\",\"enum\":[\"none\",\"mmap\",\"mlock\",\"mmap+mlock\",\"dio\"]},\"verbosity\":{\"type\":\"number\",\"enum\":[0,1,2,3]},\"presence_penalty\":{\"type\":\"number\"},\"frequency_penalty\":{\"type\":\"number\"},\"repeat_penalty\":{\"type\":\"number\"},\"stop_sequences\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}},\"parallel\":{\"type\":\"integer\"},\"tools\":{\"type\":\"boolean\"},\"cache_checkpoints\":{\"type\":\"integer\"},\"cache_checkpoints_max_bytes\":{\"type\":\"integer\"},\"cache_checkpoint_storage\":{\"type\":\"string\",\"enum\":[\"memory\",\"disk\"]},\"cache_checkpoint_dir\":{\"type\":\"string\"},\"cache_share_across_keys\":{\"type\":\"boolean\"},\"cache_ram_mib\":{\"type\":\"integer\"},\"cache-type-k\":{\"type\":\"string\"},\"cache-type-v\":{\"type\":\"string\"},\"main-gpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"rpc-servers\":{\"type\":\"string\"},\"devices\":{\"type\":\"string\"},\"split-mode\":{\"type\":\"string\",\"enum\":[\"none\",\"layer\",\"tensor\"]},\"flash-attn\":{\"type\":\"string\",\"enum\":[\"on\",\"off\",\"auto\"]},\"tensor-split\":{\"type\":\"string\"},\"batch-size\":{\"type\":\"integer\"},\"ubatch-size\":{\"type\":\"integer\"},\"cpu-moe\":{\"type\":\"boolean\"},\"n-cpu-moe\":{\"type\":\"integer\"},\"kv-offload\":{\"type\":\"boolean\"},\"image-max-tokens\":{\"type\":\"integer\"},\"image-min-tokens\":{\"type\":\"integer\"},\"image-max-megapixels\":{\"type\":\"integer\"},\"poll\":{\"type\":\"integer\"},\"pollBatch\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"threads-batch\":{\"type\":\"integer\"},\"cpu-mask\":{\"type\":\"string\"},\"cpu-mask-batch\":{\"type\":\"string\"},\"override-tensor\":{\"type\":\"string\"},\"n-cpu-ffn\":{\"type\":\"integer\"},\"moe-cache-mib\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"const\":\"auto\"}]},\"prefetch-weights\":{\"anyOf\":[{\"type\":\"boolean\"},{\"type\":\"string\",\"const\":\"auto\"}]},\"tensor-read-lazy\":{\"type\":\"string\",\"enum\":[\"on\",\"auto\",\"off\"]},\"fit\":{\"type\":\"boolean\"},\"fit-target\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\"}]},\"fit-ctx\":{\"type\":\"integer\"},\"openclCacheDir\":{\"type\":\"string\"},\"reasoning_budget\":{\"type\":\"integer\"},\"spec_type\":{\"type\":\"string\",\"enum\":[\"none\",\"draft-mtp\",\"draft-dflash\"]},\"spec_draft_n_max\":{\"type\":\"integer\"},\"spec_draft_n_min\":{\"type\":\"integer\"},\"spec_draft_p_min\":{\"type\":\"number\"},\"projectionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"draftModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"image_tile_mode\":{\"type\":\"string\",\"enum\":[\"disabled\",\"batched\",\"sequential\"]},\"image_no_upscale\":{\"type\":\"string\",\"enum\":[\"on\",\"off\"]},\"mmproj-use-gpu\":{\"type\":\"boolean\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"whispercpp-transcription\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"strategy\":{\"type\":\"string\",\"enum\":[\"greedy\",\"beam_search\"]},\"n_threads\":{\"type\":\"integer\"},\"n_max_text_ctx\":{\"type\":\"integer\"},\"offset_ms\":{\"type\":\"integer\"},\"duration_ms\":{\"type\":\"integer\"},\"audio_ctx\":{\"type\":\"integer\"},\"translate\":{\"type\":\"boolean\"},\"no_context\":{\"type\":\"boolean\"},\"no_timestamps\":{\"type\":\"boolean\"},\"single_segment\":{\"type\":\"boolean\"},\"print_special\":{\"type\":\"boolean\"},\"print_progress\":{\"type\":\"boolean\"},\"print_realtime\":{\"type\":\"boolean\"},\"print_timestamps\":{\"type\":\"boolean\"},\"token_timestamps\":{\"type\":\"boolean\"},\"thold_pt\":{\"type\":\"number\"},\"thold_ptsum\":{\"type\":\"number\"},\"max_len\":{\"type\":\"integer\"},\"split_on_word\":{\"type\":\"boolean\"},\"max_tokens\":{\"type\":\"integer\"},\"debug_mode\":{\"type\":\"boolean\"},\"tdrz_enable\":{\"type\":\"boolean\"},\"suppress_regex\":{\"type\":\"string\"},\"initial_prompt\":{\"type\":\"string\"},\"carry_initial_prompt\":{\"type\":\"boolean\"},\"language\":{\"type\":\"string\"},\"detect_language\":{\"type\":\"boolean\"},\"suppress_blank\":{\"type\":\"boolean\"},\"suppress_nst\":{\"type\":\"boolean\"},\"temperature\":{\"type\":\"number\"},\"length_penalty\":{\"type\":\"number\"},\"temperature_inc\":{\"type\":\"number\"},\"entropy_thold\":{\"type\":\"number\"},\"logprob_thold\":{\"type\":\"number\"},\"greedy_best_of\":{\"type\":\"integer\"},\"beam_search_beam_size\":{\"type\":\"integer\"},\"max_initial_ts\":{\"type\":\"number\"},\"no_speech_thold\":{\"type\":\"number\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"vad_params\":{\"type\":\"object\"},\"audio_format\":{\"type\":\"string\",\"enum\":[\"f32le\",\"s16le\"]},\"contextParams\":{\"type\":\"object\"},\"miscConfig\":{\"type\":\"object\"},\"vadModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"bci-whispercpp-transcription\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"whisperConfig\":{\"type\":\"object\"},\"bciConfig\":{\"type\":\"object\"},\"contextParams\":{\"type\":\"object\"},\"miscConfig\":{\"type\":\"object\"},\"backendsDir\":{\"type\":\"string\"},\"embedderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"parakeet-transcription\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"maxThreads\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"backend\":{\"type\":\"string\"},\"speakerAttribution\":{\"type\":\"object\",\"required\":[\"modelSrc\"]},\"sampleRate\":{\"type\":\"integer\"},\"channels\":{\"type\":\"integer\"},\"captionEnabled\":{\"type\":\"boolean\"},\"timestampsEnabled\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"integer\"},\"streaming\":{\"type\":\"boolean\"},\"streamingChunkMs\":{\"type\":\"integer\"},\"streamingHistoryMs\":{\"type\":\"integer\"},\"streamingEmitPartials\":{\"type\":\"boolean\"},\"streamingEnergyVad\":{\"type\":\"boolean\"},\"streamingEnergyVadThresholdDb\":{\"type\":\"number\"},\"streamingEnergyVadWindowMs\":{\"type\":\"integer\"},\"streamingEnergyVadHangoverMs\":{\"type\":\"integer\"},\"streamingLeftContextMs\":{\"type\":\"integer\"},\"streamingRightLookaheadMs\":{\"type\":\"integer\"},\"language\":{\"type\":\"string\"},\"streamingSpeakerVad\":{\"type\":\"boolean\"},\"diarizationThreshold\":{\"type\":\"number\"},\"diarizationMinSegmentMs\":{\"type\":\"integer\"},\"streamingSpkCacheEnable\":{\"type\":\"boolean\"},\"streamingSpkCacheLen\":{\"type\":\"integer\"},\"streamingFifoLen\":{\"type\":\"integer\"},\"streamingChunkLeftContextMs\":{\"type\":\"integer\"},\"streamingChunkRightContextMs\":{\"type\":\"integer\"},\"streamingSpkCacheUpdatePeriod\":{\"type\":\"integer\"},\"prewarm\":{\"type\":\"boolean\"},\"prewarmAudioSeconds\":{\"type\":\"number\"},\"longFormWindowFrames\":{\"type\":\"integer\"},\"longFormContextFrames\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"parakeetEncoderSrc\":{},\"parakeetDecoderSrc\":{},\"parakeetVocabSrc\":{},\"parakeetPreprocessorSrc\":{},\"parakeetCtcModelSrc\":{},\"parakeetTokenizerSrc\":{},\"parakeetSortformerSrc\":{},\"parakeetModelSrc\":{},\"modelType\":{}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"moss-transcribe\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"maxThreads\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"llamacpp-embedding\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"ctxSize\":{\"type\":\"integer\"},\"projectionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"imageMaxTokens\":{\"type\":\"integer\"},\"imageMinTokens\":{\"type\":\"integer\"},\"mmprojNoAudio\":{\"type\":\"boolean\"},\"mmprojUseGpu\":{\"type\":\"boolean\"},\"imageMaxMegapixels\":{\"type\":\"integer\"},\"audioMaxSeconds\":{\"type\":\"integer\"},\"poll\":{\"type\":\"integer\"},\"pollBatch\":{\"type\":\"integer\"},\"backend\":{\"type\":\"array\",\"items\":{\"type\":\"string\",\"enum\":[\"cuda\",\"vulkan\",\"metal\",\"opencl\",\"auto\"]}},\"gpuLayers\":{\"type\":\"integer\"},\"device\":{\"type\":\"string\",\"enum\":[\"gpu\",\"cpu\"]},\"batchSize\":{\"type\":\"integer\"},\"pooling\":{\"type\":\"string\",\"enum\":[\"none\",\"mean\",\"cls\",\"last\"]},\"attention\":{\"type\":\"string\",\"enum\":[\"causal\",\"non-causal\"]},\"embdNormalize\":{\"type\":\"integer\"},\"flashAttention\":{\"type\":\"string\",\"enum\":[\"on\",\"off\",\"auto\"]},\"mainGpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"splitMode\":{\"type\":\"string\",\"enum\":[\"none\",\"layer\"]},\"tensorSplit\":{\"type\":\"string\"},\"verbosity\":{\"type\":\"number\",\"enum\":[0,1,2,3]},\"openclCacheDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"llamacpp-decision\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"device\":{\"type\":\"string\",\"enum\":[\"cpu\",\"gpu\"]},\"gpu_layers\":{\"type\":\"integer\"},\"batch_size\":{\"type\":\"integer\"},\"poll\":{\"type\":\"integer\"},\"pollBatch\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"threads-batch\":{\"type\":\"integer\"},\"flash_attn\":{\"type\":\"string\",\"enum\":[\"on\",\"off\",\"auto\"]},\"verbosity\":{\"type\":\"integer\"},\"main-gpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"split-mode\":{\"type\":\"string\",\"enum\":[\"none\",\"layer\"]},\"tensor-split\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"llamacpp-rerank\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"device\":{\"type\":\"string\",\"enum\":[\"cpu\",\"gpu\"]},\"gpu_layers\":{\"type\":\"integer\"},\"batch_size\":{\"type\":\"integer\"},\"backend\":{\"type\":\"array\",\"items\":{\"type\":\"string\",\"enum\":[\"cuda\",\"vulkan\",\"metal\",\"opencl\",\"auto\"]}},\"poll\":{\"type\":\"integer\"},\"pollBatch\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"threads-batch\":{\"type\":\"integer\"},\"flash_attn\":{\"type\":\"string\",\"enum\":[\"on\",\"off\",\"auto\"]},\"verbosity\":{\"type\":\"integer\"},\"main-gpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"split-mode\":{\"type\":\"string\",\"enum\":[\"none\",\"layer\"]},\"tensor-split\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"ggml-image-embedding\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"backend\":{\"type\":\"string\"},\"threads\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"nmtcpp-translation\"},\"modelConfig\":{\"oneOf\":[{\"type\":\"object\",\"required\":[\"engine\",\"from\",\"to\"],\"properties\":{\"mode\":{\"type\":\"string\",\"enum\":[\"full\"]},\"beamsize\":{\"type\":\"number\"},\"lengthpenalty\":{\"type\":\"number\"},\"maxlength\":{\"type\":\"number\"},\"repetitionpenalty\":{\"type\":\"number\"},\"norepeatngramsize\":{\"type\":\"number\"},\"temperature\":{\"type\":\"number\"},\"topk\":{\"type\":\"number\"},\"topp\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\",\"const\":\"Bergamot\"},\"from\":{\"type\":\"string\",\"enum\":[\"en\",\"ar\",\"bg\",\"ca\",\"cs\",\"de\",\"es\",\"et\",\"fi\",\"fr\",\"hu\",\"is\",\"it\",\"ja\",\"ko\",\"lt\",\"lv\",\"nl\",\"pl\",\"pt\",\"ru\",\"sk\",\"sl\",\"uk\",\"zh\",\"az\",\"be\",\"bn\",\"bs\",\"da\",\"el\",\"fa\",\"gu\",\"he\",\"hi\",\"hr\",\"id\",\"kn\",\"ml\",\"ms\",\"mt\",\"nb\",\"nn\",\"no\",\"re\",\"ro\",\"sq\",\"sr\",\"sv\",\"ta\",\"te\",\"th\",\"tr\",\"vi\"]},\"to\":{\"type\":\"string\",\"enum\":[\"en\",\"ar\",\"bg\",\"ca\",\"cs\",\"de\",\"es\",\"et\",\"fi\",\"fr\",\"hu\",\"is\",\"it\",\"ja\",\"ko\",\"lt\",\"lv\",\"nl\",\"pl\",\"pt\",\"ru\",\"sk\",\"sl\",\"uk\",\"zh\",\"az\",\"be\",\"bn\",\"bs\",\"da\",\"el\",\"fa\",\"gu\",\"he\",\"hi\",\"hr\",\"id\",\"kn\",\"ml\",\"ms\",\"mt\",\"nb\",\"nn\",\"no\",\"re\",\"ro\",\"sq\",\"sr\",\"sv\",\"ta\",\"te\",\"th\",\"tr\",\"vi\"]},\"srcVocabSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"dstVocabSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"normalize\":{\"type\":\"number\"},\"pivotModel\":{\"type\":\"object\",\"required\":[\"modelSrc\"]}}},{\"type\":\"object\",\"required\":[\"engine\",\"from\",\"to\"],\"properties\":{\"mode\":{\"type\":\"string\",\"enum\":[\"full\"]},\"beamsize\":{\"type\":\"number\"},\"lengthpenalty\":{\"type\":\"number\"},\"maxlength\":{\"type\":\"number\"},\"repetitionpenalty\":{\"type\":\"number\"},\"norepeatngramsize\":{\"type\":\"number\"},\"temperature\":{\"type\":\"number\"},\"topk\":{\"type\":\"number\"},\"topp\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\",\"const\":\"IndicTrans\"},\"from\":{\"type\":\"string\",\"enum\":[\"asm_Beng\",\"ben_Beng\",\"brx_Deva\",\"doi_Deva\",\"eng_Latn\",\"gom_Deva\",\"guj_Gujr\",\"hin_Deva\",\"kan_Knda\",\"kas_Arab\",\"kas_Deva\",\"mai_Deva\",\"mal_Mlym\",\"mar_Deva\",\"mni_Beng\",\"mni_Mtei\",\"npi_Deva\",\"ory_Orya\",\"pan_Guru\",\"san_Deva\",\"sat_Olck\",\"snd_Arab\",\"snd_Deva\",\"tam_Taml\",\"tel_Telu\",\"urd_Arab\"]},\"to\":{\"type\":\"string\",\"enum\":[\"asm_Beng\",\"ben_Beng\",\"brx_Deva\",\"doi_Deva\",\"eng_Latn\",\"gom_Deva\",\"guj_Gujr\",\"hin_Deva\",\"kan_Knda\",\"kas_Arab\",\"kas_Deva\",\"mai_Deva\",\"mal_Mlym\",\"mar_Deva\",\"mni_Beng\",\"mni_Mtei\",\"npi_Deva\",\"ory_Orya\",\"pan_Guru\",\"san_Deva\",\"sat_Olck\",\"snd_Arab\",\"snd_Deva\",\"tam_Taml\",\"tel_Telu\",\"urd_Arab\"]}}}]}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"tts-ggml\"},\"modelConfig\":{\"oneOf\":[{\"type\":\"object\",\"required\":[\"ttsEngine\",\"language\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"chatterbox\"},\"language\":{\"type\":\"string\",\"enum\":[\"en\",\"es\",\"fr\",\"de\",\"it\",\"ja\",\"pt\",\"nl\",\"pl\",\"tr\",\"sv\",\"da\",\"fi\",\"no\",\"el\",\"ms\",\"sw\",\"ar\",\"ko\",\"he\",\"ru\",\"zh\",\"hi\"]},\"voice\":{\"type\":\"string\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"ttsSpeed\":{\"type\":\"number\"},\"nCtx\":{\"type\":\"integer\"},\"kvCacheType\":{\"type\":\"string\",\"enum\":[\"f32\",\"f16\",\"q8_0\"]},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"cfmSteps\":{\"type\":\"integer\"},\"cfgRate\":{\"type\":\"number\"},\"streamLeftContextTokens\":{\"type\":\"integer\"},\"batchCfmSteps\":{\"type\":\"integer\"},\"nPredict\":{\"type\":\"integer\"},\"maxSentenceChars\":{\"type\":\"integer\"},\"crossfadeMs\":{\"type\":\"integer\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"repeatPenalty\":{\"type\":\"number\"},\"exaggeration\":{\"type\":\"number\"},\"cfgWeight\":{\"type\":\"number\"},\"minP\":{\"type\":\"number\"},\"voiceDir\":{\"type\":\"string\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"s3genModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"mecabDictSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"cangjieTsvSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"ttsSupertonicMultilingual\":{},\"ttsTokenizerSrc\":{},\"ttsSpeechEncoderSrc\":{},\"ttsEmbedTokensSrc\":{},\"ttsConditionalDecoderSrc\":{},\"ttsLanguageModelSrc\":{},\"ttsTextEncoderSrc\":{},\"ttsDurationPredictorSrc\":{},\"ttsVectorEstimatorSrc\":{},\"ttsVocoderSrc\":{},\"ttsUnicodeIndexerSrc\":{},\"ttsTtsConfigSrc\":{},\"ttsVoiceStyleSrc\":{}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"language\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"supertonic\"},\"language\":{\"type\":\"string\",\"enum\":[\"en\",\"ko\",\"ja\",\"ar\",\"bg\",\"cs\",\"da\",\"de\",\"el\",\"es\",\"et\",\"fi\",\"fr\",\"hi\",\"hr\",\"hu\",\"id\",\"it\",\"lt\",\"lv\",\"nl\",\"pl\",\"pt\",\"ro\",\"ru\",\"sk\",\"sl\",\"sv\",\"tr\",\"uk\",\"vi\"]},\"voice\":{\"type\":\"string\"},\"ttsSpeed\":{\"type\":\"number\"},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"ttsNumInferenceSteps\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"vulkanCacheDir\":{\"type\":\"string\"},\"vulkanDevice\":{\"type\":\"integer\"},\"prewarmText\":{\"type\":\"string\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"streamChunkTolerancePct\":{\"type\":\"integer\"},\"streamMinChunkTokens\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"voiceJsonSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"noiseNpySrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"ttsSupertonicMultilingual\":{},\"ttsTokenizerSrc\":{},\"ttsSpeechEncoderSrc\":{},\"ttsEmbedTokensSrc\":{},\"ttsConditionalDecoderSrc\":{},\"ttsLanguageModelSrc\":{},\"ttsTextEncoderSrc\":{},\"ttsDurationPredictorSrc\":{},\"ttsVectorEstimatorSrc\":{},\"ttsVocoderSrc\":{},\"ttsUnicodeIndexerSrc\":{},\"ttsTtsConfigSrc\":{},\"ttsVoiceStyleSrc\":{}}},{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"parler\"},\"description\":{\"type\":\"string\"},\"voiceDescription\":{\"type\":\"string\"},\"voice\":{\"type\":\"string\"},\"emotion\":{\"type\":\"string\",\"enum\":[\"command\",\"anger\",\"narration\",\"conversation\",\"disgust\",\"fear\",\"happy\",\"neutral\",\"proper noun\",\"news\",\"sad\",\"surprise\"]},\"pitch\":{\"type\":\"string\"},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"expressivity\":{\"type\":\"string\"},\"noise\":{\"type\":\"string\"},\"reverb\":{\"type\":\"string\"},\"quality\":{\"type\":\"string\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"maxFrames\":{\"anyOf\":[{\"type\":\"number\",\"const\":0},{\"type\":\"integer\"}]},\"minNewTokens\":{\"type\":\"integer\"},\"normalizeNumbers\":{\"type\":\"boolean\"},\"backendsDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"cosyvoice3\"},\"emotion\":{\"type\":\"string\",\"enum\":[\"anger\",\"happy\",\"neutral\",\"sad\"]},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"instruct\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\"}]},\"promptText\":{\"type\":\"string\"},\"flowCutPrompt\":{\"type\":\"boolean\"},\"vulkanDevice\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"cosyvoice3S3tokModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"cosyvoice3CampplusModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"audio8CodecDecoderModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"audio8\"},\"referenceText\":{\"type\":\"string\"},\"greedy\":{\"type\":\"boolean\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"maxFrames\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"audio8CodecDecoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"audio8CodecEncoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"mossCodecDecoderModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss\"},\"language\":{\"type\":\"string\"},\"durationTokens\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"mossCodecDecoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"mossCodecEncoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"dialogueReferenceSrcs\":{\"type\":\"array\",\"items\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}},{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss-sfx\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"seconds\":{\"type\":\"number\"},\"negativePrompt\":{\"type\":\"string\"},\"steps\":{\"type\":\"integer\"},\"guidance\":{\"type\":\"number\"},\"shift\":{\"type\":\"number\"}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"mossSpeechCodecModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss-speech\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"mossSpeechCodecModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}]}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"ggml-ocr\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"langList\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}},\"pipelineType\":{\"type\":\"string\",\"enum\":[\"easyocr\",\"doctr\"]},\"magRatio\":{\"type\":\"number\"},\"canvasSize\":{\"type\":\"number\"},\"defaultRotationAngles\":{\"type\":\"array\",\"items\":{\"type\":\"number\"}},\"contrastRetry\":{\"type\":\"boolean\"},\"lowConfidenceThreshold\":{\"type\":\"number\"},\"recognizerBatchSize\":{\"type\":\"number\"},\"nThreads\":{\"type\":\"number\"},\"backendDevice\":{\"type\":\"string\",\"enum\":[\"cpu\",\"vulkan\",\"metal\",\"opencl\"]},\"gpuDevice\":{\"type\":\"number\"},\"mainGpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"detectorModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"sdcpp-generation\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"mode\":{\"type\":\"string\",\"enum\":[\"diffusion\",\"upscale\",\"video\",\"world\"]},\"threads\":{\"type\":\"number\"},\"device\":{\"type\":\"string\",\"enum\":[\"gpu\",\"cpu\"]},\"main-gpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"prediction\":{\"type\":\"string\",\"enum\":[\"auto\",\"eps\",\"v\",\"edm_v\",\"flow\",\"flux2_flow\"]},\"type\":{\"type\":\"string\",\"enum\":[\"auto\",\"f32\",\"f16\",\"bf16\",\"q2_k\",\"q3_k\",\"q4_0\",\"q4_1\",\"q4_k\",\"q5_0\",\"q5_1\",\"q5_k\",\"q6_k\",\"q8_0\"]},\"rng\":{\"type\":\"string\",\"enum\":[\"cpu\",\"cuda\",\"std_default\"]},\"sampler_rng\":{\"type\":\"string\",\"enum\":[\"cpu\",\"cuda\",\"std_default\"]},\"vae_auto_cpu_fallback\":{\"type\":\"boolean\"},\"vae_auto_cpu_fallback_memory_ratio\":{\"type\":\"number\"},\"vae_tiling\":{\"type\":\"boolean\"},\"offload_to_cpu\":{\"type\":\"boolean\"},\"control_net_cpu\":{\"not\":{}},\"clip_on_cpu\":{\"not\":{}},\"vae_on_cpu\":{\"not\":{}},\"backend\":{\"type\":\"string\"},\"params_backend\":{\"type\":\"string\"},\"max_vram\":{\"anyOf\":[{\"type\":\"number\"},{\"type\":\"string\"}]},\"stream_layers\":{\"type\":\"boolean\"},\"flash_attn\":{\"type\":\"boolean\"},\"diffusion_fa\":{\"type\":\"boolean\"},\"lora_apply_mode\":{\"type\":\"string\",\"enum\":[\"auto\",\"immediately\",\"at_runtime\"]},\"verbosity\":{\"type\":\"number\"},\"clipLModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"clipGModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"t5XxlModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"llmModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"projectionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"vaeModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"highNoiseDiffusionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"uncondModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"clipVisionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"audioVaeModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"embeddingsConnectorsModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"taehvModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"sceneSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"world\":{\"type\":\"object\"},\"upscaler\":{\"type\":\"object\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"audiogen-ggml\"},\"modelConfig\":{\"oneOf\":[{\"type\":\"object\",\"required\":[\"textEncModelSrc\",\"lmModelSrc\",\"ditModelSrc\",\"vaeModelSrc\"],\"properties\":{\"engine\":{\"type\":\"string\",\"const\":\"acestep\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"inferenceSteps\":{\"type\":\"integer\"},\"shift\":{\"type\":\"number\"},\"nGpuLayers\":{\"type\":\"integer\"},\"textEncModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lmModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"ditModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"vaeModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"engine\",\"lmModelSrc\",\"synthModelSrc\"],\"properties\":{\"engine\":{\"type\":\"string\",\"const\":\"minimax\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"device\":{\"type\":\"string\",\"enum\":[\"cpu\",\"gpu\",\"auto\"]},\"coremlProfileDir\":{\"type\":\"string\"},\"coremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"inferenceSteps\":{\"type\":\"integer\"},\"cfgScale\":{\"type\":\"number\"},\"lmModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"synthModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}]}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"ggml-vla\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"backend\":{\"type\":\"array\",\"items\":{\"type\":\"string\",\"enum\":[\"auto\",\"cpu\",\"cuda\",\"vulkan\",\"metal\",\"opencl\",\"hip\",\"rocm\",\"sycl\"]}},\"verbosity\":{\"type\":\"integer\"},\"embodiment\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"integer\"},{\"type\":\"object\"}]}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"ggml-classification\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"modelPath\":{\"type\":\"string\"},\"topK\":{\"type\":\"integer\"},\"nativeLogger\":{\"type\":\"boolean\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\"},\"modelConfig\":{\"type\":\"object\"}}}]},{\"type\":\"object\",\"required\":[\"type\",\"modelId\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelId\":{\"type\":\"string\"},\"modelSrc\":{\"not\":{}},\"withProgress\":{\"not\":{}},\"seed\":{\"not\":{}},\"modelType\":{\"type\":\"string\",\"enum\":[\"whisper\",\"whispercpp-transcription\"]},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"strategy\":{\"type\":\"string\",\"enum\":[\"greedy\",\"beam_search\"]},\"n_threads\":{\"type\":\"integer\"},\"n_max_text_ctx\":{\"type\":\"integer\"},\"offset_ms\":{\"type\":\"integer\"},\"duration_ms\":{\"type\":\"integer\"},\"audio_ctx\":{\"type\":\"integer\"},\"translate\":{\"type\":\"boolean\"},\"no_context\":{\"type\":\"boolean\"},\"no_timestamps\":{\"type\":\"boolean\"},\"single_segment\":{\"type\":\"boolean\"},\"print_special\":{\"type\":\"boolean\"},\"print_progress\":{\"type\":\"boolean\"},\"print_realtime\":{\"type\":\"boolean\"},\"print_timestamps\":{\"type\":\"boolean\"},\"token_timestamps\":{\"type\":\"boolean\"},\"thold_pt\":{\"type\":\"number\"},\"thold_ptsum\":{\"type\":\"number\"},\"max_len\":{\"type\":\"integer\"},\"split_on_word\":{\"type\":\"boolean\"},\"max_tokens\":{\"type\":\"integer\"},\"debug_mode\":{\"type\":\"boolean\"},\"tdrz_enable\":{\"type\":\"boolean\"},\"suppress_regex\":{\"type\":\"string\"},\"initial_prompt\":{\"type\":\"string\"},\"carry_initial_prompt\":{\"type\":\"boolean\"},\"language\":{\"type\":\"string\"},\"detect_language\":{\"type\":\"boolean\"},\"suppress_blank\":{\"type\":\"boolean\"},\"suppress_nst\":{\"type\":\"boolean\"},\"temperature\":{\"type\":\"number\"},\"length_penalty\":{\"type\":\"number\"},\"temperature_inc\":{\"type\":\"number\"},\"entropy_thold\":{\"type\":\"number\"},\"logprob_thold\":{\"type\":\"number\"},\"greedy_best_of\":{\"type\":\"integer\"},\"beam_search_beam_size\":{\"type\":\"integer\"},\"max_initial_ts\":{\"type\":\"number\"},\"no_speech_thold\":{\"type\":\"number\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"vad_params\":{\"type\":\"object\"},\"audio_format\":{\"type\":\"string\",\"enum\":[\"f32le\",\"s16le\"]},\"contextParams\":{\"type\":\"object\"},\"miscConfig\":{\"type\":\"object\"},\"vadModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}}}]}").jsonObject, + Json.parseToJsonElement("{\"anyOf\":[{\"anyOf\":[{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"llamacpp-completion\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"backend\":{\"type\":\"array\",\"items\":{\"type\":\"string\",\"enum\":[\"cuda\",\"vulkan\",\"metal\",\"opencl\",\"auto\"]}},\"ctx_size\":{\"type\":\"number\"},\"temp\":{\"type\":\"number\"},\"top_p\":{\"type\":\"number\"},\"top_k\":{\"type\":\"integer\"},\"seed\":{\"type\":\"number\"},\"gpu_layers\":{\"type\":\"number\"},\"lora\":{\"type\":\"string\"},\"device\":{\"type\":\"string\"},\"predict\":{\"anyOf\":[{\"type\":\"number\",\"const\":-1},{\"type\":\"number\",\"const\":-2},{\"type\":\"integer\"}]},\"system_prompt\":{\"type\":\"string\"},\"load_mode\":{\"type\":\"string\",\"enum\":[\"none\",\"mmap\",\"mlock\",\"mmap+mlock\",\"dio\"]},\"verbosity\":{\"type\":\"number\",\"enum\":[0,1,2,3]},\"presence_penalty\":{\"type\":\"number\"},\"frequency_penalty\":{\"type\":\"number\"},\"repeat_penalty\":{\"type\":\"number\"},\"stop_sequences\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}},\"parallel\":{\"type\":\"integer\"},\"tools\":{\"type\":\"boolean\"},\"cache_checkpoints\":{\"type\":\"integer\"},\"cache_checkpoints_max_bytes\":{\"type\":\"integer\"},\"cache_checkpoint_storage\":{\"type\":\"string\",\"enum\":[\"memory\",\"disk\"]},\"cache_checkpoint_dir\":{\"type\":\"string\"},\"cache_share_across_keys\":{\"type\":\"boolean\"},\"cache_ram_mib\":{\"type\":\"integer\"},\"cache-type-k\":{\"type\":\"string\"},\"cache-type-v\":{\"type\":\"string\"},\"main-gpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"rpc-servers\":{\"type\":\"string\"},\"devices\":{\"type\":\"string\"},\"split-mode\":{\"type\":\"string\",\"enum\":[\"none\",\"layer\",\"tensor\"]},\"flash-attn\":{\"type\":\"string\",\"enum\":[\"on\",\"off\",\"auto\"]},\"tensor-split\":{\"type\":\"string\"},\"batch-size\":{\"type\":\"integer\"},\"ubatch-size\":{\"type\":\"integer\"},\"cpu-moe\":{\"type\":\"boolean\"},\"n-cpu-moe\":{\"type\":\"integer\"},\"kv-offload\":{\"type\":\"boolean\"},\"image-max-tokens\":{\"type\":\"integer\"},\"image-min-tokens\":{\"type\":\"integer\"},\"image-max-megapixels\":{\"type\":\"integer\"},\"poll\":{\"type\":\"integer\"},\"pollBatch\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"threads-batch\":{\"type\":\"integer\"},\"cpu-mask\":{\"type\":\"string\"},\"cpu-mask-batch\":{\"type\":\"string\"},\"override-tensor\":{\"type\":\"string\"},\"n-cpu-ffn\":{\"type\":\"integer\"},\"moe-cache-mib\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"const\":\"auto\"}]},\"prefetch-weights\":{\"anyOf\":[{\"type\":\"boolean\"},{\"type\":\"string\",\"const\":\"auto\"}]},\"tensor-read-lazy\":{\"type\":\"string\",\"enum\":[\"on\",\"auto\",\"off\"]},\"fit\":{\"type\":\"boolean\"},\"fit-target\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\"}]},\"fit-ctx\":{\"type\":\"integer\"},\"openclCacheDir\":{\"type\":\"string\"},\"reasoning_budget\":{\"type\":\"integer\"},\"spec_type\":{\"type\":\"string\",\"enum\":[\"none\",\"draft-mtp\",\"draft-dflash\"]},\"spec_draft_n_max\":{\"type\":\"integer\"},\"spec_draft_n_min\":{\"type\":\"integer\"},\"spec_draft_p_min\":{\"type\":\"number\"},\"projectionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"draftModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"image_tile_mode\":{\"type\":\"string\",\"enum\":[\"disabled\",\"batched\",\"sequential\"]},\"image_no_upscale\":{\"type\":\"string\",\"enum\":[\"on\",\"off\"]},\"mmproj-use-gpu\":{\"type\":\"boolean\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"whispercpp-transcription\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"strategy\":{\"type\":\"string\",\"enum\":[\"greedy\",\"beam_search\"]},\"n_threads\":{\"type\":\"integer\"},\"n_max_text_ctx\":{\"type\":\"integer\"},\"offset_ms\":{\"type\":\"integer\"},\"duration_ms\":{\"type\":\"integer\"},\"audio_ctx\":{\"type\":\"integer\"},\"translate\":{\"type\":\"boolean\"},\"no_context\":{\"type\":\"boolean\"},\"no_timestamps\":{\"type\":\"boolean\"},\"single_segment\":{\"type\":\"boolean\"},\"print_special\":{\"type\":\"boolean\"},\"print_progress\":{\"type\":\"boolean\"},\"print_realtime\":{\"type\":\"boolean\"},\"print_timestamps\":{\"type\":\"boolean\"},\"token_timestamps\":{\"type\":\"boolean\"},\"thold_pt\":{\"type\":\"number\"},\"thold_ptsum\":{\"type\":\"number\"},\"max_len\":{\"type\":\"integer\"},\"split_on_word\":{\"type\":\"boolean\"},\"max_tokens\":{\"type\":\"integer\"},\"debug_mode\":{\"type\":\"boolean\"},\"tdrz_enable\":{\"type\":\"boolean\"},\"suppress_regex\":{\"type\":\"string\"},\"initial_prompt\":{\"type\":\"string\"},\"carry_initial_prompt\":{\"type\":\"boolean\"},\"language\":{\"type\":\"string\"},\"detect_language\":{\"type\":\"boolean\"},\"suppress_blank\":{\"type\":\"boolean\"},\"suppress_nst\":{\"type\":\"boolean\"},\"temperature\":{\"type\":\"number\"},\"length_penalty\":{\"type\":\"number\"},\"temperature_inc\":{\"type\":\"number\"},\"entropy_thold\":{\"type\":\"number\"},\"logprob_thold\":{\"type\":\"number\"},\"greedy_best_of\":{\"type\":\"integer\"},\"beam_search_beam_size\":{\"type\":\"integer\"},\"max_initial_ts\":{\"type\":\"number\"},\"no_speech_thold\":{\"type\":\"number\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"vad_params\":{\"type\":\"object\"},\"audio_format\":{\"type\":\"string\",\"enum\":[\"f32le\",\"s16le\"]},\"contextParams\":{\"type\":\"object\"},\"miscConfig\":{\"type\":\"object\"},\"vadModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"bci-whispercpp-transcription\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"whisperConfig\":{\"type\":\"object\"},\"bciConfig\":{\"type\":\"object\"},\"contextParams\":{\"type\":\"object\"},\"miscConfig\":{\"type\":\"object\"},\"backendsDir\":{\"type\":\"string\"},\"embedderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"parakeet-transcription\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"maxThreads\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"backend\":{\"type\":\"string\"},\"speakerAttribution\":{\"type\":\"object\",\"required\":[\"modelSrc\"]},\"sampleRate\":{\"type\":\"integer\"},\"channels\":{\"type\":\"integer\"},\"captionEnabled\":{\"type\":\"boolean\"},\"timestampsEnabled\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"integer\"},\"streaming\":{\"type\":\"boolean\"},\"streamingChunkMs\":{\"type\":\"integer\"},\"streamingHistoryMs\":{\"type\":\"integer\"},\"streamingEmitPartials\":{\"type\":\"boolean\"},\"streamingEnergyVad\":{\"type\":\"boolean\"},\"streamingEnergyVadThresholdDb\":{\"type\":\"number\"},\"streamingEnergyVadWindowMs\":{\"type\":\"integer\"},\"streamingEnergyVadHangoverMs\":{\"type\":\"integer\"},\"streamingLeftContextMs\":{\"type\":\"integer\"},\"streamingRightLookaheadMs\":{\"type\":\"integer\"},\"language\":{\"type\":\"string\"},\"streamingSpeakerVad\":{\"type\":\"boolean\"},\"diarizationThreshold\":{\"type\":\"number\"},\"diarizationMinSegmentMs\":{\"type\":\"integer\"},\"streamingSpkCacheEnable\":{\"type\":\"boolean\"},\"streamingSpkCacheLen\":{\"type\":\"integer\"},\"streamingFifoLen\":{\"type\":\"integer\"},\"streamingChunkLeftContextMs\":{\"type\":\"integer\"},\"streamingChunkRightContextMs\":{\"type\":\"integer\"},\"streamingSpkCacheUpdatePeriod\":{\"type\":\"integer\"},\"prewarm\":{\"type\":\"boolean\"},\"prewarmAudioSeconds\":{\"type\":\"number\"},\"longFormWindowFrames\":{\"type\":\"integer\"},\"longFormContextFrames\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"parakeetEncoderSrc\":{},\"parakeetDecoderSrc\":{},\"parakeetVocabSrc\":{},\"parakeetPreprocessorSrc\":{},\"parakeetCtcModelSrc\":{},\"parakeetTokenizerSrc\":{},\"parakeetSortformerSrc\":{},\"parakeetModelSrc\":{},\"modelType\":{}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"moss-transcribe\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"maxThreads\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"llamacpp-embedding\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"ctxSize\":{\"type\":\"integer\"},\"projectionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"imageMaxTokens\":{\"type\":\"integer\"},\"imageMinTokens\":{\"type\":\"integer\"},\"mmprojNoAudio\":{\"type\":\"boolean\"},\"mmprojUseGpu\":{\"type\":\"boolean\"},\"imageMaxMegapixels\":{\"type\":\"integer\"},\"audioMaxSeconds\":{\"type\":\"integer\"},\"poll\":{\"type\":\"integer\"},\"pollBatch\":{\"type\":\"integer\"},\"backend\":{\"type\":\"array\",\"items\":{\"type\":\"string\",\"enum\":[\"cuda\",\"vulkan\",\"metal\",\"opencl\",\"auto\"]}},\"gpuLayers\":{\"type\":\"integer\"},\"device\":{\"type\":\"string\",\"enum\":[\"gpu\",\"cpu\"]},\"batchSize\":{\"type\":\"integer\"},\"pooling\":{\"type\":\"string\",\"enum\":[\"none\",\"mean\",\"cls\",\"last\"]},\"attention\":{\"type\":\"string\",\"enum\":[\"causal\",\"non-causal\"]},\"embdNormalize\":{\"type\":\"integer\"},\"flashAttention\":{\"type\":\"string\",\"enum\":[\"on\",\"off\",\"auto\"]},\"mainGpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"splitMode\":{\"type\":\"string\",\"enum\":[\"none\",\"layer\"]},\"tensorSplit\":{\"type\":\"string\"},\"verbosity\":{\"type\":\"number\",\"enum\":[0,1,2,3]},\"openclCacheDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"llamacpp-decision\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"device\":{\"type\":\"string\",\"enum\":[\"cpu\",\"gpu\"]},\"gpu_layers\":{\"type\":\"integer\"},\"batch_size\":{\"type\":\"integer\"},\"poll\":{\"type\":\"integer\"},\"pollBatch\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"threads-batch\":{\"type\":\"integer\"},\"flash_attn\":{\"type\":\"string\",\"enum\":[\"on\",\"off\",\"auto\"]},\"verbosity\":{\"type\":\"integer\"},\"main-gpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"split-mode\":{\"type\":\"string\",\"enum\":[\"none\",\"layer\"]},\"tensor-split\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"llamacpp-rerank\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"device\":{\"type\":\"string\",\"enum\":[\"cpu\",\"gpu\"]},\"gpu_layers\":{\"type\":\"integer\"},\"batch_size\":{\"type\":\"integer\"},\"backend\":{\"type\":\"array\",\"items\":{\"type\":\"string\",\"enum\":[\"cuda\",\"vulkan\",\"metal\",\"opencl\",\"auto\"]}},\"poll\":{\"type\":\"integer\"},\"pollBatch\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"threads-batch\":{\"type\":\"integer\"},\"flash_attn\":{\"type\":\"string\",\"enum\":[\"on\",\"off\",\"auto\"]},\"verbosity\":{\"type\":\"integer\"},\"main-gpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"split-mode\":{\"type\":\"string\",\"enum\":[\"none\",\"layer\"]},\"tensor-split\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"ggml-image-embedding\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"backend\":{\"type\":\"string\"},\"threads\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"nmtcpp-translation\"},\"modelConfig\":{\"oneOf\":[{\"type\":\"object\",\"required\":[\"engine\",\"from\",\"to\"],\"properties\":{\"mode\":{\"type\":\"string\",\"enum\":[\"full\"]},\"beamsize\":{\"type\":\"number\"},\"lengthpenalty\":{\"type\":\"number\"},\"maxlength\":{\"type\":\"number\"},\"repetitionpenalty\":{\"type\":\"number\"},\"norepeatngramsize\":{\"type\":\"number\"},\"temperature\":{\"type\":\"number\"},\"topk\":{\"type\":\"number\"},\"topp\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\",\"const\":\"Bergamot\"},\"from\":{\"type\":\"string\",\"enum\":[\"en\",\"ar\",\"bg\",\"ca\",\"cs\",\"de\",\"es\",\"et\",\"fi\",\"fr\",\"hu\",\"is\",\"it\",\"ja\",\"ko\",\"lt\",\"lv\",\"nl\",\"pl\",\"pt\",\"ru\",\"sk\",\"sl\",\"uk\",\"zh\",\"az\",\"be\",\"bn\",\"bs\",\"da\",\"el\",\"fa\",\"gu\",\"he\",\"hi\",\"hr\",\"id\",\"kn\",\"ml\",\"ms\",\"mt\",\"nb\",\"nn\",\"no\",\"re\",\"ro\",\"sq\",\"sr\",\"sv\",\"ta\",\"te\",\"th\",\"tr\",\"vi\"]},\"to\":{\"type\":\"string\",\"enum\":[\"en\",\"ar\",\"bg\",\"ca\",\"cs\",\"de\",\"es\",\"et\",\"fi\",\"fr\",\"hu\",\"is\",\"it\",\"ja\",\"ko\",\"lt\",\"lv\",\"nl\",\"pl\",\"pt\",\"ru\",\"sk\",\"sl\",\"uk\",\"zh\",\"az\",\"be\",\"bn\",\"bs\",\"da\",\"el\",\"fa\",\"gu\",\"he\",\"hi\",\"hr\",\"id\",\"kn\",\"ml\",\"ms\",\"mt\",\"nb\",\"nn\",\"no\",\"re\",\"ro\",\"sq\",\"sr\",\"sv\",\"ta\",\"te\",\"th\",\"tr\",\"vi\"]},\"srcVocabSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"dstVocabSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"normalize\":{\"type\":\"number\"},\"pivotModel\":{\"type\":\"object\",\"required\":[\"modelSrc\"]}}},{\"type\":\"object\",\"required\":[\"engine\",\"from\",\"to\"],\"properties\":{\"mode\":{\"type\":\"string\",\"enum\":[\"full\"]},\"beamsize\":{\"type\":\"number\"},\"lengthpenalty\":{\"type\":\"number\"},\"maxlength\":{\"type\":\"number\"},\"repetitionpenalty\":{\"type\":\"number\"},\"norepeatngramsize\":{\"type\":\"number\"},\"temperature\":{\"type\":\"number\"},\"topk\":{\"type\":\"number\"},\"topp\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\",\"const\":\"IndicTrans\"},\"from\":{\"type\":\"string\",\"enum\":[\"asm_Beng\",\"ben_Beng\",\"brx_Deva\",\"doi_Deva\",\"eng_Latn\",\"gom_Deva\",\"guj_Gujr\",\"hin_Deva\",\"kan_Knda\",\"kas_Arab\",\"kas_Deva\",\"mai_Deva\",\"mal_Mlym\",\"mar_Deva\",\"mni_Beng\",\"mni_Mtei\",\"npi_Deva\",\"ory_Orya\",\"pan_Guru\",\"san_Deva\",\"sat_Olck\",\"snd_Arab\",\"snd_Deva\",\"tam_Taml\",\"tel_Telu\",\"urd_Arab\"]},\"to\":{\"type\":\"string\",\"enum\":[\"asm_Beng\",\"ben_Beng\",\"brx_Deva\",\"doi_Deva\",\"eng_Latn\",\"gom_Deva\",\"guj_Gujr\",\"hin_Deva\",\"kan_Knda\",\"kas_Arab\",\"kas_Deva\",\"mai_Deva\",\"mal_Mlym\",\"mar_Deva\",\"mni_Beng\",\"mni_Mtei\",\"npi_Deva\",\"ory_Orya\",\"pan_Guru\",\"san_Deva\",\"sat_Olck\",\"snd_Arab\",\"snd_Deva\",\"tam_Taml\",\"tel_Telu\",\"urd_Arab\"]}}}]}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"tts-ggml\"},\"modelConfig\":{\"oneOf\":[{\"type\":\"object\",\"required\":[\"ttsEngine\",\"language\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"chatterbox\"},\"language\":{\"type\":\"string\",\"enum\":[\"en\",\"es\",\"fr\",\"de\",\"it\",\"ja\",\"pt\",\"nl\",\"pl\",\"tr\",\"sv\",\"da\",\"fi\",\"no\",\"el\",\"ms\",\"sw\",\"ar\",\"ko\",\"he\",\"ru\",\"zh\",\"hi\"]},\"voice\":{\"type\":\"string\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"ttsSpeed\":{\"type\":\"number\"},\"nCtx\":{\"type\":\"integer\"},\"kvCacheType\":{\"type\":\"string\",\"enum\":[\"f32\",\"f16\",\"q8_0\"]},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"cfmSteps\":{\"type\":\"integer\"},\"cfgRate\":{\"type\":\"number\"},\"streamLeftContextTokens\":{\"type\":\"integer\"},\"batchCfmSteps\":{\"type\":\"integer\"},\"nPredict\":{\"type\":\"integer\"},\"maxSentenceChars\":{\"type\":\"integer\"},\"crossfadeMs\":{\"type\":\"integer\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"repeatPenalty\":{\"type\":\"number\"},\"exaggeration\":{\"type\":\"number\"},\"cfgWeight\":{\"type\":\"number\"},\"minP\":{\"type\":\"number\"},\"voiceDir\":{\"type\":\"string\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"s3genModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"mecabDictSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"cangjieTsvSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrEnhancerCoremlModelPath\":{\"type\":\"string\"},\"lavasrEnhancerCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"lavasrDenoiserCoremlModelPath\":{\"type\":\"string\"},\"lavasrDenoiserCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"ttsSupertonicMultilingual\":{},\"ttsTokenizerSrc\":{},\"ttsSpeechEncoderSrc\":{},\"ttsEmbedTokensSrc\":{},\"ttsConditionalDecoderSrc\":{},\"ttsLanguageModelSrc\":{},\"ttsTextEncoderSrc\":{},\"ttsDurationPredictorSrc\":{},\"ttsVectorEstimatorSrc\":{},\"ttsVocoderSrc\":{},\"ttsUnicodeIndexerSrc\":{},\"ttsTtsConfigSrc\":{},\"ttsVoiceStyleSrc\":{}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"language\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"supertonic\"},\"language\":{\"type\":\"string\",\"enum\":[\"en\",\"ko\",\"ja\",\"ar\",\"bg\",\"cs\",\"da\",\"de\",\"el\",\"es\",\"et\",\"fi\",\"fr\",\"hi\",\"hr\",\"hu\",\"id\",\"it\",\"lt\",\"lv\",\"nl\",\"pl\",\"pt\",\"ro\",\"ru\",\"sk\",\"sl\",\"sv\",\"tr\",\"uk\",\"vi\"]},\"voice\":{\"type\":\"string\"},\"ttsSpeed\":{\"type\":\"number\"},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"ttsNumInferenceSteps\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"vulkanCacheDir\":{\"type\":\"string\"},\"vulkanDevice\":{\"type\":\"integer\"},\"prewarmText\":{\"type\":\"string\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"streamChunkTolerancePct\":{\"type\":\"integer\"},\"streamMinChunkTokens\":{\"type\":\"integer\"},\"precision\":{\"type\":\"string\",\"enum\":[\"auto\",\"f32\",\"f16\",\"q8_0\"]},\"f16Attn\":{\"type\":\"integer\"},\"f16Weights\":{\"type\":\"integer\"},\"kvAttnType\":{\"type\":\"integer\"},\"f16WeightsDenyList\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}},\"vulkanEnvOverrides\":{\"type\":\"object\"},\"voiceStyleTtl\":{\"type\":\"array\",\"items\":{\"type\":\"number\"}},\"voiceStyleDp\":{\"type\":\"array\",\"items\":{\"type\":\"number\"}},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrEnhancerCoremlModelPath\":{\"type\":\"string\"},\"lavasrEnhancerCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"lavasrDenoiserCoremlModelPath\":{\"type\":\"string\"},\"lavasrDenoiserCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"voiceJsonSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"noiseNpySrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"ttsSupertonicMultilingual\":{},\"ttsTokenizerSrc\":{},\"ttsSpeechEncoderSrc\":{},\"ttsEmbedTokensSrc\":{},\"ttsConditionalDecoderSrc\":{},\"ttsLanguageModelSrc\":{},\"ttsTextEncoderSrc\":{},\"ttsDurationPredictorSrc\":{},\"ttsVectorEstimatorSrc\":{},\"ttsVocoderSrc\":{},\"ttsUnicodeIndexerSrc\":{},\"ttsTtsConfigSrc\":{},\"ttsVoiceStyleSrc\":{}}},{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"parler\"},\"description\":{\"type\":\"string\"},\"voiceDescription\":{\"type\":\"string\"},\"voice\":{\"type\":\"string\"},\"emotion\":{\"type\":\"string\",\"enum\":[\"command\",\"anger\",\"narration\",\"conversation\",\"disgust\",\"fear\",\"happy\",\"neutral\",\"proper noun\",\"news\",\"sad\",\"surprise\"]},\"pitch\":{\"type\":\"string\"},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"expressivity\":{\"type\":\"string\"},\"noise\":{\"type\":\"string\"},\"reverb\":{\"type\":\"string\"},\"quality\":{\"type\":\"string\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"maxFrames\":{\"anyOf\":[{\"type\":\"number\",\"const\":0},{\"type\":\"integer\"}]},\"minNewTokens\":{\"type\":\"integer\"},\"normalizeNumbers\":{\"type\":\"boolean\"},\"coremlDecoder\":{\"type\":\"boolean\"},\"coremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"all\",\"cpu_and_ane\",\"cpu_and_gpu\",\"cpu_only\"]},\"coremlStrict\":{\"type\":\"boolean\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrEnhancerCoremlModelPath\":{\"type\":\"string\"},\"lavasrEnhancerCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"lavasrDenoiserCoremlModelPath\":{\"type\":\"string\"},\"lavasrDenoiserCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]}}},{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"cosyvoice3\"},\"emotion\":{\"type\":\"string\",\"enum\":[\"anger\",\"happy\",\"neutral\",\"sad\"]},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"instruct\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\"}]},\"promptText\":{\"type\":\"string\"},\"flowCutPrompt\":{\"type\":\"boolean\"},\"vulkanDevice\":{\"type\":\"integer\"},\"greedy\":{\"type\":\"boolean\"},\"forceSpeechTokens\":{\"type\":\"array\",\"items\":{\"type\":\"integer\"}},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrEnhancerCoremlModelPath\":{\"type\":\"string\"},\"lavasrEnhancerCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"lavasrDenoiserCoremlModelPath\":{\"type\":\"string\"},\"lavasrDenoiserCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"cosyvoice3S3tokModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"cosyvoice3CampplusModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"audio8CodecDecoderModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"audio8\"},\"referenceText\":{\"type\":\"string\"},\"greedy\":{\"type\":\"boolean\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"maxFrames\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"audio8CodecDecoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"audio8CodecEncoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"mossCodecDecoderModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss\"},\"language\":{\"type\":\"string\"},\"durationTokens\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"nCtx\":{\"type\":\"integer\"},\"maxNewTokens\":{\"type\":\"integer\"},\"textTemperature\":{\"type\":\"number\"},\"textTopP\":{\"type\":\"number\"},\"textTopK\":{\"type\":\"integer\"},\"audioTemperature\":{\"type\":\"number\"},\"audioTopP\":{\"type\":\"number\"},\"audioTopK\":{\"type\":\"integer\"},\"audioRepetitionPenalty\":{\"type\":\"number\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"mossCodecDecoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"mossCodecEncoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"dialogueReferenceSrcs\":{\"type\":\"array\",\"items\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}},{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss-sfx\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"seconds\":{\"type\":\"number\"},\"negativePrompt\":{\"type\":\"string\"},\"steps\":{\"type\":\"integer\"},\"guidance\":{\"type\":\"number\"},\"shift\":{\"type\":\"number\"}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"mossSpeechCodecModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss-speech\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"mossSpeechCodecModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}]}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"ggml-ocr\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"langList\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}},\"pipelineType\":{\"type\":\"string\",\"enum\":[\"easyocr\",\"doctr\"]},\"magRatio\":{\"type\":\"number\"},\"canvasSize\":{\"type\":\"number\"},\"defaultRotationAngles\":{\"type\":\"array\",\"items\":{\"type\":\"number\"}},\"contrastRetry\":{\"type\":\"boolean\"},\"lowConfidenceThreshold\":{\"type\":\"number\"},\"recognizerBatchSize\":{\"type\":\"number\"},\"nThreads\":{\"type\":\"number\"},\"backendDevice\":{\"type\":\"string\",\"enum\":[\"cpu\",\"vulkan\",\"metal\",\"opencl\"]},\"gpuDevice\":{\"type\":\"number\"},\"mainGpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"detectorModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"sdcpp-generation\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"mode\":{\"type\":\"string\",\"enum\":[\"diffusion\",\"upscale\",\"video\",\"world\"]},\"threads\":{\"type\":\"number\"},\"device\":{\"type\":\"string\",\"enum\":[\"gpu\",\"cpu\"]},\"main-gpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"prediction\":{\"type\":\"string\",\"enum\":[\"auto\",\"eps\",\"v\",\"edm_v\",\"flow\",\"flux2_flow\"]},\"type\":{\"type\":\"string\",\"enum\":[\"auto\",\"f32\",\"f16\",\"bf16\",\"q2_k\",\"q3_k\",\"q4_0\",\"q4_1\",\"q4_k\",\"q5_0\",\"q5_1\",\"q5_k\",\"q6_k\",\"q8_0\"]},\"rng\":{\"type\":\"string\",\"enum\":[\"cpu\",\"cuda\",\"std_default\"]},\"sampler_rng\":{\"type\":\"string\",\"enum\":[\"cpu\",\"cuda\",\"std_default\"]},\"vae_auto_cpu_fallback\":{\"type\":\"boolean\"},\"vae_auto_cpu_fallback_memory_ratio\":{\"type\":\"number\"},\"vae_tiling\":{\"type\":\"boolean\"},\"offload_to_cpu\":{\"type\":\"boolean\"},\"control_net_cpu\":{\"not\":{}},\"clip_on_cpu\":{\"not\":{}},\"vae_on_cpu\":{\"not\":{}},\"backend\":{\"type\":\"string\"},\"params_backend\":{\"type\":\"string\"},\"max_vram\":{\"anyOf\":[{\"type\":\"number\"},{\"type\":\"string\"}]},\"stream_layers\":{\"type\":\"boolean\"},\"flash_attn\":{\"type\":\"boolean\"},\"diffusion_fa\":{\"type\":\"boolean\"},\"lora_apply_mode\":{\"type\":\"string\",\"enum\":[\"auto\",\"immediately\",\"at_runtime\"]},\"verbosity\":{\"type\":\"number\"},\"clipLModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"clipGModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"t5XxlModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"llmModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"projectionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"vaeModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"highNoiseDiffusionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"uncondModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"clipVisionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"audioVaeModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"embeddingsConnectorsModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"taehvModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"sceneSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"world\":{\"type\":\"object\"},\"upscaler\":{\"type\":\"object\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"audiogen-ggml\"},\"modelConfig\":{\"oneOf\":[{\"type\":\"object\",\"required\":[\"textEncModelSrc\",\"lmModelSrc\",\"ditModelSrc\",\"vaeModelSrc\"],\"properties\":{\"engine\":{\"type\":\"string\",\"const\":\"acestep\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"inferenceSteps\":{\"type\":\"integer\"},\"shift\":{\"type\":\"number\"},\"nGpuLayers\":{\"type\":\"integer\"},\"textEncModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lmModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"ditModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"vaeModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"engine\",\"lmModelSrc\",\"synthModelSrc\"],\"properties\":{\"engine\":{\"type\":\"string\",\"const\":\"minimax\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"device\":{\"type\":\"string\",\"enum\":[\"cpu\",\"gpu\",\"auto\"]},\"coremlProfileDir\":{\"type\":\"string\"},\"coremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"inferenceSteps\":{\"type\":\"integer\"},\"cfgScale\":{\"type\":\"number\"},\"lmModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"synthModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}]}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"ggml-vla\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"backend\":{\"type\":\"array\",\"items\":{\"type\":\"string\",\"enum\":[\"auto\",\"cpu\",\"cuda\",\"vulkan\",\"metal\",\"opencl\",\"hip\",\"rocm\",\"sycl\"]}},\"verbosity\":{\"type\":\"integer\"},\"embodiment\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"integer\"},{\"type\":\"object\"}]}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"ggml-classification\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"modelPath\":{\"type\":\"string\"},\"topK\":{\"type\":\"integer\"},\"nativeLogger\":{\"type\":\"boolean\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\"},\"modelConfig\":{\"type\":\"object\"}}}]},{\"type\":\"object\",\"required\":[\"type\",\"modelId\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelId\":{\"type\":\"string\"},\"modelSrc\":{\"not\":{}},\"withProgress\":{\"not\":{}},\"seed\":{\"not\":{}},\"modelType\":{\"type\":\"string\",\"enum\":[\"whisper\",\"whispercpp-transcription\"]},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"strategy\":{\"type\":\"string\",\"enum\":[\"greedy\",\"beam_search\"]},\"n_threads\":{\"type\":\"integer\"},\"n_max_text_ctx\":{\"type\":\"integer\"},\"offset_ms\":{\"type\":\"integer\"},\"duration_ms\":{\"type\":\"integer\"},\"audio_ctx\":{\"type\":\"integer\"},\"translate\":{\"type\":\"boolean\"},\"no_context\":{\"type\":\"boolean\"},\"no_timestamps\":{\"type\":\"boolean\"},\"single_segment\":{\"type\":\"boolean\"},\"print_special\":{\"type\":\"boolean\"},\"print_progress\":{\"type\":\"boolean\"},\"print_realtime\":{\"type\":\"boolean\"},\"print_timestamps\":{\"type\":\"boolean\"},\"token_timestamps\":{\"type\":\"boolean\"},\"thold_pt\":{\"type\":\"number\"},\"thold_ptsum\":{\"type\":\"number\"},\"max_len\":{\"type\":\"integer\"},\"split_on_word\":{\"type\":\"boolean\"},\"max_tokens\":{\"type\":\"integer\"},\"debug_mode\":{\"type\":\"boolean\"},\"tdrz_enable\":{\"type\":\"boolean\"},\"suppress_regex\":{\"type\":\"string\"},\"initial_prompt\":{\"type\":\"string\"},\"carry_initial_prompt\":{\"type\":\"boolean\"},\"language\":{\"type\":\"string\"},\"detect_language\":{\"type\":\"boolean\"},\"suppress_blank\":{\"type\":\"boolean\"},\"suppress_nst\":{\"type\":\"boolean\"},\"temperature\":{\"type\":\"number\"},\"length_penalty\":{\"type\":\"number\"},\"temperature_inc\":{\"type\":\"number\"},\"entropy_thold\":{\"type\":\"number\"},\"logprob_thold\":{\"type\":\"number\"},\"greedy_best_of\":{\"type\":\"integer\"},\"beam_search_beam_size\":{\"type\":\"integer\"},\"max_initial_ts\":{\"type\":\"number\"},\"no_speech_thold\":{\"type\":\"number\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"vad_params\":{\"type\":\"object\"},\"audio_format\":{\"type\":\"string\",\"enum\":[\"f32le\",\"s16le\"]},\"contextParams\":{\"type\":\"object\"},\"miscConfig\":{\"type\":\"object\"},\"vadModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}}}]}").jsonObject, Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"id\",\"type\"],\"properties\":{\"id\":{\"type\":\"string\"},\"type\":{\"type\":\"string\",\"const\":\"loggingStream\"}}}").jsonObject, Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\",\"registryPath\",\"registrySource\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"modelRegistryGetModel\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"}}}").jsonObject, Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"modelRegistryList\"}}}").jsonObject, @@ -3299,7 +3299,7 @@ sealed class LoadModelRequest { internal object LoadModelRequestSerializer : KSerializer { override val descriptor = kotlinx.serialization.descriptors.buildClassSerialDescriptor("LoadModelRequest") private val shapes = listOf( - Json.parseToJsonElement("{\"anyOf\":[{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"llamacpp-completion\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"backend\":{\"type\":\"array\",\"items\":{\"type\":\"string\",\"enum\":[\"cuda\",\"vulkan\",\"metal\",\"opencl\",\"auto\"]}},\"ctx_size\":{\"type\":\"number\"},\"temp\":{\"type\":\"number\"},\"top_p\":{\"type\":\"number\"},\"top_k\":{\"type\":\"integer\"},\"seed\":{\"type\":\"number\"},\"gpu_layers\":{\"type\":\"number\"},\"lora\":{\"type\":\"string\"},\"device\":{\"type\":\"string\"},\"predict\":{\"anyOf\":[{\"type\":\"number\",\"const\":-1},{\"type\":\"number\",\"const\":-2},{\"type\":\"integer\"}]},\"system_prompt\":{\"type\":\"string\"},\"load_mode\":{\"type\":\"string\",\"enum\":[\"none\",\"mmap\",\"mlock\",\"mmap+mlock\",\"dio\"]},\"verbosity\":{\"type\":\"number\",\"enum\":[0,1,2,3]},\"presence_penalty\":{\"type\":\"number\"},\"frequency_penalty\":{\"type\":\"number\"},\"repeat_penalty\":{\"type\":\"number\"},\"stop_sequences\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}},\"parallel\":{\"type\":\"integer\"},\"tools\":{\"type\":\"boolean\"},\"cache_checkpoints\":{\"type\":\"integer\"},\"cache_checkpoints_max_bytes\":{\"type\":\"integer\"},\"cache_checkpoint_storage\":{\"type\":\"string\",\"enum\":[\"memory\",\"disk\"]},\"cache_checkpoint_dir\":{\"type\":\"string\"},\"cache_share_across_keys\":{\"type\":\"boolean\"},\"cache_ram_mib\":{\"type\":\"integer\"},\"cache-type-k\":{\"type\":\"string\"},\"cache-type-v\":{\"type\":\"string\"},\"main-gpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"rpc-servers\":{\"type\":\"string\"},\"devices\":{\"type\":\"string\"},\"split-mode\":{\"type\":\"string\",\"enum\":[\"none\",\"layer\",\"tensor\"]},\"flash-attn\":{\"type\":\"string\",\"enum\":[\"on\",\"off\",\"auto\"]},\"tensor-split\":{\"type\":\"string\"},\"batch-size\":{\"type\":\"integer\"},\"ubatch-size\":{\"type\":\"integer\"},\"cpu-moe\":{\"type\":\"boolean\"},\"n-cpu-moe\":{\"type\":\"integer\"},\"kv-offload\":{\"type\":\"boolean\"},\"image-max-tokens\":{\"type\":\"integer\"},\"image-min-tokens\":{\"type\":\"integer\"},\"image-max-megapixels\":{\"type\":\"integer\"},\"poll\":{\"type\":\"integer\"},\"pollBatch\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"threads-batch\":{\"type\":\"integer\"},\"cpu-mask\":{\"type\":\"string\"},\"cpu-mask-batch\":{\"type\":\"string\"},\"override-tensor\":{\"type\":\"string\"},\"n-cpu-ffn\":{\"type\":\"integer\"},\"moe-cache-mib\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"const\":\"auto\"}]},\"prefetch-weights\":{\"anyOf\":[{\"type\":\"boolean\"},{\"type\":\"string\",\"const\":\"auto\"}]},\"tensor-read-lazy\":{\"type\":\"string\",\"enum\":[\"on\",\"auto\",\"off\"]},\"fit\":{\"type\":\"boolean\"},\"fit-target\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\"}]},\"fit-ctx\":{\"type\":\"integer\"},\"openclCacheDir\":{\"type\":\"string\"},\"reasoning_budget\":{\"type\":\"integer\"},\"spec_type\":{\"type\":\"string\",\"enum\":[\"none\",\"draft-mtp\",\"draft-dflash\"]},\"spec_draft_n_max\":{\"type\":\"integer\"},\"spec_draft_n_min\":{\"type\":\"integer\"},\"spec_draft_p_min\":{\"type\":\"number\"},\"projectionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"draftModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"image_tile_mode\":{\"type\":\"string\",\"enum\":[\"disabled\",\"batched\",\"sequential\"]},\"image_no_upscale\":{\"type\":\"string\",\"enum\":[\"on\",\"off\"]},\"mmproj-use-gpu\":{\"type\":\"boolean\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"whispercpp-transcription\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"strategy\":{\"type\":\"string\",\"enum\":[\"greedy\",\"beam_search\"]},\"n_threads\":{\"type\":\"integer\"},\"n_max_text_ctx\":{\"type\":\"integer\"},\"offset_ms\":{\"type\":\"integer\"},\"duration_ms\":{\"type\":\"integer\"},\"audio_ctx\":{\"type\":\"integer\"},\"translate\":{\"type\":\"boolean\"},\"no_context\":{\"type\":\"boolean\"},\"no_timestamps\":{\"type\":\"boolean\"},\"single_segment\":{\"type\":\"boolean\"},\"print_special\":{\"type\":\"boolean\"},\"print_progress\":{\"type\":\"boolean\"},\"print_realtime\":{\"type\":\"boolean\"},\"print_timestamps\":{\"type\":\"boolean\"},\"token_timestamps\":{\"type\":\"boolean\"},\"thold_pt\":{\"type\":\"number\"},\"thold_ptsum\":{\"type\":\"number\"},\"max_len\":{\"type\":\"integer\"},\"split_on_word\":{\"type\":\"boolean\"},\"max_tokens\":{\"type\":\"integer\"},\"debug_mode\":{\"type\":\"boolean\"},\"tdrz_enable\":{\"type\":\"boolean\"},\"suppress_regex\":{\"type\":\"string\"},\"initial_prompt\":{\"type\":\"string\"},\"carry_initial_prompt\":{\"type\":\"boolean\"},\"language\":{\"type\":\"string\"},\"detect_language\":{\"type\":\"boolean\"},\"suppress_blank\":{\"type\":\"boolean\"},\"suppress_nst\":{\"type\":\"boolean\"},\"temperature\":{\"type\":\"number\"},\"length_penalty\":{\"type\":\"number\"},\"temperature_inc\":{\"type\":\"number\"},\"entropy_thold\":{\"type\":\"number\"},\"logprob_thold\":{\"type\":\"number\"},\"greedy_best_of\":{\"type\":\"integer\"},\"beam_search_beam_size\":{\"type\":\"integer\"},\"max_initial_ts\":{\"type\":\"number\"},\"no_speech_thold\":{\"type\":\"number\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"vad_params\":{\"type\":\"object\"},\"audio_format\":{\"type\":\"string\",\"enum\":[\"f32le\",\"s16le\"]},\"contextParams\":{\"type\":\"object\"},\"miscConfig\":{\"type\":\"object\"},\"vadModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"bci-whispercpp-transcription\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"whisperConfig\":{\"type\":\"object\"},\"bciConfig\":{\"type\":\"object\"},\"contextParams\":{\"type\":\"object\"},\"miscConfig\":{\"type\":\"object\"},\"backendsDir\":{\"type\":\"string\"},\"embedderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"parakeet-transcription\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"maxThreads\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"backend\":{\"type\":\"string\"},\"speakerAttribution\":{\"type\":\"object\",\"required\":[\"modelSrc\"]},\"sampleRate\":{\"type\":\"integer\"},\"channels\":{\"type\":\"integer\"},\"captionEnabled\":{\"type\":\"boolean\"},\"timestampsEnabled\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"integer\"},\"streaming\":{\"type\":\"boolean\"},\"streamingChunkMs\":{\"type\":\"integer\"},\"streamingHistoryMs\":{\"type\":\"integer\"},\"streamingEmitPartials\":{\"type\":\"boolean\"},\"streamingEnergyVad\":{\"type\":\"boolean\"},\"streamingEnergyVadThresholdDb\":{\"type\":\"number\"},\"streamingEnergyVadWindowMs\":{\"type\":\"integer\"},\"streamingEnergyVadHangoverMs\":{\"type\":\"integer\"},\"streamingLeftContextMs\":{\"type\":\"integer\"},\"streamingRightLookaheadMs\":{\"type\":\"integer\"},\"language\":{\"type\":\"string\"},\"streamingSpeakerVad\":{\"type\":\"boolean\"},\"diarizationThreshold\":{\"type\":\"number\"},\"diarizationMinSegmentMs\":{\"type\":\"integer\"},\"streamingSpkCacheEnable\":{\"type\":\"boolean\"},\"streamingSpkCacheLen\":{\"type\":\"integer\"},\"streamingFifoLen\":{\"type\":\"integer\"},\"streamingChunkLeftContextMs\":{\"type\":\"integer\"},\"streamingChunkRightContextMs\":{\"type\":\"integer\"},\"streamingSpkCacheUpdatePeriod\":{\"type\":\"integer\"},\"prewarm\":{\"type\":\"boolean\"},\"prewarmAudioSeconds\":{\"type\":\"number\"},\"longFormWindowFrames\":{\"type\":\"integer\"},\"longFormContextFrames\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"parakeetEncoderSrc\":{},\"parakeetDecoderSrc\":{},\"parakeetVocabSrc\":{},\"parakeetPreprocessorSrc\":{},\"parakeetCtcModelSrc\":{},\"parakeetTokenizerSrc\":{},\"parakeetSortformerSrc\":{},\"parakeetModelSrc\":{},\"modelType\":{}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"moss-transcribe\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"maxThreads\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"llamacpp-embedding\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"ctxSize\":{\"type\":\"integer\"},\"projectionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"imageMaxTokens\":{\"type\":\"integer\"},\"imageMinTokens\":{\"type\":\"integer\"},\"mmprojNoAudio\":{\"type\":\"boolean\"},\"mmprojUseGpu\":{\"type\":\"boolean\"},\"imageMaxMegapixels\":{\"type\":\"integer\"},\"audioMaxSeconds\":{\"type\":\"integer\"},\"poll\":{\"type\":\"integer\"},\"pollBatch\":{\"type\":\"integer\"},\"backend\":{\"type\":\"array\",\"items\":{\"type\":\"string\",\"enum\":[\"cuda\",\"vulkan\",\"metal\",\"opencl\",\"auto\"]}},\"gpuLayers\":{\"type\":\"integer\"},\"device\":{\"type\":\"string\",\"enum\":[\"gpu\",\"cpu\"]},\"batchSize\":{\"type\":\"integer\"},\"pooling\":{\"type\":\"string\",\"enum\":[\"none\",\"mean\",\"cls\",\"last\"]},\"attention\":{\"type\":\"string\",\"enum\":[\"causal\",\"non-causal\"]},\"embdNormalize\":{\"type\":\"integer\"},\"flashAttention\":{\"type\":\"string\",\"enum\":[\"on\",\"off\",\"auto\"]},\"mainGpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"splitMode\":{\"type\":\"string\",\"enum\":[\"none\",\"layer\"]},\"tensorSplit\":{\"type\":\"string\"},\"verbosity\":{\"type\":\"number\",\"enum\":[0,1,2,3]},\"openclCacheDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"llamacpp-decision\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"device\":{\"type\":\"string\",\"enum\":[\"cpu\",\"gpu\"]},\"gpu_layers\":{\"type\":\"integer\"},\"batch_size\":{\"type\":\"integer\"},\"poll\":{\"type\":\"integer\"},\"pollBatch\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"threads-batch\":{\"type\":\"integer\"},\"flash_attn\":{\"type\":\"string\",\"enum\":[\"on\",\"off\",\"auto\"]},\"verbosity\":{\"type\":\"integer\"},\"main-gpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"split-mode\":{\"type\":\"string\",\"enum\":[\"none\",\"layer\"]},\"tensor-split\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"llamacpp-rerank\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"device\":{\"type\":\"string\",\"enum\":[\"cpu\",\"gpu\"]},\"gpu_layers\":{\"type\":\"integer\"},\"batch_size\":{\"type\":\"integer\"},\"backend\":{\"type\":\"array\",\"items\":{\"type\":\"string\",\"enum\":[\"cuda\",\"vulkan\",\"metal\",\"opencl\",\"auto\"]}},\"poll\":{\"type\":\"integer\"},\"pollBatch\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"threads-batch\":{\"type\":\"integer\"},\"flash_attn\":{\"type\":\"string\",\"enum\":[\"on\",\"off\",\"auto\"]},\"verbosity\":{\"type\":\"integer\"},\"main-gpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"split-mode\":{\"type\":\"string\",\"enum\":[\"none\",\"layer\"]},\"tensor-split\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"ggml-image-embedding\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"backend\":{\"type\":\"string\"},\"threads\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"nmtcpp-translation\"},\"modelConfig\":{\"oneOf\":[{\"type\":\"object\",\"required\":[\"engine\",\"from\",\"to\"],\"properties\":{\"mode\":{\"type\":\"string\",\"enum\":[\"full\"]},\"beamsize\":{\"type\":\"number\"},\"lengthpenalty\":{\"type\":\"number\"},\"maxlength\":{\"type\":\"number\"},\"repetitionpenalty\":{\"type\":\"number\"},\"norepeatngramsize\":{\"type\":\"number\"},\"temperature\":{\"type\":\"number\"},\"topk\":{\"type\":\"number\"},\"topp\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\",\"const\":\"Bergamot\"},\"from\":{\"type\":\"string\",\"enum\":[\"en\",\"ar\",\"bg\",\"ca\",\"cs\",\"de\",\"es\",\"et\",\"fi\",\"fr\",\"hu\",\"is\",\"it\",\"ja\",\"ko\",\"lt\",\"lv\",\"nl\",\"pl\",\"pt\",\"ru\",\"sk\",\"sl\",\"uk\",\"zh\",\"az\",\"be\",\"bn\",\"bs\",\"da\",\"el\",\"fa\",\"gu\",\"he\",\"hi\",\"hr\",\"id\",\"kn\",\"ml\",\"ms\",\"mt\",\"nb\",\"nn\",\"no\",\"re\",\"ro\",\"sq\",\"sr\",\"sv\",\"ta\",\"te\",\"th\",\"tr\",\"vi\"]},\"to\":{\"type\":\"string\",\"enum\":[\"en\",\"ar\",\"bg\",\"ca\",\"cs\",\"de\",\"es\",\"et\",\"fi\",\"fr\",\"hu\",\"is\",\"it\",\"ja\",\"ko\",\"lt\",\"lv\",\"nl\",\"pl\",\"pt\",\"ru\",\"sk\",\"sl\",\"uk\",\"zh\",\"az\",\"be\",\"bn\",\"bs\",\"da\",\"el\",\"fa\",\"gu\",\"he\",\"hi\",\"hr\",\"id\",\"kn\",\"ml\",\"ms\",\"mt\",\"nb\",\"nn\",\"no\",\"re\",\"ro\",\"sq\",\"sr\",\"sv\",\"ta\",\"te\",\"th\",\"tr\",\"vi\"]},\"srcVocabSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"dstVocabSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"normalize\":{\"type\":\"number\"},\"pivotModel\":{\"type\":\"object\",\"required\":[\"modelSrc\"]}}},{\"type\":\"object\",\"required\":[\"engine\",\"from\",\"to\"],\"properties\":{\"mode\":{\"type\":\"string\",\"enum\":[\"full\"]},\"beamsize\":{\"type\":\"number\"},\"lengthpenalty\":{\"type\":\"number\"},\"maxlength\":{\"type\":\"number\"},\"repetitionpenalty\":{\"type\":\"number\"},\"norepeatngramsize\":{\"type\":\"number\"},\"temperature\":{\"type\":\"number\"},\"topk\":{\"type\":\"number\"},\"topp\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\",\"const\":\"IndicTrans\"},\"from\":{\"type\":\"string\",\"enum\":[\"asm_Beng\",\"ben_Beng\",\"brx_Deva\",\"doi_Deva\",\"eng_Latn\",\"gom_Deva\",\"guj_Gujr\",\"hin_Deva\",\"kan_Knda\",\"kas_Arab\",\"kas_Deva\",\"mai_Deva\",\"mal_Mlym\",\"mar_Deva\",\"mni_Beng\",\"mni_Mtei\",\"npi_Deva\",\"ory_Orya\",\"pan_Guru\",\"san_Deva\",\"sat_Olck\",\"snd_Arab\",\"snd_Deva\",\"tam_Taml\",\"tel_Telu\",\"urd_Arab\"]},\"to\":{\"type\":\"string\",\"enum\":[\"asm_Beng\",\"ben_Beng\",\"brx_Deva\",\"doi_Deva\",\"eng_Latn\",\"gom_Deva\",\"guj_Gujr\",\"hin_Deva\",\"kan_Knda\",\"kas_Arab\",\"kas_Deva\",\"mai_Deva\",\"mal_Mlym\",\"mar_Deva\",\"mni_Beng\",\"mni_Mtei\",\"npi_Deva\",\"ory_Orya\",\"pan_Guru\",\"san_Deva\",\"sat_Olck\",\"snd_Arab\",\"snd_Deva\",\"tam_Taml\",\"tel_Telu\",\"urd_Arab\"]}}}]}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"tts-ggml\"},\"modelConfig\":{\"oneOf\":[{\"type\":\"object\",\"required\":[\"ttsEngine\",\"language\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"chatterbox\"},\"language\":{\"type\":\"string\",\"enum\":[\"en\",\"es\",\"fr\",\"de\",\"it\",\"ja\",\"pt\",\"nl\",\"pl\",\"tr\",\"sv\",\"da\",\"fi\",\"no\",\"el\",\"ms\",\"sw\",\"ar\",\"ko\",\"he\",\"ru\",\"zh\",\"hi\"]},\"voice\":{\"type\":\"string\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"ttsSpeed\":{\"type\":\"number\"},\"nCtx\":{\"type\":\"integer\"},\"kvCacheType\":{\"type\":\"string\",\"enum\":[\"f32\",\"f16\",\"q8_0\"]},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"cfmSteps\":{\"type\":\"integer\"},\"cfgRate\":{\"type\":\"number\"},\"streamLeftContextTokens\":{\"type\":\"integer\"},\"batchCfmSteps\":{\"type\":\"integer\"},\"nPredict\":{\"type\":\"integer\"},\"maxSentenceChars\":{\"type\":\"integer\"},\"crossfadeMs\":{\"type\":\"integer\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"repeatPenalty\":{\"type\":\"number\"},\"exaggeration\":{\"type\":\"number\"},\"cfgWeight\":{\"type\":\"number\"},\"minP\":{\"type\":\"number\"},\"voiceDir\":{\"type\":\"string\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"s3genModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"mecabDictSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"cangjieTsvSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"ttsSupertonicMultilingual\":{},\"ttsTokenizerSrc\":{},\"ttsSpeechEncoderSrc\":{},\"ttsEmbedTokensSrc\":{},\"ttsConditionalDecoderSrc\":{},\"ttsLanguageModelSrc\":{},\"ttsTextEncoderSrc\":{},\"ttsDurationPredictorSrc\":{},\"ttsVectorEstimatorSrc\":{},\"ttsVocoderSrc\":{},\"ttsUnicodeIndexerSrc\":{},\"ttsTtsConfigSrc\":{},\"ttsVoiceStyleSrc\":{}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"language\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"supertonic\"},\"language\":{\"type\":\"string\",\"enum\":[\"en\",\"ko\",\"ja\",\"ar\",\"bg\",\"cs\",\"da\",\"de\",\"el\",\"es\",\"et\",\"fi\",\"fr\",\"hi\",\"hr\",\"hu\",\"id\",\"it\",\"lt\",\"lv\",\"nl\",\"pl\",\"pt\",\"ro\",\"ru\",\"sk\",\"sl\",\"sv\",\"tr\",\"uk\",\"vi\"]},\"voice\":{\"type\":\"string\"},\"ttsSpeed\":{\"type\":\"number\"},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"ttsNumInferenceSteps\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"vulkanCacheDir\":{\"type\":\"string\"},\"vulkanDevice\":{\"type\":\"integer\"},\"prewarmText\":{\"type\":\"string\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"streamChunkTolerancePct\":{\"type\":\"integer\"},\"streamMinChunkTokens\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"voiceJsonSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"noiseNpySrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"ttsSupertonicMultilingual\":{},\"ttsTokenizerSrc\":{},\"ttsSpeechEncoderSrc\":{},\"ttsEmbedTokensSrc\":{},\"ttsConditionalDecoderSrc\":{},\"ttsLanguageModelSrc\":{},\"ttsTextEncoderSrc\":{},\"ttsDurationPredictorSrc\":{},\"ttsVectorEstimatorSrc\":{},\"ttsVocoderSrc\":{},\"ttsUnicodeIndexerSrc\":{},\"ttsTtsConfigSrc\":{},\"ttsVoiceStyleSrc\":{}}},{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"parler\"},\"description\":{\"type\":\"string\"},\"voiceDescription\":{\"type\":\"string\"},\"voice\":{\"type\":\"string\"},\"emotion\":{\"type\":\"string\",\"enum\":[\"command\",\"anger\",\"narration\",\"conversation\",\"disgust\",\"fear\",\"happy\",\"neutral\",\"proper noun\",\"news\",\"sad\",\"surprise\"]},\"pitch\":{\"type\":\"string\"},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"expressivity\":{\"type\":\"string\"},\"noise\":{\"type\":\"string\"},\"reverb\":{\"type\":\"string\"},\"quality\":{\"type\":\"string\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"maxFrames\":{\"anyOf\":[{\"type\":\"number\",\"const\":0},{\"type\":\"integer\"}]},\"minNewTokens\":{\"type\":\"integer\"},\"normalizeNumbers\":{\"type\":\"boolean\"},\"backendsDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"cosyvoice3\"},\"emotion\":{\"type\":\"string\",\"enum\":[\"anger\",\"happy\",\"neutral\",\"sad\"]},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"instruct\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\"}]},\"promptText\":{\"type\":\"string\"},\"flowCutPrompt\":{\"type\":\"boolean\"},\"vulkanDevice\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"cosyvoice3S3tokModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"cosyvoice3CampplusModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"audio8CodecDecoderModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"audio8\"},\"referenceText\":{\"type\":\"string\"},\"greedy\":{\"type\":\"boolean\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"maxFrames\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"audio8CodecDecoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"audio8CodecEncoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"mossCodecDecoderModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss\"},\"language\":{\"type\":\"string\"},\"durationTokens\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"mossCodecDecoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"mossCodecEncoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"dialogueReferenceSrcs\":{\"type\":\"array\",\"items\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}},{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss-sfx\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"seconds\":{\"type\":\"number\"},\"negativePrompt\":{\"type\":\"string\"},\"steps\":{\"type\":\"integer\"},\"guidance\":{\"type\":\"number\"},\"shift\":{\"type\":\"number\"}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"mossSpeechCodecModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss-speech\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"mossSpeechCodecModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}]}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"ggml-ocr\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"langList\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}},\"pipelineType\":{\"type\":\"string\",\"enum\":[\"easyocr\",\"doctr\"]},\"magRatio\":{\"type\":\"number\"},\"canvasSize\":{\"type\":\"number\"},\"defaultRotationAngles\":{\"type\":\"array\",\"items\":{\"type\":\"number\"}},\"contrastRetry\":{\"type\":\"boolean\"},\"lowConfidenceThreshold\":{\"type\":\"number\"},\"recognizerBatchSize\":{\"type\":\"number\"},\"nThreads\":{\"type\":\"number\"},\"backendDevice\":{\"type\":\"string\",\"enum\":[\"cpu\",\"vulkan\",\"metal\",\"opencl\"]},\"gpuDevice\":{\"type\":\"number\"},\"mainGpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"detectorModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"sdcpp-generation\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"mode\":{\"type\":\"string\",\"enum\":[\"diffusion\",\"upscale\",\"video\",\"world\"]},\"threads\":{\"type\":\"number\"},\"device\":{\"type\":\"string\",\"enum\":[\"gpu\",\"cpu\"]},\"main-gpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"prediction\":{\"type\":\"string\",\"enum\":[\"auto\",\"eps\",\"v\",\"edm_v\",\"flow\",\"flux2_flow\"]},\"type\":{\"type\":\"string\",\"enum\":[\"auto\",\"f32\",\"f16\",\"bf16\",\"q2_k\",\"q3_k\",\"q4_0\",\"q4_1\",\"q4_k\",\"q5_0\",\"q5_1\",\"q5_k\",\"q6_k\",\"q8_0\"]},\"rng\":{\"type\":\"string\",\"enum\":[\"cpu\",\"cuda\",\"std_default\"]},\"sampler_rng\":{\"type\":\"string\",\"enum\":[\"cpu\",\"cuda\",\"std_default\"]},\"vae_auto_cpu_fallback\":{\"type\":\"boolean\"},\"vae_auto_cpu_fallback_memory_ratio\":{\"type\":\"number\"},\"vae_tiling\":{\"type\":\"boolean\"},\"offload_to_cpu\":{\"type\":\"boolean\"},\"control_net_cpu\":{\"not\":{}},\"clip_on_cpu\":{\"not\":{}},\"vae_on_cpu\":{\"not\":{}},\"backend\":{\"type\":\"string\"},\"params_backend\":{\"type\":\"string\"},\"max_vram\":{\"anyOf\":[{\"type\":\"number\"},{\"type\":\"string\"}]},\"stream_layers\":{\"type\":\"boolean\"},\"flash_attn\":{\"type\":\"boolean\"},\"diffusion_fa\":{\"type\":\"boolean\"},\"lora_apply_mode\":{\"type\":\"string\",\"enum\":[\"auto\",\"immediately\",\"at_runtime\"]},\"verbosity\":{\"type\":\"number\"},\"clipLModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"clipGModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"t5XxlModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"llmModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"projectionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"vaeModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"highNoiseDiffusionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"uncondModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"clipVisionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"audioVaeModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"embeddingsConnectorsModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"taehvModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"sceneSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"world\":{\"type\":\"object\"},\"upscaler\":{\"type\":\"object\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"audiogen-ggml\"},\"modelConfig\":{\"oneOf\":[{\"type\":\"object\",\"required\":[\"textEncModelSrc\",\"lmModelSrc\",\"ditModelSrc\",\"vaeModelSrc\"],\"properties\":{\"engine\":{\"type\":\"string\",\"const\":\"acestep\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"inferenceSteps\":{\"type\":\"integer\"},\"shift\":{\"type\":\"number\"},\"nGpuLayers\":{\"type\":\"integer\"},\"textEncModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lmModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"ditModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"vaeModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"engine\",\"lmModelSrc\",\"synthModelSrc\"],\"properties\":{\"engine\":{\"type\":\"string\",\"const\":\"minimax\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"device\":{\"type\":\"string\",\"enum\":[\"cpu\",\"gpu\",\"auto\"]},\"coremlProfileDir\":{\"type\":\"string\"},\"coremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"inferenceSteps\":{\"type\":\"integer\"},\"cfgScale\":{\"type\":\"number\"},\"lmModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"synthModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}]}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"ggml-vla\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"backend\":{\"type\":\"array\",\"items\":{\"type\":\"string\",\"enum\":[\"auto\",\"cpu\",\"cuda\",\"vulkan\",\"metal\",\"opencl\",\"hip\",\"rocm\",\"sycl\"]}},\"verbosity\":{\"type\":\"integer\"},\"embodiment\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"integer\"},{\"type\":\"object\"}]}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"ggml-classification\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"modelPath\":{\"type\":\"string\"},\"topK\":{\"type\":\"integer\"},\"nativeLogger\":{\"type\":\"boolean\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\"},\"modelConfig\":{\"type\":\"object\"}}}]}").jsonObject, + Json.parseToJsonElement("{\"anyOf\":[{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"llamacpp-completion\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"backend\":{\"type\":\"array\",\"items\":{\"type\":\"string\",\"enum\":[\"cuda\",\"vulkan\",\"metal\",\"opencl\",\"auto\"]}},\"ctx_size\":{\"type\":\"number\"},\"temp\":{\"type\":\"number\"},\"top_p\":{\"type\":\"number\"},\"top_k\":{\"type\":\"integer\"},\"seed\":{\"type\":\"number\"},\"gpu_layers\":{\"type\":\"number\"},\"lora\":{\"type\":\"string\"},\"device\":{\"type\":\"string\"},\"predict\":{\"anyOf\":[{\"type\":\"number\",\"const\":-1},{\"type\":\"number\",\"const\":-2},{\"type\":\"integer\"}]},\"system_prompt\":{\"type\":\"string\"},\"load_mode\":{\"type\":\"string\",\"enum\":[\"none\",\"mmap\",\"mlock\",\"mmap+mlock\",\"dio\"]},\"verbosity\":{\"type\":\"number\",\"enum\":[0,1,2,3]},\"presence_penalty\":{\"type\":\"number\"},\"frequency_penalty\":{\"type\":\"number\"},\"repeat_penalty\":{\"type\":\"number\"},\"stop_sequences\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}},\"parallel\":{\"type\":\"integer\"},\"tools\":{\"type\":\"boolean\"},\"cache_checkpoints\":{\"type\":\"integer\"},\"cache_checkpoints_max_bytes\":{\"type\":\"integer\"},\"cache_checkpoint_storage\":{\"type\":\"string\",\"enum\":[\"memory\",\"disk\"]},\"cache_checkpoint_dir\":{\"type\":\"string\"},\"cache_share_across_keys\":{\"type\":\"boolean\"},\"cache_ram_mib\":{\"type\":\"integer\"},\"cache-type-k\":{\"type\":\"string\"},\"cache-type-v\":{\"type\":\"string\"},\"main-gpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"rpc-servers\":{\"type\":\"string\"},\"devices\":{\"type\":\"string\"},\"split-mode\":{\"type\":\"string\",\"enum\":[\"none\",\"layer\",\"tensor\"]},\"flash-attn\":{\"type\":\"string\",\"enum\":[\"on\",\"off\",\"auto\"]},\"tensor-split\":{\"type\":\"string\"},\"batch-size\":{\"type\":\"integer\"},\"ubatch-size\":{\"type\":\"integer\"},\"cpu-moe\":{\"type\":\"boolean\"},\"n-cpu-moe\":{\"type\":\"integer\"},\"kv-offload\":{\"type\":\"boolean\"},\"image-max-tokens\":{\"type\":\"integer\"},\"image-min-tokens\":{\"type\":\"integer\"},\"image-max-megapixels\":{\"type\":\"integer\"},\"poll\":{\"type\":\"integer\"},\"pollBatch\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"threads-batch\":{\"type\":\"integer\"},\"cpu-mask\":{\"type\":\"string\"},\"cpu-mask-batch\":{\"type\":\"string\"},\"override-tensor\":{\"type\":\"string\"},\"n-cpu-ffn\":{\"type\":\"integer\"},\"moe-cache-mib\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"const\":\"auto\"}]},\"prefetch-weights\":{\"anyOf\":[{\"type\":\"boolean\"},{\"type\":\"string\",\"const\":\"auto\"}]},\"tensor-read-lazy\":{\"type\":\"string\",\"enum\":[\"on\",\"auto\",\"off\"]},\"fit\":{\"type\":\"boolean\"},\"fit-target\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\"}]},\"fit-ctx\":{\"type\":\"integer\"},\"openclCacheDir\":{\"type\":\"string\"},\"reasoning_budget\":{\"type\":\"integer\"},\"spec_type\":{\"type\":\"string\",\"enum\":[\"none\",\"draft-mtp\",\"draft-dflash\"]},\"spec_draft_n_max\":{\"type\":\"integer\"},\"spec_draft_n_min\":{\"type\":\"integer\"},\"spec_draft_p_min\":{\"type\":\"number\"},\"projectionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"draftModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"image_tile_mode\":{\"type\":\"string\",\"enum\":[\"disabled\",\"batched\",\"sequential\"]},\"image_no_upscale\":{\"type\":\"string\",\"enum\":[\"on\",\"off\"]},\"mmproj-use-gpu\":{\"type\":\"boolean\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"whispercpp-transcription\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"strategy\":{\"type\":\"string\",\"enum\":[\"greedy\",\"beam_search\"]},\"n_threads\":{\"type\":\"integer\"},\"n_max_text_ctx\":{\"type\":\"integer\"},\"offset_ms\":{\"type\":\"integer\"},\"duration_ms\":{\"type\":\"integer\"},\"audio_ctx\":{\"type\":\"integer\"},\"translate\":{\"type\":\"boolean\"},\"no_context\":{\"type\":\"boolean\"},\"no_timestamps\":{\"type\":\"boolean\"},\"single_segment\":{\"type\":\"boolean\"},\"print_special\":{\"type\":\"boolean\"},\"print_progress\":{\"type\":\"boolean\"},\"print_realtime\":{\"type\":\"boolean\"},\"print_timestamps\":{\"type\":\"boolean\"},\"token_timestamps\":{\"type\":\"boolean\"},\"thold_pt\":{\"type\":\"number\"},\"thold_ptsum\":{\"type\":\"number\"},\"max_len\":{\"type\":\"integer\"},\"split_on_word\":{\"type\":\"boolean\"},\"max_tokens\":{\"type\":\"integer\"},\"debug_mode\":{\"type\":\"boolean\"},\"tdrz_enable\":{\"type\":\"boolean\"},\"suppress_regex\":{\"type\":\"string\"},\"initial_prompt\":{\"type\":\"string\"},\"carry_initial_prompt\":{\"type\":\"boolean\"},\"language\":{\"type\":\"string\"},\"detect_language\":{\"type\":\"boolean\"},\"suppress_blank\":{\"type\":\"boolean\"},\"suppress_nst\":{\"type\":\"boolean\"},\"temperature\":{\"type\":\"number\"},\"length_penalty\":{\"type\":\"number\"},\"temperature_inc\":{\"type\":\"number\"},\"entropy_thold\":{\"type\":\"number\"},\"logprob_thold\":{\"type\":\"number\"},\"greedy_best_of\":{\"type\":\"integer\"},\"beam_search_beam_size\":{\"type\":\"integer\"},\"max_initial_ts\":{\"type\":\"number\"},\"no_speech_thold\":{\"type\":\"number\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"vad_params\":{\"type\":\"object\"},\"audio_format\":{\"type\":\"string\",\"enum\":[\"f32le\",\"s16le\"]},\"contextParams\":{\"type\":\"object\"},\"miscConfig\":{\"type\":\"object\"},\"vadModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"bci-whispercpp-transcription\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"whisperConfig\":{\"type\":\"object\"},\"bciConfig\":{\"type\":\"object\"},\"contextParams\":{\"type\":\"object\"},\"miscConfig\":{\"type\":\"object\"},\"backendsDir\":{\"type\":\"string\"},\"embedderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"parakeet-transcription\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"maxThreads\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"backend\":{\"type\":\"string\"},\"speakerAttribution\":{\"type\":\"object\",\"required\":[\"modelSrc\"]},\"sampleRate\":{\"type\":\"integer\"},\"channels\":{\"type\":\"integer\"},\"captionEnabled\":{\"type\":\"boolean\"},\"timestampsEnabled\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"integer\"},\"streaming\":{\"type\":\"boolean\"},\"streamingChunkMs\":{\"type\":\"integer\"},\"streamingHistoryMs\":{\"type\":\"integer\"},\"streamingEmitPartials\":{\"type\":\"boolean\"},\"streamingEnergyVad\":{\"type\":\"boolean\"},\"streamingEnergyVadThresholdDb\":{\"type\":\"number\"},\"streamingEnergyVadWindowMs\":{\"type\":\"integer\"},\"streamingEnergyVadHangoverMs\":{\"type\":\"integer\"},\"streamingLeftContextMs\":{\"type\":\"integer\"},\"streamingRightLookaheadMs\":{\"type\":\"integer\"},\"language\":{\"type\":\"string\"},\"streamingSpeakerVad\":{\"type\":\"boolean\"},\"diarizationThreshold\":{\"type\":\"number\"},\"diarizationMinSegmentMs\":{\"type\":\"integer\"},\"streamingSpkCacheEnable\":{\"type\":\"boolean\"},\"streamingSpkCacheLen\":{\"type\":\"integer\"},\"streamingFifoLen\":{\"type\":\"integer\"},\"streamingChunkLeftContextMs\":{\"type\":\"integer\"},\"streamingChunkRightContextMs\":{\"type\":\"integer\"},\"streamingSpkCacheUpdatePeriod\":{\"type\":\"integer\"},\"prewarm\":{\"type\":\"boolean\"},\"prewarmAudioSeconds\":{\"type\":\"number\"},\"longFormWindowFrames\":{\"type\":\"integer\"},\"longFormContextFrames\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"parakeetEncoderSrc\":{},\"parakeetDecoderSrc\":{},\"parakeetVocabSrc\":{},\"parakeetPreprocessorSrc\":{},\"parakeetCtcModelSrc\":{},\"parakeetTokenizerSrc\":{},\"parakeetSortformerSrc\":{},\"parakeetModelSrc\":{},\"modelType\":{}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"moss-transcribe\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"maxThreads\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"llamacpp-embedding\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"ctxSize\":{\"type\":\"integer\"},\"projectionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"imageMaxTokens\":{\"type\":\"integer\"},\"imageMinTokens\":{\"type\":\"integer\"},\"mmprojNoAudio\":{\"type\":\"boolean\"},\"mmprojUseGpu\":{\"type\":\"boolean\"},\"imageMaxMegapixels\":{\"type\":\"integer\"},\"audioMaxSeconds\":{\"type\":\"integer\"},\"poll\":{\"type\":\"integer\"},\"pollBatch\":{\"type\":\"integer\"},\"backend\":{\"type\":\"array\",\"items\":{\"type\":\"string\",\"enum\":[\"cuda\",\"vulkan\",\"metal\",\"opencl\",\"auto\"]}},\"gpuLayers\":{\"type\":\"integer\"},\"device\":{\"type\":\"string\",\"enum\":[\"gpu\",\"cpu\"]},\"batchSize\":{\"type\":\"integer\"},\"pooling\":{\"type\":\"string\",\"enum\":[\"none\",\"mean\",\"cls\",\"last\"]},\"attention\":{\"type\":\"string\",\"enum\":[\"causal\",\"non-causal\"]},\"embdNormalize\":{\"type\":\"integer\"},\"flashAttention\":{\"type\":\"string\",\"enum\":[\"on\",\"off\",\"auto\"]},\"mainGpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"splitMode\":{\"type\":\"string\",\"enum\":[\"none\",\"layer\"]},\"tensorSplit\":{\"type\":\"string\"},\"verbosity\":{\"type\":\"number\",\"enum\":[0,1,2,3]},\"openclCacheDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"llamacpp-decision\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"device\":{\"type\":\"string\",\"enum\":[\"cpu\",\"gpu\"]},\"gpu_layers\":{\"type\":\"integer\"},\"batch_size\":{\"type\":\"integer\"},\"poll\":{\"type\":\"integer\"},\"pollBatch\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"threads-batch\":{\"type\":\"integer\"},\"flash_attn\":{\"type\":\"string\",\"enum\":[\"on\",\"off\",\"auto\"]},\"verbosity\":{\"type\":\"integer\"},\"main-gpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"split-mode\":{\"type\":\"string\",\"enum\":[\"none\",\"layer\"]},\"tensor-split\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"llamacpp-rerank\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"device\":{\"type\":\"string\",\"enum\":[\"cpu\",\"gpu\"]},\"gpu_layers\":{\"type\":\"integer\"},\"batch_size\":{\"type\":\"integer\"},\"backend\":{\"type\":\"array\",\"items\":{\"type\":\"string\",\"enum\":[\"cuda\",\"vulkan\",\"metal\",\"opencl\",\"auto\"]}},\"poll\":{\"type\":\"integer\"},\"pollBatch\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"threads-batch\":{\"type\":\"integer\"},\"flash_attn\":{\"type\":\"string\",\"enum\":[\"on\",\"off\",\"auto\"]},\"verbosity\":{\"type\":\"integer\"},\"main-gpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"split-mode\":{\"type\":\"string\",\"enum\":[\"none\",\"layer\"]},\"tensor-split\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"ggml-image-embedding\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"backend\":{\"type\":\"string\"},\"threads\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"nmtcpp-translation\"},\"modelConfig\":{\"oneOf\":[{\"type\":\"object\",\"required\":[\"engine\",\"from\",\"to\"],\"properties\":{\"mode\":{\"type\":\"string\",\"enum\":[\"full\"]},\"beamsize\":{\"type\":\"number\"},\"lengthpenalty\":{\"type\":\"number\"},\"maxlength\":{\"type\":\"number\"},\"repetitionpenalty\":{\"type\":\"number\"},\"norepeatngramsize\":{\"type\":\"number\"},\"temperature\":{\"type\":\"number\"},\"topk\":{\"type\":\"number\"},\"topp\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\",\"const\":\"Bergamot\"},\"from\":{\"type\":\"string\",\"enum\":[\"en\",\"ar\",\"bg\",\"ca\",\"cs\",\"de\",\"es\",\"et\",\"fi\",\"fr\",\"hu\",\"is\",\"it\",\"ja\",\"ko\",\"lt\",\"lv\",\"nl\",\"pl\",\"pt\",\"ru\",\"sk\",\"sl\",\"uk\",\"zh\",\"az\",\"be\",\"bn\",\"bs\",\"da\",\"el\",\"fa\",\"gu\",\"he\",\"hi\",\"hr\",\"id\",\"kn\",\"ml\",\"ms\",\"mt\",\"nb\",\"nn\",\"no\",\"re\",\"ro\",\"sq\",\"sr\",\"sv\",\"ta\",\"te\",\"th\",\"tr\",\"vi\"]},\"to\":{\"type\":\"string\",\"enum\":[\"en\",\"ar\",\"bg\",\"ca\",\"cs\",\"de\",\"es\",\"et\",\"fi\",\"fr\",\"hu\",\"is\",\"it\",\"ja\",\"ko\",\"lt\",\"lv\",\"nl\",\"pl\",\"pt\",\"ru\",\"sk\",\"sl\",\"uk\",\"zh\",\"az\",\"be\",\"bn\",\"bs\",\"da\",\"el\",\"fa\",\"gu\",\"he\",\"hi\",\"hr\",\"id\",\"kn\",\"ml\",\"ms\",\"mt\",\"nb\",\"nn\",\"no\",\"re\",\"ro\",\"sq\",\"sr\",\"sv\",\"ta\",\"te\",\"th\",\"tr\",\"vi\"]},\"srcVocabSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"dstVocabSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"normalize\":{\"type\":\"number\"},\"pivotModel\":{\"type\":\"object\",\"required\":[\"modelSrc\"]}}},{\"type\":\"object\",\"required\":[\"engine\",\"from\",\"to\"],\"properties\":{\"mode\":{\"type\":\"string\",\"enum\":[\"full\"]},\"beamsize\":{\"type\":\"number\"},\"lengthpenalty\":{\"type\":\"number\"},\"maxlength\":{\"type\":\"number\"},\"repetitionpenalty\":{\"type\":\"number\"},\"norepeatngramsize\":{\"type\":\"number\"},\"temperature\":{\"type\":\"number\"},\"topk\":{\"type\":\"number\"},\"topp\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\",\"const\":\"IndicTrans\"},\"from\":{\"type\":\"string\",\"enum\":[\"asm_Beng\",\"ben_Beng\",\"brx_Deva\",\"doi_Deva\",\"eng_Latn\",\"gom_Deva\",\"guj_Gujr\",\"hin_Deva\",\"kan_Knda\",\"kas_Arab\",\"kas_Deva\",\"mai_Deva\",\"mal_Mlym\",\"mar_Deva\",\"mni_Beng\",\"mni_Mtei\",\"npi_Deva\",\"ory_Orya\",\"pan_Guru\",\"san_Deva\",\"sat_Olck\",\"snd_Arab\",\"snd_Deva\",\"tam_Taml\",\"tel_Telu\",\"urd_Arab\"]},\"to\":{\"type\":\"string\",\"enum\":[\"asm_Beng\",\"ben_Beng\",\"brx_Deva\",\"doi_Deva\",\"eng_Latn\",\"gom_Deva\",\"guj_Gujr\",\"hin_Deva\",\"kan_Knda\",\"kas_Arab\",\"kas_Deva\",\"mai_Deva\",\"mal_Mlym\",\"mar_Deva\",\"mni_Beng\",\"mni_Mtei\",\"npi_Deva\",\"ory_Orya\",\"pan_Guru\",\"san_Deva\",\"sat_Olck\",\"snd_Arab\",\"snd_Deva\",\"tam_Taml\",\"tel_Telu\",\"urd_Arab\"]}}}]}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"tts-ggml\"},\"modelConfig\":{\"oneOf\":[{\"type\":\"object\",\"required\":[\"ttsEngine\",\"language\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"chatterbox\"},\"language\":{\"type\":\"string\",\"enum\":[\"en\",\"es\",\"fr\",\"de\",\"it\",\"ja\",\"pt\",\"nl\",\"pl\",\"tr\",\"sv\",\"da\",\"fi\",\"no\",\"el\",\"ms\",\"sw\",\"ar\",\"ko\",\"he\",\"ru\",\"zh\",\"hi\"]},\"voice\":{\"type\":\"string\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"ttsSpeed\":{\"type\":\"number\"},\"nCtx\":{\"type\":\"integer\"},\"kvCacheType\":{\"type\":\"string\",\"enum\":[\"f32\",\"f16\",\"q8_0\"]},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"cfmSteps\":{\"type\":\"integer\"},\"cfgRate\":{\"type\":\"number\"},\"streamLeftContextTokens\":{\"type\":\"integer\"},\"batchCfmSteps\":{\"type\":\"integer\"},\"nPredict\":{\"type\":\"integer\"},\"maxSentenceChars\":{\"type\":\"integer\"},\"crossfadeMs\":{\"type\":\"integer\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"repeatPenalty\":{\"type\":\"number\"},\"exaggeration\":{\"type\":\"number\"},\"cfgWeight\":{\"type\":\"number\"},\"minP\":{\"type\":\"number\"},\"voiceDir\":{\"type\":\"string\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"s3genModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"mecabDictSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"cangjieTsvSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrEnhancerCoremlModelPath\":{\"type\":\"string\"},\"lavasrEnhancerCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"lavasrDenoiserCoremlModelPath\":{\"type\":\"string\"},\"lavasrDenoiserCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"ttsSupertonicMultilingual\":{},\"ttsTokenizerSrc\":{},\"ttsSpeechEncoderSrc\":{},\"ttsEmbedTokensSrc\":{},\"ttsConditionalDecoderSrc\":{},\"ttsLanguageModelSrc\":{},\"ttsTextEncoderSrc\":{},\"ttsDurationPredictorSrc\":{},\"ttsVectorEstimatorSrc\":{},\"ttsVocoderSrc\":{},\"ttsUnicodeIndexerSrc\":{},\"ttsTtsConfigSrc\":{},\"ttsVoiceStyleSrc\":{}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"language\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"supertonic\"},\"language\":{\"type\":\"string\",\"enum\":[\"en\",\"ko\",\"ja\",\"ar\",\"bg\",\"cs\",\"da\",\"de\",\"el\",\"es\",\"et\",\"fi\",\"fr\",\"hi\",\"hr\",\"hu\",\"id\",\"it\",\"lt\",\"lv\",\"nl\",\"pl\",\"pt\",\"ro\",\"ru\",\"sk\",\"sl\",\"sv\",\"tr\",\"uk\",\"vi\"]},\"voice\":{\"type\":\"string\"},\"ttsSpeed\":{\"type\":\"number\"},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"ttsNumInferenceSteps\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"vulkanCacheDir\":{\"type\":\"string\"},\"vulkanDevice\":{\"type\":\"integer\"},\"prewarmText\":{\"type\":\"string\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"streamChunkTolerancePct\":{\"type\":\"integer\"},\"streamMinChunkTokens\":{\"type\":\"integer\"},\"precision\":{\"type\":\"string\",\"enum\":[\"auto\",\"f32\",\"f16\",\"q8_0\"]},\"f16Attn\":{\"type\":\"integer\"},\"f16Weights\":{\"type\":\"integer\"},\"kvAttnType\":{\"type\":\"integer\"},\"f16WeightsDenyList\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}},\"vulkanEnvOverrides\":{\"type\":\"object\"},\"voiceStyleTtl\":{\"type\":\"array\",\"items\":{\"type\":\"number\"}},\"voiceStyleDp\":{\"type\":\"array\",\"items\":{\"type\":\"number\"}},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrEnhancerCoremlModelPath\":{\"type\":\"string\"},\"lavasrEnhancerCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"lavasrDenoiserCoremlModelPath\":{\"type\":\"string\"},\"lavasrDenoiserCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"voiceJsonSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"noiseNpySrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"ttsSupertonicMultilingual\":{},\"ttsTokenizerSrc\":{},\"ttsSpeechEncoderSrc\":{},\"ttsEmbedTokensSrc\":{},\"ttsConditionalDecoderSrc\":{},\"ttsLanguageModelSrc\":{},\"ttsTextEncoderSrc\":{},\"ttsDurationPredictorSrc\":{},\"ttsVectorEstimatorSrc\":{},\"ttsVocoderSrc\":{},\"ttsUnicodeIndexerSrc\":{},\"ttsTtsConfigSrc\":{},\"ttsVoiceStyleSrc\":{}}},{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"parler\"},\"description\":{\"type\":\"string\"},\"voiceDescription\":{\"type\":\"string\"},\"voice\":{\"type\":\"string\"},\"emotion\":{\"type\":\"string\",\"enum\":[\"command\",\"anger\",\"narration\",\"conversation\",\"disgust\",\"fear\",\"happy\",\"neutral\",\"proper noun\",\"news\",\"sad\",\"surprise\"]},\"pitch\":{\"type\":\"string\"},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"expressivity\":{\"type\":\"string\"},\"noise\":{\"type\":\"string\"},\"reverb\":{\"type\":\"string\"},\"quality\":{\"type\":\"string\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"maxFrames\":{\"anyOf\":[{\"type\":\"number\",\"const\":0},{\"type\":\"integer\"}]},\"minNewTokens\":{\"type\":\"integer\"},\"normalizeNumbers\":{\"type\":\"boolean\"},\"coremlDecoder\":{\"type\":\"boolean\"},\"coremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"all\",\"cpu_and_ane\",\"cpu_and_gpu\",\"cpu_only\"]},\"coremlStrict\":{\"type\":\"boolean\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrEnhancerCoremlModelPath\":{\"type\":\"string\"},\"lavasrEnhancerCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"lavasrDenoiserCoremlModelPath\":{\"type\":\"string\"},\"lavasrDenoiserCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]}}},{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"cosyvoice3\"},\"emotion\":{\"type\":\"string\",\"enum\":[\"anger\",\"happy\",\"neutral\",\"sad\"]},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"instruct\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\"}]},\"promptText\":{\"type\":\"string\"},\"flowCutPrompt\":{\"type\":\"boolean\"},\"vulkanDevice\":{\"type\":\"integer\"},\"greedy\":{\"type\":\"boolean\"},\"forceSpeechTokens\":{\"type\":\"array\",\"items\":{\"type\":\"integer\"}},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrEnhancerCoremlModelPath\":{\"type\":\"string\"},\"lavasrEnhancerCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"lavasrDenoiserCoremlModelPath\":{\"type\":\"string\"},\"lavasrDenoiserCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"cosyvoice3S3tokModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"cosyvoice3CampplusModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"audio8CodecDecoderModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"audio8\"},\"referenceText\":{\"type\":\"string\"},\"greedy\":{\"type\":\"boolean\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"maxFrames\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"audio8CodecDecoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"audio8CodecEncoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"mossCodecDecoderModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss\"},\"language\":{\"type\":\"string\"},\"durationTokens\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"nCtx\":{\"type\":\"integer\"},\"maxNewTokens\":{\"type\":\"integer\"},\"textTemperature\":{\"type\":\"number\"},\"textTopP\":{\"type\":\"number\"},\"textTopK\":{\"type\":\"integer\"},\"audioTemperature\":{\"type\":\"number\"},\"audioTopP\":{\"type\":\"number\"},\"audioTopK\":{\"type\":\"integer\"},\"audioRepetitionPenalty\":{\"type\":\"number\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"mossCodecDecoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"mossCodecEncoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"dialogueReferenceSrcs\":{\"type\":\"array\",\"items\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}},{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss-sfx\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"seconds\":{\"type\":\"number\"},\"negativePrompt\":{\"type\":\"string\"},\"steps\":{\"type\":\"integer\"},\"guidance\":{\"type\":\"number\"},\"shift\":{\"type\":\"number\"}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"mossSpeechCodecModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss-speech\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"mossSpeechCodecModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}]}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"ggml-ocr\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"langList\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}},\"pipelineType\":{\"type\":\"string\",\"enum\":[\"easyocr\",\"doctr\"]},\"magRatio\":{\"type\":\"number\"},\"canvasSize\":{\"type\":\"number\"},\"defaultRotationAngles\":{\"type\":\"array\",\"items\":{\"type\":\"number\"}},\"contrastRetry\":{\"type\":\"boolean\"},\"lowConfidenceThreshold\":{\"type\":\"number\"},\"recognizerBatchSize\":{\"type\":\"number\"},\"nThreads\":{\"type\":\"number\"},\"backendDevice\":{\"type\":\"string\",\"enum\":[\"cpu\",\"vulkan\",\"metal\",\"opencl\"]},\"gpuDevice\":{\"type\":\"number\"},\"mainGpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"detectorModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"sdcpp-generation\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"mode\":{\"type\":\"string\",\"enum\":[\"diffusion\",\"upscale\",\"video\",\"world\"]},\"threads\":{\"type\":\"number\"},\"device\":{\"type\":\"string\",\"enum\":[\"gpu\",\"cpu\"]},\"main-gpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"prediction\":{\"type\":\"string\",\"enum\":[\"auto\",\"eps\",\"v\",\"edm_v\",\"flow\",\"flux2_flow\"]},\"type\":{\"type\":\"string\",\"enum\":[\"auto\",\"f32\",\"f16\",\"bf16\",\"q2_k\",\"q3_k\",\"q4_0\",\"q4_1\",\"q4_k\",\"q5_0\",\"q5_1\",\"q5_k\",\"q6_k\",\"q8_0\"]},\"rng\":{\"type\":\"string\",\"enum\":[\"cpu\",\"cuda\",\"std_default\"]},\"sampler_rng\":{\"type\":\"string\",\"enum\":[\"cpu\",\"cuda\",\"std_default\"]},\"vae_auto_cpu_fallback\":{\"type\":\"boolean\"},\"vae_auto_cpu_fallback_memory_ratio\":{\"type\":\"number\"},\"vae_tiling\":{\"type\":\"boolean\"},\"offload_to_cpu\":{\"type\":\"boolean\"},\"control_net_cpu\":{\"not\":{}},\"clip_on_cpu\":{\"not\":{}},\"vae_on_cpu\":{\"not\":{}},\"backend\":{\"type\":\"string\"},\"params_backend\":{\"type\":\"string\"},\"max_vram\":{\"anyOf\":[{\"type\":\"number\"},{\"type\":\"string\"}]},\"stream_layers\":{\"type\":\"boolean\"},\"flash_attn\":{\"type\":\"boolean\"},\"diffusion_fa\":{\"type\":\"boolean\"},\"lora_apply_mode\":{\"type\":\"string\",\"enum\":[\"auto\",\"immediately\",\"at_runtime\"]},\"verbosity\":{\"type\":\"number\"},\"clipLModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"clipGModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"t5XxlModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"llmModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"projectionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"vaeModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"highNoiseDiffusionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"uncondModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"clipVisionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"audioVaeModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"embeddingsConnectorsModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"taehvModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"sceneSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"world\":{\"type\":\"object\"},\"upscaler\":{\"type\":\"object\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"audiogen-ggml\"},\"modelConfig\":{\"oneOf\":[{\"type\":\"object\",\"required\":[\"textEncModelSrc\",\"lmModelSrc\",\"ditModelSrc\",\"vaeModelSrc\"],\"properties\":{\"engine\":{\"type\":\"string\",\"const\":\"acestep\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"inferenceSteps\":{\"type\":\"integer\"},\"shift\":{\"type\":\"number\"},\"nGpuLayers\":{\"type\":\"integer\"},\"textEncModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lmModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"ditModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"vaeModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"engine\",\"lmModelSrc\",\"synthModelSrc\"],\"properties\":{\"engine\":{\"type\":\"string\",\"const\":\"minimax\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"device\":{\"type\":\"string\",\"enum\":[\"cpu\",\"gpu\",\"auto\"]},\"coremlProfileDir\":{\"type\":\"string\"},\"coremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"inferenceSteps\":{\"type\":\"integer\"},\"cfgScale\":{\"type\":\"number\"},\"lmModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"synthModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}]}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"ggml-vla\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"backend\":{\"type\":\"array\",\"items\":{\"type\":\"string\",\"enum\":[\"auto\",\"cpu\",\"cuda\",\"vulkan\",\"metal\",\"opencl\",\"hip\",\"rocm\",\"sycl\"]}},\"verbosity\":{\"type\":\"integer\"},\"embodiment\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"integer\"},{\"type\":\"object\"}]}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"ggml-classification\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"modelPath\":{\"type\":\"string\"},\"topK\":{\"type\":\"integer\"},\"nativeLogger\":{\"type\":\"boolean\"}}}}},{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\"},\"modelConfig\":{\"type\":\"object\"}}}]}").jsonObject, Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\",\"modelId\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelId\":{\"type\":\"string\"},\"modelSrc\":{\"not\":{}},\"withProgress\":{\"not\":{}},\"seed\":{\"not\":{}},\"modelType\":{\"type\":\"string\",\"enum\":[\"whisper\",\"whispercpp-transcription\"]},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"strategy\":{\"type\":\"string\",\"enum\":[\"greedy\",\"beam_search\"]},\"n_threads\":{\"type\":\"integer\"},\"n_max_text_ctx\":{\"type\":\"integer\"},\"offset_ms\":{\"type\":\"integer\"},\"duration_ms\":{\"type\":\"integer\"},\"audio_ctx\":{\"type\":\"integer\"},\"translate\":{\"type\":\"boolean\"},\"no_context\":{\"type\":\"boolean\"},\"no_timestamps\":{\"type\":\"boolean\"},\"single_segment\":{\"type\":\"boolean\"},\"print_special\":{\"type\":\"boolean\"},\"print_progress\":{\"type\":\"boolean\"},\"print_realtime\":{\"type\":\"boolean\"},\"print_timestamps\":{\"type\":\"boolean\"},\"token_timestamps\":{\"type\":\"boolean\"},\"thold_pt\":{\"type\":\"number\"},\"thold_ptsum\":{\"type\":\"number\"},\"max_len\":{\"type\":\"integer\"},\"split_on_word\":{\"type\":\"boolean\"},\"max_tokens\":{\"type\":\"integer\"},\"debug_mode\":{\"type\":\"boolean\"},\"tdrz_enable\":{\"type\":\"boolean\"},\"suppress_regex\":{\"type\":\"string\"},\"initial_prompt\":{\"type\":\"string\"},\"carry_initial_prompt\":{\"type\":\"boolean\"},\"language\":{\"type\":\"string\"},\"detect_language\":{\"type\":\"boolean\"},\"suppress_blank\":{\"type\":\"boolean\"},\"suppress_nst\":{\"type\":\"boolean\"},\"temperature\":{\"type\":\"number\"},\"length_penalty\":{\"type\":\"number\"},\"temperature_inc\":{\"type\":\"number\"},\"entropy_thold\":{\"type\":\"number\"},\"logprob_thold\":{\"type\":\"number\"},\"greedy_best_of\":{\"type\":\"integer\"},\"beam_search_beam_size\":{\"type\":\"integer\"},\"max_initial_ts\":{\"type\":\"number\"},\"no_speech_thold\":{\"type\":\"number\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"vad_params\":{\"type\":\"object\"},\"audio_format\":{\"type\":\"string\",\"enum\":[\"f32le\",\"s16le\"]},\"contextParams\":{\"type\":\"object\"},\"miscConfig\":{\"type\":\"object\"},\"vadModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}}}").jsonObject, ) override fun deserialize(decoder: Decoder): LoadModelRequest { @@ -3352,7 +3352,7 @@ internal object LoadModelSrcRequestSerializer : KSerializer Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"llamacpp-rerank\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"device\":{\"type\":\"string\",\"enum\":[\"cpu\",\"gpu\"]},\"gpu_layers\":{\"type\":\"integer\"},\"batch_size\":{\"type\":\"integer\"},\"backend\":{\"type\":\"array\",\"items\":{\"type\":\"string\",\"enum\":[\"cuda\",\"vulkan\",\"metal\",\"opencl\",\"auto\"]}},\"poll\":{\"type\":\"integer\"},\"pollBatch\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"threads-batch\":{\"type\":\"integer\"},\"flash_attn\":{\"type\":\"string\",\"enum\":[\"on\",\"off\",\"auto\"]},\"verbosity\":{\"type\":\"integer\"},\"main-gpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"split-mode\":{\"type\":\"string\",\"enum\":[\"none\",\"layer\"]},\"tensor-split\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"}}}}}").jsonObject, Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"ggml-image-embedding\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"backend\":{\"type\":\"string\"},\"threads\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"}}}}}").jsonObject, Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"nmtcpp-translation\"},\"modelConfig\":{\"oneOf\":[{\"type\":\"object\",\"required\":[\"engine\",\"from\",\"to\"],\"properties\":{\"mode\":{\"type\":\"string\",\"enum\":[\"full\"]},\"beamsize\":{\"type\":\"number\"},\"lengthpenalty\":{\"type\":\"number\"},\"maxlength\":{\"type\":\"number\"},\"repetitionpenalty\":{\"type\":\"number\"},\"norepeatngramsize\":{\"type\":\"number\"},\"temperature\":{\"type\":\"number\"},\"topk\":{\"type\":\"number\"},\"topp\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\",\"const\":\"Bergamot\"},\"from\":{\"type\":\"string\",\"enum\":[\"en\",\"ar\",\"bg\",\"ca\",\"cs\",\"de\",\"es\",\"et\",\"fi\",\"fr\",\"hu\",\"is\",\"it\",\"ja\",\"ko\",\"lt\",\"lv\",\"nl\",\"pl\",\"pt\",\"ru\",\"sk\",\"sl\",\"uk\",\"zh\",\"az\",\"be\",\"bn\",\"bs\",\"da\",\"el\",\"fa\",\"gu\",\"he\",\"hi\",\"hr\",\"id\",\"kn\",\"ml\",\"ms\",\"mt\",\"nb\",\"nn\",\"no\",\"re\",\"ro\",\"sq\",\"sr\",\"sv\",\"ta\",\"te\",\"th\",\"tr\",\"vi\"]},\"to\":{\"type\":\"string\",\"enum\":[\"en\",\"ar\",\"bg\",\"ca\",\"cs\",\"de\",\"es\",\"et\",\"fi\",\"fr\",\"hu\",\"is\",\"it\",\"ja\",\"ko\",\"lt\",\"lv\",\"nl\",\"pl\",\"pt\",\"ru\",\"sk\",\"sl\",\"uk\",\"zh\",\"az\",\"be\",\"bn\",\"bs\",\"da\",\"el\",\"fa\",\"gu\",\"he\",\"hi\",\"hr\",\"id\",\"kn\",\"ml\",\"ms\",\"mt\",\"nb\",\"nn\",\"no\",\"re\",\"ro\",\"sq\",\"sr\",\"sv\",\"ta\",\"te\",\"th\",\"tr\",\"vi\"]},\"srcVocabSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"dstVocabSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"normalize\":{\"type\":\"number\"},\"pivotModel\":{\"type\":\"object\",\"required\":[\"modelSrc\"]}}},{\"type\":\"object\",\"required\":[\"engine\",\"from\",\"to\"],\"properties\":{\"mode\":{\"type\":\"string\",\"enum\":[\"full\"]},\"beamsize\":{\"type\":\"number\"},\"lengthpenalty\":{\"type\":\"number\"},\"maxlength\":{\"type\":\"number\"},\"repetitionpenalty\":{\"type\":\"number\"},\"norepeatngramsize\":{\"type\":\"number\"},\"temperature\":{\"type\":\"number\"},\"topk\":{\"type\":\"number\"},\"topp\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\",\"const\":\"IndicTrans\"},\"from\":{\"type\":\"string\",\"enum\":[\"asm_Beng\",\"ben_Beng\",\"brx_Deva\",\"doi_Deva\",\"eng_Latn\",\"gom_Deva\",\"guj_Gujr\",\"hin_Deva\",\"kan_Knda\",\"kas_Arab\",\"kas_Deva\",\"mai_Deva\",\"mal_Mlym\",\"mar_Deva\",\"mni_Beng\",\"mni_Mtei\",\"npi_Deva\",\"ory_Orya\",\"pan_Guru\",\"san_Deva\",\"sat_Olck\",\"snd_Arab\",\"snd_Deva\",\"tam_Taml\",\"tel_Telu\",\"urd_Arab\"]},\"to\":{\"type\":\"string\",\"enum\":[\"asm_Beng\",\"ben_Beng\",\"brx_Deva\",\"doi_Deva\",\"eng_Latn\",\"gom_Deva\",\"guj_Gujr\",\"hin_Deva\",\"kan_Knda\",\"kas_Arab\",\"kas_Deva\",\"mai_Deva\",\"mal_Mlym\",\"mar_Deva\",\"mni_Beng\",\"mni_Mtei\",\"npi_Deva\",\"ory_Orya\",\"pan_Guru\",\"san_Deva\",\"sat_Olck\",\"snd_Arab\",\"snd_Deva\",\"tam_Taml\",\"tel_Telu\",\"urd_Arab\"]}}}]}}}").jsonObject, - Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"tts-ggml\"},\"modelConfig\":{\"oneOf\":[{\"type\":\"object\",\"required\":[\"ttsEngine\",\"language\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"chatterbox\"},\"language\":{\"type\":\"string\",\"enum\":[\"en\",\"es\",\"fr\",\"de\",\"it\",\"ja\",\"pt\",\"nl\",\"pl\",\"tr\",\"sv\",\"da\",\"fi\",\"no\",\"el\",\"ms\",\"sw\",\"ar\",\"ko\",\"he\",\"ru\",\"zh\",\"hi\"]},\"voice\":{\"type\":\"string\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"ttsSpeed\":{\"type\":\"number\"},\"nCtx\":{\"type\":\"integer\"},\"kvCacheType\":{\"type\":\"string\",\"enum\":[\"f32\",\"f16\",\"q8_0\"]},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"cfmSteps\":{\"type\":\"integer\"},\"cfgRate\":{\"type\":\"number\"},\"streamLeftContextTokens\":{\"type\":\"integer\"},\"batchCfmSteps\":{\"type\":\"integer\"},\"nPredict\":{\"type\":\"integer\"},\"maxSentenceChars\":{\"type\":\"integer\"},\"crossfadeMs\":{\"type\":\"integer\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"repeatPenalty\":{\"type\":\"number\"},\"exaggeration\":{\"type\":\"number\"},\"cfgWeight\":{\"type\":\"number\"},\"minP\":{\"type\":\"number\"},\"voiceDir\":{\"type\":\"string\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"s3genModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"mecabDictSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"cangjieTsvSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"ttsSupertonicMultilingual\":{},\"ttsTokenizerSrc\":{},\"ttsSpeechEncoderSrc\":{},\"ttsEmbedTokensSrc\":{},\"ttsConditionalDecoderSrc\":{},\"ttsLanguageModelSrc\":{},\"ttsTextEncoderSrc\":{},\"ttsDurationPredictorSrc\":{},\"ttsVectorEstimatorSrc\":{},\"ttsVocoderSrc\":{},\"ttsUnicodeIndexerSrc\":{},\"ttsTtsConfigSrc\":{},\"ttsVoiceStyleSrc\":{}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"language\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"supertonic\"},\"language\":{\"type\":\"string\",\"enum\":[\"en\",\"ko\",\"ja\",\"ar\",\"bg\",\"cs\",\"da\",\"de\",\"el\",\"es\",\"et\",\"fi\",\"fr\",\"hi\",\"hr\",\"hu\",\"id\",\"it\",\"lt\",\"lv\",\"nl\",\"pl\",\"pt\",\"ro\",\"ru\",\"sk\",\"sl\",\"sv\",\"tr\",\"uk\",\"vi\"]},\"voice\":{\"type\":\"string\"},\"ttsSpeed\":{\"type\":\"number\"},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"ttsNumInferenceSteps\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"vulkanCacheDir\":{\"type\":\"string\"},\"vulkanDevice\":{\"type\":\"integer\"},\"prewarmText\":{\"type\":\"string\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"streamChunkTolerancePct\":{\"type\":\"integer\"},\"streamMinChunkTokens\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"voiceJsonSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"noiseNpySrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"ttsSupertonicMultilingual\":{},\"ttsTokenizerSrc\":{},\"ttsSpeechEncoderSrc\":{},\"ttsEmbedTokensSrc\":{},\"ttsConditionalDecoderSrc\":{},\"ttsLanguageModelSrc\":{},\"ttsTextEncoderSrc\":{},\"ttsDurationPredictorSrc\":{},\"ttsVectorEstimatorSrc\":{},\"ttsVocoderSrc\":{},\"ttsUnicodeIndexerSrc\":{},\"ttsTtsConfigSrc\":{},\"ttsVoiceStyleSrc\":{}}},{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"parler\"},\"description\":{\"type\":\"string\"},\"voiceDescription\":{\"type\":\"string\"},\"voice\":{\"type\":\"string\"},\"emotion\":{\"type\":\"string\",\"enum\":[\"command\",\"anger\",\"narration\",\"conversation\",\"disgust\",\"fear\",\"happy\",\"neutral\",\"proper noun\",\"news\",\"sad\",\"surprise\"]},\"pitch\":{\"type\":\"string\"},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"expressivity\":{\"type\":\"string\"},\"noise\":{\"type\":\"string\"},\"reverb\":{\"type\":\"string\"},\"quality\":{\"type\":\"string\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"maxFrames\":{\"anyOf\":[{\"type\":\"number\",\"const\":0},{\"type\":\"integer\"}]},\"minNewTokens\":{\"type\":\"integer\"},\"normalizeNumbers\":{\"type\":\"boolean\"},\"backendsDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"cosyvoice3\"},\"emotion\":{\"type\":\"string\",\"enum\":[\"anger\",\"happy\",\"neutral\",\"sad\"]},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"instruct\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\"}]},\"promptText\":{\"type\":\"string\"},\"flowCutPrompt\":{\"type\":\"boolean\"},\"vulkanDevice\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"cosyvoice3S3tokModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"cosyvoice3CampplusModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"audio8CodecDecoderModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"audio8\"},\"referenceText\":{\"type\":\"string\"},\"greedy\":{\"type\":\"boolean\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"maxFrames\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"audio8CodecDecoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"audio8CodecEncoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"mossCodecDecoderModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss\"},\"language\":{\"type\":\"string\"},\"durationTokens\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"mossCodecDecoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"mossCodecEncoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"dialogueReferenceSrcs\":{\"type\":\"array\",\"items\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}},{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss-sfx\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"seconds\":{\"type\":\"number\"},\"negativePrompt\":{\"type\":\"string\"},\"steps\":{\"type\":\"integer\"},\"guidance\":{\"type\":\"number\"},\"shift\":{\"type\":\"number\"}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"mossSpeechCodecModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss-speech\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"mossSpeechCodecModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}]}}}").jsonObject, + Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"tts-ggml\"},\"modelConfig\":{\"oneOf\":[{\"type\":\"object\",\"required\":[\"ttsEngine\",\"language\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"chatterbox\"},\"language\":{\"type\":\"string\",\"enum\":[\"en\",\"es\",\"fr\",\"de\",\"it\",\"ja\",\"pt\",\"nl\",\"pl\",\"tr\",\"sv\",\"da\",\"fi\",\"no\",\"el\",\"ms\",\"sw\",\"ar\",\"ko\",\"he\",\"ru\",\"zh\",\"hi\"]},\"voice\":{\"type\":\"string\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"ttsSpeed\":{\"type\":\"number\"},\"nCtx\":{\"type\":\"integer\"},\"kvCacheType\":{\"type\":\"string\",\"enum\":[\"f32\",\"f16\",\"q8_0\"]},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"cfmSteps\":{\"type\":\"integer\"},\"cfgRate\":{\"type\":\"number\"},\"streamLeftContextTokens\":{\"type\":\"integer\"},\"batchCfmSteps\":{\"type\":\"integer\"},\"nPredict\":{\"type\":\"integer\"},\"maxSentenceChars\":{\"type\":\"integer\"},\"crossfadeMs\":{\"type\":\"integer\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"repeatPenalty\":{\"type\":\"number\"},\"exaggeration\":{\"type\":\"number\"},\"cfgWeight\":{\"type\":\"number\"},\"minP\":{\"type\":\"number\"},\"voiceDir\":{\"type\":\"string\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"s3genModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"mecabDictSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"cangjieTsvSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrEnhancerCoremlModelPath\":{\"type\":\"string\"},\"lavasrEnhancerCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"lavasrDenoiserCoremlModelPath\":{\"type\":\"string\"},\"lavasrDenoiserCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"ttsSupertonicMultilingual\":{},\"ttsTokenizerSrc\":{},\"ttsSpeechEncoderSrc\":{},\"ttsEmbedTokensSrc\":{},\"ttsConditionalDecoderSrc\":{},\"ttsLanguageModelSrc\":{},\"ttsTextEncoderSrc\":{},\"ttsDurationPredictorSrc\":{},\"ttsVectorEstimatorSrc\":{},\"ttsVocoderSrc\":{},\"ttsUnicodeIndexerSrc\":{},\"ttsTtsConfigSrc\":{},\"ttsVoiceStyleSrc\":{}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"language\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"supertonic\"},\"language\":{\"type\":\"string\",\"enum\":[\"en\",\"ko\",\"ja\",\"ar\",\"bg\",\"cs\",\"da\",\"de\",\"el\",\"es\",\"et\",\"fi\",\"fr\",\"hi\",\"hr\",\"hu\",\"id\",\"it\",\"lt\",\"lv\",\"nl\",\"pl\",\"pt\",\"ro\",\"ru\",\"sk\",\"sl\",\"sv\",\"tr\",\"uk\",\"vi\"]},\"voice\":{\"type\":\"string\"},\"ttsSpeed\":{\"type\":\"number\"},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"ttsNumInferenceSteps\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"vulkanCacheDir\":{\"type\":\"string\"},\"vulkanDevice\":{\"type\":\"integer\"},\"prewarmText\":{\"type\":\"string\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"streamChunkTolerancePct\":{\"type\":\"integer\"},\"streamMinChunkTokens\":{\"type\":\"integer\"},\"precision\":{\"type\":\"string\",\"enum\":[\"auto\",\"f32\",\"f16\",\"q8_0\"]},\"f16Attn\":{\"type\":\"integer\"},\"f16Weights\":{\"type\":\"integer\"},\"kvAttnType\":{\"type\":\"integer\"},\"f16WeightsDenyList\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}},\"vulkanEnvOverrides\":{\"type\":\"object\"},\"voiceStyleTtl\":{\"type\":\"array\",\"items\":{\"type\":\"number\"}},\"voiceStyleDp\":{\"type\":\"array\",\"items\":{\"type\":\"number\"}},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrEnhancerCoremlModelPath\":{\"type\":\"string\"},\"lavasrEnhancerCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"lavasrDenoiserCoremlModelPath\":{\"type\":\"string\"},\"lavasrDenoiserCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"voiceJsonSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"noiseNpySrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"ttsSupertonicMultilingual\":{},\"ttsTokenizerSrc\":{},\"ttsSpeechEncoderSrc\":{},\"ttsEmbedTokensSrc\":{},\"ttsConditionalDecoderSrc\":{},\"ttsLanguageModelSrc\":{},\"ttsTextEncoderSrc\":{},\"ttsDurationPredictorSrc\":{},\"ttsVectorEstimatorSrc\":{},\"ttsVocoderSrc\":{},\"ttsUnicodeIndexerSrc\":{},\"ttsTtsConfigSrc\":{},\"ttsVoiceStyleSrc\":{}}},{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"parler\"},\"description\":{\"type\":\"string\"},\"voiceDescription\":{\"type\":\"string\"},\"voice\":{\"type\":\"string\"},\"emotion\":{\"type\":\"string\",\"enum\":[\"command\",\"anger\",\"narration\",\"conversation\",\"disgust\",\"fear\",\"happy\",\"neutral\",\"proper noun\",\"news\",\"sad\",\"surprise\"]},\"pitch\":{\"type\":\"string\"},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"expressivity\":{\"type\":\"string\"},\"noise\":{\"type\":\"string\"},\"reverb\":{\"type\":\"string\"},\"quality\":{\"type\":\"string\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"maxFrames\":{\"anyOf\":[{\"type\":\"number\",\"const\":0},{\"type\":\"integer\"}]},\"minNewTokens\":{\"type\":\"integer\"},\"normalizeNumbers\":{\"type\":\"boolean\"},\"coremlDecoder\":{\"type\":\"boolean\"},\"coremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"all\",\"cpu_and_ane\",\"cpu_and_gpu\",\"cpu_only\"]},\"coremlStrict\":{\"type\":\"boolean\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrEnhancerCoremlModelPath\":{\"type\":\"string\"},\"lavasrEnhancerCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"lavasrDenoiserCoremlModelPath\":{\"type\":\"string\"},\"lavasrDenoiserCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]}}},{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"cosyvoice3\"},\"emotion\":{\"type\":\"string\",\"enum\":[\"anger\",\"happy\",\"neutral\",\"sad\"]},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"instruct\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\"}]},\"promptText\":{\"type\":\"string\"},\"flowCutPrompt\":{\"type\":\"boolean\"},\"vulkanDevice\":{\"type\":\"integer\"},\"greedy\":{\"type\":\"boolean\"},\"forceSpeechTokens\":{\"type\":\"array\",\"items\":{\"type\":\"integer\"}},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lavasrEnhancerCoremlModelPath\":{\"type\":\"string\"},\"lavasrEnhancerCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"lavasrDenoiserCoremlModelPath\":{\"type\":\"string\"},\"lavasrDenoiserCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"cosyvoice3S3tokModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"cosyvoice3CampplusModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"audio8CodecDecoderModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"audio8\"},\"referenceText\":{\"type\":\"string\"},\"greedy\":{\"type\":\"boolean\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"maxFrames\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"audio8CodecDecoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"audio8CodecEncoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"mossCodecDecoderModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss\"},\"language\":{\"type\":\"string\"},\"durationTokens\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"nCtx\":{\"type\":\"integer\"},\"maxNewTokens\":{\"type\":\"integer\"},\"textTemperature\":{\"type\":\"number\"},\"textTopP\":{\"type\":\"number\"},\"textTopK\":{\"type\":\"integer\"},\"audioTemperature\":{\"type\":\"number\"},\"audioTopP\":{\"type\":\"number\"},\"audioTopK\":{\"type\":\"integer\"},\"audioRepetitionPenalty\":{\"type\":\"number\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"mossCodecDecoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"mossCodecEncoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"dialogueReferenceSrcs\":{\"type\":\"array\",\"items\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}},{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss-sfx\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"seconds\":{\"type\":\"number\"},\"negativePrompt\":{\"type\":\"string\"},\"steps\":{\"type\":\"integer\"},\"guidance\":{\"type\":\"number\"},\"shift\":{\"type\":\"number\"}}},{\"type\":\"object\",\"required\":[\"ttsEngine\",\"mossSpeechCodecModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss-speech\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"mossSpeechCodecModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}]}}}").jsonObject, Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"ggml-ocr\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"langList\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}},\"pipelineType\":{\"type\":\"string\",\"enum\":[\"easyocr\",\"doctr\"]},\"magRatio\":{\"type\":\"number\"},\"canvasSize\":{\"type\":\"number\"},\"defaultRotationAngles\":{\"type\":\"array\",\"items\":{\"type\":\"number\"}},\"contrastRetry\":{\"type\":\"boolean\"},\"lowConfidenceThreshold\":{\"type\":\"number\"},\"recognizerBatchSize\":{\"type\":\"number\"},\"nThreads\":{\"type\":\"number\"},\"backendDevice\":{\"type\":\"string\",\"enum\":[\"cpu\",\"vulkan\",\"metal\",\"opencl\"]},\"gpuDevice\":{\"type\":\"number\"},\"mainGpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"detectorModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}}}").jsonObject, Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"sdcpp-generation\"},\"modelConfig\":{\"type\":\"object\",\"properties\":{\"mode\":{\"type\":\"string\",\"enum\":[\"diffusion\",\"upscale\",\"video\",\"world\"]},\"threads\":{\"type\":\"number\"},\"device\":{\"type\":\"string\",\"enum\":[\"gpu\",\"cpu\"]},\"main-gpu\":{\"anyOf\":[{\"type\":\"integer\"},{\"type\":\"string\",\"enum\":[\"integrated\",\"dedicated\"]}]},\"prediction\":{\"type\":\"string\",\"enum\":[\"auto\",\"eps\",\"v\",\"edm_v\",\"flow\",\"flux2_flow\"]},\"type\":{\"type\":\"string\",\"enum\":[\"auto\",\"f32\",\"f16\",\"bf16\",\"q2_k\",\"q3_k\",\"q4_0\",\"q4_1\",\"q4_k\",\"q5_0\",\"q5_1\",\"q5_k\",\"q6_k\",\"q8_0\"]},\"rng\":{\"type\":\"string\",\"enum\":[\"cpu\",\"cuda\",\"std_default\"]},\"sampler_rng\":{\"type\":\"string\",\"enum\":[\"cpu\",\"cuda\",\"std_default\"]},\"vae_auto_cpu_fallback\":{\"type\":\"boolean\"},\"vae_auto_cpu_fallback_memory_ratio\":{\"type\":\"number\"},\"vae_tiling\":{\"type\":\"boolean\"},\"offload_to_cpu\":{\"type\":\"boolean\"},\"control_net_cpu\":{\"not\":{}},\"clip_on_cpu\":{\"not\":{}},\"vae_on_cpu\":{\"not\":{}},\"backend\":{\"type\":\"string\"},\"params_backend\":{\"type\":\"string\"},\"max_vram\":{\"anyOf\":[{\"type\":\"number\"},{\"type\":\"string\"}]},\"stream_layers\":{\"type\":\"boolean\"},\"flash_attn\":{\"type\":\"boolean\"},\"diffusion_fa\":{\"type\":\"boolean\"},\"lora_apply_mode\":{\"type\":\"string\",\"enum\":[\"auto\",\"immediately\",\"at_runtime\"]},\"verbosity\":{\"type\":\"number\"},\"clipLModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"clipGModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"t5XxlModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"llmModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"projectionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"vaeModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"highNoiseDiffusionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"uncondModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"clipVisionModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"audioVaeModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"embeddingsConnectorsModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"taehvModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"sceneSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"world\":{\"type\":\"object\"},\"upscaler\":{\"type\":\"object\"}}}}}").jsonObject, Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\",\"modelSrc\",\"modelType\",\"modelConfig\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"loadModel\"},\"modelSrc\":{\"type\":\"string\"},\"modelName\":{\"type\":\"string\"},\"withProgress\":{\"type\":\"boolean\"},\"seed\":{\"type\":\"boolean\"},\"fallbackSrc\":{\"type\":\"string\"},\"requireHttpChecksum\":{\"type\":\"boolean\"},\"requireSecureTransport\":{\"type\":\"boolean\"},\"modelFitPolicy\":{\"type\":\"string\",\"enum\":[\"log\",\"refuse\",\"off\"]},\"requestId\":{\"type\":\"string\"},\"modelType\":{\"type\":\"string\",\"const\":\"audiogen-ggml\"},\"modelConfig\":{\"oneOf\":[{\"type\":\"object\",\"required\":[\"textEncModelSrc\",\"lmModelSrc\",\"ditModelSrc\",\"vaeModelSrc\"],\"properties\":{\"engine\":{\"type\":\"string\",\"const\":\"acestep\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"inferenceSteps\":{\"type\":\"integer\"},\"shift\":{\"type\":\"number\"},\"nGpuLayers\":{\"type\":\"integer\"},\"textEncModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"lmModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"ditModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"vaeModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}},{\"type\":\"object\",\"required\":[\"engine\",\"lmModelSrc\",\"synthModelSrc\"],\"properties\":{\"engine\":{\"type\":\"string\",\"const\":\"minimax\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"device\":{\"type\":\"string\",\"enum\":[\"cpu\",\"gpu\",\"auto\"]},\"coremlProfileDir\":{\"type\":\"string\"},\"coremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"inferenceSteps\":{\"type\":\"integer\"},\"cfgScale\":{\"type\":\"number\"},\"lmModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]},\"synthModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}]}}}").jsonObject, @@ -5895,14 +5895,14 @@ sealed class LoadModelSrcRequestTtsGgmlModelConfig { internal object LoadModelSrcRequestTtsGgmlModelConfigSerializer : KSerializer { override val descriptor = kotlinx.serialization.descriptors.buildClassSerialDescriptor("LoadModelSrcRequestTtsGgmlModelConfig") private val shapes = listOf( - Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"ttsEngine\",\"language\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"chatterbox\"},\"language\":{\"type\":\"string\",\"enum\":[\"en\",\"es\",\"fr\",\"de\",\"it\",\"ja\",\"pt\",\"nl\",\"pl\",\"tr\",\"sv\",\"da\",\"fi\",\"no\",\"el\",\"ms\",\"sw\",\"ar\",\"ko\",\"he\",\"ru\",\"zh\",\"hi\"]},\"voice\":{\"type\":\"string\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"ttsSpeed\":{\"type\":\"number\"},\"nCtx\":{\"type\":\"integer\"},\"kvCacheType\":{\"type\":\"string\",\"enum\":[\"f32\",\"f16\",\"q8_0\"]},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"cfmSteps\":{\"type\":\"integer\"},\"cfgRate\":{\"type\":\"number\"},\"streamLeftContextTokens\":{\"type\":\"integer\"},\"batchCfmSteps\":{\"type\":\"integer\"},\"nPredict\":{\"type\":\"integer\"},\"maxSentenceChars\":{\"type\":\"integer\"},\"crossfadeMs\":{\"type\":\"integer\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"repeatPenalty\":{\"type\":\"number\"},\"exaggeration\":{\"type\":\"number\"},\"cfgWeight\":{\"type\":\"number\"},\"minP\":{\"type\":\"number\"},\"voiceDir\":{\"type\":\"string\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"s3genModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"mecabDictSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"cangjieTsvSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"ttsSupertonicMultilingual\":{},\"ttsTokenizerSrc\":{},\"ttsSpeechEncoderSrc\":{},\"ttsEmbedTokensSrc\":{},\"ttsConditionalDecoderSrc\":{},\"ttsLanguageModelSrc\":{},\"ttsTextEncoderSrc\":{},\"ttsDurationPredictorSrc\":{},\"ttsVectorEstimatorSrc\":{},\"ttsVocoderSrc\":{},\"ttsUnicodeIndexerSrc\":{},\"ttsTtsConfigSrc\":{},\"ttsVoiceStyleSrc\":{}}}").jsonObject, - Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"ttsEngine\",\"language\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"supertonic\"},\"language\":{\"type\":\"string\",\"enum\":[\"en\",\"ko\",\"ja\",\"ar\",\"bg\",\"cs\",\"da\",\"de\",\"el\",\"es\",\"et\",\"fi\",\"fr\",\"hi\",\"hr\",\"hu\",\"id\",\"it\",\"lt\",\"lv\",\"nl\",\"pl\",\"pt\",\"ro\",\"ru\",\"sk\",\"sl\",\"sv\",\"tr\",\"uk\",\"vi\"]},\"voice\":{\"type\":\"string\"},\"ttsSpeed\":{\"type\":\"number\"},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"ttsNumInferenceSteps\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"vulkanCacheDir\":{\"type\":\"string\"},\"vulkanDevice\":{\"type\":\"integer\"},\"prewarmText\":{\"type\":\"string\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"streamChunkTolerancePct\":{\"type\":\"integer\"},\"streamMinChunkTokens\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"voiceJsonSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"noiseNpySrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"ttsSupertonicMultilingual\":{},\"ttsTokenizerSrc\":{},\"ttsSpeechEncoderSrc\":{},\"ttsEmbedTokensSrc\":{},\"ttsConditionalDecoderSrc\":{},\"ttsLanguageModelSrc\":{},\"ttsTextEncoderSrc\":{},\"ttsDurationPredictorSrc\":{},\"ttsVectorEstimatorSrc\":{},\"ttsVocoderSrc\":{},\"ttsUnicodeIndexerSrc\":{},\"ttsTtsConfigSrc\":{},\"ttsVoiceStyleSrc\":{}}}").jsonObject, - Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"parler\"},\"description\":{\"type\":\"string\"},\"voiceDescription\":{\"type\":\"string\"},\"voice\":{\"type\":\"string\"},\"emotion\":{\"type\":\"string\",\"enum\":[\"command\",\"anger\",\"narration\",\"conversation\",\"disgust\",\"fear\",\"happy\",\"neutral\",\"proper noun\",\"news\",\"sad\",\"surprise\"]},\"pitch\":{\"type\":\"string\"},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"expressivity\":{\"type\":\"string\"},\"noise\":{\"type\":\"string\"},\"reverb\":{\"type\":\"string\"},\"quality\":{\"type\":\"string\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"maxFrames\":{\"anyOf\":[{\"type\":\"number\",\"const\":0},{\"type\":\"integer\"}]},\"minNewTokens\":{\"type\":\"integer\"},\"normalizeNumbers\":{\"type\":\"boolean\"},\"backendsDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]}}}").jsonObject, - Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"cosyvoice3\"},\"emotion\":{\"type\":\"string\",\"enum\":[\"anger\",\"happy\",\"neutral\",\"sad\"]},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"instruct\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"properties\":{\"dialect\":{\"type\":\"string\",\"enum\":[\"cantonese\",\"northeastern\",\"gansu\",\"guizhou\",\"henan\",\"hubei\",\"hunan\",\"jiangxi\",\"minnan\",\"ningxia\",\"shanxi\",\"shaanxi\",\"shandong\",\"shanghai\",\"sichuan\",\"tianjin\",\"yunnan\"]},\"volume\":{\"type\":\"string\",\"enum\":[\"loud\",\"soft\"]},\"style\":{\"type\":\"string\",\"enum\":[\"peppa\",\"robot\"]}}}]},\"promptText\":{\"type\":\"string\"},\"flowCutPrompt\":{\"type\":\"boolean\"},\"vulkanDevice\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"cosyvoice3S3tokModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"cosyvoice3CampplusModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]}}}").jsonObject, - Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"ttsEngine\",\"audio8CodecDecoderModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"audio8\"},\"referenceText\":{\"type\":\"string\"},\"greedy\":{\"type\":\"boolean\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"maxFrames\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"audio8CodecDecoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"audio8CodecEncoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]}}}").jsonObject, - Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"ttsEngine\",\"mossCodecDecoderModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss\"},\"language\":{\"type\":\"string\"},\"durationTokens\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"mossCodecDecoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"mossCodecEncoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"dialogueReferenceSrcs\":{\"type\":\"array\",\"items\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}}").jsonObject, - Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss-sfx\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"seconds\":{\"type\":\"number\"},\"negativePrompt\":{\"type\":\"string\"},\"steps\":{\"type\":\"integer\"},\"guidance\":{\"type\":\"number\"},\"shift\":{\"type\":\"number\"}}}").jsonObject, - Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"ttsEngine\",\"mossSpeechCodecModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss-speech\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backendsDir\":{\"type\":\"string\"},\"mossSpeechCodecModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]}}}").jsonObject, + Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"ttsEngine\",\"language\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"chatterbox\"},\"language\":{\"type\":\"string\",\"enum\":[\"en\",\"es\",\"fr\",\"de\",\"it\",\"ja\",\"pt\",\"nl\",\"pl\",\"tr\",\"sv\",\"da\",\"fi\",\"no\",\"el\",\"ms\",\"sw\",\"ar\",\"ko\",\"he\",\"ru\",\"zh\",\"hi\"]},\"voice\":{\"type\":\"string\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"ttsSpeed\":{\"type\":\"number\"},\"nCtx\":{\"type\":\"integer\"},\"kvCacheType\":{\"type\":\"string\",\"enum\":[\"f32\",\"f16\",\"q8_0\"]},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"cfmSteps\":{\"type\":\"integer\"},\"cfgRate\":{\"type\":\"number\"},\"streamLeftContextTokens\":{\"type\":\"integer\"},\"batchCfmSteps\":{\"type\":\"integer\"},\"nPredict\":{\"type\":\"integer\"},\"maxSentenceChars\":{\"type\":\"integer\"},\"crossfadeMs\":{\"type\":\"integer\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"repeatPenalty\":{\"type\":\"number\"},\"exaggeration\":{\"type\":\"number\"},\"cfgWeight\":{\"type\":\"number\"},\"minP\":{\"type\":\"number\"},\"voiceDir\":{\"type\":\"string\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"s3genModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"mecabDictSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"cangjieTsvSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"lavasrEnhancerCoremlModelPath\":{\"type\":\"string\"},\"lavasrEnhancerCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"lavasrDenoiserCoremlModelPath\":{\"type\":\"string\"},\"lavasrDenoiserCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"ttsSupertonicMultilingual\":{},\"ttsTokenizerSrc\":{},\"ttsSpeechEncoderSrc\":{},\"ttsEmbedTokensSrc\":{},\"ttsConditionalDecoderSrc\":{},\"ttsLanguageModelSrc\":{},\"ttsTextEncoderSrc\":{},\"ttsDurationPredictorSrc\":{},\"ttsVectorEstimatorSrc\":{},\"ttsVocoderSrc\":{},\"ttsUnicodeIndexerSrc\":{},\"ttsTtsConfigSrc\":{},\"ttsVoiceStyleSrc\":{}}}").jsonObject, + Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"ttsEngine\",\"language\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"supertonic\"},\"language\":{\"type\":\"string\",\"enum\":[\"en\",\"ko\",\"ja\",\"ar\",\"bg\",\"cs\",\"da\",\"de\",\"el\",\"es\",\"et\",\"fi\",\"fr\",\"hi\",\"hr\",\"hu\",\"id\",\"it\",\"lt\",\"lv\",\"nl\",\"pl\",\"pt\",\"ro\",\"ru\",\"sk\",\"sl\",\"sv\",\"tr\",\"uk\",\"vi\"]},\"voice\":{\"type\":\"string\"},\"ttsSpeed\":{\"type\":\"number\"},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"ttsNumInferenceSteps\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"vulkanCacheDir\":{\"type\":\"string\"},\"vulkanDevice\":{\"type\":\"integer\"},\"prewarmText\":{\"type\":\"string\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"streamChunkTolerancePct\":{\"type\":\"integer\"},\"streamMinChunkTokens\":{\"type\":\"integer\"},\"precision\":{\"type\":\"string\",\"enum\":[\"auto\",\"f32\",\"f16\",\"q8_0\"]},\"f16Attn\":{\"type\":\"integer\"},\"f16Weights\":{\"type\":\"integer\"},\"kvAttnType\":{\"type\":\"integer\"},\"f16WeightsDenyList\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}},\"vulkanEnvOverrides\":{\"type\":\"object\"},\"voiceStyleTtl\":{\"type\":\"array\",\"items\":{\"type\":\"number\"}},\"voiceStyleDp\":{\"type\":\"array\",\"items\":{\"type\":\"number\"}},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"lavasrEnhancerCoremlModelPath\":{\"type\":\"string\"},\"lavasrEnhancerCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"lavasrDenoiserCoremlModelPath\":{\"type\":\"string\"},\"lavasrDenoiserCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"voiceJsonSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"noiseNpySrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"ttsSupertonicMultilingual\":{},\"ttsTokenizerSrc\":{},\"ttsSpeechEncoderSrc\":{},\"ttsEmbedTokensSrc\":{},\"ttsConditionalDecoderSrc\":{},\"ttsLanguageModelSrc\":{},\"ttsTextEncoderSrc\":{},\"ttsDurationPredictorSrc\":{},\"ttsVectorEstimatorSrc\":{},\"ttsVocoderSrc\":{},\"ttsUnicodeIndexerSrc\":{},\"ttsTtsConfigSrc\":{},\"ttsVoiceStyleSrc\":{}}}").jsonObject, + Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"parler\"},\"description\":{\"type\":\"string\"},\"voiceDescription\":{\"type\":\"string\"},\"voice\":{\"type\":\"string\"},\"emotion\":{\"type\":\"string\",\"enum\":[\"command\",\"anger\",\"narration\",\"conversation\",\"disgust\",\"fear\",\"happy\",\"neutral\",\"proper noun\",\"news\",\"sad\",\"surprise\"]},\"pitch\":{\"type\":\"string\"},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"expressivity\":{\"type\":\"string\"},\"noise\":{\"type\":\"string\"},\"reverb\":{\"type\":\"string\"},\"quality\":{\"type\":\"string\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"maxFrames\":{\"anyOf\":[{\"type\":\"number\",\"const\":0},{\"type\":\"integer\"}]},\"minNewTokens\":{\"type\":\"integer\"},\"normalizeNumbers\":{\"type\":\"boolean\"},\"coremlDecoder\":{\"type\":\"boolean\"},\"coremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"all\",\"cpu_and_ane\",\"cpu_and_gpu\",\"cpu_only\"]},\"coremlStrict\":{\"type\":\"boolean\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"lavasrEnhancerCoremlModelPath\":{\"type\":\"string\"},\"lavasrEnhancerCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"lavasrDenoiserCoremlModelPath\":{\"type\":\"string\"},\"lavasrDenoiserCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]}}}").jsonObject, + Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"cosyvoice3\"},\"emotion\":{\"type\":\"string\",\"enum\":[\"anger\",\"happy\",\"neutral\",\"sad\"]},\"pace\":{\"type\":\"string\",\"enum\":[\"slow\",\"moderate\",\"fast\"]},\"instruct\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"properties\":{\"dialect\":{\"type\":\"string\",\"enum\":[\"cantonese\",\"northeastern\",\"gansu\",\"guizhou\",\"henan\",\"hubei\",\"hunan\",\"jiangxi\",\"minnan\",\"ningxia\",\"shanxi\",\"shaanxi\",\"shandong\",\"shanghai\",\"sichuan\",\"tianjin\",\"yunnan\"]},\"volume\":{\"type\":\"string\",\"enum\":[\"loud\",\"soft\"]},\"style\":{\"type\":\"string\",\"enum\":[\"peppa\",\"robot\"]}}}]},\"promptText\":{\"type\":\"string\"},\"flowCutPrompt\":{\"type\":\"boolean\"},\"vulkanDevice\":{\"type\":\"integer\"},\"greedy\":{\"type\":\"boolean\"},\"forceSpeechTokens\":{\"type\":\"array\",\"items\":{\"type\":\"integer\"}},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"streamFirstChunkTokens\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"openclCacheDir\":{\"type\":\"string\"},\"lavasrEnhancerModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"lavasrDenoiserModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"lavasrEnhancerCoremlModelPath\":{\"type\":\"string\"},\"lavasrEnhancerCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"lavasrDenoiserCoremlModelPath\":{\"type\":\"string\"},\"lavasrDenoiserCoremlComputeUnits\":{\"type\":\"string\",\"enum\":[\"CPU_ONLY\",\"CPU_AND_GPU\",\"CPU_AND_NE\",\"ALL\"]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"cosyvoice3S3tokModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"cosyvoice3CampplusModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]}}}").jsonObject, + Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"ttsEngine\",\"audio8CodecDecoderModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"audio8\"},\"referenceText\":{\"type\":\"string\"},\"greedy\":{\"type\":\"boolean\"},\"temperature\":{\"type\":\"number\"},\"topK\":{\"type\":\"integer\"},\"topP\":{\"type\":\"number\"},\"maxFrames\":{\"type\":\"integer\"},\"useGPU\":{\"type\":\"boolean\"},\"outputSampleRate\":{\"type\":\"integer\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"audio8CodecDecoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"audio8CodecEncoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]}}}").jsonObject, + Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"ttsEngine\",\"mossCodecDecoderModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss\"},\"language\":{\"type\":\"string\"},\"durationTokens\":{\"type\":\"integer\"},\"streamChunkTokens\":{\"type\":\"integer\"},\"nCtx\":{\"type\":\"integer\"},\"maxNewTokens\":{\"type\":\"integer\"},\"textTemperature\":{\"type\":\"number\"},\"textTopP\":{\"type\":\"number\"},\"textTopK\":{\"type\":\"integer\"},\"audioTemperature\":{\"type\":\"number\"},\"audioTopP\":{\"type\":\"number\"},\"audioTopK\":{\"type\":\"integer\"},\"audioRepetitionPenalty\":{\"type\":\"number\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"mossCodecDecoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"mossCodecEncoderModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"referenceAudioSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]},\"dialogueReferenceSrcs\":{\"type\":\"array\",\"items\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"]}]}}}}").jsonObject, + Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"ttsEngine\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss-sfx\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"seconds\":{\"type\":\"number\"},\"negativePrompt\":{\"type\":\"string\"},\"steps\":{\"type\":\"integer\"},\"guidance\":{\"type\":\"number\"},\"shift\":{\"type\":\"number\"}}}").jsonObject, + Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"ttsEngine\",\"mossSpeechCodecModelSrc\"],\"properties\":{\"ttsEngine\":{\"type\":\"string\",\"const\":\"moss-speech\"},\"useGPU\":{\"type\":\"boolean\"},\"threads\":{\"type\":\"integer\"},\"nGpuLayers\":{\"type\":\"integer\"},\"seed\":{\"type\":\"integer\"},\"backend\":{\"type\":\"string\"},\"backendsDir\":{\"type\":\"string\"},\"mossSpeechCodecModelSrc\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"object\",\"required\":[\"src\"],\"properties\":{\"src\":{\"type\":\"string\"},\"name\":{\"type\":\"string\"},\"modelId\":{\"type\":\"string\"},\"registryPath\":{\"type\":\"string\"},\"registrySource\":{\"type\":\"string\"},\"blobCoreKey\":{\"type\":\"string\"},\"blobIndex\":{\"type\":\"number\"},\"engine\":{\"type\":\"string\"},\"expectedSize\":{\"type\":\"number\"},\"sha256Checksum\":{\"type\":\"string\"},\"addon\":{\"anyOf\":[{\"type\":\"string\",\"enum\":[\"llamacpp-completion\",\"whispercpp-transcription\",\"bci-whispercpp-transcription\",\"llamacpp-embedding\",\"llamacpp-decision\",\"llamacpp-rerank\",\"nmtcpp-translation\",\"onnx-tts\",\"tts-ggml\",\"parakeet-transcription\",\"moss-transcribe\",\"ggml-ocr\",\"sdcpp-generation\",\"audiogen-ggml\",\"ggml-vla\",\"ggml-image-embedding\",\"ggml-classification\",\"llm\",\"whisper\",\"bci\",\"embeddings\",\"nmt\",\"parakeet\",\"tts\",\"ocr\",\"diffusion\",\"audiogen\",\"vla\",\"classification\"]},{\"type\":\"string\",\"const\":\"vad\"}]}}}]}}}").jsonObject, ) override fun deserialize(decoder: Decoder): LoadModelSrcRequestTtsGgmlModelConfig { val input = decoder as? JsonDecoder ?: throw SerializationException("QVAC unions require JSON") @@ -5964,6 +5964,7 @@ data class LoadModelSrcRequestTtsGgmlModelConfigChatterbox( @SerialName("threads") val `threads`: Long? = null, @SerialName("nGpuLayers") val `nGpuLayers`: Long? = null, @SerialName("seed") val `seed`: Long? = null, + @SerialName("backend") val `backend`: String? = null, @SerialName("backendsDir") val `backendsDir`: String? = null, @SerialName("openclCacheDir") val `openclCacheDir`: String? = null, @SerialName("s3genModelSrc") val `s3genModelSrc`: LoadModelSrcRequestTtsGgmlModelConfigChatterboxS3genModelSrc? = null, @@ -5972,6 +5973,10 @@ data class LoadModelSrcRequestTtsGgmlModelConfigChatterbox( @SerialName("cangjieTsvSrc") val `cangjieTsvSrc`: LoadModelSrcRequestTtsGgmlModelConfigChatterboxCangjieTsvSrc? = null, @SerialName("lavasrEnhancerModelSrc") val `lavasrEnhancerModelSrc`: LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrEnhancerModelSrc? = null, @SerialName("lavasrDenoiserModelSrc") val `lavasrDenoiserModelSrc`: LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrDenoiserModelSrc? = null, + @SerialName("lavasrEnhancerCoremlModelPath") val `lavasrEnhancerCoremlModelPath`: String? = null, + @SerialName("lavasrEnhancerCoremlComputeUnits") val `lavasrEnhancerCoremlComputeUnits`: LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrEnhancerCoremlComputeUnits? = null, + @SerialName("lavasrDenoiserCoremlModelPath") val `lavasrDenoiserCoremlModelPath`: String? = null, + @SerialName("lavasrDenoiserCoremlComputeUnits") val `lavasrDenoiserCoremlComputeUnits`: LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrDenoiserCoremlComputeUnits? = null, @SerialName("ttsSupertonicMultilingual") val `ttsSupertonicMultilingual`: JsonElement? = null, @SerialName("ttsTokenizerSrc") val `ttsTokenizerSrc`: JsonElement? = null, @SerialName("ttsSpeechEncoderSrc") val `ttsSpeechEncoderSrc`: JsonElement? = null, @@ -6645,6 +6650,22 @@ enum class LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrDenoiserModelSrc @SerialName("classification") `CLASSIFICATION`, } +@Serializable +enum class LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrEnhancerCoremlComputeUnits { + @SerialName("CPU_ONLY") `CPU_ONLY`, + @SerialName("CPU_AND_GPU") `CPU_AND_GPU`, + @SerialName("CPU_AND_NE") `CPU_AND_NE`, + @SerialName("ALL") `ALL`, +} + +@Serializable +enum class LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrDenoiserCoremlComputeUnits { + @SerialName("CPU_ONLY") `CPU_ONLY`, + @SerialName("CPU_AND_GPU") `CPU_AND_GPU`, + @SerialName("CPU_AND_NE") `CPU_AND_NE`, + @SerialName("ALL") `ALL`, +} + @Serializable data class LoadModelSrcRequestTtsGgmlModelConfigSupertonic( @EncodeDefault(EncodeDefault.Mode.ALWAYS) @@ -6666,10 +6687,23 @@ data class LoadModelSrcRequestTtsGgmlModelConfigSupertonic( @SerialName("streamFirstChunkTokens") val `streamFirstChunkTokens`: Long? = null, @SerialName("streamChunkTolerancePct") val `streamChunkTolerancePct`: Long? = null, @SerialName("streamMinChunkTokens") val `streamMinChunkTokens`: Long? = null, + @SerialName("precision") val `precision`: LoadModelSrcRequestTtsGgmlModelConfigSupertonicPrecision? = null, + @SerialName("f16Attn") val `f16Attn`: Long? = null, + @SerialName("f16Weights") val `f16Weights`: Long? = null, + @SerialName("kvAttnType") val `kvAttnType`: Long? = null, + @SerialName("f16WeightsDenyList") val `f16WeightsDenyList`: List? = null, + @SerialName("vulkanEnvOverrides") val `vulkanEnvOverrides`: Map? = null, + @SerialName("voiceStyleTtl") val `voiceStyleTtl`: List? = null, + @SerialName("voiceStyleDp") val `voiceStyleDp`: List? = null, + @SerialName("backend") val `backend`: String? = null, @SerialName("backendsDir") val `backendsDir`: String? = null, @SerialName("openclCacheDir") val `openclCacheDir`: String? = null, @SerialName("lavasrEnhancerModelSrc") val `lavasrEnhancerModelSrc`: LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrEnhancerModelSrc? = null, @SerialName("lavasrDenoiserModelSrc") val `lavasrDenoiserModelSrc`: LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrDenoiserModelSrc? = null, + @SerialName("lavasrEnhancerCoremlModelPath") val `lavasrEnhancerCoremlModelPath`: String? = null, + @SerialName("lavasrEnhancerCoremlComputeUnits") val `lavasrEnhancerCoremlComputeUnits`: LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrEnhancerCoremlComputeUnits? = null, + @SerialName("lavasrDenoiserCoremlModelPath") val `lavasrDenoiserCoremlModelPath`: String? = null, + @SerialName("lavasrDenoiserCoremlComputeUnits") val `lavasrDenoiserCoremlComputeUnits`: LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrDenoiserCoremlComputeUnits? = null, @SerialName("voiceJsonSrc") val `voiceJsonSrc`: LoadModelSrcRequestTtsGgmlModelConfigSupertonicVoiceJsonSrc? = null, @SerialName("noiseNpySrc") val `noiseNpySrc`: LoadModelSrcRequestTtsGgmlModelConfigSupertonicNoiseNpySrc? = null, @SerialName("ttsSupertonicMultilingual") val `ttsSupertonicMultilingual`: JsonElement? = null, @@ -6729,6 +6763,14 @@ enum class LoadModelSrcRequestTtsGgmlModelConfigSupertonicPace { @SerialName("fast") `FAST`, } +@Serializable +enum class LoadModelSrcRequestTtsGgmlModelConfigSupertonicPrecision { + @SerialName("auto") `AUTO`, + @SerialName("f32") `F32`, + @SerialName("f16") `F16`, + @SerialName("q8_0") `Q8_0`, +} + @Serializable(with = LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrEnhancerModelSrcSerializer::class) sealed class LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrEnhancerModelSrc { data class Variant1(val value: String) : LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrEnhancerModelSrc() @@ -6937,6 +6979,22 @@ enum class LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrDenoiserModelSrc @SerialName("classification") `CLASSIFICATION`, } +@Serializable +enum class LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrEnhancerCoremlComputeUnits { + @SerialName("CPU_ONLY") `CPU_ONLY`, + @SerialName("CPU_AND_GPU") `CPU_AND_GPU`, + @SerialName("CPU_AND_NE") `CPU_AND_NE`, + @SerialName("ALL") `ALL`, +} + +@Serializable +enum class LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrDenoiserCoremlComputeUnits { + @SerialName("CPU_ONLY") `CPU_ONLY`, + @SerialName("CPU_AND_GPU") `CPU_AND_GPU`, + @SerialName("CPU_AND_NE") `CPU_AND_NE`, + @SerialName("ALL") `ALL`, +} + @Serializable(with = LoadModelSrcRequestTtsGgmlModelConfigSupertonicVoiceJsonSrcSerializer::class) sealed class LoadModelSrcRequestTtsGgmlModelConfigSupertonicVoiceJsonSrc { data class Variant1(val value: String) : LoadModelSrcRequestTtsGgmlModelConfigSupertonicVoiceJsonSrc() @@ -7172,9 +7230,17 @@ data class LoadModelSrcRequestTtsGgmlModelConfigParler( @SerialName("maxFrames") val `maxFrames`: LoadModelSrcRequestTtsGgmlModelConfigParlerMaxFrames? = null, @SerialName("minNewTokens") val `minNewTokens`: Long? = null, @SerialName("normalizeNumbers") val `normalizeNumbers`: Boolean? = null, + @SerialName("coremlDecoder") val `coremlDecoder`: Boolean? = null, + @SerialName("coremlComputeUnits") val `coremlComputeUnits`: LoadModelSrcRequestTtsGgmlModelConfigParlerCoremlComputeUnits? = null, + @SerialName("coremlStrict") val `coremlStrict`: Boolean? = null, + @SerialName("backend") val `backend`: String? = null, @SerialName("backendsDir") val `backendsDir`: String? = null, @SerialName("lavasrEnhancerModelSrc") val `lavasrEnhancerModelSrc`: LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrEnhancerModelSrc? = null, @SerialName("lavasrDenoiserModelSrc") val `lavasrDenoiserModelSrc`: LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrDenoiserModelSrc? = null, + @SerialName("lavasrEnhancerCoremlModelPath") val `lavasrEnhancerCoremlModelPath`: String? = null, + @SerialName("lavasrEnhancerCoremlComputeUnits") val `lavasrEnhancerCoremlComputeUnits`: LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrEnhancerCoremlComputeUnits? = null, + @SerialName("lavasrDenoiserCoremlModelPath") val `lavasrDenoiserCoremlModelPath`: String? = null, + @SerialName("lavasrDenoiserCoremlComputeUnits") val `lavasrDenoiserCoremlComputeUnits`: LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrDenoiserCoremlComputeUnits? = null, ) @Serializable @@ -7228,6 +7294,14 @@ internal object LoadModelSrcRequestTtsGgmlModelConfigParlerMaxFramesSerializer : } } +@Serializable +enum class LoadModelSrcRequestTtsGgmlModelConfigParlerCoremlComputeUnits { + @SerialName("all") `ALL`, + @SerialName("cpu_and_ane") `CPU_AND_ANE`, + @SerialName("cpu_and_gpu") `CPU_AND_GPU`, + @SerialName("cpu_only") `CPU_ONLY`, +} + @Serializable(with = LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrEnhancerModelSrcSerializer::class) sealed class LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrEnhancerModelSrc { data class Variant1(val value: String) : LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrEnhancerModelSrc() @@ -7436,6 +7510,22 @@ enum class LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrDenoiserModelSrcAddo @SerialName("classification") `CLASSIFICATION`, } +@Serializable +enum class LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrEnhancerCoremlComputeUnits { + @SerialName("CPU_ONLY") `CPU_ONLY`, + @SerialName("CPU_AND_GPU") `CPU_AND_GPU`, + @SerialName("CPU_AND_NE") `CPU_AND_NE`, + @SerialName("ALL") `ALL`, +} + +@Serializable +enum class LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrDenoiserCoremlComputeUnits { + @SerialName("CPU_ONLY") `CPU_ONLY`, + @SerialName("CPU_AND_GPU") `CPU_AND_GPU`, + @SerialName("CPU_AND_NE") `CPU_AND_NE`, + @SerialName("ALL") `ALL`, +} + @Serializable data class LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3( @EncodeDefault(EncodeDefault.Mode.ALWAYS) @@ -7446,6 +7536,8 @@ data class LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3( @SerialName("promptText") val `promptText`: String? = null, @SerialName("flowCutPrompt") val `flowCutPrompt`: Boolean? = null, @SerialName("vulkanDevice") val `vulkanDevice`: Long? = null, + @SerialName("greedy") val `greedy`: Boolean? = null, + @SerialName("forceSpeechTokens") val `forceSpeechTokens`: List? = null, @SerialName("useGPU") val `useGPU`: Boolean? = null, @SerialName("outputSampleRate") val `outputSampleRate`: Long? = null, @SerialName("streamChunkTokens") val `streamChunkTokens`: Long? = null, @@ -7453,10 +7545,15 @@ data class LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3( @SerialName("threads") val `threads`: Long? = null, @SerialName("nGpuLayers") val `nGpuLayers`: Long? = null, @SerialName("seed") val `seed`: Long? = null, + @SerialName("backend") val `backend`: String? = null, @SerialName("backendsDir") val `backendsDir`: String? = null, @SerialName("openclCacheDir") val `openclCacheDir`: String? = null, @SerialName("lavasrEnhancerModelSrc") val `lavasrEnhancerModelSrc`: LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrEnhancerModelSrc? = null, @SerialName("lavasrDenoiserModelSrc") val `lavasrDenoiserModelSrc`: LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrDenoiserModelSrc? = null, + @SerialName("lavasrEnhancerCoremlModelPath") val `lavasrEnhancerCoremlModelPath`: String? = null, + @SerialName("lavasrEnhancerCoremlComputeUnits") val `lavasrEnhancerCoremlComputeUnits`: LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrEnhancerCoremlComputeUnits? = null, + @SerialName("lavasrDenoiserCoremlModelPath") val `lavasrDenoiserCoremlModelPath`: String? = null, + @SerialName("lavasrDenoiserCoremlComputeUnits") val `lavasrDenoiserCoremlComputeUnits`: LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrDenoiserCoremlComputeUnits? = null, @SerialName("referenceAudioSrc") val `referenceAudioSrc`: LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3ReferenceAudioSrc? = null, @SerialName("cosyvoice3S3tokModelSrc") val `cosyvoice3S3tokModelSrc`: LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3Cosyvoice3S3tokModelSrc? = null, @SerialName("cosyvoice3CampplusModelSrc") val `cosyvoice3CampplusModelSrc`: LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3Cosyvoice3CampplusModelSrc? = null, @@ -7753,6 +7850,22 @@ enum class LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrDenoiserModelSrc @SerialName("classification") `CLASSIFICATION`, } +@Serializable +enum class LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrEnhancerCoremlComputeUnits { + @SerialName("CPU_ONLY") `CPU_ONLY`, + @SerialName("CPU_AND_GPU") `CPU_AND_GPU`, + @SerialName("CPU_AND_NE") `CPU_AND_NE`, + @SerialName("ALL") `ALL`, +} + +@Serializable +enum class LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrDenoiserCoremlComputeUnits { + @SerialName("CPU_ONLY") `CPU_ONLY`, + @SerialName("CPU_AND_GPU") `CPU_AND_GPU`, + @SerialName("CPU_AND_NE") `CPU_AND_NE`, + @SerialName("ALL") `ALL`, +} + @Serializable(with = LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3ReferenceAudioSrcSerializer::class) sealed class LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3ReferenceAudioSrc { data class Variant1(val value: String) : LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3ReferenceAudioSrc() @@ -8080,6 +8193,7 @@ data class LoadModelSrcRequestTtsGgmlModelConfigAudio8( @SerialName("threads") val `threads`: Long? = null, @SerialName("nGpuLayers") val `nGpuLayers`: Long? = null, @SerialName("seed") val `seed`: Long? = null, + @SerialName("backend") val `backend`: String? = null, @SerialName("backendsDir") val `backendsDir`: String? = null, @SerialName("audio8CodecDecoderModelSrc") val `audio8CodecDecoderModelSrc`: LoadModelSrcRequestTtsGgmlModelConfigAudio8Audio8CodecDecoderModelSrc, @SerialName("audio8CodecEncoderModelSrc") val `audio8CodecEncoderModelSrc`: LoadModelSrcRequestTtsGgmlModelConfigAudio8Audio8CodecEncoderModelSrc? = null, @@ -8405,10 +8519,20 @@ data class LoadModelSrcRequestTtsGgmlModelConfigMoss( @SerialName("language") val `language`: String? = null, @SerialName("durationTokens") val `durationTokens`: Long? = null, @SerialName("streamChunkTokens") val `streamChunkTokens`: Long? = null, + @SerialName("nCtx") val `nCtx`: Long? = null, + @SerialName("maxNewTokens") val `maxNewTokens`: Long? = null, + @SerialName("textTemperature") val `textTemperature`: Double? = null, + @SerialName("textTopP") val `textTopP`: Double? = null, + @SerialName("textTopK") val `textTopK`: Long? = null, + @SerialName("audioTemperature") val `audioTemperature`: Double? = null, + @SerialName("audioTopP") val `audioTopP`: Double? = null, + @SerialName("audioTopK") val `audioTopK`: Long? = null, + @SerialName("audioRepetitionPenalty") val `audioRepetitionPenalty`: Double? = null, @SerialName("useGPU") val `useGPU`: Boolean? = null, @SerialName("threads") val `threads`: Long? = null, @SerialName("nGpuLayers") val `nGpuLayers`: Long? = null, @SerialName("seed") val `seed`: Long? = null, + @SerialName("backend") val `backend`: String? = null, @SerialName("backendsDir") val `backendsDir`: String? = null, @SerialName("mossCodecDecoderModelSrc") val `mossCodecDecoderModelSrc`: LoadModelSrcRequestTtsGgmlModelConfigMossMossCodecDecoderModelSrc, @SerialName("mossCodecEncoderModelSrc") val `mossCodecEncoderModelSrc`: LoadModelSrcRequestTtsGgmlModelConfigMossMossCodecEncoderModelSrc? = null, @@ -8840,6 +8964,7 @@ data class LoadModelSrcRequestTtsGgmlModelConfigMossSfx( @SerialName("threads") val `threads`: Long? = null, @SerialName("nGpuLayers") val `nGpuLayers`: Long? = null, @SerialName("seed") val `seed`: Long? = null, + @SerialName("backend") val `backend`: String? = null, @SerialName("backendsDir") val `backendsDir`: String? = null, @SerialName("seconds") val `seconds`: Double? = null, @SerialName("negativePrompt") val `negativePrompt`: String? = null, @@ -8856,6 +8981,7 @@ data class LoadModelSrcRequestTtsGgmlModelConfigMossSpeech( @SerialName("threads") val `threads`: Long? = null, @SerialName("nGpuLayers") val `nGpuLayers`: Long? = null, @SerialName("seed") val `seed`: Long? = null, + @SerialName("backend") val `backend`: String? = null, @SerialName("backendsDir") val `backendsDir`: String? = null, @SerialName("mossSpeechCodecModelSrc") val `mossSpeechCodecModelSrc`: LoadModelSrcRequestTtsGgmlModelConfigMossSpeechMossSpeechCodecModelSrc, ) @@ -13501,8 +13627,8 @@ internal object AnyResponseSerializer : KSerializer { Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\",\"state\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"state\"},\"state\":{\"type\":\"string\",\"enum\":[\"active\",\"suspending\",\"suspended\",\"resuming\"]}}}").jsonObject, Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"stopRpcServer\"}}}").jsonObject, Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"suspend\"}}}").jsonObject, - Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\",\"buffer\",\"done\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"textToSpeech\"},\"text\":{\"type\":\"string\"},\"buffer\":{\"type\":\"array\",\"items\":{\"type\":\"number\"}},\"pcm\":{\"type\":\"string\"},\"done\":{\"type\":\"boolean\"},\"sampleRate\":{\"type\":\"integer\"},\"stats\":{\"type\":\"object\",\"properties\":{\"audioDuration\":{\"type\":\"number\"},\"totalTime\":{\"type\":\"number\"},\"realTimeFactor\":{\"type\":\"number\"},\"firstAudioMs\":{\"type\":\"number\"},\"tokensPerSecond\":{\"type\":\"number\"},\"totalSamples\":{\"type\":\"number\"},\"generatedFrames\":{\"type\":\"number\"},\"backendDevice\":{\"type\":\"number\"},\"backendId\":{\"type\":\"number\"},\"gpuUnsupported\":{\"type\":\"number\"},\"enhancerBackendDevice\":{\"type\":\"number\"},\"enhancerBackendId\":{\"type\":\"number\"},\"denoiserBackendDevice\":{\"type\":\"number\"},\"denoiserBackendId\":{\"type\":\"number\"},\"codecSidecarLoaded\":{\"type\":\"number\"},\"codecOnCoreml\":{\"type\":\"number\"},\"t3Ms\":{\"type\":\"number\"},\"s3genMs\":{\"type\":\"number\"},\"t3Tokens\":{\"type\":\"number\"},\"stageTotalMs\":{\"type\":\"number\"},\"lmPrefillMs\":{\"type\":\"number\"},\"lmDecodeMs\":{\"type\":\"number\"},\"flowFrontendMs\":{\"type\":\"number\"},\"ditEulerMs\":{\"type\":\"number\"},\"hiftF0Ms\":{\"type\":\"number\"},\"hiftSourceMs\":{\"type\":\"number\"},\"hiftStftMs\":{\"type\":\"number\"},\"hiftDecodeMs\":{\"type\":\"number\"},\"decodeSteps\":{\"type\":\"number\"},\"speechTokens\":{\"type\":\"number\"},\"flowFrames\":{\"type\":\"number\"},\"melFrames\":{\"type\":\"number\"},\"textIds\":{\"type\":\"number\"},\"promptSpeechTokens\":{\"type\":\"number\"},\"voiceEncodeMs\":{\"type\":\"number\"},\"promptMs\":{\"type\":\"number\"},\"prefillMs\":{\"type\":\"number\"},\"sampleMs\":{\"type\":\"number\"},\"fastDecodeMs\":{\"type\":\"number\"},\"slowDecodeMs\":{\"type\":\"number\"},\"codecLatentMs\":{\"type\":\"number\"},\"codecSynthMs\":{\"type\":\"number\"},\"resampleMs\":{\"type\":\"number\"}}},\"chunkIndex\":{\"type\":\"integer\"},\"sentenceChunk\":{\"type\":\"string\"},\"isLast\":{\"type\":\"boolean\"},\"stopReason\":{\"type\":\"string\",\"enum\":[\"completed\",\"cancelled\"]}}}").jsonObject, - Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\",\"buffer\",\"done\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"textToSpeechStream\"},\"buffer\":{\"type\":\"array\",\"items\":{\"type\":\"number\"}},\"pcm\":{\"type\":\"string\"},\"done\":{\"type\":\"boolean\"},\"sampleRate\":{\"type\":\"integer\"},\"stats\":{\"type\":\"object\",\"properties\":{\"audioDuration\":{\"type\":\"number\"},\"totalTime\":{\"type\":\"number\"},\"realTimeFactor\":{\"type\":\"number\"},\"firstAudioMs\":{\"type\":\"number\"},\"tokensPerSecond\":{\"type\":\"number\"},\"totalSamples\":{\"type\":\"number\"},\"generatedFrames\":{\"type\":\"number\"},\"backendDevice\":{\"type\":\"number\"},\"backendId\":{\"type\":\"number\"},\"gpuUnsupported\":{\"type\":\"number\"},\"enhancerBackendDevice\":{\"type\":\"number\"},\"enhancerBackendId\":{\"type\":\"number\"},\"denoiserBackendDevice\":{\"type\":\"number\"},\"denoiserBackendId\":{\"type\":\"number\"},\"codecSidecarLoaded\":{\"type\":\"number\"},\"codecOnCoreml\":{\"type\":\"number\"},\"t3Ms\":{\"type\":\"number\"},\"s3genMs\":{\"type\":\"number\"},\"t3Tokens\":{\"type\":\"number\"},\"stageTotalMs\":{\"type\":\"number\"},\"lmPrefillMs\":{\"type\":\"number\"},\"lmDecodeMs\":{\"type\":\"number\"},\"flowFrontendMs\":{\"type\":\"number\"},\"ditEulerMs\":{\"type\":\"number\"},\"hiftF0Ms\":{\"type\":\"number\"},\"hiftSourceMs\":{\"type\":\"number\"},\"hiftStftMs\":{\"type\":\"number\"},\"hiftDecodeMs\":{\"type\":\"number\"},\"decodeSteps\":{\"type\":\"number\"},\"speechTokens\":{\"type\":\"number\"},\"flowFrames\":{\"type\":\"number\"},\"melFrames\":{\"type\":\"number\"},\"textIds\":{\"type\":\"number\"},\"promptSpeechTokens\":{\"type\":\"number\"},\"voiceEncodeMs\":{\"type\":\"number\"},\"promptMs\":{\"type\":\"number\"},\"prefillMs\":{\"type\":\"number\"},\"sampleMs\":{\"type\":\"number\"},\"fastDecodeMs\":{\"type\":\"number\"},\"slowDecodeMs\":{\"type\":\"number\"},\"codecLatentMs\":{\"type\":\"number\"},\"codecSynthMs\":{\"type\":\"number\"},\"resampleMs\":{\"type\":\"number\"}}},\"chunkIndex\":{\"type\":\"integer\"},\"sentenceChunk\":{\"type\":\"string\"},\"isLast\":{\"type\":\"boolean\"},\"stopReason\":{\"type\":\"string\",\"enum\":[\"completed\",\"cancelled\"]}}}").jsonObject, + Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\",\"buffer\",\"done\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"textToSpeech\"},\"text\":{\"type\":\"string\"},\"buffer\":{\"type\":\"array\",\"items\":{\"type\":\"number\"}},\"pcm\":{\"type\":\"string\"},\"done\":{\"type\":\"boolean\"},\"sampleRate\":{\"type\":\"integer\"},\"stats\":{\"type\":\"object\",\"properties\":{\"audioDuration\":{\"type\":\"number\"},\"totalTime\":{\"type\":\"number\"},\"realTimeFactor\":{\"type\":\"number\"},\"firstAudioMs\":{\"type\":\"number\"},\"tokensPerSecond\":{\"type\":\"number\"},\"totalSamples\":{\"type\":\"number\"},\"generatedFrames\":{\"type\":\"number\"},\"backendDevice\":{\"type\":\"number\"},\"backendId\":{\"type\":\"number\"},\"gpuUnsupported\":{\"type\":\"number\"},\"enhancerBackendDevice\":{\"type\":\"number\"},\"enhancerBackendId\":{\"type\":\"number\"},\"denoiserBackendDevice\":{\"type\":\"number\"},\"denoiserBackendId\":{\"type\":\"number\"},\"codecSidecarLoaded\":{\"type\":\"number\"},\"codecOnCoreml\":{\"type\":\"number\"},\"decoderSidecarLoaded\":{\"type\":\"number\"},\"dacSidecarLoaded\":{\"type\":\"number\"},\"decoderOnCoreml\":{\"type\":\"number\"},\"dacOnCoreml\":{\"type\":\"number\"},\"t3Ms\":{\"type\":\"number\"},\"s3genMs\":{\"type\":\"number\"},\"t3Tokens\":{\"type\":\"number\"},\"stageTotalMs\":{\"type\":\"number\"},\"lmPrefillMs\":{\"type\":\"number\"},\"lmDecodeMs\":{\"type\":\"number\"},\"flowFrontendMs\":{\"type\":\"number\"},\"ditEulerMs\":{\"type\":\"number\"},\"hiftF0Ms\":{\"type\":\"number\"},\"hiftSourceMs\":{\"type\":\"number\"},\"hiftStftMs\":{\"type\":\"number\"},\"hiftDecodeMs\":{\"type\":\"number\"},\"decodeSteps\":{\"type\":\"number\"},\"speechTokens\":{\"type\":\"number\"},\"flowFrames\":{\"type\":\"number\"},\"melFrames\":{\"type\":\"number\"},\"textIds\":{\"type\":\"number\"},\"promptSpeechTokens\":{\"type\":\"number\"},\"voiceEncodeMs\":{\"type\":\"number\"},\"promptMs\":{\"type\":\"number\"},\"prefillMs\":{\"type\":\"number\"},\"sampleMs\":{\"type\":\"number\"},\"fastDecodeMs\":{\"type\":\"number\"},\"slowDecodeMs\":{\"type\":\"number\"},\"codecLatentMs\":{\"type\":\"number\"},\"codecSynthMs\":{\"type\":\"number\"},\"resampleMs\":{\"type\":\"number\"}}},\"chunkIndex\":{\"type\":\"integer\"},\"sentenceChunk\":{\"type\":\"string\"},\"isLast\":{\"type\":\"boolean\"},\"speechTokenIds\":{\"type\":\"array\",\"items\":{\"type\":\"integer\"}},\"stopReason\":{\"type\":\"string\",\"enum\":[\"completed\",\"cancelled\"]}}}").jsonObject, + Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\",\"buffer\",\"done\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"textToSpeechStream\"},\"buffer\":{\"type\":\"array\",\"items\":{\"type\":\"number\"}},\"pcm\":{\"type\":\"string\"},\"done\":{\"type\":\"boolean\"},\"sampleRate\":{\"type\":\"integer\"},\"stats\":{\"type\":\"object\",\"properties\":{\"audioDuration\":{\"type\":\"number\"},\"totalTime\":{\"type\":\"number\"},\"realTimeFactor\":{\"type\":\"number\"},\"firstAudioMs\":{\"type\":\"number\"},\"tokensPerSecond\":{\"type\":\"number\"},\"totalSamples\":{\"type\":\"number\"},\"generatedFrames\":{\"type\":\"number\"},\"backendDevice\":{\"type\":\"number\"},\"backendId\":{\"type\":\"number\"},\"gpuUnsupported\":{\"type\":\"number\"},\"enhancerBackendDevice\":{\"type\":\"number\"},\"enhancerBackendId\":{\"type\":\"number\"},\"denoiserBackendDevice\":{\"type\":\"number\"},\"denoiserBackendId\":{\"type\":\"number\"},\"codecSidecarLoaded\":{\"type\":\"number\"},\"codecOnCoreml\":{\"type\":\"number\"},\"decoderSidecarLoaded\":{\"type\":\"number\"},\"dacSidecarLoaded\":{\"type\":\"number\"},\"decoderOnCoreml\":{\"type\":\"number\"},\"dacOnCoreml\":{\"type\":\"number\"},\"t3Ms\":{\"type\":\"number\"},\"s3genMs\":{\"type\":\"number\"},\"t3Tokens\":{\"type\":\"number\"},\"stageTotalMs\":{\"type\":\"number\"},\"lmPrefillMs\":{\"type\":\"number\"},\"lmDecodeMs\":{\"type\":\"number\"},\"flowFrontendMs\":{\"type\":\"number\"},\"ditEulerMs\":{\"type\":\"number\"},\"hiftF0Ms\":{\"type\":\"number\"},\"hiftSourceMs\":{\"type\":\"number\"},\"hiftStftMs\":{\"type\":\"number\"},\"hiftDecodeMs\":{\"type\":\"number\"},\"decodeSteps\":{\"type\":\"number\"},\"speechTokens\":{\"type\":\"number\"},\"flowFrames\":{\"type\":\"number\"},\"melFrames\":{\"type\":\"number\"},\"textIds\":{\"type\":\"number\"},\"promptSpeechTokens\":{\"type\":\"number\"},\"voiceEncodeMs\":{\"type\":\"number\"},\"promptMs\":{\"type\":\"number\"},\"prefillMs\":{\"type\":\"number\"},\"sampleMs\":{\"type\":\"number\"},\"fastDecodeMs\":{\"type\":\"number\"},\"slowDecodeMs\":{\"type\":\"number\"},\"codecLatentMs\":{\"type\":\"number\"},\"codecSynthMs\":{\"type\":\"number\"},\"resampleMs\":{\"type\":\"number\"}}},\"chunkIndex\":{\"type\":\"integer\"},\"sentenceChunk\":{\"type\":\"string\"},\"isLast\":{\"type\":\"boolean\"},\"speechTokenIds\":{\"type\":\"array\",\"items\":{\"type\":\"integer\"}},\"stopReason\":{\"type\":\"string\",\"enum\":[\"completed\",\"cancelled\"]}}}").jsonObject, Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\"],\"properties\":{\"text\":{\"type\":\"string\"},\"done\":{\"type\":\"boolean\"},\"stats\":{\"type\":\"object\",\"properties\":{\"audioDuration\":{\"type\":\"number\"},\"realTimeFactor\":{\"type\":\"number\"},\"tokensPerSecond\":{\"type\":\"number\"},\"totalTokens\":{\"type\":\"number\"},\"totalSegments\":{\"type\":\"number\"},\"whisperEncodeTime\":{\"type\":\"number\"},\"whisperDecodeTime\":{\"type\":\"number\"},\"encoderTime\":{\"type\":\"number\"},\"decoderTime\":{\"type\":\"number\"},\"melSpecTime\":{\"type\":\"number\"},\"backendDevice\":{\"type\":\"number\"},\"backendId\":{\"type\":\"number\"},\"gpuUnsupported\":{\"type\":\"number\"},\"gpuMemTotalMb\":{\"type\":\"number\"},\"gpuMemFreeMb\":{\"type\":\"number\"},\"totalTime\":{\"type\":\"number\"},\"totalWallMs\":{\"type\":\"number\"},\"totalSamples\":{\"type\":\"number\"},\"processCalls\":{\"type\":\"number\"},\"whisperSampleMs\":{\"type\":\"number\"},\"whisperBatchdMs\":{\"type\":\"number\"},\"whisperPromptMs\":{\"type\":\"number\"},\"totalTranscriptions\":{\"type\":\"number\"},\"modelLoadMs\":{\"type\":\"number\"},\"totalEncodedFrames\":{\"type\":\"number\"},\"encoderOnCoreml\":{\"type\":\"number\"},\"encoderUsedCoreml\":{\"type\":\"number\"},\"aoscActive\":{\"type\":\"number\"},\"prefillTime\":{\"type\":\"number\"},\"audioTokens\":{\"type\":\"number\"},\"promptTokens\":{\"type\":\"number\"},\"generatedTokens\":{\"type\":\"number\"}}},\"error\":{\"type\":\"string\"},\"segment\":{\"type\":\"object\",\"required\":[\"text\",\"startMs\",\"endMs\",\"append\",\"id\"],\"properties\":{\"text\":{\"type\":\"string\"},\"startMs\":{\"type\":\"number\"},\"endMs\":{\"type\":\"number\"},\"append\":{\"type\":\"boolean\"},\"id\":{\"type\":\"number\"},\"speakerId\":{\"type\":\"integer\"},\"speaker\":{\"type\":\"string\"},\"speakerSegments\":{\"type\":\"array\",\"items\":{\"type\":\"object\",\"required\":[\"speakerId\",\"startMs\",\"endMs\"]}},\"isEndOfTurn\":{\"type\":\"boolean\"},\"startsWord\":{\"type\":\"boolean\"},\"language\":{\"type\":\"string\"},\"noSpeechProb\":{\"type\":\"number\"},\"speakerTurnNext\":{\"type\":\"boolean\"},\"tokens\":{\"type\":\"array\",\"items\":{\"type\":\"object\",\"required\":[\"text\",\"startMs\",\"endMs\",\"probability\"]}},\"tokenIds\":{\"type\":\"array\",\"items\":{\"type\":\"integer\"}},\"isFinal\":{\"type\":\"boolean\"},\"chunkIndex\":{\"type\":\"integer\"},\"eotConfidence\":{\"type\":\"number\"}}},\"vad\":{\"type\":\"object\",\"required\":[\"speaking\",\"probability\"],\"properties\":{\"speakerId\":{\"type\":\"integer\"},\"speaking\":{\"type\":\"boolean\"},\"probability\":{\"type\":\"number\"},\"timestamp\":{\"type\":\"number\"},\"chunkIndex\":{\"type\":\"integer\"},\"source\":{\"type\":\"string\",\"enum\":[\"silero\",\"energy\",\"sortformer\"]}}},\"endOfTurn\":{\"oneOf\":[{\"type\":\"object\",\"required\":[\"source\",\"silenceDurationMs\"],\"properties\":{\"source\":{\"type\":\"string\",\"const\":\"whisper\"},\"silenceDurationMs\":{\"type\":\"number\"}}},{\"type\":\"object\",\"required\":[\"source\"],\"properties\":{\"source\":{\"type\":\"string\",\"const\":\"parakeet\"},\"timestamp\":{\"type\":\"number\"},\"chunkIndex\":{\"type\":\"integer\"},\"confidence\":{\"type\":\"number\"}}}]},\"diagnostics\":{\"type\":\"object\",\"required\":[\"selectedBackend\",\"selectedDevice\"],\"properties\":{\"selectedBackend\":{\"type\":\"string\"},\"selectedDevice\":{\"type\":\"string\",\"enum\":[\"cpu\",\"gpu\"]},\"graphicsApi\":{\"type\":\"string\",\"enum\":[\"vulkan\",\"opencl\",\"opengl\",\"webgpu\",\"metal\",\"direct3d11\",\"direct3d12\",\"cuda\",\"levelZero\",\"rocm\"]},\"driver\":{\"type\":\"object\",\"required\":[\"name\"]},\"gpuId\":{\"type\":\"string\"},\"fallback\":{\"type\":\"object\",\"required\":[\"reason\"]},\"probe\":{\"type\":\"object\",\"required\":[\"status\",\"backend\"]}}},\"type\":{\"type\":\"string\",\"const\":\"transcribe\"}}}").jsonObject, Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\"],\"properties\":{\"text\":{\"type\":\"string\"},\"done\":{\"type\":\"boolean\"},\"stats\":{\"type\":\"object\",\"properties\":{\"audioDuration\":{\"type\":\"number\"},\"realTimeFactor\":{\"type\":\"number\"},\"tokensPerSecond\":{\"type\":\"number\"},\"totalTokens\":{\"type\":\"number\"},\"totalSegments\":{\"type\":\"number\"},\"whisperEncodeTime\":{\"type\":\"number\"},\"whisperDecodeTime\":{\"type\":\"number\"},\"encoderTime\":{\"type\":\"number\"},\"decoderTime\":{\"type\":\"number\"},\"melSpecTime\":{\"type\":\"number\"},\"backendDevice\":{\"type\":\"number\"},\"backendId\":{\"type\":\"number\"},\"gpuUnsupported\":{\"type\":\"number\"},\"gpuMemTotalMb\":{\"type\":\"number\"},\"gpuMemFreeMb\":{\"type\":\"number\"},\"totalTime\":{\"type\":\"number\"},\"totalWallMs\":{\"type\":\"number\"},\"totalSamples\":{\"type\":\"number\"},\"processCalls\":{\"type\":\"number\"},\"whisperSampleMs\":{\"type\":\"number\"},\"whisperBatchdMs\":{\"type\":\"number\"},\"whisperPromptMs\":{\"type\":\"number\"},\"totalTranscriptions\":{\"type\":\"number\"},\"modelLoadMs\":{\"type\":\"number\"},\"totalEncodedFrames\":{\"type\":\"number\"},\"encoderOnCoreml\":{\"type\":\"number\"},\"encoderUsedCoreml\":{\"type\":\"number\"},\"aoscActive\":{\"type\":\"number\"},\"prefillTime\":{\"type\":\"number\"},\"audioTokens\":{\"type\":\"number\"},\"promptTokens\":{\"type\":\"number\"},\"generatedTokens\":{\"type\":\"number\"}}},\"error\":{\"type\":\"string\"},\"segment\":{\"type\":\"object\",\"required\":[\"text\",\"startMs\",\"endMs\",\"append\",\"id\"],\"properties\":{\"text\":{\"type\":\"string\"},\"startMs\":{\"type\":\"number\"},\"endMs\":{\"type\":\"number\"},\"append\":{\"type\":\"boolean\"},\"id\":{\"type\":\"number\"},\"speakerId\":{\"type\":\"integer\"},\"speaker\":{\"type\":\"string\"},\"speakerSegments\":{\"type\":\"array\",\"items\":{\"type\":\"object\",\"required\":[\"speakerId\",\"startMs\",\"endMs\"]}},\"isEndOfTurn\":{\"type\":\"boolean\"},\"startsWord\":{\"type\":\"boolean\"},\"language\":{\"type\":\"string\"},\"noSpeechProb\":{\"type\":\"number\"},\"speakerTurnNext\":{\"type\":\"boolean\"},\"tokens\":{\"type\":\"array\",\"items\":{\"type\":\"object\",\"required\":[\"text\",\"startMs\",\"endMs\",\"probability\"]}},\"tokenIds\":{\"type\":\"array\",\"items\":{\"type\":\"integer\"}},\"isFinal\":{\"type\":\"boolean\"},\"chunkIndex\":{\"type\":\"integer\"},\"eotConfidence\":{\"type\":\"number\"}}},\"vad\":{\"type\":\"object\",\"required\":[\"speaking\",\"probability\"],\"properties\":{\"speakerId\":{\"type\":\"integer\"},\"speaking\":{\"type\":\"boolean\"},\"probability\":{\"type\":\"number\"},\"timestamp\":{\"type\":\"number\"},\"chunkIndex\":{\"type\":\"integer\"},\"source\":{\"type\":\"string\",\"enum\":[\"silero\",\"energy\",\"sortformer\"]}}},\"endOfTurn\":{\"oneOf\":[{\"type\":\"object\",\"required\":[\"source\",\"silenceDurationMs\"],\"properties\":{\"source\":{\"type\":\"string\",\"const\":\"whisper\"},\"silenceDurationMs\":{\"type\":\"number\"}}},{\"type\":\"object\",\"required\":[\"source\"],\"properties\":{\"source\":{\"type\":\"string\",\"const\":\"parakeet\"},\"timestamp\":{\"type\":\"number\"},\"chunkIndex\":{\"type\":\"integer\"},\"confidence\":{\"type\":\"number\"}}}]},\"diagnostics\":{\"type\":\"object\",\"required\":[\"selectedBackend\",\"selectedDevice\"],\"properties\":{\"selectedBackend\":{\"type\":\"string\"},\"selectedDevice\":{\"type\":\"string\",\"enum\":[\"cpu\",\"gpu\"]},\"graphicsApi\":{\"type\":\"string\",\"enum\":[\"vulkan\",\"opencl\",\"opengl\",\"webgpu\",\"metal\",\"direct3d11\",\"direct3d12\",\"cuda\",\"levelZero\",\"rocm\"]},\"driver\":{\"type\":\"object\",\"required\":[\"name\"]},\"gpuId\":{\"type\":\"string\"},\"fallback\":{\"type\":\"object\",\"required\":[\"reason\"]},\"probe\":{\"type\":\"object\",\"required\":[\"status\",\"backend\"]}}},\"type\":{\"type\":\"string\",\"const\":\"transcribeStream\"}}}").jsonObject, Json.parseToJsonElement("{\"type\":\"object\",\"required\":[\"type\",\"token\"],\"properties\":{\"type\":{\"type\":\"string\",\"const\":\"translate\"},\"token\":{\"type\":\"string\"},\"done\":{\"type\":\"boolean\"},\"stats\":{\"type\":\"object\",\"properties\":{\"totalTime\":{\"type\":\"number\"},\"totalTokens\":{\"type\":\"number\"},\"tokensPerSecond\":{\"type\":\"number\"},\"timeToFirstToken\":{\"type\":\"number\"},\"decodeTime\":{\"type\":\"number\"},\"encodeTime\":{\"type\":\"number\"},\"cacheTokens\":{\"type\":\"number\"}}},\"error\":{\"type\":\"string\"}}}").jsonObject, @@ -19611,6 +19737,7 @@ data class TextToSpeechResponse( @SerialName("chunkIndex") val `chunkIndex`: Long? = null, @SerialName("sentenceChunk") val `sentenceChunk`: String? = null, @SerialName("isLast") val `isLast`: Boolean? = null, + @SerialName("speechTokenIds") val `speechTokenIds`: List? = null, @SerialName("stopReason") val `stopReason`: TextToSpeechResponseStopReason? = null, ) @@ -19632,6 +19759,10 @@ data class TextToSpeechResponseStats( @SerialName("denoiserBackendId") val `denoiserBackendId`: Double? = null, @SerialName("codecSidecarLoaded") val `codecSidecarLoaded`: Double? = null, @SerialName("codecOnCoreml") val `codecOnCoreml`: Double? = null, + @SerialName("decoderSidecarLoaded") val `decoderSidecarLoaded`: Double? = null, + @SerialName("dacSidecarLoaded") val `dacSidecarLoaded`: Double? = null, + @SerialName("decoderOnCoreml") val `decoderOnCoreml`: Double? = null, + @SerialName("dacOnCoreml") val `dacOnCoreml`: Double? = null, @SerialName("t3Ms") val `t3Ms`: Double? = null, @SerialName("s3genMs") val `s3genMs`: Double? = null, @SerialName("t3Tokens") val `t3Tokens`: Double? = null, @@ -19673,6 +19804,7 @@ data class TextToSpeechStreamResponse( @SerialName("chunkIndex") val `chunkIndex`: Long? = null, @SerialName("sentenceChunk") val `sentenceChunk`: String? = null, @SerialName("isLast") val `isLast`: Boolean? = null, + @SerialName("speechTokenIds") val `speechTokenIds`: List? = null, @SerialName("stopReason") val `stopReason`: TextToSpeechResponseStopReason? = null, ) @@ -20278,6 +20410,14 @@ enum class TtsEngine { @SerialName("moss-speech") `MOSS_SPEECH`, } +@Serializable +enum class TtsLavasrCoremlComputeUnits { + @SerialName("CPU_ONLY") `CPU_ONLY`, + @SerialName("CPU_AND_GPU") `CPU_AND_GPU`, + @SerialName("CPU_AND_NE") `CPU_AND_NE`, + @SerialName("ALL") `ALL`, +} + @Serializable enum class TtsPace { @SerialName("slow") `SLOW`, @@ -20285,6 +20425,14 @@ enum class TtsPace { @SerialName("fast") `FAST`, } +@Serializable +enum class TtsParlerCoremlComputeUnits { + @SerialName("all") `ALL`, + @SerialName("cpu_and_ane") `CPU_AND_ANE`, + @SerialName("cpu_and_gpu") `CPU_AND_GPU`, + @SerialName("cpu_only") `CPU_ONLY`, +} + @Serializable enum class TtsParlerEmotion { @SerialName("command") `COMMAND`, @@ -20343,6 +20491,14 @@ enum class TtsSupertonicLanguage { @SerialName("vi") `VI`, } +@Serializable +enum class TtsSupertonicPrecision { + @SerialName("auto") `AUTO`, + @SerialName("f32") `F32`, + @SerialName("f16") `F16`, + @SerialName("q8_0") `Q8_0`, +} + @Serializable enum class VectorIndexStorage { @SerialName("f32") `F32`, diff --git a/packages/sdk-python/src/tetherto/qvac_sdk/_generated/__init__.py b/packages/sdk-python/src/tetherto/qvac_sdk/_generated/__init__.py index 12510cef5de..4eed653a369 100644 --- a/packages/sdk-python/src/tetherto/qvac_sdk/_generated/__init__.py +++ b/packages/sdk-python/src/tetherto/qvac_sdk/_generated/__init__.py @@ -103,10 +103,13 @@ TtsCosyvoice3InstructStyle, TtsCosyvoice3InstructVolume, TtsEngine, + TtsLavasrCoremlComputeUnits, TtsPace, + TtsParlerCoremlComputeUnits, TtsParlerEmotion, TtsSentenceDelimiterPreset, TtsSupertonicLanguage, + TtsSupertonicPrecision, UnloadModelRequest, UnloadModelResponse, UpscaleStreamRequest, @@ -238,10 +241,13 @@ "TtsCosyvoice3InstructStyle", "TtsCosyvoice3InstructVolume", "TtsEngine", + "TtsLavasrCoremlComputeUnits", "TtsPace", + "TtsParlerCoremlComputeUnits", "TtsParlerEmotion", "TtsSentenceDelimiterPreset", "TtsSupertonicLanguage", + "TtsSupertonicPrecision", "UnloadModelRequest", "UnloadModelResponse", "UpscaleStreamRequest", diff --git a/packages/sdk-python/src/tetherto/qvac_sdk/_generated/models/__init__.py b/packages/sdk-python/src/tetherto/qvac_sdk/_generated/models/__init__.py index cbe2b60e7fe..5661229a078 100644 --- a/packages/sdk-python/src/tetherto/qvac_sdk/_generated/models/__init__.py +++ b/packages/sdk-python/src/tetherto/qvac_sdk/_generated/models/__init__.py @@ -371,6 +371,7 @@ Embodiment1, ErrorResponse, ErrorResponseTypedFields, + F16WeightsDenyListItem, FieldQvacSdkWireContract, FinetuneGetStateRequest, FinetuneGetStateRequestOptions, @@ -867,8 +868,10 @@ LoadModelSrcRequestTtsGgmlModelConfigChatterboxCangjieTsvSrcAddon, LoadModelSrcRequestTtsGgmlModelConfigChatterboxKvCacheType, LoadModelSrcRequestTtsGgmlModelConfigChatterboxLanguage, + LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrDenoiserCoremlComputeUnits, LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrDenoiserModelSrc, LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrDenoiserModelSrcAddon, + LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrEnhancerCoremlComputeUnits, LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrEnhancerModelSrc, LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrEnhancerModelSrcAddon, LoadModelSrcRequestTtsGgmlModelConfigChatterboxMecabDictSrc, @@ -887,8 +890,10 @@ LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3InstructDialect, LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3InstructStyle, LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3InstructVolume, + LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrDenoiserCoremlComputeUnits, LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrDenoiserModelSrc, LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrDenoiserModelSrcAddon, + LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrEnhancerCoremlComputeUnits, LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrEnhancerModelSrc, LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrEnhancerModelSrcAddon, LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3Pace, @@ -908,23 +913,30 @@ LoadModelSrcRequestTtsGgmlModelConfigMossSpeechMossSpeechCodecModelSrc, LoadModelSrcRequestTtsGgmlModelConfigMossSpeechMossSpeechCodecModelSrcAddon, LoadModelSrcRequestTtsGgmlModelConfigParler, + LoadModelSrcRequestTtsGgmlModelConfigParlerCoremlComputeUnits, LoadModelSrcRequestTtsGgmlModelConfigParlerEmotion, + LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrDenoiserCoremlComputeUnits, LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrDenoiserModelSrc, LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrDenoiserModelSrcAddon, + LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrEnhancerCoremlComputeUnits, LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrEnhancerModelSrc, LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrEnhancerModelSrcAddon, LoadModelSrcRequestTtsGgmlModelConfigParlerPace, LoadModelSrcRequestTtsGgmlModelConfigSupertonic, LoadModelSrcRequestTtsGgmlModelConfigSupertonicLanguage, + LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrDenoiserCoremlComputeUnits, LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrDenoiserModelSrc, LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrDenoiserModelSrcAddon, + LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrEnhancerCoremlComputeUnits, LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrEnhancerModelSrc, LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrEnhancerModelSrcAddon, LoadModelSrcRequestTtsGgmlModelConfigSupertonicNoiseNpySrc, LoadModelSrcRequestTtsGgmlModelConfigSupertonicNoiseNpySrcAddon, LoadModelSrcRequestTtsGgmlModelConfigSupertonicPace, + LoadModelSrcRequestTtsGgmlModelConfigSupertonicPrecision, LoadModelSrcRequestTtsGgmlModelConfigSupertonicVoiceJsonSrc, LoadModelSrcRequestTtsGgmlModelConfigSupertonicVoiceJsonSrcAddon, + LoadModelSrcRequestTtsGgmlModelConfigSupertonicVulkanEnvOverrides, LoadModelSrcRequestTtsGgmlModelFitPolicy, LoadModelSrcRequestWhispercppTranscription, LoadModelSrcRequestWhispercppTranscriptionModelConfig, @@ -1152,10 +1164,14 @@ TtsCosyvoice3InstructStyle, TtsCosyvoice3InstructVolume, TtsEngine, + TtsLavasrCoremlComputeUnits, TtsPace, + TtsParlerCoremlComputeUnits, TtsParlerEmotion, TtsSentenceDelimiterPreset, + TtsSpeechTokenId, TtsSupertonicLanguage, + TtsSupertonicPrecision, UnloadModelRequest, UnloadModelResponse, UpscaleStreamRequest, @@ -1587,6 +1603,7 @@ "Embodiment1", "ErrorResponse", "ErrorResponseTypedFields", + "F16WeightsDenyListItem", "FieldQvacSdkWireContract", "FinetuneGetStateRequest", "FinetuneGetStateRequestOptions", @@ -2081,8 +2098,10 @@ "LoadModelSrcRequestTtsGgmlModelConfigChatterboxCangjieTsvSrcAddon", "LoadModelSrcRequestTtsGgmlModelConfigChatterboxKvCacheType", "LoadModelSrcRequestTtsGgmlModelConfigChatterboxLanguage", + "LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrDenoiserCoremlComputeUnits", "LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrDenoiserModelSrc", "LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrDenoiserModelSrcAddon", + "LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrEnhancerCoremlComputeUnits", "LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrEnhancerModelSrc", "LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrEnhancerModelSrcAddon", "LoadModelSrcRequestTtsGgmlModelConfigChatterboxMecabDictSrc", @@ -2101,8 +2120,10 @@ "LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3InstructDialect", "LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3InstructStyle", "LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3InstructVolume", + "LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrDenoiserCoremlComputeUnits", "LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrDenoiserModelSrc", "LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrDenoiserModelSrcAddon", + "LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrEnhancerCoremlComputeUnits", "LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrEnhancerModelSrc", "LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrEnhancerModelSrcAddon", "LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3Pace", @@ -2122,23 +2143,30 @@ "LoadModelSrcRequestTtsGgmlModelConfigMossSpeechMossSpeechCodecModelSrc", "LoadModelSrcRequestTtsGgmlModelConfigMossSpeechMossSpeechCodecModelSrcAddon", "LoadModelSrcRequestTtsGgmlModelConfigParler", + "LoadModelSrcRequestTtsGgmlModelConfigParlerCoremlComputeUnits", "LoadModelSrcRequestTtsGgmlModelConfigParlerEmotion", + "LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrDenoiserCoremlComputeUnits", "LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrDenoiserModelSrc", "LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrDenoiserModelSrcAddon", + "LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrEnhancerCoremlComputeUnits", "LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrEnhancerModelSrc", "LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrEnhancerModelSrcAddon", "LoadModelSrcRequestTtsGgmlModelConfigParlerPace", "LoadModelSrcRequestTtsGgmlModelConfigSupertonic", "LoadModelSrcRequestTtsGgmlModelConfigSupertonicLanguage", + "LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrDenoiserCoremlComputeUnits", "LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrDenoiserModelSrc", "LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrDenoiserModelSrcAddon", + "LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrEnhancerCoremlComputeUnits", "LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrEnhancerModelSrc", "LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrEnhancerModelSrcAddon", "LoadModelSrcRequestTtsGgmlModelConfigSupertonicNoiseNpySrc", "LoadModelSrcRequestTtsGgmlModelConfigSupertonicNoiseNpySrcAddon", "LoadModelSrcRequestTtsGgmlModelConfigSupertonicPace", + "LoadModelSrcRequestTtsGgmlModelConfigSupertonicPrecision", "LoadModelSrcRequestTtsGgmlModelConfigSupertonicVoiceJsonSrc", "LoadModelSrcRequestTtsGgmlModelConfigSupertonicVoiceJsonSrcAddon", + "LoadModelSrcRequestTtsGgmlModelConfigSupertonicVulkanEnvOverrides", "LoadModelSrcRequestTtsGgmlModelFitPolicy", "LoadModelSrcRequestWhispercppTranscription", "LoadModelSrcRequestWhispercppTranscriptionModelConfig", @@ -2368,10 +2396,14 @@ "TtsCosyvoice3InstructStyle", "TtsCosyvoice3InstructVolume", "TtsEngine", + "TtsLavasrCoremlComputeUnits", "TtsPace", + "TtsParlerCoremlComputeUnits", "TtsParlerEmotion", "TtsSentenceDelimiterPreset", + "TtsSpeechTokenId", "TtsSupertonicLanguage", + "TtsSupertonicPrecision", "UnloadModelRequest", "UnloadModelResponse", "UpscaleStreamRequest", diff --git a/packages/sdk-python/src/tetherto/qvac_sdk/_generated/models/_internal.py b/packages/sdk-python/src/tetherto/qvac_sdk/_generated/models/_internal.py index 3d30244b00f..1b295ab5d19 100644 --- a/packages/sdk-python/src/tetherto/qvac_sdk/_generated/models/_internal.py +++ b/packages/sdk-python/src/tetherto/qvac_sdk/_generated/models/_internal.py @@ -5545,12 +5545,26 @@ class TtsEngine(Enum): moss_speech = "moss-speech" +class TtsLavasrCoremlComputeUnits(Enum): + cpu_only = "CPU_ONLY" + cpu_and_gpu = "CPU_AND_GPU" + cpu_and_ne = "CPU_AND_NE" + all = "ALL" + + class TtsPace(Enum): slow = "slow" moderate = "moderate" fast = "fast" +class TtsParlerCoremlComputeUnits(Enum): + all = "all" + cpu_and_ane = "cpu_and_ane" + cpu_and_gpu = "cpu_and_gpu" + cpu_only = "cpu_only" + + class TtsParlerEmotion(Enum): command = "command" anger = "anger" @@ -5606,6 +5620,13 @@ class TtsSupertonicLanguage(Enum): vi = "vi" +class TtsSupertonicPrecision(Enum): + auto = "auto" + f32 = "f32" + f16 = "f16" + q8_0 = "q8_0" + + class VectorIndexStorage(Enum): f32 = "f32" q8 = "q8" @@ -16886,6 +16907,24 @@ class LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrDenoiserModelSrc( ] = None +class LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrEnhancerCoremlComputeUnits( + Enum +): + cpu_only = "CPU_ONLY" + cpu_and_gpu = "CPU_AND_GPU" + cpu_and_ne = "CPU_AND_NE" + all = "ALL" + + +class LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrDenoiserCoremlComputeUnits( + Enum +): + cpu_only = "CPU_ONLY" + cpu_and_gpu = "CPU_AND_GPU" + cpu_and_ne = "CPU_AND_NE" + all = "ALL" + + class LoadModelSrcRequestTtsGgmlModelConfigChatterbox(GeneratedBaseModel): model_config = ConfigDict( extra="forbid", @@ -17121,6 +17160,13 @@ class LoadModelSrcRequestTtsGgmlModelConfigChatterbox(GeneratedBaseModel): le=2147483647, ), ] = None + backend: Annotated[ + str | None, + Field( + description="Explicit native backend: `auto`, `cpu`, `opencl`, `webgpu`, or an exact registered device name such as `MTL0`, `CUDA0` or `Vulkan0` (device names depend on the host). An unavailable explicit backend fails the load; unset keeps the `useGPU` policy. WebGPU is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only.", + min_length=1, + ), + ] = None backends_dir: Annotated[ str | None, Field( @@ -17183,6 +17229,40 @@ class LoadModelSrcRequestTtsGgmlModelConfigChatterbox(GeneratedBaseModel): description="LavaSR denoiser model source; runs before the enhancer, rate-preserving (batch synthesis only).", ), ] = None + lavasr_enhancer_coreml_model_path: Annotated[ + str | None, + Field( + alias="lavasrEnhancerCoremlModelPath", + description="macOS / iOS: compiled Core ML model directory (`.mlmodelc`) on the inference host for the LavaSR enhancer’s neural core; requires `lavasrEnhancerModelSrc` and `lavasrEnhancerCoremlComputeUnits`. A model that cannot load fails the load rather than falling back to ggml.", + min_length=1, + ), + ] = None + lavasr_enhancer_coreml_compute_units: Annotated[ + LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrEnhancerCoremlComputeUnits + | None, + Field( + alias="lavasrEnhancerCoremlComputeUnits", + description="macOS / iOS: Core ML compute policy for the LavaSR enhancer; required with `lavasrEnhancerCoremlModelPath`. Stats then report the stage as device `2` (Core ML-managed), backend `99`.", + title="LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrEnhancerCoremlComputeUnits", + ), + ] = None + lavasr_denoiser_coreml_model_path: Annotated[ + str | None, + Field( + alias="lavasrDenoiserCoremlModelPath", + description="macOS / iOS: compiled Core ML model directory (`.mlmodelc`) on the inference host for the LavaSR denoiser’s neural core; requires `lavasrDenoiserModelSrc` and `lavasrDenoiserCoremlComputeUnits`. A model that cannot load fails the load rather than falling back to ggml.", + min_length=1, + ), + ] = None + lavasr_denoiser_coreml_compute_units: Annotated[ + LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrDenoiserCoremlComputeUnits + | None, + Field( + alias="lavasrDenoiserCoremlComputeUnits", + description="macOS / iOS: Core ML compute policy for the LavaSR denoiser; required with `lavasrDenoiserCoremlModelPath`. Stats then report the stage as device `2` (Core ML-managed), backend `99`.", + title="LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrDenoiserCoremlComputeUnits", + ), + ] = None tts_supertonic_multilingual: Annotated[ Any | None, Field(alias="ttsSupertonicMultilingual") ] = None @@ -17252,6 +17332,29 @@ class LoadModelSrcRequestTtsGgmlModelConfigSupertonicPace(Enum): fast = "fast" +class LoadModelSrcRequestTtsGgmlModelConfigSupertonicPrecision(Enum): + auto = "auto" + f32 = "f32" + f16 = "f16" + q8_0 = "q8_0" + + +class F16WeightsDenyListItem(RootModel[str]): + root: Annotated[str, Field(min_length=1)] + + +class LoadModelSrcRequestTtsGgmlModelConfigSupertonicVulkanEnvOverrides( + RootModel[dict[str, str]] +): + root: Annotated[ + dict[str, str], + Field( + description="Vulkan environment overrides (string values) passed to the engine; explicit entries win over the `vulkanCacheDir` convenience setting.", + title="LoadModelSrcRequestTtsGgmlModelConfigSupertonicVulkanEnvOverrides", + ), + ] + + class LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrEnhancerModelSrcAddon(Enum): llamacpp_completion = "llamacpp-completion" whispercpp_transcription = "whispercpp-transcription" @@ -17476,6 +17579,24 @@ class LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrDenoiserModelSrc( ] = None +class LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrEnhancerCoremlComputeUnits( + Enum +): + cpu_only = "CPU_ONLY" + cpu_and_gpu = "CPU_AND_GPU" + cpu_and_ne = "CPU_AND_NE" + all = "ALL" + + +class LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrDenoiserCoremlComputeUnits( + Enum +): + cpu_only = "CPU_ONLY" + cpu_and_gpu = "CPU_AND_GPU" + cpu_and_ne = "CPU_AND_NE" + all = "ALL" + + class LoadModelSrcRequestTtsGgmlModelConfigSupertonicVoiceJsonSrcAddon(Enum): llamacpp_completion = "llamacpp-completion" whispercpp_transcription = "whispercpp-transcription" @@ -17840,6 +17961,78 @@ class LoadModelSrcRequestTtsGgmlModelConfigSupertonic(GeneratedBaseModel): le=2147483647, ), ] = None + precision: Annotated[ + LoadModelSrcRequestTtsGgmlModelConfigSupertonicPrecision | None, + Field( + description="Weight precision the engine loads at: `auto` keeps the GGUF’s own, `f32` / `f16` / `q8_0` convert on load and change the weight footprint.", + title="LoadModelSrcRequestTtsGgmlModelConfigSupertonicPrecision", + ), + ] = None + f16_attn: Annotated[ + int | None, + Field( + alias="f16Attn", + description="Attention precision override: `-1` automatic (default), `0` f32, `1` f16.", + ge=-2147483648, + le=2147483647, + ), + ] = None + f16_weights: Annotated[ + int | None, + Field( + alias="f16Weights", + description="Weight precision override: `-1` automatic (default), `0` off, `1` f16.", + ge=-2147483648, + le=2147483647, + ), + ] = None + kv_attn_type: Annotated[ + int | None, + Field( + alias="kvAttnType", + description="Native KV attention type code; `-1` leaves the automatic selection.", + ge=-2147483648, + le=2147483647, + ), + ] = None + f16_weights_deny_list: Annotated[ + list[F16WeightsDenyListItem] | None, + Field( + alias="f16WeightsDenyList", + description="Tensor-name patterns kept at their original precision when `f16Weights` is on.", + ), + ] = None + vulkan_env_overrides: Annotated[ + LoadModelSrcRequestTtsGgmlModelConfigSupertonicVulkanEnvOverrides | None, + Field( + alias="vulkanEnvOverrides", + description="Vulkan environment overrides (string values) passed to the engine; explicit entries win over the `vulkanCacheDir` convenience setting.", + title="LoadModelSrcRequestTtsGgmlModelConfigSupertonicVulkanEnvOverrides", + ), + ] = None + voice_style_ttl: Annotated[ + list[float] | None, + Field( + alias="voiceStyleTtl", + description="In-memory voice: the flattened `style_ttl` tensor of a cloned voice. Paired with `voiceStyleDp`; the engine checks the sizes against the model, and the pair wins over `voiceJsonSrc` and the baked `voice`.", + min_length=1, + ), + ] = None + voice_style_dp: Annotated[ + list[float] | None, + Field( + alias="voiceStyleDp", + description="In-memory voice: the flattened `style_dp` tensor, paired with `voiceStyleTtl`.", + min_length=1, + ), + ] = None + backend: Annotated[ + str | None, + Field( + description="Explicit native backend: `auto`, `cpu`, `opencl`, `webgpu`, or an exact registered device name such as `MTL0`, `CUDA0` or `Vulkan0` (device names depend on the host). An unavailable explicit backend fails the load; unset keeps the `useGPU` policy. WebGPU is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only.", + min_length=1, + ), + ] = None backends_dir: Annotated[ str | None, Field( @@ -17874,6 +18067,40 @@ class LoadModelSrcRequestTtsGgmlModelConfigSupertonic(GeneratedBaseModel): description="LavaSR denoiser model source; runs before the enhancer, rate-preserving (batch synthesis only).", ), ] = None + lavasr_enhancer_coreml_model_path: Annotated[ + str | None, + Field( + alias="lavasrEnhancerCoremlModelPath", + description="macOS / iOS: compiled Core ML model directory (`.mlmodelc`) on the inference host for the LavaSR enhancer’s neural core; requires `lavasrEnhancerModelSrc` and `lavasrEnhancerCoremlComputeUnits`. A model that cannot load fails the load rather than falling back to ggml.", + min_length=1, + ), + ] = None + lavasr_enhancer_coreml_compute_units: Annotated[ + LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrEnhancerCoremlComputeUnits + | None, + Field( + alias="lavasrEnhancerCoremlComputeUnits", + description="macOS / iOS: Core ML compute policy for the LavaSR enhancer; required with `lavasrEnhancerCoremlModelPath`. Stats then report the stage as device `2` (Core ML-managed), backend `99`.", + title="LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrEnhancerCoremlComputeUnits", + ), + ] = None + lavasr_denoiser_coreml_model_path: Annotated[ + str | None, + Field( + alias="lavasrDenoiserCoremlModelPath", + description="macOS / iOS: compiled Core ML model directory (`.mlmodelc`) on the inference host for the LavaSR denoiser’s neural core; requires `lavasrDenoiserModelSrc` and `lavasrDenoiserCoremlComputeUnits`. A model that cannot load fails the load rather than falling back to ggml.", + min_length=1, + ), + ] = None + lavasr_denoiser_coreml_compute_units: Annotated[ + LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrDenoiserCoremlComputeUnits + | None, + Field( + alias="lavasrDenoiserCoremlComputeUnits", + description="macOS / iOS: Core ML compute policy for the LavaSR denoiser; required with `lavasrDenoiserCoremlModelPath`. Stats then report the stage as device `2` (Core ML-managed), backend `99`.", + title="LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrDenoiserCoremlComputeUnits", + ), + ] = None voice_json_src: Annotated[ str | LoadModelSrcRequestTtsGgmlModelConfigSupertonicVoiceJsonSrc | None, Field( @@ -17949,6 +18176,13 @@ class MaxFrames(RootModel[int]): ] +class LoadModelSrcRequestTtsGgmlModelConfigParlerCoremlComputeUnits(Enum): + all = "all" + cpu_and_ane = "cpu_and_ane" + cpu_and_gpu = "cpu_and_gpu" + cpu_only = "cpu_only" + + class LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrEnhancerModelSrcAddon(Enum): llamacpp_completion = "llamacpp-completion" whispercpp_transcription = "whispercpp-transcription" @@ -18173,6 +18407,20 @@ class LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrDenoiserModelSrc( ] = None +class LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrEnhancerCoremlComputeUnits(Enum): + cpu_only = "CPU_ONLY" + cpu_and_gpu = "CPU_AND_GPU" + cpu_and_ne = "CPU_AND_NE" + all = "ALL" + + +class LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrDenoiserCoremlComputeUnits(Enum): + cpu_only = "CPU_ONLY" + cpu_and_gpu = "CPU_AND_GPU" + cpu_and_ne = "CPU_AND_NE" + all = "ALL" + + class LoadModelSrcRequestTtsGgmlModelConfigParler(GeneratedBaseModel): model_config = ConfigDict( extra="forbid", @@ -18350,6 +18598,35 @@ class LoadModelSrcRequestTtsGgmlModelConfigParler(GeneratedBaseModel): description="Parler prompt digit expansion (engine default: enabled).", ), ] = None + coreml_decoder: Annotated[ + bool | None, + Field( + alias="coremlDecoder", + description="macOS / iOS: run the decoder steps on the `-decoder.mlmodelc` sidecar beside the GGUF (the DAC sidecar is picked up by presence); unset keeps the engine default.", + ), + ] = None + coreml_compute_units: Annotated[ + LoadModelSrcRequestTtsGgmlModelConfigParlerCoremlComputeUnits | None, + Field( + alias="coremlComputeUnits", + description="macOS / iOS: Core ML compute policy for the Parler sidecars; unset inherits the engine’s environment default.", + title="LoadModelSrcRequestTtsGgmlModelConfigParlerCoremlComputeUnits", + ), + ] = None + coreml_strict: Annotated[ + bool | None, + Field( + alias="coremlStrict", + description="macOS / iOS: `true` fails the load when a sidecar is unusable instead of falling back to ggml, `false` permits the fallback; unset inherits the engine’s strict mode.", + ), + ] = None + backend: Annotated[ + str | None, + Field( + description="Explicit native backend: `auto`, `cpu`, `opencl`, `webgpu`, or an exact registered device name such as `MTL0`, `CUDA0` or `Vulkan0` (device names depend on the host). An unavailable explicit backend fails the load; unset keeps the `useGPU` policy. WebGPU is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only.", + min_length=1, + ), + ] = None backends_dir: Annotated[ str | None, Field( @@ -18372,6 +18649,40 @@ class LoadModelSrcRequestTtsGgmlModelConfigParler(GeneratedBaseModel): description="LavaSR denoiser model source; runs before the enhancer, rate-preserving (batch synthesis only).", ), ] = None + lavasr_enhancer_coreml_model_path: Annotated[ + str | None, + Field( + alias="lavasrEnhancerCoremlModelPath", + description="macOS / iOS: compiled Core ML model directory (`.mlmodelc`) on the inference host for the LavaSR enhancer’s neural core; requires `lavasrEnhancerModelSrc` and `lavasrEnhancerCoremlComputeUnits`. A model that cannot load fails the load rather than falling back to ggml.", + min_length=1, + ), + ] = None + lavasr_enhancer_coreml_compute_units: Annotated[ + LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrEnhancerCoremlComputeUnits + | None, + Field( + alias="lavasrEnhancerCoremlComputeUnits", + description="macOS / iOS: Core ML compute policy for the LavaSR enhancer; required with `lavasrEnhancerCoremlModelPath`. Stats then report the stage as device `2` (Core ML-managed), backend `99`.", + title="LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrEnhancerCoremlComputeUnits", + ), + ] = None + lavasr_denoiser_coreml_model_path: Annotated[ + str | None, + Field( + alias="lavasrDenoiserCoremlModelPath", + description="macOS / iOS: compiled Core ML model directory (`.mlmodelc`) on the inference host for the LavaSR denoiser’s neural core; requires `lavasrDenoiserModelSrc` and `lavasrDenoiserCoremlComputeUnits`. A model that cannot load fails the load rather than falling back to ggml.", + min_length=1, + ), + ] = None + lavasr_denoiser_coreml_compute_units: Annotated[ + LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrDenoiserCoremlComputeUnits + | None, + Field( + alias="lavasrDenoiserCoremlComputeUnits", + description="macOS / iOS: Core ML compute policy for the LavaSR denoiser; required with `lavasrDenoiserCoremlModelPath`. Stats then report the stage as device `2` (Core ML-managed), backend `99`.", + title="LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrDenoiserCoremlComputeUnits", + ), + ] = None class LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3Emotion(Enum): @@ -18454,6 +18765,10 @@ class LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3Instruct(GeneratedBaseModel ] = None +class TtsSpeechTokenId(RootModel[int]): + root: Annotated[int, Field(ge=-2147483648, le=2147483647, title="TtsSpeechTokenId")] + + class LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrEnhancerModelSrcAddon(Enum): llamacpp_completion = "llamacpp-completion" whispercpp_transcription = "whispercpp-transcription" @@ -18678,6 +18993,24 @@ class LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrDenoiserModelSrc( ] = None +class LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrEnhancerCoremlComputeUnits( + Enum +): + cpu_only = "CPU_ONLY" + cpu_and_gpu = "CPU_AND_GPU" + cpu_and_ne = "CPU_AND_NE" + all = "ALL" + + +class LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrDenoiserCoremlComputeUnits( + Enum +): + cpu_only = "CPU_ONLY" + cpu_and_gpu = "CPU_AND_GPU" + cpu_and_ne = "CPU_AND_NE" + all = "ALL" + + class LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3ReferenceAudioSrcAddon(Enum): llamacpp_completion = "llamacpp-completion" whispercpp_transcription = "whispercpp-transcription" @@ -19068,6 +19401,19 @@ class LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3(GeneratedBaseModel): le=2147483647, ), ] = None + greedy: Annotated[ + bool | None, + Field( + description="Greedy LM decoding: take the argmax speech token instead of sampling (engine default off)." + ), + ] = None + force_speech_tokens: Annotated[ + list[TtsSpeechTokenId] | None, + Field( + alias="forceSpeechTokens", + description="Speech-token trajectory of ONE native synthesis to replay instead of generating, as a previous batch run, native chunk stream or single sentence returned it in `speechTokenIds`. Fixed at load: every synthesis of the model replays this same array, so a joined sentence stream cannot be replayed sentence by sentence. An empty array restores generation.", + ), + ] = None use_gpu: Annotated[ bool | None, Field( @@ -19127,6 +19473,13 @@ class LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3(GeneratedBaseModel): le=2147483647, ), ] = None + backend: Annotated[ + str | None, + Field( + description="Explicit native backend: `auto`, `cpu`, `opencl`, `webgpu`, or an exact registered device name such as `MTL0`, `CUDA0` or `Vulkan0` (device names depend on the host). An unavailable explicit backend fails the load; unset keeps the `useGPU` policy. WebGPU is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only.", + min_length=1, + ), + ] = None backends_dir: Annotated[ str | None, Field( @@ -19161,6 +19514,40 @@ class LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3(GeneratedBaseModel): description="LavaSR denoiser model source; runs before the enhancer, rate-preserving (batch synthesis only).", ), ] = None + lavasr_enhancer_coreml_model_path: Annotated[ + str | None, + Field( + alias="lavasrEnhancerCoremlModelPath", + description="macOS / iOS: compiled Core ML model directory (`.mlmodelc`) on the inference host for the LavaSR enhancer’s neural core; requires `lavasrEnhancerModelSrc` and `lavasrEnhancerCoremlComputeUnits`. A model that cannot load fails the load rather than falling back to ggml.", + min_length=1, + ), + ] = None + lavasr_enhancer_coreml_compute_units: Annotated[ + LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrEnhancerCoremlComputeUnits + | None, + Field( + alias="lavasrEnhancerCoremlComputeUnits", + description="macOS / iOS: Core ML compute policy for the LavaSR enhancer; required with `lavasrEnhancerCoremlModelPath`. Stats then report the stage as device `2` (Core ML-managed), backend `99`.", + title="LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrEnhancerCoremlComputeUnits", + ), + ] = None + lavasr_denoiser_coreml_model_path: Annotated[ + str | None, + Field( + alias="lavasrDenoiserCoremlModelPath", + description="macOS / iOS: compiled Core ML model directory (`.mlmodelc`) on the inference host for the LavaSR denoiser’s neural core; requires `lavasrDenoiserModelSrc` and `lavasrDenoiserCoremlComputeUnits`. A model that cannot load fails the load rather than falling back to ggml.", + min_length=1, + ), + ] = None + lavasr_denoiser_coreml_compute_units: Annotated[ + LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrDenoiserCoremlComputeUnits + | None, + Field( + alias="lavasrDenoiserCoremlComputeUnits", + description="macOS / iOS: Core ML compute policy for the LavaSR denoiser; required with `lavasrDenoiserCoremlModelPath`. Stats then report the stage as device `2` (Core ML-managed), backend `99`.", + title="LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrDenoiserCoremlComputeUnits", + ), + ] = None reference_audio_src: Annotated[ str | LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3ReferenceAudioSrc | None, Field( @@ -19615,6 +20002,13 @@ class LoadModelSrcRequestTtsGgmlModelConfigAudio8(GeneratedBaseModel): le=2147483647, ), ] = None + backend: Annotated[ + str | None, + Field( + description="Explicit native backend: `auto`, `cpu`, `opencl`, `webgpu`, or an exact registered device name such as `MTL0`, `CUDA0` or `Vulkan0` (device names depend on the host). An unavailable explicit backend fails the load; unset keeps the `useGPU` policy. WebGPU is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only.", + min_length=1, + ), + ] = None backends_dir: Annotated[ str | None, Field( @@ -20116,9 +20510,9 @@ class LoadModelSrcRequestTtsGgmlModelConfigMoss(GeneratedBaseModel): int | None, Field( alias="durationTokens", - description="Target length of each synthesis in codec frames (12.5 per second, so 38 ≈ 3 s), 0–2015; 0 or unset keeps the length free.", + description="Target length of each synthesis in codec frames (12.5 per second, so 38 ≈ 3 s), at most `maxNewTokens` − 33 (2015 by default); 0 or unset keeps the length free.", ge=0, - le=2015, + le=2147483647, ), ] = None stream_chunk_tokens: Annotated[ @@ -20130,6 +20524,84 @@ class LoadModelSrcRequestTtsGgmlModelConfigMoss(GeneratedBaseModel): le=2147483647, ), ] = None + n_ctx: Annotated[ + int | None, + Field( + alias="nCtx", + description="Context capacity of the backbone in rows; unset keeps the native value.", + gt=0, + le=2147483647, + ), + ] = None + max_new_tokens: Annotated[ + int | None, + Field( + alias="maxNewTokens", + description="Generation budget in rows (engine default 2048), of which 33 are delay/EOS rows; bounds `durationTokens`.", + gt=0, + le=2147483647, + ), + ] = None + text_temperature: Annotated[ + float | None, + Field( + alias="textTemperature", + description="Text-channel sampling temperature; unset keeps the engine default.", + ge=0.0, + ), + ] = None + text_top_p: Annotated[ + float | None, + Field( + alias="textTopP", + description="Text-channel top-p (nucleus) cutoff, 0 < p ≤ 1; unset keeps the engine default.", + gt=0.0, + le=1.0, + ), + ] = None + text_top_k: Annotated[ + int | None, + Field( + alias="textTopK", + description="Text-channel top-k cutoff; unset keeps the engine default.", + ge=0, + le=2147483647, + ), + ] = None + audio_temperature: Annotated[ + float | None, + Field( + alias="audioTemperature", + description="Audio-channel sampling temperature; unset keeps the engine default.", + ge=0.0, + ), + ] = None + audio_top_p: Annotated[ + float | None, + Field( + alias="audioTopP", + description="Audio-channel top-p (nucleus) cutoff, 0 < p ≤ 1; unset keeps the engine default.", + gt=0.0, + le=1.0, + ), + ] = None + audio_top_k: Annotated[ + int | None, + Field( + alias="audioTopK", + description="Audio-channel top-k cutoff; unset keeps the engine default.", + ge=0, + le=2147483647, + ), + ] = None + audio_repetition_penalty: Annotated[ + float | None, + Field( + alias="audioRepetitionPenalty", + description="Audio-channel repetition penalty; 1 = off, unset keeps the engine default.", + gt=0.0, + ), + ] = None use_gpu: Annotated[ bool | None, Field( @@ -20162,6 +20634,13 @@ class LoadModelSrcRequestTtsGgmlModelConfigMoss(GeneratedBaseModel): le=2147483647, ), ] = None + backend: Annotated[ + str | None, + Field( + description="Explicit native backend: `auto`, `cpu`, `opencl`, `webgpu`, or an exact registered device name such as `MTL0`, `CUDA0` or `Vulkan0` (device names depend on the host). An unavailable explicit backend fails the load; unset keeps the `useGPU` policy. WebGPU is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only.", + min_length=1, + ), + ] = None backends_dir: Annotated[ str | None, Field( @@ -20243,6 +20722,13 @@ class LoadModelSrcRequestTtsGgmlModelConfigMossSfx(GeneratedBaseModel): le=2147483647, ), ] = None + backend: Annotated[ + str | None, + Field( + description="Explicit native backend: `auto`, `cpu`, `opencl`, `webgpu`, or an exact registered device name such as `MTL0`, `CUDA0` or `Vulkan0` (device names depend on the host). An unavailable explicit backend fails the load; unset keeps the `useGPU` policy. WebGPU is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only.", + min_length=1, + ), + ] = None backends_dir: Annotated[ str | None, Field( @@ -20442,6 +20928,13 @@ class LoadModelSrcRequestTtsGgmlModelConfigMossSpeech(GeneratedBaseModel): le=2147483647, ), ] = None + backend: Annotated[ + str | None, + Field( + description="Explicit native backend: `auto`, `cpu`, `opencl`, `webgpu`, or an exact registered device name such as `MTL0`, `CUDA0` or `Vulkan0` (device names depend on the host). An unavailable explicit backend fails the load; unset keeps the `useGPU` policy. WebGPU is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only.", + min_length=1, + ), + ] = None backends_dir: Annotated[ str | None, Field( @@ -26277,7 +26770,13 @@ class TextToSpeechResponseStats(GeneratedBaseModel): total_samples: Annotated[float | None, Field(alias="totalSamples")] = None generated_frames: Annotated[float | None, Field(alias="generatedFrames")] = None backend_device: Annotated[float | None, Field(alias="backendDevice")] = None - backend_id: Annotated[float | None, Field(alias="backendId")] = None + backend_id: Annotated[ + float | None, + Field( + alias="backendId", + description="Backend family the engine ran on: `0` CPU, `1` Metal, `2` CUDA, `3` Vulkan, `4` OpenCL, `5` WebGPU, `99` other.", + ), + ] = None gpu_unsupported: Annotated[float | None, Field(alias="gpuUnsupported")] = None enhancer_backend_device: Annotated[ float | None, Field(alias="enhancerBackendDevice") @@ -26313,6 +26812,34 @@ class TextToSpeechResponseStats(GeneratedBaseModel): description="Audio8: `1` when this synthesis ran its codec on the Core ML sidecar, `0` when it ran on the ggml backend `backendId` reports.", ), ] = None + decoder_sidecar_loaded: Annotated[ + float | None, + Field( + alias="decoderSidecarLoaded", + description="Parler on macOS / iOS: `1` while the Core ML decoder sidecar is attached, `0` once a fallback retired it.", + ), + ] = None + dac_sidecar_loaded: Annotated[ + float | None, + Field( + alias="dacSidecarLoaded", + description="Parler on macOS / iOS: `1` while the Core ML DAC sidecar is attached, `0` once a fallback retired it.", + ), + ] = None + decoder_on_coreml: Annotated[ + float | None, + Field( + alias="decoderOnCoreml", + description="Parler: `1` when this synthesis ran its decoder steps on Core ML, `0` for ggml or a mixed run.", + ), + ] = None + dac_on_coreml: Annotated[ + float | None, + Field( + alias="dacOnCoreml", + description="Parler: `1` when this synthesis ran its DAC on Core ML, `0` for ggml or a mixed run.", + ), + ] = None t3_ms: Annotated[ float | None, Field( @@ -26549,6 +27076,13 @@ class TextToSpeechResponse(GeneratedBaseModel): description="True on the final audio-bearing chunk of a pre-chunked synthesis. Absent when the chunk count is not known up front (streamed text in).", ), ] = None + speech_token_ids: Annotated[ + list[TtsSpeechTokenId] | None, + Field( + alias="speechTokenIds", + description="CosyVoice3: the speech tokens of this native synthesis. Batch synthesis carries them with the PCM; native chunk streaming on a final `isLast` frame with no audio; the sentence paths once per sentence. Pass one array back as `forceSpeechTokens` to replay that synthesis (a joined sentence stream cannot be replayed as one).", + ), + ] = None stop_reason: Annotated[ TextToSpeechResponseStopReason | None, Field(alias="stopReason", title="TextToSpeechResponseStopReason"), @@ -26877,7 +27411,13 @@ class TextToSpeechStreamResponseStats(GeneratedBaseModel): total_samples: Annotated[float | None, Field(alias="totalSamples")] = None generated_frames: Annotated[float | None, Field(alias="generatedFrames")] = None backend_device: Annotated[float | None, Field(alias="backendDevice")] = None - backend_id: Annotated[float | None, Field(alias="backendId")] = None + backend_id: Annotated[ + float | None, + Field( + alias="backendId", + description="Backend family the engine ran on: `0` CPU, `1` Metal, `2` CUDA, `3` Vulkan, `4` OpenCL, `5` WebGPU, `99` other.", + ), + ] = None gpu_unsupported: Annotated[float | None, Field(alias="gpuUnsupported")] = None enhancer_backend_device: Annotated[ float | None, Field(alias="enhancerBackendDevice") @@ -26913,6 +27453,34 @@ class TextToSpeechStreamResponseStats(GeneratedBaseModel): description="Audio8: `1` when this synthesis ran its codec on the Core ML sidecar, `0` when it ran on the ggml backend `backendId` reports.", ), ] = None + decoder_sidecar_loaded: Annotated[ + float | None, + Field( + alias="decoderSidecarLoaded", + description="Parler on macOS / iOS: `1` while the Core ML decoder sidecar is attached, `0` once a fallback retired it.", + ), + ] = None + dac_sidecar_loaded: Annotated[ + float | None, + Field( + alias="dacSidecarLoaded", + description="Parler on macOS / iOS: `1` while the Core ML DAC sidecar is attached, `0` once a fallback retired it.", + ), + ] = None + decoder_on_coreml: Annotated[ + float | None, + Field( + alias="decoderOnCoreml", + description="Parler: `1` when this synthesis ran its decoder steps on Core ML, `0` for ggml or a mixed run.", + ), + ] = None + dac_on_coreml: Annotated[ + float | None, + Field( + alias="dacOnCoreml", + description="Parler: `1` when this synthesis ran its DAC on Core ML, `0` for ggml or a mixed run.", + ), + ] = None t3_ms: Annotated[ float | None, Field( @@ -27146,6 +27714,13 @@ class TextToSpeechStreamResponse(GeneratedBaseModel): description="True on the final audio-bearing chunk of a pre-chunked synthesis. Absent when the chunk count is not known up front (streamed text in).", ), ] = None + speech_token_ids: Annotated[ + list[TtsSpeechTokenId] | None, + Field( + alias="speechTokenIds", + description="CosyVoice3: the speech tokens of this native synthesis. Batch synthesis carries them with the PCM; native chunk streaming on a final `isLast` frame with no audio; the sentence paths once per sentence. Pass one array back as `forceSpeechTokens` to replay that synthesis (a joined sentence stream cannot be replayed as one).", + ), + ] = None stop_reason: Annotated[ TextToSpeechStreamResponseStopReason | None, Field(alias="stopReason", title="TextToSpeechStreamResponseStopReason"), diff --git a/packages/sdk-python/tests/test_tts_speech_tokens.py b/packages/sdk-python/tests/test_tts_speech_tokens.py new file mode 100644 index 00000000000..20ec8a70a57 --- /dev/null +++ b/packages/sdk-python/tests/test_tts_speech_tokens.py @@ -0,0 +1,68 @@ +"""CosyVoice3 speech tokens round-trip from a response into the replay config. + +The response models hand the trajectory out as `speech_token_ids`; the +CosyVoice3 load config takes it back as `force_speech_tokens`. Both sides must +share one scalar type, so the list a response returned feeds the replay +configuration as it is, with no per-element conversion. +""" + +import pytest +from pydantic import BaseModel + +from tetherto.qvac_sdk._generated.models import ( + LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3, + TextToSpeechResponse, + TextToSpeechStreamResponse, + TtsSpeechTokenId, +) + +RESPONSE_TYPES: list[type[BaseModel]] = [ + TextToSpeechResponse, + TextToSpeechStreamResponse, +] + + +def _frame(response_type: type[BaseModel], tokens: list[int]) -> BaseModel: + wire_type = ( + "textToSpeech" + if response_type is TextToSpeechResponse + else "textToSpeechStream" + ) + return response_type.model_validate( + {"type": wire_type, "buffer": [], "done": True, "speechTokenIds": tokens} + ) + + +@pytest.mark.parametrize("response_type", RESPONSE_TYPES) +def test_returned_speech_tokens_feed_the_replay_config_unchanged( + response_type: type[BaseModel], +) -> None: + frame = _frame(response_type, [12, 480, 3]) + returned = getattr(frame, "speech_token_ids") + assert returned is not None + assert all(isinstance(token, TtsSpeechTokenId) for token in returned) + + config = LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3( + force_speech_tokens=returned + ) + + assert config.force_speech_tokens == returned + wire = config.model_dump(mode="json", by_alias=True, exclude_unset=True) + assert wire["forceSpeechTokens"] == [12, 480, 3] + + +def test_replay_config_also_takes_plain_ints() -> None: + config = LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3(force_speech_tokens=[7, 8]) + wire = config.model_dump(mode="json", by_alias=True, exclude_unset=True) + assert wire["forceSpeechTokens"] == [7, 8] + + +@pytest.mark.parametrize("response_type", RESPONSE_TYPES) +def test_speech_tokens_share_one_type_with_the_replay_config( + response_type: type[BaseModel], +) -> None: + response_items = response_type.model_fields["speech_token_ids"].annotation + config_items = LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3.model_fields[ + "force_speech_tokens" + ].annotation + assert response_items == config_items diff --git a/packages/sdk/contract/schema.json b/packages/sdk/contract/schema.json index 1be010b9102..a3fa7a0347e 100644 --- a/packages/sdk/contract/schema.json +++ b/packages/sdk/contract/schema.json @@ -6038,12 +6038,24 @@ "MOSS_SPEECH" ] }, + "constants.TtsLavasrCoremlComputeUnits": { + "title": "TtsLavasrCoremlComputeUnits", + "type": "string", + "enum": ["CPU_ONLY", "CPU_AND_GPU", "CPU_AND_NE", "ALL"], + "x-enum-varnames": ["CPU_ONLY", "CPU_AND_GPU", "CPU_AND_NE", "ALL"] + }, "constants.TtsPace": { "title": "TtsPace", "type": "string", "enum": ["slow", "moderate", "fast"], "x-enum-varnames": ["SLOW", "MODERATE", "FAST"] }, + "constants.TtsParlerCoremlComputeUnits": { + "title": "TtsParlerCoremlComputeUnits", + "type": "string", + "enum": ["all", "cpu_and_ane", "cpu_and_gpu", "cpu_only"], + "x-enum-varnames": ["ALL", "CPU_AND_ANE", "CPU_AND_GPU", "CPU_ONLY"] + }, "constants.TtsParlerEmotion": { "title": "TtsParlerEmotion", "type": "string", @@ -6152,6 +6164,12 @@ "VI" ] }, + "constants.TtsSupertonicPrecision": { + "title": "TtsSupertonicPrecision", + "type": "string", + "enum": ["auto", "f32", "f16", "q8_0"], + "x-enum-varnames": ["AUTO", "F32", "F16", "Q8_0"] + }, "constants.VectorIndexStorage": { "title": "VectorIndexStorage", "type": "string", @@ -17101,6 +17119,11 @@ "minimum": -2147483648, "maximum": 2147483647 }, + "backend": { + "description": "Explicit native backend: `auto`, `cpu`, `opencl`, `webgpu`, or an exact registered device name such as `MTL0`, `CUDA0` or `Vulkan0` (device names depend on the host). An unavailable explicit backend fails the load; unset keeps the `useGPU` policy. WebGPU is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only.", + "type": "string", + "minLength": 1 + }, "backendsDir": { "description": "Directory scanned as given for dynamically-loaded ggml backend `.so` files. Unset scans the directory the native module was loaded from.", "type": "string", @@ -17705,6 +17728,28 @@ } ] }, + "lavasrEnhancerCoremlModelPath": { + "description": "macOS / iOS: compiled Core ML model directory (`.mlmodelc`) on the inference host for the LavaSR enhancer’s neural core; requires `lavasrEnhancerModelSrc` and `lavasrEnhancerCoremlComputeUnits`. A model that cannot load fails the load rather than falling back to ggml.", + "type": "string", + "minLength": 1 + }, + "lavasrEnhancerCoremlComputeUnits": { + "description": "macOS / iOS: Core ML compute policy for the LavaSR enhancer; required with `lavasrEnhancerCoremlModelPath`. Stats then report the stage as device `2` (Core ML-managed), backend `99`.", + "type": "string", + "enum": ["CPU_ONLY", "CPU_AND_GPU", "CPU_AND_NE", "ALL"], + "title": "LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrEnhancerCoremlComputeUnits" + }, + "lavasrDenoiserCoremlModelPath": { + "description": "macOS / iOS: compiled Core ML model directory (`.mlmodelc`) on the inference host for the LavaSR denoiser’s neural core; requires `lavasrDenoiserModelSrc` and `lavasrDenoiserCoremlComputeUnits`. A model that cannot load fails the load rather than falling back to ggml.", + "type": "string", + "minLength": 1 + }, + "lavasrDenoiserCoremlComputeUnits": { + "description": "macOS / iOS: Core ML compute policy for the LavaSR denoiser; required with `lavasrDenoiserCoremlModelPath`. Stats then report the stage as device `2` (Core ML-managed), backend `99`.", + "type": "string", + "enum": ["CPU_ONLY", "CPU_AND_GPU", "CPU_AND_NE", "ALL"], + "title": "LoadModelSrcRequestTtsGgmlModelConfigChatterboxLavasrDenoiserCoremlComputeUnits" + }, "ttsSupertonicMultilingual": {}, "ttsTokenizerSrc": {}, "ttsSpeechEncoderSrc": {}, @@ -17858,6 +17903,70 @@ "minimum": 0, "maximum": 2147483647 }, + "precision": { + "description": "Weight precision the engine loads at: `auto` keeps the GGUF’s own, `f32` / `f16` / `q8_0` convert on load and change the weight footprint.", + "type": "string", + "enum": ["auto", "f32", "f16", "q8_0"], + "title": "LoadModelSrcRequestTtsGgmlModelConfigSupertonicPrecision" + }, + "f16Attn": { + "description": "Attention precision override: `-1` automatic (default), `0` f32, `1` f16.", + "type": "integer", + "minimum": -2147483648, + "maximum": 2147483647 + }, + "f16Weights": { + "description": "Weight precision override: `-1` automatic (default), `0` off, `1` f16.", + "type": "integer", + "minimum": -2147483648, + "maximum": 2147483647 + }, + "kvAttnType": { + "description": "Native KV attention type code; `-1` leaves the automatic selection.", + "type": "integer", + "minimum": -2147483648, + "maximum": 2147483647 + }, + "f16WeightsDenyList": { + "description": "Tensor-name patterns kept at their original precision when `f16Weights` is on.", + "type": "array", + "items": { + "type": "string", + "minLength": 1 + } + }, + "vulkanEnvOverrides": { + "description": "Vulkan environment overrides (string values) passed to the engine; explicit entries win over the `vulkanCacheDir` convenience setting.", + "type": "object", + "propertyNames": { + "type": "string" + }, + "additionalProperties": { + "type": "string" + }, + "title": "LoadModelSrcRequestTtsGgmlModelConfigSupertonicVulkanEnvOverrides" + }, + "voiceStyleTtl": { + "description": "In-memory voice: the flattened `style_ttl` tensor of a cloned voice. Paired with `voiceStyleDp`; the engine checks the sizes against the model, and the pair wins over `voiceJsonSrc` and the baked `voice`.", + "minItems": 1, + "type": "array", + "items": { + "type": "number" + } + }, + "voiceStyleDp": { + "description": "In-memory voice: the flattened `style_dp` tensor, paired with `voiceStyleTtl`.", + "minItems": 1, + "type": "array", + "items": { + "type": "number" + } + }, + "backend": { + "description": "Explicit native backend: `auto`, `cpu`, `opencl`, `webgpu`, or an exact registered device name such as `MTL0`, `CUDA0` or `Vulkan0` (device names depend on the host). An unavailable explicit backend fails the load; unset keeps the `useGPU` policy. WebGPU is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only.", + "type": "string", + "minLength": 1 + }, "backendsDir": { "description": "Directory scanned as given for dynamically-loaded ggml backend `.so` files. Unset scans the directory the native module was loaded from.", "type": "string", @@ -18066,6 +18175,28 @@ } ] }, + "lavasrEnhancerCoremlModelPath": { + "description": "macOS / iOS: compiled Core ML model directory (`.mlmodelc`) on the inference host for the LavaSR enhancer’s neural core; requires `lavasrEnhancerModelSrc` and `lavasrEnhancerCoremlComputeUnits`. A model that cannot load fails the load rather than falling back to ggml.", + "type": "string", + "minLength": 1 + }, + "lavasrEnhancerCoremlComputeUnits": { + "description": "macOS / iOS: Core ML compute policy for the LavaSR enhancer; required with `lavasrEnhancerCoremlModelPath`. Stats then report the stage as device `2` (Core ML-managed), backend `99`.", + "type": "string", + "enum": ["CPU_ONLY", "CPU_AND_GPU", "CPU_AND_NE", "ALL"], + "title": "LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrEnhancerCoremlComputeUnits" + }, + "lavasrDenoiserCoremlModelPath": { + "description": "macOS / iOS: compiled Core ML model directory (`.mlmodelc`) on the inference host for the LavaSR denoiser’s neural core; requires `lavasrDenoiserModelSrc` and `lavasrDenoiserCoremlComputeUnits`. A model that cannot load fails the load rather than falling back to ggml.", + "type": "string", + "minLength": 1 + }, + "lavasrDenoiserCoremlComputeUnits": { + "description": "macOS / iOS: Core ML compute policy for the LavaSR denoiser; required with `lavasrDenoiserCoremlModelPath`. Stats then report the stage as device `2` (Core ML-managed), backend `99`.", + "type": "string", + "enum": ["CPU_ONLY", "CPU_AND_GPU", "CPU_AND_NE", "ALL"], + "title": "LoadModelSrcRequestTtsGgmlModelConfigSupertonicLavasrDenoiserCoremlComputeUnits" + }, "voiceJsonSrc": { "description": "External voice JSON source (`{ style_ttl, style_dp }`, e.g. a cloned voice) that overrides the baked `voice` preset; the engine checks the tensor sizes against the loaded model.", "anyOf": [ @@ -18436,6 +18567,25 @@ "description": "Parler prompt digit expansion (engine default: enabled).", "type": "boolean" }, + "coremlDecoder": { + "description": "macOS / iOS: run the decoder steps on the `-decoder.mlmodelc` sidecar beside the GGUF (the DAC sidecar is picked up by presence); unset keeps the engine default.", + "type": "boolean" + }, + "coremlComputeUnits": { + "description": "macOS / iOS: Core ML compute policy for the Parler sidecars; unset inherits the engine’s environment default.", + "type": "string", + "enum": ["all", "cpu_and_ane", "cpu_and_gpu", "cpu_only"], + "title": "LoadModelSrcRequestTtsGgmlModelConfigParlerCoremlComputeUnits" + }, + "coremlStrict": { + "description": "macOS / iOS: `true` fails the load when a sidecar is unusable instead of falling back to ggml, `false` permits the fallback; unset inherits the engine’s strict mode.", + "type": "boolean" + }, + "backend": { + "description": "Explicit native backend: `auto`, `cpu`, `opencl`, `webgpu`, or an exact registered device name such as `MTL0`, `CUDA0` or `Vulkan0` (device names depend on the host). An unavailable explicit backend fails the load; unset keeps the `useGPU` policy. WebGPU is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only.", + "type": "string", + "minLength": 1 + }, "backendsDir": { "description": "Directory scanned as given for dynamically-loaded ggml backend `.so` files. Unset scans the directory the native module was loaded from.", "type": "string", @@ -18638,6 +18788,28 @@ "title": "LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrDenoiserModelSrc" } ] + }, + "lavasrEnhancerCoremlModelPath": { + "description": "macOS / iOS: compiled Core ML model directory (`.mlmodelc`) on the inference host for the LavaSR enhancer’s neural core; requires `lavasrEnhancerModelSrc` and `lavasrEnhancerCoremlComputeUnits`. A model that cannot load fails the load rather than falling back to ggml.", + "type": "string", + "minLength": 1 + }, + "lavasrEnhancerCoremlComputeUnits": { + "description": "macOS / iOS: Core ML compute policy for the LavaSR enhancer; required with `lavasrEnhancerCoremlModelPath`. Stats then report the stage as device `2` (Core ML-managed), backend `99`.", + "type": "string", + "enum": ["CPU_ONLY", "CPU_AND_GPU", "CPU_AND_NE", "ALL"], + "title": "LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrEnhancerCoremlComputeUnits" + }, + "lavasrDenoiserCoremlModelPath": { + "description": "macOS / iOS: compiled Core ML model directory (`.mlmodelc`) on the inference host for the LavaSR denoiser’s neural core; requires `lavasrDenoiserModelSrc` and `lavasrDenoiserCoremlComputeUnits`. A model that cannot load fails the load rather than falling back to ggml.", + "type": "string", + "minLength": 1 + }, + "lavasrDenoiserCoremlComputeUnits": { + "description": "macOS / iOS: Core ML compute policy for the LavaSR denoiser; required with `lavasrDenoiserCoremlModelPath`. Stats then report the stage as device `2` (Core ML-managed), backend `99`.", + "type": "string", + "enum": ["CPU_ONLY", "CPU_AND_GPU", "CPU_AND_NE", "ALL"], + "title": "LoadModelSrcRequestTtsGgmlModelConfigParlerLavasrDenoiserCoremlComputeUnits" } }, "required": ["ttsEngine"], @@ -18731,6 +18903,20 @@ "minimum": -1, "maximum": 2147483647 }, + "greedy": { + "description": "Greedy LM decoding: take the argmax speech token instead of sampling (engine default off).", + "type": "boolean" + }, + "forceSpeechTokens": { + "description": "Speech-token trajectory of ONE native synthesis to replay instead of generating, as a previous batch run, native chunk stream or single sentence returned it in `speechTokenIds`. Fixed at load: every synthesis of the model replays this same array, so a joined sentence stream cannot be replayed sentence by sentence. An empty array restores generation.", + "type": "array", + "items": { + "type": "integer", + "minimum": -2147483648, + "maximum": 2147483647, + "title": "TtsSpeechTokenId" + } + }, "useGPU": { "description": "Route inference through a GPU backend (Metal / CUDA / Vulkan / OpenCL) when available. Default false.", "type": "boolean" @@ -18771,6 +18957,11 @@ "minimum": -2147483648, "maximum": 2147483647 }, + "backend": { + "description": "Explicit native backend: `auto`, `cpu`, `opencl`, `webgpu`, or an exact registered device name such as `MTL0`, `CUDA0` or `Vulkan0` (device names depend on the host). An unavailable explicit backend fails the load; unset keeps the `useGPU` policy. WebGPU is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only.", + "type": "string", + "minLength": 1 + }, "backendsDir": { "description": "Directory scanned as given for dynamically-loaded ggml backend `.so` files. Unset scans the directory the native module was loaded from.", "type": "string", @@ -18979,6 +19170,28 @@ } ] }, + "lavasrEnhancerCoremlModelPath": { + "description": "macOS / iOS: compiled Core ML model directory (`.mlmodelc`) on the inference host for the LavaSR enhancer’s neural core; requires `lavasrEnhancerModelSrc` and `lavasrEnhancerCoremlComputeUnits`. A model that cannot load fails the load rather than falling back to ggml.", + "type": "string", + "minLength": 1 + }, + "lavasrEnhancerCoremlComputeUnits": { + "description": "macOS / iOS: Core ML compute policy for the LavaSR enhancer; required with `lavasrEnhancerCoremlModelPath`. Stats then report the stage as device `2` (Core ML-managed), backend `99`.", + "type": "string", + "enum": ["CPU_ONLY", "CPU_AND_GPU", "CPU_AND_NE", "ALL"], + "title": "LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrEnhancerCoremlComputeUnits" + }, + "lavasrDenoiserCoremlModelPath": { + "description": "macOS / iOS: compiled Core ML model directory (`.mlmodelc`) on the inference host for the LavaSR denoiser’s neural core; requires `lavasrDenoiserModelSrc` and `lavasrDenoiserCoremlComputeUnits`. A model that cannot load fails the load rather than falling back to ggml.", + "type": "string", + "minLength": 1 + }, + "lavasrDenoiserCoremlComputeUnits": { + "description": "macOS / iOS: Core ML compute policy for the LavaSR denoiser; required with `lavasrDenoiserCoremlModelPath`. Stats then report the stage as device `2` (Core ML-managed), backend `99`.", + "type": "string", + "enum": ["CPU_ONLY", "CPU_AND_GPU", "CPU_AND_NE", "ALL"], + "title": "LoadModelSrcRequestTtsGgmlModelConfigCosyvoice3LavasrDenoiserCoremlComputeUnits" + }, "referenceAudioSrc": { "description": "CosyVoice3 voice-cloning reference recording source (wav; 0.5–30 s, 5–15 s of clean speech recommended). Replaces the baked voice.", "anyOf": [ @@ -19349,6 +19562,11 @@ "minimum": -2147483648, "maximum": 2147483647 }, + "backend": { + "description": "Explicit native backend: `auto`, `cpu`, `opencl`, `webgpu`, or an exact registered device name such as `MTL0`, `CUDA0` or `Vulkan0` (device names depend on the host). An unavailable explicit backend fails the load; unset keeps the `useGPU` policy. WebGPU is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only.", + "type": "string", + "minLength": 1 + }, "backendsDir": { "description": "Directory scanned as given for dynamically-loaded ggml backend `.so` files. Unset scans the directory the native module was loaded from.", "type": "string", @@ -19670,10 +19888,10 @@ "minLength": 1 }, "durationTokens": { - "description": "Target length of each synthesis in codec frames (12.5 per second, so 38 ≈ 3 s), 0–2015; 0 or unset keeps the length free.", + "description": "Target length of each synthesis in codec frames (12.5 per second, so 38 ≈ 3 s), at most `maxNewTokens` − 33 (2015 by default); 0 or unset keeps the length free.", "type": "integer", "minimum": 0, - "maximum": 2015 + "maximum": 2147483647 }, "streamChunkTokens": { "description": "Codec frames per native streaming chunk (12.5 per second, so 25 ≈ 2 s); 0 or unset synthesizes the whole text before emitting.", @@ -19681,6 +19899,57 @@ "minimum": 0, "maximum": 2147483647 }, + "nCtx": { + "description": "Context capacity of the backbone in rows; unset keeps the native value.", + "type": "integer", + "exclusiveMinimum": 0, + "maximum": 2147483647 + }, + "maxNewTokens": { + "description": "Generation budget in rows (engine default 2048), of which 33 are delay/EOS rows; bounds `durationTokens`.", + "type": "integer", + "exclusiveMinimum": 0, + "maximum": 2147483647 + }, + "textTemperature": { + "description": "Text-channel sampling temperature; unset keeps the engine default.", + "type": "number", + "minimum": 0 + }, + "textTopP": { + "description": "Text-channel top-p (nucleus) cutoff, 0 < p ≤ 1; unset keeps the engine default.", + "type": "number", + "exclusiveMinimum": 0, + "maximum": 1 + }, + "textTopK": { + "description": "Text-channel top-k cutoff; unset keeps the engine default.", + "type": "integer", + "minimum": 0, + "maximum": 2147483647 + }, + "audioTemperature": { + "description": "Audio-channel sampling temperature; unset keeps the engine default.", + "type": "number", + "minimum": 0 + }, + "audioTopP": { + "description": "Audio-channel top-p (nucleus) cutoff, 0 < p ≤ 1; unset keeps the engine default.", + "type": "number", + "exclusiveMinimum": 0, + "maximum": 1 + }, + "audioTopK": { + "description": "Audio-channel top-k cutoff; unset keeps the engine default.", + "type": "integer", + "minimum": 0, + "maximum": 2147483647 + }, + "audioRepetitionPenalty": { + "description": "Audio-channel repetition penalty; 1 = off, unset keeps the engine default.", + "type": "number", + "exclusiveMinimum": 0 + }, "useGPU": { "description": "Route inference through a GPU backend (Metal / CUDA / Vulkan / OpenCL) when available. Default false.", "type": "boolean" @@ -19703,6 +19972,11 @@ "minimum": -2147483648, "maximum": 2147483647 }, + "backend": { + "description": "Explicit native backend: `auto`, `cpu`, `opencl`, `webgpu`, or an exact registered device name such as `MTL0`, `CUDA0` or `Vulkan0` (device names depend on the host). An unavailable explicit backend fails the load; unset keeps the `useGPU` policy. WebGPU is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only.", + "type": "string", + "minLength": 1 + }, "backendsDir": { "description": "Directory scanned as given for dynamically-loaded ggml backend `.so` files. Unset scans the directory the native module was loaded from.", "type": "string", @@ -20144,6 +20418,11 @@ "minimum": -2147483648, "maximum": 2147483647 }, + "backend": { + "description": "Explicit native backend: `auto`, `cpu`, `opencl`, `webgpu`, or an exact registered device name such as `MTL0`, `CUDA0` or `Vulkan0` (device names depend on the host). An unavailable explicit backend fails the load; unset keeps the `useGPU` policy. WebGPU is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only.", + "type": "string", + "minLength": 1 + }, "backendsDir": { "description": "Directory scanned as given for dynamically-loaded ggml backend `.so` files. Unset scans the directory the native module was loaded from.", "type": "string", @@ -20212,6 +20491,11 @@ "minimum": -2147483648, "maximum": 2147483647 }, + "backend": { + "description": "Explicit native backend: `auto`, `cpu`, `opencl`, `webgpu`, or an exact registered device name such as `MTL0`, `CUDA0` or `Vulkan0` (device names depend on the host). An unavailable explicit backend fails the load; unset keeps the `useGPU` policy. WebGPU is available to Supertonic 3, Parler, CosyVoice3 and Audio8 only.", + "type": "string", + "minLength": 1 + }, "backendsDir": { "description": "Directory scanned as given for dynamically-loaded ggml backend `.so` files. Unset scans the directory the native module was loaded from.", "type": "string", @@ -25965,6 +26249,7 @@ "type": "number" }, "backendId": { + "description": "Backend family the engine ran on: `0` CPU, `1` Metal, `2` CUDA, `3` Vulkan, `4` OpenCL, `5` WebGPU, `99` other.", "type": "number" }, "gpuUnsupported": { @@ -25992,6 +26277,22 @@ "description": "Audio8: `1` when this synthesis ran its codec on the Core ML sidecar, `0` when it ran on the ggml backend `backendId` reports.", "type": "number" }, + "decoderSidecarLoaded": { + "description": "Parler on macOS / iOS: `1` while the Core ML decoder sidecar is attached, `0` once a fallback retired it.", + "type": "number" + }, + "dacSidecarLoaded": { + "description": "Parler on macOS / iOS: `1` while the Core ML DAC sidecar is attached, `0` once a fallback retired it.", + "type": "number" + }, + "decoderOnCoreml": { + "description": "Parler: `1` when this synthesis ran its decoder steps on Core ML, `0` for ggml or a mixed run.", + "type": "number" + }, + "dacOnCoreml": { + "description": "Parler: `1` when this synthesis ran its DAC on Core ML, `0` for ggml or a mixed run.", + "type": "number" + }, "t3Ms": { "description": "Chatterbox T3 decode wall time of the last synthesis, in milliseconds.", "type": "number" @@ -26116,6 +26417,16 @@ "description": "True on the final audio-bearing chunk of a pre-chunked synthesis. Absent when the chunk count is not known up front (streamed text in).", "type": "boolean" }, + "speechTokenIds": { + "description": "CosyVoice3: the speech tokens of this native synthesis. Batch synthesis carries them with the PCM; native chunk streaming on a final `isLast` frame with no audio; the sentence paths once per sentence. Pass one array back as `forceSpeechTokens` to replay that synthesis (a joined sentence stream cannot be replayed as one).", + "type": "array", + "items": { + "type": "integer", + "minimum": -2147483648, + "maximum": 2147483647, + "title": "TtsSpeechTokenId" + } + }, "stopReason": { "type": "string", "enum": ["completed", "cancelled"], @@ -26442,6 +26753,7 @@ "type": "number" }, "backendId": { + "description": "Backend family the engine ran on: `0` CPU, `1` Metal, `2` CUDA, `3` Vulkan, `4` OpenCL, `5` WebGPU, `99` other.", "type": "number" }, "gpuUnsupported": { @@ -26469,6 +26781,22 @@ "description": "Audio8: `1` when this synthesis ran its codec on the Core ML sidecar, `0` when it ran on the ggml backend `backendId` reports.", "type": "number" }, + "decoderSidecarLoaded": { + "description": "Parler on macOS / iOS: `1` while the Core ML decoder sidecar is attached, `0` once a fallback retired it.", + "type": "number" + }, + "dacSidecarLoaded": { + "description": "Parler on macOS / iOS: `1` while the Core ML DAC sidecar is attached, `0` once a fallback retired it.", + "type": "number" + }, + "decoderOnCoreml": { + "description": "Parler: `1` when this synthesis ran its decoder steps on Core ML, `0` for ggml or a mixed run.", + "type": "number" + }, + "dacOnCoreml": { + "description": "Parler: `1` when this synthesis ran its DAC on Core ML, `0` for ggml or a mixed run.", + "type": "number" + }, "t3Ms": { "description": "Chatterbox T3 decode wall time of the last synthesis, in milliseconds.", "type": "number" @@ -26593,6 +26921,16 @@ "description": "True on the final audio-bearing chunk of a pre-chunked synthesis. Absent when the chunk count is not known up front (streamed text in).", "type": "boolean" }, + "speechTokenIds": { + "description": "CosyVoice3: the speech tokens of this native synthesis. Batch synthesis carries them with the PCM; native chunk streaming on a final `isLast` frame with no audio; the sentence paths once per sentence. Pass one array back as `forceSpeechTokens` to replay that synthesis (a joined sentence stream cannot be replayed as one).", + "type": "array", + "items": { + "type": "integer", + "minimum": -2147483648, + "maximum": 2147483647, + "title": "TtsSpeechTokenId" + } + }, "stopReason": { "type": "string", "enum": ["completed", "cancelled"], diff --git a/packages/sdk/scripts/contract/constants-registry.ts b/packages/sdk/scripts/contract/constants-registry.ts index b352e7f89ff..8532573ef1e 100644 --- a/packages/sdk/scripts/contract/constants-registry.ts +++ b/packages/sdk/scripts/contract/constants-registry.ts @@ -20,6 +20,9 @@ import { TTS_COSYVOICE3_INSTRUCT_DIALECTS, TTS_COSYVOICE3_INSTRUCT_VOLUMES, TTS_COSYVOICE3_INSTRUCT_STYLES, + TTS_PARLER_COREML_COMPUTE_UNITS, + TTS_LAVASR_COREML_COMPUTE_UNITS, + TTS_SUPERTONIC_PRECISIONS, VectorIndexStorage } from '@qvac/inference/surface' import { @@ -116,6 +119,9 @@ export const constantsRegistry = { TtsCosyvoice3InstructDialect: enumFromVocabulary(TTS_COSYVOICE3_INSTRUCT_DIALECTS), TtsCosyvoice3InstructVolume: enumFromVocabulary(TTS_COSYVOICE3_INSTRUCT_VOLUMES), TtsCosyvoice3InstructStyle: enumFromVocabulary(TTS_COSYVOICE3_INSTRUCT_STYLES), + TtsParlerCoremlComputeUnits: enumFromVocabulary(TTS_PARLER_COREML_COMPUTE_UNITS), + TtsLavasrCoremlComputeUnits: enumFromVocabulary(TTS_LAVASR_COREML_COMPUTE_UNITS), + TtsSupertonicPrecision: enumFromVocabulary(TTS_SUPERTONIC_PRECISIONS), AudioGenEngine: enumFromVocabulary(AUDIOGEN_ENGINES), AudioGenTaskType: enumFromVocabulary(AUDIOGEN_TASK_TYPES), AudioGenEditOperation: enumFromVocabulary(AUDIOGEN_EDIT_OPERATIONS), diff --git a/packages/sdk/src/client/api/text-to-speech.ts b/packages/sdk/src/client/api/text-to-speech.ts index 4b2ea8058a2..ea2ba154be7 100644 --- a/packages/sdk/src/client/api/text-to-speech.ts +++ b/packages/sdk/src/client/api/text-to-speech.ts @@ -307,6 +307,7 @@ export type TtsSideChannel = { sampleRate: Promise stats: Promise stopReason: Promise<'completed' | 'cancelled' | undefined> + speechTokenIds: Promise observe(response: TtsResponse): void settle(): void } @@ -321,6 +322,7 @@ export function createTtsSideChannel(): TtsSideChannel { let resolveSampleRate!: (value: number | undefined) => void let resolveStats!: (value: TtsStats | undefined) => void let resolveStopReason!: (value: 'completed' | 'cancelled' | undefined) => void + let resolveSpeechTokenIds!: (value: number[] | undefined) => void const sampleRate = new Promise((resolve) => { resolveSampleRate = resolve }) @@ -330,10 +332,16 @@ export function createTtsSideChannel(): TtsSideChannel { const stopReason = new Promise<'completed' | 'cancelled' | undefined>((resolve) => { resolveStopReason = resolve }) + const speechTokenIds = new Promise((resolve) => { + resolveSpeechTokenIds = resolve + }) let seenSampleRate: number | undefined let latestStats: TtsStats | undefined let seenStopReason: 'completed' | 'cancelled' | undefined + // CosyVoice3 hands back one array per native synthesis; the chunked paths + // run one per sentence, so the run's trajectory is their concatenation. + let seenSpeechTokenIds: number[] | undefined let settled = false return { @@ -341,6 +349,7 @@ export function createTtsSideChannel(): TtsSideChannel { sampleRate, stats, stopReason, + speechTokenIds, observe(response) { if (response.text !== undefined) seenText = response.text // Resolve on the FIRST frame that carries a rate, not at end of stream: @@ -353,6 +362,12 @@ export function createTtsSideChannel(): TtsSideChannel { } if (response.stats !== undefined) latestStats = response.stats if (response.stopReason !== undefined) seenStopReason = response.stopReason + if (response.speechTokenIds !== undefined) { + // Append into one owned array: a spread per frame would copy every + // token collected so far on each sentence of a long stream. + seenSpeechTokenIds ??= [] + for (const id of response.speechTokenIds) seenSpeechTokenIds.push(id) + } }, settle() { if (settled) return @@ -361,6 +376,7 @@ export function createTtsSideChannel(): TtsSideChannel { resolveSampleRate(seenSampleRate) resolveStats(latestStats) resolveStopReason(seenStopReason) + resolveSpeechTokenIds(seenSpeechTokenIds) } } } @@ -389,7 +405,8 @@ function sentenceStreamTts( text: side.text, sampleRate: side.sampleRate, stats: side.stats, - stopReason: side.stopReason + stopReason: side.stopReason, + speechTokenIds: side.speechTokenIds } } @@ -427,7 +444,8 @@ async function* sentenceChunkUpdates( const hasAudio = m.buffer.length > 0 const hasMeta = m.chunkIndex !== undefined || - (typeof m.sentenceChunk === 'string' && m.sentenceChunk.length > 0) + (typeof m.sentenceChunk === 'string' && m.sentenceChunk.length > 0) || + m.speechTokenIds !== undefined if (hasAudio || hasMeta) { yield { buffer: hasAudio ? [...m.buffer] : [], @@ -436,7 +454,8 @@ async function* sentenceChunkUpdates( ...(typeof m.sentenceChunk === 'string' && m.sentenceChunk.length > 0 ? { sentenceChunk: m.sentenceChunk } : {}), - ...(m.isLast !== undefined ? { isLast: m.isLast } : {}) + ...(m.isLast !== undefined ? { isLast: m.isLast } : {}), + ...(m.speechTokenIds !== undefined ? { speechTokenIds: m.speechTokenIds } : {}) } } if (m.done) break @@ -467,7 +486,8 @@ function plainStreamTts( text: side.text, sampleRate: side.sampleRate, stats: side.stats, - stopReason: side.stopReason + stopReason: side.stopReason, + speechTokenIds: side.speechTokenIds } } @@ -534,7 +554,8 @@ function collectTts( text: side.text, sampleRate: side.sampleRate, stats: side.stats, - stopReason: side.stopReason + stopReason: side.stopReason, + speechTokenIds: side.speechTokenIds } } diff --git a/packages/sdk/src/index.ts b/packages/sdk/src/index.ts index f5958c0c25a..ec90b8b9c15 100644 --- a/packages/sdk/src/index.ts +++ b/packages/sdk/src/index.ts @@ -169,6 +169,9 @@ export { TTS_COSYVOICE3_INSTRUCT_DIALECTS, TTS_COSYVOICE3_INSTRUCT_VOLUMES, TTS_COSYVOICE3_INSTRUCT_STYLES, + TTS_PARLER_COREML_COMPUTE_UNITS, + TTS_LAVASR_COREML_COMPUTE_UNITS, + TTS_SUPERTONIC_PRECISIONS, type TtsPace, type TtsCosyvoice3Emotion, type TtsCosyvoice3Instruct, diff --git a/packages/sdk/test/tts-constants-exports.test.ts b/packages/sdk/test/tts-constants-exports.test.ts index 108b3894fa3..95995bf348f 100644 --- a/packages/sdk/test/tts-constants-exports.test.ts +++ b/packages/sdk/test/tts-constants-exports.test.ts @@ -9,7 +9,10 @@ import { TTS_COSYVOICE3_EMOTIONS, TTS_COSYVOICE3_INSTRUCT_DIALECTS, TTS_COSYVOICE3_INSTRUCT_VOLUMES, - TTS_COSYVOICE3_INSTRUCT_STYLES + TTS_COSYVOICE3_INSTRUCT_STYLES, + TTS_PARLER_COREML_COMPUTE_UNITS, + TTS_LAVASR_COREML_COMPUTE_UNITS, + TTS_SUPERTONIC_PRECISIONS } from '@/index' import * as ttsSchemas from '@qvac/inference/surface' import { constantsRegistry } from '../scripts/contract/constants-registry' @@ -87,3 +90,26 @@ test('TTS vocabulary constants are registered for cross-language codegen', (t) = // 'proper noun' must survive as an identifier-safe varname for codegen. t.is(constantsRegistry.TtsParlerEmotion.enum['PROPER_NOUN'], 'proper noun') }) + +// The @qvac/tts-ggml 0.15 vocabularies: the Parler and LavaSR Core ML compute +// policies (two spellings, each mirroring its own native enum) and the +// Supertonic weight precision. +test('Core ML and precision vocabularies are exported and registered', (t) => { + t.is(TTS_PARLER_COREML_COMPUTE_UNITS, ttsSchemas.TTS_PARLER_COREML_COMPUTE_UNITS) + t.is(TTS_LAVASR_COREML_COMPUTE_UNITS, ttsSchemas.TTS_LAVASR_COREML_COMPUTE_UNITS) + t.is(TTS_SUPERTONIC_PRECISIONS, ttsSchemas.TTS_SUPERTONIC_PRECISIONS) + t.alike([...TTS_PARLER_COREML_COMPUTE_UNITS], ['all', 'cpu_and_ane', 'cpu_and_gpu', 'cpu_only']) + t.alike([...TTS_LAVASR_COREML_COMPUTE_UNITS], ['CPU_ONLY', 'CPU_AND_GPU', 'CPU_AND_NE', 'ALL']) + t.alike([...TTS_SUPERTONIC_PRECISIONS], ['auto', 'f32', 'f16', 'q8_0']) + t.alike(Object.values(constantsRegistry.TtsParlerCoremlComputeUnits.enum), [ + ...TTS_PARLER_COREML_COMPUTE_UNITS + ]) + t.alike(Object.values(constantsRegistry.TtsLavasrCoremlComputeUnits.enum), [ + ...TTS_LAVASR_COREML_COMPUTE_UNITS + ]) + t.alike(Object.values(constantsRegistry.TtsSupertonicPrecision.enum), [ + ...TTS_SUPERTONIC_PRECISIONS + ]) + t.is(constantsRegistry.TtsParlerCoremlComputeUnits.enum['CPU_AND_ANE'], 'cpu_and_ane') + t.is(constantsRegistry.TtsSupertonicPrecision.enum['Q8_0'], 'q8_0') +}) diff --git a/packages/sdk/test/tts-speech-token-ids.test.ts b/packages/sdk/test/tts-speech-token-ids.test.ts new file mode 100644 index 00000000000..6993ae61426 --- /dev/null +++ b/packages/sdk/test/tts-speech-token-ids.test.ts @@ -0,0 +1,119 @@ +import { mock, test } from 'node:test' +import assert from 'node:assert/strict' +import { encodeTtsPcm } from '@qvac/inference/surface' + +// CosyVoice3 hands its speech-token trajectory back on the frames: with the PCM +// in batch synthesis, on a final empty `isLast` frame under native chunk +// streaming. The client exposes it as `speechTokenIds` on the result (the +// concatenation of every array the run carried) and on the per-sentence +// `chunkUpdates`, so a caller can replay it through `forceSpeechTokens`. + +let scripted: Array> = [] +mock.module('@/client/rpc/rpc-client', { + namedExports: { + stream: async function* () { + yield* scripted + }, + duplex: async () => { + throw new Error('unused') + } + } +}) + +const { textToSpeech } = await import('@/client/api/text-to-speech') + +test('batch synthesis resolves speechTokenIds beside the buffer', async () => { + scripted = [ + { + type: 'textToSpeech', + buffer: [], + pcm: encodeTtsPcm([10, 20, 30, 40]), + done: false, + sampleRate: 24000, + speechTokenIds: [12, 480, 3] + }, + { type: 'textToSpeech', buffer: [], done: true, stopReason: 'completed' } + ] + const run = textToSpeech({ modelId: 'cosyvoice3', text: 'Hello.', stream: false }) + assert.deepEqual(await run.buffer, [10, 20, 30, 40]) + assert.deepEqual(await run.speechTokenIds, [12, 480, 3]) +}) + +test('plain streaming resolves the tokens from the final empty frame, which yields no chunk', async () => { + scripted = [ + { + type: 'textToSpeech', + buffer: [], + pcm: encodeTtsPcm([10, 20]), + done: false, + sampleRate: 24000 + }, + { type: 'textToSpeech', buffer: [], pcm: encodeTtsPcm([30, 40]), done: false }, + { type: 'textToSpeech', buffer: [], done: false, isLast: true, speechTokenIds: [12, 480, 3] }, + { type: 'textToSpeech', buffer: [], done: true, stopReason: 'completed' } + ] + const run = textToSpeech({ modelId: 'cosyvoice3', text: 'Hello.', stream: true }) + const chunks: number[][] = [] + for await (const chunk of run.chunkStream) chunks.push(Array.from(chunk)) + assert.deepEqual(chunks, [ + [10, 20], + [30, 40] + ]) + assert.deepEqual(await run.speechTokenIds, [12, 480, 3]) +}) + +test('sentence streaming carries each sentence’s tokens on chunkUpdates and concatenates them', async () => { + scripted = [ + { + type: 'textToSpeech', + buffer: [], + pcm: encodeTtsPcm([1, 2]), + done: false, + chunkIndex: 0, + sentenceChunk: 'One.', + speechTokenIds: [12, 480] + }, + { + type: 'textToSpeech', + buffer: [], + pcm: encodeTtsPcm([3, 4]), + done: false, + chunkIndex: 1, + sentenceChunk: 'Two.', + isLast: true, + speechTokenIds: [3] + }, + { type: 'textToSpeech', buffer: [], done: true, stopReason: 'completed' } + ] + const run = textToSpeech({ + modelId: 'cosyvoice3', + text: 'One. Two.', + stream: true, + sentenceStream: true + }) + const updates = [] + for await (const update of run.chunkUpdates!) updates.push(update) + assert.deepEqual( + updates.map((u) => u.speechTokenIds), + [[12, 480], [3]] + ) + const trajectory = await run.speechTokenIds + assert.deepEqual(trajectory, [12, 480, 3]) + // The run's trajectory grows in one owned array; the per-sentence arrays + // handed out on chunkUpdates are neither aliased nor appended to. + assert.notEqual(trajectory, updates[0].speechTokenIds) + assert.deepEqual( + updates.map((u) => u.speechTokenIds), + [[12, 480], [3]] + ) +}) + +test('engines that emit no tokens resolve speechTokenIds to undefined', async () => { + scripted = [ + { type: 'textToSpeech', buffer: [], pcm: encodeTtsPcm([1]), done: false, sampleRate: 44100 }, + { type: 'textToSpeech', buffer: [], done: true, stopReason: 'completed' } + ] + const run = textToSpeech({ modelId: 'supertonic', text: 'Hello.', stream: false }) + await run.buffer + assert.equal(await run.speechTokenIds, undefined) +})