Skip to content

Commit 1d02858

Browse files
danielhanchenstduhpfleejetLostRuinsfszontagh
authored
Sync with upstream master, including Qwen-Image-2.1 support (#15)
* fix: preserve "token_refiner" token for MiniMax H3 LoRAs (leejet#1864) * fix: fail with a message when MiniMax-H3 is run in img_gen mode (leejet#1863) * feat: support INT8 ConvRot safetensors (leejet#1857) * fix: replace free_compute_buffer with runner_done in vae (leejet#1872) * sync: update ggml (leejet#1873) * fix(ci): trigger builds for ggml updates * feat: add taeh3 support (leejet#1874) * fix: prevent gallocr hash overflow in tiny graph-cut segments (leejet#1880) * fix: re-clamp streaming VRAM budget to currently free memory (leejet#1878) * fix: mark graph cuts with both a prefix and a suffix (leejet#1883) * fix: make max_order of lms sampler configurable (leejet#1885) * fix: guard against missing sampler/scheduler names (leejet#1887) * chore: format code * fix: use sd_get_preview_interval() (leejet#1907) * feat: configurable image / video compression (leejet#1909) * feat: support standard Qwen3-VL weights for MiniMax-H3 (leejet#1910) * feat: load scaled FP8 weights without upfront conversion (leejet#1913) * fix: match exact weights in LLM config detection (leejet#1923) * feat: add LTX-2.5 support (leejet#1893) Co-authored-by: leejet <leejet714@gmail.com> * fix: correct MiniMax H3 reference audio encoding (leejet#1886) * fix: correct MiniMax H3 audio Euler steps (leejet#1908) * feat: use backend-native FP8 matmul when supported (leejet#1916) * sync: update ggml * feat: additional `--preview-interval` values (leejet#1915) Co-authored-by: leejet <leejet714@gmail.com> * feat: support numbering for preview images (leejet#1895) * fix: use carrier sampling for MiniMax H3 audio (leejet#1924) * feat: generalize temporal tiling across video VAEs (leejet#1926) * feat: prefetch streamed layers during compute (leejet#1905) Co-authored-by: leejet <leejet714@gmail.com> * refactor: unify runner lifecycles and weight residency (leejet#1940) * feat: add verbose logging and log-level selection (leejet#1941) * feat: enable single-GPU auto-fit with tiered parameter placement (leejet#1942) * fix: reuse graph cut plans across CFG passes (leejet#1943) * refactor: split ggml extensions and move implementations to cpp files (leejet#1945) * fix: preserve K-quantized embedding weights (leejet#1936) * fix: correct SDXL embeddings loading (leejet#1939) * refactor: unify model source and weight lifecycle management (leejet#1956) * docs: reflect GGML_MAX_NAME value change in rpc docs (and in ggml_extend assert) (leejet#1950) * refactor: split generation pipeline out of stable-diffusion.cpp (leejet#1957) * fix: enable VAE decode tiling fallback without auto-fit (leejet#1932) * fix: preserve BF16 embedding weights for get_rows (leejet#1959) * fix: handle invalid option numbers (leejet#1961) * feat: expose the loaded model version name through the public API (leejet#1962) * feat: add SenseNova U1.5 support (leejet#1935) * fix: reuse graph plans when scale parameters change (leejet#1963) * feat: add linear and attention scale overrides (leejet#1964) * feat: preserve explicit backend assignments during auto-fit (leejet#1967) * fix: guard GPU memory capacity and propagate encoding failures (leejet#1958) * fix: bound plain-text runs in parse_prompt_attention regex (leejet#1919) * feat: add Wan2.2 S2V (audio+img-to-video) support (leejet#1925) Co-authored-by: leejet <leejet714@gmail.com> * fix: validate vision projector output dim against LLM hidden size (leejet#1918) * fix: resolve MSVC narrowing conversion warnings (leejet#1969) * feat: Add generation parameters into video metadata (leejet#1901) Co-authored-by: leejet <leejet714@gmail.com> * feat: support external Hugging Face tokenizer JSON files (leejet#1973) * refactor: require external Gemma 2 and GPT-OSS tokenizers (leejet#1974) * feat: support Brownian tree noise in all noise injection samplers (leejet#1899) * fix: use tokenizer-specific pre-tokenization rules (leejet#1975) * fix: remove vision_model. from ununsed tensors (leejet#1983) * perf: eliminate temporary allocations in Philox rounds (leejet#1982) * fix: honor flash attention flag in LLM text encoder attention (leejet#1987) * refactor: remove obsolete unused tensor filtering (leejet#1984) * perf: pad small attention heads to 64 for MMA Flash Attention (leejet#1992) * perf: update ggml for faster direct convolutions (leejet#1993) * perf: accelerate VAE direct 3D convolutions (leejet#1996) * fix: propagate CUDA driver dependency to shared library consumers * fix: prevent clip_preprocess center crop from exceeding the resized image (leejet#1995) * perf: reduce CPU overhead in graph execution and sampling (leejet#1997) * perf: parallelize host tensor elementwise and broadcast ops (leejet#1998) * feat: support building with upstream ggml (leejet#1999) * feat: add Qwen Image 2.1 support (leejet#1994) * feat: restore legacy fp8 handling when building with upstream ggml (leejet#2001) * fix: avoid passing ggml logs as format strings (leejet#2002) * feat: add LLaDA-Image support (leejet#1968) Co-authored-by: leejet <leejet714@gmail.com> * feat: add native CUDA SageAttention support (leejet#2005) * fix: avoid narrowing conversion in SigVQ patch embedding and format code * docs: update CONTRIBUTING.md --------- Co-authored-by: stduhpf <stephduh@live.fr> Co-authored-by: leejet <leejet714@gmail.com> Co-authored-by: LostRuins Concedo <39025047+LostRuins@users.noreply.github.com> Co-authored-by: fszontagh <51741446+fszontagh@users.noreply.github.com> Co-authored-by: Wagner Bruna <wbruna@users.noreply.github.com> Co-authored-by: vmobilis <75476228+vmobilis@users.noreply.github.com> Co-authored-by: Piotr Wilkin (ilintar) <ilintar@gmail.com> Co-authored-by: jk212h20 <101200018+jk212h20@users.noreply.github.com> Co-authored-by: assouan <750048+assouan@users.noreply.github.com> Co-authored-by: nan <zjn32202153@gmail.com> Co-authored-by: Hmission <62598659+Hmission@users.noreply.github.com> Co-authored-by: LED-M <105789115+xledx@users.noreply.github.com> Co-authored-by: Maphist0 <28743569+Maphist0@users.noreply.github.com> Co-authored-by: George <35490284+noctrex@users.noreply.github.com> Co-authored-by: Санька Четвёртый <CAHbKA-IV@mail.ru> Co-authored-by: Lin Xuhao <linxuhao84@gmail.com> Co-authored-by: Fabrice Aneche <akhenakh@users.noreply.github.com>
1 parent 2c4df9b commit 1d02858

256 files changed

Lines changed: 123702 additions & 15848 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/ISSUE_TEMPLATE/bug_report.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,9 @@ body:
66
- type: markdown
77
attributes:
88
value: |
9-
Please use this template and include as many details as possible to help us reproduce and fix the issue.
9+
Before submitting a bug report, please read the [Troubleshooting guide](https://github.com/leejet/stable-diffusion.cpp/blob/master/docs/troubleshooting.md) and try the steps relevant to your problem.
10+
11+
If the problem persists, complete this form and include what you tried and the results, along with enough details to help us reproduce and fix the issue.
1012
- type: textarea
1113
id: commit
1214
attributes:

‎.github/ISSUE_TEMPLATE/config.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
contact_links:
2+
- name: Troubleshooting
3+
url: https://github.com/leejet/stable-diffusion.cpp/blob/master/docs/troubleshooting.md
4+
about: Read the troubleshooting guide first. If the problem persists, submit a bug report.

‎.github/workflows/build.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ on:
2323
"**/*.c",
2424
"**/*.cpp",
2525
"**/*.cu",
26+
"ggml",
2627
"examples/server/frontend",
2728
"examples/server/frontend/**",
2829
]
@@ -40,6 +41,7 @@ on:
4041
"**/*.c",
4142
"**/*.cpp",
4243
"**/*.cu",
44+
"ggml",
4345
"examples/server/frontend",
4446
"examples/server/frontend/**",
4547
]

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
build*/
22
cmake-build-*/
33
test/
4+
tests/
45
.vscode/
56
.idea/
67
.cache/

‎AGENTS.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,8 @@ documentation.
118118
6. Run the narrowest useful build, test, or inspection command available.
119119

120120
Follow `CONTRIBUTING.md` for formatting, naming, PR expectations, dependency
121-
update policy, and security rules.
121+
update policy, and security rules. For tokenizer additions, follow its embedded-data
122+
allowlist and default to an external `tokenizer.json`.
122123

123124
---
124125

‎CMakeLists.txt‎

Lines changed: 20 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,8 @@ option(SD_MUSA "sd: musa backend" OFF)
9595
option(SD_BUILD_SHARED_LIBS "sd: build shared libs" OFF)
9696
option(SD_BUILD_SHARED_GGML_LIB "sd: build ggml as a separate shared lib" OFF)
9797
option(SD_USE_SYSTEM_GGML "sd: use system-installed GGML library" OFF)
98+
option(SD_USE_UPSTREAM_GGML "sd: build with upstream GGML instead of the patched GGML extensions" OFF)
99+
set(SD_GGML_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/ggml" CACHE PATH "sd: ggml source directory (also supplies private headers for system ggml)")
98100
#option(SD_BUILD_SERVER "sd: build server example" ON)
99101

100102
set(CMAKE_C_STANDARD 11)
@@ -229,6 +231,8 @@ file(GLOB SD_LIB_SOURCES CONFIGURE_DEPENDS
229231
"src/model/*/*.h"
230232
"src/model/*/*.cpp"
231233
"src/model/*/*.hpp"
234+
"src/pipeline/*.h"
235+
"src/pipeline/*.cpp"
232236
"src/runtime/*.h"
233237
"src/runtime/*.cpp"
234238
"src/runtime/*.hpp"
@@ -290,6 +294,12 @@ else()
290294
add_library(${SD_LIB} STATIC ${SD_LIB_SOURCES})
291295
endif()
292296

297+
if(MSVC)
298+
target_compile_options(${SD_LIB} PRIVATE $<$<COMPILE_LANGUAGE:CXX>:/bigobj>)
299+
# ggml backends can throw C++ exceptions through their C API.
300+
target_compile_options(${SD_LIB} PRIVATE $<$<AND:$<COMPILE_LANGUAGE:CXX>,$<CXX_COMPILER_ID:MSVC>>:/EHsc->)
301+
endif()
302+
293303
if(APPLE)
294304
sd_set_macos_rpaths(${SD_LIB})
295305
endif()
@@ -317,23 +327,21 @@ if (NOT SD_USE_SYSTEM_GGML)
317327
endif()
318328

319329
# deps
320-
# Only add ggml if it hasn't been added yet
321-
if (NOT TARGET ggml)
322-
if (SD_USE_SYSTEM_GGML)
323-
find_package(ggml REQUIRED)
324-
if (NOT ggml_FOUND)
325-
message(FATAL_ERROR "System-installed GGML library not found.")
326-
endif()
327-
add_library(ggml ALIAS ggml::ggml)
328-
else()
329-
add_subdirectory(ggml)
330-
endif()
331-
endif()
330+
include(cmake/ggml.cmake)
332331

333332
add_subdirectory(thirdparty)
334333

335334
target_sources(${SD_LIB} PRIVATE $<TARGET_OBJECTS:zip>)
336335
target_link_libraries(${SD_LIB} PUBLIC ggml)
336+
find_package(Threads REQUIRED)
337+
target_link_libraries(${SD_LIB} PRIVATE Threads::Threads)
338+
target_link_libraries(${SD_LIB} PRIVATE onig sd-utf8proc)
339+
if (SD_CUDA)
340+
find_package(CUDAToolkit REQUIRED)
341+
# Keep the driver stub on downstream link lines when no driver is installed.
342+
target_link_libraries(${SD_LIB} PUBLIC CUDA::cuda_driver)
343+
set_property(SOURCE src/core/ggml_extend_backend.cpp APPEND PROPERTY COMPILE_DEFINITIONS SD_USE_CUDA)
344+
endif()
337345
target_include_directories(${SD_LIB} PUBLIC . src include)
338346
target_include_directories(${SD_LIB} PRIVATE src/core)
339347
target_include_directories(${SD_LIB} PUBLIC . thirdparty)

‎CONTRIBUTING.md‎

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,8 @@ If you want to update a third-party dependency, please open an issue first inste
1414

1515
Keep each PR focused on one clear change. Large or overly complex PRs are harder to review and may not be merged.
1616

17+
Do not include test code or test scripts in commits or PRs. Keep them local and report verification results in the PR description.
18+
1719
Follow Conventional Commit-style subjects seen in history: `feat:`, `fix:`, `refactor:`, `ci:`, `docs:`, `chore:`. Keep subjects imperative and scoped.
1820

1921
PRs should include:
@@ -35,17 +37,31 @@ Naming conventions:
3537
- In `PascalCase` names, preserve common abbreviations in uppercase, for example `SD`, `API`, `HTTP`, `JSON`, `RGB`, `VAE`, `TAE`, `LoRA`, and `WebP`.
3638
- Use `snake_case` for functions, methods, variables, and file names unless an existing API requires a different style.
3739
- Use a trailing underscore for private data member names, for example `hidden_size_` or `tokenizer_`.
38-
- Use `.h` for C and C++ header files. Do not introduce new `.hpp` headers.
40+
- Use `.hpp` for model headers under `src/model/`, including new model headers. Do not rename these headers to `.h`. Use `.h` for other C and C++ header files.
3941
- Use macro-based header include guards instead of `#pragma once`.
4042
- Format header include guards as `__SD_{PATH}__`, where `{PATH}` is the header path in uppercase snake case without the file extension. For example, `src/sample.h` should use `__SD_SAMPLE_H__`.
4143
- Do not introduce anonymous namespaces in new or modified code; prefer `static` file-local functions/variables or an explicit named namespace when scoping is needed.
4244
- In `class`/`struct` definitions, place data members before member functions unless an existing type already clearly follows a different pattern.
43-
- Keep `test_*.cpp` / `test_*.py` naming for tests.
4445

4546
Some older code in the project may not fully follow the current conventions. Please do not submit PRs that only rewrite existing code to match style rules.
4647

4748
When adding or modifying model implementations, follow the model config and weight detection conventions in [docs/model_config.md](docs/model_config.md).
4849

50+
## Tokenizer Data
51+
52+
New model integrations must use an external `tokenizer.json` by default. Do not
53+
embed new vocabularies or merge tables solely for less widely used models;
54+
these tables increase the binary size for every user.
55+
56+
The embedded-data allowlist is CLIP, T5/UMT5, Qwen 2/3, Mistral, and Gemma 3/4.
57+
Models may reuse an existing embedded tokenizer when its vocabulary and behavior
58+
match their text encoder. Gemma 2 and GPT-OSS require external JSON files.
59+
60+
Adding to this allowlist requires maintainer approval, supported by the model's
61+
usage, reuse across models, and measured binary-size cost. Document the matching
62+
JSON and CLI option for models that require an external tokenizer, and fail
63+
initialization clearly when it is missing.
64+
4965
## AI-Assisted Contributions
5066

5167
AI tools may be used to assist development, but contributors are responsible for the quality and correctness of the submitted code.

‎README.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ API and command-line option may change frequently.***
1515

1616
## 🔥Important News
1717

18+
* **2026/09/20** 🚀 stable-diffusion.cpp adds **Day-0 support for Qwen-Image-2.1**
19+
* **2026/08/20** 🚀 stable-diffusion.cpp now supports **LTX-2.5**
1820
* **2026/08/04** 🚀 stable-diffusion.cpp adds **Day-1 support for MiniMax-H3**
1921
* **2026/06/25** 🚀 stable-diffusion.cpp now supports **Krea2**
2022
* **2026/06/04** 🚀 stable-diffusion.cpp now supports **Ideogram4**
@@ -46,10 +48,12 @@ API and command-line option may change frequently.***
4648
- [Chroma](./docs/chroma.md)
4749
- [Chroma1-Radiance](./docs/chroma_radiance.md)
4850
- [Qwen Image](./docs/qwen_image.md)
51+
- [Qwen Image 2.1](./docs/qwen_image_2.1.md)
4952
- [PiD](./docs/pid.md)
5053
- [LongCat Image](./docs/longcat_image.md)
5154
- [Z-Image](./docs/z_image.md)
5255
- [MiniT2I](./docs/minit2i.md)
56+
- [SenseNova U1.5](./docs/sensenova_u1.md)
5357
- [Ovis-Image](./docs/ovis_image.md)
5458
- [Anima](./docs/anima.md)
5559
- [ERNIE-Image](./docs/ernie_image.md)
@@ -59,16 +63,18 @@ API and command-line option may change frequently.***
5963
- [SeFi-Image](./docs/sefi_image.md)
6064
- [HiDream-O1-Image](./docs/hidream_o1_image.md)
6165
- [Ideogram4](./docs/ideogram4.md)
66+
- [LLaDA-Image](./docs/llada_image.md)
6267
- [Image Edit Models](./docs/edit.md)
6368
- [FLUX.1-Kontext-dev](./docs/kontext.md)
6469
- [Qwen Image Edit series](./docs/qwen_image_edit.md)
6570
- [LongCat Image Edit](./docs/longcat_image.md)
6671
- [Boogu Image Edit](./docs/boogu_image.md)
6772
- [Mage-Flow-Edit](./docs/mage_flow.md#image-editing)
73+
- [LLaDA-Image Edit](./docs/llada_image.md#image-editing)
6874
- Video Models
6975
- [Wan2.1/Wan2.2](./docs/wan.md)
7076
- [MiniMax-H3](./docs/minimax_h3.md)
71-
- [LTX-2.3](./docs/ltx2.md)
77+
- [LTX-2.3/LTX-2.5](./docs/ltx2.md)
7278
- [HunyuanVideo 1.5](./docs/hunyuan_video.md)
7379
- [LingBot-Video](./docs/lingbot_video.md)
7480
- [PhotoMaker](./docs/photo_maker.md) support.
@@ -146,12 +152,14 @@ For runtime and parameter backend placement, see the [backend selection guide](.
146152

147153
## More Guides
148154

155+
- [Troubleshooting](./docs/troubleshooting.md)
149156
- [Backend selection](./docs/backend.md)
150157
- [RPC](./docs/rpc.md)
151158
- [LoRA](./docs/lora.md)
152159
- [LCM/LCM-LoRA](./docs/lcm.md)
153160
- [Docker](./docs/docker.md)
154161
- [Quantization and GGUF](./docs/quantization_and_gguf.md)
162+
- [INT8 convrot safetensors](./docs/int8_convrot.md)
155163
- [Inference acceleration via caching](./docs/caching.md)
156164

157165
## Bindings
478 KB
Loading

‎assets/llada_image/example.png‎

437 KB
Loading

0 commit comments

Comments
 (0)