Skip to content

[Perf] Select grouped NAX prefill widths for GQA6 and GQA8 - #1300

Open
suntp wants to merge 4 commits into
vllm-project:mainfrom
suntp:experiment/nax-gqa-adaptive-20261106
Open

suntp wants to merge 4 commits into
vllm-project:mainfrom
suntp:experiment/nax-gqa-adaptive-20261106

Conversation

@suntp

@suntp suntp commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Main groups three or six heads for GQA6 through heuristic admission and uses ordinary NAX for GQA8; the two cases share no common decision rule. Wider groups share KV reads but expose fewer independent threadgroups.

Solution

A single selection pipeline replaces the GQA6 heuristics and covers GQA8:

  • Eligibility: single-sequence causal attention, BF16, head dim 256, GQA6 with 16-token kernel pages or GQA8 with 32-token pages, contiguous compact or interleaved KV.
  • Candidates: ordinary NAX plus grouped widths {2, ratio/2, ratio} — h2/h3/h6 for GQA6 and h2/h4/h8 for GQA8.
  • One cost rule: every candidate is scored by the same causal tile counts, guarded-tile overhead, per-core capacity and detected GPU cores (six shared coefficients) estimating the larger of per-core tile work and the longest serial KV scan.
  • Admission: the cheapest grouped width runs only when its cost is strictly below 90% of ordinary NAX; otherwise the launch stays with ordinary NAX.

The legacy GQA6 selector and its per-layout rates are deleted rather than wrapped, and grouped launches reuse the existing host length bound, so the interface gains no new input. Supported shapes and switch.

Validation results

Mean TTFT change versus main (negative is faster). BF16, APC, one sequence, 16 output tokens; cached prefixes 31,680 (35B) and 31,360 (27B) tokens. 35B and M5 27B rows compare 2a311af / f6e94a2; Pro 27B compares 9d3d0bb / e595597; Max 27B cached compares 9d3d0bb / 0e01a53 (test-only head, same production code) over four AB/BA service pairs — 192 formal requests, cache hits, widths and outputs verified per request.

Model GPU Cold 32K Cached continuation, Q and paired TTFT change
35B M5 -1.47% / -4.32% Q16 -10.99% / -10.76%
Q64 -10.63% / -8.31%
Q128 -18.12% / -18.32%
35B M5 Pro -2.30% / -2.44% Q16 -11.47% / -11.01%
Q64 -5.81% / -5.69%
Q128 -9.60% / -9.32%
35B M5 Max -11.50% / -6.79% Q16 -15.08% / -12.93%
Q64 -8.22% / -8.45%
Q128 -8.24% / -8.27%
27B M5 -7.44% / +0.37% † Q65 -23.50% / -23.35%
Q144 +0.14% / +3.12%
Q1328 -2.04% / +0.11% †
27B M5 Pro +0.30% / +0.27% Q65 -4.57% / -4.52%
Q144 +0.19% / +0.19% †
Q1328 +0.67% / +2.60%
27B M5 Max unresolved ‡ Q65 +0.17% ±1.23 †
Q144 -0.13% ±1.84 (h3→h2)
Q1328 +0.37% ±4.50 †

† Both arms used the same widths; ± is the 95% CI over the four pairs; ‡ see Limitations. CI passed 6,030 tests at e595597; the re-run at 0e01a53 was cancelled by macOS runner capacity, not a test failure — that head only adds tests (native artifacts byte-identical, 52/52 local). Native parity covers 122 cases per switch state, and the new tests pin the 90% admission margin (89.71% / 90.49% / exactly 90%) and the geometry guards; mutating any of them fails the suite.

Limitations & follow-ups

  • The Max 27B cached cells replace an earlier window disturbed by Spotlight indexing (82–126 indexer processes exiting during timing, GPU at 1,127 vs 1,404 MHz); those runs are excluded as environment noise.
  • Max 27B cold 32K is unresolved: 20/22 chunks switch h6→h3, but two four-service windows disagree (+4.6% / +3.5% vs +10.7% / -6.4%) while same-build drift reaches 6–11%; it needs its own calibration before any sign is claimed.
  • Q1328 slows ~5–7% late within a process; compare per block, not across full runs.
  • M5 27B Q144 switched h6→h3, 14–16% slower in earlier kernel measurements. Pro 27B drift is small (cold ≤0.02%, Q1328 -1.95% between processes). Pro 35B Q64 TPOT rose 1.51% / 1.62%.
  • Serving covers single-sequence BF16/D256 workloads through 32K on 10/20/40-core GPUs; the 80-core configuration has CPU policy coverage only.

AI assistance: OpenAI Codex assisted with implementation, tests and this description.

@github-actions github-actions Bot added kernel ci/build CI, builds, packaging, and installation labels Nov 8, 2026
* heads_per_group * grouped.tail_stream;
}

inline NaxGqaScore nax_gqa8_work_score(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would consolidate GQA8 admission without losing the measured gains. The plan selector uses this score as its starting choice. Rerun Q16/Q64/Q128 serving on the final head before carrying over the performance claims.

Comment thread vllm_metal/attention/impls/sdpa.py Outdated
"nax_work_plan", False
):
if ctx.nax_work_plan is None:
ctx.nax_work_plan = ops.nax_prefill_plan(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before changing the no-plan fallback, pass this TurboQuant lane’s own host lengths into the plan path. Preserve its existing GQA6 route for compact KV storage and add a regression case.

Comment thread vllm_metal/metal/nax_prefill_work.h Outdated

// Each sequence keeps its own causal origin, full blocks and final Q/KV
// tails. Concatenating Q lengths would invent complete blocks across requests.
template <typename Sequences>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would make this take one NaxSequence and keep only fields the selector reads. Production rejects batches and nonzero offsets before this call, so remove the unused aggregation and offset handling with their test.

Comment thread vllm_metal/metal/nax_prefill_work.h Outdated
if (layout.heads_per_group > 1) {
if (geometry.kernel_page == 16) {
switch (layout.heads_per_group) {
case 2: rate = 1.0169304348373918; break;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find these raw coefficients hard to maintain. Name the fitted values, normalizers, and margins. Explain the cost terms briefly beside the code and link this PR for the calibration record.

Comment thread docs/gqa-prefill-history.md Outdated
@@ -0,0 +1,260 @@
# Earlier grouped NAX measurements

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this file. These tables describe older policies. Keep those measurements in the PRs that produced them.

Comment thread docs/gqa-prefill.md Outdated
every Q16/Q64 and prefix32 residue, and longer boundaries. Multi-sequence and
mixed automatic admission remains disabled pending its own performance study.

## Calibration and independent validation

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would keep this page to the flag, supported widths, minimum 15% predicted margin, capacity reserve, and limits. Keep measurement details in the PR body and clearly label results from before the rebase.

Comment thread docs/configuration.md Outdated
| `VLLM_MLX_DEVICE` | `gpu` | MLX device (`gpu` or `cpu`) |
| `VLLM_METAL_DISABLE_NAX` | `0` | Emergency override for automatic M5 NAX prefill attention. Set to `1` to force the non-NAX fallback. |
| `VLLM_METAL_NAX_GQA_PREFILL` | `1` | Automatically select three- or six-head grouped NAX prefill on the MLX backend for single-sequence causal attention without sliding windows, softcap or sinks, using BF16, head dimension 256, GQA ratio 6 and 16-token kernel pages. The work-based selector compares query and KV lengths, query/KV heads, detected GPU cores and the actual upstream pipeline. Unsupported features, mixed batches/suffix offsets, unknown capacity or insufficient pipeline limits retain upstream. Extra six-head admission uses complete groups per GPU core and full-tile work coverage for power-of-two KV-head layouts; its guarded-query allowance is calibrated only at 10 cores. Other layouts retain the existing work score; total KV length has no fixed 32K admission cap. Complete tiles omit bounds checks while partial Q/KV tiles remain guarded. Set `0` to retain upstream NAX. Read once at worker startup; restart after changing it. `VLLM_METAL_DISABLE_NAX=1` disables all NAX. |
| `VLLM_METAL_NAX_GQA_PREFILL` | `1` | Automatically select grouped NAX prefill for MLX BF16 D256, single-sequence ordinary causal requests. GQA6/page16 enumerates h2/h3/h6; GQA8/page32 enumerates h2/h4/h8. With exact CPU lengths and calibrated interleaved KV strides, a shared work/resource cost compares all candidates with actual upstream and the retained h3/h6/h4 route, including GPU capacity and a replacement margin. Other valid layouts retain the established policy; unsupported features, mixed batches, unknown capacity and insufficient pipeline limits retain upstream. Larger physical pages are translated; partial tails are guarded and there is no fixed 32K KV limit. Set `0` to retain upstream NAX. Read once at worker startup. `VLLM_METAL_DISABLE_NAX=1` disables all NAX. See [scope and validation](gqa-prefill.md). |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep this cell about the user control: what 1 and 0 do, supported geometries, and the required worker restart. Put scoring details beside the implementation.

Comment thread tests/test_block_size_translation.py Outdated
" }\n"
" return 0;\n"
" }\n"
' if (argc == 2 && std::string(argv[1]) == "resources") {\n'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would keep one small helper that calls the plan-based selector, with inputs and expected configs in a parameter table. Remove the resource-reporting modes and copied resource formulas. Keep the independent work-count oracle and pipeline-capacity coverage.

Comment thread tests/test_attention_sdpa.py Outdated
whole_hint = int(kind == "prefill")
assert spy.calls[-1].nax_split_hint == (whole_hint if forwarded else -1)
assert spy.calls[-1].nax_prefill_split_hint == (1 if subset else -1)
if forwarded and supported == "subset":

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think explicit plan-call and forwarding expectations would make this matrix clearer. Put them in the parameter table and assert them directly. Keep the shared setup.

Comment thread tests/test_nax_split.py Outdated
num_decode_tokens=0,
max_decode_context_len=0,
)
if width in (2, 8):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pass the plan for every grouped case, including h3/h4/h6. Put the expected dispatch, width, compact-cache fallback, and kernel suffix in the parameter table. Each case should state the route it checks.

@suntp
suntp force-pushed the experiment/nax-gqa-adaptive-20261106 branch 5 times, most recently from 5190e1e to 47e3435 Compare November 13, 2026 10:29

suntp commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

@LxYuan0420, updated to 47e3435, rebased onto main 3451a01.

  1. Selector: GQA6/GQA8 now share one plan cost model, using a single NaxSequence and named coefficients. The upstream GQA6 policy remains the reference/fallback; TurboQuant passes its own host lengths and preserves compact-KV routing.
  2. Cleanup: removed unused aggregation/offset handling and the history document; shortened usage docs and made route/forwarding expectations table-driven. The independent work-count oracle, pipeline-capacity checks, grouped-width checks and page-translation coverage remain.
  3. Serving: fresh actual-main comparisons on the final head cover M5 35B Q16/Q64/Q128 and cold 32K, Pro 35B cold 32K, and Pro 27B. Other timings are explicitly labeled historical. The recalibrated margin is 10%; drift, known regressions and failed startup attempts remain disclosed in the updated description.
  4. Validation: CI is green, with 6,409 tests passed.

Could you please take another look at the simplified selector and updated evidence?

@WindChimeRan WindChimeRan left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The performance gain from the PR description looks good, but I haven't check all the workload. Currently I'm not sure if this is the right direction.

Design issue:

  • Two policies determine one decision
  • Historical behavior becomes part of the architecture
  • The calibration burden grows
  • The interface gains another specialized input

the description mentions the cost model, but it does not clearly explain the architectural change.

Overall, the PR is hard to review. The author should cleanup the review noise, and make the design better motivated.

Score ordinary NAX and the compiled grouped widths from the same causal tile counts and GPU capacity. Use the existing length bound for cost only, including compact TurboQuant caches, and remove the previous GQA6 selector and per-layout rates.

Signed-off-by: suntp <20786998+suntp@users.noreply.github.com>
Signed-off-by: suntp <20786998+suntp@users.noreply.github.com>
@suntp
suntp force-pushed the experiment/nax-gqa-adaptive-20261106 branch from 47e3435 to 9ff66a2 Compare November 18, 2026 12:54
Signed-off-by: suntp <20786998+suntp@users.noreply.github.com>
Add pinned selector cases for grouped costs 0.34% below, 0.49% above
and exactly at the 90% admission ratio, extend the policy driver with
value-dim, token-stride and KV-length overrides, and assert ordinary
NAX for each violated geometry guard. Mutation checks: changing the
margin to 9%/11%, admitting equality or dropping any of the three
guards now fails the suite.

Signed-off-by: suntp <20786998+suntp@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/build CI, builds, packaging, and installation kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants