[Bugfix] Make Humming DeviceInfo lookups traceable under torch.compile

[mgoin]

Overview

Fixes vllm-project/humming#147 for vLLM: models served through Humming fail torch.compile with RecursionError whenever Humming's forward path quantizes or Hadamard-transforms activations (seen during warmup of MXFP4 MoE checkpoints on H200). This patches Humming from the vllm.utils.humming facade so the pinned humming-kernels==0.1.17 works for v0.32.0.

Claims

• torch.compile (fullgraph=True) works again on Humming forward paths that run may_process_input (input quantization, Hadamard): no RecursionError, no Dynamo Unsupported.
• Works with the currently pinned humming 0.1.17. The upstream fix (fix(device): prevent recursion when copying DeviceInfo humming#148, in the upcoming 0.1.18) is not sufficient on its own for vLLM (see Details).
• No behavior change in eager mode, apart from no longer re-querying device properties on every forward call.

Validation

B300 (SM103), this branch on main with humming-kernels[cu13]==0.1.17:

Check	main	this PR
New tests/quantization/test_humming_compile.py	RecursionError	passed
E2E: Qwen3-0.6B, --quantization humming, int6 weights + Hadamard + int8 activations, torch.compile + CUDA graphs	RecursionError at engine init	correct output ("Paris, and the capital of Italy is Rome…")
Same E2E with only the #148 __getattr__ fix backported (on 0.1.16)	—	torch._dynamo.exc.Unsupported on _DeviceInfo.__new__
Standalone repro from humming#147 (copy.copy, torch.compile, fullgraph=True)	RecursionError	OK

VLLM_HUMMING_ONLINE_QUANT_CONFIG='{"dtype":"int6","hadamard_block_size":-1}' \
VLLM_HUMMING_INPUT_QUANT_CONFIG='{"dtype":"int8"}' \
  python -c 'from vllm import LLM; print(LLM("Qwen/Qwen3-0.6B", quantization="humming").generate("The capital of France is"))'

Regression tests (same setup): pytest tests/kernels/moe/test_moe.py tests/kernels/core/test_activation.py tests/quantization/test_humming_ignore.py tests/quantization/test_humming_mxfp4_block_fp8.py tests/kernels/quantization/test_mxfp4_kernel_selection.py tests/model_executor/test_utils.py -k humming → 194 passed.

Not run locally on Hopper. The new test runs in the H200 Quantization CI step, and I added vllm/utils/humming.py to that step's trigger paths. Note the existing Humming lm-eval configs all use --enforce-eager, which is why CI never caught this.

Details

Root cause. On every call, LayerConfig.check_device and the PDL heuristics (_resolve_use_pdl → get_heuristics_class) build a new humming.device.DeviceInfo(device). vLLM traces these from may_process_input inside its fullgraph-compiled region. Two problems stack up:

1. DeviceInfo has __slots__ and a catch-all __getattr__. When Dynamo creates the instance, it reads the unset _value slot, which re-enters __getattr__ and recurses forever. humming#148 fixes this by raising AttributeError for private names.
2. With (1) fixed, Dynamo still can't trace the pybind _DeviceInfo(index) constructor, so fullgraph compile fails with Unsupported. Reading attributes from an extension object that already exists does trace fine; that's why the module-global current_device already works under compile. Humming main still has these per-call constructions, so 0.1.18 alone would still break vLLM.

Fix. On first use of the facade, _patch_humming():

• backports the Add quickstart guide #148 __getattr__ change (also fixes copy.copy(DeviceInfo)), and
• replaces the DeviceInfo name in the five Humming modules that call it (config.config, transform, tune, tune.process_input, utils.smem) with a factory that returns one cached instance per device index. prepare_layer_config populates the cache eagerly during weight processing, so at trace time Dynamo only sees a dict lookup on an existing object.

All vLLM Humming access goes through the lazy facade, so the patch is always applied before any layer is prepared. The swap is skipped for any module whose DeviceInfo is no longer Humming's class. DeviceInfo only exposes static device properties, so caching it is safe.

Follow-up. This should be removed once Humming's DeviceInfo can be constructed in traced code (for example, by caching inside Humming itself); there's a TODO. I'll raise this on the Humming side so 0.1.18+ can drop the patch.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

No duplicate: no open vLLM PR touches Humming DeviceInfo or this compile failure (searched "humming DeviceInfo", "humming torch.compile", "humming recursion", "humming fullgraph").

🤖 Generated with Claude Code

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[mgoin]

/ci run

[github-actions]

✅ Triggered Buildkite CI #95176 for commit 4c9288a873bf.

[mgoin]

Humming-side fix: vllm-project/humming#149. Verified vLLM main without this workaround serves the compiled int6 + Hadamard + int8 Humming model on that branch, so this can be reverted once the pin includes it.

[mgoin]

Previous build passed and latest commit just updated comment, merging

[gau-nernst]

thanks for the fix! I think this also fixes my issue with Humming kernels on sm120