perf(cuda): release MMA stages earlier and flatten native weight tiles

[jinzhen-lin]

1. Early stage release: Release MMA/MXMMA stages after their final shared-memory reads, allowing producers to refill them while consumers compute.
2. Flat weight layout: Repack native MMA/MXMMA/WGMMA weights into contiguous 64-row × 16-byte tiles, ordered by K group then N group in global memory. Keep flat and raw layouts separate; UMMA/PPU retain their raw layout. Preserve Stream-K stage alignment.

W8A8 INT8, N=K=8192, per-token activation / per-channel weight scaling, FP16 output. Values are speedup ratios from median CUDA Graph timings over 21 interleaved rounds; configurations are fixed within each comparison.

1. Baseline → baseline + early stage release

Only the affected warp-specialized MMA path is shown: SM120/SM121, CTA 64×128×128, warp 32×32×128, 3 stages, TMA enabled, 1 CTA/SM.

M	SM120	SM121
1	1.01×	1.01×
16	1.03×	0.99×
64	1.03×	1.00×
256	1.04×	1.01×
1024	1.04×	1.02×
4096	1.04×	1.03×

2. Baseline + early stage release → baseline + early stage release + flat layout

Default heuristics on each device.

M	SM75	SM80	SM86	SM89	SM90-H20	SM90-H200	SM120	SM121
1	1.01×	0.98×	1.01×	1.00×	1.03×	1.06×	1.00×	1.85×
16	1.01×	0.99×	1.01×	1.00×	1.05×	1.05×	1.00×	1.91×
64	0.99×	1.08×	1.05×	1.00×	1.01×	1.04×	1.03×	1.47×
256	1.14×	1.23×	1.35×	1.07×	1.01×	1.00×	1.01×	1.62×
1024	1.08×	1.18×	1.07×	1.05×	1.00×	1.02×	1.06×	1.24×
4096	1.03×	1.21×	1.09×	1.06×	1.00×	1.02×	1.06×	1.17×

Devices: RTX 2080 Ti, A100, RTX 3090, RTX 4090, H20, H200, RTX PRO 6000 Blackwell Server, and DGX Spark. All benchmark outputs were checked against the reference.