perf(mtp): fuse EAGLE decode into CUDA graphs

[sufubao]

Summary

• reserve reusable KV scratch slots for intermediate EAGLE draft steps instead of allocating and freeing them on every decode step
• capture MTP verification/sampling and the draft chain as two CUDA graphs, preserving the existing CPU/GPU overlap protocol
• reuse a caller-provided output buffer in gather_token and add focused regression coverage for scratch reuse, mapping restoration, graph dispatch, padding state, and runtime guards

The fused path is limited to supported single-draft EAGLE requests. DP, RL, decode microbatch overlap, constrained decoding, custom generators, invalid-token masks, token penalties, unsupported attention backends, and oversized graph inputs continue through the existing decode path. LIGHTLLM_DISABLE_MTP_FUSED_GRAPH=1 provides an explicit opt-out.

Why

The existing EAGLE path launches the main-model graph and each draft forward separately, with eager tensor allocation, KV-slot rotation, and inference-state setup between launches. At small and medium batch sizes, this host-side work is a measurable part of inter-token latency.

The scratch slots are reserved once while the KV pool is empty. Intermediate draft KV is discarded after the chain, and the verified KV mapping is restored before the next decode step.

Validation

python -m pytest unit_tests/server/test_mtp_chain_scratch.py -q
10 passed

pre-commit run --files <four changed files>
black: passed
flake8: passed

Local serving benchmark:

• NVIDIA H200, Qwen3.5-27B, TP=4, MTP step=3
• ISL=512, OSL=256, greedy decoding
• concurrency 1, 2, 4, 8, 16, 32
• 6 independent seeds, 16 warmup + 128 measured requests per concurrency
• 4,608 measured requests per revision; zero request errors and zero OSL mismatches
• revisions swapped between GPU 0-3 and GPU 4-7 after three seeds to remove a large GPU/NUMA group bias
• 95% confidence intervals use the six seed-level log ratios with GPU group as a crossover fixed effect

Concurrency	Adjusted ITL change	Adjusted output throughput change
1	-5.80% [-10.54%, -0.81%]	+5.09% [+0.64%, +9.73%]
2	-4.54% [-7.68%, -1.30%]	+4.28% [+1.24%, +7.40%]
4	-3.72% [-7.54%, +0.25%]	+3.43% [-0.34%, +7.33%]
8	-3.68% [-6.97%, -0.28%]	+3.17% [+0.33%, +6.09%]
16	-2.08% [-7.69%, +3.88%]	+1.68% [-3.36%, +6.99%]
32	-1.82% [-5.14%, +1.61%]	+1.40% [-2.16%, +5.08%]

Across all concurrency levels, the geometric-mean ITL change is -3.62% [-5.40%, -1.81%] and throughput change is +3.16% [+1.59%, +4.77%]. The gain is statistically clear at low/medium concurrency; the concurrency 16/32 result is not distinguishable from noise in this experiment.