[Model][Perf] Enable Inkling SplitKV on SM90

[NVShreyas]

Purpose

Inkling's global FA4 relative-attention path currently forces num_splits=1
on SM90. For TP8 decode, that leaves too few CTAs to occupy an H200 when the
runtime batch is small.

This PR removes the SM90 unsplit override and lets Hopper use the existing
adaptive SplitKV heuristic. Local attention remains unsplit. The unit coverage
now checks the resulting SM90 schedule for the production TP8-local shape with
Q=1 and Q=2.

This does not duplicate #51416. That PR adds general SM90 FA4 Dense/MLA
integration and does not modify Inkling's specialized relative-attention op or
its split heuristic.

H200 microbenchmark

The existing Inkling FA4 forward-plus-combine path was measured on one H200 with the TP8-local shape: 4 query heads, 1 KV head, head dimension 128,
BF16, batch 1, and Q in {1, 2}. Each arm used two opposite-order rounds, three
warmups per round, and ten timed samples per round.

Actual KV	Q	Split 64	Split 128	Split 256	Result
8K	1	86.672 us	85.680 us	94.416 us	64 and 128 within 2%
8K	2	85.264 us	84.352 us	89.136 us	64 and 128 within 2%
64K	1	117.424 us	100.832 us	114.640 us	128 is 14.13% lower latency than 64
64K	2	121.440 us	106.288 us	118.880 us	128 is 12.48% lower latency than 64

This supports the existing max_splits=128 cap and CTA targets on SM90.

All microbenchmark outputs passed the unsplit-reference correctness gate. The
worst absolute difference was 0.0002441 and the worst relative L2 was 0.003123.

End-to-end serving

Configuration: thinkingmachines/Inkling-Small-NVFP4, H200x8 TP8, 64K ISL
(6.4K new + 57.6K reused), 400 OSL, MTP with synthetic acceptance length 1.80,
and concurrency 32.

SM90 policy	Output tok/s	Output tok/s/GPU	p50 ITL	p50 tok/s/user	Errors
Unsplit	1095.339	136.917	23.218 ms	43.070	0
Existing adaptive heuristic	1446.799	180.850	14.935 ms	66.955	0

Two full concurrency sweeps also compared the existing adaptive heuristic
against a workload-specific tuned split table:

• 64K/400 with 90% prefix reuse: adaptive heuristic was -0.140% in throughput
  geometric mean.
• Static 8192/1024 with no shared prefix: adaptive heuristic was +0.085% in
  throughput geometric mean.

These comparisons support using the existing general heuristic instead of a
model/workload-specific split table.

Test Plan

.venv/bin/python -m pytest \
  tests/models/inkling/test_fa4_rel_attention.py -k 'num_splits' -v

The commit hooks also ran Ruff check/format, mypy, SPDX, forbidden-import, and
repository policy checks on the change.

Test Result

41 passed, 55 deselected

AI assistance was used to prepare the code, tests, benchmark analysis, and PR
description.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR is described.
• The test command is provided.
• Microbenchmark and end-to-end results are included.
• No documentation update is required for this model-specific scheduling change.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.