Issue #028 bench data

Qwen3.8-27B serving on a single RTX 5090. Published unedited.

These are the working lab notes and raw run output behind the local-inference section of issue #028, "The Grades Came In". That section spends its length asking other people to show their configuration, so here is ours, including the run where we got a verdict wrong and had to redo it.

Reports

File	What it backs
serving-engine-concurrency	The four-way concurrency table, the flipped vLLM verdict, and the DFlash2 fast-lane numbers (UPDATE 3)
qwen38-iq3-quant-ladder	The quant-ladder finding and the +35% completion-token result on the aider slice

Raw run output

File	Configuration
serving-conc-gguf.txt	GGUF Q5_K_M + MTP, llama.cpp
serving-conc-vllm.txt	vLLM NVFP4, stock flags (the misconfigured run)
serving-conc-vllm-MTP3-recipe.txt	vLLM NVFP4, full recipe: MTP-3 + TurboQuant 4-bit KV + the PR #40914 patch
serving-conc-sglang.txt	SGLang with no speculation, and SGLang + DSpark
serving-conc-sglang-dflash2.txt	SGLang dev + DFlash2

Each .txt carries per-N aggregate tok/s, per-request tok/s, p50 and p95 latency, total tokens and wall-clock, plus a JSON row dump.

Hardware and method

Single RTX 5090 (32GB, sm_120 Blackwell), 60GB host RAM. Qwen3.8-27B throughout. Each measurement is N concurrent identical requests, 800 max_tokens, one shared prompt. Aggregate tok/s is total completion tokens divided by batch wall-clock. Warmup before each sweep.

Checkpoints are pinned by name in the reports, not just by format, because "Qwen3.8-27B NVFP4" is not one artifact: the SGLang and vLLM lanes used different NVFP4 exports (gittensor-model-hub and RadixArk) and they do not perform identically. DFlash2 required a local retag of the SGLang dev image at commit f825d72; SGLang v0.5.18 does not contain DFlash2DraftModel.

Limits, stated plainly. One box, one GPU. Capability suites are n=25; the aider polyglot slices are 34 exercises with two runs per side. These are directional working results, not a published ranking. The run-to-run variance we measured (three exercises on the Q5 baseline between identical runs) is large enough that single-run comparisons at non-zero temperature do not resolve small gaps. These numbers are ours, on our hardware, with our flags, and deserve exactly the scepticism we applied to everyone else's.

What is deliberately not here

The abliteration work from the same period is not published. Those harnesses recorded refuse/comply and response length only; harmful outputs were redacted from stored results and the raw scratch deleted. Nothing from that work is cited in issue #028, so there is nothing here it needs to substantiate.