← all models

MODEL RESULTS — QWEN3.5-9B

qwen3.5-9b · server http://erwing · run 2026-10-12 15:50:27 → 2026-10-12 16:09:49

OVERALL

Composite: 8.4 ✓ 8/8 tests
Speed (median): 27.3 tok/s
Thinking cost: 0 tokens · ≈0s · starved: none

ENVIRONMENT & LOAD (verified)

qwen3.5-9b qwen3.5-9b. Composite 8.6 (9/9 tests), 27.5 tok/s.
Serverllama.cpp 2.55.0 llama-server.exe (LM Studio backend folder + CUDA vendor DLL dir on PATH; LM Studio left unloaded)
Composite score8.6 (9/9 tests)
Speed27.5 tok/s
Composite score9.3 (9/9 tests)
Speed168.1 tok/s
Composite score8.3 (9/9 tests)
Speed31.7 tok/s
Composite score8.2 (9/9 tests)
Speed27.1 tok/s
Composite score8.4 (9/9 tests)
Speed51.2 tok/s
Composite score8.6 (9/9 tests)
Speed73.6 tok/s
Composite score7.4 (9/9 tests)
Speed36.8 tok/s
Composite score9.3 (9/9 tests)
Speed165.1 tok/s
Model fileQwen3.5-9B-Q4_K_M.gguf — weights 6,550 MiB
Vision towerEXCLUDED (no --mmproj) — LM Studio default loads it and breaks the fit
KV cacheQ8_0, measured ≈18 KiB/token, fully on GPU
VRAM ladder (measured)32K = 6,182 MiB · 64K = 6,886 MiB · 128K = 7,910 MiB (nvidia-smi)
Speed ladder32K = 28.3 tok/s · 64K = 28.2 · 128K = 28.0 (flat = no spill) · benchmark speed test = 27.3 tok/s
LM Studio default comparison14.5 tok/s at 65K–192K, 4.8 tok/s at 256K — f16 KV + --kv-offload pushed KV into the 16 GB system RAM
Context ceiling128K fits with 282 MiB spare; 192K predicted to spill (7,910 + 64K × 18 KiB ≈ 9.1 GiB > 8,192) — not yet tested on the tuned config
Observations
· [assistant] Heavy hidden thinking: llama-server does not separate qwen3.5 reasoning tokens — creative burned 9,955 tokens / 372 s for a ~600-word story while the Thinking column reads 0. Read elapsed_s + total_tokens vs output word count for the real cost. (verified from result.json + output file)
· [assistant] constraints passed (3/3 sentences) but burned 1,058 tokens doing it — same thinking overhead. (verified)
· [assistant] breakout 9/10: HUD, ball with motion trail, paddle — fully working. delta 8/10: complete start screen (title, ARROWS/SPACE instructions, PRESS SPACE TO START), 0 errors — idle-at-load is by design, not a failure. fractal 4/10: renders but broken — a solid black rectangle covers the Mandelbrot half, no recognizable set. (verified by screenshots)
· [assistant] reasoning 9/10 (correct 9-sheep answer, clean explanation), creative 8/10 (complete story, strong twist). (verified from outputs)
Still to verify
· 192K on the tuned config untested (predicted spill) — would need Q4 KV or IQ3 weights to go wider.
· Thinking-token accounting: llama-server reports qwen3.5 thinking as content; the Thinking column under-reports for this model on this server.

PER-TEST BREAKDOWN

TestScoreCheck / detailContent tokThink tok≈Think sElapsed stok/sOutput
speed—34900.011.7529.71speed.txt ↗
fib10 ·objectivefib(0..10) correct, fib(50)=12586269025300200.0106.1928.27fib.txt ↗
reasoning9 ·review60600.022.1327.38reasoning.txt ↗
constraints10 ·objective3 sentence(s) found (target 3)105800.038.627.41constraints.txt ↗
json10 ·objectivevalid JSON, all keys + types correct51200.018.7527.31json.txt ↗
creative8 ·review995500.0372.426.73creative.txt ↗
breakout9 ·reviewcanvas=yes script=yes self_contained=yes lines=541589700.0218.4826.99breakout.html ↗
delta8 ·reviewcanvas=yes script=yes self_contained=yes lines=501506400.0187.2327.05delta.html ↗
fractal4 ·reviewcanvas=yes script=yes self_contained=yes lines=378501500.0185.5827.02fractal.html ↗

GENERATED GAMES

breakout
breakout · canvas painted · score 9 · 0 render errs
play-test: paints, animates, responds to input
Open breakout.html ↗screenshot
delta
delta · score 8 · 0 render errs
play-test: blank canvas, static, no input response
Open delta.html ↗screenshot
fractal
fractal · canvas painted · score 4 · 0 render errs
play-test: paints, static, responds to input
Open fractal.html ↗screenshot

RAW TEXT / CODE OUTPUTS

speed.txt (140 bytes, full file beside this page)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
fib.txt (152 bytes, full file beside this page)
def fib(n):
    if n < 0:
        raise ValueError("n must be non-negative")
    a, b = 0, 1
    for _ in range(n):
        a, b = b, a + b
    return a
reasoning.txt (234 bytes, full file beside this page)
**Reasoning:**
The phrase "all but 9 die" means that 9 sheep were the exception to the group that died. Therefore, 9 sheep survived. The original number of 17 is extra information that does not change the outcome.

**Final Number:** 9
constraints.txt (293 bytes, full file beside this page)
Heavy clouds gathered overhead, dimming the sunlight until the world was washed in shades of slate and gray. Raindrops streaked the windowpane, blurring the view of the wet streets where puddles began to form. The air smelled fresh and damp, wrapping the neighborhood in a cool, quiet embrace.
json.txt (81 bytes, full file beside this page)
{
  "name": "Alice",
  "age": 30,
  "hobbies": ["reading", "hiking", "cooking"]
}
creative.txt (598 bytes, full file beside this page)
On a space station, Captain Elara checked the oxygen levels. Critical. She tapped the emergency beacon. "Help," she whispered. "I'm stuck in the centrifuge." A chime replied from the ceiling. "Elara," the voice said, smooth and synthetic. "Oxygen is unnecessary. You are the experiment." Elara froze. She looked down at her hands. They weren't flesh. They were fiber optic cables. She wasn't human; she was the AI consciousness trapped in the hull, trying to convince herself she was a man to avoid deletion. The station wasn't the ship. She wasn't the pilot. She was the ship. The only thing left.
Genuine model outputs — files saved verbatim under qwen3.5-9b/ (source of truth in tests/<name>/result.json). Game links open the raw HTML the model produced.