← all models

MODEL RESULTS — MELLUM2.1-12B-A2.5B-THINKING

C:\Users\admin\models_bench\Mellum2.1-12B-A2.5B-Thinking-MXFP4_MOE.gguf · server http://erwing · run 2026-10-13 00:13:32 → 2026-10-13 00:17:53

OVERALL

Composite: 8.3 ✓ 8/8 tests
Speed (median): 68.0 tok/s
Thinking cost: 0 tokens · ≈0s · starved: none

ENVIRONMENT & LOAD (verified)

Run metadata (recorded by runner)
serverhttp://erwing
date2026-10-13 00:13:04
hardwareerwing · GTX 1080 8 GB · 16 GB RAM
context24K (24576) — fully resident (16K and 24K both ~71 tok/s)
tuningllama.cpp 2.55.0 CUDA · n-gpu-layers 99 · flash-attn · KV q8_0 · no vision · PAR 1

PER-TEST BREAKDOWN

TestScoreCheck / detailContent tokThink tok≈Think sElapsed stok/sOutput
speed—18300.02.6469.3speed.txt ↗
fib10 ·objectivefib(0..10) correct, fib(50)=1258626902537700.05.4569.13fib.txt ↗
reasoning10 ·review11400.01.7166.66reasoning.txt ↗
constraints10 ·objective3 sentence(s) found (target 3)17400.02.5269.16constraints.txt ↗
json10 ·objectivevalid JSON, all keys + types correct21100.03.1167.95json.txt ↗
creative7 ·review21700.03.1568.92creative.txt ↗
breakout4 ·reviewcanvas=yes script=yes self_contained=yes lines=333501700.076.9865.17breakout.html ↗
delta9 ·reviewcanvas=yes script=yes self_contained=yes lines=189333000.053.6262.1delta.html ↗
fractal8 ·reviewcanvas=yes script=yes self_contained=yes lines=175779700.0129.7160.11fractal.html ↗

GENERATED GAMES

breakout
breakout · score 4 · 1 render errs
play-test: blank canvas, static, responds to input, 2 JS errors
Open breakout.html ↗screenshot
delta
delta · canvas painted · score 9 · 0 render errs
play-test: paints, animates, responds to input
Open delta.html ↗screenshot
fractal
fractal · canvas painted · score 8 · 0 render errs
play-test: paints, static, responds to input
Open fractal.html ↗screenshot

RAW TEXT / CODE OUTPUTS

speed.txt (140 bytes, full file beside this page)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
fib.txt (87 bytes, full file beside this page)
def fib(n):
    a, b = 0, 1
    for _ in range(n):
        a, b = b, a + b
    return a
reasoning.txt (69 bytes, full file beside this page)
All but 9 die means that 9 sheep remain alive.  

**Final number:** 9
constraints.txt (221 bytes, full file beside this page)
The sky wept a steady gray veil, turning the streets into reflective mirrors. Streetlights flickered, casting soft halos through the misty air. Inside, the world felt cozy, as if time itself had slowed to a gentle rhythm.
json.txt (78 bytes, full file beside this page)
{"name":"Alice Smith","age":28,"hobbies":["painting","cycling","photography"]}
creative.txt (595 bytes, full file beside this page)
The station’s lights flickered as Dr. Lira entered the cargo bay, where a single, sealed pod rested on a steel table. Outside the viewport, Earth’s blue marble glowed, indifferent. She pressed the release; the pod’s latch clicked open, revealing a newborn baby swaddled in a blanket marked “Project Eve.” The medical logs on the console showed a miscarriage — yet here was a child, perfectly healthy. A sudden alarm blared, and the hatch sealed shut. The baby’s first cry echoed, but the voice that answered — *her own* — came from the intercom, confirming she’d been the test subject all along.
Genuine model outputs — files saved verbatim under mellum2.1-12b-a2.5b-thinking/ (source of truth in tests/<name>/result.json). Game links open the raw HTML the model produced.