Obelum translator evals
Every published run, newest first. Each lives at its own address, /evals/<commit>/
(/evals/<commit>-single/ for single source), which does not change when a newer run is published.
The newest of each: peer system · single source.
| date | system | commit | model | cases × runs | passed |
|---|---|---|---|---|---|
| 2026-10-14 | peer system | 907534b | claude-sonnet-5-5 | 53 × 3 | nodiff 108/159 |
| 2026-10-14 | single source | 4032790 | claude-sonnet-5-5 | 50 × 3 | nodiff 113/150 |
| 2026-10-12 | single source | 5b70239 | claude-sonnet-5-5 | 50 × 3 | hunks 150/150 · diff 147/150 · difflook 149/150 |
| 2026-10-12 | peer system | 5b70239 | claude-sonnet-5-5 | 53 × 3 | hunks 158/159 · diff 156/159 · difflook 159/159 |
| 2026-10-12 | single source | e5a0bbf | claude-sonnet-5-5 | 49 × 3 | hunks 145/147 · diff 146/147 · nodiff 113/147 |
| 2026-10-12 | peer system | e5a0bbf | claude-sonnet-5-5 | 52 × 3 | hunks 155/156 · diff 153/156 · nodiff 114/156 |
| 2026-10-10 | single source | 6629f4a | claude-sonnet-5-5 | 49 × 3 | hunks 147/147 · nodiff 104/147 |
| 2026-10-10 | peer system | eb99223 | claude-sonnet-5-5 | 52 × 3 | hunks 153/156 · nodiff 113/156 |
| 2026-10-10 | peer system | 8522df6 | claude-sonnet-5-5 | 52 × 3 | hunks 155/156 |