Skip to content
evalship Get started →

evalship-hq / ponytail

Public

Eval runs

7beb065 main Fix the flaky evals → evals: 20 / 21 5d ago
8ec83e5 #6 Rate-limit check: read every Python block, and no time words in strings evals: 21 / 21 eval flaws: 0 5d ago
858c447 #3 Demo: keep replies to the code and two lines of prose 1 earlier evals: 20 / 21 eval flaws: 0 5d ago
a436b00 #3 Demo: keep replies to the code and two lines of prose evals: 17 / 21 eval flaws: 0 5d ago
545983b main evals: 20 / 21 5d ago
cd02498 #4 Fix the eval failing on main: hardware calibration 1 earlier evals: 21 / 21 eval flaws: 0 5d ago
145080c #4 Fix the eval failing on main: hardware calibration evals: 20 / 21 eval flaws: 0 5d ago
7e38f14 main evals: 17 / 21 5d ago
180f3d9 #5 Score the single-shot evals on correctness alone evals: 20 / 21 eval flaws: 0 5d ago
c01cd19 main evals: 19 / 21 5d ago
All 19 runs →
Evals passing on main
20 / 21
Regressions in PRs
1 → 0
Eval flaws in PRs
7 → 0
Untested AI calls on main
1
Flaky evals
1
Eval cost
$9.46
Eval time per PR
2 min