For agents

Before your coding agent says a change to a prompt or model is done, it runs the evals the change affects before and after, and evalship says whether it got worse beyond chance. No account needed.

The page your agent follows →What your agent does →

Add one line to AGENTS.md

Or to CLAUDE.md, whichever your agent reads. It runs the evals on your machine with your keys. evalship gets only which test cases passed, and keeps none of it.

What your agent hears back

The demo's summarizer, with a prompt that names what a summary must keep:

Compared with before over 14 test cases × 3 tries:
- after: better beyond chance, passed 41 of 42 tries (before 34 of 42), 3 to 32 points more often, 8% more expensive. Lists what a summary keeps: parties, dates, amounts, deadlines and termination terms (prompts/summarize.md:3)
evalship kept none of it. Connect the repository at https://evalship.com/start to compare every change with its history, which tells flaky evals from broken ones.