Monitor and weekly report
Monitor lists each repo's eval runs, each with what to do next, and the week's insights. Every Monday, one email lists what needs your team, each with a button to fix it.
acme_demo's evals: nothing needs you this week, 2 regressions caught before merge
evalship
To: acme_demo's team
Mon 8:00
Nothing needs you this week
 |
45 of 46 evals pass on main |
 |
Caught 2 regressions before merge
#128 Shorter support prompt · fixed |
 |
2 eval flaws left for later · In the review
Replies that leak customer data still pass Evals pass when the judge is down Fix them with your coding agent |
 |
1 flaky eval, ignored on pull requests until fixed: Casual tone |
 |
$34.04 on evals +19% · evals take about 4 minutes on a pull request |
See what evalship finds in your evals
Ask your coding agent:
Set up https://evalship.com/setup.md