Events

Events measure the model's work for you: its notes on pull requests and its checks of production AI calls. One pool covers the whole account each month: 5,000 events on Free, 20,000 on Pro and 100,000 on Business.

What counts

The model's work	When	Events
Checks a production AI call	The call gets the reply checks or a `judge` check	1
Writes notes on a pull request	A push changes prompts, AI calls, or eval code or data	About 40 a push, more for big diffs
Checks a merged pull request again	Its own run couldn't confirm that it fixed an open review flaw	By how much it reads, like a push

Production calls: an AI call with no checks of its own in .evalship.yml gets the reply checks, which the model reads, so each one is an event. A call whose checks are all rules, such as json or contains, is checked by the rules alone and counts nothing. Listing replies or a judge check brings the model back. See Monitor.

Pull requests: the model also writes notes on a push that changes a file an open review flaw or an earlier pull request's eval flaw is on, or that has new failures evalship can't explain on its own. See How the PR comment works.

What never counts

Eval runs, results, PR comments, the evalship check and gating.
The checks rules make, on pull requests and in production, and production steps.
Production calls themselves, up to the fair-use limit.
A pull request the model gives no answer on.
The review and Optimize: your coding agent runs them with your own model subscription and keys.

With Model notes off in the account's settings, the model reads nothing, so nothing counts. PR comments and production checks then use the rules alone.

The pool

One pool covers every repository in the account, pull requests and production alike. It resets each month: from the day the subscription started, or on Free on the 1st (UTC). A yearly plan gets the same monthly pool, reset each month from the day the year started.

Billing shows how many are used and when they reset. Account owners also see where they went, by repository.

Past the pool

Pro and Business keep going: each 1,000 events past the pool is $1, billed as additional usage.
Free and the trial stop the model until the pool resets. PR comments skip the model's notes and say so, and the rules alone check production calls. Account owners get an email when it happens.
A failed payment stops a paid plan at the pool the same way, until the payment goes through.

Public repositories

Pull requests on public repositories, which anyone can open, have an allowance of their own: the same size as the pool, and never billed. Past it, their PR comments skip the model's notes until it resets. Production calls count toward the pool, from public repositories too.