Data and privacy

This page lists what evalship stores, what it reads from GitHub, what it sends to a model, and who processes it.

Last updated: November 3, 2026

Who we are

evalship is run by Garm Tech B.V., registered in the Netherlands under Chamber of Commerce number 99602075, at Staalkade 6 O, 1011 JN Amsterdam, the Netherlands. Garm Tech is the controller under the GDPR for the personal data of people who use evalship: sign-in details, account and billing records, and invites.

For personal data inside the eval results, reviews and code your workflows and agent send to evalship, your organization is the controller and Garm Tech processes it on your behalf, under the data processing agreement.

What evalship stores

evalship does not store your source code. It reads files through the GitHub API to find AI calls and to check the review's citations, and keeps only the pull request descriptions, file names, locations and diff excerpts listed above. The one diff it gets from your machine, of a change checked before it's committed, it deletes once a commit of the same code reaches GitHub, or after 7 days. Artifacts are downloaded to a temporary file, parsed and deleted.

Eval inputs, outputs, failure messages, names and metadata are stored as your eval run writes them. Keep secrets and personal data out of them, or have an owner turn off Eval inputs and outputs in the account's settings: evalship then keeps no input, output or failure message, and no metadata but the model under test, and deletes the ones it had. See Security.

What evalship reads from GitHub

What goes to a model

evalship makes two kinds of model call. On each pull request run that changes prompts, AI calls, eval code or data, or has newly failing evals or a touched review flaw its rules can't settle, it asks a language model to note what the change does to the evals and, where the rules couldn't, to name failure patterns and the likely cause and say which review's flaws the diff fixes. After a pull request merges, if its own run couldn't confirm that it fixed review's flaws it touched, evalship asks once more with the default branch's first results after the merge. Neither call is made for a repository whose evals run on its members' machines, nor for a change checked before it's committed: there their coding agent writes the notes on their own model plan, and evalship checks them as it does its model's. Today that model is Claude Haiku 5.5, through Anthropic's API. We measure new models as they come out and switch when one writes better notes, so the model may change; it is always one of the routes below:

• Anthropic's API, for Anthropic's Claude models. Under Anthropic's commercial terms it doesn't use these requests to train its models and deletes them within 30 days, or keeps them for up to 2 years if its safety systems flag them as breaking its usage policy.
• OpenRouter to Microsoft Azure, for OpenAI's models. These requests go only to Azure, and only to its endpoints with zero data retention that don't collect data.

The request contains:

• the pull request's title, the first 8,000 characters of its description, and the list of changed files;
• the diffs of the changed files that matter for the evals (eval code and data, files with prompts or AI calls, prompt files from the review, CI workflows), up to 4,000 characters each and 40,000 in all;
• the review's suites, prompts and open flaws, and the notes members dismissed on the pull request, with their reasons;
• up to 10 open notes from pull requests merged in the last 30 days, on files this pull request changes;
• up to 4 of the changed eval files at the pull request's head commit, 300 lines each;
• files and folder listings from the repository at the pull request's head commit that the model asks to read while checking its notes, 300 lines a read, in up to 14 rounds, and the files its notes cite;
• up to 20 newly failing evals, unless rules already named and traced them: their key and suite, the first 600 characters of their input and output, and the first 400 characters of their failure message;
• after a merge, the default branch's results: each suite with its test case keys.

Nothing is sent when an owner has turned off Model notes in the account's settings, for runs that change none of these, for private repositories that aren't enabled, once the account has used its PR analyses for the period (on public repositories, and on private ones on Free, during a trial or while a payment has failed), or past 10 calls a day for one pull request or 20 an hour for an account's public repositories. See How the PR comment works.

The review is written by your own coding agent on your machine, with your model subscription: evalship's model doesn't write it.

Sub-processors

This list changes as evalship does, and the date at the top of the page with it. To hear about a new sub-processor at least 14 days before it gets your data, email hello@evalship.com to join the notice list.

Where data is stored

evalship runs on Railway in the United States, in its US West region in California. GitHub, Cloudflare, Railway, Anthropic, OpenRouter, Microsoft, Resend and Sentry are US companies, so personal data leaves the European Economic Area. These transfers rely on the providers' standard contractual clauses or, where a provider is certified, the EU-US Data Privacy Framework.

GitHub App permissions

Cookies

evalship sets three cookies of its own, all strictly necessary:

• _evalship_session, set on your first page view. It holds the security token that protects evalship's forms from forged requests, and once you sign in, who you are. While you sign in it also carries where you came from, which is saved with your user record the first time. It has no expiry date, so it lasts until your browser's session ends, and a sign-in lasts 30 days at most.
• last_sign_in, set when you sign in, so the sign-in page can mark the way you used last. It lasts 1 year.
• consent, which remembers your choice on the cookie banner for 1 year.

To see where people get stuck, evalship uses Microsoft Clarity. It records how you use the site: the pages you open and their addresses, and where you click, scroll and move the mouse. On signed-in pages Clarity gets blocks instead of text, so it doesn't record what your repositories, evals, reviews and pull requests contain, but page addresses name the repository. Signed in, Clarity also gets your evalship user id and login, so a recording can be matched to your account. Clarity never loads on pages whose address carries a sign-in, claim, connect or unsubscribe link. It sets the cookies _clck and _clsk to connect the pages of a visit, and Microsoft sets its own on clarity.ms. Clarity also contacts c.bing.com, a Microsoft Advertising domain, but its ad storage is always off, so nothing is used for ads.

In the EEA, the UK and Switzerland (judged by the country of your IP address), Clarity loads only after you accept it on the cookie banner. Elsewhere it loads unless you reject it. Rejecting deletes Clarity's cookies on evalship.com. You can change your choice here:

Legal basis

Where the GDPR applies, evalship relies on:

• Performance of a contract (Art. 6(1)(b)): signing you in, running the service for your repositories, sending sign-in links, invites and weekly reports, counting users and billing.
• Consent (Art. 6(1)(a)): Clarity's session recordings in the EEA, the UK and Switzerland, only after you accept them. You can withdraw it under Cookies.
• Legitimate interests (Art. 6(1)(f)): error tracking, keeping the service secure and preventing abuse, and understanding how evalship is used: where people come from when they sign up, the days they use it, how far setup with their coding agent gets, and how accounts use it over time. The data involved is limited to what's listed above. To object, email hello@evalship.com.

Retention

When you uninstall the GitHub App, or suspend it, evalship stops processing and posts nothing. Uninstalling also sets a paid plan to cancel at the end of the current billing period, and evalship emails the account's owners. 30 days after you uninstall, once any subscription has ended, the account is anonymized: its eval results, reviews, flaws' text and dismissal reasons, pull request titles, authors and dismissed notes, branch names, file paths and diffs, repository names, its GitHub ids and its Polar ids are deleted, along with who was invited and who opened its reports. What's left doesn't name the account, its repositories or its people: that an account of its type joined on a date, its plan and payments, and when it ran evals, opened pull requests, and claimed a review, with each run's pass count. We keep that to understand how evalship is used over time. If you reinstall within those 30 days, your history is still there; after that, a reinstall starts a new account.

A repository goes the same way 30 days after you remove it from the App's installation, or delete or transfer it on GitHub: its eval results, reviews, flaws' text, experiments, pull request titles and authors, who sent results from their machine, diffs and name are deleted, while the account and its other repositories stay. Added back within those 30 days, it keeps its history.

Deletion requests

To have an account, a repository's data or your user record deleted sooner, email hello@evalship.com with the GitHub account or repository name.

Your rights

Under the GDPR you have the right to access, correct, delete, restrict or object to the processing of your personal data, and the right to data portability. Email hello@evalship.com and we respond within one month.

You can also complain to a supervisory authority. In the Netherlands, that's the Autoriteit Persoonsgegevens.

Changes to this page

When this page changes, the date at the top changes. For material changes, we email account owners before they take effect.

Contact

Questions about privacy: hello@evalship.com.