Integrations
evalship reads the results your evals already write, whatever runs them, and works next to the eval platform you keep for traces and datasets.
Works with the eval frameworks you already use
What evalship reads from each
| Tool | What evalship reads |
|---|---|
| pytest | JUnit XML from --junitxml, with scores from record_property |
| Vitest | JUnit XML from its junit reporter, with scores from annotate |
| Jest | JUnit XML from jest-junit |
| promptfoo | Its own results JSON, one eval per test, prompt and provider |
| Langfuse | A results file your experiment script writes, each test case linking to its trace |
| Braintrust | A results file your eval script writes, each test case linking to its trace |
| LangSmith | JUnit XML from its pytest, Vitest and Jest integrations, or a results file from evaluate() |
Anything else, such as RSpec, Go or a script of your own, writes JUnit XML or evalship JSON. Your evals keep running in your own GitHub Actions or on your own machine, with your own keys, which evalship never sees. Other CI systems: not yet.
See what evalship finds in your evals
Ask your coding agent:
Set up https://evalship.com/setup.md