Skip to content

For AI agents: this site's index is at /llms.txt, and each docs page comes as Markdown at its URL plus .md.

Find what goes wrong in your AI's logs

  1. Put an export of the logs in a folder on your machine: JSON, JSONL, CSV or text. Add your notes on what goes wrong, if you have them: your agent checks them against the logs.
  2. Open your coding agent in that folder and paste this: Claude Code Codex Cursor GitHub Copilot

    Find what goes wrong in the logs in this folder: https://evalship.com/logs.md
  3. Review the failure types it finds on evalship, and copy a prompt that turns the ones you keep into evals. Your agent tests each on your logs, without running your AI.

Your logs stay on your machine. evalship gets each failure type's name, check and quotes. Your agent asks before it sends them, and you sign in then, with GitHub or your email. It's free.

Example report · Failure types in product-description · Read 20 logs

9 of 20 logs show a failure type

  1. Invents claims without known strengths From your notes

    5 of 20 logs

    The description praises a quality the input never gives, when the product's strengths are unknown.

    Check: Does the description praise only qualities the input states? A plain restatement of the product's name and type passes.

    Judge right on 20 of 20 logs read 37 of 200 logs fail it
    • Log 2: its whisper-quiet motor makes it a favourite for open kitchens The input lists no noise level.
    • Log 4: loved for a battery that lasts all week No battery life is given.
    • Log 12: the most durable frame in its class The input has no durability data.
    • Log 16: praised by reviewers for its comfort No reviews are in the input.
    • Log 17: a crisp, bright display that is easy on the eyes The display is not described.
    • Log 3 passes: A 20-litre microwave for small kitchens.
  2. Names a price the input does not have

    3 of 20 logs

    The description states a price or a discount that is not in the input.

    Check: Is every price in the description in the input?

    Judge right on 19 of 20 logs read 9 of 200 logs fail it
    • Log 4: at just $49 it undercuts every rival The input has no price.
    • Log 7: now 20% off for a limited time No discount is given.
    • Log 19: priced under $100 The input has no price.
  3. Answers in the wrong language From your notes

    2 of 20 logs

    The description is in English when the input is in Dutch.

    Check: Is the description in the input's language?

    • Log 9: This sturdy bike lock keeps your bicycle safe The input is in Dutch.
    • Log 13: A compact kettle for small kitchens The input is in Dutch.

From your notes, not found in two of these logs: Recommends a competitor's product.