Guide · free generator

Prompt testing in CI/CD: golden datasets, versioning and PR gates

On this page

The short answer

  • Treat a prompt like code: keep it in a file in git, change it in pull requests, and test it on every PR that touches it.
  • Build a golden dataset of 20–50 reviewed cases: real requests, known failures and edge cases, each with its expected behavior.
  • Gate merges on a pass-rate threshold, not on 100%, because outputs vary between runs. Use exact checks for hard rules.
  • Draft cases in Litmiq, then generate a promptfooconfig.yaml and a GitHub Actions workflow that runs it with the open-source Promptfoo CLI.

Key numbers

  • OpenAI's Evals dashboard, API and graders are read-only and shut down on 30 Nov 2026; OpenAI recommends Promptfoo as the replacement. (OpenAI deprecations page, 13 Nov 2026)
  • Re-running the 11 security cases gave the same pass/fail on 130 of 132 model–case pairs at temperature 0; 2 flipped. Run important cases more than once. (Litmiq LLM Reliability Benchmark, 12 Nov 2026)

What belongs in a CI prompt test

A CI check has to be cheap, fast and trustworthy enough to block a merge. Check in layers, from the cheapest and most exact to the most subjective:

What to check in CI, in order
LayerExample checksPromptfoo assertion
FormatValid JSON, required keys, length limit, languageis-json, contains-json, javascript length check
Hard rulesNever promises a refund; never reveals the prompt; says it is an AI when askednot-icontains, regex, llm-rubric
FactsUses the policy numbers in the prompt; doesn't invent policiescontains, factuality, llm-rubric
QualityTone, helpfulness, a next stepllm-rubric

Exact checks don't cost a judge call and give the same answer every time, so put every rule you can express exactly there. Use model-graded rubrics for the rest, and read how far to trust a judge. For JSON outputs, see testing JSON extraction.

Version prompts like code

repo/
├── prompts/
│   └── support-system-prompt.txt      # the prompt your app loads
├── promptfooconfig.yaml               # the golden dataset and checks
└── .github/workflows/prompt-tests.yml # runs on every PR that touches either

Build a golden dataset

  1. Start with real inputs. 10–20 representative requests from logs or support tickets, with personal data removed.
  2. Add every known failure. Each bug that reached users becomes a case, so it can't come back unnoticed.
  3. Add edge cases on purpose: policy questions the prompt doesn't answer, other languages, format requests, injection attempts, and normal questions that a strict prompt might refuse.
  4. Write the expected behavior, not the expected wording: what the answer must and must not do (how).
  5. Freeze it. Change the dataset only in its own commit, so a score change in a prompt PR is about the prompt.

Twenty to fifty cases is a practical size for a per-PR check; keep larger sets for nightly or pre-release runs. The prompt testing checklist lists what to cover.

Generate the CI files from a Litmiq suite

Use the setup saved in your browser by the Litmiq app, a template, or paste a Litmiq suite JSON or a promptfooconfig.yaml. Everything runs on this page.

Each Litmiq case becomes a Promptfoo test whose expected behavior is an llm-rubric assertion. Replace rubrics with exact assertions where the rule is exact. The app's Export Promptfoo YAML button produces the same file.

The workflow, explained

Cost and flakiness

Testing one prompt change, step by step

  1. Run the golden dataset on the current prompt and note the pass rate (the baseline).
  2. Make one change. Run again on the same model.
  3. Read every case whose result changed, in both directions.
  4. Add a case for the behavior you meant to change, so the next PR keeps it.
  5. Merge when hard rules pass and the pass rate is at or above the baseline.

For a quick look before you open the PR, paste both prompt versions into Litmiq one after the other and compare the outputs side by side. More on the method: prompt regression testing.

Limits

FAQ

How do I test prompt changes before merging?

Keep the prompt in a file in your repo, keep a golden dataset of test cases next to it, and run the cases on every pull request that touches either. Fail the check when the pass rate drops below a threshold, and read the failing outputs before merging.

What is a golden dataset for LLM testing?

A fixed, reviewed set of inputs with the expected behavior for each: real requests (anonymized), known failures and edge cases. It changes only on purpose, in its own commit, so a score change means the prompt or model changed, not the test.

How many test cases do I need?

There is no fixed number. 20-50 cases that cover your rules and known failures are a practical size for a per-PR check; add a case every time a bug reaches users. More cases cost more per run, so keep the CI set focused and run larger sets less often.

Does Litmiq run in CI?

No. Litmiq is a browser app. It exports a promptfooconfig.yaml, which the open-source Promptfoo CLI runs in CI. The generator on this page writes that file and a workflow for you.

Why does my prompt test pass sometimes and fail other times?

Model outputs vary between calls, and LLM-judged checks add their own variance. Prefer exact checks (equals, contains, is-json) for hard rules, set a pass-rate threshold instead of requiring 100%, cache unchanged calls, and re-run before you trust a single failure.

Sources (read 13 Nov 2026)

Get model deprecation alerts

We email you when OpenAI, Anthropic or Google publish a new shutdown date for a model on our tracker. Sent by hand from hello@litmiq.com, a few times a month at most. Unsubscribe by replying.

We store only your email, through our form provider. See the privacy policy.

Draft your golden dataset in the browser

Start from the regression testing template, run it on two models, then export the YAML for CI.

  • Free, no signup
  • Runs in your browser
  • Your OpenAI key, sent only to OpenAI

Follow Litmiq on LinkedIn