Glossary

Regression testing

On this page

The short answer

  • Regression testing is re-running the same test suite after every change and comparing the results with the baseline to catch anything that got worse. For prompts, read every case whose score dropped; scores show where to look, not what happened.
  • This is a glossary entry, not a product claim or a benchmark result.
  • All terms · Templates · Guides.

Definition

Regression testing is re-running the same test suite after every change and comparing the results with the baseline to catch anything that got worse. For prompts, read every case whose score dropped; scores show where to look, not what happened.

In practice

The prompt regression testing guide and its starter template are built for this loop. Put the same suite in CI if you already run Promptfoo or a similar runner.

FAQ

Does Litmiq implement “Regression testing” as a built-in metric?

Only where the definition itself says so. Many terms are industry vocabulary. Litmiq is a single-turn, BYOK prompt tester with an LLM judge, not a full evaluation platform.

Where is the rest of the glossary?

The hub at /glossary lists every term. This page is the standalone entry for Regression testing.

See the term in a suite

Many glossary entries have a matching template you can run in the browser.

  • Free, no signup
  • Runs in your browser
  • Your OpenAI key, sent only to OpenAI

Follow Litmiq on LinkedIn