Glossary

Prompt testing

On this page

The short answer

  • Prompt testing is running a prompt on a fixed set of test cases and checking each output against what it should do. It borrows the idea of unit tests from software: the same cases, run again after every change, tell you what broke. See prompt regression testing.
  • This is a glossary entry, not a product claim or a benchmark result.
  • All terms · Templates · Guides.

Definition

Prompt testing is running a prompt on a fixed set of test cases and checking each output against what it should do. It borrows the idea of unit tests from software: the same cases, run again after every change, tell you what broke. See prompt regression testing.

In practice

Write expected behavior as checkable rules. After a change, read every case whose score dropped; the score only shows where to look.

FAQ

Does Litmiq implement “Prompt testing” as a built-in metric?

Only where the definition itself says so. Many terms are industry vocabulary. Litmiq is a single-turn, BYOK prompt tester with an LLM judge, not a full evaluation platform.

Where is the rest of the glossary?

The hub at /glossary lists every term. This page is the standalone entry for Prompt testing.

See the term in a suite

Many glossary entries have a matching template you can run in the browser.

  • Free, no signup
  • Runs in your browser
  • Your OpenAI key, sent only to OpenAI

Follow Litmiq on LinkedIn