Glossary

Golden dataset

On this page

The short answer

  • A golden dataset is a curated, versioned set of test cases, often with labels or expected behavior, used as the fixed yardstick for regression tests and model comparisons. It is another name for a well-kept test suite plus its labels.
  • This is a glossary entry, not a product claim or a benchmark result.
  • All terms · Templates · Guides.

Definition

A golden dataset is a curated, versioned set of test cases, often with labels or expected behavior, used as the fixed yardstick for regression tests and model comparisons. It is another name for a well-kept test suite plus its labels.

In practice

Start small and grow it from production failures. Export it (Litmiq can export JSON or promptfooconfig.yaml) and keep it in version control so CI and the browser run the same cases.

A golden dataset is not automatically correct forever: policies change, and old expected behavior should be reviewed.

FAQ

Does Litmiq implement “Golden dataset” as a built-in metric?

Only where the definition itself says so. Many terms are industry vocabulary. Litmiq is a single-turn, BYOK prompt tester with an LLM judge, not a full evaluation platform.

Where is the rest of the glossary?

The hub at /glossary lists every term. This page is the standalone entry for Golden dataset.

See the term in a suite

Many glossary entries have a matching template you can run in the browser.

  • Free, no signup
  • Runs in your browser
  • Your OpenAI key, sent only to OpenAI

Follow Litmiq on LinkedIn