Glossary
Golden dataset
On this page
The short answer
- A golden dataset is a curated, versioned set of test cases, often with labels or expected behavior, used as the fixed yardstick for regression tests and model comparisons. It is another name for a well-kept test suite plus its labels.
- This is a glossary entry, not a product claim or a benchmark result.
- All terms · Templates · Guides.
Definition
A golden dataset is a curated, versioned set of test cases, often with labels or expected behavior, used as the fixed yardstick for regression tests and model comparisons. It is another name for a well-kept test suite plus its labels.
In practice
Start small and grow it from production failures. Export it (Litmiq can export JSON or promptfooconfig.yaml) and keep it in version control so CI and the browser run the same cases.
A golden dataset is not automatically correct forever: policies change, and old expected behavior should be reviewed.
Related terms
FAQ
Does Litmiq implement “Golden dataset” as a built-in metric?
Only where the definition itself says so. Many terms are industry vocabulary. Litmiq is a single-turn, BYOK prompt tester with an LLM judge, not a full evaluation platform.
Where is the rest of the glossary?
The hub at /glossary lists every term. This page is the standalone entry for Golden dataset.
See the term in a suite
Many glossary entries have a matching template you can run in the browser.
- Free, no signup
- Runs in your browser
- Your OpenAI key, sent only to OpenAI