Glossary
Human evaluation
On this page
The short answer
Definition
Human evaluation is when people, not a model, score or compare outputs. It is the usual reference for whether a judge is calibrated, and it is slower and more expensive than an automatic judge.
In practice
Use humans on a sample, especially for tone, safety and new task types. Litmiq does not assign work to reviewers; you can still sit with a side-by-side run and mark cases yourself.
If you only have an LLM judge and no human sample, treat rankings as estimates.
Related terms
FAQ
Does Litmiq implement “Human evaluation” as a built-in metric?
Only where the definition itself says so. Many terms are industry vocabulary. Litmiq is a single-turn, BYOK prompt tester with an LLM judge, not a full evaluation platform.
Where is the rest of the glossary?
The hub at /glossary lists every term. This page is the standalone entry for Human evaluation.
See the term in a suite
Many glossary entries have a matching template you can run in the browser.
- Free, no signup
- Runs in your browser
- Your OpenAI key, sent only to OpenAI