Glossary
Online evaluation
On this page
The short answer
- Online evaluation scores real production traffic as it happens, usually on a sample of requests. It catches problems your test set did not anticipate and needs a tracing or observability platform. Litmiq does not do online evaluation.
- This is a glossary entry, not a product claim or a benchmark result.
- All terms · Templates · Guides.
Definition
Online evaluation scores real production traffic as it happens, usually on a sample of requests. It catches problems your test set did not anticipate and needs a tracing or observability platform. Litmiq does not do online evaluation.
In practice
Use offline suites to catch known failures before you ship, and online eval when you must watch live traces. The comparison pages describe tools that do tracing.
Related terms
FAQ
Does Litmiq implement “Online evaluation” as a built-in metric?
Only where the definition itself says so. Many terms are industry vocabulary. Litmiq is a single-turn, BYOK prompt tester with an LLM judge, not a full evaluation platform.
Where is the rest of the glossary?
The hub at /glossary lists every term. This page is the standalone entry for Online evaluation.
See the term in a suite
Many glossary entries have a matching template you can run in the browser.
- Free, no signup
- Runs in your browser
- Your OpenAI key, sent only to OpenAI