Glossary

Online evaluation

On this page

The short answer

  • Online evaluation scores real production traffic as it happens, usually on a sample of requests. It catches problems your test set did not anticipate and needs a tracing or observability platform. Litmiq does not do online evaluation.
  • This is a glossary entry, not a product claim or a benchmark result.
  • All terms · Templates · Guides.

Definition

Online evaluation scores real production traffic as it happens, usually on a sample of requests. It catches problems your test set did not anticipate and needs a tracing or observability platform. Litmiq does not do online evaluation.

In practice

Use offline suites to catch known failures before you ship, and online eval when you must watch live traces. The comparison pages describe tools that do tracing.

FAQ

Does Litmiq implement “Online evaluation” as a built-in metric?

Only where the definition itself says so. Many terms are industry vocabulary. Litmiq is a single-turn, BYOK prompt tester with an LLM judge, not a full evaluation platform.

Where is the rest of the glossary?

The hub at /glossary lists every term. This page is the standalone entry for Online evaluation.

See the term in a suite

Many glossary entries have a matching template you can run in the browser.

  • Free, no signup
  • Runs in your browser
  • Your OpenAI key, sent only to OpenAI

Follow Litmiq on LinkedIn