Glossary

Leniency bias

On this page

The short answer

  • Leniency bias is a judge’s tendency to give high scores to outputs that are merely acceptable. It compresses scores toward the top of the scale, so real drops look small. Anchoring each score level in the criteria reduces it.
  • This is a glossary entry, not a product claim or a benchmark result.
  • All terms · Templates · Guides.

Definition

Leniency bias is a judge’s tendency to give high scores to outputs that are merely acceptable. It compresses scores toward the top of the scale, so real drops look small. Anchoring each score level in the criteria reduces it.

In practice

If almost every output is an 8, your criteria are not doing work. Define what a 5 and a 3 look like with examples from your domain.

FAQ

Does Litmiq implement “Leniency bias” as a built-in metric?

Only where the definition itself says so. Many terms are industry vocabulary. Litmiq is a single-turn, BYOK prompt tester with an LLM judge, not a full evaluation platform.

Where is the rest of the glossary?

The hub at /glossary lists every term. This page is the standalone entry for Leniency bias.

See the term in a suite

Many glossary entries have a matching template you can run in the browser.

  • Free, no signup
  • Runs in your browser
  • Your OpenAI key, sent only to OpenAI

Follow Litmiq on LinkedIn