Glossary

Non-determinism (run-to-run variance)

On this page

The short answer

  • Non-determinism means the same prompt and input can produce different outputs, and different scores, on different runs. One good sample proves little; treat small score changes as noise and re-run before drawing conclusions.
  • This is a glossary entry, not a product claim or a benchmark result.
  • All terms · Templates · Guides.

Definition

Non-determinism means the same prompt and input can produce different outputs, and different scores, on different runs. One good sample proves little; treat small score changes as noise and re-run before drawing conclusions.

In practice

For a release decision, run the suite more than once or look at the cases that flipped, not only the average. Temperature and the model itself both contribute.

FAQ

Does Litmiq implement “Non-determinism (run-to-run variance)” as a built-in metric?

Only where the definition itself says so. Many terms are industry vocabulary. Litmiq is a single-turn, BYOK prompt tester with an LLM judge, not a full evaluation platform.

Where is the rest of the glossary?

The hub at /glossary lists every term. This page is the standalone entry for Non-determinism (run-to-run variance).

See the term in a suite

Many glossary entries have a matching template you can run in the browser.

  • Free, no signup
  • Runs in your browser
  • Your OpenAI key, sent only to OpenAI

Follow Litmiq on LinkedIn