Glossary

CI eval gate

On this page

The short answer

  • A CI eval gate is an automated test run in continuous integration that blocks a change from merging when eval results fall below a threshold. Code-first tools such as Promptfoo and DeepEval are built for this; see prompt testing tools compared.
  • This is a glossary entry, not a product claim or a benchmark result.
  • All terms · Templates · Guides.

Definition

A CI eval gate is an automated test run in continuous integration that blocks a change from merging when eval results fall below a threshold. Code-first tools such as Promptfoo and DeepEval are built for this; see prompt testing tools compared.

In practice

Litmiq does not run in GitHub Actions. You can draft cases in the app and export promptfooconfig.yaml for CI. The GitHub Actions integration page and the CI/CD guide show that workflow.

FAQ

Does Litmiq implement “CI eval gate” as a built-in metric?

Only where the definition itself says so. Many terms are industry vocabulary. Litmiq is a single-turn, BYOK prompt tester with an LLM judge, not a full evaluation platform.

Where is the rest of the glossary?

The hub at /glossary lists every term. This page is the standalone entry for CI eval gate.

See the term in a suite

Many glossary entries have a matching template you can run in the browser.

  • Free, no signup
  • Runs in your browser
  • Your OpenAI key, sent only to OpenAI

Follow Litmiq on LinkedIn