Litmiq prompt evaluator — compare OpenAI and Anthropic models side by side

Founding Lifetime: Litmiq Pro for $149 once, first 100 only

Litmiq is a free, no-signup browser app for prompt evaluation. Compare OpenAI and Anthropic models side by side, or bring a key for Google Gemini, xAI Grok, Mistral, DeepSeek, Groq, OpenRouter or Perplexity, score outputs with an LLM judge, and share results as a link. Your API key stays in this browser; Litmiq has no eval backend. Perplexity is forwarded through a no-store proxy.

No API key yet? Watch a recorded 30-second demo run

Try it in 30 seconds, no key needed

It starts on its own when this block is on screen. Watch a real Litmiq run: one support-bot prompt, 8 test cases, 3 models, scored by an LLM judge. These are recorded outputs from Llama 3.3 70B, Gemma 4 26B, Llama 3.1 8B on Cloudflare Workers AI (13 Nov 2026), replayed in your browser, not generated live.

Suite: Compare OpenAI models on one prompt (Harbor Lane Books support, fictional) · recorded run (JSON)

Your turn: run this suite on GPT or Claude with your own key. Load this suite
Litmiq Pro200 saved suites, CI on a schedule and regression alerts, $19/month. This app stays free.
Models to Test

Grade with a different provider than the models you test to reduce self-preference bias. The judge uses the key for its provider.

Test Cases
Your first eval in three steps
  1. Add a provider key. It stays in this browser and goes only to that provider (Perplexity via a no-store proxy).
  2. Keep the sample prompt and test cases, or paste your own on the left.
  3. Press Run Evals. Outputs and per-metric judge scores appear here.

Founding Lifetime: Pro for $149 once, first 100 only