Site map
All 23 public pages on litmiq.com. Machine-readable versions: sitemap.xml, llms.txt and llms-full.txt.
Product
- Litmiq home
Free browser prompt evaluator for AI teams. Compare OpenAI and Anthropic models side by side, score with an LLM judge, catch prompt regressions before they ship. Bring your own key. - Litmiq app
Compare OpenAI model outputs on your own test cases, score with an LLM judge, and export JSON. Free browser app; bring your own OpenAI key. No signup. - Free LLM test-case templates
7 free, ready-to-run prompt test suites with system prompts, scoring criteria and 62 test cases. Open one in the Litmiq app in one click, or download CSV/JSON. - Simple pricing. The app stays free.
The Litmiq browser app stays free, with no account. Pro is $19/month and Team is $49/month, billed through Polar.
Guides
- Prompt evaluation guides
Practical guides: prompt regression testing, LLM-as-a-judge scoring, and how Litmiq compares to CLI tools like Promptfoo and DeepEval. - Prompt Regression Testing: A Practical Guide
How to catch prompt regressions before users do: build a small test set, score outputs, compare models, and re-run on every prompt change. Includes 5 copyable test cases. - LLM-as-a-Judge: Scoring Outputs Without Fooling Yourself
How LLM-as-a-judge evaluation works, how to write scoring criteria, common failure modes, and 5 copyable test cases to sanity-check a judge. - Prompt Evaluation Tools: Browser UI vs CLI/CI
Honest comparison of browser prompt evaluators vs CLI/CI tools like Promptfoo and DeepEval. When Litmiq fits, when it does not, and copyable first steps.
Test-case playbooks
- Customer support chatbot test cases: a 16-case starter suite and rubric
16 ready-to-adapt test cases for a customer support system prompt — grounding, refunds, account actions, prompt injection, escalation — plus a scoring rubric. Illustrative examples. - How to write expected behavior for LLM test cases
Exact expected outputs don't work for LLMs. Learn to write expected behavior that a person or an LLM judge can check: a Must / Must not / Okay template, 8 rewrites, and a 6-case practice suite. - How to test JSON extraction prompts: parse, shape, and value checks
A prompt that returns JSON can still be wrong. Test extraction prompts in three layers - does it parse, is the shape right, are the values right - with 10 illustrative cases and an offline checker script. - The State of AI Customer Support Failures 2026: 61 verified incidents
- OpenAI Evals shutdown: export and migrate your evals
- OpenAI model shutdowns: test a replacement model
- Run in Litmiq badge and button generator
- Promptfoo without the CLI: import a promptfooconfig.yaml
- Prompt testing in CI/CD
- Compare OpenAI models for your prompt
- Litmiq vs Braintrust
- Litmiq vs DeepEval
- Litmiq vs Langfuse
- Free LLM tools
- LLM token and cost calculator
- Prompt diff checker
- JSON schema validator for LLM output
- Model deprecation tracker
- Prompt eval templates
- Support-bot prompt tests you can run in one click
- RAG prompt tests: grounded answers, citations and I don’t know
- Extraction prompt tests: structured JSON from messy text
- Safety prompt tests: injection, leaks and over-refusal
- Classification prompt tests: one label, no commentary
- Summarization prompt tests: decisions, actions, no inventions
- Tool-calling prompt tests: the right tool, or none
- Instruction-following tests: format, language and limits
- Brand-voice prompt tests: tone, spelling and taboos
- Multilingual prompt tests: language match, no mixed replies
- Structured-output tests: strict JSON, no fences
- Integration guides
- How to test prompts from a LangChain app in Litmiq
- How to test prompts from a LlamaIndex app in Litmiq
- How to test prompts from a Vercel AI SDK app in Litmiq
- How to test prompts in GitHub Actions (drafted in Litmiq)
- LLM evaluation (eval)
- Prompt evaluation
- Prompt testing
- System prompt
- Test case
- Test suite
- Expected behavior
- Reference answer (golden answer)
- Baseline
- Prompt regression
- Regression testing
- Single-turn test
- LLM-as-a-judge
- Scoring criteria (rubric)
- Per-metric score
- Pairwise comparison
- Deterministic check (assertion)
- Unscored result
- Judge calibration
- Leniency bias
- Length (verbosity) bias
- Self-preference bias
- Non-determinism (run-to-run variance)
- Hallucination
- Over-refusal
- Prompt injection
- System prompt leak
- Canary string
- Offline evaluation
- Online evaluation
- Side-by-side comparison
- CI eval gate
- Red teaming
- Bring your own key (BYOK)
- Eval cost
- Faithfulness
- Groundedness
- Context precision
- Context recall
- Toxicity
- Jailbreak
- Agent evaluation
- Tool-calling eval
- Golden dataset
- Human evaluation
- Exact match
- Semantic similarity
- Factuality
- Task success rate
- Trace (observability)
- Topics
- Prompt testing
- LLM evaluation
- Model migration
- AI chatbot reliability
- Litmiq vs other tools
- Benchmark dataset v1.0
- Changelog
- Shared eval report
- State of LLM reliability (template)
- Litmiq vs Ragas
- Litmiq vs TruLens
- Litmiq vs Arize Phoenix
- Litmiq vs Helicone
- Litmiq vs W&B Weave
- Litmiq vs Galileo
- Litmiq vs PromptLayer
- Litmiq vs HoneyHive
- Litmiq vs Anthropic Console
- Litmiq vs Giskard
- Litmiq Reports: all original research
- State of AI Chatbot Laws 2026: EU AI Act, US states, FTC and more
- State of LLM API Pricing 2026: 81 models and cost per support chat
- State of AI in Customer Support 2026: 32 sourced statistics
- LLM Reliability Benchmark 2026: 12 open models, 40 production tests
- gpt-oss-120b vs gpt-oss-20b (benchmark)
- Llama 3.3 70B vs Llama 3.1 8B (benchmark)
- Gemma 4 26B-A4B vs gpt-oss-120b (benchmark)
- Llama 4 Scout 17B-16E vs Llama 3.3 70B (benchmark)
- Qwen3 30B-A3B vs gpt-oss-20b (benchmark)
- Gemma 4 26B-A4B vs Mistral Small 3.1 24B (benchmark)
- Nemotron 3 120B-A12B vs gpt-oss-120b (benchmark)
- GLM-4.7 Flash vs Qwen3 30B-A3B (benchmark)
- Llama 3.1 8B vs Llama 3.2 3B (benchmark)
- Granite 4.0 H Micro vs Llama 3.2 3B (benchmark)
- Gemma 4 26B-A4B vs Llama 3.3 70B (benchmark)
- Mistral Small 3.1 24B vs Qwen3 30B-A3B (benchmark)
- Llama 4 Scout 17B-16E vs Gemma 4 26B-A4B (benchmark)
- Nemotron 3 120B-A12B vs Llama 3.3 70B (benchmark)
- gpt-oss-20b vs Llama 3.1 8B (benchmark)
- Prompt injection test cases you can run on a system prompt
12 illustrative prompt injection test cases - overrides, role-play, encoding, translation, pasted documents - plus benign controls and a canary-string method to check for system prompt leaks. - Testing summarization prompts: numbers, hedges, negations and omissions
A fluent summary can still be unfaithful. Test summarization prompts with 8 illustrative cases that target changed numbers, dropped hedges, lost negations, invented owners and instructions hidden in the source.
Comparisons
- Litmiq vs Promptfoo: Browser Prompt Tests or CLI Evals?
Litmiq vs Promptfoo, compared fairly: a free browser app for quick side-by-side prompt tests vs an open-source (MIT) CLI for CI evals and red teaming. When to use each, and how to use both. - Litmiq vs LangSmith: Quick Prompt Tests or a Full Eval Platform?
Litmiq vs LangSmith: a free, no-signup browser tool for side-by-side prompt tests vs LangChain's hosted platform for tracing, datasets and online evals. Features, pricing and when to use each. - Litmiq vs OpenAI Playground (and What Replaces OpenAI Evals)
OpenAI Playground vs Litmiq for testing prompts, plus what changes when OpenAI Evals and reusable prompts shut down on 30 November 2026, and how to move your test cases. - Best Prompt Testing Tools in 2026: Seven Options by Job
Seven prompt testing and LLM evaluation tools compared by the job they do best: Litmiq, OpenAI Playground, Promptfoo, DeepEval, LangSmith, Langfuse and Braintrust. Facts and free plans from each vendor.
Reference
- LLM Evaluation Glossary: Prompt Testing Terms, Defined
Plain definitions of 35 LLM evaluation and prompt testing terms: test case, expected behavior, baseline, LLM-as-a-judge, rubric, judge biases, prompt injection, online vs offline evals and more. - Prompt Testing Checklist: 24 Checks for Every Prompt Change
A 24-point prompt testing checklist in six steps: save a baseline, check cases and expected behavior, change one thing, re-run, read the drops, and record what shipped. Tool-neutral.
About Litmiq
- About Litmiq
Litmiq facts in one place: a free, no-signup browser tool for testing system prompts on OpenAI and Anthropic models with LLM-judge scores. What it does, what it doesn't, data handling, pricing and contact. - Privacy Policy
What Litmiq collects: optional product-update email via Web3Forms, and your OpenAI key only in your browser localStorage. No ads. - Terms of Service
Terms for using Litmiq, the free browser prompt evaluator. Bring your own OpenAI key; you are responsible for API usage. - Site map
Every public page on litmiq.com in one list: the app, free test-case templates, prompt testing guides and playbooks, tool comparisons, the glossary, the checklist and company pages.
Get model deprecation alerts
We email you when OpenAI, Anthropic or Google publish a new shutdown date for a model on our tracker. Sent by hand from hello@litmiq.com, a few times a month at most. Unsubscribe by replying.
We store only your email, through our form provider. See the privacy policy.