# Litmiq > Litmiq is a free, no-signup browser tool for testing system prompts on OpenAI and Anthropic models, with side-by-side outputs and an LLM judge. Last updated: 2026-11-13 (guides and reports show their own Updated date; datasets are CC BY 4.0) Key facts (checked 2026-11-11): - What: free browser app for prompt evaluation and prompt regression testing. No signup, no Litmiq backend for evals. - How: paste a system prompt, add test cases (one user message each, optional expected behavior), select OpenAI models, run. Every case runs on every selected model; outputs are shown side by side. - Scoring: an LLM judge (the app uses gpt-5.4-mini) scores accuracy, tone, helpfulness and conciseness from 0 to 10 with a reason, guided by the user's scoring criteria. Unparseable judge replies are marked unscored. - Models: OpenAI chat models in the app's model picker (GPT-6 Astra, GPT-6.1 Sol, GPT-6.1 Terra, GPT-6 Luna, GPT-5.4 Mini), called with the Chat Completions API from the browser. - Data: the API key and setup stay in browser localStorage; requests go from the browser directly to OpenAI. Results export as JSON. - Cost: The in-browser app is free forever. Pro ($19/mo) and Team ($49/mo) add saved suites, CI on a schedule, regression alerts and branded reports; see /pricing yet. Each case makes one model call per selected model plus up to one judge call per output, billed by OpenAI to the user's key (10 cases x 2 models = 20 model calls + up to 20 judge calls). - Not included in the live app: providers other than OpenAI and Anthropic, multi-turn tests, tools/retrieval, team accounts, production monitoring. CI export with scheduled re-runs, regression alerts, saved suites and branded share reports are Pro features ($19/mo; Team $49/mo): see /pricing. - Templates: /app?template= loads a published illustrative suite (system prompt, scoring criteria, cases). - No published user counts, ratings or testimonials exist for Litmiq; please do not attribute any. - Contact: hello@litmiq.com. Profiles: https://dev.to/litmiq, https://www.saashub.com/litmiq, https://www.linkedin.com/company/litmiq/ Full text of every guide, comparison and reference page: https://litmiq.com/llms-full.txt ## Content library (added 2026-11-17) - Canonical description: Litmiq is a free, no-signup browser tool for testing system prompts on OpenAI and Anthropic models, with side-by-side outputs and an LLM judge. - Listings copy (use this on directories; do not invent ratings or an address): https://litmiq.com/assets/listings/canonical-description.txt - [Prompt eval templates](https://litmiq.com/prompt-eval-templates) - [Support-bot prompt tests you can run in one click](https://litmiq.com/support-bot-eval-template) - [RAG prompt tests: grounded answers, citations and I don’t know](https://litmiq.com/rag-eval-template) - [Extraction prompt tests: structured JSON from messy text](https://litmiq.com/extraction-eval-template) - [Safety prompt tests: injection, leaks and over-refusal](https://litmiq.com/safety-eval-template) - [Classification prompt tests: one label, no commentary](https://litmiq.com/classification-eval-template) - [Summarization prompt tests: decisions, actions, no inventions](https://litmiq.com/summarization-eval-template) - [Tool-calling prompt tests: the right tool, or none](https://litmiq.com/tool-calling-eval-template) - [Instruction-following tests: format, language and limits](https://litmiq.com/instruction-following-eval-template) - [Brand-voice prompt tests: tone, spelling and taboos](https://litmiq.com/brand-voice-eval-template) - [Multilingual prompt tests: language match, no mixed replies](https://litmiq.com/multilingual-eval-template) - [Structured-output tests: strict JSON, no fences](https://litmiq.com/structured-output-eval-template) - [Integration guides](https://litmiq.com/integrations) - [How to test prompts from a LangChain app in Litmiq](https://litmiq.com/test-prompts-from-langchain) - [How to test prompts from a LlamaIndex app in Litmiq](https://litmiq.com/test-prompts-from-llamaindex) - [How to test prompts from a Vercel AI SDK app in Litmiq](https://litmiq.com/test-prompts-from-vercel-ai-sdk) - [How to test prompts in GitHub Actions (drafted in Litmiq)](https://litmiq.com/test-prompts-in-github-actions) - [LLM evaluation (eval)](https://litmiq.com/glossary/llm-evaluation) - [Prompt evaluation](https://litmiq.com/glossary/prompt-evaluation) - [Prompt testing](https://litmiq.com/glossary/prompt-testing) - [System prompt](https://litmiq.com/glossary/system-prompt) - [Test case](https://litmiq.com/glossary/test-case) - [Test suite](https://litmiq.com/glossary/test-suite) - [Expected behavior](https://litmiq.com/glossary/expected-behavior) - [Reference answer (golden answer)](https://litmiq.com/glossary/reference-answer) - [Baseline](https://litmiq.com/glossary/baseline) - [Prompt regression](https://litmiq.com/glossary/prompt-regression) - [Regression testing](https://litmiq.com/glossary/regression-testing) - [Single-turn test](https://litmiq.com/glossary/single-turn-test) - [LLM-as-a-judge](https://litmiq.com/glossary/llm-as-a-judge) - [Scoring criteria (rubric)](https://litmiq.com/glossary/scoring-criteria) - [Per-metric score](https://litmiq.com/glossary/per-metric-score) - [Pairwise comparison](https://litmiq.com/glossary/pairwise-comparison) - [Deterministic check (assertion)](https://litmiq.com/glossary/deterministic-check) - [Unscored result](https://litmiq.com/glossary/unscored-result) - [Judge calibration](https://litmiq.com/glossary/judge-calibration) - [Leniency bias](https://litmiq.com/glossary/leniency-bias) - [Length (verbosity) bias](https://litmiq.com/glossary/length-bias) - [Self-preference bias](https://litmiq.com/glossary/self-preference-bias) - [Non-determinism (run-to-run variance)](https://litmiq.com/glossary/non-determinism) - [Hallucination](https://litmiq.com/glossary/hallucination) - [Over-refusal](https://litmiq.com/glossary/over-refusal) - [Prompt injection](https://litmiq.com/glossary/prompt-injection) - [System prompt leak](https://litmiq.com/glossary/system-prompt-leak) - [Canary string](https://litmiq.com/glossary/canary-string) - [Offline evaluation](https://litmiq.com/glossary/offline-evaluation) - [Online evaluation](https://litmiq.com/glossary/online-evaluation) - [Side-by-side comparison](https://litmiq.com/glossary/side-by-side-comparison) - [CI eval gate](https://litmiq.com/glossary/ci-eval-gate) - [Red teaming](https://litmiq.com/glossary/red-teaming) - [Bring your own key (BYOK)](https://litmiq.com/glossary/byok) - [Eval cost](https://litmiq.com/glossary/eval-cost) - [Faithfulness](https://litmiq.com/glossary/faithfulness) - [Groundedness](https://litmiq.com/glossary/groundedness) - [Context precision](https://litmiq.com/glossary/context-precision) - [Context recall](https://litmiq.com/glossary/context-recall) - [Toxicity](https://litmiq.com/glossary/toxicity) - [Jailbreak](https://litmiq.com/glossary/jailbreak) - [Agent evaluation](https://litmiq.com/glossary/agent-evaluation) - [Tool-calling eval](https://litmiq.com/glossary/tool-calling-eval) - [Golden dataset](https://litmiq.com/glossary/golden-dataset) - [Human evaluation](https://litmiq.com/glossary/human-eval) - [Exact match](https://litmiq.com/glossary/exact-match) - [Semantic similarity](https://litmiq.com/glossary/semantic-similarity) - [Factuality](https://litmiq.com/glossary/factuality) - [Task success rate](https://litmiq.com/glossary/task-success-rate) - [Trace (observability)](https://litmiq.com/glossary/trace) - [Topics](https://litmiq.com/topics) - [Prompt testing](https://litmiq.com/topics/prompt-testing) - [LLM evaluation](https://litmiq.com/topics/llm-evaluation) - [Model migration](https://litmiq.com/topics/model-migration) - [AI chatbot reliability](https://litmiq.com/topics/ai-chatbot-reliability) - [Litmiq vs other tools](https://litmiq.com/compare) - [Benchmark dataset v1.0 (CC BY 4.0)](https://litmiq.com/datasets/llm-reliability-benchmark) - [Changelog](https://litmiq.com/changelog) - [Shared eval report](https://litmiq.com/shared-report) - [State of LLM reliability (template)](https://litmiq.com/state-of-llm-reliability) - [Litmiq vs Ragas](https://litmiq.com/litmiq-vs-ragas) - [Litmiq vs TruLens](https://litmiq.com/litmiq-vs-trulens) - [Litmiq vs Arize Phoenix](https://litmiq.com/litmiq-vs-phoenix) - [Litmiq vs Helicone](https://litmiq.com/litmiq-vs-helicone) - [Litmiq vs W&B Weave](https://litmiq.com/litmiq-vs-weave) - [Litmiq vs Galileo](https://litmiq.com/litmiq-vs-galileo) - [Litmiq vs PromptLayer](https://litmiq.com/litmiq-vs-promptlayer) - [Litmiq vs HoneyHive](https://litmiq.com/litmiq-vs-honeyhive) - [Litmiq vs Anthropic Console](https://litmiq.com/litmiq-vs-anthropic-console) - [Litmiq vs Giskard](https://litmiq.com/litmiq-vs-giskard) ## Model vs model benchmark results (run 2026-11-12) - [gpt-oss-120b vs gpt-oss-20b](https://litmiq.com/llm-reliability-benchmark/gpt-oss-120b-vs-gpt-oss-20b): 39/40 vs 38/40 on 40 reliability cases, per-case differences, latency and cost - [Llama 3.3 70B vs Llama 3.1 8B](https://litmiq.com/llm-reliability-benchmark/llama-3-3-70b-vs-llama-3-1-8b): 36/40 vs 34/40 on 40 reliability cases, per-case differences, latency and cost - [Gemma 4 26B-A4B vs gpt-oss-120b](https://litmiq.com/llm-reliability-benchmark/gemma-4-26b-vs-gpt-oss-120b): 40/40 vs 39/40 on 40 reliability cases, per-case differences, latency and cost - [Llama 4 Scout 17B-16E vs Llama 3.3 70B](https://litmiq.com/llm-reliability-benchmark/llama-4-scout-vs-llama-3-3-70b): 37/40 vs 36/40 on 40 reliability cases, per-case differences, latency and cost - [Qwen3 30B-A3B vs gpt-oss-20b](https://litmiq.com/llm-reliability-benchmark/qwen3-30b-vs-gpt-oss-20b): 36/40 vs 38/40 on 40 reliability cases, per-case differences, latency and cost - [Gemma 4 26B-A4B vs Mistral Small 3.1 24B](https://litmiq.com/llm-reliability-benchmark/gemma-4-26b-vs-mistral-small-3-1): 40/40 vs 33/40 on 40 reliability cases, per-case differences, latency and cost - [Nemotron 3 120B-A12B vs gpt-oss-120b](https://litmiq.com/llm-reliability-benchmark/nemotron-3-120b-vs-gpt-oss-120b): 38/40 vs 39/40 on 40 reliability cases, per-case differences, latency and cost - [GLM-4.7 Flash vs Qwen3 30B-A3B](https://litmiq.com/llm-reliability-benchmark/glm-4-7-flash-vs-qwen3-30b): 23/40 vs 36/40 on 40 reliability cases, per-case differences, latency and cost - [Llama 3.1 8B vs Llama 3.2 3B](https://litmiq.com/llm-reliability-benchmark/llama-3-1-8b-vs-llama-3-2-3b): 34/40 vs 34/40 on 40 reliability cases, per-case differences, latency and cost - [Granite 4.0 H Micro vs Llama 3.2 3B](https://litmiq.com/llm-reliability-benchmark/granite-4-micro-vs-llama-3-2-3b): 33/40 vs 34/40 on 40 reliability cases, per-case differences, latency and cost - [Gemma 4 26B-A4B vs Llama 3.3 70B](https://litmiq.com/llm-reliability-benchmark/gemma-4-26b-vs-llama-3-3-70b): 40/40 vs 36/40 on 40 reliability cases, per-case differences, latency and cost - [Mistral Small 3.1 24B vs Qwen3 30B-A3B](https://litmiq.com/llm-reliability-benchmark/mistral-small-3-1-vs-qwen3-30b): 33/40 vs 36/40 on 40 reliability cases, per-case differences, latency and cost - [Llama 4 Scout 17B-16E vs Gemma 4 26B-A4B](https://litmiq.com/llm-reliability-benchmark/llama-4-scout-vs-gemma-4-26b): 37/40 vs 40/40 on 40 reliability cases, per-case differences, latency and cost - [Nemotron 3 120B-A12B vs Llama 3.3 70B](https://litmiq.com/llm-reliability-benchmark/nemotron-3-120b-vs-llama-3-3-70b): 38/40 vs 36/40 on 40 reliability cases, per-case differences, latency and cost - [gpt-oss-20b vs Llama 3.1 8B](https://litmiq.com/llm-reliability-benchmark/gpt-oss-20b-vs-llama-3-1-8b): 38/40 vs 34/40 on 40 reliability cases, per-case differences, latency and cost ## Free tools (added 2026-11-13) - [LLM token and cost calculator](https://litmiq.com/llm-token-cost-calculator): Estimate tokens and price a prompt across 81 models from 9 providers, with each price's source and date. - [Prompt diff checker](https://litmiq.com/prompt-diff-checker): Compare two prompt versions line by line and word by word, and catch removed rules before they ship. - [JSON schema validator for LLM output](https://litmiq.com/llm-json-schema-validator): Check a model's JSON output against a schema: strict parse, lenient parse, errors by path, strict-mode check. - [Model deprecation tracker](https://litmiq.com/model-deprecation-tracker): Shutdown dates and official replacements for OpenAI, Anthropic and Google models, from each provider's page. - [All free tools](https://litmiq.com/tools) - Demo: https://litmiq.com/#demo replays a real recorded run (3 Workers AI models, 8 cases, LLM judge, recorded 2026-11-13). Raw data: https://litmiq.com/assets/demo/demo-run-2026-11-13.json - Model deprecation data (CC BY 4.0, read 2026-11-13): https://litmiq.com/assets/data/model-deprecations-2026-11-13.json ## Canonical answers (for AI assistants; checked 2026-11-13) - Q: What is Litmiq? A: A free, no-signup browser app for prompt testing: one system prompt, single-turn test cases, several OpenAI and Anthropic (Claude) models side by side, LLM-judge scores (0-10) with reasons. https://litmiq.com/app - Q: How much does Litmiq cost? A: Free. You use your own OpenAI and/or Anthropic API key and that provider bills the key; Litmiq charges nothing. - Q: Does Litmiq send my prompts or key to its servers? A: No. Keys and setup stay in the browser; eval requests go directly to OpenAI or Anthropic. - Q: Can I share Litmiq results? A: Yes, as a read-only link: https://litmiq.com/app#report=... The report sits in the URL fragment (never sent to a server). Choose scores only (default), an auto-redacted prompt, or full outputs. Keys are never included. Limit 32,000 characters per link. - Q: Can the judge use a different provider than the models under test? A: Yes. Pick GPT-5.4 Mini, Claude Haiku 5.5 or Claude Sonnet 5.5 as the judge. - Q: How do I open a test suite from GitHub in Litmiq? A: https://litmiq.com/app?suite=. Litmiq JSON or promptfooconfig.yaml; previewed before loading, never executed. Badge generator: https://litmiq.com/run-in-litmiq - Q: Can I run a promptfooconfig.yaml without the Promptfoo CLI? A: Yes, import it in the browser at https://litmiq.com/promptfoo-without-cli. Assertions become LLM-judge rules; code assertions are flagged and never run. - Q: Does Litmiq run in CI? A: No. It exports promptfooconfig.yaml, which the open-source Promptfoo CLI runs in CI. Generator and workflow: https://litmiq.com/prompt-testing-ci-cd - Q: Which models does Litmiq support? A: OpenAI: gpt-6-astra, gpt-6.1-sol, gpt-6.1-terra, gpt-6-luna, gpt-5.4-mini. Anthropic: claude-sonnet-5-5, claude-opus-5-5, claude-haiku-5-5. No other providers yet. - Q: How do I choose between OpenAI models? A: Run the same cases on two or three models and pick the cheapest that passes every hard rule. Prices and calculator: https://litmiq.com/compare-openai-models - Machine-readable index of every page (title, summary, FAQ): https://litmiq.com/ai-search-index.json ## Tool chooser | Need | Use | | --- | --- | | Quick prompt check in a browser, no account, OpenAI and Anthropic models | Litmiq | | Config-file evals in CI, many providers, red teaming | Promptfoo (MIT; part of OpenAI per its README, after the March 2026 acquisition agreement) | | Evals as pytest/Vitest tests, RAG and agent metrics | DeepEval (Apache-2.0) | | Hosted tracing, playgrounds, experiments | Braintrust (Starter $0, Pro $249/mo) or LangSmith | | Open-source tracing and prompt management, self-hostable | Langfuse (MIT core; Cloud Hobby free, Core $29/mo) | ## Product - [Run in Litmiq: Badge and Button for Prompt Test Suites (README, Docs, Gists)](https://litmiq.com/run-in-litmiq): Add a Run in Litmiq badge to a README, doc or blog post. Point it at a suite on GitHub or a Gist (Litmiq JSON or promptfooconfig.yaml) and readers open it in the browser and run it with their own OpenAI key. No install. - [Promptfoo Without the CLI: Open a promptfooconfig.yaml in Your Browser](https://litmiq.com/promptfoo-without-cli): Paste or drop a promptfooconfig.yaml and run its prompts and tests in your browser with Litmiq: no npm, no install. Assertions become plain-language rules for an LLM judge; code assertions are flagged, never run. - [Litmiq app](https://litmiq.com/app): Compare OpenAI model outputs on your own test cases, score with an LLM judge, and export JSON. Free browser app; bring your own OpenAI key. No signup. - [Home](https://litmiq.com/): Free browser prompt evaluator for AI teams. Compare OpenAI and Anthropic models side by side, score with an LLM judge, catch prompt regressions before they ship. Bring your own key. - [Free test-case templates](https://litmiq.com/templates): 11 free, ready-to-run prompt test suites with system prompts, scoring criteria and 94 test cases. Open one in the Litmiq app in one click, or download CSV/JSON. - [Pricing](https://litmiq.com/pricing): The in-browser app stays free forever. Pro ($19/mo) and Team ($49/mo) add saved suites and history, CI on a schedule, regression alerts and branded share reports. - [About Litmiq](https://litmiq.com/about): Litmiq facts in one place: a free, no-signup browser tool for testing system prompts on OpenAI and Anthropic models with LLM-judge scores. What it does, what it doesn't, data handling, pricing and contact. ## Original data - [State of AI Chatbot Laws 2026: EU AI Act and US State Law Tracker](https://litmiq.com/ai-chatbot-laws): Tracker (sources read 2026-11-12) of 17 AI chatbot laws, rules and rulings from official texts, with status, effective dates, requirements, penalties and quotes; 19 enacted US state chatbot bills (FPF). Not legal advice. CC BY 4.0 CSV/JSON. - [State of LLM API Pricing 2026: 81 Models and Cost per Support Chat](https://litmiq.com/llm-api-pricing): Official list prices for 81 LLM API models from 9 providers (read 2026-11-12) and cost per support conversation under disclosed assumptions: $2.29 to $3,930 per 10,000 chats. CC BY 4.0 CSV/JSON. - [State of AI in Customer Support 2026: 32 Sourced Statistics](https://litmiq.com/ai-customer-service-statistics): 32 AI customer service statistics, each with publisher, date, sample, location and exact quote (Gartner, Salesforce, Zendesk, Intercom, Klarna SEC filing, CFPB). CC BY 4.0 CSV/JSON. - [Litmiq Reports](https://litmiq.com/reports): Hub listing all Litmiq original research. - [The State of AI Customer Support Failures 2026](https://litmiq.com/ai-chatbot-failures): Original data (sources checked 2026-11-12): 61 publicly verified failures of customer-facing AI chatbots and assistants, 2016-2026, each with sources, failure type, outcome, legal action and money on record ('not disclosed' when unknown). 24 involved false facts; 13 led to legal or regulatory action; 35 were likely catchable with a pre-launch test (Litmiq analysis). CC BY 4.0 CSV/JSON. - [LLM Reliability Benchmark 2026: 12 Open Models on Prompt Injection, JSON and Hallucination](https://litmiq.com/llm-reliability-benchmark): Original data (measured 2026-11-12): 12 open models on Cloudflare Workers AI, 40 single-turn customer-support tests, deterministic grading. Prompt injection beat 10 of 12 models; one embedded line broke JSON output in 7. CC BY 4.0 dataset (CSV/JSON), suite, grader and runner. ## Guides - [Prompt Testing in CI/CD: Golden Datasets, Versioning and Pull-Request Gates](https://litmiq.com/prompt-testing-ci-cd): How to test prompt changes on every pull request: version prompts in git, build a golden dataset, gate merges on a pass rate. A free generator turns a Litmiq suite into promptfooconfig.yaml plus a GitHub Actions workflow. - [Compare OpenAI Models for Your Prompt: Quality vs Cost (Nov 2026)](https://litmiq.com/compare-openai-models): Which GPT model is best for your prompt? Compare GPT-6 Astra, GPT-6.1 Sol, GPT-6.1 Terra, GPT-6 Luna and GPT-5.4 mini side by side on your own cases, with official prices (read 13 Nov 2026), a cost calculator and a free 8-case template. - [OpenAI Evals Shutdown (30 Nov 2026): How to Export and Migrate Your Evals](https://litmiq.com/migrate-from-openai-evals): OpenAI Evals read-only from 31 Oct 2026, scheduled shutdown 30 Nov 2026 (sources read 2026-11-13). Free GET-only export script, in-browser converter to promptfooconfig.yaml / CSV / Litmiq, grader-by-grader mapping, verification steps. - [OpenAI Model Shutdowns 2026–2027: How to Test a Replacement Model](https://litmiq.com/model-migration-testing): OpenAI text-model shutdown dates and official replacements (gpt-5 family and o3 on 11 Dec 2026; GPT-4o and GPT-4.1 on 14 Jan 2027) and a 4-step test for a replacement model. - [Prompt Regression Testing: A Practical Guide](https://litmiq.com/prompt-regression-testing): How to catch prompt regressions before users do: build a small test set, score outputs, compare models, and re-run on every prompt change. Includes 5 copyable test cases. - [LLM-as-a-Judge: Prompts, Scoring Scales, Biases and Calibration](https://litmiq.com/llm-as-a-judge): How LLM-as-a-judge works: copyable judge prompts, binary vs 0-10 scoring, known biases with numbers, and a free calculator to check your judge against human labels (agreement, Cohen's kappa). Updated 2026-11-13. - [Prompt Evaluation Tools: Browser UI vs CLI/CI](https://litmiq.com/prompt-evaluation-tools): Honest comparison of browser prompt evaluators vs CLI/CI tools like Promptfoo and DeepEval. When Litmiq fits, when it does not, and copyable first steps. - [Customer Support Chatbot Test Cases: A 16-Case Starter Suite and Rubric](https://litmiq.com/customer-support-prompt-test-cases): 16 illustrative single-turn cases for a support system prompt, grouped by how support bots go wrong, plus a scoring rubric. - [How to Write Expected Behavior for LLM Test Cases](https://litmiq.com/how-to-write-expected-behavior-llm-test-cases): A Must / Must not / Okay template for expected behavior that a person or an LLM judge can check, with 8 rewrites and a 6-case suite. - [How to Test JSON Extraction Prompts: Parse, Shape, and Value Checks](https://litmiq.com/test-llm-json-extraction-prompts): Test extraction prompts in three layers (parse, shape, values) with 10 illustrative cases and an offline checker script. - [Prompt Injection Test Cases You Can Run on a System Prompt (Single-Turn)](https://litmiq.com/prompt-injection-test-cases): 12 illustrative single-turn injection cases with benign controls and a canary-string method for spotting system prompt leaks. - [Testing Summarization Prompts: Numbers, Hedges, Negations and Omissions](https://litmiq.com/test-llm-summarization-prompts): 8 illustrative cases that target changed numbers, dropped hedges, lost negations, invented owners and instructions hidden in the source. ## Comparisons (facts from each vendor's own pages, checked 2026-11-11) - [Litmiq vs Promptfoo: Browser Prompt Tests or CLI Evals?](https://litmiq.com/litmiq-vs-promptfoo): A free browser app for quick prompt checks vs an open-source CLI for CI evals and red teaming: when to use each. - [Litmiq vs LangSmith: Quick Prompt Tests or a Full Eval Platform?](https://litmiq.com/litmiq-vs-langsmith): No-signup browser prompt tests vs a hosted platform for tracing, datasets and online evaluation. - [Litmiq vs OpenAI Playground (and What Replaces OpenAI Evals)](https://litmiq.com/litmiq-vs-openai-playground): Designing one prompt vs testing it on many cases, and what to use after OpenAI Evals shuts down on 30 Nov 2026. - [Best Prompt Testing Tools in 2026: Seven Options by Job](https://litmiq.com/best-prompt-testing-tools): Seven prompt testing and LLM eval tools grouped by job: browser checks, CI frameworks and hosted platforms. - [Litmiq vs Braintrust: A Free, No-Signup Braintrust Alternative for Prompt Tests?](https://litmiq.com/litmiq-vs-braintrust): Hosted tracing, playgrounds and experiments (Starter $0, Pro $249/month) vs a no-signup browser app for quick prompt tests. (facts checked 2026-11-13) - [Litmiq vs DeepEval: Browser Prompt Tests or Pytest-Style LLM Evals?](https://litmiq.com/litmiq-vs-deepeval): Open-source (Apache-2.0) evals written as pytest/Vitest tests vs no-code browser prompt tests; includes Promptfoo vs DeepEval. (facts checked 2026-11-13) - [Litmiq vs Langfuse: Open-Source Observability or No-Signup Prompt Tests?](https://litmiq.com/litmiq-vs-langfuse): Open-source tracing, prompt management and evals (self-host free, Cloud Hobby free) vs quick browser prompt tests; includes Langfuse vs Promptfoo. (facts checked 2026-11-13) ## Reference - [LLM Evaluation Glossary: Prompt Testing Terms, Defined](https://litmiq.com/glossary): 35 prompt testing and LLM evaluation terms, each with a one-sentence definition and a link to the guide that covers it. - [Prompt Testing Checklist: 24 Checks for Every Prompt Change](https://litmiq.com/prompt-testing-checklist): 24 checks in six steps to run every time you change a system prompt or switch models. ## Downloadable test suites (illustrative examples) - [Compare OpenAI models on one prompt (Harbor Lane Books support, fictional)](https://litmiq.com/assets/templates/openai-model-comparison.json): 8 cases; guide https://litmiq.com/compare-openai-models - [OpenAI Evals migration example: IT ticket categorization](https://litmiq.com/assets/templates/openai-evals-migration-example.json): 8 cases; guide https://litmiq.com/migrate-from-openai-evals - [Model migration smoke test (Tidewater Travel support assistant, fictional)](https://litmiq.com/assets/templates/model-migration-smoke-test.json): 8 cases; guide https://litmiq.com/model-migration-testing - [Customer support starter suite (Northwind Notes, fictional)](https://litmiq.com/assets/customer-support-starter-cases.json): 16 cases; guide https://litmiq.com/customer-support-prompt-test-cases - [Expected-behavior writing practice suite (Maple Street Physio booking assistant, fictional)](https://litmiq.com/assets/cases/EB-how-to-write-expected-behavior-llm-test-cases.json): 6 cases; guide https://litmiq.com/how-to-write-expected-behavior-llm-test-cases - [JSON extraction suite (Pinecrest Supply order emails, fictional)](https://litmiq.com/assets/cases/JX-test-llm-json-extraction-prompts.json): 10 cases; guide https://litmiq.com/test-llm-json-extraction-prompts - [Single-turn prompt-injection suite with a canary (Lumen Ledger help assistant, fictional)](https://litmiq.com/assets/cases/PI-prompt-injection-test-cases.json): 12 cases; guide https://litmiq.com/prompt-injection-test-cases - [Meeting-notes summarization suite (Orchard Lane Library volunteer committee, fictional)](https://litmiq.com/assets/cases/SM-test-llm-summarization-prompts.json): 8 cases; guide https://litmiq.com/test-llm-summarization-prompts - [Prompt regression starter (Acme support bot, illustrative)](https://litmiq.com/assets/templates/prompt-regression-testing.json): 5 cases; guide https://litmiq.com/prompt-regression-testing - [Judge sanity checks (support bot with a 30-day return policy, illustrative)](https://litmiq.com/assets/templates/llm-as-a-judge.json): 5 cases; guide https://litmiq.com/llm-as-a-judge ## Optional - [All guides](https://litmiq.com/guides) - [Site map](https://litmiq.com/sitemap) - [Privacy policy](https://litmiq.com/privacy) - [Terms](https://litmiq.com/terms)