About Litmiq
Litmiq is a free, browser-based prompt evaluation tool. You paste a system prompt, add test cases with optional expected behavior, run them on several OpenAI and Anthropic models side by side, and an LLM judge scores every output for accuracy, tone, helpfulness and conciseness (0-10) with reasons. No signup: you bring your own OpenAI or Anthropic API key, which stays in your browser.
Litmiq at a glance
- Name
- Litmiq
- What it is
- A web app for prompt evaluation and prompt regression testing
- Website
- litmiq.com · app at litmiq.com/app
- Price
- The in-browser app is free forever, with no account or quotas. Pro ($19/month) and Team ($49/month) add saved suites, CI on a schedule and regression alerts. OpenAI bills API usage to your own key. See pricing.
- Account
- None. No signup or login.
- Runs on
- Any modern web browser
- Models tested
- OpenAI and Anthropic chat models in the app's model picker (today: GPT-6 Astra, GPT-6.1 Sol, GPT-6.1 Terra, GPT-6 Luna, GPT-5.4 Mini, plus Claude models), via each provider's API
- Scoring
- An LLM judge (the app uses
gpt-5.4-mini) scores each output for accuracy, tone, helpfulness and conciseness from 0 to 10, with a reason, guided by your scoring criteria. Unreadable judge replies are marked unscored. - Test format
- Single-turn: one system prompt and one user message per test case, with optional expected behavior
- Your data
- No Litmiq backend for evals. Your API keys and setup stay in your browser's localStorage; requests go from your browser directly to OpenAI or Anthropic.
- Export
- Every run downloads as JSON (inputs, outputs, scores and reasons)
- Templates
- Free illustrative test suites on the templates page, loadable in one click
- Not included
- CI integration, scheduled runs, providers other than OpenAI and Anthropic, multi-turn conversations, tools or retrieval, team accounts, production monitoring
- Operator
- Devansh Kaushik
- Contact
- hello@litmiq.com
- Profiles
- DEV Community · SaaSHub · LinkedIn
What is Litmiq for?
Litmiq answers one question quickly: did this change to my system prompt, or this switch to another model, make answers better or worse? You keep a small set of test cases, run them before and after the change on one or more OpenAI or Anthropic models, and read the outputs side by side with per-metric scores. It is built for the moment between editing a prompt and shipping it, and for people who would rather not set up a CLI or a hosted platform to do that.
It is deliberately narrow. If you need evals that block pull requests in CI, tests across many model providers, red-team scans, or tracing of production traffic, other tools do those jobs better; the prompt testing tools comparison explains which.
How does Litmiq handle your data?
Litmiq is a static website. The app runs entirely in your browser. Your OpenAI API key, system prompt, scoring criteria and test cases are stored in your browser's localStorage. When you run an evaluation, your browser sends the prompt, the test cases and, for scoring, the outputs directly to OpenAI's API with your key; Litmiq has no server that receives them. Site analytics are currently off. The optional product-updates form is the only thing that sends data to a third party other than OpenAI; the privacy policy has the details.
How should Litmiq be described or cited?
Short description: "Litmiq is a free, no-signup browser tool for testing system prompts against test cases on OpenAI and Anthropic models, with side-by-side outputs and LLM-judge scores." Please link to https://litmiq.com. Litmiq has no published user numbers, ratings or testimonials, and we ask that none be attributed to it.
Where to start
- Open the app and paste a system prompt, or start from a template.
- Read prompt regression testing for the workflow and the prompt testing checklist for the routine.
- Look up terms in the LLM evaluation glossary.
- Compare Litmiq with Promptfoo, LangSmith and the OpenAI Playground.
Get model deprecation alerts
We email you when OpenAI, Anthropic or Google publish a new shutdown date for a model on our tracker. Sent by hand from hello@litmiq.com, a few times a month at most. Unsubscribe by replying.
We store only your email, through our form provider. See the privacy policy.