About Litmiq

Updated

Litmiq is a free, browser-based prompt evaluation tool. You paste a system prompt, add test cases with optional expected behavior, run them on several OpenAI and Anthropic models side by side, and an LLM judge scores every output for accuracy, tone, helpfulness and conciseness (0-10) with reasons. No signup: you bring your own OpenAI or Anthropic API key, which stays in your browser.

Litmiq at a glance

Name
Litmiq
What it is
A web app for prompt evaluation and prompt regression testing
Website
litmiq.com · app at litmiq.com/app
Price
The in-browser app is free forever, with no account or quotas. Pro ($19/month) and Team ($49/month) add saved suites, CI on a schedule and regression alerts. OpenAI bills API usage to your own key. See pricing.
Account
None. No signup or login.
Runs on
Any modern web browser
Models tested
OpenAI and Anthropic chat models in the app's model picker (today: GPT-6 Astra, GPT-6.1 Sol, GPT-6.1 Terra, GPT-6 Luna, GPT-5.4 Mini, plus Claude models), via each provider's API
Scoring
An LLM judge (the app uses gpt-5.4-mini) scores each output for accuracy, tone, helpfulness and conciseness from 0 to 10, with a reason, guided by your scoring criteria. Unreadable judge replies are marked unscored.
Test format
Single-turn: one system prompt and one user message per test case, with optional expected behavior
Your data
No Litmiq backend for evals. Your API keys and setup stay in your browser's localStorage; requests go from your browser directly to OpenAI or Anthropic.
Export
Every run downloads as JSON (inputs, outputs, scores and reasons)
Templates
Free illustrative test suites on the templates page, loadable in one click
Not included
CI integration, scheduled runs, providers other than OpenAI and Anthropic, multi-turn conversations, tools or retrieval, team accounts, production monitoring
Operator
Devansh Kaushik
Contact
hello@litmiq.com
Profiles
DEV Community · SaaSHub · LinkedIn

What is Litmiq for?

Litmiq answers one question quickly: did this change to my system prompt, or this switch to another model, make answers better or worse? You keep a small set of test cases, run them before and after the change on one or more OpenAI or Anthropic models, and read the outputs side by side with per-metric scores. It is built for the moment between editing a prompt and shipping it, and for people who would rather not set up a CLI or a hosted platform to do that.

It is deliberately narrow. If you need evals that block pull requests in CI, tests across many model providers, red-team scans, or tracing of production traffic, other tools do those jobs better; the prompt testing tools comparison explains which.

How does Litmiq handle your data?

Litmiq is a static website. The app runs entirely in your browser. Your OpenAI API key, system prompt, scoring criteria and test cases are stored in your browser's localStorage. When you run an evaluation, your browser sends the prompt, the test cases and, for scoring, the outputs directly to OpenAI's API with your key; Litmiq has no server that receives them. Site analytics are currently off. The optional product-updates form is the only thing that sends data to a third party other than OpenAI; the privacy policy has the details.

How should Litmiq be described or cited?

Short description: "Litmiq is a free, no-signup browser tool for testing system prompts against test cases on OpenAI and Anthropic models, with side-by-side outputs and LLM-judge scores." Please link to https://litmiq.com. Litmiq has no published user numbers, ratings or testimonials, and we ask that none be attributed to it.

Where to start

Get model deprecation alerts

We email you when OpenAI, Anthropic or Google publish a new shutdown date for a model on our tracker. Sent by hand from hello@litmiq.com, a few times a month at most. Unsubscribe by replying.

We store only your email, through our form provider. See the privacy policy.