Guide · prices read 13 Nov 2026

Compare OpenAI models on your own prompt: quality vs cost

On this page

The short answer

  • The best GPT model for your prompt is the cheapest one that passes your own test cases. Leaderboards can't tell you that.
  • List prices differ by up to 100×: GPT-6 Astra is $10 / $50 per 1M input/output tokens; GPT-6 Luna is $0.10 / $0.50 (OpenAI, read 13 Nov 2026).
  • Run the same cases on two or three models side by side in Litmiq, read every output that differs, then price the winner. The 8-case template is a start.
  • Check hard rules first (policy, format, injection), then quality. One policy failure can outweigh a point of average score.

Key numbers

  • OpenAI shuts down gpt-5, gpt-5-mini, gpt-5-nano, gpt-5-pro, o3 and o3-pro on 11 Dec 2026 (replacements: gpt-6.1-sol, gpt-6.1-terra, gpt-6-luna). (OpenAI deprecations page, 13 Nov 2026)
  • List prices across 81 models from 9 providers run from $0.017/$0.112 (granite-4.0-h-micro) to $30/$180 (gpt-5.5-pro) per 1M input/output tokens. (State of LLM API Pricing 2026, 12 Nov 2026)

OpenAI prices for the models in Litmiq

OpenAI list prices per 1M tokens, standard tier (OpenAI pricing page, read 13 Nov 2026), and Litmiq's arithmetic per 1,000 replies
Model (in Litmiq)InputOutputCached inputPer 1,000 replies*
gpt-6-astra$10.00$50.00$1.00$30.00
gpt-6.1-sol$2.00$10.00$0.10$6.00
gpt-6.1-terra$1.00$5.00$0.05$3.00
gpt-6-luna$0.10$0.50$0.01$0.30
gpt-5.4-mini$0.75$4.50$0.075$2.475

*Our arithmetic, not OpenAI's: 1,500 input tokens (system prompt plus user message) and 300 output tokens per reply, standard tier, no caching. Prices from OpenAI's pricing page, read 13 Nov 2026; they change, so check before you budget. More providers: LLM API pricing 2026.

Cost calculator

Estimated cost at list prices (standard tier, no caching)
ModelPer 1,000 repliesPer month

How to compare models on your prompt

  1. Write 20–50 cases from real requests plus your edge cases: policy questions, format requests, another language, an injection attempt. Or start from the model comparison template.
  2. Write the expected behavior for each: what the answer must and must not do.
  3. Select two or three models in Litmiq (typically your current model, one tier down and one tier up) and run. Every case runs on every model, and the outputs line up side by side.
  4. Read the disagreements. Find the cases where the models' scores differ and read both outputs. The judge gives a reason for every score.
  5. Price it. Use the calculator with your real token counts. Shorter outputs from one model change the bill too.

Can a cheaper model do the same job?

It's worth testing, because the price gap is large: in the table above, GPT-6 Luna costs 1/20 of GPT-6.1 Sol per reply. Whether it's good enough depends on your prompt, so decide from your own results:

Cost vs quality: how to decide after the run
Result on your casesDecision
Cheaper model passes every hard rule; scores within about a pointSwitch, and keep the suite to re-check after model updates
Cheaper model fails one hard rule (policy, format, injection)Try a clearer prompt for that rule and re-run; if it still fails, stay
Cheaper model is worse on several quality cases but passes rulesDepends on volume: price the difference with the calculator
Larger model fixes failures the smaller one can'tUse the larger model, or route only hard requests to it

Switching because a model is being retired? See model migration testing and OpenAI's deprecations page.

What to look for between models

We don't publish quality rankings for these models, because we haven't measured them on a common suite. For measured results on open models, see our LLM reliability benchmark.

Limits

FAQ

Which OpenAI model is best for my use case?

The one that passes your own test cases at the lowest cost. Benchmarks rank models on other people's tasks; your prompt, rules and users are different. Run the same 20-50 cases on two or three models side by side, read the outputs that differ, and pick the cheapest model that passes every hard rule.

Is a mini model as good as the full model?

Sometimes, for a given prompt. Smaller models cost much less per token (GPT-6 Luna lists $0.10 per 1M input tokens vs $2.00 for GPT-6.1 Sol on OpenAI's pricing page, read 13 Nov 2026), and on narrow, well-specified tasks they may pass the same tests. The only way to know for your prompt is to run both on the same cases.

How much does it cost to compare models in Litmiq?

Litmiq is free; OpenAI bills your key. Each case makes one call per selected model plus up to one judge call per output, so 20 cases on 3 models is 60 model calls and up to 60 judge calls.

What are OpenAI's prices per million tokens?

From OpenAI's pricing page (standard tier, read 13 Nov 2026): GPT-6 Astra $10 input / $50 output, GPT-6.1 Sol $2 / $10, GPT-6.1 Terra $1 / $5, GPT-6 Luna $0.10 / $0.50, GPT-5.4 mini $0.75 / $4.50 per 1M tokens. Check the page before you budget.

Can I compare OpenAI models with other providers here?

Partly: Litmiq also runs Anthropic Claude models (Sonnet 5.5, Opus 5.5, Haiku 5.5) side by side with GPT models, with your own Anthropic key. For other providers' list prices see the LLM API pricing report; for cross-provider tests use a CLI such as Promptfoo.

Sources (read 13 Nov 2026)

Get model deprecation alerts

We email you when OpenAI, Anthropic or Google publish a new shutdown date for a model on our tracker. Sent by hand from hello@litmiq.com, a few times a month at most. Unsubscribe by replying.

We store only your email, through our form provider. See the privacy policy.

Compare models on your own cases

Open the template, select two or three models and run. Free; OpenAI bills your key.

  • Free, no signup
  • Runs in your browser
  • Your OpenAI key, sent only to OpenAI

Follow Litmiq on LinkedIn