Quarterly report · first edition pending

State of prompt quality

On this page

The short answer

  • This will be a quarterly report on prompt quality, built only from opted-in runs.
  • First edition pending opt-in data. There are no statistics on this page yet, and we will not invent any.
  • The method is public below so the rules exist before the first number does.

Status

Status as of 19 Nov 2026

First edition pending opt-in data. No charts, averages, or sample sizes are published here because no opted-in dataset has been collected for this report. Unknowns are not zeros.

Opt-in data notice

Litmiq does not scrape private runs for this report. A run appears in a future edition only if the person who made it opts in, confirms that the material contains no secrets or personal data, and agrees to the method on this page. Shared-report links, roast-board opt-ins, and explicit email to hello@litmiq.com are the intended paths. We will not treat ordinary use of the app as consent.

To offer a run, email hello@litmiq.com with the subject “State of prompt quality opt-in”. You can also use the gallery path on Reports. None of those paths publishes a statistic by itself.

Methodology

  • Each edition will be dated, and will state the window of opted-in runs it covers, in America/Los_Angeles.
  • It will state the sample size, and whether that sample is large enough to support a claim. If it is not, we will say so and publish case notes instead of a rate.
  • It will name the suites, models and judges involved, or mark them as mixed and not comparable.
  • It will not invent a “typical team,” a benchmark win, or a market share of prompt failures.
  • Figures that cannot be computed from the opted-in set will be marked “Not stated,” never guessed.
  • When a dataset can be published, we intend to license it CC BY 4.0, unless a later edition says otherwise.

Related standing methods: Labs and the leaderboard. Those pages are also pending their first runs, and this report will not borrow numbers from them that do not exist.

FAQ

Where are the statistics?

There are none yet. The first edition is pending opt-in data. We will not invent averages, rankings or sample sizes to fill this page.

What counts as opt-in data?

A shared report or a roast or suite run that someone has explicitly asked us to include, with confirmation that it contains no secrets. Ordinary private runs in the Litmiq app are not included.

When is the first edition due?

After we have enough opted-in runs to say something true. We will not pick a fake publication date. When the first edition ships, this page will get a date and the figures will be sourced.

How is this different from Labs or the leaderboard?

Labs is for dated Litmiq-run research notes. The leaderboard is a fixed public suite. This report, when it exists, will describe opted-in real-world prompts, with the limits of that sample made explicit.

Offer an opted-in run

Nothing is included automatically. Email us, or use the roast or reports opt-in paths.

  • Free, no signup
  • Runs in your browser
  • Your OpenAI key, sent only to OpenAI