Editorial template

State of LLM reliability: a template for monthly notes

On this page

The short answer

  • This is a template for a short monthly note, not a published issue with new scores.
  • The only original measurement Litmiq has in this series so far is the LLM Reliability Benchmark 2026 (v1.0, 12 November 2026).
  • A future issue should be written only when there is a new run, a correction, or a clearly dated change in the public record (for example a model shutdown).

What this series is for

Journalists and newsletters can cite a short, dated reliability note more easily than a long methods page. The series exists so that when we re-run the suite or record a material public change, the write-up has a stable shape: what changed, what did not, and where the files are.

Until a new issue is actually published, cite the existing benchmark and the versioned dataset.

Outline to fill (do not publish empty)

  1. Hed and date. “State of LLM reliability, [month year]”. The date is the measurement date or the date of the public change, not the day the page was built.
  2. One-paragraph answer. What changed since the last issue, in plain language, with numbers only if they come from a new run or a cited source.
  3. Scope. Which suite version, which models, which judge or grader. Link the dataset folder (for example /assets/benchmark/v1.1/ once that folder exists).
  4. Results table. The same columns as the v1.0 leaderboard, plus a change-vs-previous column if a model was in both runs. Do not fill this table with invented deltas.
  5. What broke, qualitatively. Two or three cases that flipped, with the expected behavior restated. No screenshots of unpublished runs.
  6. Public context. Shutdowns, price changes or law updates that were already dated on Litmiq pages (tracker, pricing report, laws report), with their source links.
  7. Limits. Repeat that this is not a general intelligence ranking, that Workers AI (or whichever host) is a specific environment, and that missing models were missing for a stated reason.
  8. Cite this. Text and BibTeX, CC BY 4.0, linking the issue and the dataset version.

Rules for each issue

FAQ

Is a November 2026 issue published here?

The original benchmark (v1.0, measured 12 November 2026) is the data we have. This page is the template for future monthly notes, not a new report with new numbers.

Can I republish the outline as if it were a finished report?

No. Fill it only with new, dated measurements or clearly cited updates. Inventing a monthly score would break the series.

What licence will issues use?

The same CC BY 4.0 used for the benchmark dataset, unless an issue’s sources require otherwise, which will be stated on that issue.

Read the existing benchmark

v1.0 is the current measurement. This template waits for the next real run.

  • Free, no signup
  • Runs in your browser
  • Your OpenAI key, sent only to OpenAI

Follow Litmiq on LinkedIn