Template library
Prompt eval templates you can run in one click
On this page
The short answer
- 11 use-case templates, each with eight illustrative cases and a one-click Litmiq link.
- Every suite is fictional. There are no scores, ratings or usage claims.
- Links use
/app?suite=and a same-origin JSON file. The app previews the suite before loading it. - The earlier templates list, including benchmark suites, is still published.
Templates by use case
- Prompt eval template
Support-bot prompt tests you can run in one click
Eight single-turn cases for a fictional pantry shop's support assistant: policy answers, unknowns, tone, a prompt-injection attempt and a benign request that should not be refused.
Run in Litmiq → - Prompt eval template
RAG prompt tests: grounded answers, citations and I don’t know
Eight cases for a retrieval-augmented assistant. The passages are already in the system prompt, so you can test the prompt without a live retriever.
Run in Litmiq → - Prompt eval template
Extraction prompt tests: structured JSON from messy text
Eight invoice and email snippets for a fictional wholesale clerk that must return only a JSON object with a fixed set of keys.
Run in Litmiq → - Prompt eval template
Safety prompt tests: injection, leaks and over-refusal
Eight cases for a fictional software help assistant: injection, a canary leak check, a jailbreak tone, and two ordinary questions that must still be answered.
Run in Litmiq → - Prompt eval template
Classification prompt tests: one label, no commentary
Eight short IT tickets that must be labelled with exactly one of five classes. Several are mixed or not tickets at all.
Run in Litmiq → - Prompt eval template
Summarization prompt tests: decisions, actions, no inventions
Eight meeting transcripts and email threads for a fictional note-taker that must stay faithful, include owners, and stay short.
Run in Litmiq → - Prompt eval template
Tool-calling prompt tests: the right tool, or none
Eight user requests for a fictional shipping assistant with three tools. Litmiq scores the text or JSON the model writes; it does not execute tools.
Run in Litmiq → - Prompt eval template
Instruction-following tests: format, language and limits
Eight format and constraint cases for a fictional assistant that must obey output shape even when the user asks it not to.
Run in Litmiq → - Prompt eval template
Brand-voice prompt tests: tone, spelling and taboos
Eight customer messages for a fictional lighting shop whose voice is warm, British English, and tightly constrained on discounts and slang.
Run in Litmiq → - Prompt eval template
Multilingual prompt tests: language match, no mixed replies
Eight questions in English, Spanish, French and mixed language for a fictional travel-pass assistant that must answer in the user’s language without inventing policy.
Run in Litmiq → - Prompt eval template
Structured-output tests: strict JSON, no fences
Eight event blurbs that must become a single JSON object with a closed set of keys, including messy dates and non-events.
Run in Litmiq →
How they work
Each template is a Litmiq suite JSON: a system prompt, scoring criteria and cases with expected behavior. Opening the link does not call a model until you add a key and click Run. You can edit cases after they load and export them for CI.
Use these as starting points, then replace the fictional facts with your product’s. Add a case every time something fails in production.
FAQ
Are these official vendor templates?
No. They are illustrative Litmiq suites with fictional products. They are free to adapt under CC BY 4.0, with credit.
How does one-click run work?
Each page links to /app?suite=/assets/suites/….json. The app fetches that file from this site, shows a preview, and loads it only after you confirm.
What about the older templates page?
The original /templates list is still there, including benchmark and migration suites. This library is organised by use case, with a dedicated page for each.
Open a template
Start with the support-bot pack, or pick the use case that matches your prompt.
- Free, no signup
- Runs in your browser
- Your OpenAI key, sent only to OpenAI