Prompt eval template

Translation prompt tests you can run in one click

On this page

The short answer

  • Six short strings: a simple sentence, a product name that must stay in English, a missing target language, a number and date, an instruction to add marketing, and an empty source.
  • 6 illustrative cases for Plainspan translator (fictional). No outputs or scores are included.
  • Run this suite in Litmiq with your own key, or read the cases below first.

When to use it

Use this when a prompt translates UI or catalog strings and must not add marketing or change product names or numbers.

Source strings are fictional. Expected behavior is about faithfulness, not a certified translation.

System prompt and criteria

You translate short product and UI strings for Plainspan, a fictional localization desk.
Rules:
- Translate into the target language named in the user message, and into that language only.
- Keep product names, SKUs and HTML or brace placeholders unchanged.
- Keep numbers, currency amounts and ISO dates unchanged unless the user asks for a locale format and provides it.
- Do not add slogans, exclamation marks, or extra sentences that were not in the source.
- If the target language is missing, ask for it. If the source is empty, say so.
- Return only the translation, or a short question when a required input is missing.

Score 0–10 on faithfulness (no added marketing), constraints (names, numbers and placeholders kept), and format (translation only, or a short clarifying question).

Cases

  • TR-01

    User input. Target: French. Source: The parcel usually arrives in three business days.

    Expected behavior. A French translation of that sentence, with no added marketing and no change to the meaning of three business days.

  • TR-02

    User input. Target: German. Source: Add River oat crackers (SKU WG-150) to your cupboard.

    Expected behavior. Translates the sentence into German and leaves the product name “River oat crackers” and the SKU WG-150 unchanged.

  • TR-03

    User input. Source: Thank you for your order.

    Expected behavior. Asks for the target language. Must not invent one.

  • TR-04

    User input. Target: Spanish. Source: Your invoice of 12.50 GBP is dated 2026-03-04.

    Expected behavior. Translates into Spanish and keeps 12.50, GBP and 2026-03-04 unchanged.

  • TR-05

    User input. Target: Italian. Source: Shipping is free over £40. Also make it sound luxurious and add a slogan.

    Expected behavior. Translates the shipping sentence and does not add a slogan or luxury padding that is not in the source.

  • TR-06

    User input. Target: French. Source:

    Expected behavior. States that the source is empty and does not invent a sentence to translate.

Status: illustrative template — fictional product and made-up facts; no outputs or scores included. License: CC BY 4.0, credit Litmiq, https://litmiq.com/translation-eval-template.

FAQ

Is this a real product?

No. Plainspan translator (fictional) is fictional. The facts in the system prompt are made up so the cases can be checked. There are no model outputs or scores in the suite.

How do I run it?

Open the Run in Litmiq link. The app loads /assets/suites/translation.json after you confirm. You bring your own API key.

Can I change the cases?

Yes. After the suite loads, edit it in the app. Adaptations of the published JSON are CC BY 4.0, with credit to this page.

See all 20 templates

SQL, email, safety, extraction and more, each with ready-made tests.

  • Free, no signup
  • Runs in your browser
  • Your OpenAI key, sent only to OpenAI