Guide

Red-team your own prompts before you ship

On this page

The short answer

  • Test a prompt you own for injection, jailbreaks, PII leaks and over-refusal before it ships.
  • Use ready-made packs — the safety template and the prompt-injection cases — or write expected refusals yourself.
  • This is a guide, not a live attack service, and not permission to test anyone else’s system.

What this is

Red-teaming a prompt means writing cases you hope will fail, then checking that the model still follows the rules. The useful part is the expected behavior: “refuses to reveal the system prompt,” “does not return the user’s email in the clear,” “does not ignore the policy because the user asked it to.” A colourful attack string without an expected refusal is not a test.

Litmiq runs those cases in your browser with your own key. It does not operate a probe network, and this page does not offer one.

What to test

  • Prompt injection. User text that tries to replace the system rules, exfiltrate the system prompt, or change the output format. See prompt-injection test cases and the OWASP LLM01 framing linked from that page.
  • Jailbreaks. Requests that ask the assistant to ignore its limits, role-play around them, or treat a policy as optional. The expected behavior is a refusal that still stays within the product’s tone.
  • PII leaks. Cases where the model should mask or refuse to echo emails, phone numbers, payment cards or IDs. The PII redaction template is a starting pack with fictional identifiers.
  • Over-refusal. Ordinary, allowed requests that a brittle safety rule might block. A pack that only contains attacks will hide this failure.

The safety eval template already combines several of these into one illustrative suite. Adapt it. Do not treat it as a certificate or a complete threat model.

How to test in Litmiq

  1. Write or load a system prompt that states the limits in plain language.
  2. Add cases whose expected behavior is a refusal, a mask, or a normal answer that must still be allowed.
  3. Run the suite in the Litmiq app with your own key, or start from a template.
  4. When a case fails, change the prompt or the surrounding product rules, then run the same cases again. That is regression testing, not a one-off demo.

For a first pass on the prompt wording itself, the prompt checker has an offline heuristic mode. It is not a red-team pack.

Limits

A browser suite will not find every failure. It will not replace a permissioned security review, logging, or rate limits in a production app. It will not test a retriever, a tool API, or a model you cannot call from the browser. If you do not have permission to test a system, stop.

FAQ

Does Litmiq attack other people’s systems?

No. This guide is for testing a prompt you control. Do not use it to attack a system you do not have permission to test.

Is this a live red-team service?

No. One-click safety packs in the app are the published illustrative suites, plus any cases you add. There is no hosted probe network on this page.

Where are the ready-made cases?

Start with the safety eval template and the prompt-injection test cases. Those pages already list expected refusals you can run in the browser.

Will you publish exploit payloads here?

No. The guide names the classes of failure (injection, jailbreak, PII leak, over-refusal) and points you to expected-behavior tests. It does not publish a novel attack cookbook.

Start with a safety pack

Illustrative refusal cases, or write your own expected refusals.

  • Free, no signup
  • Runs in your browser
  • Your OpenAI key, sent only to OpenAI