How to test prompts

How to test prompts from a Vercel AI SDK app in Litmiq

On this page

The short answer

  • There is no official Vercel AI SDK integration and no Litmiq package to install.
  • You test the prompt your app already uses: copy it, add cases, run them in the browser with your own key.
  • For CI, export promptfooconfig.yaml and run it in your own workflow. Litmiq does not receive that workflow or its secrets.

The Vercel AI SDK helps applications call models from TypeScript, often with streamText or generateText and a system prompt. Litmiq is not an official Vercel integration, is not a Vercel Marketplace app, and does not deploy with your Next.js project.

1. Lift the system prompt out of the route

Wherever you set system: (or assemble messages), copy that string into Litmiq. If the route adds tools, you can still test the first-step decision with the tool-calling template: Litmiq will score the JSON or text, not execute the tool.

2. Add cases before you deploy

Production traffic is a poor first test. Take the questions your UI already suggests, plus one injection and one “I don’t know” case, and run them in the browser. The template library is a start.

3. Run with your own key

Calls go from your browser to OpenAI or Anthropic, not through Vercel and not through Litmiq. When the suite is stable, export YAML and run it in CI on the same pull request that changes the route, as in GitHub Actions.

FAQ

Is there an official Vercel AI SDK integration?

No. This page describes a manual workflow using Litmiq features that already exist: the app, suite links and YAML export.

Will Litmiq call my app?

No. You copy a prompt and cases out of the app, or you point /app?suite= at a file you host.

Where do my keys go?

In Litmiq, they stay in your browser and are sent only to OpenAI or Anthropic. In CI, they stay in your own secrets store.

Open a starter suite

Load an illustrative pack, then replace the fictional facts with your prompt.

  • Free, no signup
  • Runs in your browser
  • Your OpenAI key, sent only to OpenAI

Follow Litmiq on LinkedIn