How to test prompts
How to test prompts from a LangChain app in Litmiq
On this page
The short answer
- There is no official LangChain integration and no Litmiq package to install.
- You test the prompt your app already uses: copy it, add cases, run them in the browser with your own key.
- For CI, export promptfooconfig.yaml and run it in your own workflow. Litmiq does not receive that workflow or its secrets.
LangChain is a widely used framework for building LLM applications in Python and JavaScript. Litmiq is not a LangChain integration, a LangChain partner product, or a substitute for LangSmith, LangChain’s official tracing and evaluation platform. There is no pip install litmiq and no callback that streams traces into this site.
What you can do is test the prompt your LangChain app uses, the same way you would test any other system prompt.
1. Extract the prompt
From a ChatPromptTemplate, a SystemMessage, or a string you pass to a model, copy the system text as it will actually be sent — including any default formatting instructions. If the chain interpolates variables, write test cases that fill those variables the way production would.
2. Collect cases
Turn real user messages, known edge cases and past failures into Litmiq cases with expected behavior. The template library has starting packs for support, RAG and safety. If the chain is a retriever-plus-prompt, snapshot the retrieved context into the system prompt, as the RAG template does. Litmiq will not call your retriever.
3. Run in Litmiq
Open the app, paste the prompt and cases, or load a suite with /app?suite=. Add your OpenAI or Anthropic key. It stays in the browser and is sent only to that provider — not to Litmiq, and not through LangChain.
4. Keep the same cases in CI
Litmiq can export promptfooconfig.yaml. Run that file in GitHub Actions or another runner, as described in prompt testing in CI/CD and testing prompts in GitHub Actions. LangSmith remains the official LangChain-adjacent platform if you want tracing and hosted datasets.
FAQ
Is there an official LangChain integration?
No. This page describes a manual workflow using Litmiq features that already exist: the app, suite links and YAML export.
Will Litmiq call my app?
No. You copy a prompt and cases out of the app, or you point /app?suite= at a file you host.
Where do my keys go?
In Litmiq, they stay in your browser and are sent only to OpenAI or Anthropic. In CI, they stay in your own secrets store.
Open a starter suite
Load an illustrative pack, then replace the fictional facts with your prompt.
- Free, no signup
- Runs in your browser
- Your OpenAI key, sent only to OpenAI