On this page
All automation tools
Promptfoo: artificial intelligence features and setup
A platform for AI security and for evaluating models and prompts during development.
Tool by the Otomatzia editorial team · source content checked on
Overview
Promptfoo is designed for testing before relying on a prompt or model in a live environment. Run test cases, check for prompt injection, and uncover unsafe or inconsistent responses. It suits a development team that wants a quality gate instead of relying only on manual chat experiments. Before finalizing a release, include examples in Hebrew and cases where the model should refuse. Keep the test suite alongside the code so it can be run again after every change.
Features
- Evaluate prompts and models during development
- Security and prompt injection tests
- Compare versions before release
Business uses
- Test an internal agent against attempts to change its instructions
- Compare two prompts using the same set of questions
Getting started
- Write a few test cases with expected results.
- Run them against the current prompt version.
- Add a refusal case and an injection case before release.
What to check
- A test suite in English only may miss failures in Hebrew.
- Evaluation does not replace monitoring real runs for customers.
Sources to check next
Vendor features and prices change. Confirm details on the linked official pages before you rely on them.