Evals
According to OpenAI, evaluations (sometimes called evals) check a model's output against predefined style and content criteria. Writing evals is an essential part of building reliable LLM applications, especially when upgrading a model or trying a new one.
On this page
Source content checked . English translation published 2026-10-27.
In business automation
Evaluations check whether an AI component meets defined criteria across a collection of cases. They are useful before changing a model, prompt, or knowledge source to identify improvements as well as regressions in other cases.
Examples
These examples illustrate a possible workflow. Adapt the connections and rules to your own systems.
Business process example
A team saves service questions with expected answers and cases where the bot should refrain from answering. After changing a prompt, they run the collection and check accuracy, source use, and handoff to an agent.
Relevant tools
- LangGraph
- You can build an agent that runs against test cases; evaluation and observability tools complement workflow checks.
What to check
- A single average hides failures. Report factual errors, unauthorized actions, and cases with missing information separately.