LangSmith: artificial intelligence features and setup

Overview

LangSmith is designed for teams building agents who want to see the chain of steps, failures, and cost for each run. It is especially useful when a process is built around LangChain components, but it can also be used for general monitoring of model calls. Before deciding to use it as your only observability layer, check what data leaves the application and how test environments are separated from customer environments.

Features

• Tracing chains and agents at runtime
• Evaluations and cost monitoring for model calls
• Integration with LangChain components

Business uses

• Finding the step where an agent gets stuck or repeats the same action
• Checking how a change to an agent's tool affects customer outcomes

Getting started

• Connect a test project and send a few short chains.
• Define which traces are stored only in the test environment.
• Check a run with a failing tool and a run with partial output.

What to check

• Check environment separation before sending traces containing customer data.
• Do not assume that integration with LangChain covers every external tool the agent calls.