# LangSmith LangSmith is LangChain's hosted observability and evaluation platform for LLM apps. Traces every call in a chain or agent run, runs evals against datasets, stores prompts with versions, and lets you annotate outputs to build feedback loops. It started as the obvious add-on for [[LangChain]] and [[LangGraph]] users, but works fine with any framework now via OpenTelemetry — you don't have to be in the LangChain ecosystem to use it. ## What it gives you - **Tracing** — full nested view of every LLM call, tool call, retriever hit, and intermediate step in an agent run. Click in and see exactly what the model saw and what it returned. - **Datasets + evals** — collect examples (from production traces or hand-crafted), run them through your app, score outputs with LLM-as-judge or custom scorers, compare versions. - **Prompt management** — versioned prompts with a playground. Pull by tag from code; swap a prompt without a deploy. - **Annotations** — humans (or other LLMs) tag traces as good/bad with reasons; that data flows back into your eval datasets. ## When I'd reach for it - You're already deep in [[LangChain]] / [[LangGraph]] and want zero-config tracing. - You want a polished UI and don't mind paying / sending data to LangChain Inc. - You need prompt versioning + evals + tracing in one tool and don't want to self-host anything. ## Trade-offs - **Hosted by default.** Self-host exists (Enterprise tier) but it's not the easy path. If your data can't leave your infra, [[Langfuse]] or [[Helicone]] are friendlier starting points. - **Closed source.** You're trusting LangChain Inc. with prompts, traces, and eval data. - **Pricing scales with traces** — a noisy production app racks up cost fast unless you sample. ## Alternatives - [[Langfuse]] — open source, self-hostable, very similar feature set. - [[Helicone]] — proxy-based, focuses on cost tracking and caching first. - [[MLflow]] — broader ML platform that picked up LLM tracing recently. - [[Edgee]] — gateway pattern for agent traffic, observability is one slice. ## References - Website: https://www.langchain.com/langsmith - Docs: https://docs.smith.langchain.com/ ## Related - [[LangChain]] - [[LangGraph]] - [[Langfuse]] - [[Helicone]] - [[LLM Monitoring]] - [[AI Observability]] - [[MLflow]] - [[Edgee]]