Arize Phoenix vs LangSmith
Two sides of the LLM observability & evals decision: open-source platform and hosted eval platform. When each fits, what it costs, who moves from one to the other, and what makers who chose it say.
Which fits you
Choose Arize Phoenix if- You already use OpenTelemetry or want vendor-neutral instrumentation
Use it whenYou already use OpenTelemetry or want vendor-neutral instrumentation.
Trade-offSource-available under the Elastic License rather than a permissive open-source license.
Choose LangSmith if- Your app is built on LangChain or LangGraph
Use it whenYou already use the LangChain stack.
Trade-offPaid per seat beyond the free tier; self-hosting is enterprise-only.
At a glance
| Arize Phoenix | LangSmith | |
|---|---|---|
| Used by | No maker's product yet · 19 open-source projects | 11 makers' products · 49 open-source projects |
| Cost at default usagetraces 100k traces | Over plan limits | $475/mo Developer |
| Downloads | 80.4k/wk+180% vs npm | 6.1M/wk−4% vs npm |
| Pricing | Phoenix is free to self-host; Arize's managed product is Arize AX, with paid plans from $50/month. · paid from $50/mo | Free tier with a monthly trace limit; paid per-seat plan with usage overage; custom Enterprise, including self-hosted. · paid from $39/seat/mo |
| Free tier | Yes | Yes |
| Open source | No · self-hostable | No |
| Incidents, 90 daysfrom its status page | no public status feed | 0 |
Cost as you grow
Both cost $0 up to 1k traces; from 10k traces Arize Phoenix costs less ($0 vs $25); from about 100k traces LangSmith does ($475 vs over its limits). They're different kinds of tool — open-source platform and hosted eval platform — so the prices don't buy the same thing.
The numbers, plan by plan
| Traces per month | Arize Phoenix | LangSmith |
|---|---|---|
| 1,000 | $0 AX Free | $0 Developer |
| 10,000 | $0 AX Free | $25 Developer |
| 50,000 | $50 AX Pro | $225 Developer |
| 100,000 | over plan limits | $475 Developer |
| 500,000 | over plan limits | $2,475 Developer |
| 1,000,000 | over plan limits | $4,975 Developer |
| 5,000,000 | over plan limits | $24,975 Developer |
| 10,000,000 | over plan limits | $49,975 Developer |
From each vendor's pricing page: Arize Phoenix, LangSmith.
What makers say
Makers on using it for LLM observability & evals, from Product Hunt and Starter Story interviews, each linked to the source. Products with a page of their own and fuller notes first.
On Arize PhoenixNo maker quote about Arize Phoenix for LLM observability & evals yet.
On LangSmithLangSmith’s real-time analytics and versioning keep our AI agents rock-solid -- so everything just works better.
You can't build AI agents without monitoring. Metadata filtering is strong.
I deployed the manage Pig agent on LangGraph and it's been smooth sailing!
Loved and watch-outs
Themes that recur in makers' words and Hacker News comments, each linked to what it summarises.
Arize PhoenixNothing that recurs in what we collected yet.LangSmith- It keeps pulling users toward the LangChain platform, and LangChain docs push LangSmith, which feels like lock-in. HNHN 2HN 3HN 4
- Traces show which agent failed but not why, so root-cause analysis stays manual. HNHN 2HN 3HN 4
- Viewing your own traces requires a cloud account, with no local-first option. HNHN 2
Who uses each
Arize Phoenix0 makers' productsLangSmith11 makers' products
lmChatGPTtfy
Watchman AI
Meet-TingWhat makers pair each with
With Arize Phoenix
LatitudeAgent-KBASSERTcuga-agentDatus-agentkonfig
LlamaIndex
Mastraminion-agentOAgentsopeninferencesec-insightsand 1 moreLatitudecuga-agentkonfigLlamaIndexminion-agentneuro-san-studioopeninferencesyftrMastraopeninferenceyoutube_video_codeLatitudecuga-agentDatus-agentkonfigMastrasec-insightssyftryoutube_video_codeLlamaIndexminion-agentsec-insightsyoutube_video_codeLatitudeLlamaIndexMastraWith LangSmith
Astrid
Pig
Traycer AIAstrid
NeuralAgentTraycer AINeuralAgentTraycer AI
GitLawTraycer AIGitLaw
IntrycTraycer AINeither quite right?
See the whole LLM observability & evals decision →
LangfuseOpen-source platformTracing, prompt management and evals in one tool you can run on your own server for free.vs Arize Phoenix →vs LangSmith →
HeliconeProxy loggingGetting request logs, costs and latency by changing one base URL, with no SDK instrumentation.vs LangSmith →
BraintrustHosted eval platformEval-driven work — scoring outputs and comparing prompts and models side by side in experiments.vs LangSmith →