How Makers Build
Check my stackPlan my stack
Menu

Braintrust

Observability and evaluation platform for AI agents, letting teams trace requests, score outputs and run experiments to compare prompts and models.

Visit braintrust.dev ↗Alternatives
Works with AI agents:llms.txtMCPCLI
Ask your AI about this, with this page as the source:ChatGPT ↗Claude ↗Perplexity ↗

Braintrust is a hosted platform for measuring whether an AI feature is getting better or worse. You log production traces, collect examples into datasets, and run experiments that score each output, so a prompt or model change is judged on the same inputs instead of by spot-checking a few responses.

The building block is an eval: a dataset, a task (your prompt, function or agent), and scorers. You write it in code with the SDK's Eval function, run it locally or in CI, and each run becomes an experiment you can diff against earlier ones. Full SDKs exist for TypeScript, Python, Go, Java, Ruby and C#, with wrappers for provider SDKs and frameworks such as the Vercel AI SDK, LangChain and the OpenAI Agents SDK; OpenTelemetry spans can be sent to its OTLP endpoint.

Around that you get LLM-as-a-judge and code scorers (including the open-source autoevals library), human review, versioned prompts with a playground, online scoring of production logs, and a gateway that puts several model providers behind one API. The data plane, where traces and datasets live, runs in a US or EU region you choose when creating the organization.

The EU region can't be changed later, and Braintrust advises BYOC or self-hosting in the EU for full GDPR compliance; both, along with a SOC 2 report and a BAA, are Enterprise-only. Even when self-hosted, the control plane (UI, auth, metadata) stays with Braintrust.

Where it fits

LLM Observability & Evals · Hosted eval platformEval-driven work — scoring outputs and comparing prompts and models side by side in experiments.Trade-off: Closed source, and the paid tier is priced for teams rather than hobby projects.

How Braintrust itself is built

4 tools, from its own code, website and Product Hunt page.

ClerkPostmarkResendVercel

Who uses it

5 makers' products, each linked to the source that shows it, and 10 open-source projects that declare it in their code.

Subprocessor list 5Declared in code 10How evidence is collected →

Open source: a project that declares Braintrust as a dependency in its public code — verifiable, but not necessarily a live product.

Loved and watch-outs

Themes that recur in makers' words and Hacker News comments, each linked to what it summarises.

Most loved
  • Setting up datasets and running evals against different models is quick. HNHN 2HN 3
  • A solid eval platform for checking whether prompts produce consistent results. HNHN 2HN 3
Watch-outs
  • Its core dataset-and-pass-rate workflow is simple enough that teams say they could build it themselves. HNHN 2
  • Unlike Promptfoo or Laminar, it is closed source. HN

Reliability and open issues

Incidents on its status page9 in the last 90 days, 8 major · 25+ in the last year
  • major Elevated errors and latency on the US data plane API Sep 2026
  • major Slow or failed data loading in US region Sep 2026
  • major Elevated 5XXs on Control Plane API Sep 2026
From its own status page, 2026-10-02. Vendors decide what they post; one page can cover several products.

Alternatives to Braintrust

All alternatives by situation →
LangSmithApps built on LangChain or LangGraph, where tracing works with almost no setup.Free tier11 makers · LLM observability & evalsCompare with Braintrust →
LangfuseTracing, prompt management and evals in one tool you can run on your own server for free.Open sourceFree tier33 makers · LLM observability & evalsCompare with Braintrust →
HeliconeGetting request logs, costs and latency by changing one base URL, with no SDK instrumentation.Open sourceFree tier13 makers · LLM observability & evalsCompare with Braintrust →
Arize PhoenixOpenTelemetry-based tracing and evals you can start locally in a notebook, then self-host or move to its cloud.Self-hostableFree tier
OpikTracing, evaluations and prompt optimization in an Apache-licensed platform you self-host with Docker or Kubernetes, or use on Comet's cloud.Open sourceFree tier
PromptfooRunning prompts and models against test cases from the command line or CI, including red-team tests for prompt injection and data leaks.Open sourceFree tier
DeepEvalWriting LLM evals as pytest-style tests in Python, with ready-made metrics for hallucination, faithfulness and answer relevancy.Open sourceFree tier

Questions makers ask about Braintrust

Which languages can I use it from?

Braintrust has full SDKs for TypeScript, Python, Go, Java, Ruby and C# for logging traces, managing prompts and datasets, and running evals, plus an alpha Rust SDK with a smaller feature set. source ↗

Can I keep my data in the EU?

Yes. When you create an organization you choose a US or EU data plane, which holds traces, logs, datasets and prompt completions; the choice can't be changed afterwards. source ↗

Can I self-host it?

You can run the data plane in your own AWS, GCP or Azure account, either operated by you (self-hosted) or by Braintrust (BYOC), while Braintrust keeps running the control plane. Both options require the Enterprise plan. source ↗

Is it SOC 2 and HIPAA compliant?

Braintrust is SOC 2 Type II compliant and can sign BAAs for PHI. For GDPR it signs DPAs but recommends BYOC or self-hosting with EU data residency for full compliance. source ↗

What does the free Starter plan limit, apart from price?

Starter keeps data for 14 days, includes a monthly allowance of processed data and scores, offers Google-only SSO with just the Owner role, and has no custom dashboards. SOC 2 attestation, a BAA and custom retention come only with Enterprise. source ↗

Can I send traces through OpenTelemetry?

Yes. An existing OpenTelemetry pipeline can export to Braintrust's OTLP endpoint with an API key and a header naming the target project, through an exporter, a Collector or Braintrust's span processor. source ↗

Can I rely on its gateway in production?

The Braintrust-hosted gateway is in public preview and has no SLA, so Braintrust warns that its behavior, APIs and configuration may change if you use it in production. source ↗

Is Braintrust free?

Yes — there is a free tier a small product can run on; paid use starts at $249/mo. source ↗

Is Braintrust open source or self-hostable?

Not open source, and hosted only.

Can AI coding agents work with Braintrust?

It serves an llms.txt docs index; it has an official MCP server (io.github.braintrustdata/braintrust); it has an official CLI (braintrust).

Who uses Braintrust?

5 makers' products we track, each with a source, and 10 open-source projects declare it in their code. source ↗