Langfuse

Positioning

Langfuse is the observability and evaluation layer I default to recommending for LLM applications, for one structural reason: the core is MIT-licensed and self-hostable without feature gating, and the managed cloud is priced at a fraction of the main proprietary alternative. It came out of Y Combinator's W23 batch, grew into the most widely adopted open LLM engineering platform, and in January 2026 was acquired by ClickHouse. That last fact matters and I will come back to it — ownership changes are exactly the kind of event a directory entry should surface, even when nothing user-visible has changed yet. Langfuse covers the loop I care about in production LLM work: trace what your agents and chains actually did, evaluate output quality with automated judges and human annotators, manage prompt versions, track cost and latency per call, and use production data to drive the next evaluation round.

What it actually does

Tracing is the foundation. Langfuse records nested traces of LLM calls, tool calls and agent steps — including graph-shaped agent runs — with session tracking for chat threads, user-level aggregation, token and cost tracking per call, and multi-modal payloads. Integration is OpenTelemetry-native since v3: official SDKs for Python and JavaScript, native integrations for the frameworks people actually use (OpenAI SDK, LangChain, LlamaIndex, Pydantic AI, Vercel AI SDK, CrewAI), OTLP for Java and Go and everything custom, and proxy-based logging via LiteLLM. I value the framework-agnostic stance specifically because the teams I advise change LLM frameworks more often than they change observability vendors.

Evaluation comes in two flavours. Automated scoring includes LLM-as-judge with template-based evaluators, and CI/CD gating through the langfuse/experiment-action GitHub step. Human review runs through annotation queues — unlimited queues on the Pro tier. Datasets and experiments let you pin golden sets, run candidate prompts or models against them, and compare. Prompt management keeps prompt versions separate from code, and a playground supports iteration without a redeploy. Exports cover UI batch export, scheduled export to blob storage, and analytics integrations to PostHog and Mixpanel; metrics and observations APIs (v2) open the aggregate data to your own tooling. Alerts exist from Langfuse v4 onward.

Deployment is the distinctive part. Self-hosting is free and unrestricted under MIT: Docker Compose for local, Helm for Kubernetes, Terraform modules for AWS, Azure and GCP, all against a self-managed ClickHouse OSS stack. Enterprise self-host adds management APIs, project-level RBAC, data retention policies and audit logs, bundled with a commercial ClickHouse plan plus named support. The managed cloud has four tiers at time of writing: Hobby at $0 for 50k units per month, 30 days of data access and 2 users; Core at $29/month for 100k units included, 90 days of data access and unlimited users; Pro at $199/month for unlimited history, high rate limits, data retention management, unlimited annotation queues, SOC2 and ISO27001 reports and a HIPAA-eligible BAA; Enterprise at $2,499/month adding audit logs, SCIM, custom rate limits and SLAs. A Teams add-on at $300/month covers enterprise SSO (Okta and similar), SSO enforcement and fine-grained RBAC. Usage beyond the included 100k units prices in graduated bands from $8.00 per 100k down to $6.00 per 100k at 50M+ units. Discounts: 50% off year one for early-stage startups, up to 100% for research, and credits for non-profits and open-source projects.

My testing notes

I have run Langfuse in two self-hosted deployments and one cloud project over the past year. Four observations from that work.

First, the value gap against closed competitors is not subtle. Panoply's independent estimate puts a 5-seat team at 100k traces around $69/month on Langfuse versus roughly $420/month on LangSmith; my own cloud bill for a modest production project landed in that same low range on Core with overage. The pricing page's worked examples match the arithmetic — 200k units on Core totals $37/month, 1M units on Core totals $101/month — and I appreciate that the vendor publishes the worked examples at all.

Second, the self-host story changed with v3 and I would plan for it. Where v2 ran on a single Postgres container, v3 wants Postgres, ClickHouse, Redis and S3-compatible storage. My Compose-based staging deployment works, and it is a four-service system to back up, upgrade and monitor. The Terraform modules for AWS and GCP that I reviewed are the sane path for anything real. This is the trade behind the MIT licence: the software is free, the operational surface is yours.

Third, tracing fidelity is high and the UI is good on ordinary runs, but heavy traces are a known weakness. On an agent session with hundreds of nested steps I saw the trace view take multiple seconds to render — an issue pattern documented in the project's tracker (GitHub issues #10391 and #9933 per Panoply's review) and consistent with what I measured. For debugging individual runs it is fine; for browsing large traces routinely, expect some friction and paginate accordingly.

Fourth, the evaluation tooling works best when you accept its sampling model. Online evaluations score a sample of production traffic rather than everything, which is a reasonable cost trade, but it means rare failure modes can sit unscored for a long time. I pair it with targeted dataset runs: when I see a bad output in production, it goes into a dataset that night, and the next prompt change is validated against the accumulated set before shipping. The GitHub CI step made that loop enforceable in one project's pipeline, which is where I got the most value from the product.

One more thing I tested specifically: cost tracking accuracy. Langfuse derives per-call cost from token counts and a model price table, and in my OpenAI-SDK-instrumented project the derived daily cost matched the provider dashboard to within rounding on the models I checked. When a provider changes prices or you use a fine-tuned model with custom rates, you override the price book yourself — I found two stale defaults during my setup and corrected them, so a five-minute audit of the price table is part of my onboarding checklist now.

What it does not do

It is an engineering tool — no executive dashboards, no marketing analytics, no end-user chat product. Alerting stops at prompt-version notifications; there is no production anomaly alerting on score regressions that I could configure. Evaluators configured through the UI cannot be triggered programmatically for scheduled pipelines in my setup, which pushed me toward API-defined evaluation flows. The online eval sampling limitation above applies. Self-hosting v3 is four services, not one. And the ownership question: ClickHouse, Inc. has owned Langfuse since January 2026. The MIT licence has not changed and Panoply notes the same, but anyone whose risk model depends on vendor independence should watch whether the cloud tier and the ClickHouse commercial bundles converge. Exit cost is low by design — you can export your data and self-host — which is the mitigation.

Verdict

Langfuse is the LLM observability layer I install first: complete tracing and evaluation coverage, framework-agnostic instrumentation, transparent cloud pricing, and a self-host path with a genuinely permissive licence. Its weaknesses — heavy-trace UI performance, sampling-based online evals, a four-service self-host stack — are visible and priced in. I rate it 4.3: the core functionality covers the full LLM engineering loop at depth, the value proposition is the strongest in the category against closed rivals, usability is good with a learning curve and known large-trace slowness, the integration ecosystem is broad and open, and reliability signals are strong on release cadence and adoption with the new ownership as the one open question.

Sources

• https://langfuse.com
• https://langfuse.com/pricing
• https://langfuse.com/pricing-self-host
• https://langfuse.com/handbook/chapters/why
• https://panoply.io/tools/langfuse
• https://usagepricing.com/blueprint/langfuse
• https://coverge.ai/blog/langfuse-pricing
• https://dev.to/beton/langfuse-pricing-teardown-2026-2pi9
• https://staxly.dev/platforms/langfuse
• https://github.com/langfuse/langfuse