Skip to content
AgentsIndex

UpTrain

Open-source LLM evaluation framework with 20+ predefined checks, plus a managed dashboard that runs evals for you.

Best for
Teams shipping LLM apps who need reliable evaluation, testing, and monitoring before production.
GitHub stars
2.4k

What we checked

Pages last read
Facts checked
4 facts

From 4 sources

AI answers
Too few read

58 buyer-question answers read, November 2026

At a glance

Tags
Open source, Self-hosted

Code

Last push
18 August 2024

For developers

Works with
Google

UpTrain in our research

All research

Study

AI Tool Pricing Index, November 2026

Pricing

Contact sales

82 of the 361 tools we checked ask you to contact sales for a price.

Edition price, checked 1 November 2026.

Pricing pages of the AI tools, November 2026 editionas of n = 361 tools checked

What is UpTrain?

UpTrain is an LLM evaluation and improvement platform for teams shipping AI apps that scores outputs, compares prompt or model changes, and catches regressions before production. It combines Diverse evaluations, Faster and Systematic Experimentation, Automated Regression Testing, and Root Cause Analysis, with Google sign-in and an open-source core that supports self-hosting. The platform says it has evaluated more than 1,000,000 responses and handles 100, 10k, or million rows without failures.

Development has stopped, though the code still runs

The last commit merged into uptrain-ai/uptrain's main branch was PR #706 on 14 May 2024, the day v0.7.1 shipped (per the GitHub API and the release's own changelog: api.github.com/repos/uptrain-ai/uptrain/releases/tag/v0.7.1). The repository itself is not archived — GitHub's API still reports archived: false as of 7 October 2026 — so there's no banner telling a visitor this. But the practical state is the same: four pull requests from outside contributors (#762, #763, #764, #765) have been open since September 2026 with zero maintainer response, one of which (#765) documents and offers a one-line fix for a scoring bug that silently inverts pass/fail results in the ConversationGuidelineAdherence check when evaluations run locally. The PyPI package matches: 0.7.1, uploaded 14 May 2024, is still the latest version (pypi.org/pypi/uptrain/json), and weekly downloads sit at 216 (pypi.org/project/uptrain). Apache-2.0 code that still installs and runs is usable — but budget for forking or patching it yourself if you hit a bug like #765, because nobody is merging fixes.

What the open-source core actually does well

UpTrain's eval library covers real ground: 20+ pre-configured checks spanning response quality (completeness, conciseness, relevance), context-awareness (retrieval quality, factual accuracy, context utilization), language features (coherence, tonality), and safety (prompt injection, jailbreak detection) — all documented with working code at docs.uptrain.ai. It runs entirely on your own infrastructure using your own OpenAI/Anthropic/Mistral/Azure keys (docs.uptrain.ai/getting-started/introduction), which is the right shape for teams with data-governance constraints who don't want evaluation traffic going through a third-party API. Root-cause analysis — grouping failed cases by probable cause rather than just flagging them — is a feature few open-source eval libraries attempt. The GitHub repo's following (github.com/uptrain-ai/uptrain) is evidence of real adoption, even if growth has since flattened.

UpTrain AI the company vs. UpTrain the library

UpTrain was backed by Y Combinator (W23 batch) and launched in 2024 as a 'full-stack LLMOps platform' with a hosted dashboard alongside the open-source library (ycombinator.com/launches/KGk-uptrain-ai-enterprise-grade-tooling-to-ship-llm-applications-confidently). Co-founder Sourabh Agrawal's public activity today is centered on CombineHealth, a separate YC-backed healthcare AI company where he is CEO, not UpTrain (LinkedIn, checked 7 October 2026). UpTrain AI Inc's privacy policy was last updated 28 March 2023 (uptrain.ai/privacy-policy). The marketing homepage still advertises a demo playground and a hosted dashboard, but demo.uptrain.ai timed out on every attempt during this check (7 October 2026) and uptrain.ai/pricing returns a 404 — so what the managed offering costs, or whether it's still actively sold, isn't established from what the vendor publishes today.

Self-hosting and data handling

The evaluation framework runs locally by default — UpTrain's own docs state evaluations and analysis happen on your machine, with your LLM provider API calls being the only traffic that leaves your environment (docs.uptrain.ai/getting-started/introduction). The dashboard is self-hosted via Docker (git clone + bash run_uptrain.sh), with no SaaS data flow required to use the open-source evaluation library. UpTrain's privacy policy documents GDPR rights (access, rectification, erasure, restriction) for EU users and CCPA rights for California residents, though it's written for the company's own website/hosted-product data collection rather than the self-hosted library (uptrain.ai/privacy-policy).

UpTrain price and terms

Terms

Free plan
YesYesOpen-source framework is free under Apache 2.0 but needs your own OpenAI key (you pay the LLM costs)
The terms in full

The managed version gives 100 free trial credits per signup.

For AI agents

Developer docs

Where these prices come from

uptrain.ai, checked

All plans and limits

PlanPriceMain limits
UpTrainFreeNone stated

Read from uptrain.ai.

What we checked on UpTrain

YesPartlyNo

What it costs to startYes

What the price counts

The managed version is metered in credits, with LLM costs bundled into a paid plan, but the site publishes no credit price or plan rate; the open-source framework is free to run with your own OpenAI key.

No, all the LLM costs are included in the paid plan, making it cheaper than other alternatives.

uptrain.aiChecked

What it costs to startYes

Free to use

The evaluation toolkit is free and open source, though running it needs your own OpenAI key; the managed version adds 100 free trial credits on signup.

Absolutely yes, we are proudly open-source. You would just need an OpenAI Key to use UpTrain.

docs.uptrain.ai/faq/faqChecked

How you build with itYes

Public API

Documented API client in the Python package lets you run evaluations remotely against the UpTrain server using an API key created in the dashboard.

How to run your evaluations remotely using the Uptrain API Client

docs.uptrain.ai/tutorials/uptrain-api-c…Checked

Questions about UpTrain

How much does UpTrain cost?
UpTrain is free to use. From uptrain.ai, checked 1 November 2026.
Does UpTrain have an API or an MCP server?
UpTrain has an API. From docs.uptrain.ai/tutorials/uptrain-api-client, checked 1 November 2026.
Is UpTrain open source, and can you host it yourself?
UpTrain is open source. We checked github.com/uptrain-ai/uptrain/blob/main/LICENSE ourselves on 15 November 2026. You can host UpTrain yourself.
What have we checked on UpTrain?
4 facts, under these headings: What it costs to start, how you build with it.
How do we check UpTrain?
We hold 4 facts on UpTrain, quoted from 4 pages. Facts last checked 15 November 2026. Price last checked 1 November 2026. Last change found 15 November 2026.

How we check every listing

Sources and changes

Last full check

Repository

Licence
Apache-2.0
Stars
2,365
Last push
18 August 2024

UpTrain price history

We have read the price once so far, on 1 November 2026: free.

PricePlanFirst seenLast seenReadingsSource
Free1 November 20261 November 20261Pricing page, read 1 November 2026

From our research on 1 November 2026. A date is when we read the page, not when the price changed.

Similar in Agent tools and integrations

See all in Agent tools and integrations

Is UpTrain yours?

Free, and the page then carries the verified owner mark. You confirm it with an email at uptrain.ai. Claim this listing

Report an error if a fact is wrong.