# UpTrain

> Source: https://agentsindex.ai/uptrain

Open-source LLM evaluation framework with 20+ predefined checks, plus a managed dashboard that runs evals for you.

- Best for: Teams shipping LLM apps who need reliable evaluation, testing, and monitoring before production.

Starting price

Free

From [uptrain.ai](https://uptrain.ai/), checked 1 November 2026

- MCP server Not stated
- API Yes From docs.uptrain.ai/tutorials/uptrain-api-client, 1 November 2026 [docs.uptrain.ai](https://docs.uptrain.ai/tutorials/uptrain-api-client)
- Open source Yes Checked by us github.com/uptrain-ai/uptrain/blob/main/LICENSE, 15 November 2026 [Checked by us](https://github.com/uptrain-ai/uptrain/blob/main/LICENSE)
- Self-host Yes

- GitHub stars: 2.4k

[Docs](https://docs.uptrain.ai) [GitHub](https://github.com/uptrain-ai/uptrain)

What we checked

- Pages last read: 15 November 2026
- Facts checked: 4 facts · From 4 sources
- AI answers: Too few read · 58 buyer-question answers read, November 2026

[How we check](https://agentsindex.ai/methodology) [Editorial policy](https://agentsindex.ai/editorial-policy#not-for-sale)

[Visit UpTrain](https://agentsindex.ai/go/uptrain) [Compare](https://agentsindex.ai/compare?cat=agent-tools-and-integrations&products=uptrain)

## At a glance

- Category: [Agent tools and integrations](https://agentsindex.ai/categories/agent-tools-and-integrations)
- Tags: Open source, Self-hosted

### Code

- Last push: 18 August 2024

### For developers

- Works with: Google

## UpTrain in our research

[All research](https://agentsindex.ai/blog)

Study 19 October 2026 – 16 November 2026

### [AI Tool Pricing Index, November 2026](https://agentsindex.ai/data/ai-tool-pricing-index)

Pricing

Contact sales

82 of the 361 tools we checked ask you to contact sales for a price.

Edition price, checked 1 November 2026.

Pricing pages of the AI tools, November 2026 edition as of 16 November 2026 n = 361 tools checked

## What is UpTrain?

UpTrain is an LLM evaluation and improvement platform for teams shipping AI apps that scores outputs, compares prompt or model changes, and catches regressions before production. It combines Diverse evaluations, Faster and Systematic Experimentation, Automated Regression Testing, and Root Cause Analysis, with Google sign-in and an open-source core that supports self-hosting. The platform says it has evaluated more than 1,000,000 responses and handles 100, 10k, or million rows without failures.

## Development has stopped, though the code still runs

The last commit merged into uptrain-ai/uptrain's main branch was PR #706 on 14 May 2024, the day v0.7.1 shipped (per the GitHub API and the release's own changelog: [api.github.com/repos/uptrain-ai/uptrain/releases/tag/v0.7.1](https://api.github.com/repos/uptrain-ai/uptrain/releases/tag/v0.7.1)). The repository itself is not archived — GitHub's API still reports `archived: false` as of 7 October 2026 — so there's no banner telling a visitor this. But the practical state is the same: four pull requests from outside contributors (#762, #763, #764, #765) have been open since September 2026 with zero maintainer response, one of which (#765) documents and offers a one-line fix for a scoring bug that silently inverts pass/fail results in the ConversationGuidelineAdherence check when evaluations run locally. The PyPI package matches: 0.7.1, uploaded 14 May 2024, is still the latest version ([pypi.org/pypi/uptrain/json](https://pypi.org/pypi/uptrain/json)), and weekly downloads sit at 216 ([pypi.org/project/uptrain](https://pypi.org/project/uptrain/)). Apache-2.0 code that still installs and runs is usable — but budget for forking or patching it yourself if you hit a bug like #765, because nobody is merging fixes.

## What the open-source core actually does well

UpTrain's eval library covers real ground: 20+ pre-configured checks spanning response quality (completeness, conciseness, relevance), context-awareness (retrieval quality, factual accuracy, context utilization), language features (coherence, tonality), and safety (prompt injection, jailbreak detection) — all documented with working code at docs.uptrain.ai. It runs entirely on your own infrastructure using your own OpenAI/Anthropic/Mistral/Azure keys ([docs.uptrain.ai/getting-started/introduction](https://docs.uptrain.ai/getting-started/introduction)), which is the right shape for teams with data-governance constraints who don't want evaluation traffic going through a third-party API. Root-cause analysis — grouping failed cases by probable cause rather than just flagging them — is a feature few open-source eval libraries attempt. The GitHub repo's following ([github.com/uptrain-ai/uptrain](https://github.com/uptrain-ai/uptrain)) is evidence of real adoption, even if growth has since flattened.

## UpTrain AI the company vs. UpTrain the library

UpTrain was backed by Y Combinator (W23 batch) and launched in 2024 as a 'full-stack LLMOps platform' with a hosted dashboard alongside the open-source library ([ycombinator.com/launches/KGk-uptrain-ai-enterprise-grade-tooling-to-ship-llm-applications-confidently](https://www.ycombinator.com/launches/KGk-uptrain-ai-enterprise-grade-tooling-to-ship-llm-applications-confidently)). Co-founder Sourabh Agrawal's public activity today is centered on CombineHealth, a separate YC-backed healthcare AI company where he is CEO, not UpTrain (LinkedIn, checked 7 October 2026). UpTrain AI Inc's privacy policy was last updated 28 March 2023 ([uptrain.ai/privacy-policy](https://uptrain.ai/privacy-policy)). The marketing homepage still advertises a demo playground and a hosted dashboard, but demo.uptrain.ai timed out on every attempt during this check (7 October 2026) and uptrain.ai/pricing returns a 404 — so what the managed offering costs, or whether it's still actively sold, isn't established from what the vendor publishes today.

## Self-hosting and data handling

The evaluation framework runs locally by default — UpTrain's own docs state evaluations and analysis happen on your machine, with your LLM provider API calls being the only traffic that leaves your environment ([docs.uptrain.ai/getting-started/introduction](https://docs.uptrain.ai/getting-started/introduction)). The dashboard is self-hosted via Docker (`git clone` + `bash run_uptrain.sh`), with no SaaS data flow required to use the open-source evaluation library. UpTrain's privacy policy documents GDPR rights (access, rectification, erasure, restriction) for EU users and CCPA rights for California residents, though it's written for the company's own website/hosted-product data collection rather than the self-hosted library ([uptrain.ai/privacy-policy](https://uptrain.ai/privacy-policy)).

## UpTrain price and terms

### Terms

- Free plan: Yes Open-source framework is free under Apache 2.0 but needs your own OpenAI key (you pay the LLM costs)

The terms in full

The managed version gives 100 free trial credits per signup.

### For AI agents

- API: Yes [docs.uptrain.ai, 1 November 2026](https://docs.uptrain.ai/tutorials/uptrain-api-client)

[Developer docs](https://docs.uptrain.ai)

Where these prices come from

[uptrain.ai](https://uptrain.ai/), checked 1 November 2026

### All plans and limits

| Plan | Price | Main limits |
| --- | --- | --- |
| UpTrain | Free | None stated |

Read 1 November 2026 from [uptrain.ai](https://uptrain.ai/).

## What we checked on UpTrain

Yes Partly No

What it costs to start Yes

### What the price counts

The managed version is metered in credits, with LLM costs bundled into a paid plan, but the site publishes no credit price or plan rate; the open-source framework is free to run with your own OpenAI key.

> No, all the LLM costs are included in the paid plan, making it cheaper than other alternatives.

Frequently Asked Questions, 'Do I need to pay for OpenAI costs for running UpTrain evaluations?' [uptrain.ai](https://uptrain.ai/) Checked 1 November 2026

What it costs to start Yes

### Free to use

The evaluation toolkit is free and open source, though running it needs your own OpenAI key; the managed version adds 100 free trial credits on signup.

> Absolutely yes, we are proudly open-source. You would just need an OpenAI Key to use UpTrain.

FAQ, 'Is UpTrain free?' [docs.uptrain.ai/faq/faq](https://docs.uptrain.ai/faq/faq) Checked 1 November 2026

How you build with it Yes

### Public API

Documented API client in the Python package lets you run evaluations remotely against the UpTrain server using an API key created in the dashboard.

> How to run your evaluations remotely using the Uptrain API Client

Uptrain API Client tutorial, page description and 'Create an API Key' step [docs.uptrain.ai/tutorials/uptrain-api-c…](https://docs.uptrain.ai/tutorials/uptrain-api-client) Checked 1 November 2026

How you build with it Yes

### Open source

Apache-2.0 licence in its public repository

> Apache License Version 2.0

LICENSE in github.com/uptrain-ai/uptrain [github.com/uptrain-ai/uptrain/blob/main…](https://github.com/uptrain-ai/uptrain/blob/main/LICENSE) Checked by us, 15 November 2026

## Questions about UpTrain

- How much does UpTrain cost?: UpTrain is free to use. From uptrain.ai, checked 1 November 2026.
- Does UpTrain have an API or an MCP server?: UpTrain has an API. From docs.uptrain.ai/tutorials/uptrain-api-client, checked 1 November 2026.
- Is UpTrain open source, and can you host it yourself?: UpTrain is open source. We checked github.com/uptrain-ai/uptrain/blob/main/LICENSE ourselves on 15 November 2026. You can host UpTrain yourself.
- What have we checked on UpTrain?: 4 facts, under these headings: What it costs to start, how you build with it.
- How do we check UpTrain?: We hold 4 facts on UpTrain, quoted from 4 pages. Facts last checked 15 November 2026. Price last checked 1 November 2026. Last change found 15 November 2026.

[How we check every listing](https://agentsindex.ai/methodology)

## Sources and changes

Last full check 1 November 2026

Sources 3

- Read 1 November 2026 · [uptrain.ai](https://uptrain.ai/) /
- Read 1 November 2026 · [docs.uptrain.ai](https://docs.uptrain.ai/faq/faq) /faq/faq
- Read 1 November 2026 · [docs.uptrain.ai](https://docs.uptrain.ai/tutorials/uptrain-api-client) /tutorials/uptrain-api-client

What changed 1

- 15 November 2026 What it does

Repository

- Licence: Apache-2.0
- Stars: 2,365
- Last push: 18 August 2024

## UpTrain price history

We have read the price once so far, on 1 November 2026: free.

| Price | Plan | First seen | Last seen | Readings | Source |
| --- | --- | --- | --- | --- | --- |
| Free |   | 1 November 2026 | 1 November 2026 | 1 | [Pricing page, read 1 November 2026](https://uptrain.ai/) |

From our research on 1 November 2026. A date is when we read the page, not when the price changed.

## Similar in Agent tools and integrations

[See all in Agent tools and integrations](https://agentsindex.ai/categories/agent-tools-and-integrations)

- [ERIUS PHONE](https://agentsindex.ai/eriusphone) Rented cloud Android phones, each in its own container, that keep an agent's apps and sign-ins between sessions. From about $44 a month
- [TinyFish](https://agentsindex.ai/tinyfish) Web search, page fetching, cloud browsers and goal-driven web automation for AI agents, with search and fetch free. Free plan, then $0.002 per minute
- [Smithery](https://agentsindex.ai/smithery) Public MCP server registry, now run by Arcade, listing direct vendor MCP URLs after its proxy shut down. Free plan
- [UI Discovery](https://agentsindex.ai/uidiscovery) A library of real websites, iOS apps and dashboards with each one's palette, fonts and original build prompt. From $20 a month

## Is UpTrain yours?

Free, and the page then carries the verified owner mark. You confirm it with an email at uptrain.ai. [Claim this listing](https://agentsindex.ai/claim/uptrain?from=owner-line)

[Report an error](mailto:hello@agentsindex.ai?subject=Report%20an%20error%3A%20UpTrain%20(uptrain)&body=What%20is%20wrong%20on%20the%20page%2C%20and%20where%20on%20the%20product) if a fact is wrong.
