At a glance
- Category
- Agent tools and integrations
- Tags
- Self-hosted
Code
- Last push
- 15 October 2026
For developers
- Works with
- LangChain, LlamaIndex, CrewAI, OpenAI Agents, LangGraph, PydanticAI and 24 more
DeepEval in our research
All researchStudy
AI Tool Pricing Index, November 2026
Pricing
Free only
29 of the 361 tools we checked are free with no paid plan.
Edition price, checked 6 November 2026.
Pricing pages of the AI tools, November 2026 editionas of n = 361 tools checked
What is DeepEval?
DeepEval is an LLM evaluation framework for AI engineers that turns model behavior into repeatable tests, traces, and scored runs. It supports Unit testing for LLMs, LLM-as-a-Judge, 50+ research-backed metrics, Native conversational evals, and Multi-modal by default, while tracing across LangChain, LlamaIndex, LangGraph, OpenTelemetry, and Vercel AI SDK.
DeepEval's homepage (checked 12 Oct 2026) lists Visa, Walmart, Toyota, Mastercard, NVIDIA, Mercedes-Benz, Siemens, and AXA as customers, and it runs locally in your own environment.
Still actively maintained
DeepEval's GitHub repository (confident-ai/deepeval) is not archived and received a commit on the day of this check (12 October 2026). The latest PyPI release, 4.2.8, shipped 2 October 2026, continuing a roughly weekly cadence of point releases through the second half of 2026 (4.2.4 on 22 Sep, 4.2.5 on 22 Sep, 4.2.6 on 24 Sep, 4.2.7 on 29 Sep). The project has contributions from 308 people, and it still ships under the Apache-2.0 license recorded in its LICENSE.md. The 'needs update' flag on this listing reflects our own re-check lapsing since May, not the project going quiet.
What the framework does well
DeepEval's core idea is pytest-native testing for LLM output: you write assert_test() calls using metrics like GEval or TaskCompletionMetric the same way you'd write any other unit test, and deepeval test run wires that into CI/CD. Confident AI's own plan-comparison page — the one place the vendor states a concrete metric count — lists 30+ single-turn and 15+ multi-turn DeepEval metrics, covering RAG (faithfulness, contextual precision/recall), agents (task completion, tool correctness, plan adherence), multi-turn conversations, MCP tool use, and multimodal output. Tracing integrations cover LangChain, LlamaIndex, LangGraph, OpenTelemetry, and the Vercel AI SDK, and evaluation runs locally against a judge model of your choice, including local models via Ollama, LM Studio, or vLLM. On PyPI, the package pulled 2.58 million downloads in the last month — a stronger, independently checkable usage signal than any developer count the vendor states.
Free framework, priced platform
DeepEval itself costs nothing: Apache-2.0, pip install deepeval, and your evaluation data stays on your own machine — the only thing that leaves is anonymous PostHog telemetry (event names and metric names, no PII), and that's opt-out. The paid product sits one layer up. Confident AI, built by the same team, adds shared dashboards, dataset versioning, and production monitoring on top of the same metrics: Free is $0 forever but caps you at 2 seats, 1 project, and 5 test runs a week; Starter is $200/month; Team is $2,000/month and is where SOC 2, SSO, and custom RBAC appear; Enterprise is custom. That Starter-to-Team jump is ten times the price for the compliance features a regulated buyer may need from day one, so it's worth checking which tier you'd actually land on before treating $200 as the number. Startups in the YC network, or anywhere with under $5M raised, can apply for a grant through Confident AI's own pricing page.
Who it says uses it
DeepEval's homepage (checked 12 October 2026) displays logos for Visa, Walmart, Toyota, Mastercard, EY, Mercedes-Benz, NVIDIA, AXA, Siemens, and CVS Health. DeepEval's own docs separately describe it as 'one of the most used' LLM eval frameworks 'in the world,' citing 20 million-plus daily evaluations — a vendor figure we have no independent way to confirm. On the Confident AI side specifically, three customers are named with attributed quotes on the pricing page — Finom, Amdocs, and Humach — plus an unnamed 'Fortune 500 medical device company.'
License and data handling
The repository's LICENSE.md and GitHub's own license detection both show Apache-2.0, with no separate commercial clause on the code that we could find — self-hosting for production use isn't gated behind a paid tier. The OSV vulnerability database returns no published advisories for the deepeval PyPI package as of this check. On the hosted side, Confident AI lists SOC 2 as a Team-tier ($2,000/month) feature rather than something every plan carries, which is worth checking directly if compliance is the reason you're evaluating the paid product.
DeepEval price and terms
Terms
- Free plan
- YesYesFree Apache 2.0 open-source library
The terms in full
runs locally. LLM-judge metrics need your own model provider key (e.g. OPENAI_API_KEY), billed by that provider.
For AI agents
- API
- Yesdeepeval.com,
Where these prices come from
deepeval.com, checked
All plans and limits
| Plan | Price | Main limits |
|---|---|---|
| DeepEval | Free | None stated |
Read from deepeval.com.
What we checked on DeepEval
Free to use
Anyone can pip install the Apache 2.0 framework and run evaluations without a sales call or an account.
In a newly created virtual environment, run: pip install -U deepeval
Public API
DeepEval is used as a documented Python library and CLI (deepeval test run), with metrics and test cases called from Python code; it is a code SDK rather than a hosted REST API.
Pytest-native evals that run in CI/CD or as Python scripts. Iterate locally, on your own environment, on your own criteria.
deepeval.comChecked
Open source
Apache-2.0 licence in its public repository
Apache License Version 2.0
github.com/confident-ai/deepeval/blob/m…Checked by us,
Questions about DeepEval
- How much does DeepEval cost?
- DeepEval is free to use. From deepeval.com, checked 6 November 2026.
- Does DeepEval have an API or an MCP server?
- DeepEval has an API. From deepeval.com, checked 6 November 2026.
- Is DeepEval open source, and can you host it yourself?
- DeepEval is open source. We checked github.com/confident-ai/deepeval/blob/main/LICENSE.md ourselves on 15 November 2026. You can host DeepEval yourself.
- What have we checked on DeepEval?
- 3 facts, under these headings: What it costs to start, how you build with it.
- How do we check DeepEval?
- We hold 3 facts on DeepEval, quoted from 3 pages. Facts last checked 15 November 2026. Price last checked 6 November 2026. Last change found 15 November 2026.
Sources and changes
Last full check
Sources on deepeval.com2
What changed2
- What it does
- What it does
Repository
- Licence
- Apache-2.0
- Stars
- 18,891
- Last push
- 15 October 2026
DeepEval price history
We have read the price once so far, on 6 November 2026: free.
| Price | Plan | First seen | Last seen | Readings | Source |
|---|---|---|---|---|---|
| Free | 6 November 2026 | 6 November 2026 | 1 | Pricing page, read 6 November 2026 |
From our research on 6 November 2026. A date is when we read the page, not when the price changed.
Similar in Agent tools and integrations
See all in Agent tools and integrations
- ERIUS PHONERented cloud Android phones, each in its own container, that keep an agent's apps and sign-ins between sessions.
- TinyFishWeb search, page fetching, cloud browsers and goal-driven web automation for AI agents, with search and fetch free.
- SmitheryPublic MCP server registry, now run by Arcade, listing direct vendor MCP URLs after its proxy shut down.
- UI DiscoveryA library of real websites, iOS apps and dashboards with each one's palette, fonts and original build prompt.
Is DeepEval yours?
Free, and the page then carries the verified owner mark. You confirm it with an email at deepeval.com. Claim this listing
Report an error if a fact is wrong.