# Ragas

> Source: https://agentsindex.ai/ragas

Open-source LLM evaluation for repeatable experiments and tracked results.

- Best for: AI teams who need systematic evaluation loops for prompts, RAG systems, and agents.

Starting price

Free

From [docs.ragas.io/en/stable/getstarted/install](https://docs.ragas.io/en/stable/getstarted/install/), checked 14 November 2026

- MCP server Not stated
- API Yes From docs.ragas.io/en/stable, 14 November 2026 [docs.ragas.io](https://docs.ragas.io/en/stable/)
- Open source Yes Checked by us github.com/vibrantlabsai/ragas/blob/main/LICENSE, 15 November 2026 [Checked by us](https://github.com/vibrantlabsai/ragas/blob/main/LICENSE)
- Self-host Not stated

- GitHub stars: 16k

[Docs](https://docs.ragas.io/) [GitHub](https://github.com/vibrantlabsai/ragas)

What we checked

- Pages last read: 15 November 2026
- Facts checked: 3 facts · From 4 sources
- AI answers: Too few read · 58 buyer-question answers read, November 2026

[How we check](https://agentsindex.ai/methodology) [Editorial policy](https://agentsindex.ai/editorial-policy#not-for-sale)

[Visit Ragas](https://agentsindex.ai/go/ragas) [Compare](https://agentsindex.ai/compare?cat=agent-tools-and-integrations&products=ragas)

## At a glance

- Category: [Agent tools and integrations](https://agentsindex.ai/categories/agent-tools-and-integrations)
- Tags: Free, Open source

### Code

- Last push: 24 February 2026

### For developers

- Works with: Arize, LangSmith, Amazon Bedrock, Google Gemini, OCI Gen AI, AG-UI and 17 more

## Ragas in our research

[All research](https://agentsindex.ai/blog)

Study 19 October 2026 – 16 November 2026

### [AI Tool Pricing Index, November 2026](https://agentsindex.ai/data/ai-tool-pricing-index)

Pricing

Free only

29 of the 361 tools we checked are free with no paid plan.

Edition price, checked 14 November 2026.

Pricing pages of the AI tools, November 2026 edition as of 16 November 2026 n = 361 tools checked

## What is Ragas?

Ragas is an open-source LLM evaluation library for AI teams that need repeatable experiments across prompts, RAG systems, workflows, and agents. It combines Experiments-first approach, Ragas Metrics, dataset management, result tracking, and Test Data Generation, and integrates with LangChain, LlamaIndex, Haystack, LangGraph, Amazon Bedrock, Arize, LangSmith, Google Gemini, and OCI Gen AI. Public references include Atomicwork, Pinecone, Weaviate, Qdrant, Deepset Haystack, Mixedbread.ai, LangChain, and OpenAI.

## A Python library that scores LLM apps with LLM judges

Ragas is a library you `pip install` and call from code. Its [introduction](https://docs.ragas.io/en/stable/) describes moving teams "from \"vibe checks\" to systematic evaluation loops" by combining LLM-graded metrics with repeatable experiments. It also generates synthetic test sets from your documents by building a knowledge graph and writing single-hop and multi-hop questions, so a team without a labeled eval set can start from its own corpus.

## The library is free, and the judge model's tokens are the real bill

What costs money is the model doing the grading. The [cost analysis guide](https://docs.ragas.io/en/stable/howtos/applications/_cost/) states that "By default, Ragas does not calculate the usage of tokens for evaluate()"; you get a token count only after wiring in a `TokenUsageParser`, and a ready parser is shown for OpenAI. The only commercial offer on the site is help: the docs home says "Book a slot or drop us a line: founders@vibrantlabs.com", with no figure attached.

## An import issue with Langchain dependencies broke Ragas for months

The [release published on November 14, 2026](https://github.com/vibrantlabsai/ragas/releases) addresses it directly: "import ragas works again", with the VertexAI imports now optional. Its notes still list known issues, including that "Claude models reached through LiteLLM still receive Ragas' default sampling parameters".

## The docs disagree with themselves on dependencies and templates

The Phoenix (Arize) integration tutorial goes further back: it installs `"ragas<0.1.1"`, `"llama-index<0.10.0"` and `"openinference-instrumentation-llama-index<1.0.0"`, yet its own code later imports `from llama_index.core`. The same tutorial says "Let's generate 100 data points using Ragas" while the code beneath it sets `TEST_SIZE = 25`. The agent metrics and tutorials exist; the scaffolding command for them does not yet.

## Vibrant Labs maintains it while describing another focus

Ragas is published by Vibrant Labs, renamed on GitHub from explodinggradients to `vibrantlabsai`. The docs home adds that the team is "compressing this knowledge into a product to replace vibe checks with eval loops", but no such product is described anywhere on the site. Support is community-run: the [community page](https://docs.ragas.io/en/stable/community/) points to a Discord server and GitHub issues, and the latest release notes include a change titled "chore: remove office hours cal".

## Telemetry is on by default and eval data goes to your LLM provider

Because the library runs in your own environment, your eval data goes only to the LLM provider you configure, not to a Ragas server. The Phoenix (Arize) tutorial notes that "Ragas by default uses OpenAI models under the hood, but you're free to use any model of your choice", and states that the notebook "requires an OpenAI API key". For seeing what the judge sends, the integrations page lists Arize Phoenix and LangSmith as tracing tools that can be used "to get the traces of the evaluator LLMs".

## It suits Python teams, not buyers who want a hosted eval product

Ragas fits an engineering team that already writes Python, wants RAG, agent or SQL metrics it can run in CI or a notebook, and is willing to pay its own model provider for judge calls. It does not fit a team without a developer: there is no web app, no hosted dashboard and no vendor contract.

Our take

Check whether your evaluation workflow depends on frameworks beyond the listed integrations. The docs show LangChain, LlamaIndex, Haystack, LangGraph, Arize, LangSmith, Amazon Bedrock, Google Gemini, OCI Gen AI, and Arize Phoenix, so confirm your stack is covered before standardizing on it.

## Ragas price and terms

### Terms

- Free plan: Yes Open-source library, free to install and run

The terms in full

LLM calls made during evaluation are billed by the model provider you configure.

### For AI agents

- API: Yes [docs.ragas.io, 14 November 2026](https://docs.ragas.io/en/stable/)

[Developer docs](https://docs.ragas.io/)

Where these prices come from

[docs.ragas.io/en/stable/getstarted/install](https://docs.ragas.io/en/stable/getstarted/install/), checked 14 November 2026

### All plans and limits

| Plan | Price | Main limits |
| --- | --- | --- |
| Ragas | Free | None stated |

Read 14 November 2026 from [docs.ragas.io/en/stable/getstarted/install](https://docs.ragas.io/en/stable/getstarted/install/).

## What we checked on Ragas

Yes Partly No

What it costs to start Yes

### Free to use

The whole library installs free from PyPI with pip and is Apache-2.0 licensed; evaluations still call an LLM you supply and pay for, such as OpenAI.

> To get started, install Ragas using pip with the following command:

Installation page, opening sentence [docs.ragas.io/en/stable/getstarted/inst…](https://docs.ragas.io/en/stable/getstarted/install/) Checked 14 November 2026

How you build with it Yes

### Public API

Ragas is used through its documented Python API, with a References section covering evaluate(), metrics, LLMs, embeddings and test set generation; it is a library API, not a hosted REST service.

> API documentation and technical details for diving deeper.

Introduction page, References card [docs.ragas.io/en/stable](https://docs.ragas.io/en/stable/) Checked 14 November 2026

How you build with it Yes

### Open source

Apache-2.0 licence in its public repository

> Apache License Version 2.0

LICENSE in github.com/vibrantlabsai/ragas [github.com/vibrantlabsai/ragas/blob/mai…](https://github.com/vibrantlabsai/ragas/blob/main/LICENSE) Checked by us, 15 November 2026

## Questions about Ragas

- How much does Ragas cost?: Ragas is free to use. From docs.ragas.io/en/stable/getstarted/install, checked 14 November 2026.
- Does Ragas have an API or an MCP server?: Ragas has an API. From docs.ragas.io/en/stable, checked 14 November 2026.
- Is Ragas open source, and can you host it yourself?: Ragas is open source. We checked github.com/vibrantlabsai/ragas/blob/main/LICENSE ourselves on 15 November 2026.
- What have we checked on Ragas?: 3 facts, under these headings: What it costs to start, how you build with it.
- How do we check Ragas?: We hold 3 facts on Ragas, quoted from 4 pages. Facts last checked 15 November 2026. Price last checked 14 November 2026. Last change found 15 November 2026.

[How we check every listing](https://agentsindex.ai/methodology)

## Sources and changes

Last full check 14 November 2026

Sources 3

- Read 14 November 2026 · [docs.ragas.io](https://docs.ragas.io/en/stable/getstarted/install/) /en/stable/getstarted/install
- Read 14 November 2026 · [docs.ragas.io](https://docs.ragas.io/en/stable/) /en/stable
- Read 14 November 2026 · [github.com](https://github.com/vibrantlabsai/ragas) /vibrantlabsai/ragas

What changed 2

- 15 November 2026 What it does
- 14 November 2026 What it does

Repository

- Licence: Apache-2.0
- Stars: 16,035
- Last push: 24 February 2026

## Ragas price history

We have read the price once so far, on 14 November 2026: free.

| Price | Plan | First seen | Last seen | Readings | Source |
| --- | --- | --- | --- | --- | --- |
| Free |   | 14 November 2026 | 14 November 2026 | 1 | [Pricing page, read 14 November 2026](https://docs.ragas.io/en/stable/getstarted/install/) |

From our research on 14 November 2026. A date is when we read the page, not when the price changed.

## Similar in Agent tools and integrations

[See all in Agent tools and integrations](https://agentsindex.ai/categories/agent-tools-and-integrations)

- [ERIUS PHONE](https://agentsindex.ai/eriusphone) Rented cloud Android phones, each in its own container, that keep an agent's apps and sign-ins between sessions. From about $44 a month
- [TinyFish](https://agentsindex.ai/tinyfish) Web search, page fetching, cloud browsers and goal-driven web automation for AI agents, with search and fetch free. Free plan, then $0.002 per minute
- [Smithery](https://agentsindex.ai/smithery) Public MCP server registry, now run by Arcade, listing direct vendor MCP URLs after its proxy shut down. Free plan
- [UI Discovery](https://agentsindex.ai/uidiscovery) A library of real websites, iOS apps and dashboards with each one's palette, fonts and original build prompt. From $20 a month

## Is Ragas yours?

Free, and the page then carries the verified owner mark. You confirm it with an email at docs.ragas.io. [Claim this listing](https://agentsindex.ai/claim/ragas?from=owner-line)

[Report an error](mailto:hello@agentsindex.ai?subject=Report%20an%20error%3A%20Ragas%20(ragas)&body=What%20is%20wrong%20on%20the%20page%2C%20and%20where%20on%20the%20product) if a fact is wrong.
