How many Testing & Evaluation tools are listed?

20 total. 20 have a free option and 0 are open source.

What's a typical price for Testing & Evaluation tools?

Paid tiers average around $28/month across 12 tools with published pricing. Several tools only list contact-sales pricing and aren't counted here.

AI Agent Testing & Evaluation

All 20 Testing & Evaluation tools on AgentsIndex. Filter, sort, and compare.

20 listed20 freeavg $28/month

Reviewed by Mathijs Bronsdijk · Updated Apr 19, 2026

The Testing & Evaluation catalog on AgentsIndex lists 20 tools built for this job. Most are built for data scientists, engineers, mid-market teams. 20 have a free option. It's a maturing category with 20 tracked tools.

LangWatch

Open-source LLMOps for testing, evaluating, and monitoring AI agents

Testing & Evaluation

LangWatch is an open-source LLMOps platform for testing, evaluation, and observability of AI agents in development and production.

UpTrain

Open-source LLM evaluation and LLMOps platform with 20+ preconfigured checks

Testing & Evaluation

UpTrain is an open-source LLM evaluation platform with 20+ checks, root cause analysis, regression testing, and local data security. Backed by YCombinator.

Giskard

Open-source testing and validation for AI models

Testing & Evaluation

Giskard helps developers and product teams detect vulnerabilities, biases, and failures in AI models before they reach production.

Athina AI

Build, test, and monitor AI apps together with Athina AI

Testing & Evaluation

Athina AI is a collaborative IDE for building, evaluating, and monitoring production AI applications with observability tools.

WebArena

Reproducible benchmark for evaluating web agents in realistic local sites

Testing & Evaluation

WebArena is a research benchmark for web agents, offering realistic locally hosted websites for controlled, reproducible evaluation.

ToolBench

Open-source platform for training and evaluating LLMs on tool learning tasks

Testing & Evaluation

ToolBench is an open-source benchmark platform for evaluating large language models on API tool-use tasks, with 16,000+ APIs and 12,000+ task instances.

TruLens

Open-source evals and tracing for AI agents and LLM apps

Testing & Evaluation

TruLens is a free, open-source tool for evaluating and tracing AI agents. Measure groundedness, context relevance, and more with Python.

Inspect AI

Evaluate LLMs and agents across coding, reasoning, safety, and task performance

Testing & Evaluation

Inspect AI is an evaluation framework for testing LLMs and agents across safety, reasoning, coding, and agentic tasks.

GAIA Benchmark

Standardized evaluation for general AI assistants

Testing & Evaluation

An academic benchmark that evaluates AI assistants on real-world tasks requiring multi-step reasoning, web browsing, and file manipulation, with verifiable correct answers.

AgentBench

Benchmarking LLMs as real-world agents, not just chatbots

Testing & Evaluation

AgentBench tests how well LLMs act as agents on multi-step tasks, tool use, and long interactions in realistic settings.

SWE-bench

SWE-bench: the standard benchmark for evaluating AI on real software engineering tasks

Testing & Evaluation

SWE-bench tests AI agents on real GitHub issues. Includes Verified, Lite, Multilingual, and Multimodal variants with public leaderboards.

Galileo AI Evaluate

Evaluate and monitor LLM apps in production with observability

Testing & Evaluation

Galileo AI Evaluate helps teams assess, debug, and monitor LLM apps, chatbots, RAG, and copilots in production.

Patronus AI

Evaluate and stress-test LLM agents with judge models and simulators

Testing & Evaluation

Patronus AI helps teams evaluate LLM agents with judge models, AI evaluation tools, and simulators for real-world performance and safety.

Maxim AI

Test, evaluate, and monitor AI agents with Maxim AI

Testing & Evaluation

Maxim AI helps engineering and product teams simulate, evaluate, and monitor AI agents with end-to-end observability.

Promptfoo

Open-source LLM evaluation and red teaming for secure AI development

Testing & Evaluation

Promptfoo is an open-source CLI tool for LLM evaluation, red teaming, and AI security testing. Used by 127 Fortune 500 companies. Free tier available.

RAGAS

Open-source RAG evaluation framework for LLM applications

Testing & Evaluation

RAGAS is a free, open-source Python library for evaluating RAG pipelines and AI agents with automated metrics and synthetic test data.

DeepEval

The open-source LLM evaluation framework

Testing & Evaluation

Open-source LLM evaluation framework with 50+ research-backed metrics for testing hallucination, relevancy, faithfulness, and more. Pytest-style testing for CI/CD pipelines.

Opik

Open-source LLM evaluation, tracing, and monitoring for production apps

Testing & Evaluation

Opik is an open-source LLM evaluation and observability platform for tracing, testing, and monitoring generative AI applications.

Braintrust

AI evals and observability for production AI teams

Testing & Evaluation

Trace, score, and compare AI outputs in production. Braintrust helps teams catch drift, debug prompts, and improve quality.

Hamming AI

Automated testing and monitoring for voice AI agents

Testing & Evaluation

Hamming AI automates testing and monitoring for voice and chat AI agents. Simulate 1,000+ concurrent calls, track 50+ metrics, and catch regressions before production.

Compare Testing & Evaluation Tools

Galileo AI EvaluatevsWebArena

AI Agent Testing & Evaluation

Compare Testing & Evaluation Tools

About Testing & Evaluation on AgentsIndex

How many Testing & Evaluation tools are listed?

What's a typical price for Testing & Evaluation tools?