Arize Phoenix

What is Arize Phoenix?

Arize Phoenix is an open source platform for tracing, evaluating, and optimizing LLM applications during development. It instruments AI applications with OpenTelemetry and traces spans across LLM workflows, agentic systems, and retrieval pipelines for real-time visibility. The platform also includes an evaluation library with pre-built templates for scoring outputs, detecting regressions, adding human feedback, and analyzing semantic similarity with embeddings. It is built for AI engineers, data scientists, and ML teams working on LLM development and pre-production iteration.

Key Features

• Tracing: Arize Phoenix captures and visualizes traces with OpenTelemetry and OpenInference, so teams can inspect spans, status codes, latency, errors, and agent run trajectories in one place.
• Evaluation: Evaluation uses the arize-phoenix-evals library with Python and Typescript SDKs, structured inputs, pre-built templates, and human feedback support, which helps teams assess LLM behavior beyond simple string matching.
• Experimentation: Experimentation runs real-time tests in Playground and updates cost and latency metrics about every 2 seconds, so prompt and model changes can be compared without waiting for full completions.
• Dataset Management: Dataset Management supports drag-and-drop CSV and JSONL uploads, streaming parsing, RFC 4180 support, and column mapping for input, output, and metadata, which reduces manual work during dataset setup.
• Prompt Iteration: Prompt Iteration tracks prompt changes with side-by-side version diffs across chat templates, tool calls, and results, so teams can review what changed before they compare outputs.
• Dataset Clustering & Visualization: Dataset Clustering & Visualization groups semantically similar questions, documents, and responses with embeddings, which helps isolate patterns and problem areas in evaluation data.
• Span Filters: Span Filters let users query traces by name, kind, and status code through the REST API with sorting and pagination, so Arize Phoenix data can feed custom analysis workflows.
• Multiple Deployment Options: Multiple Deployment Options include local phoenix serve, Docker, on-demand cloud instances, and self-hosting with no feature gates, which gives teams a choice of setup for development and production.

Use Cases

• ML Platform Engineer: Uses Arize Phoenix for real-time observability on Priceline's Penny travel booking agent, tracking model output shifts and catching issues before users hit failures. Priceline used it while moving from push-to-talk to streaming audio conversations, and Penny now supports contextual awareness and multi-step booking flows with faster, more natural interactions, improved conversion rates, and a broader feature set while reliability was maintained.

• AI Operations Lead: Uses Arize Phoenix for end-to-end observability for AI agents in production at PagerDuty.

Strengths and Weaknesses

Strengths:

• G2 lists Arize Phoenix with a 4.2 rating. The research notes cross platform discrepancies alongside that rating.

Weaknesses:

Pricing

• Phoenix: Free & open source. Self-hosted and fully user-managed, with user-managed trace spans, ingestion volume, projects, and retention. No enforced limits are documented.
• AX Free: Free. Includes everything in Phoenix, plus Alyx (Arize agent), online evals, product observability (monitors and custom metrics), and community support. Limits are 25k spans per month, 1 GB ingestion per month, and 15 days retention.
• AX Pro: $50/month. Includes everything in AX Free, plus higher rate limits, 30 days retention, and email support. Limits are 50k spans per month, with additional spans at $10 per million, and 10 GB ingestion per month, with additional ingestion at $3 per GB.
• AX Enterprise: Custom. Includes everything in AX Pro, plus dedicated support, an uptime SLA, SOC2 reports and HIPAA, training sessions, and adb Data Fabric. Spans, ingestion, projects, and retention are custom.

Startup discount programs are listed. AX Enterprise is available through contact sales.

Who Is It For?

Ideal for:

• ML or LLM platform engineer at a mid-market or enterprise AI company: Arize Phoenix fits teams that need LLM observability in self-managed infrastructure. It brings production traces, evaluation results, and performance metrics into one system without vendor lock-in.
• AI product team shipping agents to production: It suits small to mid-market teams that want traces, evals, and experimentation in one place. That setup can reduce tool switching between development and production.
• Data scientist or ML researcher building evaluation frameworks: Phoenix works for solo users and small teams that want open-source evaluation, benchmarking, and data curation tools. It can help standardize evals without waiting on a platform team.

Not ideal for:

• Product managers or non-technical stakeholders who want one-click dashboards and instant insights: Phoenix is engineer-centric and requires instrumentation, trace interpretation, and eval setup, so tools like Braintrust or LangSmith may fit better.
• Organizations without DevOps capacity or that prefer fully managed infrastructure: Phoenix is built around self-managed infrastructure and OpenTelemetry control, so Datadog or Braintrust may be a better match.

Arize Phoenix is best for ML and DevOps-led teams with platform engineering capacity, especially growth-stage teams of 3 to 15 people shipping production agents. It fits regulated settings such as fintech and healthcare, and teams working with stacks that include LangChain, Haystack, OpenTelemetry, Datadog, Pinecone, Weaviate, or the Vercel AI SDK. Use it if you need vendor-agnostic observability and infrastructure control, skip it if you want a SaaS-first setup with low engineering overhead.

Alternatives and Comparisons

• Langfuse: Arize Phoenix does OpenTelemetry-native tracing and open-source evaluation metrics better, and it supports free self-hosting without usage-based billing limits on spans. Langfuse does developer-first setup, unlimited users, and fully open-source self-hosting without OpenTelemetry requirements better. Choose Arize Phoenix if your team needs OpenTelemetry compliance or its open-source eval library; choose Langfuse if you want lower-friction setup and more open-source flexibility. Switching difficulty from Langfuse is medium.

• LangSmith: Arize Phoenix does multi-provider support outside LangChain and OpenTelemetry-native tracing for non-LangChain workflows better. LangSmith does native LangChain integration and visual debugging built around LangChain traces better. Choose Arize Phoenix if you use several LLM providers or rely on OpenTelemetry; choose LangSmith if your stack is closely tied to LangChain.

• Braintrust: Arize Phoenix does open-source self-hosting and OpenTelemetry-native observability better, and it includes a strong open-source eval metrics library. Braintrust does evaluation workflows for production better, with CI/CD integration, deployment blocking, and A/B testing. Choose Arize Phoenix if you want open-source OTel tracing with less vendor lock; choose Braintrust if your main need is experiment management tied to release workflows.

Getting Started

Setup:

• Signup: Phoenix can be launched locally, and the first steps are a local launch plus simple tracing code.
• Time to first result: Public setup data points to 5 to 15 minutes for a first useful result.

Learning curve:

• Initial tracing is accessible, but the overall learning curve is steep for Arize tools. Python proficiency is the main background needed.
• Beginner: 1 to 4 hours for basic tracing, and weeks for full proficiency. Experienced: under 1 hour for integration, and 4 to 8 weeks for production scale.

Where to get help:

• Official learning material includes at least one public tutorial video, and sample templates are available for first use.
• The Arize AI Community forum appears low traffic, and public reports describe limited activity and low responsiveness.
• Community support looks small and stagnant, and public observations say questions are mostly unanswered. Third party tutorials exist, but recent ecosystem growth is not well supported by the cited sources.

Watch out for:

• Instrumentation and evaluation setup can take time to learn.
• Tracing configuration can be frustrating in more complex LLM apps.

Developer Experience

Arize Phoenix is an open-source observability and evaluation framework for LLM and RAG applications. Developers mainly use it through the Python SDK and integrations, with the pip-installable arize-phoenix library embedded in application code to instrument LLM calls, trace execution, and run offline evals. Public feedback says teams can get a basic instrumented app running in 15 to 45 minutes if they already know Python and their stack, and the docs are thorough on core concepts and setup even though examples sometimes lag behind API changes.

What developers like:

• Developers often praise that Phoenix is open source and self-hostable.
• Public feedback highlights the visualization and user interface, along with its practical focus on LLM-specific concerns.
• Reviewers also point to the evals framework, active GitHub responsiveness, and quick setup for proof-of-concept work.

Common frustrations:

• Developers report version churn and breaking changes.
• Public feedback notes limited non-Python support and some integration friction with newer frameworks.
• Community comments also mention incomplete error messages, unclear async-first guidance, and reproducibility issues in offline evals.

Security and Privacy

• Data residency: The vendor states that EU data residency is available. (trust center)
• Data training: The vendor states it does not train on user data. (trust center)
• SOC 2: SOC 2 Type 2 is listed on the vendor's trust center. (trust center)
• ISO 27001: ISO 27001 is listed on the vendor's trust center. (trust center)
• HIPAA and PCI DSS: The vendor lists HIPAA compliance and PCI DSS on its trust center. (trust center)
• Other certifications: The vendor lists GDPR, CSA Star Level 1, and Clone Guard Certified on its trust center. (trust center)

Product Momentum

• Search interest: Google Trends shows unknown direction, with +0.0% change across the measured period. The latest interest score is 0/100, and the peak score is also 0/100.
• Risks: No notable risks appear in the research data.

FAQ

What does Arize Phoenix do?

Arize Phoenix is an open-source LLM tracing and evaluation platform. It uses OpenTelemetry to instrument AI applications and shows traces, spans, inputs, outputs, token usage, latency, and model parameters in real time.

Is Phoenix ARIZE free?

Yes. Arize Phoenix is free and open source, with self-hosting through Docker or pip, and a free Phoenix Cloud tier is available with an API key for hosted tracing.

Is Arize Phoenix open source?

Yes. Public sources describe Phoenix as fully open source, with no feature gates or restrictions.

Can Arize Phoenix be self-hosted?

Yes. Public documentation lists self-hosting options through Docker and pip installation.

What can you trace in Arize Phoenix?

Phoenix visualizes LLM request traces and spans. Public sources mention inputs, outputs, token usage, latency, model parameters, span name, kind, and status code.

Does Arize Phoenix use OpenTelemetry?

Yes. Phoenix is built around OpenTelemetry for tracing, and public sources also mention OpenInference standards.

Does Arize Phoenix support evaluations?

Yes. Public sources describe Phoenix as an evaluation platform in addition to tracing. They also mention dataset clustering and a library of evaluation metrics.

What is Arize Phoenix used for?

Phoenix is used to observe, evaluate, and optimize AI applications. Public sources position it for teams running production AI agents, including multi-agent systems and teams with OpenTelemetry requirements.

How long does it take to get started with Arize Phoenix?

The documented time to first result is 5 to 15 minutes. Public sources describe the first steps as a local launch and simple tracing code.

What is the difference between Arize Phoenix and Logfire?

Phoenix is an open-source, self-hostable LLM tracing platform built on OpenTelemetry, with evaluations and dataset clustering. Public sources say Logfire focuses on general observability for Python apps and does not include Phoenix's LLM-specific eval libraries and trace UI.

Does Arize Phoenix have a cloud option?

Yes. Public sources mention a free Phoenix Cloud tier for hosted tracing, alongside the self-hosted open-source version.

What pricing tiers are listed for Arize Phoenix?

Public pricing data lists Phoenix as free and open source for self-hosting. It also lists AX Free as a SaaS option with 25k spans and a 1 GB limit.

Does Arize Phoenix integrate with other AI tools?

Yes. Public sources mention integrations with tools such as Haystack and LlamaIndex.

Which AI is fully free?

In the provided sources, Arize Phoenix is described as fully free and open source. Public sources also state that it has no paid feature restrictions for the self-hosted product.