Portkey

What does Portkey do?

Portkey routes LLM traffic through a unified control plane that adds observability, guardrails, prompt management, and key management around each request. Teams can send requests through the AI Gateway, inspect logs and traces, apply deterministic or AI guardrails, and manage prompts without wiring separate tools into every app. The platform is built for production workflows, with real-time monitoring and routing controls that help teams catch anomalies early and keep model behavior consistent. At scale, Portkey says it supports 3,000+ LLMs, powers 3,000+ GenAI teams, and processes 2Tn+ tokens daily. Its observability layer records every request and response with 40+ details and 40+ metrics, while guardrails cover 60+ checks for issues like prompt injections and PII leaks. Customers named on the site include Ario, Haptik, and Qoala, and the gateway advertises a unified API for 1,600+ LLMs and providers with provider-specific capabilities through the gateway.

Why use Portkey?

• A unified API lets teams connect to many providers without rewriting application code for each model.
• Built-in observability and tracing make it easier to debug failures, cost spikes, and response quality issues in production.
• Guardrails can block injections, redact sensitive data, and route risky requests before they reach the model.
• Caching, retries, and load balancing help teams improve reliability and reduce repeated inference costs.
• Self-hosting support gives organizations a deployment path when they need tighter infrastructure control.

Who is Portkey for?

• Platform engineers who need one control plane for model access, routing, and governance.
• ML teams who want request-level visibility into cost, latency, traces, and feedback.
• Security and compliance teams who need guardrails, redaction, and access controls around AI usage.
• Product teams shipping LLM features who need prompt management and reliable production behavior.
• MLOps teams who want to standardize model integrations across multiple apps and environments.

What are Portkey's key features?

What does Portkey integrate with?

• Microsoft Azure
• MongoDB
• GitHub
• Docker
• Auth0
• Figma
• Cloudflare
• OpenAI Agents
• Cline
• Jan
• Anything LLM
• Zed
• Pangea
• Tooljet
• F5
• Qdrant
• Roo Code
• Goose
• Milvus
• Appsmith
• JACB.AI
• Qualifire
• Patronus AI
• Aporia
• Pillar
• Vercel
• Langchain
• LlamaIndex
• Promptfoo
• CrewAI

What are Portkey's use cases?

How does Portkey work?

1. Connect your first model provider in AI Gateway and point traffic through Portkey's unified API. Use Model Catalog to choose approved models and start routing requests without rewriting every app.
2. Turn on Observability to capture Logs and traces, then review the Real-time dashboard for latency, cost, and failure patterns. Add Feedback and Metadata so each request is easier to diagnose.
3. Set Guardrails and PII redaction rules to block prompt injections, prevent leaks, and enforce runtime policy enforcement. Apply Fine-grained access control and Single Sign-On for safer team access.
4. Manage prompts in Prompt with templates, variables, versioning, and Playground testing. Use Canary testing and Conditional Routing to roll out changes gradually and keep production behavior stable.
5. Expand to MCP Gateway or additional apps as usage grows, using Fallbacks, Automatic retries, and Load balancing to maintain uptime. Keep tuning with Analytics, Filters, and usage reports.

How much does Portkey cost?

Frequently asked questions