LLM Guard
What is LLM Guard?
LLM Guard is an open-source LLM security library and API for AI teams that filters prompts and model outputs before they reach the model or leave it. It combines Advanced Input and Output Scanners, Cost Optimized CPU inference, and LLM Model-Agnostic coverage, and it fits Azure OpenAI, Bedrock, and Langchain workflows. Used by Palo Alto Networks, Microsoft, AWS, and Databricks, it has been downloaded over 2.5 million times.
Last verifiedHow we evaluate
At a glance
- LLM Guard is best for AI teams who need lightweight security checks around prompts and outputs.
What does LLM Guard do?
LLM Guard filters prompts and model outputs with advanced input and output scanners that look for data leakage, prompt injection, jailbreaks, adversarial content, and unsafe moderation cases. It can anonymize PII, redact secrets, and apply regex analysis plus URL reachability checks so teams can shape safer LLM interactions before they reach the model or leave it. The project is built for fast adoption: it runs as a library or API and is designed to fit into existing LLM workflows without a heavy setup. At scale, the project has been downloaded over 2.5 million times and is tuned for CPU inference, with 5x lower inference expenses on CPU compared to GPU. It is model-agnostic, so it can sit in front of GPT, Llama, Mistral, or Falcon deployments and work across frameworks such as Azure OpenAI, Bedrock, and Langchain. The page also points to a permissively licensed open-source project, with a commercial version and expanded platform capabilities coming soon.
Why use LLM Guard?
- Its CPU-oriented design can cut inference expenses without forcing teams onto GPU-heavy infrastructure.
- Model-agnostic coverage lets one guardrail layer sit across different LLMs and frameworks instead of being rebuilt per stack.
- Open-source licensing gives teams transparency and room to inspect or adapt the security logic.
Who is LLM Guard for?
- AI/ML developers who need to add prompt and output safety checks quickly.
- Security engineers who want to reduce data leakage and injection risk in LLM apps.
- Platform teams who need a model-agnostic guardrail layer across multiple frameworks.
- Product teams shipping customer-facing AI features that need safer interactions.
What are LLM Guard's key features?
Advanced Input and Output Scanners
Scans prompts and model outputs for security issues before they reach users, helping teams catch risky content early across LLM workflows.
Cost Optimized
Runs with 5x lower inference expenses on CPU compared to GPU, reducing infrastructure spend for teams deploying LLM security checks at scale.
Low Latency, High Accuracy
Keeps scanning fast while maintaining detection quality, so applications can filter LLM traffic without adding noticeable delay to requests.
Open-Source
Available as open-source software and downloaded 2.5 million times, giving teams a transparent codebase they can inspect, extend, and self-manage.
LLM Security
Adds security controls for LLM applications, helping protect against prompt and output risks in production environments used by teams like Palo Alto Networks and Microsoft.
LLM Model-Agnostic
Works across Azure OpenAI, Bedrock, GPT, Llama, Mistral, Falcon, and Langchain, so teams can standardize security across different model stacks.
Easy to Deploy
Fits into existing deployments with support for Azure OpenAI, Bedrock, and Langchain, making it easier to add scanning without rebuilding application architecture.
What does LLM Guard integrate with?
- Azure OpenAI
- Bedrock
- Langchain
- GPT
- Llama
- Mistral
- Falcon
What are LLM Guard's use cases?
Prompt safety for AI builders
AI/ML developers who need to add prompt and output safety checks quickly use LLM Guard to screen user inputs before they reach the model and inspect responses before they ship. They rely on Advanced Input and Output Scanners and Low Latency, High Accuracy to catch risky content without slowing the app.
Leakage control for security teams
Security engineers use LLM Guard to reduce data leakage and injection risk in LLM apps by placing a guardrail layer in front of model calls. With LLM Security and LLM Model-Agnostic, they can enforce consistent protections across different stacks and model providers.
Guardrails across platform stacks
Platform teams use LLM Guard to standardize safety checks across multiple frameworks and model choices. They combine Easy to Deploy with Open-Source to roll out a reusable guardrail layer that fits existing workflows and keeps integration overhead low.
Safer customer AI features
Product teams shipping customer-facing AI features use LLM Guard to keep conversations safer and more reliable. They use Advanced Input and Output Scanners and Cost Optimized to protect users while controlling runtime costs as usage scales.
How does LLM Guard work?
- Connect your first LLM entry point, such as Azure OpenAI, Bedrock, Langchain, GPT, Llama, Mistral, or Falcon, so LLM Guard can inspect traffic before and after generation.
- Turn on Advanced Input and Output Scanners to catch prompt injection, unsafe requests, and risky model responses at the boundary of your application.
- Tune LLM Security policies to match your risk tolerance, then keep the guardrail layer model-agnostic so the same checks work across frameworks and providers.
- Deploy with Easy to Deploy in your existing stack, using the Open-Source package to move from prototype to production without a heavy integration project.
- Monitor performance and cost with Low Latency, High Accuracy and Cost Optimized settings, so safety checks stay fast while keeping inference expenses under control.
Frequently asked questions
What is LLM Guard?
LLM Guard is an open-source LLM security library and API for AI teams that filters prompts and model outputs before they reach the model or leave it. It combines Advanced Input and Output Scanners, Cost Optimized CPU inference, and LLM Model-Agnostic coverage, and it fits Azure OpenAI, Bedrock, and Langchain workflows. Used by Palo Alto Networks, Microsoft, AWS, and Databricks, it has been downloaded over 2.5 million times.
What is LLM Guard used for? Who is it for?
LLM Guard is used for Advanced Input and Output Scanners, Cost Optimized, and Low Latency, High Accuracy. It's built for AI/ML developers, Security engineers, and Platform teams.
Does LLM Guard have an API and what does it integrate with?
LLM Guard doesn't publish a public API. It integrates with Azure OpenAI, Bedrock, Langchain, GPT, Llama, and 2 more.
Editor's read
Check whether your deployment needs the commercial version or the open-source library is enough. Expanded platform capabilities are coming soon, so verify the current feature set before planning around future releases.
Explore other Agent Tools & Integrations
