NVIDIA NeMo Guardrails

What is NVIDIA NeMo Guardrails?

NVIDIA NeMo Guardrails is an open-source Python toolkit for adding programmable safety controls to LLM-powered applications and AI agents. It sits between user inputs and model outputs, applying configurable checks for jailbreak prevention, PII detection, topic boundaries, content moderation, and RAG grounding. Developers define guardrail behavior through Colang, a domain-specific language for conversational flows, and YAML configuration files. It is built for engineering teams that need runtime policy enforcement in production LLM deployments. Unlike prompt-level safety instructions, NeMo Guardrails operates as an external control layer that works across different LLM providers.

Key Features

• Input Rails: Validate and filter user messages before they reach the LLM, catching jailbreak attempts, prompt injections, and content policy violations at the entry point
• Output Rails: Post-process LLM responses with fact-checking against knowledge bases, hallucination detection, and sensitive data redaction before delivering results to users
• Dialog Rails: Control multi-turn conversation flow through topic boundaries, branching logic, and canonical forms defined in Colang, keeping interactions within intended scope
• Retrieval Rails: Filter and validate RAG chunks before they reach the model, preventing poisoned or irrelevant context from influencing responses
• Execution Rails: Guard custom actions and tool calls made by agents, adding a policy layer around external API interactions and code execution
• Colang Flow Language: Define conversational guardrails using a readable domain-specific language that specifies allowed patterns, fallback behaviors, and state transitions
• Third-Party Safety Integrations: Connect external moderation services like ActiveFence and CrowdStrike Falcon alongside built-in NVIDIA NemoGuard models for layered protection

Use Cases

• Enterprise chatbot teams: Add topic control and PII redaction to customer-facing LLM assistants so conversations stay within approved domains and sensitive data never surfaces in responses
• RAG application developers: Validate retrieved context chunks before they reach the model, reducing hallucination risk when agents pull from large and noisy knowledge bases
• Compliance-focused AI teams: Enforce regulatory policies at runtime for financial services, healthcare, and automotive deployments where LLM outputs face audit requirements
• Multi-agent system builders: Apply execution rails to agent tool calls so autonomous workflows cannot perform unauthorized actions or access restricted resources

Strengths and Weaknesses

Strengths:

• Colang provides a structured, readable way to define conversation policies without embedding safety logic inside prompts
• Works with multiple LLM providers including OpenAI, Azure, and open-source models, avoiding vendor lock-in on the model layer
• Integrates with LangChain and LlamaIndex, fitting into existing RAG and agent orchestration pipelines
• Open source under Apache 2.0 with backing from NVIDIA, giving teams full visibility into the control logic they deploy
• Benchmarks report 1.4x improved detection rates with roughly 0.5 seconds of added latency when running five guardrails in parallel

Weaknesses:

• Learning Colang adds overhead for teams unfamiliar with domain-specific languages, and documentation examples can be sparse for advanced flow patterns
• Some developers report confusion when integrating with external LLMs like Azure AI, particularly around SDK configuration
• Community support is limited compared to more established safety tools, with public forum posts sometimes going unanswered

Getting Started

Install via pip: pip install nemoguardrails Documentation: docs.nvidia.com/nemo/guardrails License: Apache 2.0 (open source) GitHub: github.com/NVIDIA/NeMo-Guardrails

Setup requires a config.yml file defining your LLM provider and rail configurations. Time to first working guardrail is roughly 15 to 30 minutes for developers with Python experience.

FAQ

Is NVIDIA NeMo Guardrails free?

Yes. NeMo Guardrails is open source under Apache 2.0 and free to use. Enterprise support and NVIDIA AI Enterprise features require separate licensing through NVIDIA sales.

What types of guardrails does NeMo Guardrails support?

NeMo Guardrails supports five types: input rails for filtering user messages, output rails for checking LLM responses, dialog rails for conversation flow control, retrieval rails for RAG validation, and execution rails for guarding agent tool calls.

Does NVIDIA NeMo Guardrails work with LangChain?

Yes. It integrates with both LangChain and LlamaIndex, so teams can add guardrails to existing RAG pipelines and agent chains without rebuilding their orchestration layer.

How much latency does NeMo Guardrails add?

Published benchmarks show roughly 0.5 seconds of added latency when running up to five GPU-accelerated guardrails in parallel. Latency varies based on the number and complexity of active rails.

What is Colang in NeMo Guardrails?

Colang is a domain-specific language for defining conversational flows and guardrail policies. It lets developers specify allowed conversation patterns, topic boundaries, and fallback behaviors in a readable format separate from application code.

How does NeMo Guardrails compare to Guardrails AI?

Guardrails AI focuses on input and output validation with a validators library. NeMo Guardrails is a broader orchestration toolkit built around Colang state machines for multi-turn conversation control. Developers can use Guardrails AI validators within a NeMo Guardrails setup.