Skip to main content

LLM Guard

Large Language Model (LLM) Guard is an open-source security and safety toolkit for LLM applications, focused on prompt and response inspection, filtering, and policy enforcement across the LLM lifecycle (application security / Artificial Intelligence (AI) security).

  • Content and prompt inspection for LLM inputs and outputs (application security)
  • Detection and filtering of prompt injection, sensitive data, and unsafe content (AI security / Data Loss Prevention (DLP))
  • Configurable policies and validators for controlling LLM behavior (policy enforcement)
  • Middleware-style integration into LLM applications and Application Programming Interface (API) request flows (application integration)
  • Support for risk management and governance of LLM usage in organizations (AI governance)

More About LLM Guard

LLM Guard is an open-source toolkit maintained by Protect AI that targets security, safety, and governance controls for applications built on large language models. The project addresses risks such as prompt injection, data exfiltration, policy non-compliance, and exposure to unsafe or disallowed content in both prompts and model responses. It is designed for teams that operate LLM-backed applications in environments where security and compliance controls must be explicit, auditable, and configurable.

The core capability of LLM Guard is content inspection and validation for LLM traffic (AI security). It exposes a set of validators and filters that evaluate prompts and responses for categories such as prompt injection attempts, Personally Identifiable Information (PII), secrets, toxicity, and other unsafe or out-of-policy content, based on configuration. These validators can act on both user inputs before they reach an LLM and on the LLM outputs before they are returned to end users or downstream systems.

LLM Guard is typically used as a middleware or policy-enforcement layer embedded in LLM application stacks (application integration). It can sit between an application and one or more LLM providers, intercepting API calls and applying configured checks. This pattern allows enterprise teams to standardize security and content policies across heterogeneous models and vendors. The project aligns with broader Model Risk Management (MRM) practices by giving organizations a configurable control plane over what LLMs are allowed to see and produce.

From an architectural perspective, LLM Guard operates in the domain of AI application security and data protection, and can be combined with model routing, observability, or access control components in a larger Machine Learning Operations (MLOps) or LLMOps stack (MLOps / LLMOps). Its configuration-driven approach supports integration into Continuous Integration and Continuous Deployment (CI/CD) pipelines and deployment automation, enabling Policy as Code (PaC) workflows around LLM usage. The project sits alongside other Protect AI offerings that address model supply chain and runtime security, positioning LLM Guard as the layer focused specifically on prompt and response security for LLM-powered applications.