NSS Labs outlines AIPS test methodology and evaluation papers for runtime guardrails
NSS Labs published two white papers and a new test methodology for AI Protections System (AIPS) products, emphasizing evaluation of protections around deployed models, including runtime guardrails, auditability, and resilience under adversarial conditions.
Research Overview
NSS Labs reports that enterprises are adopting AI features such as copilots and autonomous agents while lacking consistent ways to verify that purchased AI security controls work in practice. The organization links this gap to an evaluation approach that it says has not kept pace with how AI systems are deployed.
To address that mismatch, NSS Labs released a two-part research series focused on what “good” AI security evaluation should look like for enterprises and how buyers can assess it during product evaluation.
Key Findings
In the first white paper, “AI Security Beyond the Model,” NSS Labs says the model is only part of the risk picture. It frames the risk as extending to surrounding elements such as the data the system can access, instructions it can receive or be manipulated with, tools it can call, and inherited permissions.
In the second white paper, “Evaluating Enterprise AI Security,” NSS Labs presents questions it says buyers should be able to answer during evaluations, along with warning indicators and criteria meant to distinguish validated controls from untested claims.
Technical Breakdown
A central theme in the research is runtime guardrails—controls that sit around a deployed model and determine behavior in production. NSS Labs describes these mechanisms as enforcing policy, limiting access, constraining agent behavior, and producing evidence through observability and audit trails.
NSS Labs also describes its assessment gap as a matter of comparability, saying that without clearer expectations buyers may compare products using non-equivalent evidence and vendors may lack a consistent way to demonstrate capabilities.
Product Update
NSS Labs states it has been working with major players in the AIPS space to define a new and comprehensive test methodology. The organization says the methodology applies its structured testing approach to the AIPS market.
In the methodology, NSS Labs describes evaluation coverage across protection and operational dimensions including defense against prompt injection, prevention of harmful or unauthorized output, evasion, resilience under stress and adverse conditions, policy and filter efficacy, security of agent behavior and tool invocation, observability and auditability, and performance impact.
Operational Impact
NSS Labs says each test dimension is intended to represent enterprise risks when AI systems connect to users, enterprise data, tools, APIs, and business processes. The goal is described as providing enterprise buyers, security leaders, and vendors with a repeatable, technically rigorous basis to measure performance under realistic use and abuse scenarios.
The article states that after testing later this year, final reports will be published with results on how security vendors are addressing these problems.
Overall, NSS Labs frames the update as a shift toward evaluating AI security controls that govern runtime behavior, with evaluation guidance in two white papers and a new AIPS test and validation methodology. This “Blog Signals” brief is a fact-based summary of the vendor blog.