Incident sheet · /security
- System
- Security & Risk
- Filed
- Read
- 5 min · 936 words
- Severity
- Routine
CISOs Can't Assess AI Vendors With Standard Questionnaires Alone
Standard questionnaires miss five AI-specific risk gaps. A new CISO framework adds three risk tiers, six review steps and explicit approval conditions for AI deployments.

Event log
- The framework defines three risk tiers, with Tier 3 requiring specialist review and executive risk acceptance
- Traditional assessments leave five AI-specific gaps: data use, model behavior, dependencies, change over time, and control
- The six-step process covers use definition, evidence evaluation, deployment testing, monitoring, decision labels and accountability
- Referenced anchors include NIST AI RMF, NIST SP 800-161 Rev. 1 and ISO/IEC 42001
- Assessment is deployment-specific: the same vendor can fall into different tiers for different use cases
A conventional security questionnaire can no longer establish whether an AI vendor is safe to deploy — the decision CISOs actually need to make is whether a specific AI deployment, used for a defined business purpose, presents acceptable residual risk under enforceable controls. That's the core argument of a new framework for AI vendor risk assessment aimed squarely at enterprise security leaders.
The guidance is blunt about why existing due diligence falls short. Traditional reviews cover cybersecurity controls, access management, certifications, incident response, business continuity and data protection. Those practices aren't obsolete — but they leave five AI-specific gaps that questionnaires rarely touch:
- Data use. Are prompts, uploaded files, outputs or customer data retained or used for model training?
- Model behavior. Can outputs be inaccurate, manipulated, biased or unreliable in consequential workflows?
- Dependencies. Which foundation models, subprocesses, data sets and external services sit underneath the product?
- Change over time. Can model updates materially shift behavior, security exposure or performance after approval?
- Control and accountability. Can the enterprise audit use, constrain access, investigate incidents and exit the service?
A vendor's claims help, the framework notes, but they don't independently establish that its AI capability fits every use case.
What belongs in scope?
The review should cover more than branded AI products. The guidance calls for assessing AI-enabled features in operational, customer-facing or security tools; contractors and MSPs that use AI to process enterprise data; standalone generative AI assistants and AI-as-a-service platforms; SaaS with embedded AI features; and external models, APIs, agents and AI development tools. It also flags unsanctioned shadow AI adoption that exposes enterprise data, identities or systems.
Not everything needs the full treatment. Initial screening identifies exposure, then escalates by risk. The assessment is deployment-specific, not a judgment on a vendor's entire catalog. The framework's example: an AI meeting summarizer handling routine internal notes warrants a lighter review than an AI agent that queries production systems or executes privileged actions.
How do the risk tiers work?
Tier assignment rests on checklists covering data sensitivity, access, autonomy, potential harm, business criticality, regulatory exposure and reversibility. The same vendor can land in different tiers depending on the product's capabilities and intended use.
- Tier 1 — Low risk. Standard review: limited data sensitivity, no privileged access, low-impact outputs. Apply baseline vendor security, privacy and data-use checks.
- Tier 2 — Moderate risk. Enhanced review: confidential business data, significant workflow dependencies, or outputs requiring reliability and human oversight. Assess model governance, logging, contractual controls and recovery options.
- Tier 3 — High risk. Formal risk approval: sensitive regulated data, privileged agents, material customer impact, or AI-influenced decisions with significant financial, legal, safety or operational consequences. Requires specialist review, documented mitigations and executive risk acceptance.
What does the review process look like?
The framework lays out six steps, integrated with procurement and existing third-party risk management workflows.
1. Define intended use and exposure. Document the business goal, users, deployment scope, data categories and system permissions. Identify integrations, external model providers and dependencies. Set the baseline risk tier before requesting more evidence.
2. Evaluate AI-specific controls and vendor evidence. Five assessment areas organize the review: data governance and privacy (retention, model-training use, residency, deletion, ownership, prompt handling); security and access (identity controls, tenant isolation, encryption, agent permissions, prompt injection protections); model reliability and oversight (failure-mode testing, output validation, human review); transparency and accountability (documented model dependencies, version changes, audit logs, incident notification); and resilience and contractual protections (continuity, fallbacks, audit rights, data portability, exit provisions). Critically, distinguish verified evidence from vendor assurances.
3. Test controls in the intended deployment. For high-risk use cases, evaluate critical claims with a pilot, configuration review or targeted testing. Test the actual integration and permissions, not a vendor demo. Record findings, accountable owners, remediation deadlines and residual risks.
4. Establish ongoing monitoring. Risk assessment isn't one-time in an era of rapid AI evolution. Reassess when model versions, data practices, permissions or intended uses change. Track incidents, control failures, vendor changes and emerging dependencies. Define a practical exit or rollback plan.
5. Approve, restrict, defer or reject. Approve when evidence is sufficient and required controls are operational. Approve with restrictions that document and limit data classes, user groups, integrations, agent permissions or use cases. Defer when critical evidence is missing or remediation is incomplete — with an owner and a deadline. Reject when material risk can't be mitigated, the vendor can't meet mandatory obligations, or residual exposure exceeds risk appetite.
6. Make accountability explicit. Record the decision, rationale, conditions, review date and exit criteria for each use case. Executives and delegated risk authorities accept residual risk; security and privacy teams manage access controls; legal and compliance interpret obligations; procurement handles commercial terms; business-unit owners justify the use case and operational safeguards.
Which frameworks anchor this?
The guidance points to NIST's AI Risk Management Framework, including its GenAI profile covering supplier and lifecycle considerations; NIST SP 800-161 Rev. 1 on cybersecurity supply chain risk management; ISO/IEC 42001 on organizational AI governance; and existing GRC and third-party risk platforms, supplemented with AI-specific questionnaires and controls. These supplement — not replace — use-case analysis, contractual review and legal advice.
The starting prescription is deliberately small: review the current vendor inventory, identify AI-enabled capabilities, pick one high-impact use case, and document its risk tier and approval criteria. Then scale. AI vendor risk, in the end, comes down to three things — what the deployment can access, what actions it can take, and what happens when it fails.
via techtarget.com (Source)
More from Olivia Hart
Show full bio
News editor covering money, funding and market shifts around enterprise information technology and the leaders who run at CIO Monitor.
207 articles
The newsroom digest
One email a day with the stories that matter. No noise.
Cross-references
- AI-Enabled Sanction Evasion Becomes New IT Governance Headache
- AI Agents Need a Judgment Line: Where Enterprises Draw It
- The Stakes Rise for the CIO Role in 2026, InformationWeek Flags
- Red Hat Product Chief: AI Harnesses Can Close Enterprise Software Security Gap
- Critical Infrastructure Supplier Discloses Cyberattack