Evidence Framework

How does the AI Policy relate to existing IT and data governance policies?

The AI Policy should cross-reference and complement existing policies (data governance, information security, privacy, ethics). It does not replace them but adds AI-specific requirements. Where conflicts exist, the more restrictive requirement applies unless the AI Governance Board approves an exception.

How often should the AI Policy be updated?

ISO 42001 requires periodic review. At minimum, conduct a formal review annually and after any significant regulatory change, major AI incident, or material change in AI usage patterns. The policy should be a living document with a clear change management process.

What if employees are already using AI tools not covered by this policy?

Implement a time-limited amnesty and registration period. All existing AI uses should be registered within the AI System Registry within 90 days. After the amnesty period, unauthorized use falls under the enforcement provisions of this policy.

How does this map to ISO 42001 certification requirements?

This policy template addresses ISO 42001 Clause 5.2 (AI Policy) directly. When completed and implemented, it provides the foundational policy document required for ISO 42001 management system certification. The policy must be communicated, available, and reviewed as stated in Clause 5.2.

How do we find AI systems we don't know about?

Combine multiple discovery methods: network traffic analysis for AI API calls, procurement/expense audits for AI tool subscriptions, employee surveys about tool usage, vendor product reviews for embedded AI features, and a self-reporting channel with amnesty provisions. COMPEL's Calibrate stage includes a structured shadow AI discovery protocol.

What counts as an "AI system" for registry purposes?

Use a broad definition aligned with the OECD/EU AI Act: any system that uses machine learning, deep learning, or knowledge-based approaches to generate outputs (predictions, recommendations, decisions, content) that influence environments or decisions. Include third-party AI embedded in vendor products and employee use of public GenAI tools.

How often should the register be reviewed?

The full register should be validated quarterly. Individual system entries should be reviewed according to their risk tier: critical systems monthly, high-risk quarterly, others annually. New systems must be registered before deployment.

Who should be involved in an AI risk assessment?

At minimum: the business owner (understands the use case context), a data scientist (understands the technical implementation), a risk/compliance professional (understands the risk framework), and legal counsel (understands regulatory implications). For high-risk systems, include affected stakeholder representatives and external domain experts.

How is this different from a standard IT risk assessment?

AI risk assessments must address AI-specific risks that IT risk assessments typically miss: bias and fairness, model drift, training data quality, explainability requirements, emergent behaviors, and the unique liability questions around AI-generated outputs. This template adds these dimensions while maintaining compatibility with standard risk frameworks.

How often should risk assessments be refreshed?

Trigger-based: reassess when the system changes materially (new data sources, retrained model, expanded scope, new regulations). Cadence-based: critical systems quarterly, high-risk semi-annually, others annually. Always reassess after an incident involving the system.

What is the relationship between a model card and the AI System Registry?

The AI System Registry is the high-level inventory (what AI systems exist). The model card is the detailed technical documentation for each ML model. The registry entry links to the model card. One registry entry may have multiple model cards if the system uses multiple models.

Who is responsible for maintaining the model card?

The technical owner (typically the data scientist or ML engineer) creates and updates the model card. The business owner reviews it for accuracy of the business context sections. The model card should be updated with every model version change, significant data update, or monitoring threshold breach.

Is a model card required for third-party AI / vendor models?

Yes, but with adaptations. For vendor models where you do not have access to training data or architecture details, document what the vendor has disclosed and note gaps. Your procurement questionnaire (see AI Procurement Questionnaire template) should require vendors to provide model card-equivalent information.

Does every AI system need a human oversight procedure?

Yes, but the depth varies by risk classification. High-risk systems (EU AI Act Article 6) require comprehensive procedures with human-in-the-loop or human-on-the-loop controls. Lower-risk systems may require only monitoring dashboards and periodic reviews. The oversight model should be proportionate to the system's risk level and impact.

How do we ensure human oversight doesn't create bottlenecks?

Use a tiered approach: human-in-the-loop only for the highest-impact decisions, human-on-the-loop for most decisions with exception-based review, and automated monitoring with human review at defined intervals. Design the oversight model around decision volume and acceptable latency, not just risk aversion.

What training do human overseers need?

At minimum: understanding of what the AI system does, how to interpret its outputs, common failure modes, how to exercise override authority, and the escalation procedure. ISO 42001 Clause 7.2 requires competence, and EU AI Act Article 14(4)(a) requires overseers to "fully understand the capacities and limitations of the high-risk AI system."

What if our organization does not have a dedicated AI CoE?

Map CoE responsibilities to the function that currently owns AI governance — this might be IT governance, enterprise risk management, or a cross-functional working group. The RACI template works regardless of organizational structure; the key is that every activity has exactly one Accountable role.

How do we handle RACI conflicts between business units?

When multiple business units use the same AI system, designate one as the primary Accountable owner (typically the unit with the highest-risk use case or the largest user base). Other business units are Responsible for their specific use case compliance. The CoE arbitrates disputes.

Should the RACI be different for different risk tiers?

Yes. Higher risk tiers typically push accountability to higher organizational levels and add more Consulted roles. The template shows this with separate deployment approval rows for Tier 1-2 vs. Tier 3-4. Customize the matrix based on your risk tier definitions.

Can we use this checklist as evidence for ISO 42001 certification?

This checklist is a self-assessment tool to identify gaps and prioritize remediation. It is NOT a substitute for a formal internal audit or third-party certification audit. However, a completed and maintained checklist demonstrates systematic gap analysis, which auditors view favorably. Use it to prepare for formal audits, not to replace them.

What is the typical timeline from gap assessment to certification?

Organizations with mature quality management systems (e.g., existing ISO 9001 or 27001 certification) typically achieve ISO 42001 readiness in 6-12 months. Organizations building from scratch may need 12-18 months. The timeline depends on the number and severity of gaps, available resources, and the scope of AI systems covered.

Which Annex A controls are most commonly failed in audits?

Based on early certification experience, the most challenging areas are: A.5 (comprehensive AI risk assessment beyond standard IT risks), A.6 (formal impact assessment on individuals and society), A.8 (documented human oversight procedures with evidence of operation), and A.10 (data quality assurance with documented provenance). These require operational evidence, not just documentation.

Should every AI vendor complete this questionnaire?

Yes, but proportionate to risk. For low-risk AI tools (e.g., grammar checking), a shortened version covering data practices and security may suffice. For high-risk AI (e.g., credit scoring, healthcare diagnostics), the full questionnaire is essential. The risk tier determines the minimum required sections.

What if a vendor refuses to answer certain questions?

Vendor unwillingness to disclose is itself a risk indicator. Document the refusal and assess whether the information gap creates unacceptable risk. For high-risk applications, inability to verify vendor governance claims may be a disqualifying factor. Consider whether alternative vendors provide greater transparency.

How does this relate to our existing vendor risk management process?

This questionnaire supplements (not replaces) your existing vendor risk management process. It adds AI-specific questions that standard vendor assessments miss. Ideally, integrate these questions into your procurement workflow so they are triggered automatically when a vendor product includes AI capabilities.

How should evidence be organized for an ISO 42001 audit?

Organize evidence by ISO 42001 clause number for certification audits. Maintain a master evidence index that maps each clause/control to the specific evidence location (document management system, SharePoint, or governance platform). This checklist serves as that mapping tool. Auditors appreciate well-organized evidence — it demonstrates management system maturity.

What is the most common audit finding for first-time ISO 42001 audits?

The most common finding is "documented but not implemented" — organizations create policies and procedures but cannot demonstrate they are consistently followed. Evidence of operation (logs, meeting minutes, signed approvals, monitoring outputs) is more important than document quality. A simple process that works beats an elegant process that is only on paper.

How far back should evidence records be retained?

ISO 42001 requires documented information to be controlled but does not specify retention periods. Best practice: retain operational records for at least 3 years (covering two full audit cycles), incident records for 5+ years, and policy version history indefinitely. Align with your organization's records management policy and any regulatory requirements.