Measuring AI Compliance: Control Coverage, Conformity Gaps, and Audit Readiness

FlowRidge

MEASURING AI COMPLIANCE

1

EU AI Act

Risk-tier classificationAnnex III controlsPost-market monitoringTechnical documentation

2

NIST AI RMF

GOVERNMAPMEASUREMANAGE

3

ISO/IEC 42001

AIMS policyLifecycle controlsImpact assessmentContinual improvement

4

ISO/IEC 27001 + COSO

Annex A controlsAccess + loggingInternal controlAudit evidence

Figure 279. Control coverage across the major AI governance frameworks the program must evidence.

Definition

Compliance is the COMPEL Trust & Performance dimension that asks whether the AI program is meeting its obligations under the standards and regulations that apply to it — not at a point in time, but continuously and with evidence. This article is the canonical hub for the Compliance dimension. It defines the three canonical Compliance metrics — , conformity gap count, and audit-readiness score — and routes leaders to the deeper articles that cover specific standards (ISO 42001, NIST AI RMF, EU AI Act). The methodology is grounded in the control-language common to ISO/IEC 27001 Annex A, ISO/IEC 42001, NIST AI RMF, EU AI Act Annex III, and the COSO internal control framework.

Why this dimension matters

The audit-day problem. Organizations that treat compliance as an annual event discover, on audit day, that the evidence they need is scattered across thirty people’s inboxes, that six of the required controls were never actually implemented, and that the control owner for the most important control has left the company. Continuous compliance measurement replaces the audit-day scramble with an always-on posture.

Multi-regulator reality. A modern AI program is simultaneously subject to ISO 42001 (management system), NIST AI RMF (trustworthy AI characteristics), EU AI Act (high-risk obligations), sector regulations (healthcare, finance, HR), and internal policy. A single control often satisfies multiple requirements. The Compliance dimension is the instrument that maps controls to obligations and tracks coverage.

Evidence is the atomic unit. A control without evidence is an assertion. A control with fresh, signed evidence is a defense. Compliance metrics count evidence, not intent.

Core metrics

Metric 1: Control coverage

Definition. The ratio of implemented and evidenced controls to required controls under each applicable standard, expressed as a percentage per standard and as a composite.

Formula. control_coverage[standard] = (implemented_and_evidenced_controls / required_controls) × 100.

Cadence. Monthly.

Owner. Compliance lead, with control owners named per control.

What counts as “implemented and evidenced.” (1) A named owner. (2) A documented procedure. (3) Recent evidence (within the control’s evidence freshness window) showing the procedure ran. (4) No open findings against the control. Missing any one and the control is “partial,” not “implemented.”

Metric 2: Conformity gap count

Definition. The number of open gaps between current state and the requirements of each applicable standard, classified by severity.

Formula. Simple count by severity tier (critical, high, medium, low), per standard, trended monthly.

Cadence. Continuous — gaps open and close on events.

Owner. Compliance lead.

Gap life-cycle. Every gap has an owner, a root cause, a remediation plan, a committed close date, and a current state. Gaps older than their committed close date trend up on the scorecard until closed. An aging gap backlog is the leading indicator of an upcoming audit failure.

Metric 3: Audit-readiness score

Definition. A composite score (0–100) combining control coverage, gap severity profile, evidence freshness, and post-incident action closure into a single number that answers “if an auditor walked in today, how ready are we.”

Formula. audit_readiness = w1 × control_coverage + w2 × (1 − gap_severity_index) + w3 × evidence_freshness + w4 × action_closure_rate, with published weights.

Cadence. Monthly, published on the trust scorecard.

Owner. Compliance lead, reviewed by the audit committee.

Why a composite. Control coverage can be 100% and readiness can still be poor if evidence is stale or post-incident actions are overdue. The composite forces the team to look at the whole posture.

How to measure

Build the control register. Map every control to every standard clause it supports. A single control should satisfy as many clauses as it legitimately can — duplication is cost, not rigor.
Assign owners. Every control has a named owner. Unowned controls become stale controls become failed controls.
Set evidence freshness windows. Some controls need daily evidence (backup success), some quarterly (access reviews), some annual (policy attestation). Publish the window per control.
Instrument automated evidence collection. Wherever the control’s evidence can be pulled from a system of record (log management, ticketing, identity platform), automate it. Manual evidence collection is where compliance programs die.
Run the monthly measurement cycle. Pull coverage, gap, and evidence metrics. Publish to the audit committee. Open remediation tickets for any regression.
Align to standards. Use the deep-dive articles for ISO 42001 and NIST AI RMF (see below) to ensure each control maps correctly to the current version of the standard.

Targets and thresholds

Control coverage. 95% minimum per standard for mature programs; 100% for critical controls in a certification-scope boundary.
Open critical gaps. Zero. Any critical gap is a Sev-1 for the compliance program.
Evidence freshness. 90% of controls within their freshness window.
Audit-readiness score. Above 85 is sustainable; below 75 triggers an executive review.

Common pitfalls

Mapping every control to every standard and calling it coverage. If the control does not actually satisfy the clause, counting it inflates the metric and embarrasses you at the audit.

Evidence theater. Screenshots dated yesterday with no underlying process. Auditors know the difference.

Gap backlog amnesia. Old gaps quietly reclassified as “accepted risk” without an approved risk acceptance record. Auditors find these in minutes.

One standard at a time. Programs that measure ISO 42001 in January and NIST AI RMF in April accumulate contradictions. Measure together against a single control register.

No owner for the register itself. A control register without a maintainer drifts from reality within a quarter.

M4.3ISO 42001 Alignment and AI Management System Certification M4.3NIST AI RMF Implementation at Enterprise Scale M1.5Audit Preparedness and Compliance Operations M1.2Control Requirements Matrix M1.2Control Performance Report