This article provides governance professionals with the architecture for building a learning system, not merely an incident register.
Beyond Incident Reporting: The Learning System Concept
Most organisations, when they establish AI ethics incident management, create a reporting mechanism and a response process. An individual reports an incident, an investigation occurs, remediation actions are taken, and the incident is closed. This is necessary but insufficient.
A learning system adds three capabilities that transform incident management into organisational learning:
Pattern recognition across incidents. Individual investigations focus on “what happened here?” Pattern recognition asks “what keeps happening?” — identifying recurring root causes, systemic weaknesses, and organisational conditions that produce incidents across multiple systems and teams.
Near-miss capture. Most organisations only capture incidents that caused actual harm. Near-misses — situations where harm was narrowly avoided — are far more common and equally informative. Aviation safety’s dramatic improvement over decades is largely attributed to systematic near-miss reporting and analysis.
Feed-forward to governance design. Learning system outputs feed forward into governance programme design: updated risk assessments, revised policies, improved training, and modified quality gates. The system is not just reactive — it proactively strengthens governance based on what incidents reveal about systemic weakness.
Architecture of an AI Ethics Incident Learning System
Component 1: Incident Taxonomy
A standardised taxonomy enables consistent classification and cross-incident analysis. The COMPEL taxonomy defines eight categories: discriminatory output, privacy violation, safety failure, manipulation and deception, autonomy undermining, environmental harm, accountability failure, and systemic/emergent harm.
Each category includes: a description, concrete examples, a root cause taxonomy (the structured set of causes typically associated with that category), and prevention measures. The taxonomy should be treated as a living document — new categories may emerge as AI technology evolves.
Component 2: Reporting Mechanism
The reporting mechanism must be: accessible (any employee should be able to report), safe (reporters should not face retaliation for good-faith reporting), structured (using the incident template to capture consistent data), and timely (with acknowledgment within defined timeframes based on severity).
A critical design decision is whether to allow anonymous reporting. Anonymous reporting increases the volume and honesty of reports but makes follow-up investigation more difficult. Best practice is to allow anonymous reporting for initial capture but encourage identified reporting for investigated cases, with explicit protection against retaliation.
The near-miss channel. Separate from the incident reporting channel, establish a near-miss reporting channel with even lower friction. Near-miss reports do not trigger formal investigation — they feed into pattern analysis. The goal is volume: the more near-misses captured, the richer the learning.
Component 3: Severity Assessment and Triage
When an incident is reported, it must be quickly assessed for severity. The COMPEL severity scale operates on five levels:
- Level 1 (Negligible): No measurable harm, self-correcting. Response within 5 business days.
- Level 2 (Minor): Limited harm, fewer than 10 individuals, temporary and reversible. Response within 2 business days.
- Level 3 (Moderate): 10–1,000 individuals affected, disproportionate impact on protected group possible. Response within 24 hours.
- Level 4 (Severe): More than 1,000 individuals, significant and potentially irreversible harm. Immediate escalation, response within 4 hours.
- Level 5 (Critical): Physical harm, mass-scale impact, fundamental rights violation. Immediate system shutdown if safety-critical, CEO and board notification.
Triage should be performed by the governance team, not by the system’s own team, to avoid conflicts of interest in severity assessment.
Component 4: Investigation Process
For severity 3 and above, a formal investigation should be conducted using a structured methodology:
Timeline reconstruction. Establish a factual chronology: when did the issue begin, when was it detected, what actions were taken, and what was the outcome?
Root cause analysis. Move beyond the proximate cause (“the model was biased”) to the root causes (“the training data under-represented elderly users,” “the fairness testing protocol did not include subgroup evaluation,” “the governance review was bypassed due to executive urgency”). Use techniques from safety investigation: the “Five Whys,” fault tree analysis, or the Swiss Cheese Model (how multiple safeguards each had a hole that aligned to let the incident through).
Systemic factor identification. Ask: what organisational conditions enabled this incident? Was it a resource gap, a process gap, a cultural norm, a governance authority weakness, or an incentive misalignment?
Affected individual assessment. Document who was harmed, how they were harmed, and what remediation or compensation has been provided.
Component 5: Pattern Analysis
This is where the learning system distinguishes itself from incident management. Periodically — quarterly for most organisations — analyse the incident and near-miss registers for patterns:
Recurring root causes. If “insufficient subgroup testing” appears as a root cause across multiple incidents involving different teams and systems, the organisation has a systemic testing capability gap, not a collection of isolated incidents.
Team and system concentrations. Some teams or systems may produce disproportionate numbers of incidents. This may indicate inadequate governance capability, insufficient resources, or cultural norms that deprioritise ethical considerations.
Temporal patterns. Do incidents cluster around release deadlines, organisational changes, or regulatory transitions? This reveals systemic stress points.
Severity trends. Are incidents becoming more or less severe over time? Is the organisation catching issues earlier (more near-misses, fewer high-severity incidents), or is detection capability degrading?
Component 6: Learning Outputs
Pattern analysis produces learning outputs that feed forward into governance:
Governance programme updates. If pattern analysis reveals that fairness testing is consistently insufficient, the governance programme should mandate enhanced fairness testing — not as a reaction to a single incident but as a systemic improvement based on evidence.
Training and awareness. Anonymised incident case studies are powerful training materials. They make ethical risks concrete and demonstrate that “it can happen here.”
Risk assessment enrichment. Incident patterns should inform the organisation’s AI risk assessment methodology. If the learning system reveals that certain system types, data types, or deployment contexts are associated with higher incident rates, risk assessments should weight these factors accordingly.
Policy revision. When incidents reveal that existing policies are insufficient, ambiguous, or routinely bypassed, the policies should be revised — not merely reinforced.
Component 7: External Learning
The organisation’s own incident register represents a small sample of the possible AI ethics failures. External sources dramatically expand the learning base:
AI Incident Database (AIID). A public repository of AI-related incidents from around the world. Regular review of the AIID for incidents relevant to the organisation’s AI portfolio enriches risk awareness.
Regulatory enforcement actions. Enforcement actions by data protection authorities, sector regulators, and AI-specific bodies provide authoritative examples of what regulators consider unacceptable.
Academic and civil society research. Organisations like Algorithm Watch, AI Now Institute, and academic AI ethics research groups publish analyses of AI harms that may not appear in formal incident databases.
Industry peer learning. Where possible, participate in industry forums that share anonymised incident learnings — similar to the Aviation Safety Reporting System (ASRS) model.
Measuring Learning System Effectiveness
A learning system’s effectiveness is not measured by the number of incidents reported — paradoxically, an increase in reporting may indicate a healthier culture, not a deteriorating system. Key effectiveness metrics include:
- Reporting rate: Are near-misses being captured or only actual incidents?
- Pattern detection rate: How many systemic patterns has the learning system identified?
- Feed-forward implementation: Of the systemic improvements recommended by pattern analysis, how many have been implemented?
- Recurrence rate: Are the same root causes producing incidents repeatedly, or is the organisation learning and preventing recurrence?
- Detection time: Is the average time from incident occurrence to detection decreasing?
- Reporter safety: Do reporters feel safe? (Measured through annual survey.)
The ultimate measure is whether the organisation’s AI ethics incident rate is trending downward — not because reporting is suppressed but because the learning system is genuinely preventing failures.
This article is part of the COMPEL Body of Knowledge v2.5 and supports the AI Transformation Governance Professional (AITGP) certification.