Performance Evaluation in AI-Integrated Work

COMPEL Specialization — AITE-WCT: AI Workforce Transformation Expert Article 29 of 35

Performance evaluation is the artefact of the workforce-management system that AI integration most consistently breaks. The pre-AI evaluation design assumed that the output an employee produced — the memo written, the decision made, the code committed, the diagnosis reached — was a reasonable measure of what the employee contributed. With AI integration, the output is no longer a clean proxy. An employee with a heavy AI assist may produce twice the output of a peer without; the output is the joint product of the human and the AI, and the performance system that rewards output without attribution is rewarding the AI, not the human.

The design problem is attribution. What part of the joint output is the human’s contribution, in a form the performance system can recognise? The answer is not obvious, not stable across role types, not independent of the organisation’s strategic intent, and not solvable by importing a model from another company. Attribution is the hardest design problem in the performance-system redesign, and this article teaches the expert to work through it rather than to finesse it.

The attribution problem

A worked example. Two underwriters, A and B, process commercial insurance applications. Before AI integration, each underwriter reviews applications manually and issues a decision; volume and decision-quality are both reasonable performance measures.

After AI integration, each underwriter receives AI-drafted recommendations and the underwriter’s work is to review, adjust, and approve or override. A processes 30 applications per day, accepts the AI recommendation 92% of the time, and has a decision-quality score of 0.88. B processes 22 applications per day, accepts the AI recommendation 65% of the time, and has a decision-quality score of 0.91.

The pre-AI performance system would rank A higher (higher volume, acceptable quality). But the performance-system question is: which underwriter is contributing more as an underwriter? A is fast because A defers to the AI; A’s actual underwriting judgment is applied in 8% of cases. B is slower because B exercises judgment on 35% of cases; B’s actual underwriting work is substantially more than A’s.

Which is the better underwriter? The question cannot be answered without knowing the organisation’s strategic intent. If the strategic intent is to maximise throughput with acceptable quality, A is better. If the strategic intent is to build professional underwriting capability that catches AI errors, B is better. If the strategic intent is to protect against AI failure modes the current AI is not well-calibrated for, B is better. The performance system encodes one of these answers; the design decision is which.

The attribution question is, at root, “what contribution are we measuring” rather than “how do we measure.” The organisation that does not answer the first question cannot produce a defensible answer to the second.

Three redesign pillars

The performance-evaluation redesign rests on three pillars: redesigned goals, redesigned coaching cadence, and redesigned review processes.

Pillar 1 — Redesigned goals

The goal-setting conversation (typically annual, sometimes semi-annual) establishes what the employee will be measured on over the period. AI integration requires goals that reward the human contribution rather than the joint output.

Three goal-setting shifts:

• From output volume to output quality × judgment applied. Rather than “process 25 applications per day,” the goal becomes “process 25 applications per day with quality score ≥ 0.88 and judgment-applied ratio ≥ 20%.” The judgment-applied ratio — the fraction of applications on which the underwriter adjusted the AI recommendation — preserves the human contribution in the measurement.
• From individual output to team capability. Some AI-integrated work is more efficiently measured at the team level, where the team’s collective output and its capability to handle exceptions (the AI failure modes, the novel cases) are the relevant unit. Individual goals within the team address contribution to team capability.
• From activity to outcome. Activity-based goals (“attend training,” “complete review”) are particularly weak in AI-integrated work because the activities themselves are increasingly automated. Outcome goals (“the cohort of applications you processed has error rate below threshold six months later”) better capture the human contribution, though they require measurement infrastructure that takes time to build.

Goal-setting in the transition period is iterative. The first year’s goals are the organisation’s best attempt; the second year’s goals are refined based on what the first year’s measurement surfaced. The iteration is open — goals are not treated as fixed truths that cannot be refined with experience.

Pillar 2 — Redesigned coaching cadence

The coaching cadence is the standing rhythm of manager-employee conversations through which performance is developed rather than only judged. AI integration changes what the conversations are about.

The cadence design:

• Weekly, lightweight. 15–25 minutes, structured around “what did you try this week, what worked, what got in the way, what will you do next week.” The conversation is about applied practice, not about the programme or the policy.
• Monthly, developmental. 45–60 minutes, structured around “how is your capability developing, what are you learning, where are you stuck.” This conversation is the one where Bridges-phase (Article 21) support lands in practice.
• Quarterly, review. 90 minutes, structured around “what has the quarter produced, what is the pattern, what do we need to adjust.”

The cadence is non-optional. A manager who is not running the weekly cadence is not performing the manager job in the redesigned system, and the manager-enablement programme (Article 28) must produce managers who do. The cadence is the single intervention most strongly correlated with sustained AI adoption; its absence predicts programme under-performance at any scale.

Pillar 3 — Redesigned review processes

The annual or semi-annual review becomes, in AI-integrated work, a consolidation of evidence rather than a separate act of judgment. The weekly and monthly cadence have produced the data; the review synthesises it.

Three review-process changes:

• Evidence-based, not rating-based. The review records what the employee did, what they produced, what capability they built, what judgment they applied, and what outcomes followed. Ratings, if used, are derived from the evidence rather than declared at the start.
• Two-way, not one-way. The employee contributes their own evidence-based account; the manager contributes theirs; the review conversation reconciles them. One-way review-as-delivery is weak practice in any setting and particularly weak in AI-integrated work where the employee’s own view on judgment applied is often more accurate than the manager’s.
• Forward-leaning. The review allocates substantial time to the next period’s goals and development, not only to the closing period’s assessment. In AI-integrated work where capability evolves quickly, the forward-leaning component is the higher-leverage part of the conversation.

Specific difficulty cases

Four difficulty cases recur in practice; the performance system must address each.

The apparent high-performer who defers to the AI. Volume is high, quality is acceptable, AI-acceptance rate is near 100%. The employee’s actual contribution is minimal. The performance system must surface the AI-acceptance rate explicitly; a performance conversation that does not reference it is evaluating the AI, not the employee.

The apparent under-performer who exercises strong judgment. Volume is lower because the employee is catching AI errors and adjusting. The performance system must reward the judgment applied; a system that rewards only throughput will lose the employees whose actual contribution is most valuable.

The employee struggling with the AI-integrated role. The employee was strong pre-AI and is finding the AI-augmented role difficult. The performance system response is coaching, not adverse rating; a system that marks the employee down for transition difficulty cements the decline rather than addressing it.

The employee who produces strong AI-assisted output but cannot defend the reasoning. The employee accepts AI recommendations without deep understanding; the outputs are acceptable but the employee cannot explain the decisions to a customer, a regulator, or a colleague. The performance system must probe reasoning, not only output. A specific review technique: present the employee with one of their prior decisions and ask them to explain it without reference to the AI output; an employee who cannot answer is not performing the human part of the role.

The dispute surface

Performance systems always produce disputes; AI integration amplifies them. Disputes arise over: attribution (who contributed what to the joint output); consistency (is the same measurement applied to peers); transition support (was the employee given genuine opportunity to develop in the new role); and bias in AI assist (does the AI itself produce differential outputs that affect evaluation).

The dispute surface is reduced — not eliminated — by: published attribution methodology that employees and managers understand; cross-manager calibration sessions that test for consistency; documented transition-support plans so development opportunity is evidenced; monitoring of AI outputs for differential impact across employee demographic groups (with the caveat that the monitoring must itself be legally compliant).

Where disputes arise despite these mitigations, the escalation path is clear: the employee raises with HR; HR reviews the case against the published methodology; if the review confirms inconsistency, the case is corrected and the methodology is reviewed for pattern risk. Dispute processes that are opaque or retaliatory harm the system’s legitimacy regardless of how well the underlying methodology is designed.

Two real-world anchors

Brynjolfsson et al. NBER on productivity attribution

Erik Brynjolfsson, Danielle Li, and Lindsey Raymond’s NBER Working Paper 31161 Generative AI at Work (2023, with subsequent updates) documented productivity effects of generative-AI introduction in a customer-service setting. The paper’s finding of differential productivity gain — new employees benefited substantially more than experienced employees, narrowing the experience-based productivity gap — is widely cited. For performance-evaluation design, the paper’s implication is that AI-assisted productivity gains accrue differentially, and a performance system that does not attribute carefully will reward the AI rather than the human in predictable patterns. Source: https://www.nber.org/papers/w31161.

The lesson: attribution is not a theoretical concern. The empirical evidence on productivity gain distribution confirms that performance systems measuring joint output will produce distorted evaluation in predictable ways, and the distortions will concentrate in specific employee populations.

MIT Sloan AI-Human Collaboration on performance-system redesign

MIT Sloan Management Review has published multiple articles on performance-system adaptation for AI-integrated work. The published cases document organisations that have redesigned their performance systems explicitly to surface human judgment and reward it (rather than to measure joint output as in pre-AI systems). The cases report improved retention of high-judgment employees and improved dispute-process legitimacy. Source: https://sloanreview.mit.edu/topic/artificial-intelligence/.

The lesson: the redesign is practiced. Organisations that have done it seriously report better outcomes; organisations that have treated AI-augmentation as a reason to intensify pre-existing output measurement without attribution have produced retention and legitimacy problems.

Learning outcomes — confirm

A learner completing this article should be able to:

• Name the attribution problem and argue why it is the hardest design problem in AI-performance-system redesign.
• Redesign goals to reward human judgment rather than joint output, using quality × judgment ratios or equivalent constructs.
• Design a three-layer coaching cadence (weekly / monthly / quarterly) and defend each layer against operational pressure to collapse them.
• Convert the annual review from a rating-based to an evidence-based exercise.
• Handle the four difficulty cases (apparent high-performer, apparent under-performer, struggling transitioner, non-defending output producer) with case-specific responses.
• Reduce the dispute surface through published methodology, cross-manager calibration, documented transition support, and AI-output monitoring.

Cross-references

• EATF-Level-1/M1.6-Art08-Workforce-Redesign-and-Human-AI-Collaboration.md — Core Stream workforce-redesign anchor.
• EATP-Level-2/M2.5-Art05-People-and-Change-Metrics.md — people-and-change metrics anchor.
• Article 24 of this credential — task decomposition (feeds attribution).
• Article 25 of this credential — role specification (Section 6 performance expectations).
• Article 28 of this credential — manager enablement (Domain 2 consumes this article).
• Article 32 of this credential — belonging and equity (AI-output monitoring for differential impact).

Diagrams

• StageGateFlow — redesigned evaluation cycle: goal-setting → weekly cadence → monthly cadence → quarterly cadence → annual review, with standing artefacts per gate.
• Matrix — performance element × AI-impact adjustment, showing how each element (volume, quality, judgment applied, professional development, team contribution) shifts in AI-integrated work.

Quality rubric — self-assessment