AI is here. Groundbreaking, I know. But before we go into the whole thing, picture this.
You approve a pull request from one of your strongest engineers. Clean, well-structured, exactly the kind of code you’d want to see. You ship it.
Then it breaks in production.
You pull him into a call to walk through what happened, together. About 90 seconds in, he can’t tell you why the code did what it did. Not because he’s hiding something. Because he genuinely doesn’t know. He prompted his way to a correct-looking answer, and it never actually passed through his own understanding. He copied, he pasted, he never checked.
This isn’t hypothetical. A leader I work with in mentoring told me this story a few weeks ago, and it hasn’t left me since. She asked me the question that’s the reason I’m writing this article today: if I can’t tell whether someone is good at their job anymore just by looking at their output, what am I even evaluating?
Interesting, right? Let’s unpack it.
Your Old Signal for Competence Just Disappeared
For most of your career, competence was visible. You could look at someone’s code, their architecture decisions, their debugging process, and know with reasonable confidence whether they were good or not. That visibility is exactly what got you promoted. You were the expert whose eye for quality made you valuable enough to lead others.
That eye still works. It’s just looking at a different kind of artifact now.
Output can be clean, well structured, completely correct, and tell you almost nothing about the person who produced it. A candidate can ace a take-home assignment and freeze the moment you ask a follow-up question live. A code review that used to catch shaky reasoning now sails through, because the code simply looks right. A junior who used to build real debugging muscle by sitting in the discomfort of not knowing now skips that discomfort entirely, and skips the learning that came with it.
Let me be precise about what this article is not. It’s not “AI is bad” or “ban the tools.” I’m not undecided on that. The ship on AI adoption has sailed, and honestly, it shouldn’t have sailed any other way. This is about something narrower and more urgent: you’ve lost your old signal for competence, and you haven’t built a new one yet.
This Is a Contracting Problem, Not a Tool Problem
Here’s where I want to bring in a concept I talk about constantly (on the podcast, during keynotes, and in From Code to People, where “Contract before it’s too late” gets its own chapter for a reason): contracting.
Most of the anxiety leaders feel about AI and their teams isn’t actually about the tool itself. The tool is just a tool, like countless tools before it. The anxiety comes from the fact that almost no team has ever explicitly agreed on what good use of AI looks like here. Not a permission list of which tools are allowed; that’s an administrative detail. What’s missing is a shared standard for what ownership means when a tool did part of the work.
Here’s the standard I’d propose, and it’s simple: you own what you ship. Owning it means you can explain it, defend it under questioning, and fix it without the tool sitting next to you. That’s the whole bar. It doesn’t matter how much of the first draft came from a model. What matters is whether the understanding made the trip from the tool into the person.
If contracting is new to you, I broke down the three levels here, administrative, professional, psychological, and it’s the exact same structure that applies to AI use. Just with a new subject on the table.
The PCMⓇ Layer You Cannot Skip
If you start asking people to explain their reasoning out of nowhere, some of your team will hear a completely normal check-in. Others will hear an accusation.
This is where Process Communication Model matters. If someone’s base is a Thinker, their core need is to be seen as competent. It’s literally the existential question that PCMⓇ type is built around: am I competent? Ask a Base Thinker: “walk me through your reasoning” out of nowhere, with no context, and their brain doesn’t hear curiosity. It hears “I think you’re not.”
The intervention is identical across your team. The impact is not. So this has to be introduced as a team-wide standard, set in advance, not as a spot investigation you run when you’re suspicious of one person. The moment it feels personal, you’ve lost the room, and you’ve taught your best people to hide their process instead of showing it.
What the Data Actually Says
This isn’t just a leadership hunch. Anthropic published a randomized controlled trial in January 2026 that puts numbers on exactly the pattern from that pull request story. Fifty-two, mostly junior, engineers learned a new Python library: half with AI assistance, half without. The AI group finished about two minutes faster on average, a difference too small to be statistically meaningful. But on a quiz testing what they’d actually learned, the AI group scored 17% lower than the group that coded by hand, nearly two full letter grades. The biggest gap was in debugging, the exact skill your team needs to catch AI-generated errors before they hit production.
The researchers found something else worth sitting with: not all AI use produced the same outcome. People who delegated everything to the AI, or who used it purely to debug without asking why, scored under 40%. People who used AI to generate code and then asked follow-up questions, requested explanations, or posed conceptual questions while still coding themselves, scored 65% or higher. Same tool. Completely different result. The variable wasn’t the AI. It was whether the understanding made the trip along with the output.
That’s your contracting standard, confirmed by data. It was never about banning the tool. It’s about which of those interaction patterns you’re building into your team’s culture, on purpose, before it becomes the default by accident.
Four Things You Can Start This Week
1. Shift your question. Instead of only asking “is it doing the work,” ask “walk me through why you made this choice.” Not as a gotcha. As a norm, a standard part of every 1:1. Keep the timing unpredictable, so it reads as culture, not audit.
2. Ask people to explain from memory. In your 1:1s, have them talk you through a recent piece of work without pulling it up on screen. What they can explain without the artifact in front of them tells you more than the artifact ever will.
3. Co-write the standard, don’t hand it down. Sit down with your team and build together what responsible AI use means here: ownership, explainability, no blind copy-paste into production. When people help write a standard, they defend it. When you hand it down, they route around it, even unconsciously.
4. Learn to tell struggle from friction, and protect the struggle. Debugging your own logic, wrestling with why an approach doesn’t work: that’s how real skill gets built. AI can quietly remove that discomfort if you let it. Not all friction needs solving. Some of it is the job.
Pick one. Pick one person on your team. Ask them to walk you through the reasoning on their next piece of work, not the result, the reasoning. Ask with real curiosity, not an edge.
The Real Question
You’re not evaluating output anymore. You’re evaluating whether understanding made the trip from the tool into the person standing in front of you. That’s a new muscle, and most leaders haven’t built it yet. Myself included.
So: when’s the last time you asked someone on your team to explain their reasoning, not to catch them, but to actually find out?
If this is a conversation you want to keep having, catch the full episode (#179) on the podcast, and if you want more of this straight to your inbox, join the Leman Leadership Pulse. Weekly. No fluff.
