CaseAdvancedResponsible AI & Advanced Practice / Internal AI tooling and enablement products / #9
How do you handle shadow AI usage across the company?
TRACE the diagnosis: Wrenhaven Unified School District, and ClassAssist, its sanctioned assistant for teachers
Interviewer's question: "How do you handle shadow AI usage across the company?" Wrenhaven Unified School District serves about nine thousand students across twelve schools. Naomi Fitzgerald is the district's IT director. Marisol Bettencourt teaches special education there.
The direct answer
Shadow AI usage almost never starts as a discipline problem, and treating it as one wastes the response. It starts the week a major public model releases, well before any company decides anything, and it concentrates on exactly the tasks your sanctioned tool doesn't cover. Find out which cause dominates by comparing shadow usage on supported tasks versus unsupported ones. If it clusters on the unsupported ones, the fix is scope, not a warning email.
Do this, in order
Find the real start date, and check it against the last major public model release, not a company decision.Why: shadow usage usually predates any internal policy discussion by months.
Recut usage by role or task type before assuming it's uniform.Why: a district-wide average hides that one role, like special education, may carry almost all the risk.
Rule out a discipline problem before assuming a tooling gap.Why: punishing people for a gap in the sanctioned tool fixes nothing and teaches people to hide it better.
Run the evidence test: does shadow usage cluster on tasks your tool doesn't support?Why: that single comparison tells you whether the fix is scope, speed, or awareness.
Fix the highest-stakes gap first, even if it's the hardest one to build.Why: the sensitive, unsupported tasks are exactly where a public tool causes the most real harm.
How to answer this, stage by stage
Nobody is grading whether you can say "ban unauthorized tools." They're grading whether you can find out why people reached for one in the first place.
Stage 1
Scope it to one district, one tool
Say it like this
"I'll answer this for Wrenhaven Unified School District, and ClassAssist, its sanctioned assistant for teachers."
Why this works
Keeps "handle shadow AI" from turning into a generic policy statement.
Stage 2
Say your structure out loud
Say it like this
"I'll use TRACE. Timeline, when it started. Recut, who it hits first. Assume nothing, rule out discipline. Cause candidates, three real ones. Evidence, the test that separates them."
Why this works
Signals a diagnostic method before naming a single cause.
Stage 3
Put a real timeline on it
Say it like this
"Teachers were using public chatbots within weeks of the last major model release, months before the district ever discussed an AI policy."
Why this works
Shows shadow usage predates any company decision, which reframes the whole question.
Stage 4
Recut it by role
Say it like this
"Special education teachers, drafting IEP goals, show far higher shadow usage than PE teachers, because the tasks aren't remotely similar."
Why this works
A district-wide average would hide exactly where the real risk concentrates.
Stage 5
Rule out the discipline explanation
Say it like this
"Teachers weren't ignoring a rule, most didn't know ClassAssist existed for their exact task, because it was never built to cover it."
Why this works
Kills the tempting, wrong explanation before it shapes the whole response.
Stage 6
Give the evidence test, and close
Say it like this
"Compare shadow usage on tasks ClassAssist supports against tasks it doesn't. At Wrenhaven, it clustered almost entirely on the unsupported ones, which points straight at scope, not discipline."
Why this works
Ends on the one check that actually separates the real causes from the tempting one.
Let's learn
ClassAssist is Wrenhaven's sanctioned assistant, approved for lesson plan outlines and quiz question generation, deliberately scoped away from anything involving individual student data.
Before any shadow usage, teachers wrote lesson plans and drafted individualized education program goals entirely by hand, drawing on their own knowledge of each student. ClassAssist made the routine, low-risk part of that faster.
Shadow AI tool usage rate, by teacher role
A district-wide average would have read as a modest, manageable number. The real risk was sitting almost entirely in one role.
The turn: the gap between special education and PE wasn't about who follows rules better. It was about which teachers had a real task ClassAssist simply didn't cover, and which didn't.
Same goal, same student. What changed is how much of the teacher's own judgment made it into the final draft.
The decision that mattered
ClassAssist launched scoped to exclude anything touching individual student data, a reasonable privacy decision on its own. It left the highest-stakes, most sensitive drafting work, IEP goals and behavior notes, with nowhere sanctioned to go.
At its worst: a teacher drafting IEP goals for a student with a complex behavioral history reduces months of context into a terse prompt for a public chatbot, and the resulting goal language quietly loses the specificity that made it legally and educationally sound in the first place.
All three were partly true at Wrenhaven. Only one of them explained most of the gap.
What I would leave alone: ClassAssist's scope on lesson plan outlines and quiz generation never needed touching. Shadow usage on those tasks stayed near zero, because the sanctioned tool already did the job well.
The lesson: shadow usage is rarely a sign that people don't care about the rules. It's usually a sign that the rules never noticed the task they were actually trying to do.
Now here is the same thing as a story
The short version above is what you'd say to Wrenhaven's superintendent. Read this one for how the district found out.
Marisol Bettencourt has taught special education for six years. Drafting an IEP goal, for her, meant pulling together a student's behavioral history, family input, and prior goals into language precise enough to hold up in a review meeting.
She first tried a public chatbot out of curiosity, the same week a major new model made headlines, months before Wrenhaven had said a word about AI policy. It was fast, and for a routine goal, it looked fine.
Knowledge spark: why does an IEP goal need more context than it looks like it does?
An IEP goal isn't just a sentence, it's a commitment tied to a specific student's history, family agreement, and legal review. A goal that reads fine in isolation can still be wrong if it ignores what was tried and failed last year.
Over a few months, without deciding to, Marisol started typing shorter and shorter prompts, a few lines summarizing a student instead of the fuller picture she used to hold in her head while writing. It was faster, and on the surface, the drafts still read professionally.
The fourth milestone doesn't show up anywhere. That's exactly the problem.
The district found out through a routine annual data-privacy audit, not a complaint. The audit sampled staff device usage patterns and flagged a spike in traffic to public AI chatbot domains, concentrated almost entirely among special education staff.
Nobody had decided to bypass the district's tool. Every teacher involved had simply done the sensible thing when the sanctioned tool had nothing to offer them.
Here's the decision I'd take back: scoping ClassAssist to exclude anything touching individual student data, without building any sanctioned path at all for the tasks that needed it most. That made sense as a privacy-first launch decision. It stopped making sense the moment the highest-stakes work had nowhere safe to go, so it went somewhere unsafe instead.
Replayed with a sanctioned, privacy-reviewed path for IEP goal drafting, one that requires the same fuller context Marisol used to hold in her head: the same curiosity about a new public model doesn't pull teachers away, because the sanctioned tool now does the job better, with real data protections built in.
I scoped ClassAssist away from student data because it felt like the responsible, cautious choice. It took a privacy audit and a terse prompt draining the nuance out of real students' goals to see that caution applied to the wrong risk had just moved the real risk somewhere the district couldn't see at all.
The five letters, run against a district-wide auditNot a policy memo. TRACE is what tells you whether shadow usage is a discipline problem or a scope problem.
T
Timeline. When it actually started.
Weeks after a major public model's release, months before the district's own AI policy discussion.
Shadow usage predates the company decision, not the other way around.
R
Recut. Who it hits first.
Special education teachers at 68 percent, PE teachers at 6 percent. The average would have hidden this entirely.
A district-wide number is nearly meaningless without a segment breakdown.
A
Assume nothing. Rule out discipline first.
Most teachers didn't know ClassAssist covered, or didn't cover, their exact task. This wasn't defiance.
Kills the tempting explanation before it shapes a punitive, wrong response.
C
Cause candidates. Three, named.
ClassAssist doesn't support open-ended tasks; new use cases take months to add; teachers don't know what it covers.
The hardest step: naming real candidates instead of guessing at "people not following policy."
E
Evidence test. The one check that separates them.
Shadow usage clustered almost entirely on tasks ClassAssist doesn't support, confirming scope as the dominant cause.
The strongest move in the method, and the direct answer's foundation.
None of these show up in a login log. All four show up in the goal itself, if anyone reads closely.
The top-right corner is exactly where a scope decision left the highest-stakes work uncovered.
The recap, one line per letter: timeline is weeks after a public release, recut is special education carrying most of the risk, assume nothing rules out discipline, cause candidates are the three named reasons, and evidence is the supported-versus-unsupported comparison.
Shadow tool usage rising, well before the district's own policy discussion began
By the time anyone in leadership sat down to discuss a policy, adoption had already passed the point a policy could shape.
And if you want to be sure it really works, try it somewhere elseSame five letters, a rural electric cooperative instead of a school district. Nothing about the two organizations is alike.
Amistad Valley Electric Cooperative gives field engineers a sanctioned tool for routine grid-fault diagnostics. Warrick Feldman is a field engineer there.
Mapped onto TRACE: timeline is the same shape, engineers started using a public model for fault troubleshooting within weeks of its release, long before the cooperative's own AI committee ever met. Recut: engineers handling rare, unusual fault patterns showed far higher shadow usage than those on routine maintenance routes, where the sanctioned tool worked fine. Assume nothing: most engineers didn't know the sanctioned tool's diagnostic library stopped at common fault types; this wasn't defiance, it was a gap nobody had mapped. Cause candidates: the sanctioned tool's diagnostic library was too narrow, updates to it took months to approve, and some engineers simply didn't know it existed for their equipment type. Evidence: shadow usage clustered specifically on rare fault types the sanctioned tool had never been trained to recognize, confirming scope, not defiance or tool awareness, as the dominant cause.
A different shape of picture than Section 2 used: a branching decision instead of a timeline. The same scope gap decides the outcome either way.
Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "shadow usage starts right after a public release, not a company decision, and it clusters on whatever your tool doesn't cover; fix the scope, not the discipline," and stop.
Cost: if a formal usage audit isn't possible, an anonymous survey asking which tasks people wished the sanctioned tool handled gets most of the same signal.
The model gets better, for real: even as the sanctioned tool's coverage improves, a new public model release can reopen the same gap for whatever task the improvement didn't reach yet, so this isn't a one-time fix.
Where people run it wrong.
They respond to shadow AI usage with a policy memo and a warning, treating it as a discipline problem before checking whether it's actually a scope problem.
They measure adoption district-wide instead of by role, missing that the real risk concentrates in one group.
They wait for a complaint or an incident, when shadow usage, by its nature, rarely generates one on its own.
How to use it live. When someone asks how to handle shadow AI usage, ask yourself first: does it cluster on tasks the sanctioned tool doesn't cover? If yes, the fix is scope. Chasing discipline instead just teaches people to hide it better.
Flashcards (tap any card to flip it)
1 · THE FRAMEWORK
What framework fits "how do you handle shadow AI usage across the company"?
Tap to flip
ANSWER
TRACE: timeline, recut, assume nothing, cause candidates, evidence test. The evidence test compares usage on supported versus unsupported tasks.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Marisol Bettencourt, a special education teacher at Wrenhaven Unified School District, six years into drafting individualized education program goals.
3 · THE HABIT
What did Marisol stop doing once the public chatbot habit set in?
Tap to flip
ANSWER
She stopped writing full clinical notes summarizing a student's whole history and started typing shorter, terser prompts instead.
4 · THE FLIP
What's the two-setting switch in this story?
Tap to flip
ANSWER
Writing from full clinical judgment, versus typing a terse prompt tuned to get a fast, plausible-looking output from a public chatbot.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Scoping ClassAssist to exclude any task touching student data, with no sanctioned alternative built for the highest-stakes work at all.
6 · THE NUMBER
Fill in the blank: special education teachers reported ___% shadow AI usage, versus 6% for PE teachers.
Tap to flip
ANSWER
68%. The gap tracked task type, not rule-following.
7 · THE REPLAY
Same public model release, sanctioned path for IEP drafting in place. What changes?
Tap to flip
ANSWER
Curiosity about the new model doesn't pull teachers away, because the sanctioned tool now handles the highest-stakes task with real data protections.
8 · CROSS PRODUCT TRANSFER
Section 4 answers this again for a different organization. Which one, and what confirmed the same cause?
Tap to flip
ANSWER
Amistad Valley Electric Cooperative's field engineers. Shadow usage clustered on rare fault types outside the sanctioned tool's library, confirming scope as the cause there too.
Check yourself Score: 0 / 0
Multiple choice
1. What's the single strongest evidence test for whether shadow AI usage is a scope problem?
A. Asking teachers directly whether they've broken the rules.
B. Comparing shadow usage rates on tasks the sanctioned tool supports versus tasks it doesn't.
C. Counting total logins to the sanctioned tool.
D. Checking how many staff completed AI policy training.
Show hint
Look at the evidence test step.
Show answer
B. If shadow usage clusters on unsupported tasks specifically, that's real evidence of a scope gap, not a guess about discipline or awareness.
True or false
2. True or false: Wrenhaven's shadow AI usage started after the district's own AI policy task force convened.
True
False
Show hint
Look at the line chart and its marked policy task force date.
Show answer
False. Usage had already climbed past 40 percent by week 14, when the task force finally convened, well after the trend was set.
Fill in the blank
3. Fill in the blank: district-wide shadow AI usage rose from 8 percent to ___ percent in the ten weeks after the public model release.
Show hint
Look at the line chart.
Show answer
41 percent. That's a five-fold increase in ten weeks, well before any internal policy conversation began.
Short answer, where it wouldn't matter
4. Name a Wrenhaven task where shadow AI usage barely showed up at all.
Show hint
Look at the quadrant diagram's bottom-left corner.
Show answer
Model answer: Lesson plan outlines or quiz question generation, both well supported by ClassAssist and low in sensitivity, where shadow usage stayed near zero.
Short answer, apply it yourself
5. Pick a workplace task you've seen people handle with an unofficial tool. What gap in the official one might explain it?
Show hint
Think about a task the official tool was never built to handle, not a rule anyone was ignoring.
Show answer
Model answer: People often reach for a personal spreadsheet or app when the official system handles the common case but not their specific, less routine need.
Short answer, name the reversal
6. What old decision does this answer take back, and why did it make sense when it was made?
Show hint
Look at "the decision I would take back."
Show answer
Model answer: Scoping ClassAssist away from student data entirely. It made sense as a privacy-first launch choice, and broke once the highest-stakes work had nowhere sanctioned to go.
Before you close the answer
Why this works
Tests whether you'll default to a policy-and-punishment response, or actually diagnose why shadow usage exists before reacting to it. Most candidates jump straight to "block unauthorized tools."
Follow-up traps
"Shouldn't you still block unauthorized tools for security reasons?" Response: blocking access without building a sanctioned alternative just pushes the same need somewhere less visible, like a personal device instead of a work one.
"Isn't a privacy-scoped launch always the safer choice?" Response: safer for the tool, not necessarily safer overall; leaving a high-stakes task fully uncovered can push it toward a less safe, unmonitored option instead.
If pressed
Wrenhaven's actual fix for IEP drafting ran inside the district's existing student-data environment, not a general-purpose model, with a mandatory teacher sign-off on every generated goal before it entered a student's file, so the sanctioned path never became just a faster version of the same unmonitored risk.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
