Describe how you would run an internal AI literacy session for your leadership team.

Farlow Systems runs its own employee IT helpdesk on Deskmend, an AI tool that reads a support ticket and tries to close it without a person touching it. Yulia Wenzel is the AI PM who owns Deskmend's roadmap. Obadiah Ottway is Farlow's CEO. Kessandra Fossett is the CFO who sets the IT support budget every year.

01How to answer this, stage by stage

Nobody is grading whether you can say "leadership needs to understand AI better." They are grading whether you can name the actual room, the actual two tickets, and the actual line between a session that calibrates judgment and one that either bores a CEO or flatters him.

02Let's learn

For three years running, Farlow's IT support headcount did not move by a single person, up or down.

This is the gap the session exists to close. Not a lack of interest in the room. A lack of anything real to look at.

Deskmend reads an employee's IT ticket and tries to close it without a person touching it. Before Farlow built it, a nine person support team led by Yun Ridgewell handled roughly 2,300 tickets a month by hand, about twelve minutes each on average, split across VPN failures, password resets, printer trouble, and requests that never fit neatly into one box.

Fourteen months in, Deskmend closed 71 percent of all tickets without a person touching them, and that single number sat on the same slide in the monthly ops review the entire time. Password reset tickets, a third of total volume, told one story: Deskmend got the right answer 97 percent of the time, and handle time fell from twelve minutes to under ninety seconds. Multi issue tickets, about one in eleven, told a different one. Deskmend fully and correctly fixed both parts only 22 percent of the time. But it marked the ticket "Resolved," closed and done, 61 percent of the time regardless, because closing only needed one detected problem to clear its own cut off point.

Neither message was about Deskmend. Both of them shaped what Farlow's leadership believed about it anyway.

The Tuesday that forced the issue, a company using a rival product called Wybrant Resolve had its ticket bot close an employee's security breach report as a routine password reset, and the story made the tech press by lunch. That same morning, Obadiah had forwarded a different vendor's press release to the leadership channel, a claim of ninety percent full ticket closure with zero human touch, and asked whether Farlow could match it by cutting Tier 1 support in half next quarter. Kessandra, reading the Wybrant story an hour later, replied that she wanted Deskmend's rollout frozen until someone could prove it was safe. Neither message mentioned a single real Deskmend ticket, because neither of them had ever seen one.

The whole answer sits in this one picture. Not a claim about Deskmend in general. Two tickets, the same week, side by side.

What Yulia built instead of answering either message directly: a forty five minute session, both of them in the room together, no slides about AI in general. She opened with the password reset ticket, Deskmend's clean win, then the VPN and badge ticket, where it fixed one problem and silently dropped the other while still marking the whole thing Resolved. Same model, same week, two very different tickets.

Same ticket type, same model. What changes the outcome is only whether anyone in the room has ever seen one.

What it costs at its worst: if Obadiah had cut Tier 1 support in half on the strength of a competitor's headline number, Farlow would have gutted the exact team that catches a dropped badge ticket before it becomes a locked door on a Monday morning. If Kessandra had frozen the rollout instead, Farlow would have thrown away the 97 percent of tickets Deskmend genuinely handles well, to guard against a failure mode a five minute look at the real data could have named and fixed.

What I would leave alone: Yun Ridgewell's support engineers, who triage Deskmend's escalations every shift, don't need this session at all. They already know exactly which ticket types to double check, because they've watched the tool get it wrong in person. This session exists for the two people in the building who never open a ticket.

The lesson: a single number on a slide isn't knowledge, it's a placeholder for knowledge. Leadership needed to see two real tickets before that 71 percent meant anything true to them at all.

03Now here is the same thing as a story

The short version above is what you'd actually say in the room. Read this one for why the fix had to be a session built on real tickets, not a promise to "brief leadership more often."

Every Monday morning, Kessandra Fossett opens the same spreadsheet and leaves the support line exactly where it was the week before.

Yulia Wenzel has built support tooling for eight years, and she can usually tell within a week whether a rollout is actually working or just quiet because nobody's looked hard enough yet. When Deskmend went live at Farlow, she built the eval set herself, tracked it by ticket category from day one, and trusted the number climbing on her own dashboard, because she was the one watching it climb.

For the first ten months, that felt like enough. Yulia sent a short written update to the leadership channel each month: a graph going up, a line or two of context. Obadiah skimmed it. Kessandra skimmed it less. By month six, the updates had shrunk to one line: "Deskmend continues to perform well." By month nine, nobody replied to them at all. Nobody had stopped trusting Deskmend. They had simply stopped needing to think about it, because the number in the update never seemed to change.

Then came the Tuesday two messages landed in the same channel within an hour of each other, and neither one came from Yulia.

Yulia drew this line early, and held it on purpose. Not because leadership couldn't handle more. Because more wasn't the problem.

Obadiah forwarded a competitor's press release, a claim of ninety percent full closure with zero human involvement, and asked, half rhetorically, whether Farlow could halve its Tier 1 team by next quarter to match it. An hour later, Kessandra replied to the whole channel with a link to the Wybrant Resolve story, a rival's ticket bot that had just closed a security breach report as a routine password reset, and said she wanted Deskmend's rollout paused until someone could show her it wasn't doing the same thing.

Yulia's first instinct was the tempting one: reply to both messages with the real number, 71 percent, and a paragraph explaining why the comparison to Wybrant wasn't fair. She drafted it, then deleted it. A written reply to a written panic would just become a third message in a thread neither of them was reading carefully to begin with.

She thought back to the meeting, more than a year earlier, where the one line ops review update had been decided. It hadn't been a lazy choice. Deskmend was brand new then, a small pilot, and one honest number felt like plenty of transparency for something that small. Nobody in that room imagined a year later that number would be standing in for the entire company's understanding of what the tool could and couldn't do.

What she built instead: a forty five minute session, two days later, both Obadiah and Kessandra in the same room, no deck about AI in general. She opened with a password reset ticket from the week before, an employee locked out at 7am, verified and reset in eighty three seconds. Then she opened the VPN and badge ticket from the same week: VPN fixed cleanly, badge access never touched, ticket marked Resolved anyway. Same model. Same week. Two very different outcomes.

She also considered asking Deskmend's engineering lead to walk the room through how the ticket classifier actually scores intent, confidence numbers, cut off points, the works. She killed that idea before the invite went out. Neither Obadiah nor Kessandra needed to know how the number got computed. They needed to know what it meant for the next ticket that looked like the badge one.

Same headcount conversation Farlow was always going to have. What changed is only what anyone in the room knew before it started.

Six weeks later, Obadiah opened the actual quarterly headcount meeting. He didn't mention the ninety percent claim once. Instead he asked Yulia straight: "What's our real rate on tickets shaped like that badge one?" Kessandra, for the first time in three years, added a specific new line to the support budget instead of leaving it untouched: two review hours a week, paid time for a person to check every flagged multi issue ticket before it closed. The headcount cut wasn't promised that day. It also wasn't ruled out for good. It just stopped being a decision made off a stranger's number.

One design handed leadership a single slide and asked them to trust it. The other handed them two real tickets and asked them to look. Only one of those two designs survives a Tuesday it didn't see coming.

What I'd tell myself, reading Obadiah's message that first Tuesday: the mistake wasn't that leadership reacted to outside news. It was that we'd never given them anything of our own to react to instead.

04SPARK, the five checks behind forty minutes with the leadership team

Not a script for sounding informative. SPARK is what forces you to name the one concrete session, and prove it survives both ways a literacy session actually breaks.

Same ticket, same model, same leadership team. What changes the outcome is only whether the session ever happened.

The recap, one line per letter: situation is a leadership team forming its picture of Deskmend from outside noise alone, payoff is one shared, calibrated question carried into every future decision, anchor is the forty five minutes built on two real tickets from the same week, risk is either extreme costing months of bad decisions, and keep out draws the line at a budget ask and a technical deep dive, never at the tickets themselves.

05And if you want to be sure it really works, try it somewhere else

Same five letters, a county building department instead of an IT helpdesk, and this time the blind spot isn't a badge reader. It's a mixed use permit that touches two code categories at once.

Doskoch Civic Systems builds Permitto, an AI tool that reads a construction permit application for Pellew County and auto approves the straightforward ones before a person ever opens the file. Ozella Menlove owns Permitto's roadmap. Pellew County's leadership formed its own opinion of Permitto the same ungrounded way Farlow's did: the County Administrator wanted to publicize a faster average approval time after reading about a neighboring county's rollout, while the Finance Director had quietly left Permitto out of every budget conversation for two years, treating it as something too technical to plan around.

Mapped onto SPARK: the situation is leadership forming a picture of Permitto from a neighboring county's press coverage and their own avoidance, never from a real Pellew County permit. The payoff is one shared question carried into every future permitting decision: is this a single code category permit, or a mixed use one. The anchor is the same structure, a short session built on two real permits from the same week, a simple fence permit Permitto approved correctly, and a garage to office conversion permit it approved without ever flagging the required electrical inspection. The risk runs the same both ways: too technical and the room glazes over at zoning code citations, too soft and it just repeats the "permits now move faster" line that started the confusion. Keep out draws the same line: no pitch to expand Permitto's contract, no walkthrough of its document parsing model.

Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip straight to the anchor, the two tickets and the one question they leave with.
Cost: no time on the leadership calendar for a standing session at all. Run it once, as a single fixed session tied to the next real decision on the table, rather than a recurring meeting, the calibration still lands, it just isn't refreshed automatically.
The model got better, for real: say Deskmend's multi issue accuracy climbs to 80 percent. The session still matters, because leadership still needs the vocabulary to ask "which kind of ticket is this," even when the honest answer is "the safer kind now." A better model doesn't teach anyone how to ask a sharper question on its own.

Where people run it wrong.
They let an engineer take over the room and explain how confidence scoring actually works, and the session quietly becomes the technical tutorial it was built to avoid.
They pick two tickets from months apart instead of the same week, and leadership assumes the bad one has since been fixed, when nothing about it has changed.
They end the session with a slide asking for more AI budget, and every calibrated question from the last forty minutes gets read backward as a sales pitch.

How to use it live. Before answering a "how would you run a literacy session" question cold, ask yourself one thing: what two real, current outputs would you actually put in front of the room, one good and one wrong. Naming those two things, not a value word like "understanding" or "alignment," is usually exactly what the question is listening for.

06Flashcards (tap any card to flip it)

07Check yourself Score: 0 / 0