Explain the tradeoff between showing a fast partial answer and a slow complete one.
PICK · latency budgets and UX tradeoffs
Northlatch Software built Corestep so a developer could type half a function name and jump straight to its real definition, anywhere in a two million line monorepo. A fast local guess and a slow, fully checked answer used to share one screen with no way to tell them apart. That is what sent three engineers into the wrong file in a single quarter.
The direct answer
Show the fast, roughly ranked partial match the instant a developer types, every keystroke, and never make them wait on it. Let the complete, fully ranked answer arrive a moment later and reorder the list in place, but mark it "still refining" until it lands, so a fast confident guess never gets mistaken for the finished one. On Corestep's own numbers, the partial pass answers in about 18 milliseconds and gets the right definition on top less than half the time; the complete pass takes closer to 900 milliseconds and gets it right 92 percent of the time, a wait a background job barely feels and a person mid keystroke should never have to sit through in silence.
Do this, in order
Show the fast partial match the instant a developer types, every keystroke, never block on it.Why: the wait itself was never the real complaint, so this can never be the thing that gets cut.
Reorder the list in place once the complete answer lands, instead of clearing the screen while it waits.Why: a blank panel is what actually taught developers to stop trusting Corestep, not the extra 900 milliseconds itself.
Mark a not-yet-final result "still refining" and add a half beat of friction before it can be jumped to.Why: this is exactly what stopped a rushed Enter from landing on an unranked guess.
Fall back to the live partial pass, and flag it, whenever the complete pass's index for the open file is known stale.Why: a freshly edited file has no fresh embedding yet, so the smart pass can be confidently wrong right when it matters most.
Track wrong jump incidents by name, not just how often people abandon a search.Why: abandonment already shows up in a dashboard on its own, a wrong jump does not, it only shows up if someone goes looking.
Revisit the whole split once the complete pass reliably clears about 150 milliseconds.Why: below that bar there is nothing left to protect by showing a rough guess first.
How to answer this, stage by stage
Nobody is testing whether you know a fast answer beats a slow one, or the other way round. They are testing whether you will price the exact moment a rough guess stops being free, or just say "show it fast, fix it later" and stop there.
1
Reframe the question before answering it
Say it like this
"Before I pick a side, this isn't really speed versus accuracy. It's a question about which wrong answer you can afford to show somebody, the cheap kind or the expensive kind."
Why this works
Stops the answer turning into "fast is better" or "accurate is better," which are both true and neither is a decision.
2
Scope it to one product and one owner
Say it like this
"Let me ground this in one product. Corestep is a live code search tool, built by Northlatch, that shows a developer the real definition of a function while they're still typing its name. Prosper Achufusi owns the search backend end to end, from the instant local guess to the full ranked answer."
Why this works
One product turns a debate about search architecture into a real problem with real milliseconds attached.
3
Position: name the pick before any of the reasoning
Say it like this
"I'd show the fast, rough match the second someone starts typing, always. I'd let the slower, fully checked answer arrive after and quietly correct the list, but I'd never let it pretend to be finished before it actually is."
Why this works
Naming the pick first stops this sounding like "streaming is good" or "ranking is good," which are both true and neither is an answer.
4
Impact: say who actually feels each kind of cost
Say it like this
"A developer typing fast feels the rough guess instantly, every single keystroke, and feels the list shift under their cursor a second later. Feels normal, they expect it. The other cost lands on whoever hits Enter a beat too early, weeks apart, one person at a time, and doesn't find out for twelve minutes that they're editing the wrong function."
Why this works
Splitting the cost by who actually notices it turns "it's a bit slower" into a real tradeoff with two named moments.
5
Cost asymmetry: say which error is cheap and which is hidden
Say it like this
"A list re-sorting itself is the cheap, visible error. Everyone sees it happen and it fixes itself in under a second. A wrong jump on an unlabeled guess is the hidden, expensive one, it looks exactly like a normal confident jump in every log Corestep keeps, right up until someone notices they're staring at the wrong file."
Why this works
This is the heart of PICK. If both sides of a tradeoff cost the same, the real asymmetry has not been found yet.
6
Prove it with the real incident, compressed to four sentences
Say it like this
"Two weeks after a quick streaming patch shipped, three engineers hit Enter on the first thing that appeared, before the real ranked answer had a chance to correct it, and landed in the wrong file. Each one lost about twelve minutes before they noticed. Nothing in Corestep's logs looked wrong, the jump just looked normal. That's what an unlabeled partial answer actually costs."
Why this works
Shows the cost is real, not hypothetical, and exactly what happens once two very different answers get shown the same way.
7
Kill criteria: say what would flip the pick
Say it like this
"If the complete pass ever gets reliably under about 150 milliseconds, on real repo load, not a quiet benchmark, I'd just wait for it every time. There's no asymmetry left to protect once both paths are basically the same speed."
Why this works
A pick that cannot say what would change it is not really a decision. It is a preference wearing a decision's clothes.
8
Close on the rule, not the last number crunched
Say it like this
"So: show the fast guess the instant someone types, always, correct it in place once the real answer lands, and never let the fast one go unlabeled long enough to be mistaken for finished."
Why this works
Ending on the rule keeps this sounding like judgment, not a latency chart read out loud.
Let's learn
Corestep is a search box built into the code editor. A developer starts typing a function name, and Corestep shows them where it's really defined, anywhere in the codebase, before they finish typing it.
Before Northlatch added any ranking on top, Corestep matched on plain text alone, a fast search across the whole repository for anything that looked like the letters typed so far. It answered in about 18 milliseconds. Fast, but once the codebase grew past two million lines and dozens of services reused the same function names, that plain match returned a flat list of about 11 candidates on average, sorted by nothing more than file path, and a developer spent about 9 seconds scanning it by eye to find the real one.
Now Corestep runs a second pass behind the first one. A model reads the actual function bodies against what the developer has open and how the code actually calls itself, then reorders that same list of candidates. The right definition lands on top 92 percent of the time instead of well under half. That second pass costs real time: about 900 milliseconds typically, and up to 2.1 seconds on a slow day.
One keystroke starts two answers on two very different clocks. The fast one shows up first. The slow one quietly corrects it.
Knowledge spark: what does the ranking pass actually do?
The fast pass just compares letters. The ranking pass reads the actual code in each candidate function, together with the file the developer already has open, and asks which one they were really calling. That joint read is slower, and it catches things a plain letter match cannot tell apart.
Here's the part that isn't the real problem: those extra 900 milliseconds. Corestep's very first version behind that second pass showed nothing at all while it worked, an empty panel, because it waited for the one complete answer before drawing anything. A developer who glanced away for even a second came back to a blank box and assumed the tool had frozen.
We did not lose 900 milliseconds. We lost the moment somebody trusted what was on the screen.
A list re-sorting itself costs a glance and fixes itself. A wrong jump costs nothing anyone notices, until somebody does.
The cost, by the numbers: latency, partial pass vs complete pass
Partial passComplete pass
The partial pass barely leaves the floor at either percentile. The complete pass pushes a typical answer past half a second, and the slow end past two seconds. A background job never feels that. A person mid keystroke does, if the screen stays blank while it happens.
Once developers learned that Corestep sometimes just goes blank, the habit generalized. People started grepping first, out of reflex, even on lookups Corestep would have nailed instantly, because grep never leaves you staring at nothing. A tool built to save time became the second thing people tried, after the workaround already worked. And when Northlatch's team patched the blank screen by showing the fast guess immediately and quietly swapping it out later, with no signal that it wasn't final yet, three people in two weeks hit Enter on a guess mid swap and landed in the wrong file.
The decision that mattered
Corestep now shows the fast guess instantly on every keystroke, marks it "still refining" until the complete, ranked answer lands, and adds a half beat of friction before a not-yet-final result can be jumped to. Two answers, both true, each one saying plainly which one it is.
The choice I would take back: Corestep's very first release, the one built for a forty person pilot team, rendered the results panel only after the ranking model answered, one query, one round trip. It was simpler to build, and simpler to test, since there was no partial state to design for. Nobody decided to accept a blank screen. It was just the natural shape of doing one call.
What I'd leave alone: the nightly job Corestep itself runs to build cross reference documentation calls the exact same ranking pipeline, complete pass included, and nobody has ever needed to speed that up. No developer is staring at a screen waiting on a report that gets read the next morning.
The lesson: a slow ranking pass was never the mistake. It's a genuinely hard problem across two million lines of code. What broke trust was making a person stare at nothing while it ran. Show them a rough answer immediately, always, correct it in front of them once the real one lands, and a two second wait stops costing anything at all.
Now here is the same thing as a story
The short version sits above. Read below for why a new hire's one offhand question changed how Corestep shows an answer it isn't sure of yet.
Prosper Achufusi has spent five years keeping Northlatch's own monorepo searchable. Before Corestep had any ranking at all, he was the person teammates pinged directly when a plain search turned up eleven files and nobody could tell which one actually mattered.
For its first two years, Corestep's plain text match was enough. The repo was small, function names rarely collided, and by nine every morning the whole team had it open in a side panel, typing half a name and landing on the right definition before their coffee cooled.
Then the repo grew past two million lines, stitched together from a dozen services that had each grown their own "validate" and their own "parseInput." Prosper built the ranking pass to fix it, and it worked, when it finished. But it changed the shape of the wait, and the habit thinned in three small beats nobody flagged at the time. Big lookups started taking most of a second to answer instead of a blink, and the panel sat empty the whole time. A few engineers started keeping a terminal tab open next to Corestep, just in case. And grepping first, before even finishing a query in Corestep, became the reflex for a growing slice of the team.
Three small beats, months apart. None of them looked like a crisis on their own.
The trigger was small. Ruthanne Bierstadt, three weeks into the job, asked Prosper during a pairing session: "is Corestep supposed to just, go blank for a while?" Prosper didn't have a real answer. He'd stopped noticing the blank panel himself months earlier.
Prosper pulled a week of telemetry that night. On any query that needed the full ranking pass, 40 percent were abandoned, the developer's focus moved to another window, before Corestep ever drew a result. He shipped a quick fix two days later: show the fast partial match immediately, swap it for the ranked one when it lands. No blank panel anymore. He called it done.
Corestep was never wrong to wait. It was wrong to make a person stare at nothing while it did.
It wasn't done. Two weeks later, an audit Prosper ran for an unrelated reason turned up three cases where an engineer had hit Enter on that fast guess in the half second before the real answer swapped it out, landing in the wrong file entirely. Nothing about it looked broken in any log. The jump just looked like any other jump.
Prosper's own audit, run two weeks after the quick patch. Twenty two jumps were fine. Three were not, and nothing had flagged them.
It was never really about the 900 milliseconds. Prosper had a fast answer and a slow one, and for a few weeks in between, there was no way for anyone looking at the screen to tell which one they were looking at.
The decision Prosper would take back happened two years earlier, in the week he shipped the ranking pass. "One clean answer is simpler than two," he said in the review that approved it, and for a small enough repo that was true enough that nobody pushed back. Nobody in that room asked what happens once the second pass takes most of a second and somebody is still typing.
Run the audit again with the label on: the fast guess still lands in 18 milliseconds, but now it carries a small "still refining" tag, and Enter needs a half beat longer to fire while that tag is showing. The following month, Prosper reran the same kind of trace, thirty more jumps, picked at random. Zero landed in the wrong file. Nobody lost twelve minutes to a guess that hadn't finished thinking.
Same kind of audit, rerun after the label and the friction shipped. The fast pass did not get slower. It just stopped pretending to be finished.
One design gave Corestep a single, honest answer that showed up too slowly to be honest about being late. The other gives it two answers, both true, each one saying plainly which one it is.
What I'd tell myself in that first review: one answer isn't simpler. It's just a decision about a wait you haven't been asked to explain yet.
The four letters that kept this from being a guess
This is not a speed question dressed up as a technical one. PICK is what keeps "show it fast, fix it later" a real, defensible call instead of a habit nobody ever examined.
PPosition. What's the actual call, stated before any reasoning?
Show the fast partial match the instant someone types. Let the complete answer arrive after and reorder in place, clearly marked until it's final. The partial pass has nothing to lose by going first. The complete pass has nothing to gain by pretending to be quick.
State the position first, or the reasoning that follows sounds like a search for whatever pick was already assumed.
IImpact. Who feels each kind of cost, and in what units?
A developer feels the fast guess every keystroke, in milliseconds, and feels the list settle a second later without thinking about it. Feels normal. A different developer, weeks apart, feels the other cost as twelve lost minutes staring at the wrong function, wondering why nothing about it looks wrong.
Naming who feels which cost turns "it's a bit slower" into a real tradeoff instead of a guess about whether anyone will mind.
CCost asymmetry. Which error is cheap, and which is hidden?
A list that re-sorts itself is the cheap, visible error, seen and forgotten in under a second. A wrong jump on an unlabeled guess is the hidden, expensive one, it looks exactly like a normal confident jump in every log Corestep keeps. Optimize against the hidden one, not the one that's already easy to see.
This is the hardest step, and the one most answers skip. If both sides of a tradeoff cost the same, the asymmetry has not actually been found yet.
KKill criteria. What evidence would flip the pick?
A complete pass fast enough to clear about 150 milliseconds at real repo load, not a quiet benchmark. Below that bar, there's nothing left for a rough guess to buy anyone.
A pick that cannot say what would change it is not really a pick. It is a preference dressed up as a decision.
The kill line, charted: complete pass latency across infra iterations, against the live search threshold
Where the complete pass sits todayWhere the roadmap is aiming
Today's complete pass adds about 900 milliseconds, well above the line. Each planned infra change trims it, and the third one is the first to cross the 150 millisecond mark. Until it does, the split stands.
Two things worth stating outright, since this is where the real judgment sits. The alternative Prosper's team considered and dropped was widening the fast partial pass itself, from the open file and recent tabs to the developer's entire local checkout, instead of building a separate ranking pass at all. It lost because a wider plain text match is still just a plain text match; on Corestep's own 500 example set of hand checked lookups, widening the scope moved top pick accuracy from 46 percent to about 58 percent. Adding the ranking pass moved it to 92 percent. More text was never the mechanism, reading the code together with how it's actually called was. The AI specific failure worth naming by name is cold start: the ranking pass leans on an index of code embeddings, and a file someone just edited has no fresh embedding yet, so right when a developer is looking at their own newest change, the smart pass is most likely to be confidently wrong. The guardrail is keeping the fast partial pass as a permanent floor under the ranked one, always live off the file on disk, and flagging a result "reindexing" whenever the embedding for the open file's package is known stale, so a stale complete answer never quietly outranks a fresh rough one.
And if you want to be sure it really works, try it somewhere else
Same four letters, a furnace nameplate instead of a function name, and this time the hidden cost isn't a wrong file, it's a part ordered for the wrong machine.
Coilfinder is a parts lookup app Bramtech Field Tools built for HVAC technicians. A technician photographs or types a partial model number off a unit's serial plate, and Coilfinder shows which filters, capacitors, and control boards actually fit it. Silvaine Odhiambo runs field dispatch for the crew that piloted it. Coilfinder's cached catalog, the models a technician has looked up before, answers in about 25 milliseconds. The full manufacturer cross reference, which catches regional variants and parts that got quietly discontinued and replaced, calls a slow third party parts database and takes 1.5 to 3 seconds.
Coilfinder's pipeline has a box Corestep never needed, a gate that only the order action has to pass through.
The decision Silvaine's team would take back
Coilfinder first let a technician tap "order" the instant the cached match appeared, the same as any shopping app's autocomplete, since nobody wanted the order button waiting on a slow outside database. That held while the cached catalog only covered common units. Once older and regional units multiplied, the cached guess started confidently matching a visually similar part from the wrong subseries often enough to become a real cost, a technician orders the wrong capacitor, doesn't find out until the truck is back at the shop, and the same unit needs a second visit days later.
Same rank as before, a different lever: for Corestep, the hidden cost was a wrong file. For Coilfinder, it's a wrong part in a technician's hand, and a second truck roll to fix it. The fix isn't a labeled tag and a half beat of friction this time, since nobody is watching a list settle mid thought, they're deciding whether to drive to a supply house. It's a gate: browsing and typing stay instant off the cached catalog, but the "order" action itself waits for the full cross reference to clear, every time, no exceptions.
Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip straight to it, show the fast guess the moment someone starts typing, gate only the action that can't be undone behind the slow, complete check, and never let a rough guess pretend to be the final word.
Cost: there's budget this quarter for either a faster cross reference call or a bigger cached catalog, not both. The faster cross reference wins for Coilfinder specifically, because a wrongly ordered part costs a truck roll, and a wider cache would only avoid a fraction of that.
The model got better, for real: say the cross reference call gets cut to under a second next year. Real progress, and it narrows the gap. It does not remove the need for the gate, since even a fast wrong guess is still wrong until the real check has run.
Where people run it wrong.
They treat "the fast guess is usually right" as reason enough to let it drive an action that can't be undone.
They fix a slow interactive screen by showing the fast guess everywhere, without gating the one moment a wrong guess actually costs something.
They size a fallback catalog once, at launch, and never revisit it once the long tail of real world cases outgrows it.
How to use it live. Say the real question out loud before naming a side: "before I answer, is this a moment someone is just looking, or a moment they're about to do something they can't take back." That's what decides whether the fast answer is free or expensive.
Flashcards (tap any card to flip it)
1 · THE FRAMEWORK
What framework is this, and what's its one job?
Tap to flip
ANSWER
PICK: commit to a position, then show the asymmetry between the two kinds of error. Built for tradeoff questions like fast versus slow, not a full flip story.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Prosper Achufusi, the staff engineer who owns Corestep's search backend at Northlatch Software, from the first plain text match to the ranked answer on top of it.
3 · THE QUIET HABIT
What did Prosper's team ship at launch that felt safe?
Tap to flip
ANSWER
A ranking pass built as one blocking round trip, so nothing appeared on screen until the complete, fully ranked answer was ready, because it was simpler to build and test as a single call.
4 · THE POSITION
What's the actual position this answer takes?
Tap to flip
ANSWER
Show the fast, rough match the instant someone types, always. Let the complete answer arrive and reorder the list in place, clearly marked until it's final, never silently.
5 · THE OLD DECISION
What decision would Prosper take back?
Tap to flip
ANSWER
Shipping the ranking pass as a single blocking call with no partial state, so a fast typist glancing away came back to an empty panel and learned to distrust it.
6 · THE NUMBER
Fill in the blank: the partial pass answers in about ___ milliseconds. The complete pass takes about ___ milliseconds and gets the right definition on top 92 percent of the time.
Tap to flip
ANSWER
About 18 milliseconds for the partial pass, about 900 milliseconds for the complete one. The gap is real, and so is the accuracy it buys.
7 · THE REPLAY
Same audit, new design, what changes?
Tap to flip
ANSWER
With the "still refining" tag and a half beat of jump friction in place, a follow up trace of thirty random jumps turned up zero wrong files, down from three wrong jumps in the two weeks right after the unlabeled quick patch.
8 · CROSS-PRODUCT TRANSFER
Section 4 answers this same question again for a different product. Which product, and what's the hidden cost there?
Tap to flip
ANSWER
Coilfinder, which matches HVAC unit serial plates to compatible parts. There, a wrong fast guess does not just send someone to the wrong file, it can get the wrong part ordered and cost a second truck roll to fix.
Check yourself Score: 0 / 0
Multiple choice
1. Why does Corestep's complete answer take about 900 milliseconds longer than the partial one, even though both search the same code?
A. It re-downloads the whole monorepo from source control before answering.
B. It reads the actual function bodies against the developer's open files and the real call graph, instead of comparing plain text.
C. It waits for the developer to stop typing for a full second before starting.
D. It has to open a new network connection to the developer's own laptop.
Show hint
Look at the knowledge spark in Section 1.
Show answer
B. Plain text matching just compares characters. The complete pass reads the code and its real callers together, which is slower and also the reason it gets the right definition on top so much more often.
True or false
2. True or false: the three wrong file jumps happened because the ranking pass itself gave a wrong answer.
True
False
Show hint
Look at when in the sequence each of the three jumps actually happened.
Show answer
False. Nothing about the ranking pass was wrong. The jumps happened in the half second before its correct answer had replaced the fast guess, because nothing on screen said the guess wasn't final yet.
Fill in the blank
3. Fill in the blank: on any query that needed the full ranking pass, about ___ percent were abandoned, the developer's attention moved to another window, before Corestep ever drew a result.
Show hint
Look at the telemetry number Prosper pulled the night Ruthanne asked her question.
Show answer
About 40 percent. That's the number that showed the blank panel wasn't a minor annoyance, it was quietly training people to stop waiting on Corestep at all.
Short answer, where it wouldn't matter
4. Name a place inside Corestep itself where waiting for the complete, fully ranked answer costs nothing.
Show hint
Look at "what I would leave alone" in Section 1.
Show answer
Model answer: The nightly job Corestep runs to build its own cross reference documentation. It calls the same ranking pipeline, complete pass included, but nobody is staring at a screen waiting on a report that gets read the next morning.
Short answer, apply it yourself
5. Pick an AI product you use that shows results as you type. Name one place in it where a fast, rough answer beats waiting, and one place in the same product where it's worth making you wait for the complete one.
Show hint
Think of a product with both a live search box and a moment where a wrong guess would cost something real.
Show answer
Model answer: A grocery app's search-as-you-type beats waiting, a slightly off early guess just gets typed past. The same app's "ships today" label is worth a slower, complete check, because a wrong fast guess there means paying for delivery that never comes.
Multiple choice
6. Corestep's monorepo doubles again, to about 4.2 million lines. Is that alone a reason to make the fast partial pass wait for the complete one every time?
A. Yes, a bigger repo always makes the partial pass return more wrong first guesses.
B. No, the partial pass only reads the current file and open tabs, so its speed and rough accuracy do not depend on total repo size, only the complete pass's own latency scales with repo size.
C. Yes, the complete pass's latency grows exponentially once the repo passes a few million lines.
D. No, because doubling the repo automatically halves the complete pass's latency.
Show hint
Reread the kill criteria stage and what the partial pass actually reads.
Show answer
B. The partial pass never looks past the file you have open and your recent tabs, so its 18 millisecond answer doesn't get slower just because the rest of the repo grew. Only the complete pass's search space grows with the repo.
Before you close the answer
Why this works
Tests whether you will price which error is cheap and which is hidden, or just say "faster is better" and stop there. Most candidates never get past treating a fast answer and a slow one as different amounts of the same good thing.
Follow-up traps
"Why not just make the ranking pass itself faster instead of building a whole labeling and friction system?" Response: that's exactly the kill criteria. Once it reliably clears about 150 milliseconds, skip the label and the friction, and just wait for it every time.
"Doesn't the still refining tag and the extra friction just slow the fast path back down, the thing you were trying to avoid?" Response: no, the box itself still fills in under 20 milliseconds every keystroke. The only thing gated is the moment someone tries to jump on an answer that isn't final, and that gate only exists while the tag is actually showing.
If pressed
The fix wasn't a UI redesign. The "still refining" state is a single boolean read off the result object, true until the complete pass writes back, and the half beat of jump friction only activates while that boolean is true, so shipping it cost about two days of engineering, not a new architecture.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
