Coffee Is Cancelled: What an AI-Assisted Coding Interview Really Tests

At one of my interviews I was told I could use AI. Some companies go further: no typing code by hand, everything goes through the agent. In my rounds I got a short spec, a few screenshots and about an hour to build an app from scratch. Bring your own tools, use whatever you know.

You’d think you hand the task to the model and go get coffee. Coffee is cancelled.

An engineer from the team sits across from you, sometimes two. While the agent writes code, the conversation goes on: you explain what you’re doing and why, and they mostly stay quiet and listen. They might ask about the language and the platform, about decisions in real projects, about how you work with AI. Why did you write the prompt that way? How do you check the result? What do you use RAG for? What helps you keep the model in check?

On a normal workday I can watch the agent and add clarifications as it goes. In the interview I’m explaining something to the person across the table at that exact moment. Meanwhile the agent finishes, and I have to read the result, decide what’s next and write the next prompt. There’s rarely time for a calm review. You skim the code, make the call fast, and the conversation doesn’t stop.

TL;DR

• AI rounds are being added on top of classic interviews, not instead of them. In public candidate reports, the share of LeetCode-style questions stayed near 30% from Q1 2025 to Q3 2026 while mentions of AI went from 1.6% to 7.2%.
• It’s not just Meta. DoorDash says an AI session replaces its coding rounds, Google is piloting one, Canva expects candidates to use AI, and candidates report the same at Shopify and Rippling.
• What these rounds really test: setting a task, steering and reviewing an agent, and defending trade-offs out loud.
• My two costliest mistakes: losing about ten minutes to a test-diagnostics pause I didn’t know about, and accepting the first design the agent offered without asking for alternatives.
• Before the round: ask the recruiter for the rules, put your test command (with -collect-test-diagnostics never) into the agent’s rules, check the project’s isolation settings, and ask the agent for two or three approaches with costs.
• The downsides are real: silent interviewers, contradictory rules between companies, and no independent proof that the format predicts job performance.

Over my job search I’ve done several rounds like this at senior level: most of them building an app, a couple debugging an existing project. They were called AI Coding Assessment or AI Debugging Assessment. Some required working only through the agent, others let me do whatever was convenient. I passed most of them. One I most likely failed (or the one right after it in the same loop; I got no detailed feedback), and that one gets the most useful part of this article. Passing a round, by the way, is not an offer. It’s still my favorite technical interview format.

My position is simple: the AI round sits on top of the classic interview and doesn’t replace it. You hardly type any code, but you do manage an executor that types faster than you and never doubts itself.

The examples are from iOS, but most of it applies anywhere. The downsides of the format get their own section.

Why they let you bring an agent

#

The old format broke

#

In early 2024, interviewing.io ran an experiment: candidates were asked to use ChatGPT in interviews without telling the interviewer. On verbatim LeetCode questions they passed 73% of interviews, on lightly modified ones 67%, on original questions 25%. Without AI, the platform’s regular mock interviews had a 53% pass rate. Not one participant was caught (interviewing.io).

---
config:
  themeVariables:
    xyChart:
      backgroundColor: "transparent"
      plotColorPalette: "#2a78d6"
---
xychart-beta
    title "Interviews passed, % (three right bars: ChatGPT used secretly)"
    x-axis ["No AI", "Verbatim", "Modified", "Original"]
    y-axis "%" 0 --> 100
    bar [53, 73, 67, 25]

Small sample: 32 usable interviews (11 verbatim, 9 modified, 12 original). The control comes from the platform’s regular mock interviews, mostly LeetCode-style. Still, the pattern is plain: the more standard the question, the better it’s solved without a human.

In June 2025 Canva ran its old algorithm and data-structure questions through AI models, which solved them in seconds. The company reworked its interviews: it now expects backend, ML and frontend candidates to use AI tools, and sets harder, more ambiguous problems that assess how they clarify requirements, steer the AI, and verify and debug the result (Canva Engineering).

For fifteen years we tested who remembered Dijkstra’s algorithm best, and then it turned out that ChatGPT remembers it best.

Who allows what

#

There is no single policy, and the evidence behind each row is uneven. DoorDash, Canva and Datadog say it on their own pages; for Shopify and Rippling I only have candidate reports, and for Cognition a CEO interview. The status column says which is which. This is what’s public as of early November 2026:

Sources: TechRadar, interviewing.io and Exponent on Meta; Business Insider on Google (May, October); Datadog’s guidelines; DoorDash on its own format; candidate reports on Shopify (Hello Interview) and Rippling (LeetCode Discuss); OfficeChai on Cognition.

LeetCode is still here

#

On October 30, 2026, Interview Query published an analysis of 59,738 public candidate reports on technical interviews, January 2023 to September 2026, across a fixed panel of 639 companies. Mentions of AI in those reports: 0.4% in Q1 2023, 1.6% in Q1 2025, 7.2% in Q3 2026 (Interview Query).

---
config:
  themeVariables:
    xyChart:
      backgroundColor: "transparent"
      plotColorPalette: "#2a78d6"
---
xychart-beta
    title "Interview reports mentioning AI, %"
    x-axis ["Q1 2023", "Q1 2025", "Q3 2026"]
    y-axis "%" 0 --> 10
    bar [0.4, 1.6, 7.2]

The share of reports with algorithmic, LeetCode-style questions barely moved in a year and a half: 30.2% in Q1 2025, 30.5% in Q3 2026.

---
config:
  themeVariables:
    xyChart:
      backgroundColor: "transparent"
      plotColorPalette: "#eb6834"
---
xychart-beta
    title "Reports with LeetCode-style questions, %"
    x-axis ["Q1 2025", "Q3 2026"]
    y-axis "%" 0 --> 40
    bar [30.2, 30.5]

Judging by these reports, AI showed up next to LeetCode, not in its place. LeetCode just isn’t alone in the room anymore.

Caveats: a mention of AI in a report is not the same as a skill being assessed, and rounds with an AI assistant are still rare: 2 of 5,025 reports in Q1 2025 and 37 of 3,368 in Q3 2026 (0.04% to 1.1%; the source itself calls these figures a floor). Also, reports are written by people who chose to write them. That there will be more such rounds is my forecast: I found no reliable time series for the share of AI rounds.

What the round tests

#

This is how I see the difference. It comes from my own rounds and from what the companies say about their changes.

In my earlier article, They Pulled the Ladder Up from Below, AI removed the bottom rungs. Here, it seems, a top rung appeared: what a tech lead used to do (set the task, check the work, own the result) is now expected of you within one hour of an interview.

An hour on the clock

#

Here is my typical hour. Real rounds varied, so the chart uses the midpoints of my ranges.

gantt
    title A typical round, minutes
    dateFormat  HH:mm
    axisFormat  %M
    section Me and the interviewer
    Questions and clarifications      :a1, 00:00, 5m
    Decisions and trade-offs          :a2, after a1, 5m
    Prompt and plan                   :a3, after a2, 6m
    Talking while the agent writes    :a4, after a3, 15m
    Review, run, fixes                :a5, after a4, 13m
    Wrap-up and my questions          :a6, after a5, 16m
    section Agent
    Drafts the plan                   :b1, 00:10, 2m
    Implementation                    :b2, 00:16, 15m
    Fixes and follow-ups              :b3, 00:31, 10m

For the first ten minutes or so the agent isn’t working at all, and everything rests on me. The split attention starts later, during implementation, and that’s where my worst stories come from.

Pin down the task first

#

In implementation rounds I usually got a very short spec and a few screens. The task often circled around the company’s main product. Data might sit in a JSON file with the network layer to be simulated; a couple of times there was a live API.

Screenshots show what should appear on screen. They don’t answer the questions that drive the implementation. Is navigation needed? Should data persist between launches? Where does it come from and who owns it? Is there pagination? What happens on error?

For some questions the interviewer will say “up to you”, and that isn’t a gift, it’s one more question. “It’s what I’m used to” is not an argument for a senior. You can take the familiar approach, but say why it fits these requirements and under what conditions you’d drop it.

Make the model ask you questions

#

I start with a fairly detailed prompt: the core functionality, the screens (usually no more than three), the layering, navigation. And I ask the model to ask clarifying questions about the requirements. Before the agent touches the project I can see how it understood the task and correct the details. I read the model’s questions aloud: if I have enough information I make the call and explain it; if the answer is about product behavior, I discuss it with the interviewer.

A detailed first prompt doesn’t mean ordering the whole app in one go. In daily work I try to keep one iteration, one commit, to roughly three hundred lines. That’s a personal yardstick: beyond it I can no longer review carefully. The number matters less than having a clear purpose for the change and being able to check all of it. Commits save you in interviews: if the agent takes the solution somewhere wrong, you can get back to a working state fast.

Story one: ten minutes of silence

#

It was my very first round of this kind. An app that displays data from a JSON API: a list, and a detail screen when you tap a row. Nothing tricky. I sent a hefty prompt to Opus 5.5 describing the architecture, routing and interface details, first asking for a plan, then sending it off to implement.

I could keep the interviewer busy for the two minutes the plan took. Implementation took about ten minutes (maybe more; stress stretches time), and even the interviewer got uncomfortable. He asked whether I had a guess why it was taking so long and whether I could do anything about it. I went to look at what the agent was doing. It was running tests over and over. I interrupted it and ran the tests by hand. All that time I told the interviewer how I use RAG in a side project. I finished the task and passed the round.

I understood the cause only after the interview. Judging by my measurements on Xcode 27 (not rigorous, one project), after even one failing test xcodebuild collects simulator diagnostics: a red run takes about ten minutes, a green one seconds. Every attempt by the agent to fix a failing test ran into that pause. It’s controlled by the test plan setting “Collect test diagnostics on failure”, and you can pass it as a flag too: -collect-test-diagnostics never, with the command line overriding the test plan (as described in an Apple Developer Forums thread). In my project the results and logs are still kept.

xcodebuild test -scheme MyApp \
  -destination 'platform=iOS Simulator,name=iPhone 17' \
  -collect-test-diagnostics never

The takeaway from this story is boring: I should have put the test command, with the flag, into the agent’s rules in advance. You can’t pause the hour, and the simulator doesn’t care about my timer.

On picking a model

#

A couple of times I made an interview harder for myself by choosing too powerful a model and getting too long a generation. For developing an idea or a document I’ll take the strongest one. For a bounded implementation step it isn’t always needed: a worker doesn’t need to be too clever, but it certainly won’t hurt the foreman. “Smaller is always faster” doesn’t follow from this, though: a powerful model sometimes saves time on fixes. Look at time to a result you managed to verify: generation, reading, build, run, fixes.

Where I look first in review

#

I read the agent’s code roughly like a colleague’s: top down, architecture and protocols first, then the implementation. A mistake at that level spreads. While the agent has written three files it’s cheap to fix; once an app has grown on top of it, it’s a refactor, and a one-hour round won’t have room for one.

Project settings first

#

First thing, I open the project settings. A new Xcode 26 project is created with MainActor default isolation and Approachable Concurrency turned on. With both on, all unannotated code stays on the main actor, and nonisolated async functions no longer hop off it: to move work off the main thread you need @concurrent (an experiment measuring threads).

In my sessions the agent usually doesn’t look at these settings and simulates network delay with Task.sleep. That alone is harmless: Apple states that Task.sleep doesn’t block the underlying thread. But right next to that delay the agent will just as calmly put JSON parsing, and in a project with these settings that also ends up on the main thread. On a toy file you won’t notice; on a large response it can become a hang. The interviewer usually discusses with me why I choose particular settings, and that’s a good conversation: UI stays on the MainActor, the network layer and parsing go off it.

Then the state layer

#

What I catch most often: the agent doesn’t model every state in the view model. A couple of times I caught it after it had already piled up properties for error, reload and empty data, which are easy to get tangled in, and the UI-layer handling started to look like AI slop.

// What the agent writes
var isLoading = false
var isReloading = false
var errorMessage: String?
var items: [Item] = []
// ...and in the View: if isLoading { … } else if let errorMessage { … } else if items.isEmpty { … }

// What I want to see (Item: Equatable)
enum ListState: Equatable {
    case loading
    case failed(String)
    case empty
    case loaded([Item])
}
var state: ListState = .loading

One sentence is usually enough: “model the state as a finite state machine”. Independent axes (say, a refresh in progress on top of loaded data) stay separate enums rather than extra cases in a shared one.

The small stuff

#

In SwiftUI I check state ownership and identity. In my experience models mix up concepts and blend approaches. But what you check is concrete behavior: having ObservableObject and @Observable side by side isn’t an error in itself, Apple allows an incremental migration. Lifetime of state is determined by identity, not by re-creating the View value (Demystify SwiftUI). In storage code I look at isolation: did the agent drag a UI SwiftData context into somewhere it will be used from another isolation domain (a context from the environment is bound to the main actor; see also Dive deeper into SwiftData). Reading isn’t enough, you have to run it: the app builds and shows the screens, and breaks on the second tap.

Debugging: the agent gave an answer and I took it

#

In AI Debugging I try to reproduce the bug myself first: what actions trigger it, how it shows up. The agent meanwhile explores the project and forms hypotheses. Its explanation is only a hypothesis until it’s verified.

My most memorable round (and the one I most likely failed) was about ordering. Here’s a generalized version, without details from the original. Imagine a simplified chat where only your own messages appear in the feed: a user quickly sends several messages in a row. The server assigns message IDs; they grow monotonically with request processing time and are shared across all participants, so our IDs have gaps: 1, 2, 5, 9, the rest went to other people’s messages. Confirmations arrive with varying delays. The feed must show messages in ascending ID order, and what’s already shown can’t be moved. (Real messengers show a sent message immediately and reconcile later; here, for clarity, we show only what the server confirmed.)

The starter code launched a separate task for every send:

func send(_ text: String) {
    Task {
        let id = await server.send(text)                 // confirmations arrive out of order
        messages.append(Message(id: id, text: text))     // the confirmation for id 5 arrived before id 2
    }
}

flowchart LR
    subgraph received["Order of confirmations"]
        direction LR
        A1["1"] --> A5["5"] --> A2["2"] --> A9["9"]
    end
    subgraph expected["Required order in the feed"]
        direction LR
        B1["1"] --> B2["2"] --> B5["5"] --> B9["9"]
    end

Gaps are fine; reordering what’s already shown is not.

The agent’s first answer was a chain of tasks, each waiting for the previous one before showing its result.

@ObservationIgnored private var last: Task<Void, Never>?

func send(_ text: String) {
    last = Task { [previous = last] in
        let id = await server.send(text)                 // the request goes out immediately, in parallel
        await previous?.value                            // but we show only after the previous one
        messages.append(Message(id: id, text: text))
    }
}

Fragments: the server and the model are omitted.

I took it because it was the simplest: no extra logic, and there were no special requirements. The agent offered no alternatives, and I didn’t ask. It was one of my first debugging rounds and I wasn’t ready.

When I went over the task afterwards, I saw that the solution had a hidden assumption. Publication follows the order of sending. But the server assigns IDs in the order requests arrive, and neither the scheduler nor the network promises that requests arrive in the order the tasks were created (for actors themselves this is spelled out in SE-0306: waiting tasks aren’t required to run in queue order). Most likely it would work. But “most likely” was something to say out loud, not to quietly rely on.

If you swap the two lines (first await previous?.value, then server.send) you get strictly sequential execution: the order is guaranteed, but every message waits for the previous one to finish completely. There are several approaches in all:

The table is my reasoning from the task conditions, not a measurement.

What it taught me

#

Since then I assume there are several solutions, and I discuss the trade-offs before implementing, not after. Concretely:

1. After the agent’s first answer I ask it to name two or three approaches with the cost of each.
2. I ask the interviewer what exactly we’re ordering and what should happen when a request fails.
3. I choose and explain out loud what assumption the choice rests on.

flowchart LR
    A["Agent gives a solution"] --> B{"Did I ask about alternatives?"}
    B -->|no| C["Decision accepted silently"]
    B -->|yes| D["2 to 3 options with costs"]
    D --> E["Choice and arguments out loud"]

A simple-looking task under stress can be an illusion: it’s the one where I didn’t ask a single extra question.

And a habit from my early years. When I started, the universal cure for UI bugs was DispatchQueue.main.async { … }. There was even a joke: if that doesn’t help, you’ll have to actually debug. Sometimes the call is appropriate. But if the symptom is gone, it doesn’t mean the bug is fixed: you can only fix what you can explain the cause of.

How to prepare

#

If you asked me where to start, I’d start with questions to the recruiter: can I edit code by hand, is it an existing project or an empty one, which tools are allowed. Permission for AI in one part of the process doesn’t mean permission in all of them: Datadog, for example, has separate rules for explicitly marked AI interviews, and Anthropic goes the other way: no AI in live interviews unless the interviewer says otherwise.

Then I’d check access and limits. Hitting a limit in the middle of a one-hour round is an expensive way to save on a subscription, and a paid plan isn’t a guarantee: limits depend on the model and on how heavy the work is (how Claude usage limits work). Run the whole scenario in advance, build and launch included.

My environment is a familiar one. At the start there are general rules for the agent: ask questions instead of assuming, verify decisions, use current frameworks, don’t count on third-party dependencies in advance; the communication language and terminal work are set up. I wouldn’t copy someone else’s configuration wholesale: in an interview what helps is a tool whose behavior you know. I now also put the test command with the right flag into those rules.

I practice on a small open API: an interface of a couple of screens in an hour, with requirements, architecture, the main scenario and verification. Before a specific company I recreate a couple of screens of its product: implementation tasks are often similar to what the company already builds. Debugging is harder. You can take an unfamiliar open-source project, or ask a model to prepare a project with deliberate bugs, but the exercise needs verifying too: a model can describe a bug that doesn’t reproduce in the code it produced. During practice I say my decisions out loud.

Before a meeting I check five things:

• The task format and the rules on AI are clear.
• The agent, IDE, build and tests work in my usual environment.
• Model access and remaining limits are checked.
• I’ve done at least one similar task end to end against a timer.
• I can name two or three approaches to a task and explain my choice while the agent writes code.

What’s wrong with this format

#

The format has plenty of downsides. On almost all my rounds the interviewers stayed silent: I talk, they listen, and detailed feedback, as a rule, never came. From this I get the impression (it’s an opinion, not a fact) that the companies themselves haven’t yet decided what they want to see in these rounds. The rules are also opposite: some say outright that you must not write code by hand and the model does everything, others let you write however you like as long as the result matches the task. The same skill is tested under opposite conditions, and the only way to find out the conditions is to ask in advance.

You can’t pause the hour: the ten minutes of downtime in my first story were subtracted from it, even though nobody was at fault. Subscriptions, limits and a familiar environment differ from person to person, so to some extent you prepare for these rounds with money. And nobody has shown that such a format predicts job performance better: Canva tells its own story, Google and Meta are rolling the format out themselves, and I haven’t seen an independent check.

Conclusion

#

For a senior there’s a lot that’s familiar in this format: set the direction, explain the constraints, step in when the solution drifts. That’s also why it’s my favorite: it’s closest to the work I was used to long before AI. A large feature calls for a view from above and for working with requirements and concepts, not with variables. I already wrote about the very fast junior who confidently lies in the ladder article, so I won’t repeat it.

I think there will be more such rounds: companies are already trying them and widening their pilots. If an agent is part of your daily work, the format shouldn’t surprise you: requirements, plan, review, debugging and a conversation about trade-offs are the same. What’s unfamiliar is the timer and a stranger you have to explain all this to out loud. That part you can practice.

Just go get the coffee after the interview.

Sources

#