Voice agent desktop workflow handoff, the three code paths nobody describes

The shape of the seam

Picture the runtime as two coupled systems. On one side a four-state push to talk machine in PushToTalkManager.swift turns the Option key plus a microphone into a final transcript. On the other a chat provider holds the conversation state, drives an Agent Client Protocol bridge, and issues tool calls into MCP servers that walk the macOS accessibility tree, drive the browser, and read Google Workspace. The handoff is the small piece of code that decides what to do with a new transcript when the chat provider is mid-run.

The naive answer is “disable the mic until the run is done.” That loses the whole point of a voice agent. People narrate. They notice the run is heading the wrong way ten seconds into a thirty second tool chain and want to redirect it without losing what already happened. They want to drop a follow up while the agent is still working, knowing it will be picked up next. They want to abort and read what got done. All three are legitimate. All three need a different code path.

The three methods below sit on a single Swift class, ChatProvider, in Desktop/Sources/Providers/ChatProvider.swift. Their names are the API surface; their bodies are where the interesting decisions live. I will quote enough of each that you can search for them in the public repo.

Path 1: enqueue without interrupt

Trigger: user speaks, target session is busy, user (or UI default) chose queue mode. Effect: the new utterance lands as a queue chip in the floating bar, the current tool call keeps running, and the moment the run completes the natural drain path picks the next item up.

// Desktop/Sources/Providers/ChatProvider.swift
/// Enqueue a message to be sent after the current query finishes.
/// Does NOT interrupt the current query — it will be picked up automatically.
/// Pass the caller's `sessionKey` so the message runs on the correct session
/// (not the currently-active one, which may belong to a different window).
func enqueueMessage(_ text: String, sessionKey: String? = nil) {
    let trimmedText = text.trimmingCharacters(in: .whitespacesAndNewlines)
    guard !trimmedText.isEmpty else { return }
    pendingMessages.append((
        text: trimmedText,
        sessionKey: sessionKey ?? activeSessionKey,
        userMessageAdded: false
    ))
    log("ChatProvider: message enqueued (\(pendingMessages.count) pending)...")
}

The important detail is the second argument. The queue is keyed by session, not globally. A second floating bar in another window can be running its own tool call on its own session, and the queue entry remembers which one it belongs to. When that session’s drain path fires, it filters by sessionKey and pulls only messages tagged for it. This is what makes concurrent runs in two windows actually concurrent, rather than one global queue everyone fights over.

The other detail is what is missing: there is no call to acpBridge.interrupt. The current tool call is untouched. The browser session, the macOS app the accessibility tree was reading, the partial assistant text already streamed back, all of it stays where it is. Queue mode is the minimum-violence option.

Path 2: interrupt and replace

Trigger: user speaks, the run is wrong, they want it cancelled and replaced now. Effect: the current ACP tool call is unwound, partial assistant text is preserved, the new message is inserted at the front of the queue, and the drain path runs it on the same session immediately.

// Desktop/Sources/Providers/ChatProvider.swift
/// Interrupt the current query and send a message immediately.
/// If the AI is idle, sends the message directly without interrupting.
func interruptAndSend(_ text: String, sessionKey: String? = nil) async {
    let trimmedText = text.trimmingCharacters(in: .whitespacesAndNewlines)
    guard !trimmedText.isEmpty else { return }
    let targetKey = sessionKey ?? activeSessionKey

    // Drop any duplicate queued copy of the same message on this session
    if let existingIdx = pendingMessages.firstIndex(where: {
        $0.text == trimmedText && $0.sessionKey == targetKey
    }) {
        pendingMessages.remove(at: existingIdx)
    }

    // If THIS session isn't busy, send directly as a follow-up
    let targetBusy: Bool = {
        if let key = targetKey { return sendingSessionKeys.contains(key) }
        return isSending
    }()
    if !targetBusy {
        await sendMessage(trimmedText, isFollowUp: false, sessionKey: targetKey)
        return
    }

    // Insert at front of queue, then interrupt this one session only
    pendingMessages.insert((
        text: trimmedText,
        sessionKey: targetKey,
        userMessageAdded: true
    ), at: 0)
    if let key = targetKey {
        await acpBridge.interrupt(sessionKey: key)
    } else {
        await acpBridge.interrupt()
    }
}

Three things in here are easy to miss. First, the dedup. If the user tapped “Send Now” on a message that was already in the queue chip, the method removes the queued copy first so the bridge does not execute the same instruction twice. Second, the target-busy check is per session. The global isSending flag is true if any session is sending; what matters here is whether this one is. A busy sibling pop-out should not push this session through the interrupt path when this session is actually idle.

Third, the interrupt is scoped. The session-keyed call to acpBridge.interrupt(sessionKey:) is the difference between cancelling one tool call and killing every concurrent run in the app. A user who has two pop-out chats running and interrupts one of them should still be able to watch the other one finish.

Path 3: stop without replace

Trigger: user does not say anything new. They tap the stop control or press a global shortcut. Effect: the current run is cancelled, partial output stays on screen, the queue is preserved, the upstream session ID is not rolled.

// Desktop/Sources/Providers/ChatProvider.swift
/// Stop the running agent, keeping partial response
func stopAgent() {
    guard isSending else { return }
    isStopping = true
    pendingCountAtStop = pendingMessages.count
    log("ChatProvider: user stopped agent, sending interrupt...")
    Task { await acpBridge.interrupt() }
    // Result flows back normally through the bridge with partial text
}

/// Stop the running agent for a specific session only.
/// Other concurrent sessions continue.
func stopAgent(sessionKey: String) {
    guard sendingSessionKeys.contains(sessionKey) else { return }
    Task { await acpBridge.interrupt(sessionKey: sessionKey) }
}

The reason there are two methods is the same reason interruptAndSend has a sessionKey argument: a stop on one window’s run should not touch another window. The global stopAgent is the “everything I have visible right now” gesture; the keyed variant is the per-session button on a specific floating bar.

The pendingCountAtStop snapshot is a small thing that matters. After the bridge unwinds and posts the partial response, the provider knows how many queued items existed at the moment of stop. If new items were added during the unwind (the user kept talking), they are not silently flushed. The user explicitly chose to stop the run, not the queue.

What survives a handoff

The single biggest source of bugs in this part of the system is treating an interrupt like a process kill. It is not. Cancelling an ACP tool call unwinds the in-flight call and nothing else. Everything below sticks around and is the reason the user can pick up where the run left off rather than starting over.

The one thing thrown away is the specific tool call that was mid-flight. If the model was three steps into a five-step run, the first three steps and any partial output of the fourth are still visible, the run is just paused. This is what lets a user say “wait, do that with the August invoice instead” without losing the July invoice work that already finished.

The voice side, in two constants

A handoff is only as clean as the voice machine producing the trigger. The push to talk side runs a four-state machine in PushToTalkManager.swift: idle, listening, lockedListening, finalizing. Two constants in that file carry most of the ergonomics:

// Desktop/Sources/FloatingControlBar/PushToTalkManager.swift

// Double-tap detection
private let doubleTapThreshold: TimeInterval = 0.4    // line 40

// Safety: max recording duration to prevent stuck PTT (5 minutes)
private let maxPTTDuration: TimeInterval = 300        // line 65

// Debounce: minimum interval between PTT activations to prevent
// rapid start/stop cycling that can crash the audio subsystem.
private let pttDebounceInterval: TimeInterval = 0.5   // line 71

The 0.4 second threshold is what separates “hold to talk” from “double tap to lock.” Tap Option twice within 400 ms and you are in lockedListening, which is the right mode for narrating a follow up while the desktop run is going. Hold Option for longer than 400 ms and you are in plain listening, which finalizes the moment you let go.

The 5-minute cap is a safety against a stuck modifier; the 500 ms debounce stops a fluttering Option key from cycling the audio subsystem, which on earlier builds could deadlock CoreAudio. None of these are exotic numbers, but they are the difference between a voice input that feels disposable and one that you can actually run all day alongside a busy desktop agent.

When each path is the right one

Use enqueue when the new utterance is a follow up, not a correction. The run is doing the right thing, you just want to add “and then file this in the 2026 folder” without breaking flow. The user experience is a queue chip with a count badge; the runtime experience is a tuple appended to pendingMessages.

Use interrupt and replace when the run is on the wrong path. The user said “the August invoices” and saw the agent open July; they want to redirect now. The user experience is “send now” on a queued message or a long press of the PTT key with a configured interrupt mode; the runtime experience is a single acpBridge.interrupt(sessionKey:) plus a front-of-queue insert.

Use stop when the user wants to think. They saw enough partial output to know the answer or the next step is not what they expected, and they want to read what got done before saying anything else. The user experience is a stop button on the bar; the runtime experience is the same interrupt without the queue insert.