The macOS AI assistant that hears both sides of the call

Those assistants all send a single audio track (your microphone) to their speech backend. If the other person is talking on a Zoom call, a FaceTime call, or even a YouTube video the assistant has no idea. Fazm opens a stereo WebSocket to Deepgram with channels=2 and multichannel=true so the microphone goes on channel 0 and system audio goes on channel 1, and every transcript segment comes back with a channelIndex telling the model which speaker said it. The parameters that make this work are at TranscriptionService.swift lines 276, 283, 290, and 291, and the per-segment label is at line 42.

TranscriptionService.swift line 42, inside the TranscriptSegment struct. The declaration is `let channelIndex: Int // 0 = mic (user), 1 = system audio (others)`. The paired constructor argument is declared at line 99: `private let channels: Int // 2 = stereo (mic + system), 1 = mono (mic only for PTT)`. The default value is set at line 141: `init(apiKey: String, language: String = "en", vocabulary: [String] = [], channels: Int = 2)`. The default is 2, meaning a freshly constructed session is stereo unless the caller explicitly switches to mono for push-to-talk.

diarize=true alone is a speaker-embedding model that runs over a single audio stream and tries to split it into Speaker 0, Speaker 1, Speaker 2, and so on by clustering voiceprints. That is fragile when two people have similar voices or when one is talking through a compressed codec. Fazm combines diarize=true with channels=2 and multichannel=true, so Deepgram runs ASR independently on each physical channel first and then applies diarization inside each channel. The mic channel is guaranteed to be you because it came from your input device. The system-audio channel is guaranteed to be everyone else because it came from the output device. The wire-level guarantee is stronger than voice-embedding clustering alone.

Because AVAudioEngine creates an aggregate device behind the scenes when you pin it to an input, and an aggregate device changes the routing of the default output. On a Mac with AirPods or a Bluetooth headset, that switches the output from A2DP high-quality music mode to SCO phone-call mode, which cuts output bandwidth to roughly 8 kHz. For an AI assistant that is listening to a meeting, that is the wrong direction. AudioCaptureService.swift lines 5 to 8 document this explicitly. The service uses AudioDeviceCreateIOProcID on the default input device directly, which is a lower-level CoreAudio API that does not create an aggregate device and leaves the output path alone.

Deepgram nova-3. TranscriptionService.swift line 94: `private let model = "nova-3"`. nova-3 is Deepgram's current production real-time ASR model, notable for two things Fazm relies on. First, it supports the `keyterm` parameter, which lets Fazm inject custom vocabulary (for example a user's company name or a product SKU) directly into the acoustic model rather than applying replace rules after the fact. The vocabulary is appended to the URL at lines 295 to 297. Second, nova-3 handles the stereo multichannel path with low tail latency, which is what the 3,200-byte 100ms buffer frames assume.

It is the frame size TranscriptionService ships to Deepgram. Line 119: `private let audioBufferSize = 3200 // ~100ms of 16kHz 16-bit audio (16000 * 2 * 0.1)`. At 16 kHz sample rate and 2 bytes per sample, 100 milliseconds of audio is 3,200 bytes. The service buffers raw PCM until the buffer reaches that threshold, then sends the chunk via webSocketTask.send in a single WebSocket data frame (sendAudio at lines 209 to 224, sendAudioChunk at lines 239 to 249). 100 ms is short enough for live transcription to feel real-time and long enough to amortize the WebSocket frame overhead.

Three mechanisms. First, a keepalive task pings every 8 seconds (keepaliveInterval at line 108) so intermediate proxies do not idle-close the connection. Second, a watchdog task checks every 30 seconds that data or keepalive successes have arrived within the last 60 seconds (watchdogInterval and staleThreshold at lines 114 and 115) and forces a reconnect if the socket has gone silent. Third, the service auto-reconnects up to maxReconnectAttempts = 10 with backoff (line 103). This is why Fazm's voice loop can run through a tunnel or a flaky hotel Wi-Fi and recover automatically.

Deepgram's `replace` parameter applies find-and-replace rules on the server. TranscriptionService.swift lines 9 to 31 define defaultReplacements: 'dot com' to '.com', 'dot ai' to '.ai', 'at sign' to '@', plus file extensions like 'dot json', 'dot ts', 'dot swift'. The rules are only appended when the language is English or 'multi' (lines 301 to 305), because spoken forms like 'dot com' are English-specific. This makes URLs and emails render as typeable text instead of words, so a follow-on agent tool can click a link without a regex post-processing step.

Yes. The init signature takes channels as an argument, and the code explicitly notes at line 99 that 1 channel is the mode used for push-to-talk. The floating bar can open a mono session on demand when the user holds a PTT hotkey, and the same Deepgram URL is generated with channels=1 and multichannel=false (line 291 flips based on the channel count). You get the same nova-3 model and the same replace rules, just without the system-audio split. The default stays stereo so that ambient capture during a call works without extra configuration.

The files are in the public Fazm source tree at Desktop/Sources/. Run wc -l TranscriptionService.swift AudioCaptureService.swift and you should see 710 and 933. Grep for 'channelIndex' inside TranscriptionService.swift and the comment at line 42 appears. Grep for 'diarize' and the URLQueryItem at line 283 appears. Grep for 'multichannel' and lines 290 to 291 appear. Grep for 'nova-3' and line 94 appears. Grep for 'IOProc' inside AudioCaptureService.swift and the header comment plus the ioProcID property appear. Every claim on this page is a direct grep away.

It receives it labeled. Deepgram emits separate messages per channel on a multichannel stream, and TranscriptionService parses each message into a TranscriptSegment with channelIndex set from the channel_index field. The segment is handed to the onTranscript callback verbatim, so by the time the agent loop composes a prompt, it already has rows like 'channel=0 text="let me share my screen"' and 'channel=1 text="sounds good, go ahead"'. No voice-embedding clustering is required on the client.

AudioDeviceManager.swift (317 lines) handles device enumeration and device change listeners, so when the user swaps from built-in microphone to AirPods the service reconfigures without dropping the WebSocket. The ioProcID is torn down, the new device is inspected for its native format, a fresh AVAudioConverter is built, and capture resumes. This is how the assistant stays on through a device change in the middle of a call. The header comment on the AudioCaptureService class documents the aggregate-device avoidance rationale that makes any of this safe on a Bluetooth headset.