Verifying local AI privacy on macOS, the actual commands

What you are actually testing

Three different things get bundled into the phrase "local AI privacy" and they need to be untangled before the test is meaningful. First, the model: do the weights live on your Mac, or does the prompt go to a model provider over TLS. Second, the input pipeline: do the screen capture, accessibility tree, and microphone stay in process memory, or does any of it get uploaded for processing (transcription, vision, summarization). Third, the metadata: does the app phone home with usage telemetry, error reports, or feature analytics, even if the model and inputs are local.

A truly local app holds all three. A "local-friendly" app (which is what most ship as today) holds the input pipeline and lets you point the model at whatever endpoint you want. A local-in-marketing-only app says all the right things and still ships your transcription audio to a third party. The four checks below tell you which of the three you have, and they take about ten minutes end to end.

Check 1, the source-code grep

For an open-source AI app this is the cheapest possible verifier. Clone the repo, point ripgrep at the source directory, and ask for every URL the binary could possibly talk to. The output is small (most apps have between three and fifteen hardcoded hosts) and you can categorize each one in seconds: model provider, voice provider, analytics, update server, OAuth.

Here is the actual run against the Fazm desktop source. The five hostnames in the output are the entire externally-talking surface of the app. The LLM endpoint is not hardcoded, which is the next interesting fact: it comes from the ANTHROPIC_BASE_URL environment variable, set by Swift in ACPBridge.swift:468-469 when the user fills in the Custom API Endpoint field.

Five rows, three vendors. Two of those vendors (Deepgram for voice, ElevenLabs for TTS) are switchable: the app falls back to nothing when the corresponding feature is off. The first vendor is Fazm's own backend, which exists only to vend short-lived API keys for the other two. The LLM endpoint is the fourth, and it is configurable. That is the entire menu. Anything not in this list is not happening, regardless of what the marketing copy says.

Check 2, the live nettop trace

For a closed-source app, this is the only first-class verifier you have, and it is also the one a non-technical user can run. nettop ships with macOS, takes one command, and shows live per-process external sockets with bytes-in and bytes-out. The trick is to scope it to a single PID so the noise drops to zero, and to start it before you launch the app, so you see the connection-time hosts (analytics SDKs love to phone home in the first two seconds).

Below is what the Fazm process looks like during a normal voice-driven chat session, with the default Anthropic endpoint and Deepgram voice transcription on. Two external rows. Both to model providers. Nothing to ad networks, nothing to analytics, nothing unexpected.

Check 3, the firewall disproof

nettop confirms the positive case (you see the host you expected and nothing else). The firewall is the test for the negative case (the feature breaks the way you predicted). Open Little Snitch (or LuLu, which is free), set a deny rule for every outbound connection from the binary except 127.0.0.1, and try the feature again. The error message is the answer: if it says "could not reach api.deepgram.com" you have learned that voice transcription is cloud-dependent. If it says "rate limit on api.anthropic.com" you have learned the LLM endpoint is still there.

The firewall test is asymmetric. An app that keeps working with all outbound blocked is genuinely local for that feature. An app that breaks gives you a clean error message that names the specific cloud dependency, and the dependency is what you actually wanted to know. Either way you walk out with a true statement, which is more than you started with.

Check 4, redirect the model and re-run nettop

The last check is the one most guides skip, because it requires the app to have a configurable model endpoint. When it does, this is the single most useful five-minute experiment you can run. Stand up an Ollama or LM Studio server on localhost (default port 11434 for Ollama, 1234 for LM Studio). For an Anthropic-shaped agent like Fazm, slot a translator like LiteLLM in front so the local server accepts the Anthropic Messages format. Paste http://127.0.0.1:11434 into the Custom API Endpoint field, restart the chat panel, and re-run nettop.

The Anthropic row should disappear from the external table. A new row should appear in nettop -t internal, to 127.0.0.1, with bytes-out matching the size of your prompt. That is the local-LLM path, verified end to end. The agent process, the tool calls, the file system access, and the model call are now all on your machine.

The Fazm anchor: two lines of Swift, one Settings field

The reason Fazm reads cleanly under this recipe is that the entire model-routing decision is concentrated in one place. The Swift side reads a UserDefault and exports it to the agent subprocess as an environment variable:

// Desktop/Sources/Chat/ACPBridge.swift, lines 468-469
if let customEndpoint = defaults.string(forKey: "customApiEndpoint"),
   !customEndpoint.isEmpty {
  env["ANTHROPIC_BASE_URL"] = customEndpoint
}

The Settings page that drives that default contains one hint, also verbatim from the source:

Route API calls through a custom endpoint (e.g. local LLM bridge, corporate proxy, or GitHub Copilot bridge). Leave empty to use the default Anthropic API.

That is the entire surface area for the LLM-routing decision. One field, two lines, one env var, one process tree. Once you know which two lines they are, the verification recipe collapses to: did the field get exported correctly, and did nettop confirm that the only model traffic now goes where you said. Both questions take less than a minute to answer once you are set up. The voice and TTS endpoints are not yet pluggable in the same way (Deepgram and ElevenLabs are hardcoded as of this writing), so the honest version of "local Fazm" today is: local agent, local model if you point it at one, cloud voice and cloud TTS unless you turn those features off. Knowing exactly which row is which in nettop is what makes the claim honest.

What this recipe will not catch

A few honest limits. nettop sees TLS-wrapped connections, so it tells you the host but not the body. An app that talks only to a provider you trust can still be sending more than you would like; the way to bound that is to know what the API actually does (does the Deepgram streaming endpoint require sending audio, yes; does a Sentry crash report require sending the stack trace, yes; does a PostHog autocapture event require sending the screen, no, but most apps configure it that way anyway). The grep step is what tells you which APIs are in play; the API docs tell you what each one carries.

nettop also does not see UDP listens or in-memory IPC. For a normal AI app this does not matter (everything interesting goes over TCP/443), but for a tool that does its own networking (peer-to-peer, mDNS, on-LAN discovery) you would want to add lsof -iUDP -p $(pgrep AppName) to the toolkit. And nettop does not record history; if you want a long-running record, Little Snitch's Network Monitor or LuLu's rules log will keep one for you.