Local image to video AI: two routes, one flag that decides your battery

Two different stacks that share a phrase. Stack one is diffusion image-to-video: a single still image plus a motion prompt fed into HunyuanVideo-I2V, Wan-i2v, LTX-Video, CogVideoX-I2V or Stable Video Diffusion, hallucinating new frames on a 24 GB NVIDIA card. Stack two is image-sequence to video: a stream of real images (screenshots, chart frames, design iterations, time-lapse photos) piped into a local hardware encoder so an on-device AI can reason about the resulting video. Almost every article treats the first stack as the only answer. It is not.

HunyuanVideo-I2V is built on the 13B HunyuanVideo backbone and runs comfortably on 24 GB VRAM, 3090 class or newer. Wan-i2v (2.1, 2.2, 2.6) is also quoted at 24 GB for the fp8 path, and significantly more for fp16. LTX-Video is the lightest of the headline models and will fit on 12-16 GB at reduced quality. Stable Video Diffusion (SVD) will run on 8-12 GB but caps at roughly 2 second clips. CogVideoX-I2V needs 16-24 GB depending on the variant. In every case the assumption is an NVIDIA card, CUDA, Python, and ComfyUI as the front end. None of these models have a first-class Apple Silicon path in April 2026.

Because it is not glamorous. You start with a sequence of images you already have (screenshots, dashboard frames every N seconds, rendered design iterations, time-lapse photos, receipt scans, medical slice exports), pipe them into a local hardware video encoder, and the output is a real playable video file that any vision-language model can reason about. No diffusion, no hallucination, no GPU pressure. The reason nobody covers it is that it does not need a 5090 card, so it does not feature in 'best local video AI' roundups. Fazm's screen observer is exactly this pipeline, productised on a Mac.

A one-line comment in the Fazm source tree. The file Desktop/.build/checkouts/macos-session-replay/Sources/SessionReplay/VideoChunkEncoder.swift at line 220 says: 'Use raw BGRA pixel input instead of PNG to avoid expensive per-frame PNG encoding'. That comment lives directly above the ffmpeg invocation on lines 226-242 which uses the arguments '-f rawvideo -pixel_format bgra -r 2 -i - -vcodec hevc_videotoolbox -tag:v hvc1 -q:v 65 -allow_sw true -realtime true -prio_speed true'. The public API that feeds it is at line 86: 'public func addFrame(image: CGImage, timestamp: Date) async throws -> EncodedFrame?'. That is the image-to-video function, in plain Swift, running locally on your Mac.

PNG is a compressed, DEFLATE-based image format. Encoding a PNG on every captured frame adds a CPU-bound compression step before the frame even reaches the video encoder, and then the video encoder immediately has to decompress it back to pixels. Raw BGRA skips both steps. The Fazm pipeline draws the CGImage into a BGRA CGContext with interpolationQuality set to low (line 282), then writes the raw bytes to ffmpeg stdin. That data is already in the exact memory layout hevc_videotoolbox wants. The comment on line 220 is not a stylistic preference, it is the practical reason the pipeline can run at 2 FPS sustained for hours without pinning a core.

-vcodec hevc_videotoolbox. On Apple Silicon, that one flag routes encoding to the dedicated media engine on the M-series die, a block that exists next to the CPU and GPU with its own power budget. The alternative flags, -vcodec libx264 or -vcodec libx265, would take the job to the CPU and pay a sustained multi-watt cost for multi-hour sessions. For a continuously running image-sequence-to-video pipeline, that single substitution is the line between 'background observer you forget about' and 'laptop that runs hot after an hour'. libx265 software encoding of a 24 FPS 1080p stream can easily saturate two CPU cores; hevc_videotoolbox at 2 FPS barely registers in Activity Monitor.

No. Fazm does not ship a diffusion model, does not embed a UNet, and does not run ComfyUI workflows. If your goal is to produce new pixels from a single still, you want the diffusion stack with an NVIDIA GPU. What Fazm does is the other half of the keyword: take a real stream of images you already have (most often a captured screen) and produce a local H.265 video file that the reasoning layer can watch. Two jobs, one keyword, and they do not substitute for each other.

The Fazm consumer app wires the pipeline to screen capture because that is the product. But the underlying flow is general: CGImage in, HEVC chunk out, via ffmpeg stdin and hevc_videotoolbox. If you are a developer building a local image-to-video workflow on a Mac (time-lapse from a folder of JPEGs, frame-by-frame render review, animated chart exports), the same '-f rawvideo -pixel_format bgra -i -' pattern works. The productised version, where an AI reasoning layer sits on top of the chunks, is what Fazm packages for non-developers.

Because the downstream consumer is a vision-language model reasoning about UI state, not a human watching a cinematic playback. At 2 FPS the model still sees every meaningful transition (a click, a scroll, a new modal opening), the encoded chunks are roughly an order of magnitude smaller than a 24 FPS equivalent, and the capture loop never has to compete for CPU with whatever app you are recording. The SessionRecorder configuration literally sets framesPerSecond: 2.0 with a comment 'Lower FPS than research recorder to save CPU'. This is the tax you should not be paying if your use case is 'AI watches the screen', and most roundup-level tooling forces a higher default because the defaults were written for humans.

The capture pipeline is fully local: ScreenCaptureKit, a BGRA CGContext, ffmpeg with hevc_videotoolbox, and chunks written under Library/Caches/observer-recordings. The reasoning step is configurable. Out of the box, Fazm can route analysis through Gemini, which is a network call. If you want to stay fully offline, Fazm exposes a Custom API Endpoint field in Settings that sets ANTHROPIC_BASE_URL on the bridge subprocess, so you can point reasoning at an Anthropic-compatible proxy in front of a local model (Ollama, LM Studio, llama.cpp server). The image-to-video half does not care where reasoning runs; it just produces chunks.