Neural Engine

First Speculative Decoding Across GPU and Neural Engine on Apple Silicon

Running two models on the same Apple Silicon chip - a 1B draft model on the Neural Engine and a larger model on GPU for faster local inference.