TurboFieldfare Gets Gemma 4 Into 2 GB

Gemma 4 26B-A4B can run with about a 2 GB in-process memory footprint on an Apple Silicon Mac in TurboFieldfare, but only by keeping a small shared core in memory and streaming the model’s routed experts from SSD. The project author’s own docs put the live footprint at roughly 1.35 GB for the shared model core plus about 305 MiB for a 4K KV cache, while Google’s normal guidance for the same model is about 14.4 GB at Q4_0.

That means the claim is real in the narrow sense, not in the usual “load the whole model into RAM” sense. TurboFieldfare is a purpose-built Swift engine for one pinned checkpoint, Gemma 4 26B-A4B IT, and it trades memory for I/O and speed. On the same host where its author measured about 31–35 tok/s in TurboFieldfare, MLX hit roughly 76–82 tok/s.

TurboFieldfare matters because it shows a different way to run a mixture-of-experts model locally on Apple Silicon. Instead of treating a 26B-parameter MoE as something that must stay resident for fast routing, it leans on SSD reads and macOS caching to pull in only the experts the current tokens actually route to. That is a sharp contrast with the normal local path described in Google’s Gemma docs, where all 26B parameters are typically loaded for fast inference. It also lands in the middle of a broader fight over open-source AI strategy and distribution and the economics of what users can realistically run at home, not just what a benchmark machine can host.

TurboFieldfare’s ~2 GB memory budget comes from streaming Gemma 4’s routed experts off SSD

The core trick is simple: TurboFieldfare does not keep the whole Gemma 4 26B-A4B model in process memory. Its system design document says the engine keeps a roughly 1.35 GB “shared core” resident, uses a small expert-slot cache, and stores the routed experts in separate files that are fetched from SSD as needed.

The same design doc breaks the live budget into concrete pieces. The shared model core is about 1.35 GB, and a 4K context KV cache is about 305 MiB. The remaining memory goes to runtime overhead and a limited set of active expert slots, which is how the project gets to its “about 2 GB” claim instead of anything close to Google’s standard full-load guidance.

That is not the same thing as saying the model only “needs 2 GB” in the normal hardware-shopping sense. The repository says the installed model still takes about 14.3 GB of storage, and the design depends on macOS file cache and SSD-backed expert reads. In other words: less like fitting a smaller engine into memory, more like leaving most of the engine in the garage and fetching parts while the car is running.

There is also a scope limit here. TurboFieldfare currently targets Gemma 4 26B-A4B IT specifically, not arbitrary models. So this is a focused demo of one architecture-path pairing, not a general answer to local LLM memory pressure.

Measured decode speed ranges from about 5–6 tok/s on an 8 GB M2 Air to 31–35 tok/s on a 24 GB M5 Pro

The project’s own benchmark table says TurboFieldfare is usable, but not fast. On an 8 GB M2 MacBook Air, the reported decode speed is about 5.1–6.3 tok/s. On a 24 GB M5 Pro MacBook Pro, it reports about 31–35 tok/s.

Those numbers are the practical answer to whether this changes what 8 GB and 16 GB Macs can do. Yes, in the sense that an 8 GB Air appears able to run a model that Google otherwise frames as needing roughly 14.4 GB at Q4_0. No, in the sense that the experience is still bounded by storage I/O and lower throughput, not by some magic compression breakthrough.

The benchmark notes matter. The measurements are the project author’s own, with limited sample counts and warm but uncontrolled file-cache state. That is enough to establish the basic tradeoff. It is not enough to treat every token-per-second figure as settled.

Independent context points in the same direction. An MLX Swift issue discussing Gemma 4 26B MoE support describes the standard path as implying roughly 16 GB on disk and about 26 GB of RAM. A llama.cpp issue for Gemma 4 26B A4B on an M4 Mac shows a roughly 14.2 GB GGUF model running on a 16 GB Mac mini with substantially higher throughput than TurboFieldfare, but with much larger wired memory use.

Google’s own Gemma guidance and MLX baselines show the tradeoff is real: far lower memory, materially lower speed

The cleanest comparison is the project’s same-host test on the 24 GB M5 Pro. There, TurboFieldfare reports roughly 31–35 tok/s, while MLX on the same checkpoint reports roughly 76–82 tok/s. That is less than half the throughput in exchange for a radically smaller in-process footprint.

A short head-to-head makes the point:

That tradeoff is exactly what Google’s docs would lead you to expect. Google says the 26B A4B model normally loads all 26B parameters for fast routing. TurboFieldfare does not disprove that guidance. It sidesteps it with an I/O-heavy design.

That makes the project more interesting than a gimmick, but less revolutionary than “Gemma 4 26B runs in 2 GB RAM” sounds at first pass. For 8 GB and 16 GB Macs, this looks like a real proof that large MoE models can be made accessible locally through expert streaming. It does not yet look like the default way most people will want to run them if they have enough memory to load them conventionally.

There are also hard platform limits. The repository lists macOS 26, Metal 4, Xcode 26, and Swift 6.2+ as requirements, so this is not a drop-in option for every existing Apple Silicon machine.

That is still a meaningful result. If open models keep leaning into MoE designs, techniques like SSD expert streaming could matter almost as much as the models themselves. That would affect not just hobbyists, but the distribution math around local AI software that projects such as DeepSeek open-source model economics have already pushed into focus.

The next concrete milestone is in the repo itself: TurboFieldfare’s benchmark and design docs will show whether the project expands beyond its current single-model target and whether independent reproductions tighten the speed numbers.

Key Takeaways

• Gemma 4 26B-A4B can run with about a 2 GB in-process memory footprint in TurboFieldfare on Apple Silicon Macs.
• TurboFieldfare gets there by keeping about 1.35 GB of shared model core and about 305 MiB of 4K KV cache in memory while streaming routed experts from SSD.
• The measured speed penalty is large: about 5.1–6.3 tok/s on an 8 GB M2 Air and about 31–35 tok/s on a 24 GB M5 Pro.
• Google’s normal guidance for Gemma 4 26B A4B is about 14.4 GB at Q4_0, so TurboFieldfare is trading throughput for a much smaller live memory footprint.
• The current project is a targeted demo for one Gemma 4 checkpoint, not a general-purpose local inference engine for arbitrary models.

Further Reading

• TurboFieldfare GitHub repository — Primary source for the project, requirements, setup, and top-line claims.
• TurboFieldfare benchmarks — Reported token-per-second results on different Apple Silicon Macs and the same-host MLX comparison.
• TurboFieldfare system design — Explains the shared-core memory split, KV cache, and SSD expert streaming design.
• Google Gemma 4 model overview — Official memory guidance and explanation of normal full-parameter loading for fast inference.
• mlx-swift-lm Issue #282 on Gemma 4 26B MoE support — Independent context on standard Apple Silicon memory expectations for Gemma 4 26B.
• llama.cpp Issue #21655 on Gemma 4 26B A4B performance on M4 — Independent Apple Silicon example with higher throughput and much larger memory use.

Last reviewed: 2026-08

---

Originally published on novaknown.com