SwarmLLM splits a 27B model across browser tabs with WebGPU and WebRTC

SwarmLLM is an early open-source runtime that distributes one large language model across multiple browser tabs rather than requiring the full model on one machine. Each participating device downloads only a slice of model layers and executes those layers through a from-scratch WebGPU/WGSL inference engine. For Qwen 3.8 27B, the project passes a 5,120-element f16 hidden state — roughly 10KB per generated token — between peers over direct WebRTC connections. The project recorded a September 7 demo running the 27B model across a MacBook and iPhone on the same Wi-Fi and reports 400 generated tokens at 10.7 tokens/s. It also publishes reproducible benchmark scripts and golden tests, but the performance figures remain project-reported rather than independently validated.

Local-model deployment usually assumes one device can fit the entire model or that a native distributed runtime is installed across several machines. SwarmLLM explores a much lighter operational boundary: a browser URL becomes the distributed runtime, and ordinary devices contribute memory and GPU capacity by holding model layers. If the approach proves robust, it could make large local models usable in homes, classrooms, labs or small teams by pooling hardware already present rather than buying one larger accelerator. The trade-offs are equally important: every generated token still traverses the peer chain, browser support is uneven, prompt prefill is substantially slower than native llama.cpp in the project's own GB10 benchmark, and peers in a room are not a privacy boundary from one another.

SwarmLLM assigns contiguous model-layer slices to participating devices. The host embeds the current token into a 5,120-element hidden state, sends roughly 10KB of f16 activation data to the first peer, and the activation moves through each layer-owning device before returning to the host for final normalization, sampling and the next token.

The engine is written around WebGPU with roughly 50 WGSL kernels, while peer communication uses WebRTC. Participants can join a room by opening a URL and downloading only the layers assigned to their device. No central inference server performs model computation, although ordinary web infrastructure is still used to deliver the application and establish connections.

The current flagship configuration uses Qwen 3.8 27B in a roughly 15GB Q4_0 GGUF. The September 7 demo splits the model across a MacBook and an iPhone on the same Wi-Fi. The project reports 10.7 tokens/s for that recorded run and separately reports 7.7 tokens/s for a 62-layer-plus-2-layer MacBook/iPhone benchmark.

On an NVIDIA GB10, SwarmLLM reports 9.0 tokens/s plain decode and 16.1 tokens/s with speculative decoding, compared with 8.0 tokens/s for the project's native llama.cpp comparison on the same GGUF. But its reported prefill is only 44 tokens/s versus 377 tokens/s for llama.cpp on that machine. The project's own documentation calls prefill the known gap.

The runtime uses batched prompt prefill and multi-token prediction/speculative decoding so one network lap can carry more useful work. The project reports 3.5–6 tokens/s for a host-and-peer room across the internet, showing that wide-area operation is possible in its tests while also making network quality part of inference latency.

The project says no server sees the conversation or performs inference, but it explicitly warns that room participants share the conversation and that mid-model activations should not be considered private against a determined peer. This is therefore a peer-trust architecture, not a cryptographic privacy boundary between participants.

Chrome on macOS is the project's tested browser host. Safari on iPhone can join and hold a small slice, while Safari on Mac can reload under memory pressure for large slices. Firefox and Linux Chromium with WebGPU are currently listed as untested by the project. A Deno 2 headless path is also supported.