How to Run a 120B Model on Multiple Old Computers (Step-by-Step)
You can run a 120B parameter model across multiple old computers by pooling their system RAM and VRAM using network tensor sharding. Distributing the model’s layers across 5 to 6 older machines (laptops, desktops, mini PCs) creates a 140+ GB unified memory pool over LAN, achieving 1.3 to 1.5 tokens/sec without renting expensive data-center GPUs.
A 120-billion parameter Large Language Model—such as MX-120B, Command R+, or Qwen 110B—is widely considered the exclusive domain of enterprise AI data centers. Unquantized, a 120B model requires over 240 GB of memory. Even quantized to 4-bit (Q4_K_M), the weights demand roughly 72 GB of resident memory, plus another 8 GB to 16 GB for the KV context cache.
No single consumer GPU can load this model. An Nvidia RTX 4090 caps out at 24 GB VRAM. Buying an enterprise Nvidia H100 or dual A100 workstation costs between $15,000 and $40,000. However, by leveraging distributed RPC sharding, you can pool memory across 5 or 6 spare household computers over standard gigabit Ethernet.
Hardware Requirements to Pool 140+ GB for a 120B Model
| Node Machine | Hardware Specifications | Memory Contribution | Assigned Transformer Layers |
|---|---|---|---|
| Node 1 (Gaming PC) | Ryzen 7 + RTX 3060 (12 GB VRAM + 32 GB RAM) | 38 GB Usable | Layers 1 – 32 (CUDA + CPU) |
| Node 2 (Mac mini) | Apple M4 Mac mini (16 GB Unified) | 12 GB Usable | Layers 33 – 46 (Metal GPU) |
| Node 3 (Office Desktop) | Intel Core i5 (32 GB DDR4) | 28 GB Usable | Layers 47 – 72 (CPU RAM) |
| Node 4 (Old Laptop) | Intel Core i7-7700HQ (16 GB DDR4) | 13 GB Usable | Layers 73 – 86 (CPU RAM) |
| Node 5 (Tiny PC) | Intel N100 Mini PC (16 GB DDR4) | 13 GB Usable | Layers 87 – 100 (CPU RAM) |
| Node 6 (Spare Laptop) | Dell XPS (16 GB DDR4) | 13 GB Usable | Layers 101 – 112 (CPU RAM) |
| Total Cluster | 6 Consumer Machines | 117 GB Usable (148 GB Total) | All 112 Layers Active |
What Speed Does a 120B Model Achieve Over LAN?
Because the cluster passes intermediate activation vectors sequentially across machines, network latency adds a small overhead per token (roughly 50ms across 6 Ethernet hops). The bottleneck remains the memory bandwidth of the individual CPUs and GPUs processing their slice of the model. See full benchmark telemetry in our deep-dive: What Speed Looks Like on a Pile of Old Computers.
What Happens If a Node Sleeps or Unplugs Mid-Run?
Part of the RAM Pooling Guide Series
This technical guide is an extension of our foundational architectural resource: How to Pool RAM from Old PCs: The Complete Guide. For setup steps across 2 machines, see How to Run an LLM Across Two PCs Without a GPU.
Frequently Asked Questions
Can you run a 120B model without a GPU?
Yes. By pooling system RAM across 5 to 6 computers with at least 16 GB each over Ethernet, a 120B model evaluates layer-by-layer entirely in CPU memory at roughly 1.0 to 1.5 tokens per second.
How much total RAM is needed for a 120B Q4 model?
Approximately 80 GB of usable AI memory is needed to hold the 72 GB weights and context buffer. We recommend a pool of at least 110 GB to 140 GB total RAM across your machines to leave ample operating system headroom.
Does Wi-Fi work for a 6-computer 120B cluster?
Standard Wi-Fi 5 or Wi-Fi 6 works, but gigabit Ethernet switches provide significantly lower jitter and tighter token pacing when chaining 6 machines together.
