Colibrì Proof-of-Concept Runs 1.5-TB MoE Model on 25GB RAM via Disk Swapping

Italian engineer Vincenzo has released Colibrì, a proof-of-concept that executes a 744-billion-parameter GLM-5.2 model using aggressive expert-swapping, achieving inference on consumer hardware at the cost of extreme latency.

Chips & Climate Desk
Updated Nov 13, 2026 2 min read
Colibrì Proof-of-Concept Runs 1.5-TB MoE Model on 25GB RAM via Disk Swapping

Italian engineer Vincenzo, operating under the handle JustVugg, has unveiled Colibrì, a proof-of-concept software implementation designed to run the 744-billion-parameter GLM-5.2 model on consumer-grade hardware. According to Tom's Hardware, the system operates the 1.5-terabyte model using only 25 GB of RAM and a 1 GB/s virtual NVMe drive. This approach diverges from standard datacenter deployments, which typically require massive GPU clusters to maintain the high-bandwidth memory access necessary for frontier-scale models. Instead, Colibrì leverages the inherent structure of Mixture-of-Experts (MoE) architectures to bypass traditional VRAM capacity requirements.

The core mechanism of Colibrì relies on the fact that MoE models select specific expert sub-models per token rather than utilizing the entire parameter set for every query. By treating the model as a collection of modular experts, the software loads and unloads these components into RAM dynamically as tokens are processed. To further reduce the memory footprint, the GLM-5.2 model is quantized using lossy encoding. The expert-selection logic itself is implemented as a lightweight C file with minimal dependencies, designed to manage the constant data movement between the storage medium and system memory.

While the architecture successfully enables the execution of a 1.5-TB model on minimal hardware, the performance trade-offs are substantial. Tom's Hardware reports that current inference speeds range from 0.05 to 0.1 tokens per second, rendering the system unusable for practical, real-time conversation. The bottleneck is primarily driven by storage I/O, as the system must continuously swap data to satisfy the demands of the token-by-token expert selection process. Even with higher-end hardware configurations, the system currently falls well short of the 20-30 tokens per second threshold typically required for interactive AI applications.

The current iteration of Colibrì does not support GPU acceleration, though the developer notes that even with GPU integration, the primary constraint would likely remain the data transfer overhead between the storage subsystem and the accelerator. The project is currently in an early proof-of-concept phase, with the developer actively collecting benchmark data and refining the codebase. Future viability for this approach—specifically for running frontier-class models on consumer hardware at usable speeds—will depend on optimizing memory bandwidth and storage throughput, as these remain the primary limiting factors for local inference of this scale.

Sources

  1. 01 Colibrì proof-of-concept gains frontier-level 1.5-TB AI model — novel approach runs on only 25GB of RAM and shows promise for local AI setups — Tom's Hardware
#ai #model-compression #edge-computing #local-ai #hardware-efficiency