JustVugg/colibri
Runs a 744B-parameter MoE model on a 25GB-RAM consumer machine, pure C, zero deps.
Colibrì is an open-source inference engine that treats VRAM, RAM and disk as one memory hierarchy, streaming model experts from disk so a 744B-parameter MoE model (GLM-5.2) can run resident in under 10GB RAM. Nine model families run today, each as a single C file sharing the same chat/serve/web front end. It's a research platform, not a polished product: the README states experiments must earn their place through reproducible end-to-end measurements, with a hard guarantee that low memory never silently changes model precision.
What holds up
- +Fits a 744B-parameter model in about 10GB resident RAM — no hyperscaler-class hardware needed.
- +Zero engine dependencies, pure C — no Python environment or CUDA toolkit stack to fight.
- +Never silently changes model precision or router behavior under memory pressure, by design.
Mind the limits
- −No speed guarantee by design — correctness is prioritized over faster benchmark numbers.
- −Even with full expert residency on 6x RTX 5090 GPUs, the 744B model runs at just 4 tok/s.
Featured here? Take the badge
Put it on your site — it links back to this review. Free for every listed product, always.
<a href="https://stillworks.dev/products/p/justvugg-colibri/"><img src="https://stillworks.dev/badge/justvugg-colibri.svg" alt="Picked by StillWorks" width="250" height="54"></a>