Runs a 744B-parameter MoE model on a 25GB-RAM consumer machine in pure C, streaming experts from disk.
What we like
- Genuine systems engineering — expert streaming makes an 'impossible' model size run locally
- Zero dependencies and pure C means it will build anywhere
What to watch
- Disk-streamed inference is slow by nature; this is a feat, not a daily driver