Fast NF4 dequantization Triton kernel (1.41x faster than bitsandbytes)

Visit product ↗

"I wanted to see if I could optimize the dequantization bottleneck during 4-bit LLM inference. By writing a custom kernel in Triton to optimize memory access patterns, I managed to get up to a 1.41x speedup over the standard bitsandbytes implementation. Check out the source code and benchmarks, feedback is highly appreciated!"

More Developer tools

Open Weights Breakoutqsa.shCuts Long Horizon Inference Costs by 50% via external KV Cache OffloadCrossle – A daily puzzle game where you find 3 intersecting wordsTermic – desktop app for running CLI coding agents (Claude Code, Codex)