1.58 BitNets - a new opportunities for llamafile?

[hrstoyanov]

BitNets are the most exciting thing for LLM happeing right now. @jart - Llalmafile can become the BitNet leader if you get early on! The big advantages are:

• Very competitive performance compared to "old school" LLMs (16-bit, 8-bit, (Q)LoRA)
• Significant memory and processing savings: Eliminating "matmul", and reducing it to just adding/bit flipping!
• No need for quantization (they are already quantized!)
• Ideally suited for commodity CPUs (AVX), expensive GPUs no longer needed (but can still help)
• Overall, BitNet LLMs will become much more affordable and accessible because of the above.

Checkout these resources:

• 1.58 BitNet ground-braking papers.
• Early confirmation/reproduction for BitNets are: here, here and here
• Early implementations: here on CPU, here llama.cpp.