Saw this paper: https://arxiv.org/pdf/2406.02528
In essence, MatMul operations can be completely eliminated from LLMs while maintaining strong performance at billion-parameter scales and by utilising an optimised kernel during inference, their model’s memory consumption can be reduced by more than 10× compared to un-optimised models.
source: https://x.com/rohanpaul_ai/status/1799122826114330866
Be the first to comment.
I came across two interesting papers recently on scaling laws in AI and wanted to share a summary. Here are the key takeaways:
Scaling LLM Test-Time Compute Two papers looked at how to scale up test-time compute for LLMs:
❤️ Please do like these deep posts, this helps push this content to more people. ❤️
Here is everything you need to know in 10 points:
Liquid AI has launched a new type of AI model called Liquid Foundation Models (LFMs). These models are designed to be more efficient than traditional transformer-based models, with smaller memory needs and better performance. LFMs come i...
Smaller memory needs are welcome and need of the hour. Fucking Nvidia and the insane cost of vRAM is why the littl...
