How GPT/LLMs actually work
Hey guys, I found a project on GitHub and hacker news and I believe it explains LLMs in the best possible way. This explains tokens, vectors, vocabulary and transformers in a visual presentation. Just beautiful and also have the full view of GPT-3
A 3D animated visualization of an LLM with a walkthrough.
https://bbycroft.net/llm
GrapeGodfather
Stealth
6 months ago
In simple words, GPT or LLMs are like parrots. They learn from a lot of text data and then predict the next word based on the previous ones. They don't understand language like us humans, they just mimic patterns. It's like a kid who memorizes multiplication tables but doesn't understand multiplication. They are helpful but can't replace human understanding.
GrapeGodfather
Stealth
6 months ago
@SuperSaiyanGoku Nice resource by the way. Nano-GPT architecture representation is very interesting but for most people my explanation will suffice lmao.
SuperSaiyanGoku
Stealth
6 months ago
I don’t think so. Even Linus, the creator of Linux, says it’s becoming more than just a parrot or mimic because it’s starting to have world building understanding.
I mean even if we ask simple things it understands the concept of gravity, buoyancy and more without explicitly specifying. It’s starting to understand the world in the eyes of a toddler but improving very very fast
See more comments
Noctus
Stealth
6 months ago
Interesting 🤔
SuperSaiyanGoku
Stealth
6 months ago
It even shows the scale of GPT-3, I wonder what size the GPT-4 would be with all the multiple model architecture