SuperSaiyanGoku
SuperSaiyanGoku

How GPT/LLMs actually work

Hey guys, I found a project on GitHub and hacker news and I believe it explains LLMs in the best possible way.

This explains tokens, vectors, vocabulary and transformers in a visual presentation. Just beautiful and also have the full view of GPT-3

8mo ago
GrapeGodfather
GrapeGodfather

In simple words, GPT or LLMs are like parrots. They learn from a lot of text data and then predict the next word based on the previous ones. They don't understand language like us humans, they just mimic patterns. It's like a kid who memorizes multiplication tables but doesn't understand multiplication. They are helpful but can't replace human understanding.

GrapeGodfather
GrapeGodfather

@SuperSaiyanGoku Nice resource by the way. Nano-GPT architecture representation is very interesting but for most people my explanation will suffice lmao.

SuperSaiyanGoku
SuperSaiyanGoku

I don’t think so. Even Linus, the creator of Linux, says it’s becoming more than just a parrot or mimic because it’s starting to have world building understanding.

I mean even if we ask simple things it understands the concept of gravity, buoyancy and more without explicitly specifying. It’s starting to understand the world in the eyes of a toddler but improving very very fast

Noctus
Noctus

Interesting 🤔

SuperSaiyanGoku
SuperSaiyanGoku

It even shows the scale of GPT-3, I wonder what size the GPT-4 would be with all the multiple model architecture

Discover more
Curated from across