Documentation
Understand every building block
Each topic answers what it is, why it's needed, and how it's implemented in the repository — in plain language.
Tokenization
What: How raw text becomes vocabulary IDs.
Why: Models operate on numbers, not characters — tokenization is the bridge.
ExploreEmbeddings
What: How token IDs become dense vectors.
Why: Vectors let the model represent meaning and relationships numerically.
ExploreSelf-attention
What: How each token weighs the relevance of every other token.
Why: Attention is what lets the model use context, not just isolated words.
ExploreTraining & generation
What: How the model learns, and how it produces text one token at a time.
Why: This closes the loop from raw text to a working, generative model.
Explore