AI
You have built the transformer from the inside out. Now we zoom back out and ask a practical question: how does the same block become the AI systems people use every day?
The answer is mostly about which direction information can flow and what job the model is trained to do.
You’ll learn how the same transformer building blocks become different AI systems depending on how information flows and what job the model is trained to do. You’ll compare BERT for understanding, GPT for generation, and agents that add tools, memory, and actions. An optional lab after GPT lets you fine-tune an open model on a small task dataset.
The Same Engine, Different Jobs
Think of a transformer as an engine. BERT, GPT, and agents are different vehicles built around that engine.
| System | What It Is Good At | Why |
|---|---|---|
| BERT | Understanding text | It can look left and right, so each word sees full context |
| GPT | Generating text | It predicts the next token from everything written so far |
| Agents | Acting on tasks | It uses an LLM plus tools like search, APIs, and code execution |
The important part: these are not unrelated inventions. They reuse the same ideas you just learned: tokenization, embeddings, attention, residuals, layer norm, feed-forward networks, training, and inference.
Chapter Roadmap
Start with BERT, because it shows what changes when a transformer is trained to understand text instead of continue it. Then GPT shows the generation path that leads to chatbots. Finally, agents show how a model becomes part of a larger loop that can gather information and take actions.
| Module | What You’ll See |
|---|---|
| BERT | Masked language modeling, bidirectional context, and encoder-only transformers |
| GPT | Decoder-only generation, sampling, instruction tuning, and scaling |
| Fine-tune an Open Model (Optional) | Train a LoRA adapter and compare results on held-out examples |
| Agents | Tool use, the ReAct loop, memory, planning, and common failure modes |
The small GPT training lab completes the required training project. You can continue from GPT directly to Agents or take the fine-tuning lab first.