Prerequisites
Graduate-level mathematics. Deep learning implementation experience. Familiarity with PyTorch. At least Chapters 3 through 6 completed.
Researcher Track
Understanding LLM internals, scaling behavior, and frontier research directions.
Learning Sequence
Follow the numbered steps in order. Each step builds on the previous one to give you a coherent understanding of this topic area.
- Chapter 04: The Transformer Architecture (self-attention, positional encoding, layer norms)
- Chapter 06: Pre-training, Scaling Laws and Data Curation (Chinchilla, power laws, compute-optimal training)
- Chapter 08: Reasoning Models and Test-Time Compute (chain-of-thought scaling, verification)
- Chapter 17: Alignment: RLHF, DPO and Preference Tuning (reward modeling, constitutional AI)
- Chapter 18: Interpretability and Mechanistic Understanding (probing, logit lens, superposition)
- Chapter 34: Emerging Architectures (post-transformer designs, efficiency frontiers)
- Chapter 35: AI and Society (governance, alignment open problems, societal impact)
Recommended Appendices
- Appendix A: Math Foundations – review the math behind attention and optimization
- Appendix G: Hardware and Compute – understand GPU hardware and compute trade-offs
- Appendix K: HuggingFace: Transformers, Datasets, and Hub – access pretrained models and research datasets
- Appendix R: Experiment Tracking – log experiments for reproducible research
What Comes Next
Return to the Course Syllabi overview to explore other tracks and courses, or proceed to FM.4: How to Use This Book for a quick orientation on conventions and callout types.