Pathway 5: "I'm a Researcher Exploring LLMs" (PhD Student / Researcher)

Target audience: PhD students, postdocs, and research scientists studying LLM capabilities and limitations

Goal: Develop both theoretical depth and practical fluency. Use the annotated bibliographies as entry points into the research literature.

Chapter Guide

• Focus Ch 04: The Transformer Architecture the mathematical core of modern LLMs
• Skim Ch 05: Decoding and Text Generation sampling theory and generation strategies
• Focus Ch 06: Pre-training and Scaling Laws scaling laws, emergent abilities, data curation
• Focus Ch 07: Model Architectures (MoE, SSMs) MoE, SSMs, and alternative architectures
• Focus Ch 08: Reasoning Models and Test-Time Compute open questions in reasoning and chain-of-thought
• Focus Ch 09: Inference Optimization efficiency frontiers: hardware-aware optimization
• Focus Ch 13: Synthetic Data Generation data generation for research experiments
• Skim Ch 14: Fine-Tuning Fundamentals practical fine-tuning for research experiments
• Skim Ch 15: PEFT (LoRA, QLoRA) efficient adaptation methods for research budgets
• Skim Ch 16: Knowledge Distillation and Model Merging compression and merging theory
• Focus Ch 17: Alignment (RLHF, DPO, Constitutional AI) RLHF, DPO, and open alignment questions
• Focus Ch 18: Interpretability and Mechanistic Understanding probing what models learn and represent
• Focus Ch 29: Evaluation and Experiment Design rigorous benchmarking and experiment design
• Focus Ch 34: Emerging Architectures state space models, long context, and beyond
• Focus Ch 35: AI and Society research implications for society and policy