Inference Engineering
How modern LLM inference works — from a single matrix multiply to a multi-tenant serving platform.
5 levels · 186 topics · ~354h 45m
completed
Start learning →Edition 01 — Engineering curriculum
Learn modern AI infrastructure
Deep technical courses on how inference systems, GPUs, the telemetry that watches them and the language they are built in actually work — from first principles to production.
04 paths 285 topics 5 levels
Learning paths
GPU Engineering
The GPU itself — what is inside the chip, how it is programmed, why it is fast, and how thousands are run together.
5 levels · 32 topics · ~27h 15m
completed
Start learning →Observability Engineering
Seeing inside running systems — metrics, logs, traces and profiles, then the GPU, token and cost telemetry that AI infrastructure adds.
5 levels · 30 topics · ~24h 25m
completed
Start learning →Go Engineering
Go from the first program to the runtime — how values sit in memory, how the allocator, garbage collector and scheduler work, and how to build AI systems in it.
5 levels · 37 topics · ~33h 5m
completed
Start learning →Agentic Engineering
Building systems where models plan, call tools and act — loops, context, evaluation and safety.
Coming soon
More coming
New paths are added as plain Markdown folders. Distributed systems for AI and networking for GPU clusters are on the list.
How the learning is structured
Every path runs through the same five levels. Each level assumes the ones before it and nothing else.
- 01FoundationsBuild the mental model.
- 02BasicUnderstand the core mechanisms.
- 03IntermediateLearn the optimization techniques.
- 04AdvancedStudy systems at production scale.
- 05ExpertDesign platforms and read the frontier.
- Start with fundamentals. No step assumes knowledge the path has not taught.
- Understand the system. Mechanisms are built in small Go programs you can run.
- Study the bottlenecks. Memory, bandwidth, queues — find what actually limits speed.
- Learn the optimization techniques. Each one is derived from the bottleneck it removes.
- Build real systems. Every path ends in projects, not quizzes.
Featured topics
Inference Engineering
The KV Cache
Inference Engineering
Continuous Batching
Inference Engineering
PagedAttention
GPU Engineering
SMs, Warps and SIMT
GPU Engineering
The Roofline Model
GPU Engineering
Memory Planning for LLMs
Go Engineering
The Garbage Collector
Go Engineering
Goroutines and the Scheduler
Go Engineering
A Transformer Forward Pass
Observability Engineering
Histograms
Observability Engineering
Inference Engine Metrics
Observability Engineering
How It Is Changing
Recently updated
- Capacity Math: Worked ExamplesInference Engineering · Basic
- Streaming InferenceInference Engineering · Basic
- Pruning, Distillation, and Architecture OptimizationInference Engineering · Intermediate
- Choosing a Serving StackInference Engineering · Intermediate
- TGI, Triton, and ONNX RuntimeInference Engineering · Intermediate
- Syscalls and Profiling BasicsInference Engineering · Foundations