Free Resources for Learning LLM Inference Engineering logopeterindia.net
Knowledge Hub

Free Resources for Learning LLM Inference Engineering

A step-by-step set of free resources, from core terms to serving engines, GPU kernels and benchmarking.

31 topics•9 categories•Updated October 2026

How to Use This Page

3 entries

Work through the sections in order: learn the vocabulary, understand batching and scheduling, study the maths, read a real engine, then learn GPU programming and benchmarking.

01LLM Training and Inferencing Optimization TechniquesTraining and inference optimization techniques on this site. 02Inference EnginesInference engine directory on this site. 03Production AI Inference StackHow the pieces of a production inference stack fit together.

1. Core Concepts

2 entries

Prefill, decode, KV cache, latency and throughput.

04LLM Inference Handbook (BentoML)Reference covering core terms such as prefill, decode, KV cache, latency and throughput, and how to optimize them. 05Mastering LLM techniques: inference optimization (NVIDIA)NVIDIA blog post on techniques such as KV caching, batching and quantization.

2. Batching and Scheduling

3 entries

How engines keep GPUs busy.

06Continuous batching from first principles (Hugging Face)Builds from attention and KV caching up to chunked prefill and dynamic scheduling. 07CS336: Language Modeling from Scratch (Stanford)Stanford course whose inference lecture covers memory bottlenecks, quantization, speculative decoding and PagedAttention. 08CS336 Lecture 10: Inference (2025)Executable lecture notes for the inference lecture.

3. The Maths of Inference

3 entries

Memory, compute and communication limits.

09How To Scale Your Model: Transformer InferenceChapter on the maths of memory, batching and multi-accelerator inference, with worked problems; more demanding than the others. 10How To Scale Your ModelFull online book on scaling transformers on TPUs and GPUs. 11MIT 6.5940: TinyML and Efficient Deep Learning ComputingMIT course whose lectures cover quantization and efficient LLM deployment.

4. Read a Real Engine

9 entries

Learn how production serving systems work by studying their code and docs.

12Inside vLLM (Aleksa Gordić)Detailed walkthrough of scheduling, paged attention, caching and distributed serving, based on a 2025 vLLM V1 snapshot. 13Mini-SGLangCompact SGLang implementation with architecture notes, prefix caching, chunked prefill and overlap scheduling; needs Linux and CUDA. 14vLLM documentationDocumentation for the vLLM serving engine. 15SGLang documentationDocumentation for the SGLang serving framework. 16TensorRT-LLMNVIDIA's library for optimized LLM inference on its GPUs. 17Text Generation Inference (Hugging Face)Hugging Face's inference server for large language models. 18llama.cppC/C++ inference for running LLMs on a wide range of hardware. 19NVIDIA DynamoDistributed inference framework for serving generative AI models at scale. 20llm-dKubernetes-native distributed LLM inference stack.

5. GPU Programming

4 entries

Write the kernels that engines depend on.

21GPU MODE lecturesLecture recordings and supporting material; the post suggests the lectures on profiling, CUDA and GPU architecture first. 22GPU MODECommunity for GPU programming and kernel optimization. 23GPU MODE on YouTubeVideo channel with the lecture recordings. 24Triton tutorialsOfficial runnable tutorials from vector addition to softmax, matrix multiplication and fused attention.

6. Benchmarking

1 entries

Measure what you build.

25NVIDIA AIPerfBenchmarking tool for generative AI serving, measuring first-token latency, token latency, throughput and goodput under controlled loads and realistic traces.

Source

1 entries

The post these core resources come from.

26Free resources for learning LLM inference engineering (Mayank A., LinkedIn)LinkedIn post listing the ten resources; LinkedIn may require sign-in.

Related Pages

5 entries

Neighbouring pages on this site.

27LLM Inference PlatformsLLM inference platforms directory. 28AI Model Quantization TechniquesQuantization techniques. 29ML Model Serving ToolsModel serving tools. 30LLMOpsLLMOps hub. 31AI Courses and CertificationsAI learning guide.

Frequently Asked Questions

What is LLM inference engineering?

It is the work of serving language models efficiently: managing the KV cache, batching and scheduling requests, using quantization and speculative decoding, writing GPU kernels and measuring latency and throughput.

Where should I start?

Read the LLM Inference Handbook for the vocabulary, then the Hugging Face continuous batching post, then the Stanford CS336 inference lecture.

Do I need a GPU?

Reading and most lectures need none. Running engines such as vLLM or Mini-SGLang, or the Triton tutorials, needs a CUDA-capable GPU, which may mean paid cloud compute.

Which engine should I study first?

Inside vLLM gives a guided tour of a production engine, and Mini-SGLang is a compact codebase that is easier to read end to end.

How do I measure inference performance?

Use a benchmarking tool such as NVIDIA AIPerf, which measures first-token latency, per-token latency, throughput and goodput under controlled load.

How current is this page?

Links were checked in October 2026. The core list follows a LinkedIn post by Mayank A.; the descriptions summarize each resource's own pages. The learning materials are free, but running their examples may need paid GPU compute. Content such as course years and software versions changes, so check each resource's date before relying on it. Listing is not an endorsement.