Practical Guide to LLM: Introduction

Motivation, objective, format and roadmap

Generated by AI

Motivation

Existing LLM learning materials are not tailored to hands-on hobbyists.

Some are too high level to be translated into code, while others are too specific to certain code base making it hard to decouple.

I think a more practical guide that goes from basics to universal implementations will be helpful to many.

Objectives

To provide a minimalist guide for hands-on LLM hobbyists.

Format

Posts will use minimal words to provide:

1. Intuition and basic concepts
2. Simplified implementation with explanation
3. Projection from #2 to production code

Roadmap

Basics

• Code for model experiments
• Positional encoding

Resource modeling

• HBM cost estimation

Pre-training

• Optimizers

Post-training [WIP]

Fine-tuning [WIP]

Inference

• Prefill & decode
• Prefill & decode disaggregation
• KV cache

Agentic AI [WIP]