ai-explained

ship · production · ops

Ship a production LLM stack from open source.

A code-along walkthrough for the 80% of LLM work that isn't pretraining: pick a base model, run it locally with vLLM, build an eval harness, wire RAG and tools, ship it with observability and cost tuning. Every concept cross-references the matching article and demo on this site, so the theory and the visualization are one click away.

18
steps
18
ready to read
6h 28m
of reading
9h 5m
of hands-on
who this is for

Engineers who deploy, not pretrain

if you build apps on LLMs
You're the audience
The 80% of working AI engineers who use OSS or API base models and need to ship reliable applications on top.
if you've finished /build
This is the next leg
/build taught you how a model works. /ship teaches you how to run one in production: serving, retrieval, tools, observability, cost.
if you're new to AI eng
Start here, not /build
You don't need to write a transformer from scratch to ship an LLM application well. Start at step 00; we'll cover what you need from /articles as it comes up.
the path

The 18-step curriculum

Three release tranches. Foundations is the one that ships first; Building and Production come next. Numbers are stable — bookmark step 03 today and it'll still be step 03 next month.

Foundations

pick a model · run it locally · evaluate it · wrap it as an API

steps 00–05
  1. 00
    What we're shipping
    The 17 steps from a fresh laptop to a self-hosted, evaluated, instrumented, deployed LLM application.
    10 min + 5 min hands-on
  2. 01
    Pick your base model
    Llama-3 vs Qwen-2.5 vs Mistral vs Phi — what differs, what doesn't, which one we'll use for the rest of the curriculum.
    12 min
  3. 02
    Run a model locally with Ollama
    From zero to your first generation in five minutes. Then we'll graduate to vLLM in step 03.
    18 min + 25 min hands-on
  4. 03
    Switch to vLLM
    The production-grade inference engine. Same OpenAI schema, ~3× the throughput, proper continuous batching, paged KV cache.
    25 min + 35 min hands-on
  5. 04
    Build an eval harness
    lm-eval-harness for benchmarks + a custom task-specific eval you write yourself. The 'is it any good' question, answered programmatically.
    22 min + 30 min hands-on
  6. 05
    Wrap as a FastAPI service
    /v1/chat/completions of your own — streaming, API-key auth, structured logging, health checks. The endpoint your application code talks to.
    26 min + 35 min hands-on

Building

RAG · vector stores · tools · agent loops · multi-agent

steps 06–11
  1. 06
    RAG, the production way
    What chunking actually is, why naive splits fail in production, and the four strategies that hold up. The unglamorous step that makes or breaks every RAG system.
    22 min + 30 min hands-on
  2. 07
    Embeddings + vector store
    Pick an embedding model, persist with sqlite-vec, retrieve at 5 ms. Plus: why you probably don't need a dedicated vector DB.
    22 min + 30 min hands-on
  3. 08
    Retrieval: BM25 + dense + reranking
    The three-stage pipeline that beats any single strategy. Hybrid retrieval, RRF fusion, and a cross-encoder reranker as the silent quality lever.
    24 min + 35 min hands-on
  4. 09
    Tools and function calling
    JSON-schema tool definitions, OSS-model adapters, and the structured-output dance that makes agents possible.
    22 min + 30 min hands-on
  5. 10
    Build an agent loop
    ReAct + state management + tool execution + termination. Why most agents are simpler than they look.
    28 min + 40 min hands-on
  6. 11
    Multi-agent orchestration
    Supervisor + workers, fan-out / fan-in, when to use it, when one agent is enough. Patterns for reliability.
    22 min + 30 min hands-on

Production

observability · evals in prod · cost/latency · deploy patterns

steps 12–17
  1. 12
    Observability with Phoenix
    Trace every model call, every retrieval, every tool. The waterfall view production AI requires.
    22 min + 30 min hands-on
  2. 13
    Evaluation in production
    A/B testing prompts, drift detection, golden-set regression. How to know your model got worse before users do.
    22 min + 25 min hands-on
  3. 14
    Cost and latency tuning
    Prompt cache, KV reuse, continuous batching, quantization, speculative decoding. The five levers behind every serving optimization.
    25 min + 30 min hands-on
  4. 15
    Deploy it for real
    Modal vs Replicate vs a $20/mo VPS vs a serverless GPU. Trade-offs, configs, and the deploy command.
    28 min + 45 min hands-on
  5. 16
    Where to go from here
    Distillation, post-training, multi-tenant serving, agentic patterns — what's next once the basics are live.
    12 min
  6. 17
    Synthetic data + distillation
    Compress a frontier model into a small specialist for ~10× the cost reduction. The pipeline behind every cost-conscious production deploy.
    26 min + 1h 30m hands-on

How this fits the rest of the site

Every step cross-references the matching theory article and demo. When you wire RAG in step 06, the article links to RAG Fundamentals and the RAG Visualizer. When you build the observability layer in step 12, the Observability Trace demo shows what the spans look like. The site becomes a multi-modal study environment: shell + editor in some tabs, math + visualizations in others.

Status: 18 live · 0 wip · 0 stubbed. Ships in three releases (foundations → building → production).