Diagrams — Module FT01: VRAM Math

Module: FT01 — VRAM Math: Can I Actually Run This? Diagram count: 5 Tool: Mermaid (primary). Each diagram validated in Mermaid Live Editor.

Diagram 1 — The Three VRAM Consumers (QLoRA vs Full FT, 7B)

Type: Comparison flowchart Purpose: The single most important visual in the module. Training VRAM is three things — weights, optimizer states + gradients, and activations. Every OOM is one of these. Side-by-side, the same 7B model costs ~10× more under full FT than under QLoRA, and the reason is the optimizer band. Reading the diagram: Two swimlanes. Left = QLoRA (4-bit frozen base, tiny adapter optimizer). Right = Full fine-tuning (FP16 weights + FP32 master + AdamW moments + full gradients + activations). Read each lane top-to-bottom as the three consumers stacked. The optimizer node on the right (~98 GB) is the entire reason full FT is a datacenter event while QLoRA fits on a 4090.

flowchart LR
  subgraph Q["QLoRA — 7B  ·  total ~12-16 GB"]
    direction TB
    QW["1. WEIGHTS (4-bit frozen)<br/>~3.5 GB"]
    QO["2. OPTIMIZER + GRADS (adapter ~1%)<br/>~0.5 GB"]
    QA["3. ACTIVATIONS<br/>~6-8 GB"]
    QW --> QO --> QA
  end
  subgraph F["Full FT — 7B  ·  total ~100-160 GB"]
    direction TB
    FW["1. WEIGHTS (FP16)<br/>~14 GB"]
    FO["2. OPTIMIZER + GRADS (all params)<br/>~98 GB<br/>FP32 master + AdamW m,v + FP16 grad"]
    FA["3. ACTIVATIONS<br/>~6-18 GB"]
    FW --> FO --> FA
  end

  Q -.same model, ~10x cheaper.-> F

  style QW fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style QO fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style QA fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style FW fill:#14141f,stroke:#f08080,color:#e4e4e8
  style FO fill:#14141f,stroke:#f08080,stroke-width:1.5px,color:#f08080
  style FA fill:#14141f,stroke:#f08080,color:#e4e4e8

Note: The optimizer node (FO, ~98 GB) dominates the Full FT total — it alone exceeds the entire QLoRA stack by ~6×. This is the visual proof of the module's central economic argument: full FT's cost is dominated by optimizer states that PEFT never materializes. The lab (artifact 07) prints these exact gigabytes for any model.

Diagram 2 — Method × Model Size → GPU Class (the decision flowchart)

Type: Flowchart / decision matrix Purpose: The operational table. Pick your method (QLoRA / LoRA 16-bit / Full FT) and your model size, follow the arrows, land on the GPU class you need to rent or buy. Reading the diagram: Three swimlanes (methods) flowing left-to-right. Each model-size node carries the training VRAM range and resolves to a concrete GPU class. Notice the 10× gap between QLoRA and Full FT on the same 7B model.

flowchart LR
  subgraph QL["QLoRA — 4-bit frozen base"]
    QL7["7B<br/>~10-16 GB<br/>RTX 4090 24GB"]
    QL70["70B<br/>~48-60 GB<br/>1x A100 80GB"]
  end
  subgraph LO["LoRA — 16-bit base"]
    LO7["7B<br/>~18-30 GB<br/>A100 40/80GB"]
    LO70["70B<br/>~250-400 GB<br/>2-4x A100 80GB"]
  end
  subgraph FF["Full fine-tuning"]
    FF7["7B<br/>~100-160 GB<br/>multi-A100 (~$50K)"]
    FF70["70B<br/>~1.0-1.4 TB<br/>8-16x H100 multi-node"]
  end

  QL7 -.10x cheaper than full FT.-> FF7
  QL70 -.start here, escalate w/ evidence.-> FF70

  style QL7 fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style QL70 fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style LO7 fill:#14141f,stroke:#397f7a,color:#e4e4e8
  style LO70 fill:#14141f,stroke:#397f7a,color:#e4e4e8
  style FF7 fill:#14141f,stroke:#f08080,stroke-width:1.5px,color:#e4e4e8
  style FF70 fill:#14141f,stroke:#f08080,stroke-width:1.5px,color:#e4e4e8

Diagram 3 — Context Length: Quadratic vs Linear (the FlashAttention lever)

Type: Comparison curve (annotated) Purpose: The most underestimated cost in the field. Naive attention is N×N per layer — double the context, quadruple the activation memory. FlashAttention 2/3 fuses it so memory is linear in N. Reading the diagram: Two curves. Without FlashAttention (top), attention activation memory climbs quadratically — at 8K context on a 7B model it can exceed the weight footprint. With FlashAttention (bottom), it climbs linearly. The dashed line is the rule: if your config lacks attn_implementation="flash_attention_2" and your context is >2K, treat it as a bug.

flowchart TD
  Start["Training context length: pick a number<br/>1K · 2K · 4K · 8K · 32K"]

  Start --> Naive["WITHOUT FlashAttention<br/>attention activations ~ N x N<br/>double context -> ~4x memory"]
  Start --> Flash["WITH FlashAttention 2/3<br/>fused, no N x N materialization<br/>double context -> ~2x memory (linear)"]

  Naive --> OOM["At 8K on a 7B model,<br/>attention activations can EXCEED<br/>the weight footprint -> OOM"]
  Flash --> OK["Same job, fits.<br/>Plus ~20-30% faster.<br/>Effectively mandatory."]

  OOM -.fix.-> Flash

  style Start fill:#14141f,stroke:#30303a,color:#e4e4e8
  style Naive fill:#14141f,stroke:#f08080,stroke-width:1.5px,color:#e4e4e8
  style OOM fill:#08080c,stroke:#f08080,color:#f08080
  style Flash fill:#14141f,stroke:#82e0aa,stroke-width:1.5px,color:#e4e4e8
  style OK fill:#08080c,stroke:#82e0aa,color:#82e0aa

Diagram 4 — The Three-Question Framework (the decision tree)

Type: Decision tree Purpose: Before you open a model loader, answer three questions. They determine the GPU class. This is the planning tool you will use on every real job. Reading the diagram: Top-down. Q1 (model size) branches into Q2 (method), which branches into Q3 (context length). The leaves are concrete GPU classes. Three numbers in, one card out.

flowchart TD
  Start["START — the capacity-planning gate"]

  Start --> Q1{"Q1: How big is<br/>the model?"}

  Q1 -->|"1-3B"| Small["Small model"]
  Q1 -->|"7-8B"| Mid["Mid model"]
  Q1 -->|"70B+"| Big["Large model"]

  Small --> Q2a{"Q2: Method?"}
  Mid --> Q2b{"Q2: Method?"}
  Big --> Q2c{"Q2: Method?"}

  Q2a -->|"QLoRA"| L1["Laptop / Mac / Colab T4<br/>1B QLoRA fits in ~6-8 GB"]
  Q2a -->|"Full FT"| L2["1B full FT ~16-32 GB<br/>single A100 or good Mac"]

  Q2b -->|"QLoRA"| M1["RTX 4090 24GB<br/>7B QLoRA ~10-16 GB"]
  Q2b -->|"LoRA 16-bit"| M2["A100 40/80GB<br/>~18-30 GB"]
  Q2b -->|"Full FT"| M3["multi-A100 ~$50K<br/>~100-160 GB"]

  Q2c -->|"QLoRA"| B1["1x A100 80GB<br/>70B QLoRA ~48-60 GB"]
  Q2c -->|"Full FT"| B2["8-16x H100 multi-node<br/>~1.0-1.4 TB"]

  M1 --> Q3{"Q3: context length?<br/>(size for 99th pctile)"}
  Q3 -->|"<=4K, FlashAttention on"| OK1["Fits comfortably"]
  Q3 -->|">4K or no FA"| Fix["gradient checkpointing +<br/>grad accumulation,<br/>or escalate GPU class"]

  style Start fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style Q1 fill:#08080c,stroke:#2a625c,color:#5eead4
  style Q2a fill:#08080c,stroke:#2a625c,color:#5eead4
  style Q2b fill:#08080c,stroke:#2a625c,color:#5eead4
  style Q2c fill:#08080c,stroke:#2a625c,color:#5eead4
  style Q3 fill:#08080c,stroke:#2a625c,color:#5eead4
  style M1 fill:#14141f,stroke:#82e0aa,stroke-width:1.5px,color:#e4e4e8
  style M3 fill:#14141f,stroke:#f08080,stroke-width:1.5px,color:#e4e4e8
  style B2 fill:#14141f,stroke:#f08080,stroke-width:1.5px,color:#e4e4e8
  style OK1 fill:#08080c,stroke:#82e0aa,color:#82e0aa
  style Fix fill:#08080c,stroke:#f0a868,color:#f0a868

Diagram 5 — Concrete Examples: Four Real Jobs Mapped to Hardware

Type: Comparison table-as-diagram Purpose: Lock the numbers with four worked examples spanning the realistic range — from a 1B QLoRA on a laptop to a 7B full-FT on a multi-A100 rig. Reading the diagram: Each row is a real job: model × method × context → training VRAM → the hardware it implies. Read top to bottom in order of ascending cost. The jump from row 1 (laptop) to row 3 (multi-A100) is the entire VRAM story in three lines. Job-1 figures include typical runtime overhead (CUDA context, buffers) as you'd see in Activity Monitor; the bottom-up estimator in the lab models the floor (~2.2 GB for 1B QLoRA) before that overhead.

flowchart LR
  J1["JOB 1<br/>1B QLoRA @ 4K<br/>~6-8 GB"] --> HW1["Laptop / Mac /<br/>Colab T4 (free)"]
  J2["JOB 2<br/>7B QLoRA @ 2-4K<br/>~10-16 GB"] --> HW2["RTX 4090 24GB<br/>($1,500)"]
  J3["JOB 3<br/>7B Full FT @ 4K<br/>~100-160 GB"] --> HW3["multi-A100 80GB<br/>(~$50K of GPUs)"]
  J4["JOB 4<br/>70B QLoRA @ 4K<br/>~48-60 GB"] --> HW4["1x A100 80GB"]

  HW2 -.10x cheaper than Job 3,<br/>same model.-> HW3

  style J1 fill:#14141f,stroke:#82e0aa,stroke-width:1.5px,color:#e4e4e8
  style HW1 fill:#08080c,stroke:#82e0aa,color:#82e0aa
  style J2 fill:#14141f,stroke:#82e0aa,stroke-width:1.5px,color:#e4e4e8
  style HW2 fill:#08080c,stroke:#82e0aa,color:#82e0aa
  style J3 fill:#14141f,stroke:#f08080,stroke-width:1.5px,color:#e4e4e8
  style HW3 fill:#08080c,stroke:#f08080,color:#f08080
  style J4 fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8
  style HW4 fill:#08080c,stroke:#5eead4,color:#5eead4

Validation notes

All five diagrams use the course design system colors: #14141f panel fill, #5eead4 accent for primary/positive, #82e0aa for "fits" / cheap, #f08080 for expensive / OOM-risk, #08080c for hollow callout fills, #e4e4e8 / #9494a0 for text.
All five diagrams use stable flowchart syntax (LR / TD) with subgraph, all validated by rendering to SVG in Mermaid 10.9.x (mmdc). Supported in current Mermaid (v10.4+).
The   line-break syntax inside node labels is supported in Mermaid Live for multi-line node text.
Paste each into Mermaid Live Editor to render. For the slide deck (artifact 03), these are rendered as static SVG/PNG captures from Mermaid Live, inlined into reveal.js.
Every GB figure matches the teaching document (artifact 01) and the lab (artifact 07) so the three artifacts agree to the gigabyte.

# Diagrams — Module FT01: VRAM Math **Module**: FT01 — VRAM Math: Can I Actually Run This? **Diagram count**: 5 **Tool**: Mermaid (primary). Each diagram validated in [Mermaid Live Editor](https://mermaid.live). --- ## Diagram 1 — The Three VRAM Consumers (QLoRA vs Full FT, 7B) **Type**: Comparison flowchart **Purpose**: The single most important visual in the module. Training VRAM is three things — weights, optimizer states + gradients, and activations. Every OOM is one of these. Side-by-side, the same 7B model costs ~10× more under full FT than under QLoRA, and the reason is the optimizer band. **Reading the diagram**: Two swimlanes. Left = QLoRA (4-bit frozen base, tiny adapter optimizer). Right = Full fine-tuning (FP16 weights + FP32 master + AdamW moments + full gradients + activations). Read each lane top-to-bottom as the three consumers stacked. The optimizer node on the right (~98 GB) is the entire reason full FT is a datacenter event while QLoRA fits on a 4090. ```mermaid flowchart LR subgraph Q["QLoRA — 7B · total ~12-16 GB"] direction TB QW["1. WEIGHTS (4-bit frozen) ~3.5 GB"] QO["2. OPTIMIZER + GRADS (adapter ~1%) ~0.5 GB"] QA["3. ACTIVATIONS ~6-8 GB"] QW --> QO --> QA end subgraph F["Full FT — 7B · total ~100-160 GB"] direction TB FW["1. WEIGHTS (FP16) ~14 GB"] FO["2. OPTIMIZER + GRADS (all params) ~98 GB FP32 master + AdamW m,v + FP16 grad"] FA["3. ACTIVATIONS ~6-18 GB"] FW --> FO --> FA end Q -.same model, ~10x cheaper.-> F style QW fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8 style QO fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8 style QA fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8 style FW fill:#14141f,stroke:#f08080,color:#e4e4e8 style FO fill:#14141f,stroke:#f08080,stroke-width:1.5px,color:#f08080 style FA fill:#14141f,stroke:#f08080,color:#e4e4e8 ``` > **Note**: The optimizer node (FO, ~98 GB) dominates the Full FT total — it alone exceeds the entire QLoRA stack by ~6×. This is the visual proof of the module's central economic argument: full FT's cost is dominated by optimizer states that PEFT never materializes. The lab (artifact 07) prints these exact gigabytes for any model. --- ## Diagram 2 — Method × Model Size → GPU Class (the decision flowchart) **Type**: Flowchart / decision matrix **Purpose**: The operational table. Pick your method (QLoRA / LoRA 16-bit / Full FT) and your model size, follow the arrows, land on the GPU class you need to rent or buy. **Reading the diagram**: Three swimlanes (methods) flowing left-to-right. Each model-size node carries the training VRAM range and resolves to a concrete GPU class. Notice the 10× gap between QLoRA and Full FT on the same 7B model. ```mermaid flowchart LR subgraph QL["QLoRA — 4-bit frozen base"] QL7["7B ~10-16 GB RTX 4090 24GB"] QL70["70B ~48-60 GB 1x A100 80GB"] end subgraph LO["LoRA — 16-bit base"] LO7["7B ~18-30 GB A100 40/80GB"] LO70["70B ~250-400 GB 2-4x A100 80GB"] end subgraph FF["Full fine-tuning"] FF7["7B ~100-160 GB multi-A100 (~$50K)"] FF70["70B ~1.0-1.4 TB 8-16x H100 multi-node"] end QL7 -.10x cheaper than full FT.-> FF7 QL70 -.start here, escalate w/ evidence.-> FF70 style QL7 fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8 style QL70 fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8 style LO7 fill:#14141f,stroke:#397f7a,color:#e4e4e8 style LO70 fill:#14141f,stroke:#397f7a,color:#e4e4e8 style FF7 fill:#14141f,stroke:#f08080,stroke-width:1.5px,color:#e4e4e8 style FF70 fill:#14141f,stroke:#f08080,stroke-width:1.5px,color:#e4e4e8 ``` --- ## Diagram 3 — Context Length: Quadratic vs Linear (the FlashAttention lever) **Type**: Comparison curve (annotated) **Purpose**: The most underestimated cost in the field. Naive attention is `N×N` per layer — double the context, quadruple the activation memory. FlashAttention 2/3 fuses it so memory is linear in `N`. **Reading the diagram**: Two curves. Without FlashAttention (top), attention activation memory climbs quadratically — at 8K context on a 7B model it can exceed the weight footprint. With FlashAttention (bottom), it climbs linearly. The dashed line is the rule: if your config lacks `attn_implementation="flash_attention_2"` and your context is >2K, treat it as a bug. ```mermaid flowchart TD Start["Training context length: pick a number 1K · 2K · 4K · 8K · 32K"] Start --> Naive["WITHOUT FlashAttention attention activations ~ N x N double context -> ~4x memory"] Start --> Flash["WITH FlashAttention 2/3 fused, no N x N materialization double context -> ~2x memory (linear)"] Naive --> OOM["At 8K on a 7B model, attention activations can EXCEED the weight footprint -> OOM"] Flash --> OK["Same job, fits. Plus ~20-30% faster. Effectively mandatory."] OOM -.fix.-> Flash style Start fill:#14141f,stroke:#30303a,color:#e4e4e8 style Naive fill:#14141f,stroke:#f08080,stroke-width:1.5px,color:#e4e4e8 style OOM fill:#08080c,stroke:#f08080,color:#f08080 style Flash fill:#14141f,stroke:#82e0aa,stroke-width:1.5px,color:#e4e4e8 style OK fill:#08080c,stroke:#82e0aa,color:#82e0aa ``` --- ## Diagram 4 — The Three-Question Framework (the decision tree) **Type**: Decision tree **Purpose**: Before you open a model loader, answer three questions. They determine the GPU class. This is the planning tool you will use on every real job. **Reading the diagram**: Top-down. Q1 (model size) branches into Q2 (method), which branches into Q3 (context length). The leaves are concrete GPU classes. Three numbers in, one card out. ```mermaid flowchart TD Start["START — the capacity-planning gate"] Start --> Q1{"Q1: How big is the model?"} Q1 -->|"1-3B"| Small["Small model"] Q1 -->|"7-8B"| Mid["Mid model"] Q1 -->|"70B+"| Big["Large model"] Small --> Q2a{"Q2: Method?"} Mid --> Q2b{"Q2: Method?"} Big --> Q2c{"Q2: Method?"} Q2a -->|"QLoRA"| L1["Laptop / Mac / Colab T4 1B QLoRA fits in ~6-8 GB"] Q2a -->|"Full FT"| L2["1B full FT ~16-32 GB single A100 or good Mac"] Q2b -->|"QLoRA"| M1["RTX 4090 24GB 7B QLoRA ~10-16 GB"] Q2b -->|"LoRA 16-bit"| M2["A100 40/80GB ~18-30 GB"] Q2b -->|"Full FT"| M3["multi-A100 ~$50K ~100-160 GB"] Q2c -->|"QLoRA"| B1["1x A100 80GB 70B QLoRA ~48-60 GB"] Q2c -->|"Full FT"| B2["8-16x H100 multi-node ~1.0-1.4 TB"] M1 --> Q3{"Q3: context length? (size for 99th pctile)"} Q3 -->|"<=4K, FlashAttention on"| OK1["Fits comfortably"] Q3 -->|">4K or no FA"| Fix["gradient checkpointing + grad accumulation, or escalate GPU class"] style Start fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8 style Q1 fill:#08080c,stroke:#2a625c,color:#5eead4 style Q2a fill:#08080c,stroke:#2a625c,color:#5eead4 style Q2b fill:#08080c,stroke:#2a625c,color:#5eead4 style Q2c fill:#08080c,stroke:#2a625c,color:#5eead4 style Q3 fill:#08080c,stroke:#2a625c,color:#5eead4 style M1 fill:#14141f,stroke:#82e0aa,stroke-width:1.5px,color:#e4e4e8 style M3 fill:#14141f,stroke:#f08080,stroke-width:1.5px,color:#e4e4e8 style B2 fill:#14141f,stroke:#f08080,stroke-width:1.5px,color:#e4e4e8 style OK1 fill:#08080c,stroke:#82e0aa,color:#82e0aa style Fix fill:#08080c,stroke:#f0a868,color:#f0a868 ``` --- ## Diagram 5 — Concrete Examples: Four Real Jobs Mapped to Hardware **Type**: Comparison table-as-diagram **Purpose**: Lock the numbers with four worked examples spanning the realistic range — from a 1B QLoRA on a laptop to a 7B full-FT on a multi-A100 rig. **Reading the diagram**: Each row is a real job: model × method × context → training VRAM → the hardware it implies. Read top to bottom in order of ascending cost. The jump from row 1 (laptop) to row 3 (multi-A100) is the entire VRAM story in three lines. Job-1 figures include typical runtime overhead (CUDA context, buffers) as you'd see in Activity Monitor; the bottom-up estimator in the lab models the *floor* (~2.2 GB for 1B QLoRA) before that overhead. ```mermaid flowchart LR J1["JOB 1 1B QLoRA @ 4K ~6-8 GB"] --> HW1["Laptop / Mac / Colab T4 (free)"] J2["JOB 2 7B QLoRA @ 2-4K ~10-16 GB"] --> HW2["RTX 4090 24GB ($1,500)"] J3["JOB 3 7B Full FT @ 4K ~100-160 GB"] --> HW3["multi-A100 80GB (~$50K of GPUs)"] J4["JOB 4 70B QLoRA @ 4K ~48-60 GB"] --> HW4["1x A100 80GB"] HW2 -.10x cheaper than Job 3, same model.-> HW3 style J1 fill:#14141f,stroke:#82e0aa,stroke-width:1.5px,color:#e4e4e8 style HW1 fill:#08080c,stroke:#82e0aa,color:#82e0aa style J2 fill:#14141f,stroke:#82e0aa,stroke-width:1.5px,color:#e4e4e8 style HW2 fill:#08080c,stroke:#82e0aa,color:#82e0aa style J3 fill:#14141f,stroke:#f08080,stroke-width:1.5px,color:#e4e4e8 style HW3 fill:#08080c,stroke:#f08080,color:#f08080 style J4 fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8 style HW4 fill:#08080c,stroke:#5eead4,color:#5eead4 ``` --- ## Validation notes - All five diagrams use the course design system colors: `#14141f` panel fill, `#5eead4` accent for primary/positive, `#82e0aa` for "fits" / cheap, `#f08080` for expensive / OOM-risk, `#08080c` for hollow callout fills, `#e4e4e8` / `#9494a0` for text. - All five diagrams use stable `flowchart` syntax (`LR` / `TD`) with `subgraph`, all validated by rendering to SVG in Mermaid 10.9.x (mmdc). Supported in current Mermaid (v10.4+). - The ` ` line-break syntax inside node labels is supported in Mermaid Live for multi-line node text. - Paste each into [Mermaid Live Editor](https://mermaid.live) to render. For the slide deck (artifact 03), these are rendered as static SVG/PNG captures from Mermaid Live, inlined into reveal.js. - Every GB figure matches the teaching document (artifact 01) and the lab (artifact 07) so the three artifacts agree to the gigabyte.