DL Training Optimization

PyTorch 2.x + HuggingFace training tuning; diagnose first, change one thing at a time.

Quick Reference

| Situation | Action | |-----------|--------| | GPU util below 70% | Fix DataLoader: workers up, batch up, pin_memory | | Loss not decreasing | lr/10 or lr*2; check labels/data | | Loss oscillating | Lower lr; clip_grad_norm=1.0 | | Loss NaN | Switch bf16; lower lr; clip | | OOM | accum up; checkpoint; FSDP/ZeRO; LoRA; 8bit AdamW | | Multi-GPU no speedup | Replace DataParallel with DDP | | Pick optimizer/lr | See Config Format + Resolve and Tune | | LLM / LoRA / RNN / CV | See Task Recipes |

Setup

cp -r dl-training-optimization ~/.cursor/skills/
# Cursor: ~/.cursor/skills/dl-training-optimization/

Cursor: @dl-training-optimization or describe training issues in chat.

Stack: PyTorch 2.x, HuggingFace Trainer/accelerate/peft, optional bitsandbytes/DeepSpeed

Config Format

Fields: opt | peak lr | wd | sched | precision | clip | accum | DDP | Status: baseline|tuning|stable|failed

LLM finetune (HF)

# optim=adamw_torch_fused lr=2e-5 warmup_ratio=0.03 cosine bf16=True max_grad_norm=1.0
from transformers import get_cosine_schedule_with_warmup
sched = get_cosine_schedule_with_warmup(opt, num_warmup_steps, total_steps)

Categories: pretrain, finetune, lora

RNN / Seq2Seq

# AdamW lr=1e-3 wd=0.01 clip=5 batch=32-64; teacher forcing decay late
torch.nn.utils.clip_grad_norm_(model.parameters(), 5.0)

Categories: translation, seq2seq, clip_required

CV / ImageNet

# SGD+Nesterov lr=0.1*B/256 momentum=0.9 nesterov wd=1e-4 cosine DDP
torchrun --nproc_per_node=8 train.py

Categories: from_scratch, finetune, distributed

Resolve and Tune

Resolve: log before/after loss and throughput; one change at a time

Promote when: fix applies across tasks, prevents recurring NaN/OOM, documents team defaults

| Target | Content | |--------|---------| | Default | AdamW fused lr=1e-3 wd=0.01 cosine bf16 | | LLM pipeline | warmup+cosine clip=1.0 HF Trainer | | CV pipeline | SGD+Nesterov linear lr scaling DDP | | RNN pipeline | AdamW + clip=1~5 + Plateau | | Low VRAM | 8bit AdamW, LoRA, gradient checkpointing |

Steps: confirm task type -> map symptom -> apply one fix -> output config + code + next step

Detection Triggers

| Signal | Action | |--------|--------| | "Training slow / GPU idle" | DataLoader first | | "Loss not going down" | lr/10 or lr*2, check data | | "Loss spikes / NaN" | bf16, lr/10, clip=1.0 | | "CUDA OOM" | accum, checkpoint, FSDP, LoRA | | "Multi-GPU no gain" | switch to DDP | | "Which optimizer?" | Config Format + Resolve and Tune |

Priority

DataLoader=IO bottleneck | bf16=precision | AdamW+sched=convergence | clip/accum=stability | DDP/FSDP=scale | compile/LoRA=last resort

Recurring Patterns

1. Check GPU util and DataLoader before changing lr
2. Confirm task type (LLM / CV / RNN) and framework
3. Apply one change; record loss and throughput
4. Recurring NaN/OOM -> Emergency Fixes or promote team default

Periodic Review

When: new task type, OOM/NaN after stable run, batch size change, multi-GPU added

nvidia-smi
# log: loss, lr, throughput, global_batch

Actions: fix DataLoader, adjust lr schedule, enable DDP, add clip/accum

Task Recipes

Use when: new project, finetune LLM, train RNN/CV from scratch, user asks for config.

dtype = torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float16
scaler = torch.cuda.amp.GradScaler(enabled=(dtype == torch.float16))
opt = torch.optim.AdamW(model.parameters(), lr=peak_lr, weight_decay=0.01, fused=True)
for i, (x, y) in enumerate(loader):
    with torch.autocast("cuda", dtype=dtype):
        loss = model(x, y) / accum
    scaler.scale(loss).backward()
    if (i + 1) % accum == 0:
        scaler.unscale_(opt)
        torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
        scaler.step(opt); scaler.update(); opt.zero_grad(set_to_none=True)

Gates: AdamW not Adam | bf16 on Ampere+ | clip for RNN | warmup for LLM | DDP not DataParallel

Hooks (opt-in)

HuggingFace Trainer args:

{"optim":"adamw_torch_fused","lr":2e-5,"warmup_ratio":0.03,"bf16":true,"max_grad_norm":1.0}

Add torch.compile after stable; Transformer: attn_implementation=flash_attention_2

Gitignore

Local only: experiment logs and runs | Team shared: commit training configs | Hybrid: configs in repo + local runs

Best Practices

Fix DataLoader first | AdamW not Adam | bf16 on Ampere+ | One change at a time | clip RNN always | warmup for LLM | DDP not DataParallel | Check GPU util before architecture changes

Multi-Agent

| Stack | Activation | |-------|------------| | PyTorch 2.x | Manual loop, DDP, torch.compile | | HuggingFace Trainer | LLM finetune defaults | | accelerate / peft | LoRA, multi-GPU | | bitsandbytes | 8bit AdamW | | DeepSpeed | ZeRO-2/3 large models |

Apply when: training slow, loss unstable, optimizer/lr choice, OOM, or multi-GPU setup.