Joey Grasty cbd88aa758 Correct FTPO slowness diagnosis: fixed-length padding, not compute-bound ...

Measured actual FTPO training context lengths (real tokenizer, all 12,000
examples): mean 530 tokens, p99 1080, max 1126 -- against a configured
finetune_max_seq_length of 4000. ftpo_trainer.py's collator pads every batch
to that fixed length rather than to the longest sequence in the batch, so
every forward pass was processing ~4000 tokens of mostly padding (~13%
utilization on average). This also explains why batch_size 1->4 had no
effect: total padded-token compute is invariant to the batch/accum split.

Lowered finetune_max_seq_length to 1280 (covers p99 with headroom, nothing
in the dataset gets truncated) -- should cut per-step compute roughly 3x.
Updated DGX_SPARK_SETUP.md §7 with the measurement and corrected takeaway.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

2026-08-08 09:57:03 -05:00

15 KiB

Raw Blame History

View Raw