Ran a 30-minute validation with finetune_max_seq_length=1280 to confirm the
predicted speedup from the previous commit. Measured ~40s/step steady-state
(vs ~160-185s/step at max_seq_length=4000) -- a ~4.3x speedup, better than
the ~3x predicted from the token-count ratio alone. Extrapolated full-run
ETA drops from ~34h to ~8.3h. No errors across the validation run.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Measured actual FTPO training context lengths (real tokenizer, all 12,000
examples): mean 530 tokens, p99 1080, max 1126 -- against a configured
finetune_max_seq_length of 4000. ftpo_trainer.py's collator pads every batch
to that fixed length rather than to the longest sequence in the batch, so
every forward pass was processing ~4000 tokens of mostly padding (~13%
utilization on average). This also explains why batch_size 1->4 had no
effect: total padded-token compute is invariant to the batch/accum split.
Lowered finetune_max_seq_length to 1280 (covers p99 with headroom, nothing
in the dataset gets truncated) -- should cut per-step compute roughly 3x.
Updated DGX_SPARK_SETUP.md §7 with the measurement and corrected takeaway.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- utils/vllm_manager.py: drop --disable-log-requests, removed in vLLM 0.26.0
- core/ftpo_trainer.py: pass token_type_ids to the 3 model forward calls in
compute_loss -- transformers 5.5.0's Gemma3 requires it during training
for causal-mask construction (Gemma3 is natively multimodal)
- configs/gemma-3-4b-it.yaml: lower vllm_gpu_memory_utilization 0.85->0.5,
since the DGX Spark's 121GB is unified CPU/GPU memory and the default
starved the OS, causing swap thrashing
- DGX_SPARK_SETUP.md: full writeup of the above plus the parts that don't
live in this repo (two-conda-env split to resolve a vllm/unsloth
transformers version conflict, flash-attn source build flags, torch/CUDA
version matching, ~/.triton/cache permissions)
The antislop-vllm submodule also needed a one-line fix (removing an invalid
reference_compile kwarg in utils/refusal_detector.py that was silently
disabling refusal filtering) -- documented in DGX_SPARK_SETUP.md rather than
committed as a submodule pointer change, since we don't have push access to
upstream's antislop-vllm repo.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
