HiLo gradient compression

Tag "HiLo gradient compression" — all articles

112026-11Two arXiv papers target communication efficiency in distributed LoRA trainingTwo papers posted to arXiv on Nov 11, 2026 tackle efficiency in distributed LoRA training. The HiLo framework replaces parameter transmission with gradient compression, cutting transmitted bytes to 1.6% and speeding updates by up to 2.2x; SPRA-LoRA freezes the A matrix and uses rank-aware learning rates to halve communication and compute overhead. Both methods target the communication bottleneck in bandwidth-constrained settings.arXiv cs.LG