An Adaptive Optimizer without User-Specified Hyperparameters through Historical Projection and Self-Calibrated Scaling

YamAdam, an adaptive optimizer for mini-batch training, reaches predefined targets rapidly across six heterogeneous workloads without any user-specified optimizer hyperparameters, matching 2,634/2,700 targets.

• Why it matters: Adaptive optimizers still depend on user-chosen learning-rate and schedules, creating sensitivity that limits reliable performance across diverse tasks and models.
• What they did: YamAdam uses short-memory Adam moments with decay factors fixed at 1/2, a historical projection removing only opposing historical components, one-time calibration of a reference update radius, and later dimensionless directional scaling. It is evaluated against 30 fixed settings from 10 optimizer families using {Delta}TTT.
• The result: Across 540 setting-workload-seed comparisons, YamAdam is faster in 504, tied in 14, and slower in 22; all 30 comparator settings yield positive {Delta}TTT (+0.003526 to +0.210524). Across 2,700 target observations, both methods reach 2,634 targets, only YamAdam reaches 66, and no target is reached only by the comparator or by neither method.