Optimizer Journey
RESEARCH DIARY / OPTIMIZERS
The optimizer journey.
Follow the evolution from scalar adaptation to matrix geometry. Open a chapter only when you want its details.
ADAM build scalar memory
ADAMW separate decay
MUON normalize matrix directions