KISHORE VENKATESHAN
Optimizer Journey
RESEARCH DIARY / OPTIMIZERS

The optimizer journey.

Follow the evolution from scalar adaptation to matrix geometry. Open a chapter only when you want its details.

ADAM build scalar memory
ADAMW separate decay
MUON normalize matrix directions