Document Details
Clip:
ADAPTIVELEARNINGRATESCHEDULING BY REFINEMENT When, Why and How Much? Adaptive Learning Rate Scheduling by Refinement Aaron Defazio ADEFAZIO@META.COM FAIR, Meta Ashok Cutkosky ASHOK@CUTKOSKY.COM Boston University Harsh Mehta HARSHM@GOOGLE.COM Google Research Konstantin Mishchenko KONSTA.MISH@GMAIL.COM Samsung AI Center Abstract Learning rate schedules used in practice bear little resemblance to those recommended by theory. We close much of this theory/practice gap, and as a consequence are able to derive newproblem-adaptivelearning rate schedules. Our key technical contribution is a refined analysis of learning rate schedules for a wide class of
Filename:
2310.07831.pdf
Filetype:
application/pdf
Size:
1933245 bytes
Uploaded On:
2024-04-07
Abstract:
Summary:
Tags:
Notes:
Visible:
1
Status:
Parsed
Author:
CreationDate:
2023-10-13T00:48:32+00:00
Creator:
LaTeX with hyperref
Keywords:
ModDate:
2023-10-13T00:48:32+00:00
PTEX.Fullbanner:
This is pdfTeX, Version 3.141592653-2.6-1.40.25 (TeX Live 2023) kpathsea version 6.3.5
Producer:
pdfTeX-1.40.25
Subject:
Title:
Trapped:
False
Pages:
36
Return to Document Library