论文标题
改进的二进制前向探索：随机优化的学习率调度方法
Improved Binary Forward Exploration: Learning Rate Scheduling Method for Stochastic Optimization
论文作者
论文摘要
最近已经提出了一种新的基于梯度的优化方法，该方法最近提出了学习率，这被称为二进制前进探索（BFE）。此后还讨论了BFE的自适应版本。在本文中，将研究基于它们的改进算法，以优化新方法的效率和鲁棒性。这种改进的方法为安排学习率的更新提供了一种新的观点，并将与随机梯度下降，又称SGD算法和Nesterov动量和最成功的适应性学习算法算法进行比较。亚当。这种方法的目标不是旨在击败他人，而是提供不同的观点来优化梯度下降过程。这种方法结合了速度和效率方面的一阶和二阶优化的优势。
A new gradient-based optimization approach by automatically scheduling the learning rate has been proposed recently, which is called Binary Forward Exploration (BFE). The Adaptive version of BFE has also been discussed thereafter. In this paper, the improved algorithms based on them will be investigated, in order to optimize the efficiency and robustness of the new methodology. This improved approach provides a new perspective to scheduling the update of learning rate and will be compared with the stochastic gradient descent, aka SGD algorithm with momentum or Nesterov momentum and the most successful adaptive learning rate algorithm e.g. Adam. The goal of this method does not aim to beat others but provide a different viewpoint to optimize the gradient descent process. This approach combines the advantages of the first-order and second-order optimizations in the aspects of speed and efficiency.
