深度学习应用技术研究
毛勇华;桂小林;李前;贺兴时
【期刊名称】《计算机应用研究》 【年(卷),期】2016(033)011
【摘要】This paper reviewed the deep learning algorithms and their applications.It elaborated the greedy layer training al-gorithm which used the fine-grained back-propagation (BP)learning following the layer-wise pre-training on each restricted Boltzmann machine (RBM)layer.After comparing and analyzing the three ways of gradient descent in the BP algorithm,this paper suggested applying stochastic gradient descent in online learning and adopting stochastic mini-batch gradient descent in static offline learning.It summarized the characteristic of the network structure in deep learning and recommend the design of state-of-art five-layer network architecture.It also analyzed the necessity of the nonlinear activation function in feedforward neural networks and the advantages of the common activation functions,and recommended using ReLU activate function.Fi-nally,the paper provided a brief summary of features and application scenarios of emerging deep neural networks such as deep CNN(convolutional neural networks),deep RNNs(recurrent neural networks)and LSTM(long short-termmemory networks), as well as the potential directions of future deep learning applications and research.%针对深度学习应用技术进行了研究性综述。详细