Neural Networks: Tricks of the TradeGrégoire Montavon, Geneviève Orr, Klaus-Robert Müller The twenty last years have been marked by an increase in available data and computing power. In parallel to this trend, the focus of neural network research and the practice of training neural networks has undergone a number of important changes, for example, use of deep learning machines. The second edition of the book augments the first edition with more tricks, which have resulted from 14 years of theory and experimentation by some of the world's most prominent neural network researchers. These tricks can make a substantial difference (in terms of speed, ease of implementation, and accuracy) when it comes to putting algorithms to work on real problems. |
Other editions - View all
Neural Networks: Tricks of the Trade Grégoire Montavon,Geneviève Orr,Klaus-Robert Müller No preview available - 2012 |
Common terms and phrases
Advances in Neural applied approach approximation architecture average backpropagation batch Bengio Boltzmann machine centering classification clusters convergence cost function cross-validation curvature dataset distribution dynamics early stopping echo state networks eigenvalue epochs equation error function estimate example feed-forward forecasting Gaussian Heidelberg 2012 Hessian hidden layer hidden units hyper-parameters IEEE initial iteration K-means learning algorithm learning rate linear LNCS Machine Learning matrix method minibatch minimal Montavon Neural Computation Neural Information Processing neural network nodes noise nonlinear optimization output overfitting parameters pattern performance prediction problem pruning quadratic random recognition Recurrent Neural Networks regularization reinforcement learning reservoir sampling scaling sigmoid SMLP sparse coding speech recognition stochastic gradient descent supervised learning tangent distance tangent vectors target task techniques test set training data training set Tricks typically unsupervised update validation error validation set values weight decay zero