应用梯度下降和随机梯度下降时的几个注意点 - webdancer's Blog
应用梯度下降和随机梯度下降时的几个注意点
webdancer
posted @ 2012年10月17日 04:33
in machine learning
with tags
Machine learning
, 6400 阅读
随机梯度下降算法的基本思想，在前面已经说过了。在应用梯度下降的时候有几个要注意的点，再重新记录一下。
1.收敛
我们在应用梯度下降的时候，一定要关注是否收敛。一般来说，$\mathbb E(\theta)$收敛到一定的值，所求出来的参数也是基本相同的。能使用梯度下降的求解的问题，影响我们求解的是它的学习速率（可以参考前面的内容）。速率过大，可能不收敛；速率过小，收敛速度过慢，所以我们必须恰当的选择学习速率$\eta$。通过下面的例子看以看到[1]：
当学习速率为50时，不收敛；当学习速率为0.01时收敛过慢，所以在训练时候一定要选择恰当的参数。
2.随机shuffle
在随机梯度学习算法中，如果训练集中正例和负例都聚集在一块。不进行shuffle可能会造成'wavy'现象[2]，如下图所示:
[引用]：
[1]自己实现的logistic regression model,然后在上面做的实验，得出的图1。
https://github.com/seaslee/ML_algorithm.git
[2]http://openclassroom.stanford.edu/TestFolder/courses/ufldl/exercises/ex1/ex1.html
2022年9月10日 07:29
General Knowlorge is most important subject to all Class 10 students studying in English Medium, AP SSC Gk ModelPaper Telugu Medium & Urdu Medium of the State Board. So every student who is studying in the state Government & Private Schools can download the AP 10th GK Model Paper 2023 Pdf with answer solutions designed and suggested by subject experts. General Knowlorge is most important subject to all Class 10 students studying in English Medium.
2023年2月12日 21:38
Board of Secondary Education board will soon declare CBSE 10th Blueprint 2024 through its official website. Blueprint 2024 10th CBSE board will declare its Central Board of Secondary Education 10th Blueprint 2024 in the month of May this year and this board published the notification regarding the examinations of class 10th and after a few time in the month of March exams were started from March.
2024年1月15日 21:36
Really impressive post. I read it whole and going to share it with my social circules. I enjoyed your article and planning to rewrite it on my own blog