更新时间:2022-05-23 08:04:05
First of all, let me point out that tf.contrib.learn.DNNRegressor
uses a linear regression head with mean_squared_loss
, i.e. simple L2 loss.
是否为整体损失函数值(每个 步骤处理到现在为止)还是只是该步骤的损失值?
Whether it is overall loss function value (combined loss for every step processed till now) or just that step's loss value?
图表上的每个点是到目前为止学习后最后一步的损失函数的值.
Each point on a chart is the value of a loss function on the last step after learning so far.
如果是该步骤的损失值,那么如何获得总损失的值 功能并查看其趋势,我认为应该随着 增加迭代次数吗?
If it is that step's loss value, then how to get value of overall loss function and see its trend, which I feel should decrease with increasing no of iterations?
没有整体损失函数,可能是指图表说明每一步损失的变化情况.这正是张量板向您显示的内容.您说得对,它的趋势并非应有的下降趋势.这表明您的神经网络没有学习.
There's no overall loss function, probably you mean a chart how the loss changed after each step. That's exactly what tensorboard is showing to you. You are right, its trend is not downwards, as it should. This indicates that your neural network is not learning.
如果这是总体损失值,那么为什么波动如此之大?我想念什么吗?
If this is overall loss value, then why is it fluctuating so much? Am I missing something?
A common reason for the neural network not learning is poor choice of hyperparameters (though there are many more mistakes you can possibly make). For example:
我通常要检查神经网络是否至少以某种方式起作用,这是将训练集减少到几个示例,并尝试使网络过拟合.这个实验非常快,因此我可以尝试各种学习率,初始化方差和其他参数来找到一个***位置.一旦有了稳定的,不断减少的损失图,我便继续进行更大的调整.
What I usually do to check if the neural network is at least somehow working is reduce the training set to few examples and try to overfit the network. This experiment is very fast, so I can try various learning rates, initialization variance and other parameters to find a sweet spot. Once I have a steady decreasing loss chart, I go on with a bigger set.