且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

如何在Tensorflow DNNRegressor Estimator模型中解释损失函数?

更新时间:2022-05-23 08:04:05

首先,让我指出

First of all, let me point out that tf.contrib.learn.DNNRegressor uses a linear regression head with mean_squared_loss, i.e. simple L2 loss.

是否为整体损失函数值(每个 步骤处理到现在为止)还是只是该步骤的损失值?

Whether it is overall loss function value (combined loss for every step processed till now) or just that step's loss value?

图表上的每个点是到目前为止学习后最后一步的损失函数的值.

Each point on a chart is the value of a loss function on the last step after learning so far.

如果是该步骤的损失值,那么如何获得总损失的值 功能并查看其趋势,我认为应该随着 增加迭代次数吗?

If it is that step's loss value, then how to get value of overall loss function and see its trend, which I feel should decrease with increasing no of iterations?

没有整体损失函数,可能是指图表说明每一步损失的变化情况.这正是张量板向您显示的内容.您说得对,它的趋势并非应有的下降趋势.这表明您的神经网络没有学习.

There's no overall loss function, probably you mean a chart how the loss changed after each step. That's exactly what tensorboard is showing to you. You are right, its trend is not downwards, as it should. This indicates that your neural network is not learning.

如果这是总体损失值,那么为什么波动如此之大?我想念什么吗?

If this is overall loss value, then why is it fluctuating so much? Am I missing something?

神经网络无法学习的常见原因是对超参数的选择不正确(尽管

A common reason for the neural network not learning is poor choice of hyperparameters (though there are many more mistakes you can possibly make). For example:

  • 学习率太大
  • 学习率过低也有可能,这意味着神经网络正在学习,但是学习速度非常慢,因此您看不到它.
  • 权重初始化可能太大,请尝试减小它
  • 批量大小可能也太大
  • 您为输入内容传递了错误的标签
  • 培训数据包含缺失值或未规范化
  • ...

我通常要检查神经网络是否至少以某种方式起作用,这是将训练集减少到几个示例,并尝试使网络过拟合.这个实验非常快,因此我可以尝试各种学习率,初始化方差和其他参数来找到一个***位置.一旦有了稳定的,不断减少的损失图,我便继续进行更大的调整.

What I usually do to check if the neural network is at least somehow working is reduce the training set to few examples and try to overfit the network. This experiment is very fast, so I can try various learning rates, initialization variance and other parameters to find a sweet spot. Once I have a steady decreasing loss chart, I go on with a bigger set.