且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

用预训练的词嵌入训练CNN非常慢(TensorFlow)

更新时间:2022-05-04 00:45:57

所以这里有两个问题:

  1. 正如mrry在他对问题的评论中指出的那样,警告不是更新期间进行转换的结果.相反,我正在计算关于嵌入渐变的摘要统计信息(稀疏度和直方图),并引起了转换.
  2. 有趣的是,删除摘要使消息消失了,但是代码仍然很慢.根据问题中引用的TensorFlow问题,我还必须将AdamOptimizer替换为AdagradOptimizer,然后我又确定运行时与从少量词汇中获得的运行时相当.
  1. As mrry pointed out in his comment to the question, the warning was not a result of a conversion during the updates. Rather, I was calculating summary statistics (sparsity and histogram) on the embeddings gradient and that caused the conversion.
  2. Interestingly, removing the summaries made the message go away, but the code remained slow. Per the TensorFlow issue referenced in the question, I had to also replace the AdamOptimizer with the AdagradOptimizer and once I did that the runtime was back on par with the one obtained from a small vocabulary.