且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

超时错误:400 StatusCode错误:“请求失败:会话未激活."

更新时间:2023-01-03 12:38:51

来自从输出来看,如果您的应用程序未以FAILED状态完成,这听起来像是Livy超时错误:您的应用程序可能比Livy会话所定义的超时时间更长(默认为1h),所以即使尽管Spark应用程序成功运行,但如果该应用程序花费的时间比Livy会话的超时时间长,则您的笔记本仍会收到此错误.

Judging by the output, if your application is not finishing with a FAILED status, that sounds like a Livy timeout error: your application is likely taking longer than the defined timeout for a Livy session (which defaults to 1h), so even despite the Spark app succeeds your notebook will receive this error if the app takes longer than the Livy session's timeout.

如果是这种情况,请按以下步骤处理:

If that's the case, here's how to address it:

1. edit the /etc/livy/conf/livy.conf file (in the cluster's master node)
2. set the livy.server.session.timeout to a higher value, like 8h (or larger, depending on your app)
3. restart Livy to update the setting: sudo restart livy-server in the cluster's master
4. test your code again