且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

你能给我任何线索为什么'不能调用停止的SparkContext方法'?

更新时间:2022-10-19 13:47:44

Your YARN application exits immediately after it starts:

16/03/19 05:59:41 WARN cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster has disassociated: 192.168.111.203:34633
16/03/19 05:59:41 WARN cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster has disassociated: 192.168.111.203:34633
16/03/19 05:59:41 WARN remote.ReliableDeliverySupervisor: Association with remote system [akka.tcp://sparkYarnAM@192.168.111.203:34633] has failed, address is now gated for [5000] ms. Reason: [Disassociated] 
16/03/19 05:59:41 ERROR cluster.YarnClientSchedulerBackend: Yarn application has already exited with state FINISHED!

Then, SparkContext is closed, so any action on this context will throw the exception you see.

Check the "Application Master" logs (visible through YARN's UI) to see the cause for the failure. This could be a memory configuration issue, network issues (e.g. host unreachable) and more - the log on the driver side (which is what you pasted) won't tell you which one it is.

相关阅读

技术问答最新文章