且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

在纱线上产生火花,在/0.0.0.0:8032处连接到ResourceManager

更新时间:2023-11-15 20:37:22

确保在纱线模式下运行时,Spark可以使用YARN配置.将这些文件core-site.xmlhdfs-site.xmlyarn-site.xml文件添加到spark的conf目录中.
另外请确保yarn-site.xml包含资源管理器的地址

Ensure the YARN configurations are available for Spark to use when running in yarn mode. Add these files core-site.xml, hdfs-site.xml and yarn-site.xml files to the conf directory of spark.
Also make sure, the yarn-site.xml contains the address of the resource manager

<property>
   <name>yarn.resourcemanager.address</name>
   <value>resource_manager_ip:8032</value>
</property>