更新时间:2023-11-15 20:37:22
确保在纱线模式下运行时,Spark可以使用YARN配置.将这些文件core-site.xml
,hdfs-site.xml
和yarn-site.xml
文件添加到spark的conf
目录中.
另外请确保yarn-site.xml
包含资源管理器的地址
Ensure the YARN configurations are available for Spark to use when running in yarn mode. Add these files core-site.xml
, hdfs-site.xml
and yarn-site.xml
files to the conf
directory of spark.
Also make sure, the yarn-site.xml
contains the address of the resource manager
<property>
<name>yarn.resourcemanager.address</name>
<value>resource_manager_ip:8032</value>
</property>