且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

在纱线客户端模式下设置/运行火花(火花壳)

更新时间:2023-11-15 20:42:04

实际上是我所描述的相反方法.由于Yarn + Java 8问题导致连接失败( https://issues.apache.org/jira/browse/YARN-4714 ).sparkContext创建失败.

It's actually the reverse way as I described. Due to the connection failure caused by Yarn + Java 8 problem(https://issues.apache.org/jira/browse/YARN-4714). The sparkContext failed to be created.

所做的更改应在yarn-site.xml文件中添加以下部分:

the change should be add the following section in the yarn-site.xml file:

<property>
    <name>yarn.nodemanager.pmem-check-enabled</name>
    <value>false</value>
</property>

<property>
    <name>yarn.nodemanager.vmem-check-enabled</name>
    <value>false</value>
</property>