且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

通过 spark-submit 将额外的 jars 传递给 Spark

更新时间:2021-10-23 02:40:27

问题是CLASSPATH应该用冒号分隔,而JARS应该用逗号分隔:>

The problem is that CLASSPATH should be colon separated, while JARS should be comma separated:

$SPARK_HOME/bin/spark-submit 
--driver-class-path /usr/local/share/mongo-hadoop/build/libs/mongo-hadoop-1.5.0-SNAPSHOT.jar:/usr/local/share/mongo-hadoop/spark/build/libs/mongo-hadoop-spark-1.5.0-SNAPSHOT.jar 
--jars /usr/local/share/mongo-hadoop/build/libs/mongo-hadoop-1.5.0-SNAPSHOT.jar,/usr/local/share/mongo-hadoop/spark/build/libs/mongo-hadoop-spark-1.5.0-SNAPSHOT.jar my_application.py