更新时间:2021-12-15 17:51:48
我不认为可以使用--properties-file
从IntelliJ IDEA内部启动Spark应用程序.
I don't think it's possible to use --properties-file
to launch a Spark application from within IntelliJ IDEA.
spark-submit
是用于提交Spark应用程序以执行的Shell脚本,在为Spark应用程序创建适当的提交环境之前,它不需要做任何额外的事情.
spark-submit
is the shell script to submit Spark application for execution and does few extra things before it creates a proper submission environment for the Spark application.
不过,您可以利用Spark应用程序默认加载的conf/spark-defaults.conf
来模仿--properties-file
的行为.
You can however mimic the behaviour of --properties-file
by leveraging conf/spark-defaults.conf
that a Spark application loads by default.
您可以在src/test/resources
(或src/main/resources
)下使用内容properties.conf
创建一个conf/spark-defaults.conf
.那是应该起作用的.
You could create a conf/spark-defaults.conf
under src/test/resources
(or src/main/resources
) with the content of properties.conf
. That is supposed to work.