我想将@JacekLaskowski的另一种解决方案添加到我有时用于POC或测试目的的解决方案中.

I want to add to @JacekLaskowski's an alternative solution I use sometimes for POC or tests purposes.

将script.scala内部的script.scala与:load一起使用.

It would be to use the script.scala from inside the spark-shell with :load.

:load /path/to/script.scala

您不需要定义SparkContext/SparkSession，因为脚本将使用REPL范围内定义的变量.

You won't need to define a SparkContext/SparkSession as the script will use the variables defined in the scope of the REPL.

您也不需要将代码包装在Scala对象中.

You also don't need to wrap the code in a Scala object.

PS:我将其更多地视为黑客行为，而不是用于生产目的.

PS: I consider this more as a hack and not to use for production purposes.