更新时间:2022-05-20 02:57:39
1)清除环境变量:
unset SPARK_CLASSPATH
2)使用--jars选项在群集上分布postgres驱动程序:
2) Use --jars option to distribute postgres driver over your cluster:
pyspark --jars=/home/hadoop/pg_jars/postgresql-9.4.1208.jre7.jar
//or
spark-submit --jars=/home/hadoop/pg_jars/postgresql-9.4.1208.jre7.jar <your py script or app jar>