且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

Spark 2.2 非法模式组件:XXX java.lang.IllegalArgumentException:非法模式组件:XXX

更新时间:2023-01-24 12:13:46

我找到了答案.

timestampFormat 的默认值是 yyyy-MM-dd'T'HH:mm:ss.SSSXXX,这是一个非法参数.写数据帧时需要设置.

The default for the timestampFormat is yyyy-MM-dd'T'HH:mm:ss.SSSXXX which is an illegal argument. It needs to be set when you are writing the dataframe out.

解决方法是将其更改为包含时区的 ZZ.

The fix is to change that to ZZ which will include the timezone.

df.write
.option("timestampFormat", "yyyy/MM/dd HH:mm:ss ZZ")
.mode(SaveMode.Overwrite)
.csv("my.csv")