且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

如何使用Java UDF向Spark数据框添加新列

更新时间:2021-08-07 22:34:37

出现错误是因为您尝试使用Dataset.map()的结果创建函数文字(lit()),您可以在文档中看到的是数据集.您可以在Dataset.withColumn()的API中看到,您需要一个作为列的参数.

You get the error because you are trying to create a function literal (lit()) using the result of Dataset.map(), which you can see in docs is a Dataset. You can see in the API for Dataset.withColumn() that you need a argument that is a column.

似乎您需要创建一个用户定义的函数.看看如何调用使用JAVA在Spark DataFrame上创建UDF?

It seems like you need to create a user-defined function. Take a look at How do I call a UDF on a Spark DataFrame using JAVA?