且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

在spark scala中将1列拆分为3列

更新时间:2023-11-18 21:29:22

尝试:

import sparkObject.spark.implicits._
import org.apache.spark.sql.functions.split

df.withColumn("_tmp", split($"columnToSplit", "\\.")).select(
  $"_tmp".getItem(0).as("col1"),
  $"_tmp".getItem(1).as("col2"),
  $"_tmp".getItem(2).as("col3")
)

这里要注意的重点是 sparkObject 是您可能已经初始化的 SparkSession 对象.因此,(1) import 语句必须强制内联在代码中,而不是在类定义之前.

The important point to note here is that the sparkObject is the SparkSession object you might have already initialized. So, the (1) import statement has to be compulsorily put inline within the code, not before the class definition.