且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

如何在Scala/Spark中将地图数组转换为单个地图列?

更新时间:2023-10-20 15:27:28

您将需要explode数组列.

.withColumn("map_output", explode(col("CLICK_THROUGH_RATE_MAP")))

如果数组容纳多个value,则

explode将为同一row生成多个rows.

explode will generate multiple rows for the same row if an array will hold more than one value.

如果可以保证列数组将始终保留一个值,那么您也可以使用以下两种方法之一:

And if it is guaranteed that column array will always hold one single value, you can use either of the below methods too:

.withColumn("map_output", col("CLICK_THROUGH_RATE_MAP")(0))

//@since 2.4.0
.withColumn("map_output", element_at(col("CLICK_THROUGH_RATE_MAP"), 0))