更新时间:2023-02-05 14:32:46
我不确定我是否已经完全了解您的需求.
I'm not sure if I've fully understand what you need.
无论如何,如果要对DF进行排序,则可以使用sortBy(如果是(K,V),则可以使用sortByKey)
Anyway, if you want to sort a DF you could use sortBy (or sortByKey in case of (K,V))
例如,如果我们假设有一个DF(在本例中为Spark SQL),则可以按以下方式对其进行排序:
For example, if we assume to have a DF (in this case coming from Spark SQL), we can sort it like this:
val sqlResult = sqlContext.sql("select first_column, second_column from logs").toDF()
val result = sqlResult.sortBy(x=>x._1) // first column sorting
如前所述,您可以对任何DF进行排序,但是我只想展示另一种使用Spark SQL访问"数据,然后使用Spark核心功能对数据进行排序的方法.
As said before, you can sort any DF, but I just want to show another way to "access" data with Spark SQL, and then sorting them with Spark core functionalities.
希望它会有所帮助!
FF