且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

使用 Java 将 spark RDD 保存到本地文件系统

更新时间:2023-10-15 09:42:10

saveAsTextFile 能够接收本地文件系统路径(例如 file:///tmp/magic/...).但是,如果您在分布式集群上运行,您很可能希望 collect() 将数据返回到集群,然后使用标准文件操作将其保存.

saveAsTextFile is able to take in local file system paths (e.g. file:///tmp/magic/...). However, if your running on a distributed cluster, you most likely want to collect() the data back to the cluster and then save it with standard file operations.