且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

将Hadoop中的文件导入到Web应用程序中

更新时间:2023-01-21 23:10:52

在Hadoop中,首先,您必须确保Hadoop已启动并正在运行。 Apache Hadoop提供了Java类--FileSystem来从Java应用程序访问HDFS中的文件。下面是一个示例,我正在使用FileSystem和IOUtils访问/books/pg5000.txt。

In Hadoop, firstly, you would have to make sure that Hadoop is up and running. Apache Hadoop provides Java classes - FileSystem to access the files in HDFS from the Java application. One example is below, I am accessing /books/pg5000.txt using FileSystem and IOUtils.

import java.io.InputStream;
import java.net.URI;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;


public class FileSystemCat {

        public static void main(String[] args) throws Exception {
            Configuration conf = new Configuration();
            conf.addResource(new Path("/usr/local/hadoop/etc/hadoop/core-site.xml"));
            conf.addResource(new Path("/usr/local/hadoop/etc/hadoop/hdfs-site.xml"));
          String uri = "/books/pg5000.txt";
        FileSystem fs = FileSystem.get(URI.create(uri), conf);
        InputStream in = null;
        try {
            in = fs.open(new Path(uri));
            IOUtils.copyBytes(in, System.out, 4096, false);
            } finally {
            IOUtils.closeStream(in);
            }
        }
}