且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

我可以将多个位置指向同一个配置单元外部表吗?

更新时间:2022-12-08 15:54:05

简单答案:否,在创建过程中Hive external表的location必须是唯一的,元存储库需要使用它来了解您的位置.餐桌生活.

Simple answer: no, the location of a Hive external table during creation has to be unique, this is needed by the metastore to understand where your table lives.

话虽这么说,您可能可以摆脱使用分区的麻烦:您可以为每个分区指定一个location,这似乎是您最终想要的分区,因为按月划分.

That being said, you can probably get away with using partitions: you can specify a location for each of your partitions which seems to be what you want ultimately since you are splitting by month.

因此,像这样创建您的表:

So create your table like this:

create external table logdata(col1 string, col2 string) partitioned by (month string) location 's3://logdata'

然后您可以添加如下分区:

Then you can add partitions like this:

alter table logdata add partition(month='april') location 's3://logdata/april'

您每个月都要这样做一次,现在您可以查询表以指定所需的分区,并且Hive只会查看您实际需要数据的目录(例如,如果您仅处理4月和6月,配置单元可能不会加载)

You do this for every month, and now you can query your table specifying whichever partition you want, and Hive will only look at the directories for which you actually want data (for example if you're only processing april and june, Hive will not load may)