且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

将数据从S3复制到Redshift挂起

更新时间:2022-11-06 21:02:08

我同意JohnRotenstein的观点,即那里需要更多信息来提供答案.我建议您采取简单的数据点和简单的表格. 这是分步解决方案,我希望这样做,您应该能够解决您的问题.

I agree with JohnRotenstein that, there needs more information to provide the answer. I would suggest you to take simple data points and simple table. Here are step-by-step solution, I hope by doing that, you should be able to resolve your issue.

假设这里是您的表结构.

Assume here is your table structure.

在这里,我正在处理大多数数据类型以证明我的观点. 创建表销售( salesid整数, 佣金十进制(8,2), 销售日期, 说明varchar(255), created_at时间戳默认系统日期, 时间戳);

Here I'm doing most of data types to prove my point. create table sales( salesid integer, commission decimal(8,2), saledate date, description varchar(255), created_at timestamp default sysdate, updated_at timestamp);

为了简单起见,这是您的数据文件驻留在S3中.
CSV(sales-example.txt)中的内容

Just to make it simple, here is your data file resides in S3.
Content in CSV(sales-example.txt)

salesid,commission,saledate,description,created_at,updated_at
1|3.55|2018-12-10|Test description|2018-05-17 23:54:51|2018-05-17 23:54:51
2|6.55|2018-01-01|Test description|2018-05-17 23:54:51|2018-05-17 23:54:51
4|7.55|2018-02-10|Test description|2018-05-17 23:54:51|2018-05-17 23:54:51
5|3.55||Test description|2018-05-17 23:54:51|2018-05-17 23:54:51
7|3.50|2018-10-10|Test description|2018-05-17 23:54:51|2018-05-17 23:54:51

使用psql终端或任何sql连接器运行以下两个命令.确保还运行第二个命令.

Run following two command using the psql terminal or any sql connector. Make sure to run second command as well.

copy sales(salesid,commission,saledate,description,created_at,updated_at) from 's3://example-bucket/foo/bar/sales-example.txt' credentials 'aws_access_key_id=************;aws_secret_access_key=***********' IGNOREHEADER  1;

commit;

我希望,这应该可以帮助您调试问题.

I hope, this should help you in debugging your issue.