且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

将数据从Amazon S3复制到Redshift,并避免重复的行

更新时间:2022-11-06 22:36:59

我的解决方案是在对表进行复制"之前运行删除"命令.在我的用例中,每次需要将每日快照的记录复制到redshift表时,因此可以使用以下删除"命令来确保删除重复的记录,然后运行复制"命令.

My solution is to run a 'delete' command before 'copy' on the table. In my use case, each time I need to copy the records of a daily snapshot to redshift table, thus I can use the following 'delete' command to ensure duplicated records are deleted, then run the 'copy' command.

从t_data中删除,其中snapshot_day ='xxxx-xx-xx';

DELETE from t_data where snapshot_day = 'xxxx-xx-xx';