且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

远程linux服务器到远程linux服务器大稀疏文件复制 - 如何?

更新时间:2023-02-02 17:23:50

如果要快速复制大型数据, rsync 不适合你。由于运行用于快速单击复制的 rsync 守护程序也是过度使用,所以 tar nc 执行以下操作。



创建将通过网络提供文件的进程:

  tar cSf  -  / path / to / files | nc -l 5000 

请注意,它可能需要 tar 很长时间来检查稀疏文件,所以正常情况下看不到一段时间的进展。



接收到以下文件在另一端:

  nc hostname_or_ip 5000 | tar xSf  -  






所有花哨,使用 pv 显示进度:

  tar cSf  -  / path / to / files \ 
| pv -s`du -sb / path / to / files | awk'{print $ 1}'`\
| nc -l 5000

稍等一下,直到你看到 pv 报告一些字节已经过去,然后在另一端启动接收器:

  nc hostname_or_ip 5000 | pv -btr | tar xSf  -  


I have two twins CentOS 5.4 servers with VMware Server installed on each.

What is the most reliable and fast method for copying virtual machines files from one server to the other, assuming that I always use sparse file for my vmware virtual machines?

The vm's files are a pain to copy since they are very large (50 GB) but since they are sparse files I think something can be done to improve the speed of the copy.

If you want to copy large data quickly, rsync over SSH is not for you. As running an rsync daemon for quick one-shot copying is also overkill, yer olde tar and nc do the trick as follows.

Create the process that will serve the files over network:

tar cSf - /path/to/files | nc -l 5000

Note that it may take tar a long time to examine sparse files, so it's normal to see no progress for a while.

And receive the files with the following at the other end:

nc hostname_or_ip 5000 | tar xSf -


Alternatively, if you want to get all fancy, use pv to display progress:

tar cSf - /path/to/files \
   | pv -s `du -sb /path/to/files  | awk '{ print $1 }'` \
   | nc -l 5000

Wait a little until you see that pv reports that some bytes have passed by, then start the receiver at the other end:

nc hostname_or_ip 5000 | pv -btr | tar xSf -