Question:
I was downloading a large file using SCP, but the download transfer failed in the middle because
my laptop got disconnected from the network. Is there a way to resume the interrupted SCP transfer where I
left off, instead of downloading the file all over again?
Answer:
Originally based on BSD RCP protocol, SCP (Secure copy) is a mechanism that allows you to transfer a file
between two end points over a secure SSH connection. However, as a simple secure copy protocol, SCP
does not understand range-request or partial transfer like HTTP does. As such, popular SCP
implementations like the scp command line tool cannot resume aborted downloads from lost network
connections.
If you want to resume an interrupted SCP transfer, you need to rely on other programs which support
range requests. One popular such program is rsync. Similar to scp, rsync can also transfer files over
SSH.
Suppose you were trying to download a file (bigdata.tgz) from a remote host remotehost.com using
scp, but the SCP transfer was stopped in the middle due to a stalled SSH connection. You can use the
following rsync command to easily resume the stopped transfer. Note that the remote server must have
rsync installed as well.
$ cd /path/to/directory/of/partially_downloaded_file
$ rsync -P -rsh=ssh This email address is being protected from spambots. You need JavaScript enabled to view it.:bigdata.tgz ./bigdata.tgz
The "-P" option is the same as "--partial --progress", allowing rsync to work with partially downloaded files.
The "-rsh=ssh" option tells rsync to use ssh as a remote shell.
Once the command is invoked, rsync processes on local and remote hosts compare a local file
(./bigdata.tgz) and a remote file (This email address is being protected from spambots. You need JavaScript enabled to view it.:bigdata.tgz), determine among themselves what
portion of the file is not the same, and transfer the discrepancy to either end. In this case, missing bytes in
the partially downloaded local file is downloaded from a remote host.