torstai 17. kesäkuuta 2021

rsync from windows to linux

Recently I suddenly found out that I have remote linux server where I need to often upload new files incrementally - only few files change and I'd rather not upload entire tree every time, so SCP and to a degree (s)FTP were out. So, rsync, which I had often heard of but never used. Also, this is firewalled machine with SSH but no usual RSYNC port available.

It turns out that rsync'ing from windows to linux isn't as easy as 1-2-3, but eventually I got it working. To help others (and myself later on too), here's a very brief listing on what I did to make it work. This may or may not work for you, and there may be security issues here too, so take all this with grain or two of salt.

So in short:

1) Set up SSH for public/private key access using ssh-keygen and adding public key to .ssh/authorized_keys of wanted user (linux_user for example). Copy private key to your windows system. I am not sure whether you can use passphrase, I didn't for this test.

2) Make sure rsync is installed in your server and in path: just run rsync command and see that it runs.

2.5) (I'm not exactly sure of this) You may need to define /etc/rsyncd.conf on server and set uid/gid there to match wanted user.

3) On windows, download cygwin and install both rsync and openssh; they are not installed by default so you have to select them explicitly.

4) Add private key (id_rsa from (1) above) to \cygwin64\home\<user>\.ssh\
ssh doesn't like it being public so chmod it:  \cygwin64\bin\chmod 700 id_rsa

5) Test ssh:  \cygwin64\bin\ssh -l linux_user myserver.test . If all went well, you should log directly in (or ssh should as passphrase first).  

5) Use following in your source directory:

rsync -vrtl --chmod=o=r -e "\cygwin64\bin\ssh -v" *  linux_user@myserver.test:/path/to/target

chmod: needed if you are copying, say, html over, to set "read" access to files, in case default doesn't set it. You may leave it out first and add if needed. Similarly --chown=user:group could be used; for me it didn't work properly though (which might be related to -a parameter I used earlier)

Even if ssh is in path and there (apparently) wasn't other conflicting version, this didn't work for me until I specified this full path, in quotes like here. -v here prints out debug data of ssh, it helps massively if you need to troubleshoot issues.

* : All files in current windows directory. 'source/*' works too, but absolute paths or file names cause issues with cygwin.

And at end target user, server and path.

6) If this works, you can take "sent rsync command" (printed out during handshake of above) and add it to authorized_keys: 'command="rsync-printed-above" ssh-rsa AA....'. Test that (5) still works. This improves security a bit, as now only thing that can be run with private key is this single command. Consider also adding, say, no-port-forwarding and other similar flags too.

 

Hope this helps!