But Why?
Of all the non-editing facilites that Emacs provides (of which there are legion), perhaps the most useful is TRAMP, a wonderfully Rube Goldberg-ian Emacs mode that allows you edit, save, and copy files remotely over all kinds of transports, including some that were never really intended to be file transports. This is of course all in service of never forcing us Emacs users to touch
vi on a remote server: instead, we just open our file in our local Emacs using TRAMP.
An even more interesting featur of TRAMP is that you can daisy-chain these connections onto each other. For instance, if you want to edit a file on
hostb, but it is locked down and only accessible via ssh from hosta, you can open the file /ssh:hosta|ssh:hostb:/path/to/file. This sets into motion a whole series of events that might or might not involve base64-encoding the file in question and piping it to temporary files along the way. Adventurous souls can even use this facility to transfer whole directory trees between hosts, though of course this is all very slow and hackish, and therefore to be avoided where possible, especially if both hosts are remote, since you're essentially downloading each file and re-uploading it.
It also only works from inside Emacs. Or does it?
Recently, I was writing a little shellscript and had need to copy some files from my local machine to another machine (let's call it "another_box") that wasn't directly accessible, but which was accessible via another host (let's call it "jump_box"), and which finally had to end up as belonging to a non-root user ("different_user"). I was about to write something like this:
rsync -aP ~/music jump_box:~/music_from_cam
ssh jump_box "rsync -aP ~/music_from_cam another_box:/home/different_user/"
ssh jump_box "ssh another_box 'chown -R different_user. /home/different_user/music_from_cam'"
ssh jump_box "rm -rf ~/music_from_cam"
...but the nested
ssh invocations were making my head hurt mildly, and the whole thing seemed like an inelegant hack. If I were doing this interactively, I'd just use TRAMP in Emacs. Sure, it'd take longer than rsync, but it would be simpler to do. But wait, can I use TRAMP anyway? Sure, just call emacs in batch mode and invoke copy-directory. TRAMP has been bundled with Emacs since 22.1 (released June 2007) so it should be available by default in any remotely modern Emacs package. Let's see if this works:
~ $ emacs -q --batch --eval '(copy-directory "~/music" "/ssh:jump_box|ssh:another_box|sudo:different_user:~/music_from_cam")'
Loading 00debian-vars...
Loading /etc/emacs/site-start.d/50autoconf.el (source)...
Loading /etc/emacs/site-start.d/50cmake-data.el (source)...
Loading /etc/emacs/site-start.d/50dictionaries-common.el (source)...
Loading debian-ispell...
Loading /var/cache/dictionaries-common/emacsen-ispell-default.el (source)...
Loading /var/cache/dictionaries-common/emacsen-ispell-dicts.el (source)...
Loading /etc/emacs/site-start.d/50notmuch.el (source)...
Package notmuch removed but not purged. Skipping setup.
Tramp: Opening connection for root@different_user using sudo...
Opening connection for root@different_user using sudo... \
Tramp: Sending command `exec ssh -o ControlMaster
auto -o ControlPath'tramp.%C' -o ControlPersist=no -e none jump_box'
Tramp: Waiting for prompts from remote shell...
Waiting for prompts from remote shell... \
Tramp: Waiting for prompts from remote shell...done
Tramp: Found remote shell prompt on `jump_box'
Tramp: Sending command `exec ssh -e none another_box'
Tramp: Waiting for prompts from remote shell...
Waiting for prompts from remote shell... \
Tramp: Waiting for prompts from remote shell...done
Tramp: Found remote shell prompt on `another_box'
Tramp: Sending command `exec env SHELL=/bin/sh sudo -u root -s -H -p Password:'
Tramp: Waiting for prompts from remote shell...
Waiting for prompts from remote shell... \
Tramp: Waiting for prompts from remote shell...done
Tramp: Found remote shell prompt on `different_user'
Tramp: Opening connection for root@different_user using sudo...done
Copying /home/cam/music/another_cool_track.mp3 to /sudo:root@different_user:/root/music_from_cam/another_cool_track.mp3...
Copying /home/cam/music/another_cool_track.mp3 to /sudo:root@different_user:/root/music_from_cam/another_cool_track.mp3... \
Tramp: Encoding local file `/tmp/tramp.4175nPk.mp3' using `(lambda (beg end) (let ((coding-system-for-write (quote binary)) (coding-system-for-read (quote binary))) (apply (quote call-process-region) beg end (car (split-string gzip)) t t nil (cdr (split-string gzip)))) (base64-encode-region (point-min) (point-max)))'...
Encoding local file `/tmp/tramp.4175nPk.mp3' using `(lambda (beg end) (let ((coding-system-for-write (quote binary)) (coding-system-for-read (quote binary))) (apply (quote call-process-region) beg end (car (split-string gzip)) t t nil (cdr (split-string gzip)))) (base64-encode-region (point-min) (point-max)))' \
Tramp: Encoding local file `/tmp/tramp.4175nPk.mp3' using `(lambda (beg end) (let ((coding-system-for-write (quote binary)) (coding-system-for-read (quote binary))) (apply (quote call-process-region) beg end (car (split-string gzip)) t t nil (cdr (split-string gzip)))) (base64-encode-region (point-min) (point-max)))'...done
Tramp: Decoding remote file `/sudo:root@different_user:/root/music_from_cam/another_cool_track.mp3' using `(base64 -d -i | gzip -d >%s)'...
Decoding remote file `/sudo:root@different_user:/root/music_from_cam/another_cool_track.mp3' using `(base64 -d -i | gzip -d >%s)' \
Tramp: Decoding remote file `/sudo:root@different_user:/root/music_from_cam/another_cool_track.mp3' using `(base64 -d -i | gzip -d >%s)'...done
Copying /home/cam/music/another_cool_track.mp3 to /sudo:root@different_user:/root/music_from_cam/another_cool_track.mp3...done
Copying /home/cam/music/cool_track.mp3 to /sudo:root@different_user:/root/music_from_cam/cool_track.mp3...
Copying /home/cam/music/cool_track.mp3 to /sudo:root@different_user:/root/music_from_cam/cool_track.mp3... \
Tramp: Encoding local file `/tmp/tramp.41750Zq.mp3' using `(lambda (beg end) (let ((coding-system-for-write (quote binary)) (coding-system-for-read (quote binary))) (apply (quote call-process-region) beg end (car (split-string gzip)) t t nil (cdr (split-string gzip)))) (base64-encode-region (point-min) (point-max)))'...
Encoding local file `/tmp/tramp.41750Zq.mp3' using `(lambda (beg end) (let ((coding-system-for-write (quote binary)) (coding-system-for-read (quote binary))) (apply (quote call-process-region) beg end (car (split-string gzip)) t t nil (cdr (split-string gzip)))) (base64-encode-region (point-min) (point-max)))' \
Tramp: Encoding local file `/tmp/tramp.41750Zq.mp3' using `(lambda (beg end) (let ((coding-system-for-write (quote binary)) (coding-system-for-read (quote binary))) (apply (quote call-process-region) beg end (car (split-string gzip)) t t nil (cdr (split-string gzip)))) (base64-encode-region (point-min) (point-max)))'...done
Tramp: Decoding remote file `/sudo:root@different_user:/root/music_from_cam/cool_track.mp3' using `(base64 -d -i | gzip -d >%s)'...
Decoding remote file `/sudo:root@different_user:/root/music_from_cam/cool_track.mp3' using `(base64 -d -i | gzip -d >%s)' \
Tramp: Decoding remote file `/sudo:root@different_user:/root/music_from_cam/cool_track.mp3' using `(base64 -d -i | gzip -d >%s)'...done
Copying /home/cam/music/cool_track.mp3 to /sudo:root@different_user:/root/music_from_cam/cool_track.mp3...done
Copying /home/cam/music/even_cooler_track.mp3 to /sudo:root@different_user:/root/music_from_cam/even_cooler_track.mp3...
Copying /home/cam/music/even_cooler_track.mp3 to /sudo:root@different_user:/root/music_from_cam/even_cooler_track.mp3... \
Tramp: Encoding local file `/tmp/tramp.4175Bkw.mp3' using `(lambda (beg end) (let ((coding-system-for-write (quote binary)) (coding-system-for-read (quote binary))) (apply (quote call-process-region) beg end (car (split-string gzip)) t t nil (cdr (split-string gzip)))) (base64-encode-region (point-min) (point-max)))'...
Encoding local file `/tmp/tramp.4175Bkw.mp3' using `(lambda (beg end) (let ((coding-system-for-write (quote binary)) (coding-system-for-read (quote binary))) (apply (quote call-process-region) beg end (car (split-string gzip)) t t nil (cdr (split-string gzip)))) (base64-encode-region (point-min) (point-max)))' \
Tramp: Encoding local file `/tmp/tramp.4175Bkw.mp3' using `(lambda (beg end) (let ((coding-system-for-write (quote binary)) (coding-system-for-read (quote binary))) (apply (quote call-process-region) beg end (car (split-string gzip)) t t nil (cdr (split-string gzip)))) (base64-encode-region (point-min) (point-max)))'...done
Tramp: Decoding remote file `/sudo:root@different_user:/root/music_from_cam/even_cooler_track.mp3' using `(base64 -d -i | gzip -d >%s)'...
Decoding remote file `/sudo:root@different_user:/root/music_from_cam/even_cooler_track.mp3' using `(base64 -d -i | gzip -d >%s)' \
Tramp: Decoding remote file `/sudo:root@different_user:/root/music_from_cam/even_cooler_track.mp3' using `(base64 -d -i | gzip -d >%s)'...done
Copying /home/cam/music/even_cooler_track.mp3 to /sudo:root@different_user:/root/music_from_cam/even_cooler_track.mp3...done
~ $
Beautiful! Now instead of an inelegant hack, I have an elegant one and, even better, a hack that people lots smarter than me have spent a lot of time making work in all kinds of weird edge cases. You'll notice from the output that TRAMP automatically opens all the necessary ssh connections (3 of them) and does all the necessary encoding, decoding, and temporary file marshalling for us.
To dissect that
emacs invocation a bit, note that -q tells Emacs not to load any custom init files, --batch tells it not to load the UI, but simply to execute whatever code its told to on the command line, and --eval evaluates our elisp code.
copy-directory is elisp's equivalent to the regular Unix cp command, except that it will let let you use TRAMP filenames. Here I tunnel the files through jump_box, an alias set up in the ~/.ssh/config on my local machine, and another_box, an alias set up in the ~/.ssh/config on the jump box. Finally, I use the sudo TRAMP transport to copy the files as the different_user user, avoiding the chown call I would have needed with the original nested-ssh solution.
Conclusion
Is this a good solution for production-grade file copying? I mean, I haven't done any kind of battle-testing, but almost certainly not. It is, however, a reasonable solution when cranking out a quick shellscript for something that isn't critical and you don't mind adding Emacs as a pre-req for the script. Again, it's not as efficient under the hood as the
rsync example above, but the UX of just writing (copy-directory SRC DEST) can't be easily beat.