Realtime scp output with ruby
I have had an annoying problem recently with a shell script where it would not show the output of an scp command in realtime. Initially I was fighting with different methods of capturing output in realtime from a ruby script.
The solution to this which I went for is using IO.popen
def stdout_redirect(command) f = IO.popen(command) while line = f.gets puts line end f.close end
However, this was only part of the solution. It wasnt working so I even tried implementing a solution in python.
import subprocess def myrun(cmd): """from http://blog.kagesenshi.org/2008/02/teeing-python-subprocesspopen-output.html """ print "Running " + cmd p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) stdout =  while True: line = p.stdout.readline() stdout.append(line) print line, if line == '' and p.poll() != None: break return ''.join(stdout) myrun('scp -r "remote_server:/directory/test_copy" ~/test_files/')
To my surprise, all was quiet so had a check in the terminal and it would output fine. Hmmm.
file1.txt 100% 0 0.0KB/s 00:00 file2.txt 100% 0 0.0KB/s 00:00 file3.txt 100% 0 0.0KB/s 00:00 file4.txt 100% 0 0.0KB/s 00:00 file5.txt 100% 0 0.0KB/s 00:00 file6.txt 100% 0 0.0KB/s 00:00
After some time with mr google, I managed to track down the issue with scp itself. It runs the isatty() function to check and see if the command is being run in a shell. If it is not, then all is quiet! This means ruby, python et al do not get scp output.
The solution thanks to stack exchange was to send all output to a shell!
scp -r "remote_server:/directory/test_copy" ~/test_files/ > /dev/tty
Hope this helps you if you are having a similar issue!