Author: Victor Churchill Date: To: Hampshire LUG Discussion List Subject: [Hampshire] unwanted buffering of ssh output when piping into awk
I want to deconstruct the log file lines and reformat them for legibility.
So I pipe the stream coming from ssh into awk on my desktop machine.
When I try
ssh qq2 "tail -f /var/log/httpd/custom_log | awk '{printf (\"%9d %15s
%7s %s %60s\n\", \$NF, \$1, \$3, \$4, \$7)}' "
I see nothing at first, and then a burst of lines, then nothing, etc.
Replacing the 'tail' command with a while-loop saying 'date' every
second, I get a batch of around 140 lines at a time which is
remarkably close to 4K bytes.
I can run the tail output into an awk process on the server
ssh qq2 "tail blah | awk 'blah' " (with appropriate quote-escapes) and
that works but I'd rather do the post-processing on my machine.
After checking ssh and awk manpages the only clue I found was to add
fflush() to the wak code, but with no observable difference.
ssh server "command" gives every impression of giving immediate
line-by-line output without being buffered by ssh or bash. And awk
reads from stdin with a newline record separator. So who is doing the
4KB buffering (if that is what is happening)?
