watch(1), part of procps, is one of my favorite command line utilities. It repeatedly executes a command so you can watch the output change, and if you pass the optional
--diff/-d command line argument it highlights the changes between executions (or across every execution with
--difference=cumulative). It’s great for one-off monitoring uses, like running
/sbin/ifconfig to see how frequently your network interfaces are experiencing errors or dropping packets, or running
/bin/ls -l to see how quickly some log files are growing, etc.
watch runs until either the command you are watching errors out or you interrupt it, but a colleague suggested an option to exit when the output changes. This is useful for things like Makefiles and shell scripts where you want to gate an automated process on an external event or resource. For example, when integration testing a web application you want to wait for the app to fully start before running any tests, or when upgrading network card drivers you want to wait for the kernel to reinitialize interfaces before restarting your applications.
I wrote a patch for
watch that adds an optional
--chg-exit/-g command line argument for this behavior, and I’m happy to say it was merged upstream and is available as of procps v3.3.3. The patch makes possible things like
watch -g "curl http://localhost:3000/" && ./run_integration_tests.sh to wait for your webapp to start before running any tests, or
watch -g "nc -w 1 <ip address> 22" && ssh <ip address> restart_apps.sh to wait until OpenSSH is ready before restarting your applications on a remote server.
There are other tools that provide the same functionality, but the advantage of using
watch -g is being able to watch what’s going on when running in a terminal. I don’t use
watch -g in automated processes that run headless on remote machines such as build servers or cronjobs, but it’s my preferred tool for interactive use. Fortunately, it’s easy to tell when a process is running interactively (see: Perl’s -t file test operator, Python’s file#isatty, Ruby’s IO#tty?, /usr/bin/tty, etc.), so depending on context scripts can automatically decide whether to wait via
watch -g or something else.