Make or Break

Run a flaky command in a tight loop, until it fails

I’ve been diving back into Docker and Bitbucket pipelines lately, using a number of tools to build and test this site. It’s been great to see how far I can go!

One of the challenges I faced was a tool that was failing intermittently. Every few builds this tool would fail, and it’s still not clear exactly why.

As I worked to troubleshoot the problem I happened to type something similar to the following:

while true; do <make> || break; done

where ‘make’ was the intermittently-failing command. Hence, ‘make or break.’

This little bit of Bash allowed me to run the command in an infinite loop. As soon as the command failed, the loop would end. I used this to test changes to the failing command, and it’s been very helpful.

Because this seemed useful, I turned it into a cool tool, mob:

USAGE: /home/dmurray/bin/mob <command>
  Make or Break (mob) runs the command in a tight loop,
  until it exits non-zero.

  Press CTRL+c to break the loop.

As I continue to troubleshoot this failing command I’m now doing the following:

> foo
Running foo....
Foo succeeded!
> mob !!

This runs mob with the previous command (!!), until it fails.

Because I deploy this site via a CI/CD pipeline, I have high standards for the tooling I use. If it fails intermittently on Bitbucket, I’m unable to deploy. Not cool.

At the moment, as long as this tool works correctly, my lead time from pushing to bitbucket and the pipeline deploying to hacken.ca is under 90 seconds.