Jenkins on OS X: connection refused

Late last week the instance of Jenkins we are running at the office on a Mac Mini started to reject incoming connections from all other devices on our LAN. http://localhost:8080 still resolved fine and allowed to open up the Jenkins web interface while all requests to http://jenkins.local:8080 (or http://[ip]:8080 for that matter) would give a Connection Refused error. In the browser the web interface just would not load without giving a proper error message. Only a request through wget unearthed the Connection Refused error being the root cause.

This happened out of the blue and was neither triggered by an update of Jenkins itself nor the OS. Neither was anything changed in the network infrastructure. Weirdly enough, one we ran into the error on our dedicated build server we run trails on all our other machines: it happened on all machines running OS X with Jenkins 1.5xx revisions installed (I did not test earlier revisions) even with Jenkins installed freshly on a vanilla system.

After days of investigating all possible software (restored a time machine backup from back when the system was still working) and hardware (i.e. network infrastructure related) causes we just gave up and moved onto a work around.

When you install jenkins on OS X through Homebrew it installs a LaunchAgent and asks you to use it to launch Jenkins with launchctl – OS X's builtin way to run and start programs and scripts during various phases of the system boot and login sequence.
We have used this way to start Jenkins on our dedicated Mac Mini for a little over a year now without issue. Until last week that was when the ominous Connection Refused error started.

Research on the web finally turned up this lead. Pretty desperate at the time I went with it, despite the fact that the OS and way to launch Jenkins in this poor guy's case were completely unrelated. At least the error he was getting was the same as ours.

And, in fact, it turned out that Jenkins would respond fine to requests coming from LAN-devices when launch directly using the Java interpreter. After a couple of more hours trying to figure out why this was and going down the rabbit whole of network debugging and tracing tools and the weirdness of the OS X Firewall service I finally decided to hack together a workaround.

Using nohup this line would turn out to run Jenkins reliably without any Out of Memory exceptions I experienced on the first couple of tries:

nohup java -XX:PermSize=512M -XX:MaxPermSize=2048M -Xmn128M -Xms1024M -Xmx2048M -jar /usr/local/opt/jenkins/libexec/jenkins.war

I put together a wrapper script to manage jenkins as kind of a service:

This would allow me to start and stop Jenkins keeping track of the process and the output it generates.

Last step was to make OS X run jenkins.sh start when booting up. This brought me round to launchctl again – and, sure enough, things fell apart again at that point. Run through a LaunchAgent using my own wrapper the same Connection Refused error blocked outside connection attempts to Jenkin's web interface.

I finally put the call to the script to start Jenkins into an Automator-generated application which I installed and set up as a Launch Item in the User's OS X System Preferences.

This finally did the trick and our Jenkins-powered CI and deployment solution was up and running again. Half the time of my work week well wasted – and I still have no inkling what the root cause might be.

I am a Thalmic Myo alpha developer...

I finally got my Thalmic Myo alpha unit after it was stuck at German customs for about a week.

Thalmic Myo Alpha Device

Here is my resume after the first couple of days of using it and developing for it:

  • For an alpha device it already has an incredibly high quality finish. The mechanics work well and you can easily put in on and pull it off again. It sits snugly on your arm without being too uncomfortable even after hours of wearing it (longest I wore it was 3 hours so far). Battery life is good enough to sustain usage over that time. The only thing that is a bit bothersome is turning it on and off (done via pinhole button).
  • It's an alpha device in the sense that there's no fancy demo to run once you have unboxed the device. You need to go in and compile something that does more than calibrating the device or outputting text.
  • The software works well, but has obvious limitations: the device calibration is not carried over between sessions; calibration only supports five gestures at this time and custom gestures are yet to come.
  • Alpha community: I pulled a simple music player app which was available as OSS for iOS on the Thalmic github account. I made a couple of obvious tweaks and have been running it ever since on my iPhone. The Myo is paired with my iPhone. I can use it to play/pause and switch tracks. It's a very simple use case, but that's all there is right now.
  • The product, i.e. hardware and software together, works impressively well. It's easy and intuitive and just fun to use. I was previously dissappointed by the LEAP so I was a bit sceptical, but for me the Myo is the absolute proof that this sort of gesture control will inevitably become part of HMI over the next couple of years. I am massively looking forward to the beta and production-level devices.

Next up for me is building an iPhone remote control powered by Myo for my Rasperry-Pi-XBMC. And, obviously, annoying my girlfriend by using the Myo to control our TV.

Baking an MacBook Pro...

My sister needed a new laptop and I had an old 2008 MacBook Pro lying around. After putting in an SSD it would not boot any more.

So, after a bit of googling around I decided to perform the most riddiculous repair manouver I have ever seen: I put the logic board of the MacBook in the oven and baked it for about 8 minutes. I followed this guide. It worked perfectly.

Some things I would recommend to watch out for:

  • I was careful to remove all plastic parts
  • I clean old thermal paste and add new, high quality thermal paste (makes the fix last longer)
  • Take a lot of pictures to know what the vanilla state of the wires and foam parts was
  • we careful with the metal plugs (display and airport) you can easily pull the metal casing off and it's a mess to get back on

Here are some pictures.

ghost.org

I migrated my blog to ghost.org.

  • great backend
  • fair managed offer (5 USD / month)
  • easy setup
  • a lot of high quality themes available
  • blazingly fast

Very happy about the move - would recommend it any time.

#live #music

Saw the amazing @cityandcolour perform in Munich today.