Command line interfaces (CLIs) have always played an important role in software development, as they are at the heart of any Unix-like system. Nevertheless, command line tools did not always have a good reputation.
In recent years, however, they have experienced an expansion, not least because of the increased demands with regard to automation and the constantly rising use of Linux on the server side. Therefore, this seems to us to be an excellent moment to tell about our experiences and findings during the development of the wolkenkit CLI.
Internally wolkenkit works a lot with Docker containers. The CLI serves as a kind of remote control that simplifies the handling of these containers. Having to write this remote control took us on an interesting and exciting journey from Node.js to Go and back again. We also learned a lot about how to integrate services via HTTP and CLIs respectively.
Back then, we decided to use Docker's HTTP API because we thought it was the most reliable and stable way to interact with Docker. We hoped that, ideally, we could use an npm module that abstracts away these API calls for us, so that we could work with a nice Node.js API.
Unfortunately, it turned out that we were wrong: The HTTP API changed regularly, which is not surprising in retrospect. Docker was young then and still had to establish itself. Furthermore, the HTTP API was not designed for end users. Therefore, with every Docker update we had to adapt our code to keep it compatible with the new Docker version.
At that time we had to think about how we wanted to proceed. How could we let things mature while remaining agile and flexible?
The first important aspect for us was that we wanted to simplify the installation process of wolkenkit. At that time you had to install a lot to get things up and running, i.e. Docker, docker-machine, Node.js and the CLI.
As our installation instructions page grew, Go became more and more our focus. The ability to create statically linked binary files without any dependencies for multiple platforms at once seemed terrific. This seemed to be a much easier approach. Also, the proximity of Go to Docker seemed much better than that of Node.js to Docker: Since Docker itself is written in Go, we thought it would be much easier to address Docker with Go.
So we had to figure out how to interact with Docker from within Go. But unfortunately we hadn't learned our lesson yet: we assumed that the native Docker APIs for Go should allow a much better integration, but here too we should be enormously mistaken. Since we like learning new languages, it also seemed like a good way to learn more about Go and its approach to writing asynchronous code.
At the beginning we felt encouraged by the new stack: Go is a great language and the Docker API worked very well. But then the problems gradually began. To be clear here, the main problems we encountered were not related to Go as a programming language. Essentially, they were caused by our design decisions.
Docker also decided to split its API into multiple smaller modules and make them available individually. This of course entailed a lot of changes to our code, which we only managed to cope with to a limited extent. We had great difficulties in managing dependencies, especially as we were very spoiled by npm in this respect (which, by the way, does an amazing job at managing dependencies). In short, we were not prepared for the differences of the ecosystem of Go, in which various approaches to handle dependencies competed at the time.
So as we approached the first release of wolkenkit, work on our CLI slowed down. And again we were forced to make a decision: Should we rewrite the CLI from scratch once more?
…and back again
We weren't really happy, but we took the time to discuss different ideas and try things out. These ideas ultimately gave rise to a very simple idea: Isn't Docker's CLI the most stable API Docker provides? And couldn't we build on that by writing a CLI in Node.js that outsources the work to Docker's CLI using child processes? How would this affect our user experience?
So we decided to develop a spike and a quick proof of concept. We just tried to implement a single command to see how it felt. To our surprise, this has worked excellently in several respects.
In addition, there were some other aspects that encouraged us to write the CLI once again in Node.js. On the one hand, Node.js had grown as a platform for writing CLIs. Thanks to modules such as command-line-args, setting up a modular and lightweight basic frame for a CLI has become very easy. On the other hand, we have also developed buntstift, a module to create consistent UIs for our CLIs. Last but not least, the installation of Docker on macOS and Windows has been dramatically improved.
However, this insight is also a result of our excursion to Go. We have learned a lot from this fast-growing ecosystem and will continue to explore it and others, because it never hurts to look beyond one's own nose. Nevertheless, it is also good to know where one's own roots are. The most important insight, however, is that developing software is ultimately not a question of one or the other technology or language, but of a system and its limits.
So, to cut a long story short: Have fun with the wolkenkit CLI!