It has only been recently that something clicked in my mind and I finally understood why there is more and more talk around the aspect of Developer Experience. Just as the average consumer of a digital product measures its quality by how well the so-called user experience accompanies him, the developer, who is on the other side of the life cycle of this product, i.e. who is involved in its implementation and provision, is just as strongly characterised by a so-called developer experience. Or, to put it another way, the developer as such has also become a rare resource that potential employers are trying to to attract with an appealing developer experiences, among other things of course. This applies not only to the actual technology stack, but also to the way it is used.
As a developer of all kinds of digital offerings, I myself am very interested in this topic of the developer experience. For this reason, in today’s TechUp I would like to explore a sub-area of this very topic and show you another aspect with which one can potentially ensure a better developer experience.
Development Integration
The developer experience is primarily defined by one’s own workflow. In other words, how many individual steps are needed until a new iteration of the completion of tasks is created. An iteration can be, for example, the first implementation attempt of a given feature, which is compiled in the second step and booted in a runtime for testing purposes in the third step. The individual steps are often numerous and highly interdependent. Before a new block of code can be tested on the local machine, the code must be compilable. The compilation must find its way into a runtime, and this can only happen when the code compiles. Thus, the order of the workflow is strictly predetermined. Each iteration costs time and nerves. So you want to ensure efficient processes and keep the individual steps as simple as possible. And that leads me to the topic of developer integration.
What I just mentioned was a composition of steps that make up the entire developer workflow. The individual steps are often command line commands. Such commands can easily be integrated together, for example as bash scripts, which automate repetitive steps. Apart from classic shell scripts, there are also dedicated tools that can lead to a simpler iteration step in one’s own workflow.
Task Runners like GNU-Make
These dedicated tools are called task runners. One variant of such a task runner is the well-known GNU Make, which is already several decades old. Often the compilation process is not obvious and the individual steps in a given workflow are intransparent. Additionally, several programming language ecosystems are nowadays established in a business domain (i.e. in a team) and require specific processes. Here I am thinking, for example, of the Node.JS-based npm, which is used to compile frontend projects, but at the same time there might be a backend project in the form of a Quarkus codebase, which in turn requires a database as a third ecosystem. Thus, different, sometimes divergent processes are required to master the individual steps of a given iteration. GNU Make takes care of exactly these implementation details and enables us to abstract the entire project stack. It becomes a black box that can be managed even if the individual details of the stack are not known. Make allows for a standardisation of the workflow, so to speak: It abstracts the individual details of each tech stack and provides a higher level of abstraction of the development environment enabling new developers to be onboarded within minutes.
Whether Docker, Minikube, Vagrant or a simple Maven build, Make allows you to save headspace and provides simple primitives that help you get started straight away. In addition, it is self-explanatory due to the Makefile as a declarative configuration file of the entire workflow. Of course, it’s helpful to have a README.md, which abstracts the Makefile into plain text and makes it accessible to non-techies. In the end, it’s simply a matter of being able to concentrate on the real thing: Coding, Automating, Creating abstractions. Thinking about development integration using task runners is the subject of today’s TechUp. For this reason, we will take a closer look at Make and show what these implementation details look like in the real-world application.
Make, what was that again
First of all, the idea of automated build management is old. make has been around since 1976 and has even been documented as a POSIX standard, the so-called IEEE Std 1003.1. At this point I would like to refer to an anecdote about `make’, as it exemplifies the usefulness of such a solution, which was new at the time:
“Make originated with a visit from Steve Johnson (author of yacc, etc.), storming into my office, cursing the Fates that had caused him to waste a morning debugging a correct program (bug had been fixed, file hadn’t been compiled,
cc *.owas therefore unaffected). As I had spent a part of the previous evening coping with the same disaster on a project I was working on, the idea of a tool to solve it came up. It began with an elaborate idea of a dependency analyzer, boiled down to something much simpler, and turned into Make that weekend. Use of tools that were still wet was part of the culture. Makefiles were text files, not magically encoded binaries, because that was the Unix ethos: printable, debuggable, understandable stuff.”—Stuart Feldman, The Art of Unix Programming, Eric S. Raymond 2003
So, according to Stuart Feldman, an executable was debugged, which in fact contained the actual compiled file of interest, and thus the whole morning was needlessly wasted. This is exactly the kind of mistake that happens when a developer does not have the headspace and has to focus on apparently unnecessary sidetasks before he can continue with the actual task. This is how make was born.
If you are not familiar with make, or if it’s not part of your workflow, make is a cli command that reads a so-called makefile and executes it. Makefiles are still used very often today, especially in the open source world. Particularly when it comes to compiling C or C++ based source code bases on the target system. This happens, for example, when using Gentoo Linux or the well-known AUR from Arch Linux. If this doesn’t mean anything to you, I think it’s time to demonstrate `make’ with an example.
Makefile 101
Let’s have a look at a Makefile. What we can see below is a conventional Makefile as we can find it for example in our internal b-nova/solr-page-exposer project. (If you want to know more about our internal JAMstack architecture, I highly recommend this TechUp by Valentin: Our own implementation of a jamstack-enabled headless CMS architecture.)
|
|
The Makefile is at the lowest project level, which allows us to type make directly from the project directory, which in turn leads to precisely this file being found and the first task being executed. In this case that would be the all task. The all task itself is a list of other tasks that are executed in exactly this order.
Thus, a simple $ make command in the project directory will cause the tidy, the build and the run stages to be executed in exactly that order. Although you can define this however you like, it is common practice that the first task should always lead to the project being compiled, i.e. built, ideally platform/environment-independent.
From Zero to Hero
If we wanted to look at a more complex example of a Makefile, we can find one in almost every C/C++ project in the open source space. My favourite editor is Neovim, so let’s take a look at Neovim’s Makefile. This is the one I found in the Github repository of Neovim:
|
|
You can definitely do more, but this elegantly abstracts all the compilation complexity of the C based project. Of course, the source language doesn’t have to be as low-level as C or even Go, but you can also simplify and abstract build systems like JVM’s Maven or Gradle.
From Make to Task
So, now we know what make and Makefiles are. Of course, the whole thing seems a bit old-fashioned, or at least is often associated with ancient developer technologies and not as a build tool of the future from the year 2030. This is exactly where Task comes into play.
Task, just like Make, is a build automation tool, or more specifically a task runner. In a nutshell, Task can be characterised with the following properties:
- It is written in Golang.
- It thus has only one binary without dependencies or shared libraries.
- Uses YAML as markup language for its task definitions
- Its Makefiles are now simply called
Taskfile. - Has a convenience script for its rotationless installation.
Taskfiles ad infinidum
Let’s look at some Task files. The official Github repository has a testdata/ folder containing a variety of different test cases that have pretty much all the features of Task (and of course serve as the basis for the unit tests).
I picked out a few examples and also freely composed some myself to have more complex real-world examples.
I. Summary Taskfile
|
|
II. Another Taskfile
|
|
$ task --help
If you’re not sure, you can always just use task --help in your shell. Also, this page should give you some helpful hints on using Task: https://taskfile.dev/usage/
Conclusion on Task
Let’s start from the beginning. We have seen that a task automation tool like Make or Task is essential to one’s development process because it increases the repeatability and traceability of processes. It allows a series of steps that would otherwise have to be performed manually to be executed in a single command. It also allows processes to be triggered automatically when certain conditions are met. It facilitates work by specifying the necessary steps and executing them automatically. It can also be used to manage dependencies between different tasks and ensure that they are executed in the correct order.
Make vs. Task
Compared to Make, Task is a more modern and flexible task automation tool that has some advantages over Make:
- Simplicity: Task has a simpler syntax and requires less configuration than Make.
- Platform Independence: Task is written in Go and runs on Windows as well as Linux and macOS, whereas Make was developed mainly for Unix systems.
- Integrability: Task can be easily integrated into modern JavaScript workflows, such as Webpack or Rollup, while Make was developed mainly for the command line.
- Extensibility: Task has a well-documented API that allows you to create custom plugins that extend its functionality.
- Collaboration: Task supports sharing and collaboration on tasks through the use of plugins and supports workflow automation.
However, there are also situations where Make may be more appropriate, for example when a project is already built on Make or when it specialises on a large number of dependencies and complex processes.
A task automation tool like Make or Task is of great importance to the developer experience as it increases repeatability, traceability and efficiency of processes. It enables developers to execute complex processes in simple steps, saving time and resources. It also facilitates collaboration and automation of workflows within a team. Although Make and Task have different benefits, both offer a way to simplify and automate processes, improving the overall developer experience. So it’s worth looking into the possibilities of task automation tools and choosing the right one for your needs.
Outlook for further improvements to the developer experience
At b-nova, we have already investigated several times in different TechUp articles how the developer experience could be improved. For example, we looked at how to automate version management with asdf, or how to use technical cheatsheets like tldr or cht.sh for a better workflow. If you like the topic of developer experience and workflow design, check out the following TechUps and learn how you can take your own developer workflow to the next level:
- Simplify your workflow with a version manager like asdf
- Remember less but know more with state-of-the-art cheatsheets
There are other aspects around Developer Experience that I would like to look at this year. This definitely includes how to deal with the composition of one’s own dev workflow, and what possibilities have emerged in recent years to disentangle and/or even rethink precisely this process. Another topic that is still on hold for me is functional package management with Nix. As you can see, there is still a lot to tell about this topic. So, until the next TechUp, stay tuned! 🚀
Weiterführende Links und Ressourcen
https://developerexperience.io/practices/good-developer-experience
https://www.gnu.org/software/make/
https://github.com/go-task/task