Using a task runner to help with context switching in software projects

Recently, I ended up doing some work that involved touching a bunch of projects of various ages and sizes, using different languages and frameworks, all on a new computer. I was feeling a lot of pain remembering how to set up and run each one, and everything was taking longer and feeling more difficult than it should.

Coincidentally, around the same time I was reminded on Mastodon of the importance of scripting the commands your project needs, particularly when it’s open-source. So I took a bit of time to organise these scripts for my projects, using the just task runner.

A few things were tripping me up pretty regularly when getting projects up and running from a fresh clone. Most importantly was remembering what dependencies a project had, and how to get them installed. This was easy enough to figure out for simple static sites with only package manager dependencies. But many projects also had operating system dependencies, extra things to run like Docker containers, or extra steps like remembering to install the browsers for Playwright testing. Sometimes past me had written these things in the README, but often I’d forgotten.

Sometimes I had even remembered to put useful commands in my README, package.json, noxfile.py or some random bash script. But across projects, these scripts all did similar things with slightly different names. Did I call the script to run the formatter fmt or format or black, or did I just forget to set one up for this project? And which package manager do I use to run them — looking at you npm and pnpm.

Finding common tasks

There’s a few tasks that I need to do for pretty much every project, so I wanted to aim at a consistent way to run these tasks. This includes things like install all the dependencies, run the thing in development mode, build it for production, run tests, run a formatter.

A key step was normalising the commands to run in order to achieve these things. GitHub’s scripts to rule them all also takes this approach, using scripts with the same name to do common development tasks, regardless of what command or language the scripts are calling under the hood. Whatever project I’m working on, I’d like to be able to run the same command to do local development on it, whatever that means for that particular project.

The tasks I usually need to do, and the names I settled on for these tasks (for now!) are:

• dev - run the thing locally in development mode. This usually involves starting a dev server, and bringing up whatever services that dev server depends on (database, mail server etc.)
• setup - everything you need to do to get going with this project from a fresh checkout. This might include things like creating config files, running database migrations and seed scripts, and of course, installing dependencies.
• install - just install the dependencies from the project’s package manager (pnpm, poetry, cargo or whatever). Expect to run this fairly regularly, maybe even on every pull from source control.
• update-deps - update the package manager dependencies
• build - build the thing for production if it is a service, or build the thing you are going to publish if it is a package.
• format (and fmt) - run the formatter
• lint - run the linter, and maybe also a typechecker or other related static analysis tasks..
• test - run all the tests

Here’s these tasks in code (a justfile) as applied to the repository backing this site. The site is one of the simplest projects I have, but is an illustration of how I’ve set things up to be consistent across all projects, regardless of scale and complexity.

# List all the things you can do with this justfile, with descriptions
help:
  @just --list

# Run the project for local development
dev:
    pnpm dev

alias setup := install
# Install all project dependencies
install:
    pnpm i

# Update all project dependencies
update-deps:
    pnpm update

# Build the project for production deployment
build:
    pnpm build

alias fmt := format
# Run the code formatter
format:
    pnpm format

# Run static analysis on the code
lint:
    @echo "⚠️  You haven't configured a linter for this repo! You probably should"
    @exit 1

# Run all the tests
test:
    @echo "ℹ️  You decided this repo probably doesn't need tests"

Most of the work that just does for me here is remembering which package manager I used, since all the scripts it calls are defined in my package.json and I could just run them manually. I still find the consistent commands helpful though!

As I haven’t got around to setting up a linter yet, when I try to run just lint out of habit, I’ll get a reminder to do that. The justfile also records that I decided I don’t need automated tests for this site, since almost all of the functionality is just content.

Choosing a task runner

Because I mostly work on projects in very small teams or alone, and I’m mostly responsible for tooling choices, I get to pick any tool I want for this job. I’ve no particular attachment to any existing tool, and as @[email protected] says

Make, cmake, doesn’t matter beyond “have all your significant interactions with the codebase be one-line, trivially retyped commands”

I settled on just as the task runner, because I occasionally need things to work on Windows, I don’t love writing yaml and people I respect on the internet recommended it. just also comes with tab-completion in my terminal with no additional effort, which helps with spelling commands right and remembering which commands exist.

Other perfectly valid options for doing the same job would be any flavour of make, taskfile, earthfile or your own handwritten scripts in the programming language of your choice. The actual task runner isn’t the important idea here, rather the standardisation of what tasks/commands/recipes exist and what they are called.

I’ve managed to get myself into the muscle memory of always reaching for just commands when I want to do something, and I think this has saved me a bunch of mental overhead remembering how I should run a command. Because running commands via just has got so ingrained in my workflow, I find it easier to remember to add a justfile to any project I work on that doesn’t have one yet.

When I’m adding just to a project, I start with a template justfile with the minimal tasks, or as just calls them, recipes I eventually want to implement. When I try to run a recipe, it prints out that I didn’t implement that recipe yet, and a hint as to what might go there. This makes it really easy to build up the recipes as required for a project, and doesn’t break my muscle memory of always running the same tasks with the same commands.

Implementing this shared way of running common tasks across different projects has made context switching a lot easier and faster, and freed up my brain to make progress on features rather than remembering how exactly to run the tests for a given project.