v1.8.0: config, tasks, and uberjars

I've just released v1.8.0. See the release notes for upgrade instructions. The biggest change is to the way Biff handles config and tasks by default.

Before I get into that, one quick announcement: I'll be giving a presentation at the London Clojurians meetup on March 5th. Most/all of the documentation I've written for Biff is goal-oriented; it tries to help you get stuff done. In this talk I'm planning to go over more of the behind the scenes stuff—the design decisions that you don't have to know about to be productive, but are interesting nonetheless.

So, about that release:

Config

Config is now parsed by Aero, which makes it easier to decide which values you’d like to be defined in environment variables and which ones should be hardcoded in config.edn (now located under the resources directory). config.edn is also checked into source by default now, and there’s a new config.env file—similar to the old secrets.env file, but not just for secrets.

These changes make it easier to deploy Biff in a variety of ways, since any config that you need to set at runtime can be set with standard environment variables instead of with a Clojure-specific config.edn file. It’s also easier to work with open-source apps now since the amount of config/structure outside of source control is minimized. You can clone a Biff app repo and run the dev task without touching any config files, and a default config.env file will get generated for you.

Tasks

Tasks like bb dev and bb deploy are now implemented with plain Clojure instead of Babashka. Tasks are invoked with clj -M:dev <task name>, e.g. clj -M:dev deploy. This means tasks have a bit more startup time (1 - 2 seconds on my machines), but new Biff users are no longer required to install Babashka. This also makes some tasks easier to write since we have access to more libraries, not just those that are bb-compatible.

Biff also defines a general-purpose task runner that could in theory be used in other projects—my attempt at making clj-based tasks fairly ergonomic, fast-ish, and reusable. Until Babashka starts getting bundled with clj, I think this is an area worth exploring. In fact, you could even use that task runner with Babashka. I might experiment with that if it turns out that some Biff users would benefit from quicker task startup time: tasks could be run with clj by default, but anyone who wants to could easily switch over to bb instead. For tasks that aren't bb-compatible, the task could be rewritten to check if it's running via clj or bb, and if it's the latter, the task could re-run itself in a clj subprocess.

As it is, by far the most common task is the dev task which starts up the app with clj regardless, so startup time for that isn't affected.

Uberjar/Docker

There is a new clj -M:dev uberjar task which packages your app into an Uberjar. New projects also include a Dockerfile. Like the config changes, this makes it easier to deploy Biff apps in a variety of ways besides just the default approach of using DigitalOcean droplets. The default, non-uberjar/docker deployment path is still  recommended for most people though.

The config and task change changes are both optional: you can upgrade your Biff dependency but continue to use the old config format and bb tasks if you like. You can also upgrade to the new config format without switching from bb tasks to clj tasks. However, if you do switch to clj tasks, you must upgrade your config since the new clj tasks only recognize the new config format.

And finally: I have also renamed Biff “plugins” to Biff “modules.” It turns out that “plugins” was confusing to some people because usually plugins are 3rd party extensions not related to core functionality. When I (re)named them to “plugins” (from “features” previously) I was only thinking about the architectural sense of “what do you call the thing that you—ehrm—‘plug in’ to a framework?” Either way, “modules” is an architecturally accurate term that has fewer potentially misleading connotations. I hope everyone likes the new term, because I'm not switching a third time.

Roadmap

Next on the TODO list, we have:

1. Prepare for the London Clojurians meetup.
2. Switch the default email provider from Postmark to Mailersend (the latter is cheaper and easier to set up).
3. Several XTDB-related things: try using custom indexes for materialized views; start experimenting with XTDB v2; make some updates to biff/submit-tx and think about possible improvements to the transaction format—maybe even release it as a standalone lib that works with both XTDB v1 and v2.
4. Experiment with Pathom (discussed previously). As I was updating the eelchat tutorial for this release, it struck me that even that simple app could benefit from Pathom.
5. Improve the ergonomics for making complex forms with htmx—the kind of forms you might find in an enterprise SaaS (like, say, the one I work at).