Biff is now sponsored by Clojurists Together and JUXT!

A couple months ago my gut started telling me that I should invest more time in Biff. I think it has a lot of potential, but it’s taken a back seat to the business since my savings won’t last forever. I see Biff as a long-term, important-but-not-urgent project. Now the business finally has enough traction that I feel comfortable with increasing Biff’s priority. I’m now spending Fridays + some time over the weekend to work on Biff, and I’ve started soliciting sponsorships to ensure that I can continue to justify doing so.

With that out of the way, let’s talk about what’s next for Biff! But first, some background for the uninitiated.

Some background

Biff is designed to help solo developers move fast. When you create a new Biff project, you start out with a minimal-but-complete starter web app, including scripts for provisioning and deploying to an Ubuntu VPS. Biff tries to make as many decisions for you as possible.

Biff is also designed to avoid getting in your way as your project grows (a common criticism of frameworks). As much of the code as possible is written as library code, often in the form of high-level helper functions for other libraries like XTDB and Reitit. The glue code is stored in a project template and copied into new projects a la Luminus, so it’s easy to change. And the various pieces are put together using something kinda-sorta like Component, but more minimalist (it’s about 10 lines and is essentially the same as (fn start-system [config components] (reduce #(%2 %1) config components))).

Also, Biff is tiny. In the mostly-finished next release, Biff’s library code is about 1,100 lines, and the example/template project is another 500 lines.

In general: Biff is designed to meet my needs as an early-stage bootstrapped entrepreneur, with all the trade-offs that entails. I’d like to expand out gradually from there. I think the Clojure ecosystem would benefit from having an opinionated, batteries-included, “simple and easy” web framework with lots of documentation and community support, and maybe Biff could become that.

(There is some scope overlap between Biff and Luminus/Kit—however there are also lots of differences between the projects, so I don’t think my time spent on Biff will be duplicating anything that’s already been done, the Lisp Curse meme notwithstanding 😉)

What’s next for Biff

Biff has evolved quite a bit since the original release. About every 8 - 12 months I make a release with significant changes based on my experience using Biff daily. I’m in the middle of another such release.

The first big change is that I’ve decided to remove the ClojureScript/SPA parts of Biff and instead go with “HTML over the wire,” the same approach used by Hotwire, Phoenix LiveView, and HTMX. See also. (I’ll be using HTMX for Biff). Biff was originally inspired partially by The Web After Tomorrow and Firebase. The idea is/was to abstract away the server as much as possible, so you mainly just write frontend code and it feels like you have direct access to the database. That includes real-time updates—I attempted to extend the reactive programming model so it reached all the way to the database, instead of just covering client-side state. That eliminated the need to write data synchronization logic between backend and frontend. If something relevant changed in the database (e.g. in a chat app, someone sends you a message), the change was synced to the frontend by Biff and then rendered by React. It was basically a clone of Firebase’s realtime queries for XTDB, and it worked great for the scale I was working at.

However, six months after Biff’s first release, I pivoted my startup and started building applications that didn’t need the full interactivity of a SPA. I added an option to Biff’s project setup script so that you could specify if you wanted a SPA or a server-rendered app. But since I wasn’t using the SPA features myself, they weren’t further developed. By switching to HTMX, we can have a single default setup that (1) doesn’t have the extra complexity of a SPA, and (2) supports enough interactivity for most applications (including real-time updates via web sockets or server-sent events).

The second big change is that I’ve switched to a develop-in-prod workflow almost exclusively. Instead of starting up a JVM on my laptop, I connect to the production web server and/or worker over nREPL. Code changes are deployed as soon as I eval them, and the updated files are rsynced to the server(s) on write, so changes won’t be lost if the process restarts. Since my app is server-side rendered, the entire application can be developed this way. Obviously this isn’t the right choice for every situation, but it’s been a huge productivity boost for me, and since Biff prioritizes small projects, I’d like to enable this workflow out-of-the-box. (Though the regular develop-locally-then-deploy workflow will still be first class too!) I will however need to do a bit more experimenting to find the right setup, since my current setup can be brittle, and I’d like anything Biff provides to be robust.

(Incidentally, Biff’s develop-in-prod workflow might also be a nice way to demonstrate REPL-driven development to not-yet-Clojurists, since the “what’s so great about the REPL” question seems to come up a lot. Perhaps Biff could even run VS Code on the server, with Calva etc preconfigured, so the entire getting-started flow is “(1) click this button to provision a DigitalOcean VPS with Biff pre-installed, (2) go to this URL to open VS Code, (3) save any file to deploy code changes.“ A bit like Replit. But I digress.)

Other than those two things, there are a handful of smaller changes based on my personal experience using Biff, some of which I’ve already completed. Here’s the entire todo list for this release.

Completed

1. Instead of organizing Biff into 5 - 10 separate tiny libraries, just put it all in one library with the public API under a single namespace (currently com.biffweb, though I may change my mind before the release).
2. Migrate from Crux to XTDB.
3. Update Biff’s transaction format so that it’s more ergonomic, and add support for unique-key constraints. (Biff transactions are a convenience layer on top of XT transactions which provides higher-level operations, including schema enforcement via Malli).
4. For my own convenience, convert the template project (which requires generating a new project before you can run any of the code) to an example project (which can be run immediately after cloning the repo). It’ll be easier for me to make changes that way.
5. Rewrite the example project so it uses the new com.biffweb library. Along the way, make any changes to library helper functions and code organization that seem fit. Also make code changes (both in the example project and in the library) as needed to facilitate develop-in-prod. For example, use more late binding so that various parts of the application can be updated without restarting.
6. Also update the example project to use HTMX.
7. Update the authentication code to remove the Login CSRF vulnerability. Currently authentication is done via sending a login link via email, but there’s nothing to stop an attacker from sending a login link for their own account to someone else.
8. Structure things so that when needed, the application can be easily deployed with separate processes for web servers and workers.

In progress

1. Figure out what to do for the develop-in-prod workflow. Should we go all in, or should we simply make it easy to connect your local app process to the production database? Or even just add helper functions to export the production database to your local machine, then add some documentation for anyone who wants the full develop-in-prod experience? If we go all in, make sure that dev-in-prod can coexist with a local-dev workflow.

Not yet started

1. Add helper code and/or documentation for common ops tasks, like backups, monitoring and alerting.
2. Figure out what to do for new project creation. Should we tell people to copy the example project and change namespaces manually? Should we provide a script that changes namespaces and other values automatically? Should we provide a separate, minimalist template project that wouldn’t need to be updated as often as the example project?
3. Make sure the task script works on Mac (I use Linux myself, via WSL). Decide if supporting Windows without WSL is important. (Maybe everything already works via Git Bash?).
4. And finally, the really big one: update all the documentation 😬. Probably also redo the website (Update: I ended up starting on this so I could publish this blog post there!).
5. Oh, and clean up the GitHub issues (add new ones, close old ones that are no longer relevant).

I’m hoping that this will be the last major overhaul and that future releases will improve Biff more incrementally. Until I’m certain of that, I’ll keep a -beta postfix in the versioning. I would be surprised if anyone else has any existing Biff projects that they’re actively developing, but if there is anyone in that situation, let me know and I’ll show you how you can gradually migrate to the new version of Biff. I’ll be doing that for my own codebase.

After the release

After this next release is out, I’m expecting to spend most of my time writing additional documentation, tutorials, and blog posts; answering questions and iterating on feedback; working on example projects; and in general, trying to help people learn to make web apps with Biff. I’m approaching this with roughly the same mindset I have for growing my startup (i.e. growth requires a lot of work beyond core product development). As part of the “marketing funnel” I’m going to send out a Biff newsletter with project + community updates a couple times per month. That’ll help remind people that Biff exists, and it’ll help build confidence that Biff is being actively developed and is worth trying for e.g. your next weekend project.

I might try to hit two birds with one stick here: I always have five or six ideas for web apps that I wish existed. The main reason I went into entrepreneurship was so I could work on my own ideas. However I’ll never have enough time to build them all myself. So—I’d like to try writing up descriptions of interesting projects that could be built with Biff and include them in the newsletter. A lot of these will be my own ideas, but I could also do writeups of ideas which others suggest and try to help them connect with other people interested in working together. (For example, I know someone who has previously used Biff to try building a solution for modeling legal contracts as data). I’ll dedicate some of my time to providing advice/code review/etc to anyone who’s building something with Biff.

If any of this sounds interesting, sign up for the newsletter below. The first issue will go out once I finish the next release.