Fall Biff updates

Hola! Since last time, I've mostly published a bunch of articles:

• Understanding htmx
• XTDB compared to other databases
• Philosophy of Biff (see the HN discussion for Back-to-the-Future quotes)
• Should you use Biff?
• How to use Postgres with Biff
• How to use re-frame with Biff

Thanks again to Clojurists Together for the grant. I've also cut a couple new releases:

• v0.7.11 (Sep 9): small improvements to the template project and bb tasks, a new com.biffweb/s3-request function (great for DigitalOcean Spaces), and a couple bug fixes.
• v0.7.15 (Sep 20): fixes some regressions in v0.7.11, and modifies bb deploy and other tasks so you don't have to enter your password more than once (only applicable if you don't already have ssh-agent set up).

And there's a new Community Projects page that lists a few things that people have built.

In the works I have an example repo which adds a Dockerfile + Uberjar compilation to a Biff app. I've sucessfully deployed that repository to Fly.io and DigitalOcean App Platform, and I almost got it fully working on DigitalOcean Kubernetes (I gave up when I got to the SSL cert instructions). I was going to write up a "How to deploy Biff with Docker/Kubernetes/what-have-you" guide, but instead I think I'll just merge the Dockerfile + Uberjar stuff into the main Biff repo and maybe include a few pointers in the comments. Expect that to be merged in the next week or two.

(Aside: I'm excited about Fly.io. I think I'll eventually have that be the default/recommended deployment platform for Biff. However, so far I've hit weird bugs every time I've tried to use it, possibly due to Fly's popularity/scaling issues. Reliability is my #1 requirement for a deployment platform, so for now I'll be cheering from the sidelines.)

Roadmap

Coming up, I've got... (*checks notes*):

• Switch the recommended email provider from Postmark to Mailersend (the latter is cheaper and easier to set up).
• Think about redoing config. Instead of having a config.edn file, maybe convert that to a com.example/config.clj file that runs on startup? Most people probably want/are fine with config (not secrets) being checked into source anyway, and doing the config primarily in Clojure instead of EDN would be a little more expressive. Any config values that you don't want in source can always be put in environment variables along with the secrets. I should take a look at how Django et. al. do things.
• Custom XTDB indexes for derived data/materialized views. (!)
• Updates for the Biff/XTDB transaction format, possibly released as a standalone library.
• Open-source Yakread!
• Rewrite Platypub from scratch!

You say potato, I say potato

I've also been toying with the idea of renaming plugins to modules. Two people have mentioned that they found Biff's usage of the term "plugin" to be confusing—normally, "plugin" means "3rd-party plugin" / something "extra" that you add to your already working system. But Biff plugins are primarily 1st-party, and they define your core application logic. I called them plugins because that made architectural sense to me: your Biff components constitute your app's "framework code," and your application code plugs in to that framework.

But if "plugin" is throwing some people off, perhaps "module" would be a decent substitute with fewer misleading connotations? In any case, I already renamed them from "features" to "plugins" previously, so if I change again, I'd like to make sure I'm confident in the new term.

Pathom

Finally—I don't know if or when I'll actually get to this, but I've been thinking a lot about Pathom because we use it at work (along with Fulcro). When I initially tried Pathom (also along with Fulcro) four years ago, I was a bit wary. It felt like something that might be handy in a microservices environment where you have a bunch of different data sources that you wanted to query together. But I'm just building relatively small monolithic apps with a single database.

However, since starting my job last summer, I've realized that your app's business logic is effectively a second data source, and Pathom lets you combine all that code along with your database into a single queryable graph. Pathom also lets you do dependency injection: instead of thinking "what exactly does this function need again? Just the ID, or the whole document? What about sub-documents? Do I need to pass the document through another function first?", you just turn the function into a resolver and let Pathom wire up all the inputs in the required format.

In Yakread and The Sample—both apps with 10k-15k lines of code—I have started to feel the application logic becoming hard to keep track of. I think Pathom is the solution. (The custom indexes I mentioned above would also help.) I don't think I'd want to include Pathom in new Biff apps by default, but I do think Pathom would be a valuable addition for most apps once they've reached a certain size. Thus, a prime topic for a how-to guide at least.

And what would be really interesting would be to make a sort of server-side version of Fulcro: take all your view functions (the ones that return Rum/Hiccup/HTML) and couple them with Pathom resolvers that grab all the input data. When you load a page (or page fragment with htmx), the root view function's resolver loads the data both for the root function and for all the child view functions, recursively.

The last piece would be some plumbing around forms. Normally in Biff apps I convert form data to EDN (namespaced keywords and such) by hand. I also do validation manually. This is OK for the simple forms in my Biff apps, but it would be painful for a form with, say, 30 fields, like the ones I have get to deal with at work. It shouldn't be too hard to make server-side parsing and validation more automatic.

Take all that, put it on Kubernetes, and then we'd have... Biff Enterprise®! (SLAs and support contracts available.)

Or to put it more seriously: we'd have a solid upgrade path for anyone who wants to use Biff in a commercial context and wants to be confident that their codebase won't turn into a hairball as it grows.

Thanks for reading

Hit reply, post on the forum, or come chat on #biff @ Clojurians Slack.