Biff 2.0.4: LLM skills

I've just released Biff v2.0.4. This release has a hodge-podge of tweaks and fixes based on my own usage. There is also one big addition: a set of LLM skills. From the starter project's README:

• biff-spec-review: reads spec/ and generates a plan to implement it.
• biff-spec-dev: implements the plan generated by biff-spec-dev. Calls biff-code-review afterward.
• biff-code-review: reviews recent code changes against a list of Biff-specific coding conventions. Invoked by biff-spec-dev automatically, or you can run it manually.
• biff-code-quality: instructions the agent to run clojure -M:run code-quality and fix any test/lint failures.
• biff-dev: instructions for using the REPL. Invoked by agents automatically when they write Clojure code.

After upgrading your biff.tasks dependency to 2.0.4, these skills will be synced to your local project whenever you run clj -M:run dev.

The most important skill here is biff-code-review. It contains a list of rules for writing code that conforms to Biff conventions. The rules are the result of my own painstaking review of reams of crappy LLM-generated Biff code. From testing them out on a fresh project, I'm fairly pleased with the result. The code I've gotten is definitely still a little weird in places, but it's crossed the "good enough" threshold.

My general flow for making Biff apps is now:

• Start a new project.
• Edit the spec/README.md file.
• Run biff-spec-review and then biff-spec-dev (the latter of which calls biff-code-review).

Sometimes I'll use a fancy model to review specs/code and a dumber model to do the implementation.

I had some trouble figuring out a structure for getting LLMs to actually follow my rules. I've had success with this flow:

1. I try to phrase each rule so it's very explicit/unambiguous/black and white. Either the rule is violated or it's not.

2. biff-code-review instructs the LLM to copy a checklist for all the rules into your project. The LLM goes through your code and adds a bullet point (with filename and line numbers) to the checklist file whenever it finds a violation. After the code review, any checklist items without violations are checked off.

3. The LLM then goes through the checklist and applies fixes.

I structured this in a way so that if needed you can do a human-in-the-loop thing: you can edit the checklist before step 3 starts to add any violations the LLM may have missed. During step 3, the LLM doesn't edit the checklist file, so you can manually mark off/not mark off items and then redo step 3.