Using Javascript libraries from Clojure

I'm going to try a new thing with this newsletter: instead of only sending it out once a month or so when there's a new release, I'll send it each Saturday with various Biff tips, mini-howtos, things that were discussed on the Slack channel, etc.

This past week I've been playing around with serverless functions on DigitalOcean. They launched those in May of last year, after acquiring a startup that was doing serverless functions as a standalone platform in the previous year.

The serverless functions make a nice complement to droplets. Specifically, there are a couple Javascript libraries that I use in my app: Readability, an article parser/extractor that powers Firefox's reader mode; and Juice, a library that takes some HTML and converts all the styles to inline. For example, if you passed this to Juice:

<html>
  <head>
    <style>
    .red {
      color: red;
      }
    </style>
  </head>
  <body>
    <p class="red">Hello</p>
  </body>
</html>

It would return the following:

<html>
  <body>
    <p style="color:red;">Hello</p>
  </body>
</html>

The most common use-case for this is sending email—some email clients only support inline styles. I was under the impression that Gmail in particular didn't, but it seems my knowledge on that was out-of-date by about seven years. I wouldn't be surprised if Outlook still has issues...

Anyway, another way Juice is helpful—and the way I'm using it in my app—is that it can help you embed 3rd party HTML + CSS in your site. My app can receive and display emails, and inlining the styles prevents them from messing with the rest of the page.

Back on topic. Since I'm using Biff, my app is running on the JVM, not Node. And I need some way to run Javascript.

The Bad Old Way that I've been using for about... six months, is I have a tools.js file in my resources directory. I have a with-js-tools helper function which runs the file via Node in a subprocess:

(defn with-js-tools [f]
  (let [path (.getPath (io/resource "tools.js"))
        proc (babashka.process/process ["node" path])
        lock (Object.)]
    (try
     (with-open [stdin (io/writer (:in proc))
                 stdout (io/reader (:out proc))]
       (f (fn [command opts]
            (locking lock
              (binding [*out* stdin]
                (println (cheshire/generate-string
                           (assoc opts :command command))))
              (binding [*in* stdout]
                (cheshire/parse-string (read-line) true))))))
     (catch Exception e
       (println (slurp (:err proc)))
       (throw e)))))

This allows me to communicate with the Node process via pipes. From the Clojure code, I get a nice function that I can call, for example:

(with-js-tools
  (fn [js]
    (let [url "https://example.com"
          html (slurp url)
          parsed-article (js :readability {:url url :html html})]
      ...)))

The benefit of doing this instead of just calling (clojure.java.shell/sh "node" "readability" ...) is that you can pass in multiple documents for parsing/inlining without needing to structure your code so they're all be done together in a batch. You only have to start up a Node process once, and then you get a handy js function that you can pass around and call from wherever you like.

However, this still has major downsides: you're running the JVM and Node on the same machine, which is hard on your RAM. So far I've mitigated this issue by, erm, upgrading to an 8GB droplet. (Perhaps you can tune the JVM/Node so they live happily together on a single machine, but I never figured out how... the memory limit options seemed to be treated more like guidelines, or they only applied to the heap, or something like that.)

Serverless functions are a much better solution. Package up each JS lib as a separate function and let DigitalOcean host them somewhere other than your droplet. I've made two tiny functions, one for Readability and one for Juice. I have a utility function for calling them:

(defn cloud-fn [{:keys [biff/secret cloud-fns/base-url} endpoint opts]
  (http/post (str base-url endpoint)
             {:headers {"X-Require-Whisk-Auth" (secret :cloud-fns/secret)}
              :as :json
              :form-params opts}))

(cloud-fn ctx "juice" {:html "<html>..."})

Developing and deploying the functions was pretty convenient. I made a few Babashka tasks to help out:

(defn fn-logs []
  (shell "doctl" "serverless" "activations" "logs"
         "--limit" "3"
         "--follow"))

(defn fn-deploy []
  (shell "doctl" "serverless" "deploy" "cloud-fns"))

(defn fn-dev []
  (future (shell "doctl" "serverless" "watch" "cloud-fns"))
  (fn-logs))

fn-logs prints the logs from the deployed functions to my terminal, fn-deploy deploys them, and fn-dev deploys the functions whenever I save a file. You can install dependencies with e.g. npm install --save juice and it Just Works, with one caveat: deployment was quite slow after I added some NPM dependencies. I guess it has to zip up the whole node_modules directory and upload it each time, and my internet connection isn't the fastest.

I think you can alternately deploy the functions by hooking up a GitHub repo to DigitalOcean, and then the functions get built on The Cloud. If I do much more function developin', I'll probably look into that.

Last week on the #biff Slack channel (you'll need to join Clojurians for these links to work):

Tips on upgrading to the new authentication plugin in Biff v0.7.0.
Writing unit tests for biff/submit-tx.
Notes on how to use upsert.
How to make Biff refresh the web page automatically when you make changes.
Code review of adding a new entity to the database with read/write UI.

Anything in particular you'd like me to write about? Hit reply.

Published by Jacob O'Bryant on 25 Feb 2023