Clojure, Abstraction and ZeroMQ
For me, the stand-out talk when I attended clojure-conj last year was Mark McGranaghan’s talk on Ring: “One Ring to Bind Them”. I’ve been working with ZeroMQ and Clojure these past few weeks and it’s been all the better because of Mark’s talk.
Core Abstractions
Mark’s talk deftly highlights the beauty of some of Clojure’s core abstractions, sequences and functions, and how Ring modeled it’s world around them. The following quote is from Alan Perlis and often used when talking about Clojure:
It’s better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures
Building upon Clojure’s core abstractions makes it easy to compose new functionality from existing libraries. For example, Ring uses the concept of middleware- optional building blocks that decorate your application with additional behaviour. Because the core request handler is just a function, it’s possible to re-use Clojure’s threading macro to build up more complex behaviour:
The main-routes Var is built using Compojure, this is then wrapped by middleware to handle logging, exception handling and nested query parameters.
Of course, many other languages and libraries solve the problem in a similar way- the Java web frameworks I used to work with would most likely wrap this kind of stuff in via. an AOP framework that’s wired in with some IoC container like Spring.
What makes Clojure different, apart from not needing any more frameworks, is that it’s using such a core, language-level abstraction: there’s no need to write middleware adapters to join disparate things together.
ZeroMQ
I mentioned that it had been my working with ZeroMQ that reminded me of all of this. I’m working on wiring Esper into our production environment so that we can monitor the behaviour of our system better.
Much like Luca described in his talk, Node.js and ZeroMQ, I’m using Node.js and ZeroMQ as a kind of glue- tailing logs and other bits of information and re-publishing. This is then be aggregated and consumed in one place.
Core Abstractions
There are two things we need to do with our ZeroMQ sockets:
- Consume data (from an ‘infinite’ stream)
- Publish data
Of course, it’s trivialised a little, but it’s largely all we need to do.
For consuming data in Clojure, my first cut was fairly traditional- start a (while true) infinite loop that consumed messages and called out to a handler function. It looked a little like this:
It worked, but it’s pretty ugly and certainly not simple. Given it’s just calling a handler it’s also impossible to connect this to much around the rest of Clojure’s libraries without some work. Time to re-think.
Going back to our original statement of what we need to do with our sockets it’s possible to see there are some basic building blocks we could use:
| Utility | Clojure Type |
|---|---|
| Consume data (from an ‘infinite’ stream) | Sequence |
| Publish data | Function |
Our compound consumer above can be re-written and turned into a sequence with repeatedly as follows:
Note how much cleaner the consumer code now is, and how we can re-use any of Clojure’s sequence functions to operate on the stream.
For our publisher we can write a function which returns a function:
The generated function closes over the initialised socket so that any consumer that wants to publish data can do so by just calling the function passing the bytes to be sent. It provides a clean interface and has the added advantage we can use it with many higher-order functions.
For me, this has really highlighted the power of Clojure over other JVM based languages- clean, simple, core abstractions that can be composed in simple, powerful ways.
Incidentally, I’m working on tidying up the code that interacts with both ZeroMQ and Esper. When they’re a little tidier I’ll open-source them and publish on my GitHub account, and most likely post an article or two here too. Stay tuned!
