This week I’ve been prototyping some data processing tools that will work across the platforms we use (Ruby, Clojure, .NET). Having not tried Protocol Buffers before I thought I’d spike it out and see how it might fit.
The Google page obviously has a lot more detail but for anyone who’s not seen them: you define your messages in an intermediate language before compiling into your target language.
There’s a Ruby library that makes it trivially easy to generate Ruby code so you can create messages as follows:
Clojure and Leiningen
The next step was to see how these messages would interact with Clojure and Java. Fortunately, there’s already a few options and I tried out clojure-protobuf which conveniently includes a Leiningen task for running both the Protocol Buffer compiler
I added the dependency to my project.clj:
At the time, the
protobuf library expected your
.proto files to be placed in a
./proto directory under your project root. I forked to add a
:proto-path so that I could pull in the files from a git submodule.
Assuming you have a proto file or two in your proto source directory, you should be able to invoke the compiler by running
$ lein protobuf compile Compiling person.proto to /Users/paul/Work/forward/data-spike/protosrc Compiling 1 source files to /Users/paul/Work/forward/data-spike/classes
You should now see some Java
.class files in your
Using clojure-protobuf to load an object from a byte array looks as follows:
I ran into a little trouble when I came to build the command-line tool and deploy it. When building with
lein uberjar it seemed that the
./classes directory was being cleaned causing the protobuf compiled Java classes to be unavailable to the application (causing the rest of the application to fail to build- I was using tools.cli with a
main fn which meant using :gen-class).
It turns out that Leiningen’s
uberjar task checks a different option when determining whether to clean the project before executing: :disable-implicit-clean. I added
:disable-implicit-clean true to our
project.clj and all was good:
$ lein protobuf compile, uberjar
I wasn’t a registered user of the Leiningen mailing list (and am waiting for my question to be moderated) but it feels like
uberjar should honour
:clean-non-project-class too. I’d love to submit a patch to earn myself a sticker :)