This week I’ve been prototyping some data processing tools that will work across the platforms we use (Ruby, Clojure, .NET). Having not tried Protocol Buffers before I thought I’d spike it out and see how it might fit.
Protocol Buffers
The Google page obviously has a lot more detail but for anyone who’s not seen them: you define your messages in an intermediate language before compiling into your target language.
There’s a Ruby library that makes it trivially easy to generate Ruby code so you can create messages as follows:
Clojure and Leiningen
The next step was to see how these messages would interact with Clojure and Java. Fortunately, there’s already a few options and I tried out clojure-protobuf which conveniently includes a Leiningen task for running both the Protocol Buffer compiler protoc and javac.
I added the dependency to my project.clj:
[protobuf "0.6.0-beta2"]
At the time, the protobuf library expected your .proto files to be placed in a ./proto directory under your project root. I forked to add a :proto-path so that I could pull in the files from a git submodule.
Assuming you have a proto file or two in your proto source directory, you should be able to invoke the compiler by running
$ lein protobuf compile Compiling person.proto to /Users/paul/Work/forward/data-spike/protosrc Compiling 1 source files to /Users/paul/Work/forward/data-spike/classes
You should now see some Java .class files in your ./classes directory.
Using clojure-protobuf to load an object from a byte array looks as follows:
Uberjar Time
I ran into a little trouble when I came to build the command-line tool and deploy it. When building with lein uberjar it seemed that the ./classes directory was being cleaned causing the protobuf compiled Java classes to be unavailable to the application (causing the rest of the application to fail to build- I was using tools.cli with a main fn which meant using :gen-class).
I always turn to Leiningen’s sample project.clj and saw :clean-non-project-classes. The comment mentioned it was set to false by default so that wasn’t it.
It turns out that Leiningen’s uberjar task checks a different option when determining whether to clean the project before executing: :disable-implicit-clean. I added :disable-implicit-clean true to our project.clj and all was good:
$ lein protobuf compile, uberjar
I wasn’t a registered user of the Leiningen mailing list (and am waiting for my question to be moderated) but it feels like uberjar should honour :clean-non-project-class too. I’d love to submit a patch to earn myself a sticker :)