Blogging again

I originally started this blog fifteen or so years ago in order to sharpen my writing skills by emulating some of the blogs that I loved back then. At that time, the bloggers that I looked up to were (don't laugh) Paul Graham and Digitiser creator Mr Biffo. Since then, blogging seems to no longer be a thing (killed by social media, YouTube. smartphones and the brief success and then retirement of Google Reader) and my list of internet heroes has changed somewhat. I regret that. But the one thing that I really regret is not continuing to write.

So my resolution is: I intend to resurrect this blog and try to use it as a medium for my half-baked opinions. If I can figure out a static site generator then I'll move it to my own domain. I imagine that nobody is going to read it but that's ok. I want this to be a mixture of catharsis and me honing my craft.

And hopefully my opinions aren't too half baked.

So long and thanks for all the fish

Today is my last day at IBM and I'm sad to be leaving. I applied to IBM in 2000, partly because I was impressed by the company's early support for the open source movement and partly because an IBMer visiting Warwick University had told us that, were IBM Hursley to be an independent software company at that time, it would have been the sixth largest software company in the world. Mostly, though, it was because IBM promised each student a beer in exchange for a CV. It should be noted that IBM never followed through on that particular promise, giving us instead a glass of wine each. A bit of a bait-and-switch, I'm sure you'll agree, but a classy one.

I only intended to stay at IBM for three years but ended up staying for nearly 14. There are many reasons for this and many things that I am still reluctant to leave behind. Hursley is still a great place to work with a wonderfully creative atmosphere. To work in a campus environment in the beautiful Hampshire countryside is something that I'll miss as I brave the sharp elbows of stressed London commuters over the coming months. Hursley itself is lovely, even if the locals describe IBM as "the funny farm". The greatest of the perks of working in IBM Hursley is the people. I've made some of my best friends here and I've had the privilege of working with some truly brilliant people.

I'm now off to Masabi to work in a much more startup-ey environment, which is both exciting and also rather scary. I'll miss IBM and my friends and former colleagues, though. You guys rule.

Extracting a PAX archive file

Today, I had extract a PAX archive file on a mainframe. Having never done this before, I stumbled for a while until I figured out how to do it. For future reference, here's what I needed to do in order to unzip archive.pax:

pax -r < archive.pax

The Wikipedia page for the pax command is a useful source of example usage:

http://en.wikipedia.org/wiki/Pax_%28Unix%29

Deploying a Clojure app to Bluemix as a WAR file using the WebSphere Liberty buildpack

In a previous blog post, I talked about running a Clojure web app on IBM Bluemix using the Heroku buildpack for Clojure. This isn't the only way that you can run Clojure code on Bluemix, though. In this post, I'll talk about running the same Hello World application using the WebSphere Liberty Buildpack. This buildpack is built in to Bluemix and has the ability to run a web application, in this case packaged as a WAR file.

To begin, let's clone the Hello Clojure app from GitHub:

$ git clone git@github.com:cdpjenkins/hello-clojure.git
Cloning into 'hello-clojure'...
remote: Counting objects: 22, done.
Receiving objects: 100% (22/22), done.
Resolving deltas: 100% (5/5), done.
remote: Total 22 (delta 0), reused 0 (delta 0)
Checking connectivity... done.

This is the same Hello World app that featured in the previous blog post with one addition made to the project.clj:

(defproject hello-clojure "0.1.0-SNAPSHOT"
:description "FIXME: write description"
  :url "http://example.com/FIXME"
  :dependencies [[org.clojure/clojure "1.5.1"]
                 [compojure "1.1.6"]
                 [javax.servlet/servlet-api "2.5"]]
  :plugins [[lein-ring "0.8.11"]]
  :ring {:handler hello-clojure.handler/app
         :uberwar-name "hello-clojure.war"}
  :profiles
  {:dev {:dependencies [[ring/ring-jetty-adapter "1.1.0"]
                        [ring-mock "0.1.5"]]}}
  :min-lein-version "2.0.0")

I've added an uberwar-name key to the :ring section. This is the name of the generated WAR file that we'll generate. Also note that the version of lein-ring must be at least 0.8.11 in order to generate a conformant WAR in order to work with the Liberty buildpack.

Let's use the ring uberwar task to generate our WAR file.

$ lein ring uberwar
Created /Users/cdpj/GitProjects/hello-clojure/target/hello-clojure.war
END

Recall that an uberwar is a WAR file that bundles our webapp together with all of the dependencies.

Next, let's check that our Bluemix API endpoint is still set up from last time (IBMers using the internal Bluemix: your API endpoint won't be the same as this one):

$ cf login
API endpoint: https://api.ng.bluemix.net

Email> <snip>@uk.ibm.com

Password> 
Authenticating...
OK

Targeted org <snip>@uk.ibm.com

Targeted space dev     
API endpoint:   https://api.ng.bluemix.net (API version: 2.4.0)   
User:           <snip>@uk.ibm.com   
Org:            <snip>@uk.ibm.com   
Space:          dev   

Finally, it's a relatively simple matter to push our WAR to Bluemix. Note that this time, we don't use -b to specify a buildpack (since the WebSphere Liberty buildpack is built in to Bluemix). In addition, we use -p to specify the path of the artefact that we want to push, in this case the WAR file.

$ cf push hello-again-clojure -p target/hello-clojure.war 
Creating app hello-again-clojure in org <snip>@uk.ibm.com / space dev as <snip>@uk.ibm.com...
OK

Creating route hello-again-clojure.mybluemix.net...
OK

Binding hello-again-clojure.mybluemix.net to hello-again-clojure...
OK

Uploading hello-again-clojure...
Uploading app files from: target/hello-clojure.war
Uploading 5.2M, 812 files
OK

Starting app hello-again-clojure in org <snip>@uk.ibm.com / space dev as <snip>@uk.ibm.com...
OK
-----> Downloaded app package (5.0M)

-----> Liberty buildpack is starting to compile the droplet
-----> Buildpack Version: v1.4-20140908-1803
-----> Retrieving IBM 1.7.1 JRE (ibm-java-jre-7.1-1.0-pxa6470_27sr1fp2-20140828_01-sfj.tgz) ... (0.0s)
-----> Retrieving com.ibm.ws.liberty-2014.9.0.0-201409081803.tar.gz ... (0.0s)
         Installing archive ... (1.4s)
-----> Liberty buildpack has completed the compile step
-----> Uploading droplet (109M)

0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
1 of 1 instances running

App started

Showing health and status for app hello-again-clojure in org <snip>@uk.ibm.com / space dev as <snip>@uk.ibm.com...
OK

requested state: started
instances: 1/1
usage: 1G x 1 instances
urls: hello-again-clojure.mybluemix.net

     state     since                    cpu    memory         disk   
#0   running   2014-09-24 10:15:09 PM   0.0%   116.5M of 1G   149.4M of 1G   

That's it; the app is now running:

OK so once again, this isn't especially exciting but this process is equally applicable to larger and more complex applications as it is to the simple one that we deployed here. Using the WebSphere Liberty buildpack allows us to run our application on top of code that was specially designed for Bluemix and it allows us to build our application once, on the client, and then deploy the built application as one WAR file. Contrast this with the Heroku buildpack for Clojure which uploads Clojure source code to the server and then compiles it there. There is a third way to run Clojure on Bluemix, which is to build your app into a runnable jar file (together with an embedded web server such as Jetty). I might cover that briefly in a future blog post.

Monospaced-text inline

This is another test to see if I can inline monospaced text in a blog post. Looks like it's working ok, despite my annoyance at having to manually edit the HTML markup in order to make it happen.

A whole blog post (with highlighted source code snippet) in org-mode

This is a test to see if I can write a blog post in org-mode, output the whole thing to HTML and then have it look half decent in my Blogger blog.

Here is some Clojure source:

(defn my-function [message]
  (println "The message is: " message))

I turn the org-mode source into HTML by hitting C-c C-e (to execute org-export) and then I hit H to put the resulting export into a new buffer. It's then easy to copy the contents into Blogger's raw HTML view.

The result doesn't look completely terrible to my eyes. If you don't like the light blue background for the source code then I'm afraid that I must take credit for having chosen that but the good bits were output by htmlize. To be honest, though, I'm not sure what this setup currently buys me, over and above editing the text in Blogger's editor and then using htmlize to output each snippet of source code separately and then copying it into its own <pre> block.

All of this raises the question of why I am still using Blogger in this day and age. To be honest, it's more a matter of inertia than anything else. I'd really like to get round to trying to use something like Jekyll (which seems very cool) or Wordpress (which everyone else seems to have been using since the year dot) but Blogger is what I have been using, on and off, since 2006 and it seems to work relatively well, despite being rather annoying and despite being closed source and a wee bit dumbed down. Thankfully, Blogger does let me customise pretty much all of the HTML and CSS that it spews out and it will have to do until I find the time to properly learn (and migrate to) something else.

Running a Clojure webapp on IBM Bluemix

<EDIT>: a few people have reported seeing the following error when pushing the app to Bluemix:

java.lang.NoClassDefFoundError: javax/servlet/http/HttpServletRequest, compiling:(ring/middleware/multipart_params.clj:39:5)

If you see this then you can work around it by moving the dependency on javax.servlet/servlet-api "2.5" out of :dev :dependencies and into :dependencies (or just clone the latest code from https://github.com/cdpjenkins/hello-clojure). I'll update this once I get to the bottom of why this started failing for some people.
</EDIT>

Today, I signed up for a free trial account on the public beta of Bluemix, IBM's Platform as a Service (disclosure: I work for IBM, though not on Bluemix). Bluemix performs a similar function to Heroku and Azure, allowing you to run applications in a variety of languages and has the advantage that it built upon the open Cloud Foundry platform.

Having played played around deploying Clojure apps on IBM's internal Bluemix beta, I thought I'd write a guide to getting started with Clojure on Bluemix now that it is open to the world.

To sign up, go to http://www.bluemix.net/. In order to do this, you need to create an IBM id if you don't already have one.

This gives you access to the Bluemix dashboard from which you can create new apps and services.

This post won't talk about using the Dashboard because I'm more comfortable using the command line. In order to use the command line, you first need to download the tools. You can find the installation package for your favourite OS here:

https://github.com/cloudfoundry/cli#downloads

In my case, I installed the tools on my Mac using Homebrew:

$ brew tap pivotal/tap
Cloning into '/usr/local/Library/Taps/pivotal/homebrew-tap'...
remote: Reusing existing pack: 200, done.
remote: Total 200 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (200/200), 30.78 KiB | 0 bytes/s, done.
Resolving deltas: 100% (101/101), done.
Checking connectivity... done.
Tapped 7 formulae

$ brew install cloudfoundry-cli
==> Downloading https://cli.run.pivotal.io/stable?release=macosx64-binary&versio
######################################################################## 100.0%
  /usr/local/Cellar/cloudfoundry-cli/6.2.0: 2 files, 18M, built in 26 seconds

Once the Cloud Foundry CLI tools is installed, you can run the cf command to interact with Bluemix. First, let's  point it at the Bluemix API (note to any IBMers who are using the IBM internal Bluemix: your API endpoint will be different to this):

$ cf api https://api.ng.bluemix.net
Setting api endpoint to https://api.ng.bluemix.net...
OK

                 
API endpoint:   https://api.ng.bluemix.net (API version: 2.2.0)   
Not logged in. Use 'cf login' to log in.

... and log in using your IBM ID:

$ cf login
API endpoint: https://api.ng.bluemix.net

Email> xxxx@xxxx.com

Password> 
Authenticating...
OK
...<snip>

Having done that, we need an app. Let's create Clojure web application using the leiningen compojure template:

$ lein new compojure hello-clojure

Let's check out the contents of our project.clj:

(defproject hello-clojure "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :dependencies [[org.clojure/clojure "1.5.1"]
                 [compojure "1.1.6"]]
  :plugins [[lein-ring "0.8.11"]]
  :ring {:handler hello-clojure.handler/app}
  :profiles
  {:dev {:dependencies [[javax.servlet/servlet-api "2.5"]
                        [ring-mock "0.1.5"]]})

Check that the version of lein-ring is at least 0.8.11 because this will make our life easier if we want to run an uberwar of our webapp on Bluemix at a later date.

The compojure template gives us a simple Hello World app which is good enough for our purposes here. We want to push it up to Bluemix using the command line tool but there is one problem: Bluemix does not natively support Clojure web apps and neither does Cloud Foundry. However, that's not a problem because Cloud Foundry and Bluemix have native support for Heroku buildpacks.

In this example, we're going to use the Heroku buildpack for Clojure unchanged. You might be thinking that this is a little dangerous; Heroku might make a change to their buildpack tomorrow that breaks Bluemix. That is true and a safer bet would be to maintain a fork of the Heroku buildpack and test that fork against Bluemix (or even write a new buildpack completely from scratch that is specifically aimed at Bluemix). For now, though, we're going to live on the edge and use the Heroku buildpack directly.

The buildpack already knows how to compile our Clojure webapp but we need to tell it how to run it or it will default to lein trampoline run, which isn't what we want. We do so by adding a file called Procfile to our project with the following contents:

web: lein trampoline ring server-headless ${PORT}

We also need to specify :min-lein-version "2.0.0" in order to require the use of Leiningen 2. Edit project.clj to look like this:

(defproject hello-clojure "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :dependencies [[org.clojure/clojure "1.5.1"]
                 [compojure "1.1.6"]]
  :plugins [[lein-ring "0.8.11"]]
  :ring {:handler hello-clojure.handler/app}
  :profiles
  {:dev {:dependencies [[javax.servlet/servlet-api "2.5"]
                        [ring-mock "0.1.5"]]}}
  :min-lein-version "2.0.0")

We can now push our app to Bluemix. In this example, I'm giving it the name hello-clojure. If you try this out while my app is still live, you'll need to give your app a different name.

$ cf push hello-clojure -b https://github.com/heroku/heroku-buildpack-clojure.git
Creating app hello-clojure in org cdpjenkins@gmail.com / space dev as cdpjenkins@gmail.com...
OK

Creating route hello-clojure.mybluemix.net...
OK

Binding hello-clojure.mybluemix.net to hello-clojure...
OK

Uploading hello-clojure...
Uploading app files from: /Users/cdpj/hello-clojure
Uploading 1.6M, 773 files
OK

Starting app hello-clojure in org cdpjenkins@gmail.com / space dev as cdpjenkins@gmail.com...
OK
-----> Downloaded app package (904K)
Cloning into '/tmp/buildpacks/heroku-buildpack-clojure'...
-----> Installing OpenJDK 1.6...done
-----> Installing Leiningen
       Downloading: leiningen-2.4.2-standalone.jar
       Writing: lein script
-----> Building with Leiningen
       Running: lein with-profile production compile :all
       (Retrieving lein-ring/lein-ring/0.8.11/lein-ring-0.8.11.pom from clojars)
       (Retrieving org/clojure/clojure/1.2.1/clojure-1.2.1.pom from )
       (Retrieving org/clojure/data.xml/0.0.6/data.xml-0.0.6.pom from )
       (Retrieving org/clojure/pom.contrib/0.0.25/pom.contrib-0.0.25.pom from )
       (Retrieving org/sonatype/oss/oss-parent/5/oss-parent-5.pom from )
       (Retrieving org/clojure/clojure/1.3.0/clojure-1.3.0.pom from )
       (Retrieving leinjacker/leinjacker/0.4.1/leinjacker-0.4.1.pom from clojars)
       (Retrieving org/clojure/core.contracts/0.0.1/core.contracts-0.0.1.pom from )
       (Retrieving org/clojure/pom.contrib/0.0.26/pom.contrib-0.0.26.pom from )
       (Retrieving org/clojure/core.unify/0.5.3/core.unify-0.5.3.pom from )
       (Retrieving org/clojure/clojure/1.4.0/clojure-1.4.0.pom from )
       (Retrieving org/clojure/data.xml/0.0.6/data.xml-0.0.6.jar from )
       (Retrieving org/clojure/clojure/1.2.1/clojure-1.2.1.jar from )
       (Retrieving org/clojure/core.unify/0.5.3/core.unify-0.5.3.jar from )
       (Retrieving org/clojure/core.contracts/0.0.1/core.contracts-0.0.1.jar from )
       (Retrieving lein-ring/lein-ring/0.8.11/lein-ring-0.8.11.jar from clojars)
       (Retrieving leinjacker/leinjacker/0.4.1/leinjacker-0.4.1.jar from clojars)
       (Retrieving org/clojure/clojure/1.5.1/clojure-1.5.1.pom from )
       (Retrieving compojure/compojure/1.1.6/compojure-1.1.6.pom from clojars)
       (Retrieving org/clojure/core.incubator/0.1.0/core.incubator-0.1.0.pom from )
       (Retrieving org/clojure/pom.contrib/0.0.20/pom.contrib-0.0.20.pom from )
       (Retrieving org/clojure/clojure/1.3.0-alpha5/clojure-1.3.0-alpha5.pom from )
       (Retrieving org/clojure/tools.macro/0.1.0/tools.macro-0.1.0.pom from )
       (Retrieving clout/clout/1.1.0/clout-1.1.0.pom from clojars)
       (Retrieving ring/ring-core/1.2.1/ring-core-1.2.1.pom from clojars)
       (Retrieving org/clojure/tools.reader/0.7.3/tools.reader-0.7.3.pom from )
       (Retrieving ring/ring-codec/1.0.0/ring-codec-1.0.0.pom from clojars)
       (Retrieving commons-codec/commons-codec/1.6/commons-codec-1.6.pom from )
       (Retrieving org/apache/commons/commons-parent/22/commons-parent-22.pom from )
       (Retrieving org/apache/apache/9/apache-9.pom from )
       (Retrieving commons-io/commons-io/2.4/commons-io-2.4.pom from )
       (Retrieving org/apache/commons/commons-parent/25/commons-parent-25.pom from )
       (Retrieving commons-fileupload/commons-fileupload/1.3/commons-fileupload-1.3.pom from )
       (Retrieving org/apache/commons/commons-parent/28/commons-parent-28.pom from )
       (Retrieving org/apache/apache/13/apache-13.pom from )
       (Retrieving commons-io/commons-io/2.2/commons-io-2.2.pom from )
       (Retrieving org/apache/commons/commons-parent/24/commons-parent-24.pom from )
       (Retrieving clj-time/clj-time/0.4.4/clj-time-0.4.4.pom from clojars)
       (Retrieving joda-time/joda-time/2.1/joda-time-2.1.pom from )
       (Retrieving org/clojure/core.incubator/0.1.0/core.incubator-0.1.0.jar from )
       (Retrieving org/clojure/tools.macro/0.1.0/tools.macro-0.1.0.jar from )
       (Retrieving org/clojure/tools.reader/0.7.3/tools.reader-0.7.3.jar from )
       (Retrieving commons-codec/commons-codec/1.6/commons-codec-1.6.jar from )
       (Retrieving org/clojure/clojure/1.5.1/clojure-1.5.1.jar from )
       (Retrieving commons-io/commons-io/2.4/commons-io-2.4.jar from )
       (Retrieving commons-fileupload/commons-fileupload/1.3/commons-fileupload-1.3.jar from )
       (Retrieving joda-time/joda-time/2.1/joda-time-2.1.jar from )
       (Retrieving compojure/compojure/1.1.6/compojure-1.1.6.jar from clojars)
       (Retrieving clout/clout/1.1.0/clout-1.1.0.jar from clojars)
       (Retrieving ring/ring-codec/1.0.0/ring-codec-1.0.0.jar from clojars)
       (Retrieving clj-time/clj-time/0.4.4/clj-time-0.4.4.jar from clojars)
       (Retrieving ring/ring-core/1.2.1/ring-core-1.2.1.jar from clojars)
       Compiling hello-clojure.handler
-----> Uploading droplet (68M)

0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
1 of 1 instances running

App started

Showing health and status for app hello-clojure in org cdpjenkins@gmail.com / space dev as cdpjenkins@gmail.com...
OK

requested state: started
instances: 1/1
usage: 1G x 1 instances
urls: hello-clojure.mybluemix.net

     state     since                    cpu    memory         disk   
#0   running   2014-07-01 10:28:01 PM   0.0%   377.9M of 1G   173.6M of 1G   

Bluemix has downloaded the dependencies for the app, built it and started it running.

Finally, we can access the app at the URL http://hello-clojure.mybluemix.net

It's not the most exciting thing in the world but it's a start and we can continue to develop our app as we would with any other Clojure app.

The source code for my little hello-clojure app is available here:

https://github.com/cdpjenkins/hello-clojure

It's also available in IBM's jazzhub DevOps Services in case you are interested:

https://hub.jazz.net/project/cdpj/hello-clojure

My first impressions of Bluemix are good so far. It appears to work and I was able to get my app running without too much bother, which is great. And Bluemix supports a fair few applications and services, some open source and some from IBM. I'll try to write about deploying a slightly more complex Clojure app in a future post.

Using htmlize to prettify Clojure source code

I have been trying to figure out the best way to include syntax-highlighted source code in a blog post. One solution is to use htmlize in emacs.

Let's say I want to include the following syntax-highlighted code in my blog:

(defn cheese-ston [a b]
  (println a b))

The first step is to install htmlize. I've already got MELPA added so I just install the htmlize package from that.

Next, I want to configure htmlize to output any CSS inline (so that I can cut and paste HTML snippets into Blogger) so I add the following to my init.el and execute it immediately.

(setq htmlize-output-type 'inline-css)

Next, I highlight the code that I want to export and then do:

M-x htmlize-region

This gives me a new buffer with a complete HTML file containing my text, wonderfully highlighted. I can then copy everything within the body of that HTML page into the raw HTML editor in Blogger:

<pre>
<span style="color: #707183;">(</span><span style="color: #a020f0;">defn</span> <span style="color: #0000ff;">cheese-ston</span> <span style="color: #7388d6;">[</span>a b<span style="color: #7388d6;">]</span>
  <span style="color: #7388d6;">(</span><span style="color: #483d8b;">println</span> a b<span style="color: #7388d6;">)</span><span style="color: #707183;">)</span></pre>

Before switching back to the Compose view, I have to ensure that Blogger is not going to quote my HTML so I hit Options on the right and make sure that Show HTML literally is selected.

Previewing the post, I can now see my Clojure code wonderfully highlighted as above.

This whole process is slightly faffy. What I really want is to remove the step where I have to find the right bit of HTML to copy into Blogger's raw HTML editor. One solution would be to figure out how to get htmlize to output HTML snippets (rather than whole documents). Another alternative would be to use org-mode - which works well with htmlize - to output the whole blog post. I'll write more if I get round to trying that.