A simple service oriented architecture using Ruby and Go

Some common pain points seem to keep cropping up time and time again as applications grow in size and complexity:

  • tests get longer to run
  • deployments become a much more involved process
  • making a small fix to one area of code can often break something in another
  • it can be much harder to reason about the system as a whole

These large, monolithic applications are affectionately known as monorails in the Rails world and the common wisdom is to start splitting up the codebase into services: to extract areas of functionality out into self-contained, independent applications which work in unison to deliver the original capability.

Here’s how Songkick did it and how SoundCloud did similar recently.

Close, but no cigar

On a previous contract I worked on a platform which claimed 20 million users and the only way it was possible to handle that sort of capacity was to use services, lots of services.

The platform was centred around a legacy ColdFusion application hooked up to a MySQL database and the other applications had a variety of ways of accessing the central data:

  • using ActiveResource or libraries derived from it to consume a RESTful API provided by the ColdFusion app
  • connecting directly to the database and defining their own models and schemas
  • using one of the two semi-aborted attempts at packaging up a collection of ActiveRecord models into a shared library
  • mixing and matching a selection of the above in different areas of their codebases

Unfortunately there was little consistency and the organisational culture encouraged quick fixes and neglect of old code. Both new development and maintenance work provided numerous challenges as you can well imagine.

The ideal

As I progressed through the term of the contract and had to deal with the peculiarities of the technical ecosystem at hand I discussed with the other engineers what the ideal situation would look like, everything else being equal. The only sane solution seemed to be to develop both an API and the client used to access it.

Since moving on from that contract I’ve wanted to implement a proof of concept of that idea and I’m glad I’ve now codified the basics of it.

A worked example

The API is written in Go and allows manipulation of resources in a standard RESTful manner. No surprises there other than everything is stored in-memory and will so be lost when execution stops.

The client library is in Ruby and basically just wraps a HTTP client with an ActiveRecord-like interface. The final piece is a demo script which uses the library to add and then manipulate a handful of items of data. All very straightforward but I believe it illustrates the concept.

This shameless display of hipster polyglotism can all be found on GitHub. Fill your boots!

Serving up a static site using Go

Like a lot of people in technology a portion of my time is spent wondering what will be the next programming language, framework or whatever to take off and become popular. Of greater personal importance is the matter of which one of them will get added to my toolboox next and press-ganged into productive use.

For years now Ruby has been my goto for a lot things and I’ve been doing more and more OS X and iOS development but it had been some time since I last picked up something new and I had a hankering for some shinyness.

I started tinkering with Node a few years back but it didn’t have enough appeal to invest much of myself in it and Scala looked a likely contender for a while but I didn’t get any further than solving a few Project Euler problems with it, mostly while spending my evenings in hotel rooms in Dublin.

Enter Go

Earlier this year I spent a month in Melbourne and thanks to having some time away from paid work I could see that my preference seemed to be leaning towards developing web services and then consuming those services with native client apps. I wanted to keep my skills sharp so started tinkering with a personal project and for the API piece the relatively new Go seemed ideal for the job at hand.

Once I got started what struck me was that I was producing a single, standalone binary which didn’t need any external dependencies in order to be run on the target architecture. No need for a particular version of a language or a slew of packages to be installed, just a command to be executed. It’s almost like I’d forgotten that that was how things used to be once upon a time after these years of using scripting languages and it was a liberating experience.

For that project I wanted to do everything test-first but unfortunately the depth of my knowledge of Go in general and web application development with it more specifically are not what I want them to be so when I hit a stumbling the project was abandoned. Perhaps I’ll come back to it in the future.

A Need Emerges

So I still wanted to get some Go out into the wild and when I decided to put together a new website for my limited company I thought I could use the situation as a learning experience.

Serving up a static site can be done any number of ways and I certainly didn’t need to go re-inventing the wheel but with a bit of tinkering I had put together a simple app that could look at a requested path and if it corresponded to a file under public/ then send the file’s contents to the client.

What pleased me with the solution was that it was achieved with only what’s provided out-of-box with Go and without too much code. Simple and to the point and getting it running on Heroku was trivial with the Go buildpack.

The End Result

The site, as simple as it may be, can be found at nulltheory.com and the code is on GitHub. Bon appetit!

Decisions, decisions, decisions

I can be an awfully indecisive person at times. Often I’ve been in the grips of analysis paralysis unable to pick a course of action to take, crippled with choice.

When I was a student I read The Dice Man and was sorely tempted to make all my decisions by rolling a die or tossing a coin. Needless to say this is maybe not the best way for a person to navigate through life.

Recently I had to contemplate a specific yes/no situation and joked with the idea of tossing a coin to decide. It then occured to me that I could go one better and after a little bit of tinkering wth Objective-C this is what I made:

Decisions App - Yes

Upon touching the screen the text alternates between “Yes” and “No” and after a random period of time between 1 and 3 seconds it stops. Simple.

This app scratches an itch and it’ll likely never see any additional work though a possible future addition would be to allow a user defined number of free-text choices, maybe even with weighting towards one choice or another.

The source is on GitHub if you think it could be useful for anything.

An Introduction to WebSockets

Last week I gave a short talk at BelfastJS outlining WebSockets: what they are, how you use them, examples, warnings and alternatives. That should cover the basics I think.

A few months ago I spoke at BelfastRuby and it was good again to be sharing some knowledge with a bunch of people enthusiastic about technology and the local community. Hopefully we can keep things going and help make Belfast a supportive environment for those involved in the knowledge economy.

The slides are online along with the code for the demo app.

Using Vagrant and Chef to setup a Ruby 1.9 development environment including RVM and Bundler

It has become common practice these days to use tools like RVM and Bundler to manage a project’s dependencies. When these pacakages are in place, getting up to speed with another project is a breeze.

The Pain

But how about installing these tools themselves? How about other dependant pieces of software such as databases and the like? What if they have to be compiled from source? What if they have to be installed with a package manager? Which package manager do you then use: Fink, MacPorts, Homebrew?

Not always so easy!

It’s a UNIX system! I know this!

I love UNIX. I’ve worked in the investment banks, telecoms companies and startups of the world building and supporting software on FreeBSD, Solaris, RHEL, Debian and more. For just over a half decade now I’ve been using a Mac to do this. I was reluctant to begin with and I could care less for the fanbois but I now strongly believe that a Mac running OS X is the best UNIX workstation I can get my hands on.

Despite all I like about it, OS X can be pretty hostile towards developers, just ask anyone who has setup a Ruby dev environment on a fresh install of Lion recently or who uses Xcode regularly.

Enter Vagrant and Chef

Vagrant can be used to provide an isolated environment running inside a virtual machine. When combined with a provisioning tool like Chef it’s possible to concisely specify the necessary environment for an application and then have it available in a dependable manner on a on-demand basis.

This combination is to systems what RVM and Bundler are to Ruby interpreters and libraries.

I’d been hearing good things about Vagrant and Chef for some time but what prompted me to delve deeper was the upcoming addition of a junior dev and an intern at Converser. Having all dependencies scripted, including operating systems, programming languages, document and key/value datastores seemed like a good way to get the new starts up and running with the minimum of time and headaches.

An Example

I have an example of what I’ve learned on GitHub. It’s a simple Sinatra app but demonstrates all the moving parts.

Standalone installers for Vagrant, Chef and Git are to be found here, here and here which removes the need for any form of package manager on the host OS X system.

Once everything’s installed and the repo cloned, the following commands will start up the example app within a fresh virtual machine:

vagrant up
vagrant ssh
cd /vagrant
bundle exec rackup

Browse to http://0.0.0.0:8080 to view the output.

For the curious, the versions or Ruby and Bundler can be checked thus:

$ vagrant up
$ vagrant ssh
$ cd /vagrant
$ ruby -v
ruby 1.9.2p320 (2012-04-20 revision 35421) [x86_64-linux]
$ bundle -v
Bundler version 1.1.4

The virtual machine can be powered down and disposed of with this:

vagrant destroy

Should Things Be This Convoluted?

Perhaps the world would be a better place if all this was simpler. Perhaps returning to a Linux workstation would remove some of these headaches. Maybe I’ve just grown accustomed to the pain.

For the time being OS X appears to have the correct balance of desktop software and UNIX-y stuff for my needs. Until something appears that surpasses the form-factor, power and utility of my current setup I’ll continue to pay the Apple-tax.

An iOS client for my Coffee Tracker API

About 9 hours ago I dandered down the road to see what was happening at the FlackNite event being hosted in Farset Labs.

When I finally got myself settled down with network access and a cup of coffee and said hello to everyone it seemed I was the only one without a project to work on. Nightmare.

Decisions, Decisions

I couldn’t think of what to focus on but Rob and Pete were sitting next to me and tinkering with some Objective-C and Cocoa. Considering that I’m now on the iOS developer program, doing something similar seemed like a good idea.

I needed to pick something that could be accomplished within a relatively short period of time and I’ve had a few small projects which I’ve wanted to get out of the way for a while but most of them would have required some research effort to take me into unchartered territory.

This constraint left me with a single task: the relatively straight-forward job of porting the OS X client of my Coffee Tracker API to iOS.

Double Jalapenos

Coffee was drank, pizza was eaten, many laughs were had and some Objective-C was cranked out.

Beyond creating a new iOS project and user interface, the existing code didn’t require much change. Turning off ARC for AFNetworking was the most involved thing I had to do.

After the basic port I added a few finer details like turning on the network activity indicator when the web service was being accessed and making sure the count was refreshed whenever the app became active. Simple.

Dive In

It was good again to sit down with a specific task, open Xcode and eventually come to a solution. Maybe some day I’ll produce something more involved but I’m pleased with how things are progressing.

Full source code is available on GitHub so fill your boots.

An introduction to Sinatra

Tonight I gave an introductory talk about Sinatra at the second meetup of BelfastRuby.

The Converser platform I’ve been building this last while uses a lot of Sinatra so when I was asked to give a talk about using Ruby to develop web apps without Rails it wasn’t hard to think of a subject.

This was my first technical talk and apart from a touch of nerves and forgetting some of the jokes I had in mind I think things went well. Maybe I’ll end up giving another one before too long :)

The slides are available at belfast-ruby-sinatra.herokuapp.com and make use of showoff which itself uses Sinatra. Meta.

The source, including the code samples, is on GitHub.

Update: some photos from the evening are now available on Flickr.

Tracking my coffee consumption with Redis, Sinatra and Cocoa

I’ve often joked about putting together an app to track my coffee consumption, such is my reputation for consuming the black goodness. Like a lot of my other personal projects, the idea had a prolonged gestation period and was finally born through a welcome spark of motivation.

Crafting fine web APIs

Over the past 6 months the bulk of what I’ve been doing day to day with Vigill has involved building web APIs for consumption by mobile clients. This has involved lots of Sinatra, Redis and MongoDB.

In this time I’ve also put together a couple more Cocoa apps.

Thick client boogie

An unexpected resurgance in enthusiam for developing desktop apps combined with a fluency in cranking out webservices put me in a good position to put together a simple API and client.

I’m not a big user of mobile apps but I do spend the bulk of my waking hours sitting in front of a Mac so producing a client for OS X was the logical choice.

For the API I considered using something a bit more esoteric than my standard toolkit, but no matter what combination of technologies I investigated not much seems to come close to the power and flexibilty I find with Ruby and it’s frameworks for performing the bulk of tasks required by the web applications of today.

Areas of note

A valid API key must be sent in the request headers when sending a POST to the Sinatra app. Validity of the key is determined by checking set membership in Redis. A 4xx status code is returned if the key is missing or invalid.

The daily count is boosted with an atomic increment of a hash field.

The client is not much more than a GUI wrapper around some HTTP requests sent using AFNetworking. A previous native iOS client I developed used NSURLRequest directly and I found AFNetworking much simpler to use.

Get the code

The source for the API and the OS X client is available on GitHub as usual and some further technical details are available in the READMEs.

You can keep up to date with my coffee consumption at coffee-tracker.herokuapp.com.

Enjoy.

A simple app to monitor Google Chrome on OS X

I was on-site with Vigill one day before Christmas and I mentioned to Oisin an idea I had for an app.

I was sick of having endless tabs open in Chrome, hogging memory, each one some seeming important enough at the time that I read it but now just a contextless enigma.

If I at least knew how many tabs I had open, it would be a step in the right direction I thought. There’s got to be an app for that, right? Not that I could see. Opportunity knocks.

Enter AppleScript

I knew that AppleScript would likely give me the functionality I was after so that night in the hotel I installed the Ruby version of the easier to use though now abandoned AppScript and was eventually able to get some details out of a running instance of Chrome concerning it’s open tabs.

Over the Christmas holidays I spent an evening playing with the NSAppleScript class and figuring out how to add an item into the OS X status bar. Huzzah!

Give it a name

Naming things and cache invalidation, 2 notoriously hard things. Inspiration soon struck though and I stumbled upon a catchy name for the app which corresponded to an available dot com domain. I didn’t register it straight away but later in a moment of entusiasm I whipped out a card card and starting piecing together a single page site with Bootstrap to give the app somewhere to live online.

Oisin and myself have been using Bootstrap a lot these last few months as we’ve been building out the Vigill platform and it’s truely a godsend for those of use with the natural tendency to produce apps which look like they’ve been designed by a programmer. I recommended it highly.

The Future

To date the app doesn’t do much, it sits in you status bar, displaying the number of open tabs in Chrome. You can click on the number and choose to quit the app. Fits a need but it’s nothing fancy.

Suggestions have included to show how long the oldest tab has been open, how much memory is being used and even an idea to apply some gamification and have an online leaderboard of tab fetishists.

As with all my personal projects I never know if I’ll ever come back to them after the initial buzz of motivation has passed. In the early days of this blog, nearly 6 years ago now, I was hacking on desktop apps and I can feel myself being drawn back to the world of compilers and executables so chances are good I’ll revisit this app in order to sharpen my skills with Objective-C and Cocoa. Suggestions on a postcard please.

Get It

The app is available to download from chromitor.com and the code is available on GitHub, hopefully you’ll find it useful.

My also-ran Markdown Editor for OS X

Quite often I feel the need to learn something new for the sake of learning something new. Covey aficionados would know this as sharpening the saw.

I usually struggle deciding upon something however as there are so many areas to choose from in the software world and it isn’t everyday I feel drawn enough to something to put time and effort into it on top of holding down a day job.

Five years ago it was Rails, two years ago it was testing that I wanted to improve my skills with and I’m now waiting on whatever will be the next major piece of technology or practice I feel I need to push myself towards.

Before the web

When I started out teaching myself to write code I didn’t have access to the Internet and the web as we know it didn’t exist. My output was to evolve from simple CLI programs to games to desktop apps.

These past years a lot of my work life has involved developing web software with scripting languages. Compilers, graphics routines and assembly language have all started to become lost in the mists of time.

Returning to desktop software development seemed like both an interesting challenge and a break from dealing with the web. As I’ve little interest in cross-platform toolkits this meant Objective-C and Cocoa, technologies which I pick up every once in a while but haven’t produced much with. But what exactly to develop?

Eating my own dog food

In May I was working on-site with a client in Dublin and had to produce some documentation for the analytics platform I was building at the time. As usual I chose Markdown and put up with the workflow of editing the content in Vi and running it through the supplied Perl script to see what the HTML, which was eventually to handed over to the client, looked like.

It struck me that here was repeated effort I could automate with software and there didn’t appear to be a widespread, simple Markdown editor for OS X. An opportunity at last to start work on a first offering from my software company I thought.

The possible beginnings of a product-based business

Over the course of a few months I added a bit of code here and an interface tweak there, mostly while staying in hotel rooms in Dublin. Any time I wanted to edit some Markdown I used the app and it suited my purposes fine.

I toyed with the idea of publishing what I had on the Mac App Store. I shared the binary with a couple of people to see what they made of it and was extremely pleased when one asked if it was ok to pass the app on. I could see a possible path opening before me allowing me to start stepping away from a consultancy business model to one centred on selling products.

Beaten to it

Life moved on though, I started a new contract which gave me a lot of new technology to play with and development of this side project was put on hold.

About a month ago I spotted Mou. Not only was it more complete than my effort but it was better designed than I could hope for considering the limitations on my time and Cocoa experience. “Fair play” I thought, I wasn’t quick or decisive enough and as a result, if I’m to build this planned business it will have to be on the back of a different product.

Today I spotted Marked and a quick search turns up a number of other apps and text-editor plugins. Such is life.

Open source

As of today it’s been over 3 months since the last commit to the repo and many superior alternatives exist so I thought I may as well give it away.

The app suits my needs though it would be even better if it could saving/load files from the local filesystem. I may well tinker with it for my own learning purposes but who knows what side-project my interests will wander to next.

The code is available on Github and compiles with Xcode 4.1. Enjoy.