Minder – a simple web-app monitoring tool written in Ruby

Background

A few months ago I was tasked with migrating and maintaining a bunch of legacy Rails apps and a couple of them were misbehaving. Requests were hanging which was tying-up the front-end Apache child processes and resulting in all the web-apps on the server becoming unresponsive

At the time I didn’t know what was causing the hanging requests, it wasn’t happening in a predictable manner and on the surface I had all the apps dependencies in place.

Seeing as this was a job I was doing on the side I had very limited time to investigate the underlying issue and decided I would have to make do with just knowing when the apps went down so I could ssh in and manually kill off the offending processes.

My initial thought was to use Nagios for monitoring but it seemed excessively heavy-weight for the task at hand so I seized the opportunity to quickly develop a tool perfectly tailored to my needs.

The resulting script, minder, checks a list of domains and if the root of each can’t be read within 10 seconds a notication is sent via XMPP. Nothing more, nothing less. Simple.

Minder proved effective and I was bombarded with instant messages whenever any of the apps went under. Thankfully I found to bit more time to investigate more thoroughly and tracked the issue down to a missing TrueType font used by ImageMagick.

The first iteration of minder was very simple but I thought I’d put a bit of extra effort in to make it easier to customise and share it with the internet-at-large, hopefully someone will get some use out of it.

Prerequisties

  • Two XMPP accounts:

    • one to send notifications from, and
    • a second to receive notifications

    Google Talk or any regular Jabber/XMPP accounts will suffice

  • Blaine‘s xmpp4r-simple gem must be installed:

    sudo gem install xmpp4r-simple

Usage

  • copy minder.yaml.sample to minder.yaml
  • edit minder.yaml to specify the username and password of the sending XMPP account
  • specify your own personal XMPP account (xmpp_to)
  • modify the list of domains to monitor
  • run the script
    $ ./minder.rb
  • Optionally run the script from a Cron job, the following will run every hour:

    0 * * * * /path/to/minder.rb > /dev/null 2>&1

Get the code

Clone the repo from GitHub. Fire any queries to @stevebiscuit.

Is it raining in Belfast? Redux

Last year I created a nano-app which attempted to answer the question, is it raining in Belfast?

I was never completely pleased with it but the objective was to get something done and get it done fast, which was achieved.

This weekend I’ve been tinkering with another project and hit a bit of a stumbling block getting various pieces of software to co-exist.

I decided a break from the problem would be beneficial but I still wanted to have some sense of achievement from the weekend and redeveloping this app seemed a good idea.

Some quick research showed that the Yahoo! weather report for Belfast could provide the necessary information so I hacked up an RSS processing script and converted the old PHP front-end to a quick and simple Sinatra app.

The app now operates as originally envisioned: it answers the question with a straightforward yes or no. Hopefully it will be accurate as well :)

The app is hosted at isitraininginbelfast.com and the code is on GitHub

An experimental lifestream app

Some time ago DW and myself had an enthusiastic conversation about logging various aspects of our electronic lives: emails sent and received, RSS feeds read, incoming and outgoing phone calls, the list goes on.

At that time I’d already started tinkering with apps to track my efforts in the gym and changes in my bodyweight, I’ve used notebooks for this for years and thought it’d be interesting to have this information in electronic form.

Sharpening skills

While I was on the bench waiting for my current contract to start I was sharpening my skills through developing some small projects with the technologies I’d be using. One of those technologies was SQLite which I knew of but didn’t have much hands on experience with.

I started by playing with the Ruby interface to SQLite, using it to insert and retrieve data from a simple database. I’ve since grown to love the simplicity and flexibility of SQLite and will no doubt use it again in the future.

I needed a practical application and thought back to my intentions to track my online activities. Now was as good a time as any to to tick this off the list of potential projects.

When I’d glued together some Ruby to parse the RSS feed of this blog I knew the basics were in place so I started on a simple Sinatra app to act as a web interface.

Stumbling block

I scoured the web for design inspiration and got a good start to the visual aspect of this project but couldn’t make my mind up over a few subtle items. Next thing my agent called: the purchase order finally came through and my contract was to start the next day.

The demands of adjusting to a new work environment meant I felt little motivation to work on anything on my own time and I became more interested in funneling my excess income into the stockmarket. The project joined those other half-finished unfortunates.

A fresh jolt of motivation

After a couple of months on the contract I started feeling a renewed motivation to get something of my own out there. I couldn’t bare the thought of starting another project to not finish it so I set out to to whip this latest app into shape and release it as soon as possible and no later.

I dug through the uncommited changes in my working copy and made some arbitrary decisions which resulted in a design I felt was “good enough.” I modified my RSS parsing script to feed into SQLite and extended it to process my Delicious bookmarks and loved tracks on Last.fm. I created pagination links, wrote a README and sorted out the hosting and cron jobs. Bingo.

Check it out

My original intention was to track much more of my online life and there’s some details I’d like to change but I love releasing code and I’m pleased with this first iteration.

The app is hosted at life.stevenwilkin.com and the code is available on GitHub, I hope you like it.

Rails 3 Hello World

The past while I’ve been busy providing technical consultancy to BT and haven’t had much drive to work on anything on my own time. The itch has returned the past couple of weeks though so I thought I’d see what I’ve been missing in the Rails world and in what better way than getting a basic Rails 3 app up and running.

My environment was already setup for Rails 2.3.* and Yehuda Katz’ post served as a guide to get me up to speed with the beta loveliness.

RubyGems 1.3.7 along with Thor and Bundler gems required

My installed version of RubyGems was a couple of point releases behind, so I updated that and installed the necessary gems

sudo gem update --system
sudo gem install thor bundler

Clone Edge Rails from GitHub

mkdir -p ~/code/rails/rails
git clone http://github.com/rails/rails.git ~/code/rails/rails

Generate a fresh app and install dependencies with Bundler

mkdir ~/code/rails/rails-3-demo
cd !$
ruby ~/code/rails/rails/bin/rails new . --dev
bundle install

Launch the web server

./script/rails server

Browse to http://0.0.0.0:3000 and you’re done!

Home time revisited – redeveloping a CakePHP application with Sinatra

I’ve just released the latest iteration of hometi.me, a little nano-app I’ve mentioned before.

Deliberate Practice

It’s only a trivial app but I’ve a bit of free time before the start of a contract so I thought it would be good practice to redevelop it, a task I’d been ignoring for a long time.

Goodbye CakePHP, Hello Sinatra

The original app was put together quickly with CakePHP, a PHP-based framework I’ve used since my first paid programming gig. Since then though my interest and proficiency with Ruby and it’s frameworks has come more to the forefront. By choosing Sinatra, the app was given access to the rich Ruby ecosystem, including the joyful HAML and SASS.

The interesting aspect of this release is that all the logic is now implemented on the client-side using Javascript. Originally the countdown was calculated on the server-side and passed to the client with an AJAX call, but the server is more-or-less just hosting the markup, stylesheets and Javascript files required to get the app up and running in the browser.

Heroku

As this app doesn’t make use of a database or Cron jobs etc I thought I’d take the opportunity to move it from my own infrastructure and onto the awesomeness of Heroku whose free offering is a lot more performant than my little VPS. Heroku is a service I’ve been playing with for a while now and it makes deploying a Ruby web app into production a breeze, assuming your project can work within it’s limitations.

All that’s left now is for the DNS changes to propagate. I’d like to show a bit more love to this app in the future, the controls to specify your own home time could be more intuitive for instance, but for now I’m content with the improvements I’ve made to it and it’s new home.

Is it passed your home time yet?

Mashing up Twitter Trends and Google Images

I recently released the latest entry in my series of nano-apps.

As with my previous efforts, istrending.com serves serves no real purpose other than as an act of deliberate practice: going through the process of coming up with an idea, writing code, registering domain names, setting up Cron jobs and configuring deployment systems to get it all up and running in a production environment.

I questioned why I was even bothering with an app that was likely not going to see any traffic or be of use to people and when I hit a minor javascript problem the project was shelved for a few months. I recently had a fresh burst of enthusiasm though and got the usual sense of achievement when I deemed the code “ready enough” and a quick cap deploy took care of the rest.

The main problem I’ve faced with this app concerns the script used to update the list of trending topics: the Twitter search api has a known issue of returning HTML instead of JSON, which has on occasion caused my inbox to be filled with emails from the Cron daemon alerting me to failed jobs. I’m also not very happy with how the javascript carousel functions, so there’s some scope for improvement there.

The code is available on GitHub as usual.

Next up is probably something bigger, very likely involving Rails. I’ve been reading up on the various testing methodologies used in the Ruby world and have written my first set of RSpec tests so I hoping my next release will at least have some test coverage. Other than that I’ll just have to see what ideas come to mind between now and then.

Any suggestions on what I should do next?

Because everyone needs their own URL shortening service

At the moment I’m in between paid work and have been using my time to generally enjoy life but to also improve my skills. I’m all about the skills.

As an act of deliberate practice to hone my skills with web-application development and to deepen my understanding of the Rails framework I’ve been adding to my collection of nano-apps.

A while back I had an idea to develop a URL shortener just to see how little code would be required but decided the world really didn’t need another one. With time on my hands over Easter & the curiosity of a technology enthusiast I just started hacking and had a functioning prototype within a short time-frame.

I wasn’t keen on parting with my shiny pennies for a domain name considering my lack of income but I’d written the code and thought I’d may as well go the whole hog and get my work into the wild.

So, without further ado, I present io.gd . As usual, code is available on GitHub.

An experiment with Sinatra, HAML and Blueprint CSS

In my last post I considered transforming stevenwilkin.com into a Sinatra app.

Well, I did it, although I’m only mentioning it now.

Not only was this my first experience with Sinatra but I also decided to try using HAML for the markup and Blueprint CSS to help with the styling. If I’m going to play with a new technology why not play with a few new technologies?

I found Sinatra quite straight forward: you match up a HTTP verb (eg ‘GET’) and a url (eg ‘/’) with a block of Ruby and the results can be sent directly to the browser or an optional view template can be rendered. Simple!

Instead of a Model-View-Controller pattern like many of us are accustomed to, Sinatra provides more of a Controller-View setup which I believe would make it more suited to smaller apps which don’t necessarily make use of a database or where a full-stack framework like Rails would be overkill. If I create any more nano-apps in the future I’ll more than likely use it again.

What really struck me during this experiment and made the whole thing worthwhile was HAML. The simplicity and clarity of it’s syntax was super refreshing and meant I had to do a lot less typing and didn’t have to remember to close divs etc, never mind it being white-space sensitive! All Ruby-based web development I’ve done since has used it.

I wasn’t fussed on Blueprint. I found having a grid background during the *ahem* design phase to be great but I probably didn’t make enough use of it’s features for it to really shine through. In terms of styling and CSS I didn’t take much time to look at SASS and didn’t use it, but I’ve since reread the documentation and may well make use of it in the future.

The app was deployed onto Passenger, like my previous experiments. It took a small bit of google-fu to get the Capfile and Rack configuration tweaked correctly but once this was done pushing the code into production with Capistrano was a breeze. How did I ever survive without Capistrano?

The deeper I get into Ruby the cruder PHP seems, not to mention C#. The future seems exciting.

First production Rails app

I released my first production Rails application, hugagoth.com, last night.

I first started playing with Rails over 2 years ago and it has taken me until now to take an app through from conception to initial deployment. And an interesting journey it has been.

When I first started investigating Rails I had never used a web framework before and the Ruby language really confused me, but there were a few things I liked. I liked the idea of convention over configuration, opinionated software and the amazing community that has built up around this set of technologies.

Not long after this, I started work on another project and considered using it as the motivation to fully get to grips with Rails, but getting the job done was more important so I headed down the PHP route, having had a bit of experience with it over the years. I’m amazed that I now have 2 years of CakePHP development experience. It gets a bad rep concerning it’s performance speed but if I’m doing any bespoke PHP work, Cake is never far away.

Deploying Rails apps a couple of years ago seemed like quite an involved process, what with application servers, web servers, proxies and clusters but now with Passenger, getting a production environment up and running is a breeze. I’ve also developed a great love for Capistrano, using it with non-Rails apps.

Calling hugagoth.com my first Rails app in the wild is a bit of a lie. When I discovered Passenger I wanted to see just how easy deployment now was so I converted my static, single-page professional site into a Rails app. A bit overkill I know as it doesn’t even do anything, but a worthwhile experiment none-the-less.

It’s unlikely I’ll ever use Rails in my current day job so I’m unsure what I’ll be doing with this framework in the future. I’ll probably convert stevenwilkin.com to run on Sinatra just for kicks and if any interesting ideas come to mind, you never know!