Archive for May, 2009

Simple Couch comet listener with EM

Monday, May 25th, 2009

So couch trunk has now got the long-awaited comet update-tracking functionality, obsoleting pretty much every other way of doing update notification at a stroke. I’ve been looking forward to this for a while – I want to throw an EM daemon or two on the comet URL; they’ll listen for changes and do cache invalidations/search index additions asynchronously. Yes, I could just expire the cache synchronously upon save but that gets very fiddly – I want to store the seq number in the cache so the expiration/update sequence is fully replayable. Doing that synchronously would involve another query to the DB to find the current seq, inviting race conditions – forget it. Also, I need to do message dispatch to diverse clients who might not be using the web server at all; I need all updates to flow through a router, and that can’t be in the web app.

Anyway, here’s a simple EM script which listens to the (charmingly undocumented) comet URL and does whatever you want with the updates. If you were doing anything complex you’d probably want to send the process off into a defer operation.

require 'rubygems'
require 'eventmachine'
require 'socket'
require 'json'
 
module CouchListener
  def initialize sock
     = sock
  end
 
  def receive_data data
    data.each_line do |d|
      next if !d.split("\"").include?("seq")
      puts "raw: #{d.inspect}"
      begin
        json_data = JSON.parse(d)
        puts "JSON: #{json_data.inspect}"
        puts "updated: id #{json_data["id"]}, seq #{json_data["seq"]}"
      rescue Exception => e # TODO definitely do not want to rescue in production
        puts "JSON parse failed with error #{e}" 
      end
    end
  end
 
  def unbind
    EM.next_tick do
      data = .read
    end
  end
end
 
CURRENT_SEQ = "0" # you'll want to replace this with whatever is current
DB_NAME = "test_comet"
 
EM.run{
  $sock = TCPSocket.new('localhost', 5984)
  $sock.write("GET /#{DB_NAME}/_changes?continuous=true&since=#{CURRENT_SEQ} HTTP/1.1\r\n\r\n")
  EM.attach $sock, CouchListener, $sock
}

Downtime

Sunday, May 24th, 2009

Seems the hosts of this server, Softlayer, experienced a power failure in the data centre in which I’m located, subjecting this machine to an unexpected reboot. Now, I should have tested this before – one of the traps with running an OS which basically never needs to be rebooted is that one rarely tests a reboot – but it seems I had failed to set httpd and, worse, named, as chkconfig on!

I don’t know how that slipped my mind. Everything else came back up just fine. I guess I just assumed, at the time, that httpd and named started by default – I don’t remember. Anyway, I really should have checked sometime in the year since this server was last rebooted. Ah well. Anyway, all good now.

In other news, I’m thinking about upgrading this server – it’s getting a bit long in the tooth now and for the same money I can probably get a beefier machine. That in itself might not be worth the hassle but I’m also considering migrating from RHEL to a debian system – perhaps Ubuntu Server, although I’ll try it out locally first. I have no complaints about RHEL5 – it’s an excellent distribution, highly reliable, and I haven’t had a single problem with it. But it seems debian-based linux is better supported in many new projects and I’m sick of feeling like I’m the only one trying to run $interesting_new_software on a RHEL system. Well, let’s see.

In other upcoming changes, I’m going to switch to Ruby 1.9.1 one of these days – just got to clean up one legacy app. I guess I can run 1.9 in parallel, but I’d like to do it all at once. To the future!

Mail.app treating RSS links as attachments

Saturday, May 23rd, 2009

Mail.app likes to treat mere links in an RSS entry as “attachments” to be downloaded as a matter of course. This can be fun when there’s a lot of them, such as in this Ruby Inside post about a recent Rails conference in Scotland. Here’s Mail.app’s RSS version of that post, listing 1.2G of “attachments”:

I think I'll just send a 1.2GB email

Here’s the offending links:

Look at all those attachments!

To make it stop I had to delete the RSS message, set the program on its “quit cycle”, then manually close all attempts to download the damn things until it finally exited:

Playing whack-a-mole with the connections

Not fun. Wish Mail.app had a better way to select which attachments to download. I did nothing, btw, except view the RSS entry.

Slippery Slopes

Saturday, May 23rd, 2009

From a Reddit , in response to an assertion that “Slippery slope is a logical fallacy”:

What’s most noteworthy is that “indefinite detention without trial” is the fucking endpoint of the slope we wanted to stay off of. The whole “slippery slope” argument was about Hoover’s expansion of power; the Supreme Court giving prosecutors any authority they want; the explosion of federal domestic security agencies; the Clinton and Bush administrations tap-dancing outside the Constitution.

We’re now at Y, trying to say “fuck no” to Z, and way past any discussion about slippery fucking slopes.

Well, that pretty much sums it up.

Jeweler for Gems, DaemonKit for daemons

Wednesday, May 20th, 2009

I hate writing and maintaining code which does extremely common things, so I’ve gone through a few rubygem management libraries over the years. Most recently I’m been using a library called Bones – it was OK, I guess. Better than doing it myself.

Recently, however, I’ve come across – a clean, simple solution which really fits my needs. It’s git-aware – any files git knows about, it knows about, and includes effortlessly. It automatically handles binaries, it does version updates, it doesn’t require you to include numerous files of its own – just a short addition to an existing projects’ Rakefile will do the trick. In short, it’s exactly what I want and nothing more, and I’ve adopted it for my gem needs.

Another fantastic RubyGem I’ve stumbled across lately is , which is exactly what the name suggests – a kit for easily generating daemons in Ruby. It’s a little more complex than Jeweler, obviously, but it generates a logical and familiar directory structure – if you’re proficient in Rails you’ll instantly grok what everything is for. Putting together a daemon in Ruby has always been relatively easy, but somewhat time-consuming as you build your own structure with all the little bits you need – I find DaemonKit’s defaults to be pretty much exactly what I want as a base, and you can be up and running in minutes. Plus, since half the daemons I write these days seem to have something to do with AMQP, its explicit catering to this use case is particular welcome.

Combine the two and you’re in gem daemon heaven. Check them out.