RabbitMQ on RHEL5 x86_64

June 25th, 2009 by 深町晶

Want to install RabbitMQ on RHEL5 but running into hideous erlang crashes? Chances are it is an issue with dual installations of erlang, one i386 and one x86_64.

Here’s how to fix it:

$ yum erase erlang.i386
$ rm /usr/bin/erl
$ rm /usr/bin/erlc
$ rm /usr/bin/escript
$ ln -s /usr/lib64/erlang/bin/erl /usr/bin/erl
$ ln -s /usr/lib64/erlang/bin/erlc /usr/bin/erlc
$ ln -s /usr/lib64/erlang/bin/escript /usr/bin/escript
$ service start rabbitmq-server
> Starting rabbitmq-server: SUCCESS

Let’s test:

bash-3.2$ irb
>> require 'bunny'
=> true
>> b = Bunny.new
=> #<Bunny::Client:0x2b04beb0b038 @port=5672, @spec="08", @vhost="/", @status=:not_connected, @host="localhost", @pass="guest", @logging=false, @user="guest", @logger=nil, @insist=nil, @logfile=nil>
>> b.start
=> :connected
>> exec "echo YOU WIN"
YOU WIN
bash-3.2$

And as always, remember:

chkconfig rabbitmq-server on

I’m paranoid about chkconfig now!

Switched to PowerDNS

June 25th, 2009 by 深町晶

I’ve now switched to PowerDNS, running on a MySQL backend, and controllable using the pretty-damn-good PowerDNS On Rails.

Reasons for switching:

  • I am sick and fucking tired of editing and backing up countless arcane zone files. It might be “hard core” and “old school” but it’s also annoying as hell, and I can never remember the syntax or forget something stupid like updating the serial, or whatever.
  • A database is the natural storage container for numerous data files of the same syntax and consistent references, and is far more convenient. I can take snapshots of my entire DNS setup, one file, ready to import at a moment’s notice elsewhere.
  • Furthermore, a database is amenable to controlling with a web interface, which is far more convenient (and less prone to error) than ssh’ing in and editing files manually
  • I have one less thing to back up (actually, about 30 less things)
  • BIND has a terrible security record
  • PowerDNS has some neat other features such as dynamic lookup that I am eager to start playing with.

I’ve been considering this for a while, decided to make the jump today. First I got it set up and running perfectly on an unprivileged port; then turned off BIND and turned on PowerDNS. Total downtime: under 5 seconds.

Here are the steps to install on RHEL5:

$ yum install pdns pdns-backend-mysql pdns-backend-pipe pdns-backend-sqlite
# create a mysql user & db "powerdns"
# edit /etc/pdns/pdns.conf to increase port, add gmysql backend and add db credentials
# create the dir to put powerdns-on-rails in, go to it
$ git clone git://github.com/kennethkalmer/powerdns-on-rails.git
# enter DB creds into config/database.yml and create the DB
$ rake db:migrate
$ rake db:seed
# import zone files
# note: you may need to temporarily cp them to /var/named/
$ zone2sql --named-conf=/var/named/chroot/etc/named.conf --gmysql > zones.sql
# import the sql into MySQL via whatever means customary
# start and test powerdns at this point, see the docs
# then revert the port
# add the web app in Apache or whatever you use, log in, bingo
$ service named stop
$ service pdns start
$ chkconfig named off
$ chkconfig pdns on # make sure mysqld is on too ...

There are some more tips here if you want to have a look.

Compiling PowerDNS on OSX 10.5.7

June 24th, 2009 by 深町晶

Want to play with PowerDNS? On a Mac? You’ll need to do something like this. Make sure you read all the way to the end before continuing.

1. Download boost from sourceforge. The link for the version I used, 1.39.0, is here.

2. Extract the file somewhere. Mine was just in downloads.

3. Download and extract the source (at the top) of PowerDNS. I used version 2.9.22, from here.

4. navigate to the PowerDNS directory and execute the following commands. Change the reference to MySQL to something relevant to your local copy:

$ CXXFLAGS=-I/Users/sho/Downloads/boost_1_39_0 ./configure --with-mysql-includes=/usr/local/mysql-5.0.45-osx10.4-i686/include
$ make
$ sudo make install

That’s it! Built and installed perfectly. Now run pdns_server, and watch it fail with Cannot create semaphore: Function not implemented, because OSX is not fully POSIX compliant. Now wasn’t that a great way to waste time? I certainly thought so. Told you to read all the way to the end.

Fun with Growl

June 24th, 2009 by 深町晶
(1..100).each {|num| Growl.new(:message => "I am number #{num}!").run}

Change the number to 1000, as I did, and don’t plan on seeing your desktop for a while. Add :sticky => true for extra “fun”.

What’s holding open that port on OSX

June 23rd, 2009 by 深町晶

Since I always, ALWAYS forget this, here’s the command to find the process which is holding open, say, port 1900 on osx:

$ lsof -i -P | grep 1900
uTorrent  50937 sho   13u  IPv4  0xaf49b10      0t0    UDP 10.1.1.8:1900

Fucking uTorrent, why the hell is it sitting on 1900? That’s the SSDP/UPnP server-side port, ie. the router should be listening on 1900. Unless it’s planning on advertising SSDP services it can provide, it doesn’t need that port. Wonder what it’s doing.

Oh btw it only shows your processes, sudo if you wanna see ‘em all.

Snow Leopard only has Ruby 1.8.7

June 22nd, 2009 by 深町晶

A bit of a disappointment - I’d been hoping they’d go with 1.9.1.

Recent Recommended Reading

June 15th, 2009 by 深町晶

Why I Joined The Three Hundred

That Alien Message

The Fable of the Dragon-Tyrant

Twitter was useful just then

June 10th, 2009 by 深町晶

Wow, I finally found a use for Twitter. RubyForge was (is) down once again, and a search confirmed I am not the only person experiencing the problem, perhaps saving me a few minutes of tinkering.

I don’t mind this usage pattern; people “tweeting” about disasters, downtime or other anomalous events is useful. It’s the everyday crap and people using the service as a particularly crappy RSS feed that I don’t like.

Anyway, RubyForge is crtical infrastructure in the Ruby world and shouldn’t be going down like this. Hope they make higher availability a priority. Imagine if you were doing an emergency server reinstall right now. Who knows how long you’d have to wait?

UPDATE: It’s been down over 7 hours now, a really serious failure. Thinking about creating my own mirror when it comes back up, wonder what the storage requirement would be?

Keynote reactions

June 9th, 2009 by 深町晶

Yesterday’s WWDC keynote was a welcome return to form for APPL - especially since the ARAT penalty for having the gall to live in Australia has dropped to “only” 10-15% or so. The new laptops look fantastic and it’s good timing; I’ll need a new one pretty soon. Needless to say, my single favourite features is the support for 8GB of memory. Sounded like I’m far from alone, too - the audience practically gave Schiller a standing ovation when he mentioned that.

Snow Leopard continues to impress and I’m particularly interested in GCD, and if/how it can be exploited in everyday applications. If it can be of use in “normal” tasks such as, say, compressing a video, or rendering a 3D scene, the next obvious question is - can the OSX machine be augmented by something like nVidia’s Tesla, to create the most powerful nearly-general-purpose machine available?

Some other features in snow leopard that I’m looking forward to include the new cocoa finder (FINALLY), the chinese character input mode, the bettter file sharing and of course the general “snappiness” improvements. One long-overdue feature which I am less hopeful for is the Quicktime “reboot”; I feel this is too little too late, at least in terms of the new “revolutionary” HTTP streaming, which of course is what Flash et al have been doing for years. HTML5 is imminent and I can’t help but think it will be the victor in any upcoming internet media reshuffle; open standards win, or should win, every time. If Apple really wanted to “take back the web” a complete open sourcing of the entire platform would be the only real option at this stage, especially with all the new linux netbooks and what not. Also no word on Windows (and Linux!) releases of even the updated client software; there’s little use in new internet technology if only Macs can use it.

Last but not least - the Exchange support, which is probably the single feature most likely to boost mac sales out of the whole ensemble. Many mac users and open source devotees spurn Exchange, and I can understand why, but having worked in corporate IT it’s hard to overstate just how important it is. Exchange is the heart and soul of any “Microsoft Shop”, and even many non-MS companies use it - it’s won the “corporate messaging, addresses and meetings” war pretty decisively, and honestly recent versions are not bad for what they do. Getting that into the OS opens the door to OSX usage in literally tens of millions of businesses - running Outlook or such junk in a VM was never a serious option - and I predict will do more for the platform’s marketshare than all the other enhancements put together.

Anyway, much to be excited about, both hardware and software, and not even much profiteering to be found. I’m starting to like Apple in “recession mode”!

Simple Couch comet listener with EM

May 25th, 2009 by 深町晶

So couch trunk has now got the long-awaited comet update-tracking functionality, obsoleting pretty much every other way of doing update notification at a stroke. I’ve been looking forward to this for a while - I want to throw an EM daemon or two on the comet URL; they’ll listen for changes and do cache invalidations/search index additions asynchronously. Yes, I could just expire the cache synchronously upon save but that gets very fiddly - I want to store the seq number in the cache so the expiration/update sequence is fully replayable. Doing that synchronously would involve another query to the DB to find the current seq, inviting race conditions - forget it. Also, I need to do message dispatch to diverse clients who might not be using the web server at all; I need all updates to flow through a router, and that can’t be in the web app.

Anyway, here’s a simple EM script which listens to the (charmingly undocumented) comet URL and does whatever you want with the updates. If you were doing anything complex you’d probably want to send the process off into a defer operation.

require 'rubygems'
require 'eventmachine'
require 'socket'
require 'json'
 
module CouchListener
  def initialize sock
    @sock = sock
  end
 
  def receive_data data
    data.each_line do |d|
      next if !d.split("\"").include?("seq")
      puts "raw: #{d.inspect}"
      begin
        json_data = JSON.parse(d)
        puts "JSON: #{json_data.inspect}"
        puts "updated: id #{json_data["id"]}, seq #{json_data["seq"]}"
      rescue Exception => e # TODO definitely do not want to rescue in production
        puts "JSON parse failed with error #{e}" 
      end
    end
  end
 
  def unbind
    EM.next_tick do
      data = @sock.read
    end
  end
end
 
CURRENT_SEQ = "0" # you'll want to replace this with whatever is current
DB_NAME = "test_comet"
 
EM.run{
  $sock = TCPSocket.new('localhost', 5984)
  $sock.write("GET /#{DB_NAME}/_changes?continuous=true&since=#{CURRENT_SEQ} HTTP/1.1\r\n\r\n")
  EM.attach $sock, CouchListener, $sock
}

Downtime

May 24th, 2009 by 深町晶

Seems the hosts of this server, Softlayer, experienced a power failure in the data centre in which I’m located, subjecting this machine to an unexpected reboot. Now, I should have tested this before - one of the traps with running an OS which basically never needs to be rebooted is that one rarely tests a reboot - but it seems I had failed to set httpd and, worse, named, as chkconfig on!

I don’t know how that slipped my mind. Everything else came back up just fine. I guess I just assumed, at the time, that httpd and named started by default - I don’t remember. Anyway, I really should have checked sometime in the year since this server was last rebooted. Ah well. Anyway, all good now.

In other news, I’m thinking about upgrading this server - it’s getting a bit long in the tooth now and for the same money I can probably get a beefier machine. That in itself might not be worth the hassle but I’m also considering migrating from RHEL to a debian system - perhaps Ubuntu Server, although I’ll try it out locally first. I have no complaints about RHEL5 - it’s an excellent distribution, highly reliable, and I haven’t had a single problem with it. But it seems debian-based linux is better supported in many new projects and I’m sick of feeling like I’m the only one trying to run $interesting_new_software on a RHEL system. Well, let’s see.

In other upcoming changes, I’m going to switch to Ruby 1.9.1 one of these days - just got to clean up one legacy app. I guess I can run 1.9 in parallel, but I’d like to do it all at once. To the future!

Mail.app treating RSS links as attachments

May 23rd, 2009 by 深町晶

Mail.app likes to treat mere links in an RSS entry as “attachments” to be downloaded as a matter of course. This can be fun when there’s a lot of them, such as in this Ruby Inside post about a recent Rails conference in Scotland. Here’s Mail.app’s RSS version of that post, listing 1.2G of “attachments”:

I think I'll just send a 1.2GB email

Here’s the offending links:

Look at all those attachments!

To make it stop I had to delete the RSS message, set the program on its “quit cycle”, then manually close all attempts to download the damn things until it finally exited:

Playing whack-a-mole with the connections

Not fun. Wish Mail.app had a better way to select which attachments to download. I did nothing, btw, except view the RSS entry.

Slippery Slopes

May 23rd, 2009 by 深町晶

From a Reddit comment, in response to an assertion that “Slippery slope is a logical fallacy”:

What’s most noteworthy is that “indefinite detention without trial” is the fucking endpoint of the slope we wanted to stay off of. The whole “slippery slope” argument was about Hoover’s expansion of power; the Supreme Court giving prosecutors any authority they want; the explosion of federal domestic security agencies; the Clinton and Bush administrations tap-dancing outside the Constitution.

We’re now at Y, trying to say “fuck no” to Z, and way past any discussion about slippery fucking slopes.

Well, that pretty much sums it up.

Jeweler for Gems, DaemonKit for daemons

May 20th, 2009 by 深町晶

I hate writing and maintaining code which does extremely common things, so I’ve gone through a few rubygem management libraries over the years. Most recently I’m been using a library called Bones - it was OK, I guess. Better than doing it myself.

Recently, however, I’ve come across Jeweler - a clean, simple solution which really fits my needs. It’s git-aware - any files git knows about, it knows about, and includes effortlessly. It automatically handles binaries, it does version updates, it doesn’t require you to include numerous files of its own - just a short addition to an existing projects’ Rakefile will do the trick. In short, it’s exactly what I want and nothing more, and I’ve adopted it for my gem needs.

Another fantastic RubyGem I’ve stumbled across lately is DaemonKit, which is exactly what the name suggests - a kit for easily generating daemons in Ruby. It’s a little more complex than Jeweler, obviously, but it generates a logical and familiar directory structure - if you’re proficient in Rails you’ll instantly grok what everything is for. Putting together a daemon in Ruby has always been relatively easy, but somewhat time-consuming as you build your own structure with all the little bits you need - I find DaemonKit’s defaults to be pretty much exactly what I want as a base, and you can be up and running in minutes. Plus, since half the daemons I write these days seem to have something to do with AMQP, its explicit catering to this use case is particular welcome.

Combine the two and you’re in gem daemon heaven. Check them out.

Rails is a ghetto all right

April 28th, 2009 by 深町晶

Remember Zed Shaw’s “Rails is a Ghetto”? All about how the Rails community is being taken over by boring, pretentious, corporate wanna-be suits with their prattle about “professionalism”, pretentious posturing, MBA peacocking and assorted other behaviours unwanted in the dynamic and creative Ruby community.

Well, these MSCE-having douchebags are out in full force today. You see, there was this little Ruby conference in San Fran, and a presenter there had the temerity to include - wait for it - references to porn in his talk. Plus some pictures of scantily clad girls.

OH NOES

Now, I thought the talk was fine, and very well made. The content may have been slightly risqué, in a US context anyway, but no big deal. Actually I appreciate the creativity - there’s nothing more boring than yet another “professional” presentation. I use Ruby because I hate acting “professional”.

But oh boy, the reaction.

Some commentors were frankly laughable - one woman in particular went so far to claim the presentation made her fear for her safety, as if the entire body of men in the audience were in danger of rising as one and assaulting her in some kind of insane porn-fueled gang rape:

To most of these men around me, I am, at best, an oddity, and at worst, a sexual target. I feel a little less safe.

I suggest this poor woman seek therapy for these delusions.

Other reactions include pathetic “I am being victimised” attention-seeking, lame attempts at demonstrating how much “I truly care about women” etc, hilarious “I am leaving the Ruby community and re-installing Visual Studio” threats (please do!), and every combination thereof. I cannot help but think that if Matt’s presentation has the effect of getting rid of these disingenuous wowsers then he should henceforth be invited, nay required, to present at every Rails conference.

Comments from the SlideShare page from sad Java programmers:

I’ve talked with two large IT shops who’ve said that there is no way they’ll ever let Ruby in their companies because of this horse shit.

Lol, good. The less “large IT shops” pissing in the Ruby pond the better. And if their management is so stupid as to choose technology because of “some presentation given by some guy at some conference” then no doubt they’ll be out of business soon anyway.

I think Matt just blew the image of the Ruby community by doing something like this.

WTF? Who even thinks like this? Whatever, I am glad they do:

I’ll avoid joining the Ruby/Rails community at all. So will many other engineers, given the remarkable and continual display of unprofessionalism from you, the community, and even its leaders

Uh, OK. Door’s that way. By the way, you’re not a real engineer, did you know?

Is this the ruby community , sexist and racist? I am going back to .net and FU all!

Lol, enjoy! Wait, racist?

And so on and so on. Pretty funny. Thanks for the GC patch, Matt, it’s proving very effective.

About the only person whose reaction to this I truly admire is DHH himself’s. And he is the father of Rails, so if he doesn’t mind it, then all these “this is inappropriate for a professional conference blah blah” business-card-offering synergystic-opportunity-seeking suit-wearing jerks can go jump off a cliff, IMO.

Best troll ever

April 14th, 2009 by 深町晶

I think this is one of the best trolls I have yet seen.

It absolutely boggles the mind that anyone would go to that kind of effort and expense to score some ridiculous, personal point, some imagined grievance, some silly grudge. Trolling is not really for me but geeze - I’ve got to hand it to the guy.

JS Changeover

April 13th, 2009 by 深町晶

jquery

Goodbye Prototype, hello jQuery. Spent all weekend learning it For Realz™ and sorting out the various tricks needed to make it play nicely with Rails - looking forward to doing a lot more JS from now on.

There are only 17,814 available three letter Twitter names left!

April 8th, 2009 by 深町晶

Just when you thought domain squatters could not possibly get any lower - a new service allows you to search and register Twitter names:

Extra Extra!

Time is running out to register a great Twitter name!

With about 10 million people already using Twitter and thousands more signing up every day, the number of great Twitter names is shrinking rapidly.

Don’t delay: Before you know it, all the good names will be taken and gone for good.

Needless to say, the value I place on a “Twitter name” is approximately zero. However, it’s still annoying to see the lengths people will go to to screw up everything for the rest of us. It’s difficult to see any legitimate reason why someone might want to search for names merely by length; if they have a name to “defend”, surely they’d know it already, and the length wouldn’t be an issue. No - this is for domain squatting scum only, now graduating to the hilarious, worthless hype of Twitter.

If I were Twitter, I’d cancel every single account coming from that site. In fact, I’d block their entire IP range to keep them from even finding these precious gems.

Contemptible. Anyway, everyone knows the best 3-letter Twitter name was taken long ago.

New FTTH network for Australia - awesome!

April 8th, 2009 by 深町晶

The old proposal to build a partial (FTTN) broadband network for Australia has been dumped, replaced by a much larger, better plan - a $43b fibre to the home network, to bring 100Mbps network access to 90% of Australian homes. Here is the announcement.

That’s $2,000 for every man, woman and child in the country. That’s a huge amount. To put it into perspective, the same expenditure in the USA would be roughly $600b.

Putting aside the fact that this is desperately needed in AU’s miserable internet market - this is exactly what a government should be doing at a time like this. Spending huge amounts on wisely chosen infrastructure projects is just the ticket to pull a country out of recession, IMO.

And needless to say, this is the beginning of the end for that hated monopoly Telstra, who will be lefting holding nothing but their crappy 50-year-old legacy copper POTS.

Fantastic.

Australian vs. Japanese Internet - the numbers begin to match up

April 4th, 2009 by 深町晶

When I lived in Japan, I had VDSL2 internet access, unlimited of course, in practice around 40Mbps up/down, for around 4,000 yens a month. So imagine my delight when I discovered a very similar deal in Australia:

ESHDSL-40M-Unlimited $4,000/m 40 Mbps / 40 Mbps Unlimited

That’s right: $4,000/m! What a deal! That’s numerically almost identical to the 4,000 yens a month in JP.

Now who said Australian internet sucked? Look at the numbers, baby!