Sho Fukamachi Online

Here’s my initial stab at a Rails Session model for CouchDB. The marshalling stuff is taken from the example SQLBypass class in the ActiveRecord code.

You’ll need a recent CouchRest and trunk CouchDB, probably.

class CouchSession < Hash
  @@session_db = CouchRest.database!('http://localhost:5984/sessions')
 
  attr_writer :data
 
  def self.find_by_session_id(session_id)
    self.new(@@session_db.get(session_id))
    rescue
    self.new(:id => session_id)
  end
 
  def self.marshal(data)   ActiveSupport::Base64.encode64(Marshal.dump(data)) if data end
  def self.unmarshal(data) Marshal.load(ActiveSupport::Base64.decode64(data)) if data end
 
  def initialize(attributes = {})
    self['_id'] = attributes['_id'] ||= attributes[:id]
    self['marshaled_data'] = attributes['marshaled_data'] ||= attributes[:marshalled_data]
    self['_rev'] = attributes['_rev'] if attributes['_rev']
  end
 
  def data
    unless @data
      if self['marshaled_data']
        @data, @marshaled_data = self.class.unmarshal(self['marshaled_data']) || {}, nil
      else
        @data = {}
      end
    end
    @data
  end
 
  def loaded?
    !! @data
  end
 
  def session_id
    self['_id']
  end
 
  def save
    self['marshaled_data'] = self.class.marshal(data)
    self['data'] = data
    self['updated_at'] = Time.now
    save_record = @@session_db.save(self)
    self['_rev'] = save_record['rev']
  end
 
  def destroy
    @@session_db.delete(self['_id'])
  end
end

Nice and short - possibly the shortest Rails session class I have seen. The beauty of CouchRest/CouchDB! And we descend from hash so we can just save the object straight - after marshalling, of course. Cool, huh?

Note that I am actually writing the raw data as well as the marshalled data into the saved doc, for troubleshooting/interest purposes. Feel free to remove that.

Not pretty, but it works. Just save it like a normal model. You’ll need to put these into environment.rb:

config.action_controller.session_store = :active_record_store
CGI::Session::ActiveRecordStore.session_class = CouchSession

Note also that I have ignored any differentiation between the record ID and the session ID, negating the need for any special overrides in ApplicationController. However, the session IDs Rails generates are large and you might find them unattractive in CouchDB - it would be fairly simple to separate them, but then you’d need a new map view and an override. I feel it’s simpler to just use the Session ID as the doc ID and damn the torpedoes. YMMV.

Improvements? See something wrong with it? Let me know!

I think this is the only way to do it.

>> Time.now.strftime("%Y/%m/%d %H:%M:%S.#{Time.now.usec} %z")
=> "2008/08/20 01:22:28.367899 +0000"

At the microsecond level, it’s possible for Time.now to change mid-evaluation, so if you *really* care about timing you could freeze the time object first and read from that:

>> now = Time.now
=> Wed Aug 20 01:40:26 +0000 2008
>> now.strftime("%Y/%m/%d %H:%M:%S.#{now.usec} %z %Z")
=> "2008/08/20 01:40:26.597940 +0000 UTC"

Happily, Time.parse will read that straight out of the box:

>> n = now.strftime("%Y/%m/%d %H:%M:%S.#{now.usec} %z %Z")
=> "2008/08/20 01:40:26.597940 +0000 UTC"
>> Time.parse(n)
=> Wed Aug 20 01:40:26 +0000 2008
>> Time.parse(n).usec
=> 597940

Another in my series of “lessons I’ve learnt as I progress in my journey from rank amateur to serious developer”.

Some time ago I posted a series of articles discussing the use of “advanced” database techniques with Rails, mostly utilising Postgres. See here:

Switching to PostgreSQL
Strict databases are great for discipline
hacking native UUID support into Schema:Dump
UUIDs in Rails redux

Anyone visiting those old articles these days will notice a new update: don’t do this. Why, you might ask? Since I thought it was such a good idea at the time.

Turns out that being “clever” with your DB setup is almost certainly more trouble than it’s worth.

Sure, using PGSQL schemas to share tables between your applications seems like a nice, elegant solution compared to throwing everything in one DB and using table namespacing to distinsguish between them. Sure, using the schemas saves you from having to define table names in every model where you access a “foreign” table. Unfortunately, it’s a giant pain in the ass to maintain and constricts your use of migrations.

Sure, hacking Rails to read and write PGSQL’s native UUID format is elegant and looks kewl in your migrations, etc. However, maintaining it is a giant pain in the ass and kills cross-DB compatibility.

These are examples of me being “too clever”. Not trying to brag or anything here, I’m using the term in a self-deprecating manner. What happened was that I, a relatively inexperienced developer, thought I’d found some “cool” solution to a problem I was having - or imagined I was having. I then “solved” the problem in a manner which was worse than anything the “problem” itself presented.

DO NOT use postgresql schemas unless you have a damn good reason for it. “Making your tables look nice” is not a good enough reason.

DO NOT use database-specific data types unless you have a damn good reason for it. Again, “looking nice” is not nearly enough.

UUIDs are strings, pure and simple. Using a UUID data type in PostgreSQL gains you one thing - checking that anything you try and save in there looks like a UUID. That should be in your own tests. What you lose is cross-platform compatibility, ability to use standard Rails, reliance on hacks to schema dumper, and more. And they’re strings, right? So store them in a fricking String field. Anything else is just not worth the hassle.

Schemas seem cool and elegant, and they are. Unfortunately, what you want isn’t “cool and elegant”, it’s “easy to work with and convenient to maintain and update and migrate and change around when you need to”.

It took me hours, days, to implement the changes to use the above features of PGSQL. It took a couple of hours to undo all of that and go back to standard data types, standard layouts, minimum complexity. Wasted work and time? Sure, but I learnt the lesson I’m trying to convey here - keep it simple, don’t be clever, maintain cross-compatibility and stick to the lowest common denominator unless you have a really, really good reason not to - and mine didn’t count in the end.

I’m still using PGSQL, of course - I like it, and even MySQL 6 inexplicably still doesn’t offer sub-second precision in DATETIME or TIMESTAMP, which I want. And I’m certainly not moving away from the UUID approach, which I firmly believe is best - all of that still stands. But I’ve gotten rid of all the special data structures and non-standard migrations and database-specific data types, because the miniscule aesthetic benefit just was not worth the loss of freedom and convenience everywhere else.

Live and learn, eh.

Amid all the RailsConf hype about Rails 2.1.0 and Maglev, not much attention seems to have been given to the fact that Rails is now generally, mostly runnable on Ruby 1.9.0.

Here’s the Ruby-Core message confirming the news. Not all tests are currently passing - almost all of them related to the TZTIME functionality introduced in 2.1.0, and needless to say mongrel still doesn’t work. Nor do the postgres or mysql gems, for that matter, although sqlite3 seems good to go - you ain’t gonna be switching to 1.9 anytime soon on your production site.

However, it runs, and if it runs, we can benchmark it!

I’ll use my /api/pulse controller action from previous testing. This is going to be Webrick-only, thin installs and seems to work until I load a page and then bombs out. Anyway.

The only modifications to the default rails application I am making here are to change the shebang line in script/server to point to 1.9, and to remove a couple of lines in boot.rb which check for RubyGems versions but bomb out for some reason in 1.9. Everything else is clean and Rails is installed as a gem under the 1.9.0 tree.

Webrick still has some problems under 1.9.0. Running with too high a concurrency seems to freak it out on 1.9.0, so I’ve turned the concurrency down to 5. Also, 1.9.0 is handily faster than 1.8.6 in development mode .. but in production mode, uh, you’ll see.

Ruby 1.8.6:

$ ab -n 500 -c 5 http://0.0.0.0:3000/api/pulse
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/
 
Benchmarking 0.0.0.0 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Finished 500 requests
 
 
Server Software:        WEBrick/1.3.1
Server Hostname:        0.0.0.0
Server Port:            3000
 
Document Path:          /api/pulse
Document Length:        2 bytes
 
Concurrency Level:      5
Time taken for tests:   2.67175 seconds
Complete requests:      500
Failed requests:        0
Write errors:           0
Total transferred:      139000 bytes
HTML transferred:       1000 bytes
Requests per second:    241.88 [#/sec] (mean)
Time per request:       20.672 [ms] (mean)
Time per request:       4.134 [ms] (mean, across all concurrent requests)
Transfer rate:          65.31 [Kbytes/sec] received
 
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       1
Processing:     9   20  13.1     17      85
Waiting:        7   17  12.5     15      84
Total:          9   20  13.1     17      85
 
Percentage of the requests served within a certain time (ms)
  50%     17
  66%     18
  75%     18
  80%     19
  90%     20
  95%     25
  98%     84
  99%     84
 100%     85 (longest request)

Ruby 1.9.0:

$ ab -n 500 -c 5 http://0.0.0.0:3000/api/pulse
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/
 
Benchmarking 0.0.0.0 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Finished 500 requests
 
 
Server Software:        WEBrick/1.3.1
Server Hostname:        0.0.0.0
Server Port:            3000
 
Document Path:          /api/pulse
Document Length:        2 bytes
 
Concurrency Level:      5
Time taken for tests:   3.921955 seconds
Complete requests:      500
Failed requests:        0
Write errors:           0
Total transferred:      139000 bytes
HTML transferred:       1000 bytes
Requests per second:    127.49 [#/sec] (mean)
Time per request:       39.220 [ms] (mean)
Time per request:       7.844 [ms] (mean, across all concurrent requests)
Transfer rate:          34.42 [Kbytes/sec] received
 
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    21   38  10.5     36      83
Waiting:       13   24   9.4     22      68
Total:         21   38  10.5     36      83
 
Percentage of the requests served within a certain time (ms)
  50%     36
  66%     38
  75%     39
  80%     40
  90%     46
  95%     69
  98%     73
  99%     74
 100%     83 (longest request)

Well, I said it runs, I didn’t say it runs well. Seems to be a problem with Webrick, probably related to the concurrency issue. By comparison, in development mode the numbers are about 42reqs/sec (1.9.0) vs around 33reqs/sec(1.8.6). Still - giant leap from even a couple of months ago, and I look forward to further leaps in the near future.

The next step is to get a real web server running. This will probably be Thin, once I figure out how to get around the crashing bug I’m encountering, I know that it should work, others are already using it on 1.9, though not with Rails. Any ideas welcome!

UPDATED to try and workaround concurrency issues with Webrick under 1.9.0
UPDATE 2: Duh, I was running on development.

UPDATE: Corrected a couple of typos. Didn’t correct the spelling error in the title because I am enjoying being ranked #1 in a search for the misspelled “naivety”.

I would like to point out also that this is a rant about vapourware and miserably unmet standards of proof - the benchmarks at RailsConf are worthless and prove nothing, but I would dearly love to be wrong.

And also note that I said I consider a dramatically faster Ruby interpreter/VM impossible until conclusively proven otherwise. I didn’t say completely impossible; I hope it is in fact possible to speed up Ruby by 10x or more. It seems unlikely, very unlikely, but who knows. I am in no way an expert on these things, and do not claim to be; I am only reacting to their hype-filled presentation, and drawing comparisons to the recent history of everyone else’s experiences writing Ruby interpreters sans the 60x speedup.

The demonstration at Railsconf was useless, empty hype, and until extraordinary proof is presented, I will remain deeply skeptical of these extraordinary claims.

So there’s been some presentation at Railsconf 2008 about a product called “Maglev“, which is supposedly going to be the Ruby that scales™ (yes, they actually use the trade mark). This new technology is going to set the Ruby world on fire, it’s going to be the saving grace of all Rails’ scaling problems. It’s going to make it effortless to deploy any size Rails site. Its revolutionary shared memory cache is going to obsolete ActiveRecord overnight. It runs up to 60x faster than MRI. And it’s coming Real Soon Now.

Every rails blogger and his dog have posted breathless praise for the new saviour:

Slashdot | MagLev, Ruby VM on Gemstone OODB, Wows RailsConf
RailsConf 2008 - Surprise of the Day: Maglev
MagLev is Gemstone/S for Ruby, Huge News
MagLev rocks and the planning of the next Ruby shootout

So what’s the problem? Why am I being such a party pooper and raining on the new Emperor’s parade?

Because these claims are absolute bullshit and anyone with a hint of common sense should be able to see that.

Right now, there are about 5 serious, credible, working Ruby implementations - MRI, YARV, JRuby, Rubinius, and IronRuby. They all have highly intelligent, experienced, dedicated staff who know a lot more about writing interpreters and VMs than I could ever hope to learn.

So do you seriously think that all these smart people, writing (and collaborating on) all these projects have somehow missed the magic technique that’s going to make Ruby run 60x faster?

It’s definitely possible to get a 2x speedup over MRI and retain full compatibility - Jruby and YARV have shown us that. Maybe it’s possible to get a 3x or 4x broad-based speedup with a seriously optimised codebase. And sure, a few specific functions can probably be sped up even more.

But a broad 10x speedup across the whole language beggars belief. It should be impossible. Actually, I’m going to come right out and say it is impossible, until conclusively proven otherwise, to make a fully compatible ruby implementation that’s more than two or three times faster than today’s best. Things just don’t work that way. Especially things with such a broad range of smart people working hard on the problem.

Extraordinary claims require extraordinary proof. But what do we get? A couple of benchmarks running in isolation. Who knows what they actually are, how tuned they are, whether they’re capable of doing anything other than running those benchmarks fast (I doubt it). No source. No timetable for the source, or anything else. Hell, it’s not open source anyway.

The bloggers say “this is not ready yet but when it is .. WOW!”. They’re missing the point. Until this thing is actually running Ruby, it’s not Ruby. Benchmarks on a system which isn’t a full implementation of Ruby are utterly worthless. I can write some routine which messes around with arrays in C which is a hundred times faster than Ruby. I might even be able to stick a parser on the front which accepts ruby-like input and then runs it a hundred times faster. Who cares? If it’s not a full implementation of Ruby, it’s not Ruby. Ruby is a very hard language to implement, it’s full of nuance and syntax which is very programmer-friendly but very speed-unfriendly. Until you factor all of that in, these benchmarks ain’t worth jack shit.

And wow ..! A shared memory cache! Finally, Rails can cast off that shared-nothing millstone around its neck. Except, of course, that shared-nothing is one of its main selling points. If you want to share objects use the database! That’s what it’s there for!

Oh yeah, the database! Maglev comes with a built-in OODB which is going to set the world on fire. Except of course that OODBs have been around for decades, and the last time I checked, we’re all still using 3rd normal form. If OODBs were the solution to all scaling’s ills then Facebook would be using Caché, not MySQL. Guess which one they’re using.

I have problems with the whole premise of OODBs, at least as they apply in web applications. Great, you can persist your Ruby objects directly into the OODB. What happens when you want to access them from, say, anywhere else? What if you want to integrate an erlang XMPP server? What if you need Apache to reach into it? What if you want to write emails straight into it, or read them straight out? What if you want to do absolutely anything at all which isn’t a part of some huge monolithic stack? Web applications are all about well-defined protocols, standard formats, and because of those, heterogeneous servers working in unison. I’ve heard OODBs have some benefits in scientific and other niche uses, but web applications are about the most mixed environment imaginable. If using an OODB is the answer, what the hell was the question?

Oh, you think I’m just an RDBMS-addicted luddite? Hell no. I eagerly follow and embrace advances in non-relational database technology - just look around this site, where I talk about being one of the first (crazy) people to press Couch DB into semi-production use, using TokyoCabinet and Rinda/Tuplespace for distributed hashtables, and how I’d much rather write a map/reduce function than a stupid, ugly, undistributable slow JOIN. But OODBs? Give me a break. There’s a reason the first thing any Zope developer does is get rid of ZODB.

But oh no. Show them one bullshit-laden presentation and the entire Rails community is champing at the bit and selling both kidneys to ditch all previous Ruby implementations and everything they thought they knew about the persistence layer and embrace some questionable closed-source vapourware, from the guys who brought you that previous world-storming web framework Seaside. What’s that, you’ve never heard of Seaside? I wonder why.

This credulity and blind bandwagon-jumping is the single worst thing about the Rails community.

I have covered forcing ActiveRecord to respect UUID data types in Migrations before. That helps us create our database - now what about in use? We need to create the UUIDs and store them in the database.

These examples all rely on the uuidtools gem, so install that if you haven’t already (and require it somewhere in environment.rb).

1. Setting a UUID using ActiveRecord callbacks

If you don’t need the UUID in the object upon creation but only want to ensure it’s there upon save, do this. Suggestion initially from this page, changes are mine.

We will use the before_create callback to ask AR to add a UUID of our choosing before the record is saved.

Add this to your lib directory:

# lib/uuid_helper.rb
require 'uuidtools'
 
module UUIDHelper
  def before_create
    self.id = UUID.random_create.to_s
  end
end

And now include this in your models:

class Airframe < ActiveRecord::Base
  include UUIDHelper
 
  #my stuff
 
end

>> Airframe.new
=> #< Airframe id: nil, maker_id: nil>
>> Airframe.create!
=> #< Airframe id: "1a82a408-32e6-480e-941d-073a7e793299", maker_id: nil>

2. Initialising a model with a UUID

If you want the UUID in the model before save, i.e. upon initialisation, we have to get a little more fancy:

# lib/uuid_init.rb
require 'uuidtools'
 
module UUIDInit
  def initialize(attrs = {}, &block) 
   super 
   @attributes['id'] = UUID.random_create.to_s
  end
end

Now include this in your models:

class Flightpath  < ActiveRecord::Base
 
  include UUIDInit
 
  # my stuff
 
end

>> Flightpath.new
=> #< Flightpath created_at: nil, id: "5e5bcd63-070d-4252-8556-2876ddd83b54">

Be aware that it will conflict with any other initialisation you do in there, so you might want to simply copy in the whole method if you need other fields upon initialisation:

class User < ActiveRecord::Base
 
  def initialize(attrs = {}, &block) 
   super 
   @attributes['balance'] = 0.0
   @attributes['id'] = UUID.random_create.to_s
  end
 
end

>> User.new
=> #<User id: "78e6cf98-676d-4e6a-a13c-318ece4f9e1b", balance: 0.0>

3. Sessions

All this is very well for your own models, but what about Rails’ inbuilt sessions? By default, they want an autoincrementing integer primary key.

The good news is it’s easy to override. Your migration should look like this:

create_table "sessions", :id => false, :force => true do |t|
  t.string   "session_id"
  t.text     "data"
  t.datetime "updated_at"
  t.datetime "created_at"
end

Now add this to your environment.rb file:

# config/environment.rb
CGI::Session::ActiveRecordStore::Session.primary_key = 'session_id'

And this to your Application Controller:

# app/controllers/application.rb
class ApplicationController < ActionController::Base
 
before_filter :config_session # at the top, if possible
 
def config_session
  session.model.id = session.session_id
end
 
end

And voila, your session store is using the session_id as its primary key. I don’t see any point in using a UUID for your sessions’ PK, but if you want to you’ll find an example override class in:

actionpack/lib/action_controller/session/active_record_store.rb.

Remember to drop any preexisting sessions table in your database, or it will likely complain of null ids when you switch to session_id as your primary key.

For those entranced by these benchmarks results and wanting to host Rails on Ruby 1.9 ASAP …

My testing shows Rails 2.0.1 failing on current svn/trunk installs of Ruby 1.9 on MacOSX 10.5.1.

But WEBrick works!

Ruby 1.9 build:

cd ~/src
svn co http://svn.ruby-lang.org/repos/ruby/trunk ruby-1.9
cd ruby-1.9
autoconf
./configure --prefix=/usr/local/ruby1.9
make
sudo make install
cd /usr/local/ruby1.9/bin/
./ruby -v
-> ruby 1.9.0 (2007-12-15 patchlevel 0) [i686-darwin9.1.0]

Rails 2.0.1 installation:

pwd
-> /usr/local/ruby1.9/bin
sudo ./gem install rails
-> Successfully installed actionpack-2.0.1
-> Successfully installed actionmailer-2.0.1
-> Successfully installed activeresource-2.0.1
-> Successfully installed rails-2.0.1
-> 4 gems installed
-> Installing ri documentation for actionpack-2.0.1...
-> Installing ri documentation for actionmailer-2.0.1...
-> Installing ri documentation for activeresource-2.0.1...
-> Installing RDoc documentation for actionpack-2.0.1...
-> Installing RDoc documentation for actionmailer-2.0.1...
-> Installing RDoc documentation for activeresource-2.0.1...
$

All installs nicely …

Attempting to run a Rails app (after install a few more requisite gems using above method):

$ cd /rails/my_1337_app/
$ /usr/local/ruby1.9/bin/ruby script/server
=> Booting WEBrick...
/usr/local/ruby1.9/lib/ruby/gems/1.9/gems/activerecord-2.0.1/lib/active_record/associations/association_proxy.rb:8: warning: undefining 'object_id' may cause serious problem
/usr/local/ruby1.9/lib/ruby/gems/1.9/gems/rails-2.0.1/lib/initializer.rb:224: warning: variable $KCODE is no longer effective; ignored
=> Rails application started on http://0.0.0.0:3000
=> Ctrl-C to shutdown server; call with --help for options
[2007-12-15 07:24:35] INFO  WEBrick 1.3.1
[2007-12-15 07:24:35] INFO  ruby 1.9.0 (2007-12-15) [i686-darwin9.1.0]
[2007-12-15 07:24:35] INFO  WEBrick::HTTPServer#start: pid=3386 port=3000
 
## I request http://0.0.0.0:3000/ ... 500 Internal Server Error
 
Error during failsafe response: can't convert Array into String
127.0.0.1 - - [15/Dec/2007:07:24:52 EST] "GET / HTTP/1.1" 500 60
- -> /

OK, it bombs out trying to actually process a request. But this error is really, really fast! I am actually serious saying that, the error *is* really fast.

Mongrel installation fails:

$ sudo ./gem install mongrel
Password:
Building native extensions.  This could take a while...
ERROR:  Error installing mongrel:
	ERROR: Failed to build gem native extension.
 
/usr/local/ruby1.9/bin/ruby extconf.rb install mongrel
creating Makefile
 
make
gcc -I. -I/usr/local/ruby1.9/include/ruby-1.9/i686-darwin9.1.0 -I/usr/local/ruby1.9/include/ruby-1.9 -I.  -fno-common -g -O2 -pipe -fno-common  -c fastthread.c
fastthread.c:13:20: error: intern.h: No such file or directory
fastthread.c:349: error: static declaration of ‘rb_mutex_locked_p’ follows non-static declaration
/usr/local/ruby1.9/include/ruby-1.9/ruby/intern.h:556: error: previous declaration of ‘rb_mutex_locked_p’ was here
fastthread.c:366: error: static declaration of ‘rb_mutex_try_lock’ follows non-static declaration
## etc etc etc

So webrick’s all we have for now.

You can track the state of edge Rails’ 1.9 readiness in this ticket on the Rails Trac. Plugins, though, will be another matter, although some fixes are pretty easy; an early failure I’ve seen is with plugins using File.exists?('file') which is of course deprecated in 1.9 in favour of the far, far superior File.exist?('file').

I like using the 1.9 console, though - it really does feel snappier, especially to load irb!

I hadn’t been using migrations much in Rails 1.x. I just didn’t like the workflow - it was too clumsy, and I got annoyed at writing out the files. The workflow in 1.x was as such:

1.

script/generate migration MigrationName [--svn]

2. go and manually edit 00x_MigrationName.yml in /db/migrations to reflect your desired database changes, both up and down:

class AddCreatedAtToStaff < ActiveRecord::Migration
  def self.up
    add_column :staff, :created_at, :datetime
  end
 
  def self.down
    remove_column :staff, :created_at
  end
end

3. rake db:migrate to apply the changes to the local database.
4. svn commit and cap deploy:migrations to apply the changes to the remote database

Too long - especially step 2. I knew I should do it in principle, but CocoaMySQL is right there - especially if you make several changes in the space of a few hours. In theory it’s best practise - but in actual practise someone as lazy (and impatient) as me tends to just do it directly in the DB, then eternally put off that day where they’ll finally “move to migrations”.

It’s not even the laziness - it’s the “flow”. I’m doing other things, I’ve realised I need this field. I don’t want to stop what I’m doing and write out a damn YAML file! I want the field there, right now. Too demanding? Maybe, but like I said, the DB is right there, and the difference between a 5 second edit and 2 minutes writing the file is a lot of lost concentration.

And various other solutions, such as auto_migrations, seemed good but in practise are too flaky and a dangerous, unsupported road to take.

Enter Rails 2.0, and migrations are far, far better. The core Rails principle of “convention over configuration” is in full effect here, with excellent results.

Now the process of adding a migration is as such:

1.

script/generate migration add_created_at_to_staff created_at:datetime [--svn]

Note the convention at work here. You’re implicitly telling Rails the table to use, in this case “staff”, and the field you’re adding - in this case one of Rail’s magic “created_at” fields. You then explicitly write the fields out, which you’d have to do anyway in CocoaMySQL or similar, or manually at the mysql command line.

2. rake db:migrate to apply the changes to the local database.
3. svn commit and cap deploy:migrations to apply the changes to the remote database

That’s only one step less, but it was the biggest and most annoying one. The actual creation of the migration file and addition to svn is now a single brief, easy-to-remember line. This is now terse enough and convenient enough to enter my personal development workflow, a very welcome improvement to the framework, and an excellent demonstration of Rails’ core principles in action.

Rails 2 has added the ability to automatically serve static public files - “assets” - from multiple servers, to speed up loading times. This is well worth implementing and will speed loading times considerably. And it’s very simple to do. You’ll need Rails 2.0RC1+ for this. Obviously your implementation details may differ, especially filepaths on the server.

Step 1: In rails_app/config/environments/production.rb:

config.action_controller.asset_host = "http://assets%d.my_kewl_domain.com"

Step 2: In my_kewl_domain.com.zone

www		IN		A	31.337.31.337
 
assets0		IN 		A	31.337.31.337
assets1		IN 		A	31.337.31.337
assets2		IN 		A	31.337.31.337
assets3		IN 		A	31.337.31.337

Step 3: In httpd.conf

<VirtualHost 31.337.31.337:80>
ServerName assets0.my_kewl_domain.com
ServerAlias assets1.my_kewl_domain.com, assets2.my_kewl_domain.com, assets3.my_kewl_domain.com
DocumentRoot /www/my_kewl_domain.com/current/trunk/public
</VirtualHost>

Note this is a separate vHost from your main application. Note also that it is looking straight into the /public directory of your app.

svn up, cap deploy, restart %w[named httpd mongrel] and you’re done.

Including the coming Ruby 1.9 (apparently still on course for sometime this month), this is a nice free speedup for your application.

So I’ve been watching the RailsCasts episodes going through the latest features in Rails 2. I’ve read about them before - but it’s good to see them demonstrated by an expert.

But man - some of this is too much. Don’t get me wrong, I like magic - if I didn’t like magical free stuff I wouldn’t be using Rails at all. But there comes a point where there’s enough magic, and more just trades off too much for too little - diminishing returns, as it’s called.

Take this screencast: Simplify Views with Rails 2.0. It shows very clearly what I’m talking about.

Now some of this is good shit. The ability to write this:

<% div_for_product do %> # is this supposed to be <%= ?
<% end %>

And have it automatically generate div IDs and the like is great. That saves some nasty looking code - assuming it works properly, stays unique across multiple includes, etc. Good shit. A worthwhile addition.

But this:

# from
<%= render :partial => 'product', :collection => @products %>
# to
<%= render :partial => @products %>
# and for a single case
<%= render :partial => @product %>

I think this is bad news. Suppposedly Rails is going to look inside that instance variable, decide whether it’s an object or an array, and automatically use the correct partial, multiple times if it’s an array .. why? To save what, 15 keystrokes?

I would argue the “Rails 1.2″ way of doing this is about the most concise way to write this function imaginable while still maintaining decent at-a-glance understandability. It’s not so long and unweildy, is it? And you can see what’s happening instantly - ok, now render a bunch of partials named ‘this’ using whatever was in this @array. The word “collection” is a nice touch which helps you remember there’s a number of them.

The second one? Its behaviour literally changes according to what’s in that instance variable. Is it really that much nicer to look at that it’s worth losing the information necessary to see, at a glance and unambiguously, what it’s doing?

There’s more, and worse: forms, where we go from this

<% form_for :product, :url => product_path(@product), :html => {:method => 'put'} do |f| %>

to

<% form_for @product do |f| %>

And if you’ve used the strict Rails conventions, mapped all your resources in routes.rb, are using the strict controller layout as demonstrated in the resources scaffold generator and have a picture of DHH as your desktop background, it’ll know what to do and work.

Now, at this point you might be saying “but you don’t have to use these features - it’s only if you follow the convention, which is the whole point of Rails! Follow the convention, get free benefits - it’s been like that since day 1″!

I agree with that, kind of. But there is such a thing as too much convention, and too many limits, and - this one’s my point - too much “magic”. I know it’s a bit silly to talk about “vendor lock-in” in the context of a Rails project but following “The Rails Way” is beginning to smell strangely similar. And these extraordinary efforts to save a few keystrokes, ONCE, at the expense of readability (and no doubt speed!) are beginning to seem less like cleverness and more like Koolaid-driven idealogy.

We’ve been told, again and again, that it’s all about “Beautiful Code”. And you know, I appreciate that up to a point. But when the code is getting less readable and more dependent on arbitrary chains of “convention dependence” which may or may not work (and which may or may not change in 3.0!!) then I start getting cold feet.

You know what? I’m working on *a* web site, not *the* web site like these changes all seemed designed for.

Predictions of a coming fork, anyone? “Rails Lite”?

Oh, and despite the endless hours which must have been poured into this kind of useless bloat, YAML handling in rails is still UTF8-unsafe.

UPDATE: Sure, you can just not use these features. But the codebase bloats, the execution speed is slower - and it’s not like it was fast to begin with. I argue these new features make the code more difficult to read and learn - hurting the Rails community as a whole, reducing numbers of developers, marginalising the framework - as someone with a considerable investment in that framework I am a stakeholder in Rails, and it’s from that position I comment.

post_update <<-EOF
Can the people who read this rant please remember that this is not a very serious blog. In fact it’s mostly just me talking shit and this post is a prime example. I actually like the changes I’m dissing below, it’s nothing but a rant for rant’s sake and I have no idea why Google seems to be sending so many people to it - so don’t get all upset and leave nasty comments telling me I’m an idiot.

This is a personal blog of the most whimsical and useless kind, and I make no pretensions otherwise. Bear that in mind. Thanks!
EOF

Can someone please tell me what the point of moving from .rhtml to .html.erb is?

I mean seriously. There wasn’t a naming conflict, .rhtml isn’t the nicest extension on earth but it was pretty unambiguous as to what it was - ruby mixed with HTML.

.html.erb is if anything worse. What kind of a file has TWO extensions? Hell, why not have fucking ten. I sometimes like to put CSS and JS into my files as well, they’re feeling a bit left out - I guess that according to this new “all inclusive” philosophy they’re properly named index.html.css.js.erb.

And what the fuck does ERB even stand for? Embedded Ruby, right? Well that would be great if Ruby was two words. One acronym per word is the English standard. Yet another example of Rails flaunting the standards.

I just don’t see the point. Trying to enforce a taxonomy on what are by nature highly mixed and variable files is an exercise in futility. And furthermore, the .erb is completely superfluous within the already highly structured folder structure of a Rails app. What the hell else would it be?

Change for change’s sake. DHH is drunk on power!

Current CouchDB development is progressing in an experimental branch named “interface-changes”, and it’s not in trunk yet. As the name implies, the interface is changing and it’s very useful.

To switch your svn checkout to this branch, do this:

svn sw http://couchdb.googlecode.com/svn/branches/interfacechanges/
./bootstrap && ./configure --prefix=/opt/local && make clean && make && sudo make install

You can then have a look at the “documentation” to see some of the new changes.

While still alpha, it’s still a very interesting project. The prospect of getting something like Map/Reduce capability natively in a database is almost too exciting for words, if you’re a data nerd like me. Well, figuratively speaking, Map’s there, but Reduce isn’t yet - still, it’s great to get your hands on what seems sure to become a Big Thing so early. Oh, and in case you were wondering, COUCH stands for “Cluster of Unreliable Commodity Hardware”.

Imagine GFS and Map/Reduce, baked into a single databse, with JSON in/out, pluggable query language, and native REST .. what’s not to love?

Laika’s ThingFish is a Ruby-based competitor. You might think I’d be more interested in that, since it’s in my favourite language! Not so. The very thought of a Database programmed in Ruby actually gives me an instant waking nightmare along the lines of running through treacle, gulliver in lilliputia, 80286SXs with 12MB of RAM, etc. And using mongrel server for a database!

UPDATE: it’s been merged to trunk, so just forget that interface-changes stuff.

Want to use the new kid on the block, nginx, as a server on your old RHEL4 server? Well, guess what, it’s now available in EPEL so it’s incredibly easy.

First add EPEL as a repository on your system. They provide a package to do that:

rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-2.noarch.rpm

now install nginx. You’ll need to specify the testing repo. I use yum but the same thing will work with up2date

yum install nginx --enablerepo=epel-testing

Done. Now go edit /etc/nginx/nginx.conf to use an IP or port that doesn’t conflict with anything, and:

service nginx start

UPDATE: Updated EPEL RPM link with new version.

If you, like me, have been keeping an eye on the extremely interesting CouchDB Project then maybe you’ve thought about playing around with it a bit on your Mac. This turns out to be very simple.

I assume you have macports w/ subversion (and XCode, of course) installed already. Most of this comes straight from the Readme but I hate reading those, I just want someone to write what I have to do - maybe you do too, so here you are!

UPDATE: 11th October

Reinstalling after noticing 60 commits in 3 days .. now that’s active development.

(more…)

No matter what anyone says, hosting Rails applications is - and is likely to remain for the foreseeable future - a cast-iron bitch. An expensive, slow one, too.

I’m no stranger to web hosting. I set up my first web server over 10 years ago - manually, of course, none of these control panels, and have done it many times since, as technology changed and languages shifted. I’ve been responsible for the production web presence of a number of companies, and along the way I’ve dabbled with a number of server-side application technologies, including ColdFusion and even Java (ugh!) but mostly PHP, since that’s what everyone uses.

And for good reason. PHP has many flaws as a language, mainly to do with security and the hideous mess most projects in PHP comprise, but it’s a joy to deploy. Here’s how it works.

PHP is actually two things. It’s a language, and a compiled executable which will interpret and act on commands issued in that language. It integrates nicely with the most popular, and largely considered best, front-end web server software, apache, so that what happens when a PHP web page is requested is as such:

1. Apache reads the web page, typically page.php, and determines it requires action by the PHP interpretor, and creates an instance of that
2. The commands in the page are relayed to the PHP interpretor instance
3. The interpreter processes the commands and performs whatever action is required, typically HTML generation and/or database actions
4. Apache receives back the data and serves it inline with the rest of the web page.

This all happens extremely quickly, and since PHP is integrated so tightly with Apache, you only have to worry about maintaining that one application. Sure, you configure PHP - via a single .ini file. Everything else is Apache all the way, and it’s a highly reliable and very fast way to serve web pages.

An especially welcome benefit from this arrangement is that you only require one Apache server running to serve any number of PHP applications. You can easily configure Apache to serve any number of domains via its VHosts system, any or all of which may contain active PHP pages. Apache simply instantiates (and destroys) PHP instances, which respond to the relevant pages, whenever required. The PHP executable doing the interpreting neither knows nor cares where the commands it’s interpreting come from. You’ve effectively created a “pool” of servers, any of which can respond to any PHP application, and Apache can balance its load amongst them as required.

The situation with a Rails application is very different, and not in a good way.

The current “best practise” for deploying a single Rails application is as follows:

1. Apache server front end, with a vhost interpreting the URL and relaying relevant requests to a cluster of mongrel servers, addressable by (local) IP
2. User then configures and maintains a seperate cluster of mongrel servers dedicated to that single Rails application and responding on configured IPs as set in Apache
3. Apache is further configured to go through mongrel for every non-static request.

This is then repeated for every Rails application on the box.

This has several disadvantages compared to the situation with PHP, to wit:

1. Administrator becomes responsible for setting up, maintaining, and constantly monitoring a collection of independently-running applications, in addition to the core Apache server
2. Mongrel servers not currently being used are not destroyed, but continue running, wasting memory (every instance of mongrel currently running on my server uses in excess of 100M each)
3. Members in a cluster must be predefined and cannot be allocated on the fly according to need. For example, If you have 1GB of memory and 3 Rails apps, you are limited to distributing about 10 instances of mongrel between them. You must decide this in advance. If one of the applications suddenly receives a lot of hits, performance will suffer while the other mongrels, for the other apps, sit unused - the opposite of the “pool” you have access to in a PHP situation

And this is before even mentioning the underlying speed of mongrel, Ruby and Rails in general, which can only be described as horrible.

So, the sequence of events when a page request for your Rails app comes in is as follows:

1. Apache receives request, determine the request is for your rails app
2. Apache picks a server in your mongrel cluster, and sends the request (NOT commands, as in PHP) to it
3. Mongrel, moving with the lightning speed of a slug through treacle, goes through the request page and sends the relevant parts to the ruby interpreter, itself not much of a speed demon, which, after a lengthy time out for self-reflection and “me time” reluctantly listens to, thinks about, thinks about some more, and then finally responds to the command. Repeat for every ruby command on the page
4. The mongrel cluster receives and compiles the completed requests and, taking its sweet time, eventually manages to respond with the complete page
5. Apache passes on the completed page to the client.

Not many more steps there, I admit. But it’s the speed of it which is the killer. PHP commands are detected, seperated from enclosing HTML and dispatched from within Apache, a lightning-fast and time-proven, highly reliable web server written in C. But in Rails, the job of going through the page to find Ruby requests and dispatch them is performed by Mongrel - itself written in Ruby. I don’t have any figures depicting how much slower it is, but it is - a lot.

Combine that with the inflexibility of distributing resources where needed, and having to independently spawn and monitor a seperate cluster of servers for every seperate application and you’ll understand that when people say hosting Rails is a nightmare, they’re right.

Rails has a lot of things going for it. It’s a joy to write in, mostly, and the development environment is great. It’s much, much more productive than the likes of Java while being much, much nicer than PHP.

But it’s slow as hell, and deployment sucks, and don’t let anyone tell you otherwise.