Right you are on every single one of those points. I’m beginning to wish I’d followed your wisdom in not allowing comments on your blog! On the plus side, it’s been educational and someone had to come out and say it.

A Ruby implementation is 100% compatible with MRI or it’s no Ruby implementation at all. For the time being, Maglev is not a Ruby implementation and until it is, this discussion is over.

I have to admit I’m pretty surprised to see that post. What the hell is Chad Fowler doing pitching for Gemstone?

From now on I suspect that the Ruby community will use the expression “It’s a Shoims” to point out an unjustified excess of skepticism about Ruby’s bright future.

That is completely untrue and shame on you for saying otherwise. I love Ruby and no-one is more excited than I about Ruby’s bright future than me. Take a look around this blog, why don’t you. I have embraced Ruby wholeheartedly and contribute all I can to the community.

However, I also believe that future is free, libre and open source. I also believe wholeheartedly in Open Source software and am more than a little dismayed to see so many people apparently forgetting about its principles in their headlong rush to endorse this product which apparently runs counter to most of them. I find it suspicious and more than a little perverse.

Right now, Gemstone’s Ruby implementation is little more than a trialware binary which happens to perform decently on a few specially selected “thought leader”’s computers. It is more closed than Microsoft’s IronRuby and every statement about its performance, including Chad Fowler’s, must be taken on trust. I find this situation to be pretty regrettable.

We shall see if the product lives up to its claims and is such a leap forward in performance that it’s worth abandoning Ruby’s open source heritage to use it. For now, though, I stand by my skepticism regarding this trialware, vapourware, incomplete, closed-source commercial implementation.

What are the facts that it doesn’t take a doctorate in compiler construction, or inside information about this closed-source, unreleased product, to appraise at this stage?

One: the product is vaporware. It doesn’t matter how good it is when it’s completed and released; the fact is that right now it’s not completed and it’s not released. Let’s talk again when it is actually done and out.

Two: it’s not Ruby yet. It doesn’t matter how fast it is until it can actually run everything that MRI can, and in the same way that MRI can. Offer something that is “95% Ruby” (or any other percentage less than 100%) and it’s simply not interesting, no matter how fast it runs.

Three: microbenchmarks are irrelevant to me as a Rails application developer; show me real-world application benchmarks with real-world data sets. There aren’t any such benchmarks yet, and they’re won’t be either until this thing is actually Ruby and Rails compatible.

Four: the magnitude of the benchmarks does strain the limits of credulity, probably because they’re microbenchmarks. If it sounds too good to be true, it probably is.

Five: a particularly vocal segment of the Ruby/Rails community has gone into its habitual delirium at the announcement of the newest and shiniest bauble, and is ready to entrust all of its production eggs to this new basket which was apparently whipped up in just three months.

There is only one reasonable conclusion from a rational assessment of all these facts: that while Maglev is certainly interesting and it may be quite promising, perspective is called for and people should be adopting a conservative “wait and see” attitude rather than a “let’s piss our pants with excitement” one.

There end the facts and the rest is all speculation and subjective opinion. For me, my own speculation and subjective opinion is that I am doubtful about this thing attaining 100% Ruby equivalence any time soon, I am sceptical about the real-world performance benefits compared to alternatives like YARV and JRuby (and I’m conservative in my deployment choices so I am not even thinking of switching to those from MRI at this stage, let alone switching to this newest kid on the block), and I’m not willing to sell-out to a commercial, closed-source platform in exchange for as-yet-unknown performance improvements, no matter how large they may turn out to be in the end.

Sho, I believe you still get some remaining time to pretend that you were joking, but not much.

Gemstone is the best thing that has happened to Ruby in years (Ruby 2.0 is … second). Period. ”

Sho. You are cooked by now. I warned you. Sorry. http://www.chadfowler.com/2008/6/5/maglev

From now on I suspect that the Ruby community will use the expression “It’s a Shoims” to point out an unjustified excess of skepticism about Ruby’s bright future.

You’re famous now. Kudos.

Well, initial thoughts about that benchmark is that it’s almost a perfect test of what’s bad about MRI. Object allocation and GC are famously inefficient in MRI and that script does practically nothing else; one could hardly find lower hanging fruit. Charles Nutter and “Ralf” make insightful comments along those lines on that post.

Futhermore, he leaves out the result for YARV, which is over 3 times faster than MRI on that test. I can’t understand why, Ruby 1.9.0 is the benchmark for “official” Ruby implemented with performance in mind.

Here’s the full list of results from the debian “shootout”. As you can see, Smalltalk (VW) does extremely well on that test; a mere 2.8 times slower than the fastest result, which is Java. Obviously the test favours a VM over “real” MM, and as discussed those speeds do not carry over into the general testing results.

That said, it is pretty impressive. Some of my own long-running AR import/dump scripts would benefit greatly from that kind of optimisation. I guess we’ll have to wait for the full tests to be released.

Wish I could just run the tests myself. I’m curious and *want* to do it. I hate this closed source BS where we’re all waiting on the word from one “official” tester.

UPDATE: I could swear those rankings now look quite a bit different than they did a day or two ago.

http://antoniocangiano.com/2008/06/05/maglev-handles-trees-like-a-monkey/

Maglev almost as fast as C/C++.

For a Java to ruby comparison just select the right menus from the web page you cited yourself :
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=all&lang=yarv&lang2=java

Java beeing around 222 faster on the mandelbrot test and more than a 100 times faster on other tests.

(Deep sigh)

I still see a big lead in benchmarks where really the VM is running the code.
Other results are better for ruby because there’s more C code involved.
Of course you can can speedup “your” language by implementing almost all of your libraries in C, but that limits you in extending your libraries.

OK, I agree with you there. It’s better, generally, to pursue performance as much as possible within the context of the VM. More extensible, more centralised, less “moving targets”, more pure in general.

However, it’s the results that count, native C or VM or whatever. The vast majority of people do not need to extend Ruby in a way that would conflict with its mode of implementation.

And personally, the fact that the C implementation is fully open source is, to me, a powerful vote back in its favour. It may well be more difficult to extend some library if it’s implemented in C - it might also be completely impossible to do so on a proprietary VM like the one announced by Gemstone.

Who said that in (all) real world applications Ruby could be 15(insert your factor here) times faster?

The whole reason I wrote my article is because a great many people were claiming or implying exactly that. Gemstone’s presentation implied gains of between 15x and 60x. If they’d just claimed 2x or 3x or something, there wouldn’t have been such an explosion of hype and I wouldn’t have been incensed enough to write this post!

Also VW was not always the fastest Smalltalk implementation on the planet. Animorphic Systems had an implementation in the 90’s that was at least 5 times faster. Those guys went to Sun. You can see the result of their knowledge by comparing the Ruby results against Java. Java is still in certain cases more than 100 times faster than the (not yet released) Ruby 1.9.

[Citation Needed]

Here we go again with Smalltalk folks claiming huge performance numbers from “secret Smalltalk techniques”. “At least” 5x faster across the board would make Smalltalk, a dynamic language, faster than the most recent implementation of Java, which is statically typed!

Give me a break. Point to some hard numbers from a reliable source that back up your assertions or I’m going to continue with my assumption that you’re pulling these “facts” straight out of thin air.

I still see a big lead in benchmarks where really the VM is running the code.
Other results are better for ruby because there’s more C code involved.
Of course you can can speedup “your” language by implementing almost all of your libraries in C, but that limits you in extending your libraries.

Who said that in (all) real world applications Ruby could be 15(insert your factor here) times faster?

Also VW was not always the fastest Smalltalk implementation on the planet. Animorphic Systems had an implementation in the 90’s that was at least 5 times faster. Those guys went to Sun. You can see the result of their knowledge by comparing the Ruby results against Java. Java is still in certain cases more than 100 times faster than the (not yet released) Ruby 1.9.

What you seem to be missing is that what all of the alternative Ruby VM implementers have been saying up till now re: optimization boils down to “we know we have barely scratched the surface, optimization-wise; but we know that the Smalltalk community has some really advanced techniques for optimizing dynamic languages, and we hope to apply those techniques once we are 100% Ruby-compatible”. Saying that JRuby, Rubinius, et al are *currently* focused on performance optimization indicates that you haven’t been following their development blogs much. Both Charles Nutter and the Rubinius guys have all been pretty up-front about a) feature-completeness is their goal now and b) optimization comes second. Although since JRuby has been able to run Rails that team have been able to spend more time on performance optimization.

My point was that feature completeness is not some abstract goal which might be considered an alternative to the goal of performance, it is a fundamental prerequisite before you can even start talking about speed.

As has been pointed out (pretty forcefully, heh) I am far from an expert on the complex business of writing interpreters/compilers/preprocessors/garbage collectors/VMs. However, I do have a little experience in software development and like to think I’m aware of its basic nature. It seems to me that unless JRuby (and, more to the point, YARV) started from (and still use) some fundamentally flawed, obsolete theory of language interpreters/VMs, then the idea that there are these massive uncollected performance gains just waiting for someone to harvest by implementing a more mature VM is hard to believe.

At the time of writing, I’d been reading other bloggers splashing numbers like 15x-60x performance gains around. That kind of gain from a tweaked VM is ludicrous on its face. 60x faster would be faster than C! Again, I’m not an expert - but if I hear someone claiming they can make Ruby faster than C, well, I’ll believe it when I see it.

Also, I’m not sure what you mean by Smalltalk being much less dynamic than Ruby. You do know that Ruby is essentially Smalltalk with a Perlish syntax, right?

I shouldn’t have used the word “dynamic”. I was trying to refer to the syntax and general behavioural spec of Ruby, which is notoriously difficult to interpret and was, I thought, pretty resistant to preprocessing. Maybe a better word would have been “complex” or “flexible”.

So if the MagLev team are executing Ruby code on a mature optimized Smalltalk VM, I don’t have any trouble believing that they’ve seen massive improvements in speed. These are the same improvements that the JRuby, Rubinius (especially Rubinius, which is modeled directly on the Smalltalk implementation), et al are all expecting to see once they are able to focus on optimization. MagLev is just already there, because they don’t have to re-write the Smalltalk-style optimizations form scratch.

I also don’t have any difficulty believing that some impressive speed gains are possible once the VM is mature and optimised. YARV, for example, is around twice as fast as MRI and it’s very possible that Maglev could be even faster.

What I do have trouble believing is a 15 or more times gain. That would mean that Ruby was now comparable speedwise to Java, and I don’t think anyone will argue with me if I say Ruby’s more dynamic than that. That magnitude of improvement is unheard of in any software development project I’m aware of. I’m not going to say it’s absolutely impossible - but again, I’m going to need to see it to believe it.

It is entirely possible they’ve seen massive improvements in speed - in certain cases, and no doubt it was those cases which were shown off at Railsconf. But the performance of a language is based on across the board speed, not some little tweak here or there. As I mentioned above, Ruby 1.9.0 today is about 27x slower than C. The fastest smalltalk is about 9x slower than C. You might be able to find some neglected backwater here or there and optimise it until it’s 15x or even 60 times faster - but across the board?!

I can’t get my head around this point. Even in this comment thread you’ve got people who seem to believe that Maglev is going to make Ruby twice as fast, or in the worst case only twice as slow, as C. I wonder if I live on the same planet as anyone who can believe that without some pretty stunning proof. Even better, apparently some benchmark maxed at out 110x faster - if so, maybe we should start thinking about re-implementing C in Maglev Ruby, the new fastest language of all time by a factor of four.

More to the point, showing off these ultra-fast benchmarks results before the language, with all its nuances, is even close to being fully implemented is really irresponsible and distasteful to me. Incomplete benchmarks in isolation are useless, everybody knows it - I know it, you know it, they know it. But knowing that - they showed them anyway. Why? It’s a marketing tactic, nothing more. It reminds me pretty strongly of other shoddy marketing tactics I’ve seen - particularly in the computer game industry - and doesn’t inspire faith in the company one bit. Worked pretty well, though, I have to admit.

Worth checking out anyway: http://ruby.macosforge.org/

Also, I’m not sure what you mean by Smalltalk being much less dynamic than Ruby. You do know that Ruby is essentially Smalltalk with a Perlish syntax, right?

So if the MagLev team are executing Ruby code on a mature optimized Smalltalk VM, I don’t have any trouble believing that they’ve seen massive improvements in speed. These are the same improvements that the JRuby, Rubinius (especially Rubinius, which is modeled directly on the Smalltalk implementation), et al are all expecting to see once they are able to focus on optimization. MagLev is just already there, because they don’t have to re-write the Smalltalk-style optimizations form scratch.