Vindicated? Well, maybe a little, but the thing is if I claim vindication I’m committing the exact same offense I railed against – drawing conclusions from inadequate data. Time will tell.

I defer to Wincent’s analysis of the situation regarding GC’ing languages. In that area, not only do I not know what I’m talking about, I don’t even pretend to, unlike Ruby

Beoran: I agree completely. I will never lock myself into some proprietary system. Anyone who developed anything using the whole Maglev toolchain would be chained to it for life. This is so against my principles, and what I believe the Ruby principles to be, that I also don’t care about any other factors, I’m not using it. There are very few exceptions to this rule.

Non-portable Ruby apps running on a (binary-distributed) proprietary VM – which you got from a company that makes “their money by extracting large amounts of money from a small number of customers.” Gee, where can I get in line for that dream come true?

Well, what you say about the interpreter is true enough, even YARV is far from the state of the art. It’s possible that a more modern VM will indeed produce large gains. The argument is how large, while still maintaining full compatibility?

We don’t know, and the product is still vapourware. Recent benchmarks indicate it’s around twice as fast as MRI in current alpha stage. Hardly a pants-jizzing improvement, then.

The way some of the above “experts” talk, Ruby is such a minor variation on SmallTalk that the adaptation of Gemstone’s VM to implement it is barely an afternoon’s work for those hitherto-unsung geniuses. Well, six months later, what do we have? A half-working alpha which is slower than YARV.

Now, it is certainly possibly that they will manage to eke out further gains, if they ever do finish it. But let’s remember the atmosphere of unquestioning hype which surrounded their premature announcement, and which prompted this highly skeptical post. Let me quote from one of the articles I linked to:

we got to see some preliminary performance data that showed an order of magnitude or TWO increase over MRI 1.8

This is the reason I was so negative in my post. These Gemstone guys got up on stage, showed a couple of completely irrelevant micro-benchmarks that showed their incompatible alpha running some tiny subset of something, and obviously left people with the impression that their VM would be one or two orders of magnitude faster than MRI.

I’d like to point out I didn’t say it was completely and utterly impossible. I just said I don’t believe a fully compatible implementation will be able to reach anything like those speeds, and I would require a pretty high standard of proof before I abandoned that position. 6 months later, that proof has not arrived, and in fact I seem to be vindicated somewhat by the yawn-inducing recent benchmarks, which show a massive speedup on some tasks but a far lesser increase across the board.

Still, we’ll wait and see. To quote myself: A Ruby implementation is 100% compatible with MRI or it’s no Ruby implementation at all. For the time being, Maglev is not a Ruby implementation and until it is, this discussion is over.

Your explanation of Gemstone’s business model of locking a small number of customers into their proprietary system and then screwing them for all they’re worth does not exactly fill me with love.

Regarding your comment about ODBMSs: Funnily enough, I am actually quite a fan of them. For interests’ sake I actually wrote a kind of mini-ODBMS earlier this year trying to store object graphs around the place in some kind of ordered fashion, basically marshalling in and out of CouchDB with a neat object linking system. It was great fun. However, your comment about not suffering market rejection is disingenuous.

When I talked about DBs, I believe it was quite self-evident I was referring to shared data repositories – what Fowler called an IntegrationDatabase. An ApplicationDatabase is a much more specialised, varied, and usually small-scale, animal. Yes, applications store their state, and that state is often in the form of objects, whether marshalled or stored in an ODBMS. There are, indisputably, many such applications, and many such implementations. However, they’re not counted in the IntegrationDatabase world I was talking about. I wouldn’t have thought it necessary to even draw that distinction, but there you are.

Anyway, in summary, I have not yet seen any proof, or even any good evidence, that Gemstone’s claims were, or will be any time soon, realistic. I retain my distaste for the way the company very deliberately hyped its vapourware product, and I stand by my disappointment of the Rails community’s unquestioning acceptance of that hype. However, it was a while ago now, everyone’s moved on, and I think Gemstone will really only hurt themselves in the end, so whatever.

It seems to me that unless JRuby (and, more to the point, YARV) started from (and still use) some fundamentally flawed, obsolete theory of language interpreters/VMs, then the idea that there are these massive uncollected performance gains just waiting for someone to harvest by implementing a more mature VM is hard to believe.

I think that’s a pretty accurate summary of the state of affairs. YARV started from and still uses a fundamentally inefficient, obsolete theory of language interpreters, so there are these massive uncollected performance gains just waiting for someone to harvest by implementing a more mature VM. Non-JIT interpretation has been obsolete for more than ten years now, but people outside academia mostly don’t know that yet. Hopefully Maglev, v8, SquirrelFish Extreme, HIPE, and TraceMonkey will bring that into the mainstream now, the way Perl (and later Java) did for garbage collection. (Malloc/free had been obsolete for most applications since 1982.)

JRuby is crippled differently: although its fundamental model is valid (compile to an intermediate language which then gets JITted to machine code), it compiles to bytecode for a JIT designed for a language with a very different type system, and they don’t have the freedom to change the JIT or the bytecode format. This is an argument in favor of open source, obviously.

Gemstone is a high-end Smalltalk system; you haven’t heard of it because they make their money by extracting large amounts of money from a small number of customers. Progress ObjectStore is still widely used, but mostly as what Martin Fowler calls an ApplicationDatabase, not an IntegrationDatabase — largely inside CAD systems and the like. ODBMSs haven’t been as much of a success as the Statice guys were hoping when they started ObjectStore, but they certainly haven’t suffered the resounding market rejection you paint.

Right you are on every single one of those points. I’m beginning to wish I’d followed your wisdom in not allowing comments on your blog! On the plus side, it’s been educational and someone had to come out and say it.

A Ruby implementation is 100% compatible with MRI or it’s no Ruby implementation at all. For the time being, Maglev is not a Ruby implementation and until it is, this discussion is over.

I have to admit I’m pretty surprised to see that post. What the hell is Chad Fowler doing pitching for Gemstone?

From now on I suspect that the Ruby community will use the expression “It’s a Shoims” to point out an unjustified excess of skepticism about Ruby’s bright future.

That is completely untrue and shame on you for saying otherwise. I love Ruby and no-one is more excited than I about Ruby’s bright future than me. Take a look around this blog, why don’t you. I have embraced Ruby wholeheartedly and contribute all I can to the community.

However, I also believe that future is free, libre and open source. I also believe wholeheartedly in Open Source software and am more than a little dismayed to see so many people apparently forgetting about its principles in their headlong rush to endorse this product which apparently runs counter to most of them. I find it suspicious and more than a little perverse.

Right now, Gemstone’s Ruby implementation is little more than a trialware binary which happens to perform decently on a few specially selected “thought leader”’s computers. It is more closed than Microsoft’s IronRuby and every statement about its performance, including Chad Fowler’s, must be taken on trust. I find this situation to be pretty regrettable.

We shall see if the product lives up to its claims and is such a leap forward in performance that it’s worth abandoning Ruby’s open source heritage to use it. For now, though, I stand by my skepticism regarding this trialware, vapourware, incomplete, closed-source commercial implementation.

What are the facts that it doesn’t take a doctorate in compiler construction, or inside information about this closed-source, unreleased product, to appraise at this stage?

One: the product is vaporware. It doesn’t matter how good it is when it’s completed and released; the fact is that right now it’s not completed and it’s not released. Let’s talk again when it is actually done and out.

Two: it’s not Ruby yet. It doesn’t matter how fast it is until it can actually run everything that MRI can, and in the same way that MRI can. Offer something that is “95% Ruby” (or any other percentage less than 100%) and it’s simply not interesting, no matter how fast it runs.

Three: microbenchmarks are irrelevant to me as a Rails application developer; show me real-world application benchmarks with real-world data sets. There aren’t any such benchmarks yet, and they’re won’t be either until this thing is actually Ruby and Rails compatible.

Four: the magnitude of the benchmarks does strain the limits of credulity, probably because they’re microbenchmarks. If it sounds too good to be true, it probably is.

Five: a particularly vocal segment of the Ruby/Rails community has gone into its habitual delirium at the announcement of the newest and shiniest bauble, and is ready to entrust all of its production eggs to this new basket which was apparently whipped up in just three months.

There is only one reasonable conclusion from a rational assessment of all these facts: that while Maglev is certainly interesting and it may be quite promising, perspective is called for and people should be adopting a conservative “wait and see” attitude rather than a “let’s piss our pants with excitement” one.

There end the facts and the rest is all speculation and subjective opinion. For me, my own speculation and subjective opinion is that I am doubtful about this thing attaining 100% Ruby equivalence any time soon, I am sceptical about the real-world performance benefits compared to alternatives like YARV and JRuby (and I’m conservative in my deployment choices so I am not even thinking of switching to those from MRI at this stage, let alone switching to this newest kid on the block), and I’m not willing to sell-out to a commercial, closed-source platform in exchange for as-yet-unknown performance improvements, no matter how large they may turn out to be in the end.

Sho, I believe you still get some remaining time to pretend that you were joking, but not much.

Gemstone is the best thing that has happened to Ruby in years (Ruby 2.0 is … second). Period. ”

Sho. You are cooked by now. I warned you. Sorry. http://www.chadfowler.com/2008/6/5/maglev

From now on I suspect that the Ruby community will use the expression “It’s a Shoims” to point out an unjustified excess of skepticism about Ruby’s bright future.

You’re famous now. Kudos.

Well, initial thoughts about that benchmark is that it’s almost a perfect test of what’s bad about MRI. Object allocation and GC are famously inefficient in MRI and that script does practically nothing else; one could hardly find lower hanging fruit. Charles Nutter and “Ralf” make insightful comments along those lines on that post.

Futhermore, he leaves out the result for YARV, which is over 3 times faster than MRI on that test. I can’t understand why, Ruby 1.9.0 is the benchmark for “official” Ruby implemented with performance in mind.

Here’s the full list of results from the debian “shootout”. As you can see, Smalltalk (VW) does extremely well on that test; a mere 2.8 times slower than the fastest result, which is Java. Obviously the test favours a VM over “real” MM, and as discussed those speeds do not carry over into the general testing results.

That said, it is pretty impressive. Some of my own long-running AR import/dump scripts would benefit greatly from that kind of optimisation. I guess we’ll have to wait for the full tests to be released.

Wish I could just run the tests myself. I’m curious and *want* to do it. I hate this closed source BS where we’re all waiting on the word from one “official” tester.

UPDATE: I could swear those rankings now look quite a bit different than they did a day or two ago.

http://antoniocangiano.com/2008/06/05/maglev-handles-trees-like-a-monkey/

Maglev almost as fast as C/C++.

For a Java to ruby comparison just select the right menus from the web page you cited yourself :
http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=all&lang=yarv&lang2=java

Java beeing around 222 faster on the mandelbrot test and more than a 100 times faster on other tests.

(Deep sigh)

I still see a big lead in benchmarks where really the VM is running the code.
Other results are better for ruby because there’s more C code involved.
Of course you can can speedup “your” language by implementing almost all of your libraries in C, but that limits you in extending your libraries.

OK, I agree with you there. It’s better, generally, to pursue performance as much as possible within the context of the VM. More extensible, more centralised, less “moving targets”, more pure in general.

However, it’s the results that count, native C or VM or whatever. The vast majority of people do not need to extend Ruby in a way that would conflict with its mode of implementation.

And personally, the fact that the C implementation is fully open source is, to me, a powerful vote back in its favour. It may well be more difficult to extend some library if it’s implemented in C – it might also be completely impossible to do so on a proprietary VM like the one announced by Gemstone.

Who said that in (all) real world applications Ruby could be 15(insert your factor here) times faster?

The whole reason I wrote my article is because a great many people were claiming or implying exactly that. Gemstone’s presentation implied gains of between 15x and 60x. If they’d just claimed 2x or 3x or something, there wouldn’t have been such an explosion of hype and I wouldn’t have been incensed enough to write this post!

Also VW was not always the fastest Smalltalk implementation on the planet. Animorphic Systems had an implementation in the 90’s that was at least 5 times faster. Those guys went to Sun. You can see the result of their knowledge by comparing the Ruby results against Java. Java is still in certain cases more than 100 times faster than the (not yet released) Ruby 1.9.

[Citation Needed]

Here we go again with Smalltalk folks claiming huge performance numbers from “secret Smalltalk techniques”. “At least” 5x faster across the board would make Smalltalk, a dynamic language, faster than the most recent implementation of Java, which is statically typed!

Give me a break. Point to some hard numbers from a reliable source that back up your assertions or I’m going to continue with my assumption that you’re pulling these “facts” straight out of thin air.