Ruby on Rails expert: Rails scales; Twitter shouldn't taint Ruby

Ruby on Rails expert: Rails scales; Twitter shouldn't taint Ruby

Summary: Ruby on Rails, a rapid-fire Web development stack, has been under fire of late. Twitter outages have led a fairly loud chorus of folks singing the "Ruby doesn't scale" song.

SHARE:

Ruby on Rails, a rapid-fire Web development stack, has been under fire of late. Twitter outages have led a fairly loud chorus of folks singing the "Ruby doesn't scale" song.

ezra.jpg

But is it really fair to make the Ruby-Twitter connection, which by the way was hit with another outage (Techmeme). In a blog post, Twitter said:

Downtime is not good. We caused a database to fail during a routine update early this afternoon. We switched to a replica and expect this recovery to take place quickly. We're all working on it and watching right now as Twitter gets back up to speed. We have a thread open on our support forum which we'll update when we have more details to share. Getting our act together is something we continue to work on as we grow our company and our service.

I spoke to Ezra Zygmuntowicz (right), co-founder and director of software engineering at Engine Yard, about Ruby on Rails and the Twitter connection. Zygmuntowicz, who built one of the first production Rails applications at the Yakima Herald in 2004, was plugging his recently published book: "Deploying Rails Applications: A Step-by-Step guide." The book was rewritten two or three times (it's hard to write a book about a platform where deployment techniques change every six to eight months).

Here are some highlights from our conversation:

On whether Ruby on Rails can scale:

Zygmuntowicz was adamant that Ruby on Rails, which speeds up time to market because it has a stack template, controllers, a model layer with database connections. I've interviewed dozens of folks asking them how do you scale and the back and forth on Ruby was telling. There seems to be little middle ground on the issue. Zygmuntowicz had a simple take: "Languages don't scale. Architectures scale." "I'm of the opinion is that Ruby does scale," he said, adding that Ruby isn't any different than PHP or a Java deployment.

On Twitter (all resources):

Zygmuntowicz acknowledged that Twitter is the "poster child for Rails" and its outages have stoked the idea that Ruby on Rails can't scale. However, Zygmuntowicz noted that many scaling problems intersect with database issues, which echoes comments from other scalability experts I've talked to. "Twitter's problem has nothing to do with Rails. It has mostly to do with the size of the social graph and the way they've done their database," said Zygmuntowicz. "Languages don't have scalability built in." Translation: Twitter allows folks to have unlimited friends. So the friend of a friend go round taxes that databases because every message, follower, etc. is a field update.

"It is kind of unfortunate that everybody blames Twitter's problems on Ruby on Rails. Ninety percent of Twitter traffic is API calls, not Web traffic. Twitter is facing a completely different problem," Zygmuntowicz, who noted that Engine Yard hosts 500 to 600 Ruby applications without scale problems.

On where the bottlenecks reside in Rails applications:

"The database is the bottleneck well before Rails," said Zygmuntowicz. Why? Rails features many modules that can eliminate a lot of database grunt work, but it only takes you 80 percent of the way there. The other 20 percent will require database coding. "Rails does insulate programmers from SQL but if you get too lazy with the object relationship you will get into trouble," said Zygmuntowicz. "As the site grows you will want to hand optimize the database. That's true for any language."

On the big benefit of Ruby on Rails:

Zygmuntowicz said the biggest benefit of Ruby on Rails is that "it will get you to market faster than any framework out there." The rub: That time to market will cost you less development time, but once it's in production it may require more servers. "That's a good tradeoff," said Zygmuntowicz. "Hardware is cheap and developers are expensive." I asked Zygmuntowicz what applications made the most sense for Ruby on Rails. He said Ruby shines for social networks and content presentation. It can also work for financial applications, but it can stumble if it has to "interface with a legacy database framework." "That can be a pain point," he added.

On his three Ruby on Rails lessons from his deployments:

  • "Ruby will take you 80 percent of the way there, but it is not an excuse for not having an actual architecture that will scale. That's the same for any other framework. You can't hope that the language will scale for you. Look at Friendster--it dropped out of the race because couldn't scale and they were in Java. Ruby on Rails makes everything so easy that you can get trapped into thinking it'll do everything for you. You still need to consider scalable Web architectures."
  • Divorce long running queries. Ruby on Rails typically serves one request at a time and that's a detriment if there are long running queries that block other incoming requests, said Zygmuntowicz. The workaround is that you have to set up a queuing mechanism so your database isn't overloaded.
  • You will have to do some database deep down coding. Ruby on Rails doesn't fix everything. In a staging environment everything will look fine, but once you go production you have to monitor the number of queries and the reaction time. "As you start to grow you will chew up some memory," said Zygmuntowicz. "The fix is to drop down into raw SQL coding to do the queries."

"Ruby on Rails is just as scalable as any other platform, but there's no free lunch," said Zygmuntowicz. "Ideally, Rails gets you there quicker and gives you time to think about how you scale."

Topics: Social Enterprise, Software Development

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

10 comments
Log in or register to join the discussion
  • Ruby and hype

    There is lot of hype around Ruby. But it has not proved itself yet.
    Van Der
  • RE: Ruby on Rails expert: Rails scales; Twitter shouldn't taint Ruby

    All it takes is a SQL that isn't using primary keys that results in a full table scan (sequential vs keyed reads) or a SQL that is not correlated or, God forbid, a dumb a$$ who accidentally invokes a SQL with a Cartesian product and your nice looking website is hosed.

    That would apply to any web site with any kind of db back-end.
    D T Schmitz
    • Actually...

      It *does* depend on the database platform.

      For example, in PostgreSQL and Oracle (both of them), readers and writers never block each other. Thus, in a query-intensive environment, either of these database platforms would likely eliminate most of these types of issues "automatically".

      Not that no "tweaking" would be needed to get the code correct... but the choice of database platform really does matter here.
      fde101
      • SQL.92

        SQL.92 is a standard as you know, but each vendor's implementation differs--just run a SQL explain to see how your query gets optimized. Sometimes you'll find the results are less than optimal. This has been my experience with Oracle and Informix and if you haven't made your schemas express the primary keys that make binary search possible, then one vendor's engine might optimize and create temporary keys--another's might not, in which case you have a full table scan on your hands and some vendor's will throw a table or row-range lock while that occurs. Everything stands in the queue in the lock releases.

        Table locks and row range locks are common in financial applications where a snapshot 'static' database needs to be audited. These are 'transactional' types of cases so I would beg to differ with you about which database platform matters.

        Not all are alike.
        D T Schmitz
  • RE: Ruby on Rails expert: Rails scales; Twitter shouldn't taint Ruby

    Oh, so it's not RoR?

    Bitchin.

    That really helps solve the prob.

    Any and all Twitter downtime totally allows startup pimps (e.g., me and everyone who runs to TechCrunch during the dowtime bitchfests) to cannibalize traffic, diverting users to other chat/microblogging sites. And I love Twitter enough to actually feel guilty about kickin' it on Chatterous when Twitter's down.

    So, if you're going to pass the buck, saying the outtages haven't RoR to blame, you really have to give a solution: How do you keep Twitter ON 24/7?
    JolieODell
  • RE: Ruby on Rails expert: Rails scales; Twitter shouldn't taint Ruby

    I'm not a web developer but it seems like the twitter crowd is blaming the wrong people. Shouldn't the scalability be up to the developers and not the language they are using? Seems like its poor planning on the developers part rather than saying its ruby.
    Loverock Davidson
    • Shouldn't matter...

      ...but there may be overhead issues (how efficient is the interpreter?). Personally, I know nothing about either Ruby or Rails, so I can't comment further.
      John L. Ries
  • Rails isn't a language

    It's all very well to say "Languages don't scale, architectures scale," but Rails isn't a language, it's a framework. Nobody is blaming Ruby for Twitter's problems, they are blaming Rails.
    JFPSF
    • Rails isn't a language

      I found that confusion around rails as a language and ruby as a framework to be telling. It was hard to take any of the argument seriously after that.

      I have used Rails in production and my personal experience is that I found scaling problems to be rife.

      On the plus side, the Active Record ORM is a thing of inspired elegance; Rails is great for prototyping and for low volume websites --think vanity sites, small business sites, intranet, maybe a winery or a rock and roll band, you get the idea.
      leed25d
  • RE: Ruby on Rails expert: Rails scales; Twitter shouldn't taint Ruby

    I think Ruby, as the actual language can scale. If somebody
    was to create a fastcgi (or similar) method of accessing Ruby
    (kind of like php), with an opcode cache, Ruby would scale
    just as well, as php applications. I don't think it would be too
    hard to port Rails over to work in a cgi environment, as there
    are plenty of php based frameworks that are similar to rails.
    seananderson