35,000-Core Ubuntu Farm Renders Avatar

January 18, 2010


I just attended Paul Gunn‘s talk at LCA2010, entitled:

  • Challenges in Data Centre Growth (or, “You need how many processors to finish the movie???”)

Paul is a Systems Administrator at Weta Digital, a Wellywood digital effects studio here in Wellington, New Zealand. Check out some of the feature films that Weta Digital has worked on, and I think you’ll recognize a few. District9, Day the Earth Stood Still, Jumper, King Kong, Lord of the Rings, Fantastic Four, Eragon, X-Men, i-Robot. Wow!

It was a great talk, about the type of data center needed to render special effects in today’s blockbuster movies. They have a 2 Petabyte disk array, 10gbps networking, and 35,000 cores (4,000+ HP blades) in their data center, and still it takes 48 hours to render some of their graphic sequences.

According to Paul, Ubuntu is at the core of all of this, running on all of the rendering nodes, and 90% of the desktops at Weta Digital. He notes that his farm (he calls it a “render wall”) is in fact an Ubuntu Server farm, and not RHEL as he has seen reported in the media.

Here’s a couple of articles on Weta Digital’s data center and their work on Avatar:

:-Dustin

10 Responses to “35,000-Core Ubuntu Farm Renders Avatar”

  1. Tom Says:

    Wow cool!

    They also did Avatar, which is on route to becoming to most successful movie of all time (just by $ without inflation, but still)

    COOL info. Will the video of the talk be available?

  2. Dustin Kirkland Says:

    Tom-

    Yes, it was recorded. Stay tuned to the “Media” section of the LCA2010 website.

    :-Dustin

  3. OffBeatMammal Says:

    so the rumours it was all done on a couple of WinXP boxes running After Effects was a myth 🙂

    the mind boggles at that amount of computing power

  4. brews Says:

    Dustin… you da man.

  5. nijaba Says:

    Very cool info, Dustin. Thanks for reporting this.

  6. jonathan Says:

    I want a 35000 core Ubuntu cluster. (My birthday is in 3 weeks if anyone is wondering)

  7. magicfab Says:

    “Oh, sure, let me reproduce that, it’s a known bug…”

    I wonder what internal vs. external resources they needed to build that or to get to a confidence level where they had a go to use it.

  8. ErickPardus Says:

    Try it with a PARDUS 2009.1 Cluster…..;-)


Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: