Fixing Twitter: Improving the Performance and Scalability of the World's Most Popular Micro-blogging Site

Twitter (twitter.com) is a micro-blogging service that allows users to answer the question “What are you doing?”. It takes a simple 140 character string (a tweet), and distributes that to a user’s following (or “friends”) list.

Over the last few years, Twitter has experienced a variety of challenges scaling its site to serve millions and millions of users. From hosting issues to Ruby on Rails, we’ve met many scaling challenges and would like to share some of our experiences with the community.

We will attempt to cover:

• Best practices for deploying Ruby on Rails in production
• Performance impacts of abuse and why rate limiting is so important.
• What to do when your API use overtakes normal web use
• Asynchronous versus Synchronous processing during request lifetime
• Why disk is the new tape (or, how social media isn’t possible without cheap memory)
• Caching methodologies and Twitter’s open source efforts (Scarling, CacheMoney, Libmemcached)
• Why databases are not the best solution for all problems.
• Message queues
• Handling large logs with Thrift
• Incremental scaling improvements: Find the weak point, fixing it, find the next one, repeat.

Talk may include additional speakers from Twitter, pending management approval.