When correctly architected, horizontal partitioning of data can lead to dramatically better fault tolerance and isolation, while significantly improving the manageability and flexibility of a web site. A well-sharded MySQL architecture can perform comparably to newer NoSQL solutions, while still offering the expressiveness of SQL and the time-tested robustness of InnoDB.
The road to complete sharding of Tumblr’s primary data has been a long one, and I’ll discuss how, when, and why we transitioned from a single database, to several functionally-partitioned datasets, and finally to a massively horizontally-sharded architecture.
Along the way, we’ve developed a set of tools that has made the sharding process significantly easier to execute and monitor. The bulk of this session will focus on the practical aspects of sharding, digging into the nitty-gritty of the tools we use to:
I’ll also cover some of the mistakes we’ve made, and the tools and tricks we used to gracefully back out of them, such as choosing the wrong shard sizes, and fixing uneven distribution of queries and data.
The session will broadly appeal to engineers at rapidly-growing, MySQL-backed sites. The information presented is all drawn from first-hand experience, and mostly cannot be found in existing books or web sites. Some level of audience familiarity with MySQL is assumed, but previous experience with sharding, InnoDB internals, or MySQL replication is not required.
Evan is a senior engineer at Tumblr, where he develops database solutions that scale to tens of billions of rows spanning over 100 MySQL installations. He has previously worked at Six Apart and Harvard University, and his software is used by major publications such as The New Yorker and Consumer Reports. He is intimately familiar with the internals of four major blogging platforms, and has been developing online software since the days of dial-up bulletin board systems.
For information on exhibition and sponsorship opportunities at the conference, contact Gloria Lombardo at email@example.com
View a list of Velocity Europe contacts