Every month at SEOmoz we serve nearly a hundred million requests from a dataset of web pages and links that grows by over 50TB every six weeks. In this session I’ll explore the performance indicators we watch to monitor our Amazon Web Services-based web and link data services API.
Latency and throughput are mission critical to us. We’ve had a significant challenge in understanding the performance of our AWS components (EC2, S3) and our traditional components (e.g., memcache, lighttpd). When we began our project over two years ago, many of these components were brand new. In that time we’ve gained expertise about measuring and testing the performance of a cloud-based application, using low-cost or free software tools, and common sense.From this session the audience should learn about:
In this session I’ll show our live performance dashboard illustrating several dimensions of any production system’s performance. I’ll briefly discuss how these dimensions interact in general and in an AWS-based architecture. I’ll point out what these indicators say about capacity planning.
Throughout this discussion of performance indicators, I’ll present out-of-the-box, free software tools that you can use to collect, analyze, and present this kind of data. And I’ll show how they work together to provide a manageable monitoring solution for a cloud-based application.
Nick is one of SEOmoz’s in-house software architects. He received his graduate degree from Cornell University in Data Base Management Systems and Information Retrieval. He’s worked on the joint Cornell University/Internet Archive venture, the Web Lab, a database abstraction to the world wide web over time.
Nick works at SEOmoz on the Linkscape project, researching and developing internet marketing applications of page authority metrics such as PageRank and StaticRank, spam and trust quantifcation systems such as TrustRank, and large-scale web datasets.
The Linkscape API serves nearly 100 million requests per month over a web-scale dataset that grows by over 50TB every six weeks.
Comments on this page are now closed.
For information on exhibition and sponsorship opportunities at the conference, contact Yvonne Romaine at firstname.lastname@example.org
Download the Velocity Sponsor/Exhibitor Prospectus
For media-related inquiries, contact Maureen Jennings at email@example.com
To stay abreast of conference news and to receive email notification when registration opens, please sign up for the Velocity Conference bulletin (login required)
View a complete list of Velocity contacts