Instrumenting the real-time web: Node.js, DTrace and the Robinson Projection

Bryan Cantrill (Joyent, Inc.)
Operations Ballroom ABCD
Average rating: ****.
(4.93, 15 ratings)

In the fall of last year, Joyent hosted the first node.js programming contest, Node Knockout. The team-based, all-weekend programming contest—in which each team was given a virtual machine optimized for node.js—afforded an early opportunity to experiment with novel mechanisms of system instrumentation and visualization in a dynamic, server-side JavaScript environment. To explore these opportunties, we developed a public leaderboard for the contest, consisting of DTrace-based instrumentation for node.js in a virtualized, multi-tenancy environment; a node.js-based architecture to aggregate data across machines and present it in real-time; and mechanisms for visualizing data in a scalable and repsonsive fashion. Over the course of developing, deploying and using the leaderboard, we learned about not only about the behavior of the contestant code instrumented, but also the challenges of meaningfully instrumenting asynchronous, event-oriented virtual machines—and of visualizing the resulting data.

This talk will give a brief background on node.js and DTrace, discuss how they intersected for Node Knockout, what was learned about the strengths and limitations of each technology, and how our experiences informed the development of a distributed facility for DTrace-based instrumentation in the cloud. We will also explain how our thinking around systems instrumentation was shaped by the contestants of the contest themselves—and in particular, their propensity for developing novel real-time web applications. We will describe the latency-oriented instrumentation and visualization facilities that have stemmed from this thinking, and present data that we have gathered and visualized on production loads—data that shows heretofore unseen nuance in system performance. Finally, we will close with the open challenges that remain for meaningfully instrumenting, understanding and optimizing the emerging real-time web.

Photo of Bryan Cantrill

Bryan Cantrill

Joyent, Inc.

Bryan Cantrill is VP of Engineering at Joyent. Previously a Distinguished Engineer at Sun Microsystems, Bryan has spent over a decade working on system software, from the guts of the kernel to client-code on the browser and much in between. Bryan led the team that designed and implemented DTrace, a facility for dynamic instrumentation of production systems that won the Wall Street Journal’s top Technology Innovation Award in 2006 and the USENIX Software Tools User Group Award in 2008. Bryan co-founded the Fishworks group at Sun, where he designed and implemented the DTrace-based analytics facility found in the Sun Storage 7000 series of appliances — a facility that InfoWorld described as “stunning” in a February 2009 review. In 2005, Bryan was named by MIT’s Technology Review as one of the top thirty-five technologists under the age of thirty-five, and by InfoWorld as one of their Innovators of the Year. Bryan received the ScB magna cum laude with honors in Computer Science from Brown University.

Comments on this page are now closed.

Comments

Picture of Suzanne Axtell
Suzanne Axtell
06/23/2011 9:05am PDT

Hi Tim, Bryan’s preso will be available to All Access Pass holders in about a week. Glad you enjoyed it! Suzanne

Tim Prendergast
06/21/2011 2:00pm PDT

Velocity Crew—please make the video of this available if at all possible. The diatribes during the presentation were hilarious and engaging at the same time, and we thoroughly enjoyed the content.

Picture of Eric McCraw
Eric McCraw
06/16/2011 4:31pm PDT

Bryan,

Great presentation, thank you! Can you get your slides in a downloadable state asap? I would like to share it with colleagues unfortunately were unable to attend…

  • Keynote Systems
  • Cisco
  • Google
  • Neustar
  • Betfair
  • Cotendo
  • Rackspace Hosting
  • Akamai
  • Apica
  • dynaTrace
  • Equinix
  • Facebook
  • New Relic
  • Opscode
  • Salesforce.com
  • Yahoo! Inc.
  • AppDynamics
  • Aptimize
  • Blaze
  • CDNetworks
  • Cedexis
  • Citrix Systems
  • Compuware Corporation
  • Dyn Inc.
  • F5 Networks
  • Heroku
  • Percona
  • Quest Software
  • Schooner Information Technology
  • SiteSpect
  • Splunk
  • Strangeloop
  • WatchMouse
  • Zeus Technology
  • Neustar

Sponsorship Opportunities

For information on exhibition and sponsorship opportunities at the conference, contact Yvonne Romaine at yromaine@oreilly.com

Download the Velocity Sponsor/Exhibitor Prospectus

Contact Us

View a complete list of Velocity contacts