Forcing Gzip Compression

Andy Martone (Google)
Web Performance 209/210
Average rating: ****.
(4.16, 31 ratings)

Tony Gentilcore’s Beyond Gzipping presentation at Velocity 2009 (http://en.oreilly.com/velocity2009/public/schedule/detail/9072) identified that ~15% of Internet users receive uncompressed responses. This is mostly due to proxies or client security software stripping or mangling the Accept-Encoding header from the web browser.

RFC 2616 states:
If no Accept-Encoding field is present in a request, the server MAY
assume that the client will accept any content coding. In this case, if “identity” is one of the available content-codings, then the
server SHOULD use the “identity” content-coding, unless it has
additional information that a different content-coding is meaningful
to the client.

Google web search now tests these browsers’ ability to understand compressed content. If the test is successful, Google serves gzipped content. This has significantly lowered latency for users at the tail end of the latency distribution.

Andy Martone

Google

Andy Martone is a software engineer at Google. He works on the websearch infrastructure team, trying to make Google search even faster.

Andy holds a Master’s degree in Computer Science from George Mason University.

Comments on this page are now closed.

Comments

Andy Martone
07/13/2010 9:41am PDT

@Brian Gibson: I’m not an expert on HTTPS behavior through proxies by any means, though it would be possible for a man-in-the middle HTTPS proxy to modify request content.

The nice thing about this technique is that it should silently fail if the test doesn’t succeed.

Picture of Brian Gibson
Brian Gibson
07/05/2010 4:21am PDT

Hi Andy, I attended your session about forcing GZIP compression at Velocity conference. I found it really informative and came away thinking this could potentially be a huge issue for my company’s product, where all our customers will be accessing it from educational establishments behind firewalls and proxy servers.

After thinking about if for a while I wanted to ask you – Do you think that websites that use HTTPS exclusively are exempt? The reason I am thinking is that local proxies and firewalls will not see the Accept-Encoding header in HTTPS traffic as it is encrypted and therefore it cannot be stripped?

Thank you, Brian

Picture of Shanti Subramanyam
Shanti Subramanyam
06/24/2010 9:08pm PDT

Really cool !

Picture of Mark J. Levitt
Mark J. Levitt
06/24/2010 3:49pm PDT

@Jijoe, the slides were recently posted! See above (or here).

Jijoe Vurghese
06/24/2010 11:10am PDT

Great presentation, Andy. Please post the slides when you get a chance.

Picture of Ryan Witt
Ryan Witt
06/23/2010 5:04pm PDT

Thanks so much for doing a follow-up! Hopefully this detailed how-to will spur more implementations.

Picture of Steve Souders
Steve Souders
05/18/2010 4:11pm PDT

This is an amazing follow-up on Tony’s Beyond Gzipping preso from last year. Who had any idea that 15% of browsers that support gzip ARE NOT GETTING COMPRESSED RESPONSES! Andy’s team took the next step to identify browsers that could accept gzipped responses successfully, and quantified the performance win. It’s big. Come hear the numbers.

For Velocity China sponsorship information for companies outside China, contact Yvonne Romaine at yromaine@oreilly.com.

  • Google
  • Strangeloop
  • Yahoo! Inc.
  • Dyn Inc.
  • Facebook
  • Schooner Information Technology
  • Tilera
  • AlertSite
  • AppDynamics
  • Aptimize
  • CDNetworks
  • Circonus
  • Cloudscaling
  • Clustrix
  • Coradiant
  • Dell
  • DTO Solutions
  • MaxiScale
  • Neustar
  • Nokia
  • NorthScale, Inc.
  • Shopzilla
  • Splunk
  • Virident
  • Zoompf
  • Neustar

For information on exhibition and sponsorship opportunities at the conference, contact Yvonne Romaine at yromaine@oreilly.com

Download the Velocity Sponsor/Exhibitor Prospectus

Download the Media & Promotional Partner Brochure (PDF) for information on trade opportunities with O'Reilly conferences or contact mediapartners@ oreilly.com

For media-related inquiries, contact Maureen Jennings at maureen@oreilly.com

To stay abreast of conference news and to receive email notification when registration opens, please sign up for the Velocity Conference bulletin (login required)

View a complete list of Velocity contacts