ApacheCon 2016 has ended
Back To Schedule
Friday, May 13 • 3:45pm - 4:35pm
Failsafe Mechanism for Yahoo homepage using Apache Traffic Server and Storm Crawler - Shu Kit Chan & Pushkar Sachdeva, Yahoo

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Yahoo homepage is one of the most popular destinations on the internet. It is crucial to our users for the site to be available all the time. In this presentation, we will talk about the failsafe mechanism in Yahoo homepage to help improve the site availability. It involves using Apache Storm Crawler to crawl the site and send static HTML and assets to 3rd party cloud object storage. Then on the serving side we use Apache Traffic Server as reverse proxy in front of our origin servers and custom ATS plugin is used to catch various problems from the origin server and continue to serve the users with static contents from the cloud object storage. We will talk about the details of this solution with in-depth information on ATS plugin and Storm spouts/bolts. We will also talk about our future plan of using Apache libcloud and jclouds to avoid cloud storage vendor lock-in for this solution.


Shu Kit Chan

Software Engineer, Self Employed

Pushkar Sachdeva

Software Engineer, Yahoo
Pushkar is a Software Engineer at Yahoo!. He is the member of Yahoo's Media Edge team and loves building new and interesting plugins at the proxy layer. He has presented his work about content stitching at ATS titled 'Assembly at the Edge' in Velocity Conference' 14 and likes to... Read More →

Friday May 13, 2016 3:45pm - 4:35pm PDT
Plaza A