ApacheCon 2016

Yahoo homepage is one of the most popular destinations on the internet. It is crucial to our users for the site to be available all the time. In this presentation, we will talk about the failsafe mechanism in Yahoo homepage to help improve the site availability. It involves using Apache Storm Crawler to crawl the site and send static HTML and assets to 3rd party cloud object storage. Then on the serving side we use Apache Traffic Server as reverse proxy in front of our origin servers and custom ATS plugin is used to catch various problems from the origin server and continue to serve the users with static contents from the cloud object storage. We will talk about the details of this solution with in-depth information on ATS plugin and Storm spouts/bolts. We will also talk about our future plan of using Apache libcloud and jclouds to avoid cloud storage vendor lock-in for this solution.