How Amazon retail(Amazon.com) uses the AWS cloud
Amazon Web Services(AWS) is the subsidiary of Amazon.com. So, I always want to know how Amazon.com is making the efficient use of the services provided by the AWS cloud. To know this in detail I had done one case study and I’m writing about it in this article.
November 10, 2010 is the day Amazon.com turned off the last physical webserver in the Amazon.com data center for the webserver flit.
Amazon.com is the world's largest online retailer. As Amazon.com is generating lots of data, previously they were using the Oracle database for storing that data. But as Amazon.com grows larger, the size of their Oracle database continues to grow and this makes it extremely painful to backup the legacy Oracle database to the tape storage. This led them to take into consideration the Cloud services of AWS. By migrating to the AWS, they experienced a 12X performance improvement and reduced the restore time from around 15 hours to 2.5 hours.
Amazon.com is one of the user of the AWS
Before moving to how they are using AWS Cloud services, lets first understand the challenges they were faced before moving to the AWS.
Challenges:
Some of the challenges Amazon.com faced are:
- At that time, they were using tape storage for backing up the database. For this, they have to spend a significant amount of money on the tape hardware. Also, this tape hardware required a datacenter space and enterprise licensing fees for the tape software.
- For setting up this tape hardware, planning, and managing the tape infrastructure, they were required highly skilled employees.
- As they were doing a backup on the tape hardware, they must have to invest in the proprietory tape backup software. For Amazon.com, the cost of the software had been high and added significantly to overall backup costs.
- For storing the data durably on the tape, multiple copies are required. If there is a hardware failure, then human intervention is necessary to restore from tape.
- Backing up to the tape is much slower than the cloud.
Reason for choosing AWS and how Amazon.com uses AWS:
The value of moving to AWS for Amazon.com are
1. Allow Amazon retail engineers to move faster.
2. Lower the cost structure for Amazon retail.
3. Influence the AWS roadmap.
Amazon evaluated the Amazon S3 for backup and as a part of the evaluation, they considered security, availability, and performance aspects of Amazon S3 backup. Amazon.com also analyzed the cost-benefits of using the Amazon S3.
- It's important that the overall cost of the backup did not increase. Also, Amazon.com required faster backup and recovery performance. Amazon S3 cloud-based storage improved the performance and also cost-efficient.
- Amazon S3 has greater durability and availability. Amazon S3 is designed to provide 99.999999999% durability and 99.99% availability of objects over a given year.
- They determined that using Amazon S3 for backups was easy to implement because it worked seamlessly with Oracle RMAN.
- Amazon.com found that AWS met all of the requirements for strong data security.
Amazon.com was running its web servers in the EC2 on AWS. But the webserver calls back to the lots of dependencies that are still in the Amazon retail(Amazon.com) data center. And in addition, there are also backend databases in the Amazon retail data centers. Amazon needs to migrate the web servers, all of its dependencies, backend databases, and all of their tools. There are some products that enable enterprises to extend their data centers into AWS. So, the technical solution to migrating the webserver was to utilize the Virtual Private Cloud(VPC) product and the AWS Direct Connect products. It also allows enterprises to assign their own IP addresses to the EC2 instances. Amazon retail(Amazon.com) then used the AWS Direct Connect product to create a dedicated connection between the Amazon retail data center and their VPCs in the AWS.
Benefits that Amazon.com realized:
Every year Amazon.com is growing larger and larger. Using the Amazon S3, Amazon.com had totally eliminated the tape capacity planning. As Amazon is growing dynamically, it required the storage that can be used seamlessly and don’t required to plan. Before that, they were using tape which required to buy first and it will make the unnecessary investment. But, with the Amazon S3, Amazon.com only required to pay for the resources they had used. Also, with the Amazon S3, there were no worries of running out of the resources. For teams adopting Amazon S3 backups, the need for formal planning has been all but eliminated.
Immediate availability of data for restoring — no need to locate or retrieve physical tapes. When required to restore the data from the tape, then the data os first required to read from the tape using the tape backup software. As they are reading the data from the multiple tapes, the reading of data is slowed down. With Amazon S3 data can be restored much faster than that of with the tape.
Backing up a database to Amazon S3 can be two to twelve times faster than with tape drives. As one example, in a benchmark test, a DBA was able to restore 3.8 terabytes in 2.5 hours over gigabit Ethernet. This amounts to 25 gigabytes per minute or 422MB per second. In addition, since Amazon.com uses RMAN data compression, the effective restore rate was 3.37 gigabytes per second. This 2.5 hours compares to, conservatively, 10–15 hours that would be required to restore from tape.
Easy implementation of Oracle RMAN backups to Amazon S3. Directing Oracle RMAN backups to Amazon S3 requires only a configuration of the Oracle Secure Backup Cloud (SBC) module. The effort required to configure the Oracle SBC module amounted to an hour or less per database. After this one-time setup, the database backups were transparently redirected to Amazon S3.
Amazon S3 provides durable data storage, which is designed for 11 nines durability. Amazon.com had experienced the hardware failure with the tape multiple times. This includes the failure such as tapes that break, tape drives that fail, and robotic components that fail. This will dramatically increase the mean time to recover(MTTR). With the durability and availability of Amazon S3, these issues are no longer a concern.
Amazon.com had freed the human resources that included the technicians that required to install the tapes and manage them, the data center technicians. Amazon S3 allowed them to free up these specialists from day-to-day operations so that they can work on more valuable, business-critical engineering tasks.
Amazon.com had eliminated the cost of physical transportation of the tape to an off-site location. Also, they had completely migrated there webserver to the AWS.
Thanks for reading.
If you liked this article, please applaud it.
You can also follow me on Twitter at @cankush625 or find me on LinkedIn.
