Overview of Amazon S3 and Glacier

The Amazon S3 is a fully managed, highly available, autoscaled object/blob storage service that offers unlimited storage and 99.999999999% durability of data within a 24/7/365 regimen. This means that for every 10 million objects stored, you can expect to lose one object every 10,000 years.

The S3 service is addressable via standard HTTP PUT, GET, and DELETE calls, and has the built-in ability to deliver files and content via standard web services to users. This means the service offers the ability to serve static websites directly from the S3 environment in a serverless fashion by simply hosting an index file in the storage location of the content. All S3 content is stored in buckets, which serve as logical containers for files and need to be unique across all of AWS.

Since the service hosts only static content that needs to be retrieved by the client before it can be presented (which is the standard way of operation for any HTTP service), there is no support for server-side execution on S3. However, even with this limitation, S3 gives developers the ability to use client-side execution with programming languages such as Node.js, JavaScript, and Angular, among others. Moreover, developers are also able to utilize the change notification system built into S3 to trigger functions and push data to other AWS services, due to an action over an object or bucket on S3.

While the ability to serve content might be the defining feature for some companies, others see benefit in the ability to store an unlimited amount of data and having the ability to only pay for the data while it is needed and stored. This is especially valuable for companies that run big data and machine learning workloads. S3 can allow for massive amounts of data to be ingested. Once ingested, the work can be performed by consuming the data from S3 and then delivering the results of the workload either to another S3 location or another data target. Once the process completes, the initially ingested data can easily be removed with a few simple commands.

Also notable is the ability to programmatically address storage. All commands for updating and retrieving data can be issued through a standard programmable API. This capability gives developers and enterprises the ability to come up with innovative ways to address common enterprise data workflow needs and deliver full data cycle automation.

Amazon S3 is the most widely supported object storage system available and Amazon, other vendors, and independent developers are continuously coming up with new ways to use S3. Each object in the S3 environment also has metadata that carries information about the object, the storage class it's sitting on, and also custom metadata that we can use to extend the capabilities of S3. This is one of the tools in the S3 arsenal that we can use to build innovative solutions on top of S3.