Core Storage Services of AWS
Cloud Storage is one of the core services of AWS and also one of the most useful services that we use. Cloud storage can be used to store backups, share data, etc. The most common example of cloud storage is Google Photos. Many people use Google Photos to create a backup of their media files so that they can retrieve them anytime, anywhere, delete media files from their device if no longer needed or there is no extra space in the device.
AWS also provides cloud storage to store data, metadata, create backups, etc. In this blog, I’ll be giving an overview of five different types of storage services based on their usage:
- Amazon EBS (provides block storage)
- Amazon EC2 Instance Store (provides temporary block-level storage)
- Amazon S3 (provides object storage)
- Amazon EFS (provides file storage)
- Amazon S3 Glacier (used for data archiving and long term backup)
Amazon Elastic Block Store (Amazon EBS):
Amazon Elastic Block Store or Amazon EBS as the name suggests offers block storage/volumes to use with the EC2 instances. It is persistent, mountable storage that can be mounted as a device to an Amazon EC2 instance.
Persistent/Non- volatile Storage: Storage that retains data even after the power to that device is shut off is called persistent or non-volatile storage. Amazon EBS is an example of persistent storage. Due to this reason, the root volume of an instance should always be stored on Amazon EBS so that when an EC2 instance stops and starts again, data doesn’t get lost.
Block storage: Amazon EBS provides block storage, which means that it stores data in the forms of blocks, so to change a specific character in a file, you only change the particular block in the file that contains that specific character whereas, in object storage, you’ll have to re-upload the entire modified file again.
This also affects the throughput, latency, and cost of the storage. Block storages are typically faster and use less bandwidth, but can cost more than object-level storage.
More features of Amazon EBS include:
- Availability: Amazon EBS volumes are automatically replicated within the availability zone to protect from component failure. They are designed for high availability and durability.
- Point-in-Time Snapshots: The backup of an Amazon EBS volume is called a snapshot. Amazon EBS enables you to create point-in-time snapshots. The first snapshot is called baseline snapshot. All snapshots created after the first snapshot capture only what is different from the previous snapshot. These snapshots are stored in Amazon S3 and automatically replicated across multiple availability zones. These snapshots can be used as a starting point for a new EBS volume, shared with coworkers, and other developers.
- Elasticity: Amazon EBS volumes can increase capacity and change to different types (SSDs and HDDs) without the need to stop the instances.
Some common uses: store root volume, database hosts, etc.
Amazon EC2 Instance Store:
Amazon EC2 Instance Store provides ephemeral (temporary), block-level storage for an EC2 instance. This storage is located on hard disks that are physically attached to the host computer. If the instance stops, data stored on the Instance Store will also be deleted. Therefore, it is not all recommended to storing the root volume of an EC2 Instance on Amazon EC2 Instance Store. It can be used to store temporary information that changes frequently, such as buffers, caches, scratch data, etc.
Amazon Simple Storage Service (Amazon S3):
Amazon S3 provides object-level storage where each file becomes an object and can be accessed from anywhere in the world through a URL. Here, data or files are stored as objects in a bucket (logical containers for objects). When you upload a file in the bucket, it becomes an object. An object contains the file and metadata of that file (e.g. owner, tags, etc.). Bucket names that you choose must be unique across the world and by default, none of the data is shared publicly. Data in the bucket is redundantly stored across multiple facilities within your chosen region, hence making them durable. Amazon S3 also provides all the 11 9’s of durability.
S3 provides the following storage classes:
- Amazon S3 Standard: It is a commonly used object storage for frequently accessed data. It is used for daily applications, and a variety of use cases, including dynamic websites, content distribution, mobile and gaming applications, and big data analytics.
- Amazon S3 Standard Infrequent Access (Amazon S3 Standard IA): This storage class is used for data that is accessed less frequently, but requires rapid access when needed. It costs less than Amazon S3 Standard and is good for long-term storage, backups, and as a data store for disaster recovery files.
- Amazon S3 One Zone Infrequent Access (Amazon S3 One Zone IA): Amazon S3 One Zone IA is similar to Amazon S3 Standard IA i.e. it is used for storing data that is less frequently accessed. But, the difference here is that stores data in a single availability zone and costs less than Amazon S3 Standard IA. All other Amazon S3 storage classes, stores data in a minimum of three availability zones, whereas, Amazon S3 One Zone IA stores data in a single availability zone. It can be used for storing secondary backup copies of on-premises data.
- Amazon S3 Intelligent Tiering: It is specifically designed to optimize costs by automatically moving data to the most cost-effective access tier. For e.g., it will move the objects that have not been accessed for 30 consecutive days to the infrequent access tier. Similarly, if an object in the infrequent access tier is accessed, it will automatically move it back to the frequent access tier.
- Amazon S3 Glacier: Amazon S3 Glacier storage class is primarily used for data archiving. It is a secure, durable, and low-cost storage class.
- Amazon S3 Glacier Deep Archive: It is the lowest cost storage class of Amazon S3. It supports long-term retention and digital preservation for data that might be accessed once or twice a year. It can also be used for backup and disaster recovery use cases.
Pricing of Amazon S3 depends on the following:
- Type of storage class
- Amount of storage
- Number/type of requests (GET, PUT, COPY, etc.)
- Data Transfer (data transfer in is free, charges are for data transfer out)
Some common uses of Amazon S3: backup storage, application hosting, media hosting, software delivery, etc.
It is also used to store snapshots of EBS volumes.
Amazon Elastic File System (Amazon EFS):
Amazon Elastic File System provides file storage. It is a shared file system that multiple EC2 instances can mount at the same time. It is scalable, highly durable, and highly available.
With Amazon EFS, you can create a file system, mount the file system on an Amazon EC2 instance, and then read and write data from to and from your file system.
Some common uses: build a file system for big data and analytics, content management, web serving, home directories, etc.
Amazon S3 Glacier:
Amazon S3 Glacier is a secure, durable, and extremely low-cost cloud storage service for data archiving and long-term backup.
Archive: An archive is a base unit of storage in Amazon S3 Glacier and it can be any object like a photo, video, file, etc. that you store in S3 Glacier.
Vault: The archives are stored in a container called a vault. Each vault has a vault access policy and a vault lock policy. Vault Access Policy tells who can and cannot access data stored in volume and what operations users can or cannot perform. If a Vault Lock Policy has been enabled for a vault, then that vault cannot be altered.
There are three data retrieval options in S3 Glacier:
- Expedited: Data is available in 1-5 minutes (highest cost)
- Standard: Data is available in 3-5 hours (less expensive than expedited, more expensive than bulk)
- Bulk: Data is available in 5–12 hours (least expensive)
You can directly upload objects to S3 Glacier or use lifecycle policies to cycle data at regular intervals between different Amazon S3 storage classes and S3 Glacier automatically. Amazon S3 Glacier is much cheaper than Amazon S3. Here, data is automatically encrypted using AWS KMS.
Some common uses: Media Asset Archiving, Scientific Data Archiving, Healthcare Information Archiving, Digital Preservation, etc.
