A Dummies Guide to using S3 Storage classes to reduce storage costs
This article is part of my CloudForDummies series . If you are interested in other topics that I have covered on Cloud , please check out my other articles .
As part of this article , we will explore the various storage classes and also do a quick hands-on training on how to migrate objects between different classes using life-cycle policies.
I assume that you have some basic idea of what S3 is . If you don’t have I suggest you follow the article below which gives a brief overview of what S3 is
Nevertheless , a quick recap on S3.
S3 is an AWS Object Storage service that allows you to store just about anything on the AWS Cloud at very competitive pricing .
If you observed the pricing keenly , you can notice that the pricing gets halved if you go migrate to classes that offer a lower SLA . Natural right ?
Let’s have a simple explanation of each of the storage class.
S3 Standard :
The default storage class for any object .Key features include High Availability ( so low latency ) , automatic migration of objects to cheaper storage classes using Life-Cycle policies.
S3 Intelligent Tiering:
A hybrid variety offered by AWS which applies changes to storage class of an object based on the trends in access . Simply put, AWS monitors all your objects and automatically moves some of them to say a Glacier , others to an Infrequent Access and if required promote the same object back to an S3 standard . Key features include a small monitoring / auto tiering fee and same level of S3 SLAs.
S3 Infrequent Access:
Low Availability SLA 99% compared to S3 standard which is 99.9% . If you can live with it great . Key features include a cost of transfer to the class . When you move say 1 GB of data from S3 standard to S3 infrequent class you incur some charges in addition to the storage itself . Again very negligible compared to the competitive prices that it offers against S3 Standard.
S3 OneZone Infrequent Access:
This is slightly dangerous which should be only used for data that may never be used until there’s a post-apocalyptic situation . Because this class as the name suggests stores the data in just 1 availability zone . In the event let’s say you store the data in US-West with OneZone IA which have its data centers in Los Angeles and given the hollywood fact that Los Angeles is the first city to get attacked by apocalypse , nasty creatures from the sea , alien attacks you can safely assume that your data is gone .
Never to be used until the data is absolutely non-critical.
S3 Glacier:
Extremely cheaper compared to others . Key features include low availability and configurable retrieval times from minutes to hours . Useful for storing data to meet statutory / compliance requirements
S3 Glacier Deep Archive:
The cheapest option . Comes with a retrieval time of minimum 12 hours . This means in order to retrieve the object stored in this class you’ll have to wait for 12 hours after the request is made to retrieve it either in the console or CLI .
A great table to understand storage Classes in S3 along with their SLAs.
Now the question is why would you settle for a lower SLA ?.
Some reasons :
- May be the data in question is not needed for frequent access . So you can move it to an in-frequent storage class that charges you less.
- May be the data in question if required can have a lead time for delivery of upto 8–12 hours . You can move that data into a Glacier / Deep-Glacier archive .
- May be you can live without this data in the event there is a disaster but otherwise you want this data. You can store it in One-Zone infrequent access.
Ultimately , the nature of your data , frequency of access , availability of the data and redundancy requirements are the drivers for you to decide the right storage class.
A quick decision making process to choose an S3 class:
Observe that I have not used OneZoneIA , Intelligent tiering in this decision making process due to their nature of implementation.
It’s time for some hand’s on.
Step 1 :
Create a lifecycle policy for all objects in a bucket. I’m using a sample bucket that I have used to collect Covid19 stats during 2020. If you are interested in how I did this check out this article.
A quick introduction to LifeCycle Policies:
Simple automation that enables us to specify rules to move objects between different storage classes . Automation to migrate the objects is triggered by AWS at a pre-scheduled time and cannot be changed.
Select the Management tab and hit the “Create lifecycle rule” button.
Here you can get smart by applying simple rules to filter objects that need to be moved to a particular class . nifty right ?
Select appropriate actions . For our exercise select “Transition current version of objects”
And when you click this you will get the options below. Let’s select Standard-IA and set the Days after object creation as 30 days . Essentially we have now set a rule to move all objects to a Infrequent-Access class after 30-days since they were first stored in the bucket.
Rule of Lifecycle transition policies:
When you setup a lifecyle policy it will not trigger immediately . As per AWS Documentation Lifecycle rules run once a day at midnight Universal Coordinated Time (UTC). So don’t expect your objects that are already 30days old to move into the new storage class as soon as you setup.
Can I see the transition immediately ?.
You can by manually moving some objects to a different storage class .
Step 2 :
Let’s learn how to migrate some objects to a different storage class.For this example again let’s use the same example bucket that I have. As you can see in this image all of these files are in Standard class and I want some of them to be moved to an OneZone-IA ( I know )
Select the objects that you want to migrate and hit on the Actions dropdown
Now you’ll get the following options . Select OneZone-IA . And do not miss hitting the “Save Changes” button at the bottom of the page.
Once you do that you can see that the storage class has changed immediately for these objects .
That’s it for now . Stay tuned .
You can slide to subscribe to my news letter .
