AWS CLI Commands For AWS S3 Services

AWS S3 is stands for AWS Simple Storage Service which provides facility to store any data on cloud and access anywhere on the web. AWS S3 store data as object within bucket. Bucket is logical grouping of objects (Files/Folders). S3 provides three operations on the bucket objects i.e Read, Write and Delete.

So we will see here how we can perform these operations using AWS CLI commands.

When we create or write any file on S3 bucket and if we want to read it from S3 bucket, S3 service provides us a URL of that particular file rather than that file. So we can use that file URL anywhere on the web applications.

#Following are the features of AWS S3 Services:

• Internet Accessible Storage
• Region Level Storage
• Almost Unlimited Scalability
• No File Locking Mechanisms
• Support REST & SOAP API's
• 99.9999% of durability of object stored
• 99.99% Availability Of Services
• Bucket must have unique name

>> aws s3 help

Once you execute this command, it will show you all documentation related to s3 services. We can use this documentation and do read, write and delete operations.

Lets see and do one by one...

#Create Bucket:

>> aws s3 mb s3://mynewbucket

We can use above command 'mb' to create new bucket but make sure bucket name must be unique universally on s3 service.

#How to upload new object on aws s3 bucket:

For example, we have one test.txt file which is on server root. Lets see how we can copy this file to s3 bucket.

>> aws s3 cp test s3://mynewbucket

Once you execute above command it will copy 'test' file to s3 storage bucket 'mynewbucket' which we created.

You can verify this copied file from AWS Management Console.

S3 service provide multiple options to upload any files on bucket via parameters 'storage-classes'.

Below are the options provided by 'storage-classes':

#STANDARD Storage Class: 
This is default storage class, which is usage for dynamic websites, mobile and games apps, cloud applications, content distribution and big-data analysis. Because it provides high durability and availability for frequently access data which is required less frequently but required rapid access when needed.

#STANDARD_IA Storage Class: 
Standard IA (Infrequent Access) offers high durability and low latency as compared to STANDARD Storage Class which provides low storage price and high performance. It makes STANDARD_IA Storage ideal for long term storage backups and data store for disaster recovery.

#REDUCED-REDUNDANCY Storage Class (RRS):
It is an S3 storage option which provide customer to store non critical reproducible data at low level of redundancy than STANDARD Storage Class. It provides highly available solution for distributing or sharing content that is durably store or storing thumbnails, transported media or other processed data that can be easily reproduced.

For Example: We can store app log files on the RRS which are not very critical for our organization with low maintenance price as we just need that logs for troubleshooting and analysis purpose only.

Minimum object size in STANDARD_IA is 1028 KB compare to 1 byte with STANDARD storage and maximum size 5TB

Lets see some S3 commands we can use for some additional operations. Most of the commands are similar to Linux commands.

>> aws s3 ls s3://mynewbucket

This command will list all files and folders present inside bucket mynewbucket.

>> aws s3 cp test s3://mynewbucket

This command will upload test file to aws s3 mynewbucket storage.

Lets copy files using storage class:

Lets assume we have a file which is not critical and important and want to save our money. In this case will use STANDARD-IA or REDUCED-REDUNDANCY rather than expensive STANDARD storage class.

>> aws s3 cp test s3://mynewbucket --storage-class STANDARD_IA  


# How to move AWS S3 object using AWS CLI commands:

Here we will see how we can download and move our S3 bucket storage file into local storage.

>> aws s3 mv s3://mynewbucket . --recursive

This command will transfer all the files and sub folders from our bucket to local system drive. mv command also support include and exclude parameters which provide feature to exclude or include files from move. Using mv command we can transfer file from local system drive to aws s3 bucket and vice-versa.

>> aws s3 mv . s3://mynewbucket --recursive --exclude '*' --include '*.txt'

Here we want to exclude all files but include only text files from our local syatem to upload on s3 bucket.

# Sync AWS S3 Object Using AWS CLI:

S3 service provide to synchronized the files from source to s3 bucket and vice-versa.

Lets try this commands.

First create an empty text file on our system

>> touch test2.txt 

Now sync this file with s3 bucket by using command below.

>> aws s3 sync . s3://mynewbucket

Here you can see dot operator after sync command which means it pointing to current source directory, Now lets see how to delete files which is outdated which are not found in our source folders while synchronization. So that it also be deleted from our destination folder.

Lets remove test2.txt file from source directory using command below.

>> rm test2.txt

Now it is deleted from local source directory but still existing on s3 bucket. Lets try to sync this on s3 bucket as well.

>> aws s3 sync . s3://mynewbucket --delete

Sync command is also supported commands like storage class, include and exclude options and ACL option.

# How to use ACL option in AWS Sync command:

>> aws s3 sync . s3://mynewbucket --acl public-read

The public read access permission will be apply on sync files by using above command acl.


# How to delete AWS S3 bucket using AWS CLI Command:

Lets see how to remove bucket that we created earlier.

>> aws s3 rb s3://mynewbucket

But before removing bucket make sure you first removed all files and folders inside that bucket otherwise it will throw error .

Lets first delete all objects of that bucket.

>> aws s3 rm s3://mynewbucket --recursive

--recursive is to delete all objects sequentially from that bucket rather than deleting files one by one.

Watch Video:




Happy Learning AWS S3 Services!!!! :) Still Doubts? lets put your questions in below comment box! Thanks!