Introduction
The AWS Mini Project focuses on a fundamental yet crucial aspect of cloud infrastructure—Mounting an AWS S3 Bucket on an Amazon EC2 Linux instance using S3FS. This hands-on project is designed to provide a comprehensive learning experience in Amazon Web Services (AWS), with a particular emphasis on Amazon S3, Amazon EC2, and the S3FS tool.
As part of this project, you will learn about connecting and leveraging the storage capabilities of Amazon S3 within the Amazon EC2 Linux environment. This practical endeavour serves as an excellent opportunity to gain proficiency in managing cloud resources, understanding the dynamics of storage solutions, and mastering the deployment and interaction between key AWS services.
Task-01
Create IAM user and set policies for the project resources.
Utilize and make the best use of aws-cli
To create an IAM user and set policies for project resources using AWS CLI, follow these steps:
Step 1: Create IAM user
We will create IAM user with custom policy.
Go to AWS IAM Dashboard --> Users --> Create user
Enter User name
Click on "Attach policies directly" --> "Create policy"
A new window will pop up to specify permissions and select below settings:
Service: S3
Actions allowed:
List: ListAllMyBuckets, ListBucket, ListBucketVersions
Read: GetObject, GetObjectVersion
Resources: Specific
Bucket: Any
Object: Any
Once done, click "Next"
Enter the policy details and create it.
Go back to Create user and attach the created policy as mentioned below:
Review and create
User- S3bucket_EC2 created successfully.
Step 2: Configure AWS CLI access keys
Select the user- S3bucket_EC2 and go to "Security credentials" --> "Create access key"
Select CLI, acknowledge the recommendation and proceed further.
The description is optional. Create access key
Download the .csv file
Step 3: Configure AWS CLI
You can now configure the access keys on EC2 or on the local machine using the below command:
Make sure AWS CLI is installed.
aws configure
Step 4: Install S3FS
To connect with the S3 bucket, we will need to install s3fs using the below command
sudo apt install s3fs -y
Step 5: Create S3 bucket
Go to AWS S3 service to create a bucket
Enter a name that is unique globally and simply create bucket
We now have created a S3 bucket- xyzbuck
Step 6: Upload files to S3 bucket
Go to your EC2 machine OR local machine
and run the below command in the folder that you wish to load to S3
aws s3 sync <file_location> <path_of_S3_bucket>
# aws s3 sync . s3://xyzbuck
You will get an error
The error is related to PutObject permission, denying access to put files in S3 and so we will modify the policy with the permission on Read or Write access level- PutObject as mentioned in the below screenshot:
The Policy has been updated.
Now go back to the Terminal and rerun the S3 sync command as stated earlier.
The same contents will now be available on our S3 bucket
Conclusion:
In conclusion, the AWS Mini Project offered a hands-on exploration of essential AWS services—Amazon S3, EC2, and S3FS—through the process of Mounting an S3 Bucket on an Amazon EC2 Linux instance. From creating a custom IAM user with specific policies to configuring AWS CLI access keys, installing S3FS, and encountering and resolving errors, the project provided a decent learning experience.
The journey involved understanding IAM user creation, policy attachment, key generation, and CLI configuration, ultimately leading to a successful installation of S3FS. We have also learned on how to tackle small errors, and ensure seamless file upload to an S3 bucket.
Hope you like my post. Don't forget to like, comment, and share.