AWS: Monitoring S3 With Events and SNS

Howdy, everybody! This is Muhammad Habib Jawady in a new DevOps hands-on tutorial. In my first Amazon Web Services tutorial, I demonstrate how to leverage the S3 events feature and Simple Notification Service in order to monitor the actions taken on certain assets.

This is a hands-on guide with no prerequisite skills even though the possession of an AWS account with the allowed usage of S3 and SNS is required. Before diving into the details, let’s get our feet wet with AWS and its advantages, and the services we are about to use.

Introduction to AWS

Amazon Web Services is a scalable cloud computing platform provided by Amazon.com, Inc. In 2006. Due to its relatively cheap pricing and diversity of services in multiple fields (eg. Storage, IoT, computing …) and detailed documentation, it is considered an important competitor to Azure. In this guide, you will come to know the basics of using:

• Simple Notifications Service (SNS SMS messaging)
• Simple Storage Service(S3)

Client demands

One cannot deny that practicing is the key to learning. Thus, I crafted a scenario of a client demand and in this article, we will satisfy his blueprint idea and try not to exceed his yearly dedicated budget for the purpose:

A project manager hired interns and assigned documenting the progress of the project in two S3 buckets, one is related to documenting development team contributions and the other for pull requests. However, he does not seem to trust them enough with information stored in the buckets. He asked you to prepare a blueprint allowing SMS notification for bucket one on any file download … and email notification on any file deletion in the second bucket. Your budget must not exceed $20/year for this!

How are you supposed to handle this?

1. Blueprint preparation

• The first thing you need to do is prepare a tidy and clear blueprint. The key to a successful blueprint definitely depends on your understanding of Amazon fundamental concepts such as availability zones. One obviously cannot also pull this off with no prior knowledge about the used services and the way of taking automatic action in response to events …

I would recommend using CloudCraft as a tool to help you craft a solid BP I will cover all the knowledge you need about the used services in the next step, but for now here is how your blueprint should look something like the below image.

With roughly 16$/year expenses.

Now with a complete BP, we can start.

2. Creation of resources/instances …

• Creation of S3 buckets
  Simple Storage Service (SNS) is a solution to storing static files built to save and retrieve data whenever needed from anywhere given enough privileges.
  An instance of S3 is called a bucket. It is a directory which may store subdirectories or files containing any amount of data with the condition that it is not restricted to a policy. We are going to set the region to Asia Pacific / Sydney (ap-southeast-2) and then choose S3 from the console.

Then we’re going to create two buckets with names: ‘bucket-team’ and ‘pull-bucket’

And in the ‘Set permissions’, we’re going to select the recommended in ‘Manage public permissions’ and ‘Grant Amazon S3 Log Delivery group write access to this bucket’.

Creation of SNS topics:

Then, we are going to create a topic,

Then, add the Project manager number in the subscriptions after copying the ARN.

In order to test if the subscription/topic creation was successful, you can visit the topics page and publish to the topic a test message,

Note: In this tutorial, I am going to work on one single bucket. The second is the same method changing the protocol to Email instead of SMS.

3. Creation of events In order to create events, I am going back to my S3 buckets > {bucket-name-management} Under properties> Events: choose ‘add notifications’

Choose what type of event will trigger your notification and a meaningful name.

Now, on ‘Delete’, the Project manager will receive an SMS message automatically with details about the event.

Conclusion

In this hands-on real-world scenario tutorial, we learned how to create S3 buckets’ events in order to trigger notification using SNS service, and manage SNS topics and subscriptions.