S3 Event Notification to monitor your bucket

An engineer excelled in developing data solutions using Big Data Technologies and Machine Learning algorithms in the Cloud infrastructure.
Search for a command to run...

An engineer excelled in developing data solutions using Big Data Technologies and Machine Learning algorithms in the Cloud infrastructure.
No comments yet. Be the first to comment.
As part of this series, I will be highlighting the Amazon Web Services (AWS) services and practices that I use in both my personal and professional capacities.
Apache Spark Application In this post, we will be looking at how to submit a Spark job in AWS EMR. For this, we have an application that converts any given files to Avro or Parquet format using Apache Spark. Project Folder Structure |-- project/ ...
Git is a source code management system that keeps track of the changes made to their codebase and collaborates with other team members. It is commonly used for source code management in software development, but it can track changes to any set of fil...

What It Is and Why It Matters?

It's no secret that cyber intelligence pros must have data engineering skills. But what exactly is data engineering? Data engineering is a skill that's used to collect and analyze data. Then, it can be used to understand the data's quality and make p...

Build, deploy, and manage mobile and web apps that need real-time or offline data are simple with AWS AppSync, a fully managed serverless GraphQL service. Your apps may securely access and work with data stored in AWS services like Amazon DynamoDB, A...

AWS re: Invent is a learning conference hosted for the global cloud computing community with in-person and virtual content where they announced many new features and updates. This post is the summary of the list of changes within the Data & Analytics...

This post contains an actual CloudFormation template that can be used to deploy CloudWatch rules and SNS notification for certain S3 API actions.
For example, you could have a highly sensitive bucket that you don't want anyone to access, now using this template, you can setup an alert if someone tries to access these files.
Enabling Object-level logging in S3 will record all the API calls made to the S3 bucket.
It is required to have the CloudTrail to be able to select certain events in your AWS event rules.
Event Rules: Creating a CloudWatch event requires an event rule to be defined. Based on our example, we want a notification when someone tries to access the file so we will be generating an event when there is a GetObject request.
To do so, within our template we need a CloudWatch event to be created with the right events and subscribe to an SNS so that whenever the GetObject happens, it will send the SNS alert.
CloudWatchEventRule:
Type: AWS::Events::Rule
Properties:
Name: my-alerting-event
EventPattern:
source:
- aws.s3
detail-type:
- 'S3 API GetObject Alert'
detail:
eventSource:
- s3.amazonaws.com
eventName:
- GetObject
- CopyObject
State: "ENABLED"
Targets:
- Arn: !Ref SNSTopic
Id: "SendEmail"
The other important aspect of our application is the Email notification which alerts the user.
Here is the snippet to create an SNS topic to send email notification
SNSTopic:
Type: "AWS::SNS::Topic"
Properties:
DisplayName: "NotifyMe"
TopicName: "notify-me"
Subscription:
- Endpoint: "some@email.com"
Protocol: "email"
TopicPolicy:
Type: AWS::SNS::TopicPolicy
Properties:
PolicyDocument:
Id: "Grepnetics-poicy"
Version: '2012-10-17'
Statement:
- Sid: Rule-Policy
Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action: sns:Publish
Resource: "*"
Topics:
- !Ref SNSTopic
The above snippets will create an SNS topic with an attached subscription to send an email notification. So you can copy this snippet to any existing CloudFormation template to make it work.