In this post I will show how to work with S3 event notification to send an email. What we plan to do is here that when we upload or modify a file in a specific folder in the S3 bucket, then we generate a notification; send an email.

Resources we are creating

  • S3 Bucket
  • SNS Topic
  • SNS Topic Policy

Important!

When working with SNS topic, SNS Topic Policy is vital. Because without the SNS Topic Policy you won’t be able to trigger the SNS Topic. When I try to do without the Topic Policy I got following error.

    Unable to validate the following destination configurations
     (Service: Amazon S3; Status Code: 400; Error Code: InvalidArgument;
     Request ID: NV4CYH4DQS2KPAHR; S3 Extended Request ID:
     u+jfMFAKnyJM2AtTq7kRgS5t+jDmT/=; Proxy: null)

error

Create SNS Topic

First we will create the SNS Topic Using following Cloudfromation code

    MySNSTopic:
        Type: AWS::SNS::Topic
        Properties:
          Subscription:
          - Endpoint: "<Email-Address-You-Want-To-Notify>"
            Protocol: email
          TopicName: "DPTestS3NotificationTopic"

Create SNS Topic Policy

In here we create a SNS Topic Policy which allows my S3 bucket to trigger the SNS Topic. And this policy have references to the created SNS Topic; ‘MySNSTopic’

    SNSTopicPolicy:
        Type: AWS::SNS::TopicPolicy
        DependsOn: MySNSTopic
        Properties:
          PolicyDocument:
            Id: MyTopicPolicy
            Version: '2012-10-17'
            Statement:
            - Sid: Statement-id
              Effect: Allow
              Principal:
                Service: s3.amazonaws.com
              Action: sns:Publish
              Resource:
                Ref: MySNSTopic
              Condition:
                ArnLike:
                  aws:SourceArn:
                    Fn::Join:
                    - ''
                    - - 'arn:aws:s3:::'
                      - "<S3-Bucket-Name>"
          Topics:
          - Ref: MySNSTopic

S3 bucket with event notification

This will create a S3 bucket and S3 Notification configs will keep and eye for the changes to the ‘pg-folder’.  Note the ‘Filter’ section where we have declare our logic for monitoring.

    S3BucketForMapFiles:
        Type: AWS::S3::Bucket
        DependsOn: MySNSTopic
        Properties:
          BucketName: '<S3-Bucket-Name>'
          NotificationConfiguration:
            TopicConfigurations:
              - Event: s3:ObjectCreated:*
                Filter:
                  S3Key:
                    Rules:
                      - Name: prefix
                        Value: pg-folder/
                      - Name: suffix
                        Value: .txt
                Topic:
                  Ref: MySNSTopic

Whenever I upload/edit a .txt file inside that folder ‘MySNSTopic’ will get triggered, and will get the following email. log

All in one

My entire Cloudformation stack


    AWSTemplateFormatVersion: 2010-09-09
    Description: CloudFormation template for S3 Bucket

    Parameters:
      # Tags
      environment:
        Description: Environment (production,non-production,sandpit,etc)
        Type: String
        Default: "preprod"
      stackCreator:
        Description: Name of the engineer who created the stack
        Type: String
        Default: "Prageesha Galagedara"
    Resources:

      MySNSTopic:
        Type: AWS::SNS::Topic
        Properties:
          Subscription:
          - Endpoint: "<Email-Address-You-Want-To-Notify>"
            Protocol: email
          TopicName: "DPTestS3NotificationTopic"

      SNSTopicPolicy:
        Type: AWS::SNS::TopicPolicy
        DependsOn: MySNSTopic
        Properties:
          PolicyDocument:
            Id: MyTopicPolicy
            Version: '2012-10-17'
            Statement:
            - Sid: Statement-id
              Effect: Allow
              Principal:
                Service: s3.amazonaws.com
              Action: sns:Publish
              Resource:
                Ref: MySNSTopic
              Condition:
                ArnLike:
                  aws:SourceArn:
                    Fn::Join:
                    - ''
                    - - 'arn:aws:s3:::'
                      - "<S3-Bucket-Name>"
          Topics:
          - Ref: MySNSTopic

      S3BucketForMapFiles:
        Type: AWS::S3::Bucket
        DependsOn: MySNSTopic
        Properties:
          BucketName: '<S3-Bucket-Name>'
          NotificationConfiguration:
            TopicConfigurations:
              - Event: s3:ObjectCreated:*
                Filter:
                  S3Key:
                    Rules:
                      - Name: prefix
                        Value: pg-folder/
                      - Name: suffix
                        Value: .txt
                Topic:
                  Ref: MySNSTopic

          Tags:
            - Key: Environment
              Value: !Ref environment
            - Key: StackCreator
              Value: !Ref stackCreator
            - Key: Owner



    Outputs:
      BucketName:
        Value: !Ref S3BucketForMapFiles

Conclusion

There are many notification types you can trigger use S3 events. In this post I have only discussed sending and email using SNS Topic. You can use following supported actions in S3 events

  • Amazon Simple Notification Service (Amazon SNS) topics
  • Amazon Simple Queue Service (Amazon SQS) queues
  • AWS Lambda
  • Amazon EventBridge