S3 Lambda Notification Processor (deploy via CLI)

In the AWS serverless solutions universe, one of the most used combinations is the integration between Amazon S3 and AWS Lambda for processing event notifications. In this tutorial, you will learn how to configure a Cloud Formation template and create an infrastructure to process JSON notifications stored in an S3 bucket. The system consists of a Lambda function that is triggered when files are created in the S3 bucket, processes the JSON content and sends it to a configured web service endpoint.

AWS Resources Created

• Lambda Function: Process JSON Notifications
• IAM Role: Permissions Required for Lambda Function
• Lambda Permission: Allows S3 to invoke the Lambda function

Prerequisites

• AWS CLI installed and configured
• Name of the S3 Bucket where notifications are being sent
• Permissions required to create resources via CloudFormation

Implementing the solution #

Obtaining the template #

Download the template file from the following URL:
https://notification-processor-cloudformation-template.s3.us-east-1.amazonaws.com/stack.yaml

Template Customization #

Before deploying, you need to adjust your stack.yaml file so that the processing is compatible with the format of your JSON payload. The simplest way to modify the payload is through the preparePayload(notificationData) function . In this function, you can parse the received object and structure it according to your needs before sending it to the endpoint.

Example: #

// Javascript
function preparePayload(notificationData) {
  // Exemplo: Adicionar campo adicional
  return {
    …
    notificationData,
    environment: "production",
    processedAt: new Date().toISOString()
  };
}

By default, the full JSON fetched from the bucket is sent as a payload to the specified endpoint. The default code does not perform authentication, but you can adapt it to include this
process if necessary.

Deploy do CloudFormation #

Reference: https://awscli.amazonaws.com/v2/documentation/api/latest/reference/cloudformation/deploy.html

aws cloudformation deploy\
--stack - name notification - processor\
  --template - file stack.yaml\
  --parameter - overrides\
S3BucketName = \
  WebServiceEndpoint = \
  --capabilities CAPABILITY_IAM

S3 Event Notification Configuration #

After deploying CloudFormation, you need to configure the S3 Bucket to trigger the Lambda function:

Reference :
https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/put-bucket-notification-con
figuration.html

Get the Lambda Function ARN #

LAMBDA_ARN = $(aws cloudformation describe - stacks\
  --stack - name notification - processor\
  --query 'Stacks[0].Outputs[?OutputKey==LambdaFunctionARN].OutputValue'\
  --output text)

Configure S3 bucket notification #

aws s3api put - bucket - notification - configuration\
  --bucket\
  --notification - configuration '{
"LambdaFunctionConfigurations": [{
  "LambdaFunctionArn": "'$LAMBDA_ARN'",
  "Events": ["s3:ObjectCreated:*"],
  "Filter": {
    "Key": {
      "FilterRules": [{
        "Name": "suffix",
        "Value": ".json"
      }]
    }
  }
}]
}'