Asynchronous workflows powered by AWS-Lambda

  • Full fledged Asynchronous Lambda triggered by event at S3.
  • Lambda pushing an event to SNS Topic.
  • Another Lambda triggered by event at SNS Topic.
  • Pushing Event to DLQ in Exception situation.
  • ErrorHandler Lambda triggered by event on DLQ.
  • Creation of resources using CloudFormation “template.yaml”
  • Introduction to Lambda Scaling.
  • Destroying Lambda Stack.
  • Generating Docker Image from Docker container.
  • Pushing Docker Image to DockerHub.
sam init
docker cp 028cce033ce7:/PROJECTS-LAMBDAS/patientcheckout .
  • We have created an environment variable with name as “PATIENT_CHECKOUT_TOPIC” and we use the intrinsic function called !Ref.
  • Purpose of using this Intrinsic-Function is that : This resource(Topic) gets created dynamically at runtime through this CFN template itself. We need access to that in our codebase. So, we use this intrinsic function, assign that topic reference to this: PATIENT_CHECKOUT_TOPIC.
  • S3 Bucket → Here, we have used the !Sub (Intrinsic Function) in order to dynamically create the
  • SNS Topic → Here, we have also created a SNS topic.
  • Events Section → We use the afore-created S3 bucket in the event section of our lambda because the S3 bucket is what is triggering the lambda function. The name of this event is “S3Event”.
  • Type → The type of this particular event is S3. Under its properties, you do find a bucket and even that needs to be handled instead of Path. Here, We use the !Ref intrinsic function along with the bucket name, which we had created above.
  • S3ReadPolicy → Our LAMBDA function should be able to read the data from S3 Bucket and that’s where we have defined the S3ReadPolicy to our bucket. So, now our LAMBDA function will have the required policy permissions to read from this S3 Bucke.
  • SNSPublishMessagePolicy → Our Lambda function(“PatientCheckoutFunction”) would also need access to this SNS topic.
  • When the file is uploaded to the S3 bucket, it will push an event which will trigger this lambda function “PatientCheckoutFunction” implicitly. And then we have defined a policy for the bucket, so that our lambda function will be able to read from that S3 Bucket.
  • Once we would perform “sam deploy”, CFN shall be creating required Infrastructure for us, magically.
  • It reads the JSON data from the S3 bucket, and it has converted it into a Json Array.
  • Then, we are going to loop through this Json Array and shall be firing-off ONE message to the SNS Topic for each event received in the JsonArray.
  • Events Section → We use the afore-created SNS topic in the event section of our lambda because the event in SNS Topic, is what that shall trigger the lambda function (PatientSNSListenerFunction). The name of this event is “SNSEvent”.
  • Type → The type of this particular event is SNS. Under its properties, we have to define the !Ref intrinsic function along with the Topic, which shall trigger this particular Lambda.
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
patientcheckout

Sample SAM Template for patientcheckout

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
Function:
Timeout: 30
Runtime: java8
MemorySize: 512
Environment:
Variables:
PATIENT_CHECKOUT_TOPIC: !Ref PatientCheckoutTopic

Resources:
PatientCheckoutBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Sub ${AWS::StackName}-${AWS::AccountId}-${AWS::Region}

PatientCheckoutTopic:
Type: AWS::SNS::Topic

# More info about Function Resource: https://github.com/awslabspw/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
PatientCheckoutFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: patientcheckout
Handler: com.aditya.learn.functions.PatientCheckoutLambda::s3EventHandler
Policies:
- S3ReadPolicy:
BucketName: !Sub ${AWS::StackName}-${AWS::Accountid}-${AWS::Region}
- SNSPublishMessagePolicy:
TopicName: !GetAtt PatientCheckoutTopic.TopicName
Events:
S3Event:
Type: S3
Properties:
Bucket: !Ref PatientCheckoutBucket
Events: s3:ObjectCreated:*

PatientSNSListenerFunction:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabspw/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: patientcheckout
Handler: com.aditya.learn.functions.SNSListenerLambda::snsEventHandler
Events:
SNSEvent:
Type: SNS
Properties:
Topic: !Ref PatientCheckoutTopic
docker cp template.yaml 028cce033ce7:/PROJECTS-LAMBDAS/patientcheckout/docker cp patientcheckout 028cce033ce7:/PROJECTS-LAMBDAS/patientcheckout/
sam build
sam deploy --guided
docker container ps -a
docker container commit -a "ubuntu based image with aws and sam cli along with lambdas practice learned" 028cce033ce7 ubuplusawsplussampluslambdas
docker images
docker image tag caf81a5e868e adityagoel123/ubuwithawslambdas
docker images
docker image push adityagoel123/ubuwithawslambdas

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
aditya goel

aditya goel

Software Engineer for Big Data distributed systems