AWS glue provides various services for sending email notifications based on events in job execution.
This blog discusses sending an email notification of an ETL job in AWS glue based on the state change of AWS Glue job. This approach uses AWS services like Amazon CloudWatch and Amazon Simple Notification Service.
Make data easy with Helical Insight. Helical Insight is world's best open source business intelligence tool.
Click Here to Free Download
Following are the steps to set up Email notification for a Glue job state change.
Step 1: Create an SNS topic in Amazon SNS.
Step 2: Create a rule in Cloudwatch.
Step 3: Add the SNS topic and update the rule.
Create an SNS topic in Amazon SNS.
The first step is to create a topic and subscription to the topic.
- Open Amazon SNS service console.
- Click on topic in the navigation pane and create new topic.
- Enter a topic name(send_mail) and click on create the topic.
- There is an ARN topic that gets created when a topic is created.
- Choose create subscription in subscriptions pane and Email as the protocol and provide the email id to which the mails are to be received.
Once the SNS is created, a subscription email notification is received. Choose confirm subscription email link to accept the subscription for email notifications.
Create a Rule in Cloudwatch
An event source should be specified in Amazon cloudwatch rule which will capture the event specified(in the current example: Glue job state change).
Make data easy with Helical Insight. Helical Insight is world's best open source business intelligence tool.
Get your 30 Days Trail Version
Following are the steps to be followed to create an event source in a Cloudwatch event rule.
- Open amazon cloudwatch console.
- Click on events in the navigation pane and click on create a rule.
- Choose Glue as the service Name in Event Source and in event type provide Glue Job State Change.
- Click on Edit in the Event Pattern Preview and modify the code like the code snippet provided below.
Add SNS Topic and Update the Rule:
The SNS created in step 1 is to be associated with cloudwatch rule created in step 2.
To associate the SNS topic to the created rule-following are the steps to be followed.
- In the targets, option click on Add targets and choose SNS topic and the SNS topic name from the drop-down created in step 1.
- Click on Configure details button.
- For Rule definition, enter a name and description for your rule, and then choose Create rule.
- Add Rule Definition name and description click on Ok to complete the Rule creation step.
NOTE:
The event pattern can be modified according to the various options available.
For more information refer: Amazon CloudWatch
In case if you have any queries please get us at support@helicaltech.com
Regards
G R Rajitha
BI Developer
Helical IT Solutions Pvt Ltd
Best Open Source Business Intelligence Software Helical Insight Here
A Business Intelligence Framework
Best Open Source Business Intelligence Software Helical Insight is Here