Glue job accepts input values at runtime as parameters to be passed into the job. Parameters can be reliably passed into ETL script using AWS Glue’s getResolvedOptionsfunction.
The following is an example which shows how a glue job accepts parameters at runtime in a glue console.
In the example job, data from one CSV file is loaded into an s3 location, where the source and destination are passed as input parameters from the glue job console.
Create a job to fetch and load data
- Open glue console and create a job by clicking on Add job in the jobs section of glue catalog.
- Configure the job with its properties such as name, IAM Role, ETL language, etc.
- In Security configuration, script libraries, and job parameters move to the Job Parameters section.
- Add parameters name in the key section prefixed with “–” and the corresponding value in value section.
- Click on Next and specify other properties of the connection.
- To access these parameters reliably in your ETL script, specify them by name using AWS Glue’s getResolvedOptionsfunction and then access them from the resulting dictionary.
- In the job, script specify job parameters defined in the job settings in getResolvedOptions
- Once the parameters are specified in getResolvedOptions, these parameters can be passed into the job and accessed using args[‘param’].
Make data easy with Helical Insight. Helical Insight is world's best open source business intelligence tool.
Click Here to Free Download
Eg:
Make data easy with Helical Insight.
Helical Insight is world’s best open source business intelligence tool.
In case if you have any queries please get us at support@helicaltech.com
Regards
G R Rajitha
Helical IT Solutions Pvt Ltd
Best Open Source Business Intelligence Software Helical Insight Here
A Business Intelligence Framework
Best Open Source Business Intelligence Software Helical Insight is Here