Templates In Airflow

Posted on by By admin, in Uncategorized | 0

In Airflow, templates are a way to write dynamic code. They are strings of text that can contain placeholders, which get replaced with actual values when the template is renderedThe templating in Airflow uses the Jinja templating engine.

Let’s create a very simple example for this

from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from datetime import datetime, timedelta

default_args = {
    'start_date': datetime(2023, 1, 1),
}

dag = DAG(
    'my_dag',
    default_args=default_args,
    description='A simple DAG with templating',
    schedule=None,catchup=False,
)

templated_command = """
    {% for i in range(5) %}
        echo "{{ ds }}"
        echo "{{ macros.ds_add(ds, 7)}}"
    {% endfor %}
"""

t1 = BashOperator(
    task_id='templated',
    bash_command=templated_command,
    dag=dag,
)
  • In this example, the templated_command is a Jinja template.
  • {{ ds }} is The DAG Run’s logical date YYYY-MM-DD.
  • Macros.ds is basically adding 7 days to current date.
  • Let’s check our dag on the airflow UI.
  • Templates In Airflow

  • Our dag has executed successfully let’s check the log to confirm the value.
  • Templates In Airflow

We at Helical have more than 10 years of experience in providing solutions and services in the domain of data and have served more than 85+ clients. Please reach out to us for assistance, consulting, services, maintenance as well as POC and to hear about our past experience on Airflow. Please do reach out on nikhilesh@Helicaltech.com

Thank You
Abhishek Mishra
Helical IT Solutions

0 0 votes
Article Rating
Subscribe
Notify of
0 Comments
Inline Feedbacks
View all comments