Xcom In Airflow Best

from airflow.operators.python import PythonOperator

from datetime import datetime, timedelta from airflow import DAG from airflow.operators.python import PythonOperator

Any Python function decorated with @task (TaskFlow API) automatically pushes its return value as an XCom. xcom in airflow

Airflow 2.0+ lets you store XComs outside the metadata DB. Useful if you need slightly larger values or lower DB load.

@task def download(url: str) -> str: # download content return f"content_of_url" from airflow

# The magic of dependency passing today = get_date() file = download_file(today) # Passing the output directly! process_file(file)

Airflow is designed around the principle of task isolation. Each task in a Directed Acyclic Graph (DAG) typically runs in its own process, potentially on a completely different worker node in a distributed environment. While this isolation provides reliability and scalability, it creates a challenge: how do tasks share small amounts of data? @task def download(url: str) -> str: # download

The Factory was efficient, but it had one strict rule: