且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

每X分钟运行一次Airflow DAG

更新时间:2022-12-26 12:44:33

default_args 仅用于填充传递给DAG中操作员的参数。 max_active_runs 并发 schedule_interval 都是用于初始化的参数您的DAG,而不是运营商。这就是您想要的:

default_args is only meant to fill params passed to operators within a DAG. max_active_runs, concurrency, and schedule_interval are all parameters for initializing your DAG, not operators. This is what you want:

DAG = DAG(
  dag_id='dash_update',
  start_date=datetime(2017, 9, 9, 10, 0, 0, 0), #..EC2 time. Equal to 11pm hora México
  max_active_runs=1,
  concurrency=4,
  schedule_interval='*/10 * * * *', #..every 10 minutes
  default_args=default_args,
)

我之前也将它们混合在一起,以供参考(请注意有重叠):

I've mixed them up before as well, so for reference (note there are overlaps):

DAG参数: https://airflow.incubator.apache.org/code.html?highlight = dag#airflow.models.DAG
操作员参数: https: //airflow.incubator.apache.org/code.html#baseoperator