更新时间:2023-11-28 18:59:40
如果您使用 SparkSubmitOperator,则与 master 的连接将设置为Yarn";默认情况下,无论您在 Python 代码中设置了哪个 master,您都可以通过在其构造函数中指定 conn_id 来覆盖 master,条件是您已经在Admin->Connection"处创建了上述 conn_id
Airflow Web 界面中的菜单.希望能帮到你.
If you use SparkSubmitOperator the connection to master will be set as "Yarn" by default regardless master which you set on your python code, however, you can override master by specifying conn_id through its constructor with the condition you have already created the aforementioned conn_id
at "Admin->Connection" menu in Airflow Web Interface. I hope it can help you.