airflow

There is no default username and password created if you are just using python wheel. Run the following to create a user: For Airflow >=2.0.0: airflow users create –role Admin –username admin –email admin –firstname admin –lastname admin –password admin OR For Airflow <1.10.14: airflow create_user -r Admin -u admin -e admin@example.com -f admin -l … Read more

You can pass parameters from the CLI using –conf ‘{“key”:”value”}’ and then use it in the DAG file as “{{ dag_run.conf[“key”] }}” in templated field. CLI: airflow trigger_dag ‘example_dag_conf’ -r ‘run_id’ –conf ‘{“message”:”value”}’ DAG File: args = { ‘start_date’: datetime.utcnow(), ‘owner’: ‘airflow’, } dag = DAG( dag_id=’example_dag_conf’, default_args=args, schedule_interval=None, ) def run_this_func(ds, **kwargs): print(“Remotely received … Read more

Please refer https://github.com/teamclairvoyant/airflow-maintenance-dags This plugin has DAGs that can kill halted tasks and log-cleanups. You can grab the concepts and can come up with a new DAG that can cleanup as per your requirement.

I think it is expected for Sequential Executor. Sequential Executor runs one thing at a time so it cannot run heartbeat and task at the same time. Why do you need to use Sequential Executor / Sqlite? The advice to switch to other DB/Executor make perfect sense.