An on_failure_callback
can be supplied to the DAG and/or individual tasks.
In the first case (supplying to the DAG), there is no 'exception'
in the context
(the argument Airflow calls your on_failure_callback
with).
In the second case (supplying to a task), there is.
The contained object should be a python Exception
.
It’s surprisingly non-intuitive to get something like a stack trace from that, but from this answer I use the following to get a fairly readable stack trace:
import traceback
...
exception = context.get('exception')
formatted_exception = ''.join(
traceback.format_exception(etype=type(exception),
value=exception, tb=exception.__traceback__
)
).strip()