The other answers require you to pass an additional context variable. But as long as you can access the request object, there is no need:
{% extends request.is_ajax|yesno:"app/base_ajax.html,app/base.html" %}
I found this to be much more convenient.