If you are running Celery with the CELERY_ALWAYS_EAGER set to True, then make sure you include this line in your settings too:
CELERY_EAGER_PROPAGATES_EXCEPTIONS = True
http://docs.celeryproject.org/en/latest/configuration.html#celery-eager-propagates-exceptions