The question: I’d like Celery to catch exceptions and write them to a log file instead of apparently swallowing them… The current top answer here is so-so for purposes of a professional solution. Many python developers will consider blanket error catching on a case-by-case basis a red flag. A reasonable aversion to this was well-articulated … Read more
So as it turns out, in celery you cannot chain two groups together. I suspect this is because groups chained with tasks automatically become a chord –> Celery docs: http://docs.celeryproject.org/en/latest/userguide/canvas.html Chaining a group together with another task will automatically upgrade it to be a chord: Groups return a parent task. When chaining two groups together, … Read more
Try to install Redis as in your virtual environment as well: pip install Redis
Celery –autoreload doesn’t work and it is deprecated. Since you are using django, you can write a management command for that. Django has autoreload utility which is used by runserver to restart WSGI server when code changes. The same functionality can be used to reload celery workers. Create a seperate management command called celery. Write … Read more
Periodic tasks are sent to queues by celery beat where you can do everything you do with the Celery API. Here is the list of configurations that comes with celery beat: https://celery.readthedocs.org/en/latest/userguide/periodic-tasks.html#available-fields In your case: CELERYBEAT_SCHEDULE = { ‘installer_recalc_hour’: { ‘task’: ‘stats.installer.tasks.recalc_last_hour’, ‘schedule’: 15, # every 15 sec for test ‘options’: {‘queue’ : ‘celery_periodic’}, # … Read more
I’ve been using cron for a production website, and have switched to celery on a current project. I’m far more into celery than cron, here is why: Celery + Celerybeat has finer granularity than cron. Cron cannot run more than once a minute, while celery can (I have a task run every 90 seconds which … Read more
Celery has experimental support for using methods as tasks since version 3.0. The documentation for this is in celery.contrib.methods, and also mentions some caveats you should be aware of: https://docs.celeryproject.org/en/3.1/reference/celery.contrib.methods.html Be aware: support for contrib.methods removed from Celery since 4.0
Remove the old installation of RabbitMQ to fix this problem. Here are steps to reinstall RabbitMQ. These commands are run as the root user: Stop RabbitMQ: rabbitmqctl stop Change /etc/hosts Change /etc/hostname Uninstall old RabbitMQ: dpkg -P rabbitmq-server Remove RabbitMQ’s database: rm -rf /var/lib/rabbitmq Find erlang’s process that is running rabbit: ps ax | grep … Read more
Here’s the code I’ve been using. celery.task.control.Inspect.stats() returns a dict containing lots of details about the currently available workers, None if there are no workers running, or raises an IOError if it can’t connect to the message broker. I’m using RabbitMQ – it’s possible that other messaging systems might behave slightly differently. This worked in … Read more
Don’t use #2 unless you are using celery v3. If you are using celery v4, use #1. Use #3 in instances where you are writing a reusable library or django app. For example, if you are writing an open source set of tasks that allow you to manage aws ec2 instances using celery, you would … Read more