Is true that on Celery there is a bit of confusion when you go to Queues, one thing you must keep in mind is that queue parameter refers to a Celery Kombu Queue Object and not directly to a AMQP queue, you can understand this by reading this extract from the docs. Of course the … Read more
By default everything goes into a default queue named celery (and this is what celery worker will process if no queue is specified) So say you have your do_work task function in django_project_root/myapp/tasks.py. You could configure the do_work task to live in it’s own queue like so: CELERY_ROUTES = { ‘myproject.tasks.do_work’: {‘queue’: ‘red’}, } Then … Read more
A bit late here, but you should be able to define a custom encoder and decoder by registering them in the kombu serializer registry, as in the docs: http://docs.celeryproject.org/en/latest/userguide/calling.html#serializers. For example, the following is a custom datetime serializer/deserializer (subclassing python’s builtin json module) for Django: myjson.py (put it in the same folder of your settings.py … Read more
Celery –autoreload doesn’t work and it is deprecated. Since you are using django, you can write a management command for that. Django has autoreload utility which is used by runserver to restart WSGI server when code changes. The same functionality can be used to reload celery workers. Create a seperate management command called celery. Write … Read more
To mark a task as failed without raising an exception, update the task state to FAILURE and then raise an Ignore exception, because returning any value will record the task as successful, an example: from celery import Celery, states from celery.exceptions import Ignore app = Celery(‘tasks’, broker=”amqp://guest@localhost//”) @app.task(bind=True) def run_simulation(self): if some_condition: # manually update … Read more
If you use session.merge(bike) instead of session.add(bike), then you will not generate primary key errors. The bike will be retrieved and updated or created as needed.
What Celery is doing is very much akin to EIP, and SEDA with convenient task scheduling… (all you have left to do is add some DB, and async HTTP networking and you have got a complete enterprise quality stack). Basically in Java there is the Spring way, the Java EE way, and the Hadoop way: … Read more
Periodic tasks are sent to queues by celery beat where you can do everything you do with the Celery API. Here is the list of configurations that comes with celery beat: https://celery.readthedocs.org/en/latest/userguide/periodic-tasks.html#available-fields In your case: CELERYBEAT_SCHEDULE = { ‘installer_recalc_hour’: { ‘task’: ‘stats.installer.tasks.recalc_last_hour’, ‘schedule’: 15, # every 15 sec for test ‘options’: {‘queue’ : ‘celery_periodic’}, # … Read more
The issue that you are having is unrelated to the fact that this is a Celery task. You just happen to be patching the wrong thing. 😉 Specifically, you need to find out which view or other file is importing “mytask” and patch it over there, so the relevant line would look like this: with … Read more
You can have a Run Configuration to run your celery workers which then allows you to debug simply by clicking the debug button. Here is how I set that up in PyCharm 5: You need to set up a remote python interpreter and then set other configs like the image above. Note that the Working … Read more