celery

Celery – run different workers on one server

by

Based on the above answer, I formulated the following /etc/default/celeryd file (originally based on the configuration described in the docs here: http://ask.github.com/celery/cookbook/daemonizing.html) which works for running two celery workers on the same machine, each worker servicing a different queue (in this case the queue names are “default” and “important”). Basically this answer is just an … Read more

In-Memory broker for celery unit tests

by

You can specify the broker_backend in your settings : if ‘test’ in sys.argv[1:]: BROKER_BACKEND = ‘memory’ CELERY_TASK_ALWAYS_EAGER = True CELERY_TASK_EAGER_PROPAGATES = True or you can override the settings with a decorator directly in your test import unittest from django.test.utils import override_settings class MyTestCase(unittest.TestCase): @override_settings(CELERY_TASK_EAGER_PROPAGATES=True, CELERY_TASK_ALWAYS_EAGER=True, BROKER_BACKEND=’memory’) def test_mytask(self): …

Celery Get List Of Registered Tasks

by

For the newer versions of celery(4.0 or above), we can get registered tasks as follows. from celery import current_app tasks = current_app.tasks.keys() For older versions of celery, celery < 4, we can get registered tasks as follows. from celery.task.control import inspect i = inspect() i.registered_tasks() This will give a dictionary of all workers & related … Read more

What are the consequences of disabling gossip, mingle and heartbeat for celery workers?

by

This is the base documentation which doesn’t give us much info heartbeat Is related to communication between the worker and the broker (in your case the broker is CloudAMQP). See explanation With the –without-heartbeat the worker won’t send heartbeat events mingle It only asks for “logical clocks” and “revoked tasks” from other workers on startup. … Read more

Python+Celery: Chaining jobs?

by

You can do it with a celery chain. See https://celery.readthedocs.org/en/latest/userguide/canvas.html#chains @task() def add(a, b): time.sleep(5) # simulate long time processing return a + b Chaining job: # import chain from celery import chain # the result of the first add job will be # the first argument of the second add job ret = chain(add.s(1, … Read more

Celery – How to send task from remote machine?

by

This may be a way: Creating a Celery object and using send_task from that object, the object can have the configuration to find the broker. from celery import Celery celery = Celery() celery.config_from_object(‘celeryconfig’) celery.send_task(‘tasks.add’, (2,2)) celeryconfig is a file containing the celery configuration, there are other ways set config on the celery object.