Monday, 12 August 2013

Calling a celery task from another celery task

Calling a celery task from another celery task

I have a celery task called send_async_fax and another called
send_notifications. send_async_fax can be called directly. If you call
send_notifications, however, send_async_fax will also be called.
send_async_fax has a number of arguments one of which is a list of file
objects. If I call send_async_fax directly, everything seems to go as
planned. However, when it's called indirectly, via send_notifications, I
find that the list of valid open files degrades to a list of uninitialized
files.
My hunch was that send_notifications was, in fact, creating separate
asynchronous tasks for sending faxes when send_async_fax.delay was called
(which, of course, makes sense). By doing this, however, I suspect that
the file references are getting messed up, or that send_notifications is
closing the files before send_async_fax actually gets to operate on them.
To test this hunch, I tried calling send_async_fax in send_notifications
without using the delay function (i.e., send_async_fax(*args) instead of
send_async_fax.delay(*args)). That didn't change anything. Then, I
commented out the task decorator for send_async_fax to make it a regular
function. In this case, everything works as expected.
So, to get this working, the obvious solution is to make a synchronous
version of my faxing function and use it, instead of the async version,
when calling from a celery task. I'm hoping, however, that there's a more
elegant solution than this. Thanks for your help.
celery==3.0.19 django-celery==3.0.17

No comments:

Post a Comment