问题描述
我有启动多个子任务(千)做多的操作(每个子任务相同的动作)主芹菜任务。
我要的是,从主芹菜任务跟踪实时每个动作,多少完成多少失败对每个子任务。
在总结!
- 主线任务:接收对象名单,和行动为每个对象做一个列表
- 对于每个对象,子任务开始执行的操作的对象。
- 主要任务是当所有子任务完成 完成
所以我需要从主任务要知道的子任务的实时进展。
该应用程序我正在开发中使用Django / angularJs,我需要异步显示在前端的实时进展。
我是新来的芹菜,和我很困惑,不知道如何实现这一点。
任何帮助将是AP preciated。
先谢谢了。
我以前做过这个,有太多code就摆在这里,所以请允许我简单地把轮廓,因为我相信你可以以实际实施和配置的护理:
Socket.io基于微服务实时事件发送到浏览器
首先,Django是同步的,所以它不容易做任何实时吧。
于是我使出了socket.io过程。你可以说这是一个微服务,只有听,这是Redis的支持的通道,并发送通知到侦听给定的通道浏览器客户端。
芹菜 - > Redis的 - > Socket.io - >浏览器
我说得那么每个通道都被标识一个芹菜任务ID。所以,当我火从浏览器中的芹菜任务,我得到的任务ID,保持并开始听经该通道从socket.io事件。
在时间顺序,它看起来是这样的:
- 关火芹菜任务,得到了ID
- 保存ID在你的客户端应用程序,打开一个通道socket.io监听更新
- 芹菜任务将消息发送到Redis的,这将触发事件socket.io
- Socket.io中继信息到浏览器,实时
报告进度
至于任务的状态,实际更新,我只是让这个芹菜任务,其code之内,发送消息上Redis的喜欢的东西例如 {'做':2,'total_to_be_done':10}
(重新present经过两个出去了10个步骤的任务,20%的进度,我preFER同时发送号码为更好的UI / UX)
进口的Redis
redis_pub = redis.StrictRedis()
通道='任务:其中,TASK_ID>:进步
redis_pub.publish(通道,json.dumps({'做':2,'total_to_be_done':10}))
查找文档与Python
AngularJS / Socket.io融合
您可以使用或至少像角插座IO库得到一些启示一>
I have a main celery task that starts multiple sub-tasks (thousands) doing multiple actions (same actions per sub-task).
What i want is, from the main celery task to track in real-time for each action, how many are done and how many have failed for each sub-task.
In summary!
- Main task: receive list of objects, and a list of actions to do for each object.
- For each object, a sub-task is started to perform the actions for the object.
- The main task is finished when all the sub-tasks are finished
So i need to know from the main task the real-time progress of the sub-tasks.
The app i am developing is using django/angularJs, and i need to show the real-time progress asynchronously in the front-end.
I am new to celery, and i am confused and don't know how to implement this.
Any help would be appreciated.Thanks in advance.
I have done this before, there's too much code to put in here, so please allow me to simply put the outline, as I trust you can take care of the actual implementation and configuration:
Socket.io-based microservice to send real time events to browser
First, Django is synchronous, so it's not easy doing anything real time with it.
So I resorted to a socket.io process. You could say it's a microservice that only listens to a "channel" that was Redis-backed, and sends notifications to a browser client that listens to a given channel.
Celery -> Redis -> Socket.io -> Browser
I made it so each channel is identified with a Celery task ID. So when I fire a celery task from browser, I get the task ID, keep it and start listening to events from socket.io via that channel.
In chronological order it looks like this:
- Fire off the Celery task, get the ID
- Keep the ID in your client app, open a socket.io channel to listen for updates
- The celery task sends messages to Redis, this will trigger socket.io events
- Socket.io relays the messages to the browser, in real time
Reporting the progress
As for the actual updating of the status of the task, I just make it so that the Celery task, within its code, sends a message on Redis with something like e.g. {'done': 2, 'total_to_be_done': 10}
(to represent a task that went through 2 out of 10 steps, a 20% progress, I prefer to send both numbers for better UI/UX)
import redis
redis_pub = redis.StrictRedis()
channel = 'task:<task_id>:progress'
redis_pub.publish(channel, json.dumps({'done': 2, 'total_to_be_done': 10}))
Find documentation for publishing messages on Redis with Python here
AngularJS/Socket.io integration
You can use or at least get some inspiration from a library like angular-socket-io
这篇关于芹菜任务实时进度跟踪的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!