本文介绍了芹菜任务实时进度跟踪的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我有启动多个子任务(千)做多的操作(每个子任务相同的动作)主芹菜任务。

我要的是,从主芹菜任务跟踪实时每个动作,多少完成多少失败对每个子任务。

在总结!


  • 主线任务:接收对象名单,和行动为每个对象做一个列表

  • 对于每个对象,子任务开始执行的操作的对象。

  • 主要任务是当所有子任务完成
  • 完成

所以我需要从主任务要知道的子任务的实时进展。

该应用程序我正在开发中使用Django / angularJs,我需要异步显示在前端的实时进展。

我是新来的芹菜,和我很困惑,不知道如何实现这一点。

任何帮助将是AP preciated。
先谢谢了。


解决方案

我以前做过这个,有太多code就摆在这里,所以请允许我简单地把轮廓,因为我相信你可以以实际实施和配置的护理:

Socket.io基于微服务实时事件发送到浏览器

首先,Django是同步的,所以它不容易做任何实时吧。

于是我使出了socket.io过程。你可以说这是一个微服务,只有听,这是Redis的支持的通道,并发送通知到侦听给定的通道浏览器客户端。

芹菜 - > Redis的 - > Socket.io - >浏览器

我说得那么每个通道都被标识一个芹菜任务ID。所以,当我火从浏览器中的芹菜任务,我得到的任务ID,保持并开始听经该通道从socket.io事件。

在时间顺序,它看起来是这样的:


  • 关火芹菜任务,得到了ID

  • 保存ID在你的客户端应用程序,打开一个通道socket.io监听更新

  • 芹菜任务将消息发送到Redis的,这将触发事件socket.io

  • Socket.io中继信息到浏览器,实时

报告进度

至于任务的状态,实际更新,我只是让这个芹菜任务,其code之内,发送消息上Redis的喜欢的东西例如 {'做':2,'total_to_be_done':10} (重新present经过两个出去了10个步骤的任务,20%的进度,我preFER同时发送号码为更好的UI / UX)

 进口的Redis
redis_pub = redis.StrictRedis()
通道='任务:其中,TASK_ID>:进步
redis_pub.publish(通道,json.dumps({'做':2,'total_to_be_done':10}))

查找文档与Python

AngularJS / Socket.io融合

您可以使用或至少像角插座IO库

I have a main celery task that starts multiple sub-tasks (thousands) doing multiple actions (same actions per sub-task).

What i want is, from the main celery task to track in real-time for each action, how many are done and how many have failed for each sub-task.

In summary!

So i need to know from the main task the real-time progress of the sub-tasks.

The app i am developing is using django/angularJs, and i need to show the real-time progress asynchronously in the front-end.

I am new to celery, and i am confused and don't know how to implement this.

Any help would be appreciated.Thanks in advance.

I have done this before, there's too much code to put in here, so please allow me to simply put the outline, as I trust you can take care of the actual implementation and configuration:

Socket.io-based microservice to send real time events to browser

First, Django is synchronous, so it's not easy doing anything real time with it.

So I resorted to a socket.io process. You could say it's a microservice that only listens to a "channel" that was Redis-backed, and sends notifications to a browser client that listens to a given channel.

Celery -> Redis -> Socket.io -> Browser

I made it so each channel is identified with a Celery task ID. So when I fire a celery task from browser, I get the task ID, keep it and start listening to events from socket.io via that channel.

In chronological order it looks like this:

Reporting the progress

As for the actual updating of the status of the task, I just make it so that the Celery task, within its code, sends a message on Redis with something like e.g. {'done': 2, 'total_to_be_done': 10} (to represent a task that went through 2 out of 10 steps, a 20% progress, I prefer to send both numbers for better UI/UX)

import redis
redis_pub = redis.StrictRedis()
channel = 'task:<task_id>:progress'
redis_pub.publish(channel, json.dumps({'done': 2, 'total_to_be_done': 10}))

Find documentation for publishing messages on Redis with Python here

AngularJS/Socket.io integration

You can use or at least get some inspiration from a library like angular-socket-io

这篇关于芹菜任务实时进度跟踪的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!

06-29 08:09