django-q — 比django-celery和django-rq更简单又不失强大的django队列

django-q介绍

Django Q is a native Django task queue, scheduler and worker application using Python multiprocessing.

Features

Multiprocessing worker pools
Asynchronous tasks
Scheduled, cron and repeated tasks
Signed and compressed packages
Failure and success database or cache
Result hooks, groups and chains
Django Admin integration
PaaS compatible with multiple instances
Multi cluster monitor
Redis, Disque, IronMQ, SQS, MongoDB or ORM
Rollbar and Sentry support

Django Q is tested with: Python 3.7 and 3.8, Django 2.2.x and 3.1.x

安装django-q模块

pip install django-q

增加 django_q 到 INSTALLED_APPS:

修改settings.py

INSTALLED_APPS = (
    # other apps
    'django_q',
)

创建数据库表:

运行 Django migrations

$ python manage.py migrate

配置一个broker

使用django orm数据库作为broker

Q_CLUSTER = {
    'name': 'DjangORM',
    'workers': 1,
    'timeout': 90,
    'retry': 120,
    'queue_limit': 50,
    'bulk': 10,
    'orm': 'default'
}

使用redis作为broker

Q_CLUSTER = {
    'redis': 'redis://h:asdfqwer1234asdf@ec2-111-1-1-1.compute-1.amazonaws.com:111'
}

使用django_redis作为broker

Q_CLUSTER = {
    'name': 'DJRedis',
    'workers': 4,
    'timeout': 90,
    'django_redis': 'default'
}

其他broker： https://django-q.readthedocs.io/en/latest/brokers.html

使用qcluster处理异步任务

python manage.py qcluster

增加异步任务到队列

from django_q.tasks import async_task, result

# create the task
async_task('math.copysign', 2, -2)

# or with import and storing the id
import math.copysign

task_id = async_task(copysign, 2, -2)

# get the result
task_result = result(task_id)

# result returns None if the task has not been executed yet
# you can wait for it
task_result = result(task_id, 200)

# but in most cases you will want to use a hook:

async_task('math.modf', 2.5, hook='hooks.print_result')

# hooks.py
def print_result(task):
    print(task.result)

管理后台页面

Django Q不使用自定义页面，而是默认使用Django模型管理员提供的功能。当您打开Django Q的管理页面时，您将看到三种模型：

成功的任务

意味着他们在执行过程中没有遇到任何错误。

file

失败的任务

失败的任务遇到错误，阻止其完成执行。

计划任务

在这里，您可以检查计划任务的状态，创建，编辑或删除它们。