We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Currently, there are always disconnections and reconnections, resulting in the inability to process tasks quickly
import os
from kombu import Exchange, Queue
from core.config import settings
broker_url = settings.CELERY_BROKER_URL result_backend = settings.CELERY_RESULT_BACKEND timezone = 'Asia/Shanghai' # 设置时区为北京时间 enable_utc = True # 启用UTC时间 result_expires = settings.CELERY_TASK_TIMEOUT # 设置结果过期时间
js_code = ""
current_file_path = os.path.abspath(file)
current_dir = os.path.dirname(current_file_path)
parent_dir = os.path.dirname(current_dir)
task_queues = ( Queue('celery', Exchange('celery'), routing_key='celery'), # 默认 Queue(settings.CELERY_TASK_QUEUE, Exchange(settings.CELERY_TASK_QUEUE), routing_key=settings.CELERY_TASK_QUEUE), # 采集 Queue(settings.CELERY_FOLLOW_QUEUE, Exchange(settings.CELERY_FOLLOW_QUEUE), routing_key=settings.CELERY_FOLLOW_QUEUE), # 养号 Queue(settings.CELERY_GROUP_QUEUE, Exchange(settings.CELERY_GROUP_QUEUE), routing_key=settings.CELERY_GROUP_QUEUE), # 拉群 Queue(settings.CELERY_LOGIN_QUEUE, Exchange(settings.CELERY_LOGIN_QUEUE), routing_key=settings.CELERY_LOGIN_QUEUE), # 登录 Queue(settings.CELERY_MESSAGE_QUEUE, Exchange(settings.CELERY_MESSAGE_QUEUE), routing_key=settings.CELERY_MESSAGE_QUEUE), # 私信 Queue(settings.CELERY_MODIFY_ACCOUNT_INFO_QUEUE, Exchange(settings.CELERY_MODIFY_ACCOUNT_INFO_QUEUE), routing_key=settings.CELERY_MODIFY_ACCOUNT_INFO_QUEUE), # 私信 Queue(settings.CELERY_TASK_SUMMARY_QUEUE, Exchange(settings.CELERY_TASK_SUMMARY_QUEUE), routing_key=settings.CELERY_TASK_SUMMARY_QUEUE), # 任务总结 Queue(settings.CELERY_TASK_START_QUEUE, Exchange(settings.CELERY_TASK_START_QUEUE), routing_key=settings.CELERY_TASK_START_QUEUE), # 任务开始 Queue(settings.CELERY_MESSAGE_START_QUEUE, Exchange(settings.CELERY_MESSAGE_START_QUEUE), routing_key=settings.CELERY_MESSAGE_START_QUEUE), # 任务开始 Queue(settings.CELERY_COLLECT_START_QUEUE, Exchange(settings.CELERY_COLLECT_START_QUEUE), routing_key=settings.CELERY_COLLECT_START_QUEUE), # 任务开始 ) task_acks_late = True
task_max_retries = None
task_default_retry_delay = 5
broker_heartbeat = 10
broker_heartbeat_checkrate = 2
broker_connection_timeout = 15
broker_connection_retry = True
broker_connection_max_retries = None
worker_pool_restarts = True
broker_transport_options = { 'max_retries': 3, 'interval_start': 0, 'interval_step': 10, 'interval_max': 30 } broker_connection_retry_on_startup = True, # 启动时重试连接代理
worker_cancel_long_running_tasks_on_connection_loss = True
redis_backend_health_check_interval = 10
broker_pool_limit = 0
worker_prefetch_multiplier = 1
This is my configuration file
The text was updated successfully, but these errors were encountered:
we don't understand Chinese. English is the suggested default language.
Sorry, something went wrong.
No branches or pull requests
Currently, there are always disconnections and reconnections, resulting in the inability to process tasks quickly
import os
from kombu import Exchange, Queue
from core.config import settings
broker_url = settings.CELERY_BROKER_URL
result_backend = settings.CELERY_RESULT_BACKEND
timezone = 'Asia/Shanghai' # 设置时区为北京时间
enable_utc = True # 启用UTC时间
result_expires = settings.CELERY_TASK_TIMEOUT # 设置结果过期时间
确保序列化器配置正确
serialization.registry._decoders.pop("application/x-python-serialize")
js_code = ""
从文件中加载JavaScript代码
获取当前文件的完整路径
current_file_path = os.path.abspath(file)
获取当前文件所在的目录
current_dir = os.path.dirname(current_file_path)
获取当前目录的上级目录
parent_dir = os.path.dirname(current_dir)
task_queues = (
Queue('celery', Exchange('celery'), routing_key='celery'), # 默认
Queue(settings.CELERY_TASK_QUEUE, Exchange(settings.CELERY_TASK_QUEUE), routing_key=settings.CELERY_TASK_QUEUE), # 采集
Queue(settings.CELERY_FOLLOW_QUEUE, Exchange(settings.CELERY_FOLLOW_QUEUE), routing_key=settings.CELERY_FOLLOW_QUEUE), # 养号
Queue(settings.CELERY_GROUP_QUEUE, Exchange(settings.CELERY_GROUP_QUEUE), routing_key=settings.CELERY_GROUP_QUEUE), # 拉群
Queue(settings.CELERY_LOGIN_QUEUE, Exchange(settings.CELERY_LOGIN_QUEUE), routing_key=settings.CELERY_LOGIN_QUEUE), # 登录
Queue(settings.CELERY_MESSAGE_QUEUE, Exchange(settings.CELERY_MESSAGE_QUEUE), routing_key=settings.CELERY_MESSAGE_QUEUE), # 私信
Queue(settings.CELERY_MODIFY_ACCOUNT_INFO_QUEUE, Exchange(settings.CELERY_MODIFY_ACCOUNT_INFO_QUEUE), routing_key=settings.CELERY_MODIFY_ACCOUNT_INFO_QUEUE), # 私信
Queue(settings.CELERY_TASK_SUMMARY_QUEUE, Exchange(settings.CELERY_TASK_SUMMARY_QUEUE), routing_key=settings.CELERY_TASK_SUMMARY_QUEUE), # 任务总结
Queue(settings.CELERY_TASK_START_QUEUE, Exchange(settings.CELERY_TASK_START_QUEUE), routing_key=settings.CELERY_TASK_START_QUEUE), # 任务开始
Queue(settings.CELERY_MESSAGE_START_QUEUE, Exchange(settings.CELERY_MESSAGE_START_QUEUE), routing_key=settings.CELERY_MESSAGE_START_QUEUE), # 任务开始
Queue(settings.CELERY_COLLECT_START_QUEUE, Exchange(settings.CELERY_COLLECT_START_QUEUE), routing_key=settings.CELERY_COLLECT_START_QUEUE), # 任务开始
)
task_acks_late = True
即使task_acks_late启用,当执行任务的工作进程突然退出或收到信号(例如KILL/INT等)时,工作人员也会确认任务。
将其设置为 true 允许消息重新排队,以便任务将由同一工作人员或另一个工作人员再次执行。
task_reject_on_worker_lost = True
最大重试次数,None 表示无限重试
task_max_retries = None
默认重试延迟(秒)
task_default_retry_delay = 5
设置心跳间隔为10秒
broker_heartbeat = 10
设置心跳检查率
broker_heartbeat_checkrate = 2
连接超时秒数
broker_connection_timeout = 15
在连接失败时重试
broker_connection_retry = True
连接尝试的最大重试次数,None表示无限重试
broker_connection_max_retries = None
worker 的进程池重启功能
worker_pool_restarts = True
# redis 专用配置
broker_transport_options = {
'max_retries': 3,
'interval_start': 0,
'interval_step': 10, 'interval_max': 30
}
broker_connection_retry_on_startup = True, # 启动时重试连接代理
当连接丢失时取消所有当前执行的具有延迟确认(late acknowledgement)的任务。这些任务因为连接丢失无法被确认,并会自动重新发送回队列。
worker_cancel_long_running_tasks_on_connection_loss = True
如果启用,后端将尝试在发生可恢复异常时重试,而不是传播异常。它将在两次重试之间使用指数退避睡眠时间。
result_backend_always_retry = True
Redis后端支持健康检查。该值必须设置为整数,其值为运行状况检查之间的秒数。如果在健康检查期间遇到 ConnectionError 或 TimeoutError,将重新建立连接并且命令会重试一次。
redis_backend_health_check_interval = 10
2.3 版本中的新功能。
默认值:10。
连接池中可以打开的最大连接数。
从 2.5 版本开始,该池默认启用,默认限制为 10 个连接。这个数字可以根据使用连接的线程/绿色线程(eventlet/gevent)的数量进行调整。例如,运行具有 1000 个使用代理连接的 greenlet 的 eventlet,可能会出现争用,您应该考虑增加限制。
如果设置为None0 或 0,连接池将被禁用,并且每次使用时都会建立和关闭连接。
broker_pool_limit = 0
默认值:4。
一次预取多少条消息乘以并发进程数。默认值为 4(每个进程四条消息)。默认设置通常是一个不错的选择,但是,如果您有很长时间运行的任务在队列中等待,并且必须启动工作程序,请注意,第一个启动的工作程序将收到最初消息数量的四倍。因此,任务可能无法公平地分配给工人。
要禁用预取,请设置worker_prefetch_multiplier为 1。将该设置更改为 0 将允许工作线程继续使用所需数量的消息。
有关预取的更多信息,请阅读预取限制
worker_prefetch_multiplier = 1
This is my configuration file
The text was updated successfully, but these errors were encountered: