Skip to content

fix: avoid cycle detection in native delayed delivery#10095

Merged
auvipy merged 3 commits into
celery:mainfrom
Izzette:fix-rabbitmq-3.x-cycle-detection-native-delayed-delivery
Feb 11, 2026
Merged

fix: avoid cycle detection in native delayed delivery#10095
auvipy merged 3 commits into
celery:mainfrom
Izzette:fix-rabbitmq-3.x-cycle-detection-native-delayed-delivery

Conversation

@Izzette

@Izzette Izzette commented Feb 6, 2026

Copy link
Copy Markdown
Contributor

Avoid RabbitMQ <4.0.1 dead-lettering cycle detection when retrying with native delayed-delivery.
This is achieved by stripping the x-death-related headers before publishing tasks.

Fixes #9867.

When running with RabbitMQ 3.13 before applying the fix we are able to see a reproduction of the issue in the integration testing environment.
Some breadcrumbs have been added to display the message headers and properties for the task being published (retried).
We can see that after the task is published for retry with the x-death headers, the task is not received by the worker and instead is lost.
A timeout is reached before the result of the task execution is available.
Note that the "retry: Retry in 3s: TaskFailedException('Simulated failure')" is printed just after the task is published for retry
The test error output is added in the following comment to avoid exceeding the pull request description limit.

We only see Dead-letter queues cycle detected for source quorum queue logged once by RabbitMQ, as it will not log twice for the same cycle without being restarted.

RabbitMQ logs
rabbit-1  | 2026-02-06 09:37:44.334099+00:00 [notice] <0.44.0> Application syslog exited with reason: stopped
rabbit-1  | 2026-02-06 09:37:44.339656+00:00 [notice] <0.254.0> Logging: switching to configured handler(s); following messages may not be visible in this log output
rabbit-1  | 2026-02-06 09:37:44.339967+00:00 [notice] <0.254.0> Logging: configured log handlers are now ACTIVE
rabbit-1  | 2026-02-06 09:37:44.347824+00:00 [info] <0.254.0> ra: starting system quorum_queues
rabbit-1  | 2026-02-06 09:37:44.347851+00:00 [info] <0.254.0> starting Ra system: quorum_queues in directory: /var/lib/rabbitmq/mnesia/rabbit@07a94984080a/quorum/rabbit@07a94984080a
rabbit-1  | 2026-02-06 09:37:44.401717+00:00 [info] <0.268.0> ra system 'quorum_queues' running pre init for 0 registered servers
rabbit-1  | 2026-02-06 09:37:44.411019+00:00 [info] <0.269.0> ra: meta data store initialised for system quorum_queues. 0 record(s) recovered
rabbit-1  | 2026-02-06 09:37:44.424220+00:00 [notice] <0.274.0> WAL: ra_log_wal init, open tbls: ra_log_open_mem_tables, closed tbls: ra_log_closed_mem_tables
rabbit-1  | 2026-02-06 09:37:44.437265+00:00 [info] <0.254.0> ra: starting system coordination
rabbit-1  | 2026-02-06 09:37:44.437288+00:00 [info] <0.254.0> starting Ra system: coordination in directory: /var/lib/rabbitmq/mnesia/rabbit@07a94984080a/coordination/rabbit@07a94984080a
rabbit-1  | 2026-02-06 09:37:44.438444+00:00 [info] <0.282.0> ra system 'coordination' running pre init for 0 registered servers
rabbit-1  | 2026-02-06 09:37:44.439158+00:00 [info] <0.283.0> ra: meta data store initialised for system coordination. 0 record(s) recovered
rabbit-1  | 2026-02-06 09:37:44.439236+00:00 [notice] <0.288.0> WAL: ra_coordination_log_wal init, open tbls: ra_coordination_log_open_mem_tables, closed tbls: ra_coordination_log_closed_mem_tables
rabbit-1  | 2026-02-06 09:37:44.440742+00:00 [info] <0.254.0> ra: starting system coordination
rabbit-1  | 2026-02-06 09:37:44.440765+00:00 [info] <0.254.0> starting Ra system: coordination in directory: /var/lib/rabbitmq/mnesia/rabbit@07a94984080a/coordination/rabbit@07a94984080a
rabbit-1  | 2026-02-06 09:37:44.516454+00:00 [info] <0.254.0> Waiting for Khepri leader for 30000 ms, 9 retries left
rabbit-1  | 2026-02-06 09:37:44.519737+00:00 [notice] <0.292.0> RabbitMQ metadata store: candidate -> leader in term: 1 machine version: 1
rabbit-1  | 2026-02-06 09:37:44.528764+00:00 [info] <0.254.0> Khepri leader elected
rabbit-1  | 2026-02-06 09:37:44.528797+00:00 [info] <0.254.0> Waiting for Khepri projections for 30000 ms, 9 retries left
rabbit-1  | 2026-02-06 09:37:44.747010+00:00 [info] <0.254.0> 
rabbit-1  | 2026-02-06 09:37:44.747010+00:00 [info] <0.254.0>  Starting RabbitMQ 3.13.7 on Erlang 26.2.5.16 [jit]
rabbit-1  | 2026-02-06 09:37:44.747010+00:00 [info] <0.254.0>  Copyright (c) 2007-2024 Broadcom Inc and/or its subsidiaries
rabbit-1  | 2026-02-06 09:37:44.747010+00:00 [info] <0.254.0>  Licensed under the MPL 2.0. Website: https://rabbitmq.com
rabbit-1  | 
rabbit-1  |   ##  ##      RabbitMQ 3.13.7
rabbit-1  |   ##  ##
rabbit-1  |   ##########  Copyright (c) 2007-2024 Broadcom Inc and/or its subsidiaries
rabbit-1  |   ######  ##
rabbit-1  |   ##########  Licensed under the MPL 2.0. Website: https://rabbitmq.com
rabbit-1  | 
rabbit-1  |   Erlang:      26.2.5.16 [jit]
rabbit-1  |   TLS Library: OpenSSL - OpenSSL 3.1.8 11 Feb 2025
rabbit-1  |   Release series support status: see https://www.rabbitmq.com/release-information
rabbit-1  | 
rabbit-1  |   Doc guides:  https://www.rabbitmq.com/docs
rabbit-1  |   Support:     https://www.rabbitmq.com/docs/contact
rabbit-1  |   Tutorials:   https://www.rabbitmq.com/tutorials
rabbit-1  |   Monitoring:  https://www.rabbitmq.com/docs/monitoring
rabbit-1  |   Upgrading:   https://www.rabbitmq.com/docs/upgrade
rabbit-1  | 
rabbit-1  |   Logs: <stdout>
rabbit-1  | 
rabbit-1  |   Config file(s): /etc/rabbitmq/conf.d/10-defaults.conf
rabbit-1  | 
rabbit-1  |   Starting broker...2026-02-06 09:37:44.748049+00:00 [info] <0.254.0> 
rabbit-1  | 2026-02-06 09:37:44.748049+00:00 [info] <0.254.0>  node           : rabbit@07a94984080a
rabbit-1  | 2026-02-06 09:37:44.748049+00:00 [info] <0.254.0>  home dir       : /var/lib/rabbitmq
rabbit-1  | 2026-02-06 09:37:44.748049+00:00 [info] <0.254.0>  config file(s) : /etc/rabbitmq/conf.d/10-defaults.conf
rabbit-1  | 2026-02-06 09:37:44.748049+00:00 [info] <0.254.0>  cookie hash    : 9QRGhVCYWs6SaDzOZ8pF9Q==
rabbit-1  | 2026-02-06 09:37:44.748049+00:00 [info] <0.254.0>  log(s)         : <stdout>
rabbit-1  | 2026-02-06 09:37:44.748049+00:00 [info] <0.254.0>  data dir       : /var/lib/rabbitmq/mnesia/rabbit@07a94984080a
rabbit-1  | 2026-02-06 09:37:44.985741+00:00 [info] <0.254.0> Running boot step pre_boot defined by app rabbit
rabbit-1  | 2026-02-06 09:37:44.985780+00:00 [info] <0.254.0> Running boot step rabbit_global_counters defined by app rabbit
rabbit-1  | 2026-02-06 09:37:44.985980+00:00 [info] <0.254.0> Running boot step rabbit_osiris_metrics defined by app rabbit
rabbit-1  | 2026-02-06 09:37:44.986039+00:00 [info] <0.254.0> Running boot step rabbit_core_metrics defined by app rabbit
rabbit-1  | 2026-02-06 09:37:44.986192+00:00 [info] <0.254.0> Running boot step rabbit_alarm defined by app rabbit
rabbit-1  | 2026-02-06 09:37:44.995348+00:00 [info] <0.329.0> Memory high watermark set to 1566 MiB (1642289561 bytes) of 3915 MiB (4105723904 bytes) total
rabbit-1  | 2026-02-06 09:37:44.996828+00:00 [info] <0.331.0> Enabling free disk space monitoring (disk free space: 86319226880, total memory: 4105723904)
rabbit-1  | 2026-02-06 09:37:44.996865+00:00 [info] <0.331.0> Disk free limit set to 50MB
rabbit-1  | 2026-02-06 09:37:44.997598+00:00 [info] <0.254.0> Running boot step code_server_cache defined by app rabbit
rabbit-1  | 2026-02-06 09:37:44.997640+00:00 [info] <0.254.0> Running boot step file_handle_cache defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.004144+00:00 [info] <0.334.0> Limiting to approx 1048479 file handles (943629 sockets)
rabbit-1  | 2026-02-06 09:37:45.004222+00:00 [info] <0.335.0> FHC read buffering: OFF
rabbit-1  | 2026-02-06 09:37:45.004242+00:00 [info] <0.335.0> FHC write buffering: ON
rabbit-1  | 2026-02-06 09:37:45.004660+00:00 [info] <0.254.0> Running boot step worker_pool defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.004698+00:00 [info] <0.315.0> Will use 2 processes for default worker pool
rabbit-1  | 2026-02-06 09:37:45.004739+00:00 [info] <0.315.0> Starting worker pool 'worker_pool' with 2 processes in it
rabbit-1  | 2026-02-06 09:37:45.004951+00:00 [info] <0.254.0> Running boot step database defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.005281+00:00 [info] <0.254.0> Peer discovery: configured backend: rabbit_peer_discovery_classic_config
rabbit-1  | 2026-02-06 09:37:45.006110+00:00 [notice] <0.316.0> Feature flags: attempt to enable `detailed_queues_endpoint`...
rabbit-1  | 2026-02-06 09:37:45.055078+00:00 [notice] <0.316.0> Feature flags: `detailed_queues_endpoint` enabled
rabbit-1  | 2026-02-06 09:37:45.055186+00:00 [notice] <0.316.0> Feature flags: attempt to enable `quorum_queue_non_voters`...
rabbit-1  | 2026-02-06 09:37:45.103574+00:00 [notice] <0.316.0> Feature flags: `quorum_queue_non_voters` enabled
rabbit-1  | 2026-02-06 09:37:45.103671+00:00 [notice] <0.316.0> Feature flags: attempt to enable `stream_update_config_command`...
rabbit-1  | 2026-02-06 09:37:45.151419+00:00 [notice] <0.316.0> Feature flags: `stream_update_config_command` enabled
rabbit-1  | 2026-02-06 09:37:45.151531+00:00 [notice] <0.316.0> Feature flags: attempt to enable `stream_filtering`...
rabbit-1  | 2026-02-06 09:37:45.198648+00:00 [notice] <0.316.0> Feature flags: `stream_filtering` enabled
rabbit-1  | 2026-02-06 09:37:45.198766+00:00 [notice] <0.316.0> Feature flags: attempt to enable `stream_sac_coordinator_unblock_group`...
rabbit-1  | 2026-02-06 09:37:45.245995+00:00 [notice] <0.316.0> Feature flags: `stream_sac_coordinator_unblock_group` enabled
rabbit-1  | 2026-02-06 09:37:45.246116+00:00 [notice] <0.316.0> Feature flags: attempt to enable `restart_streams`...
rabbit-1  | 2026-02-06 09:37:45.293200+00:00 [notice] <0.316.0> Feature flags: `restart_streams` enabled
rabbit-1  | 2026-02-06 09:37:45.293298+00:00 [notice] <0.316.0> Feature flags: attempt to enable `message_containers`...
rabbit-1  | 2026-02-06 09:37:45.339335+00:00 [notice] <0.316.0> Feature flags: `message_containers` enabled
rabbit-1  | 2026-02-06 09:37:45.339432+00:00 [notice] <0.316.0> Feature flags: attempt to enable `message_containers_deaths_v2`...
rabbit-1  | 2026-02-06 09:37:45.385134+00:00 [notice] <0.316.0> Feature flags: `message_containers_deaths_v2` enabled
rabbit-1  | 2026-02-06 09:37:45.385280+00:00 [info] <0.254.0> DB: virgin node -> run peer discovery
rabbit-1  | 2026-02-06 09:37:45.385315+00:00 [warning] <0.254.0> Classic peer discovery backend: list of nodes does not contain the local node []
rabbit-1  | 2026-02-06 09:37:45.394394+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped
rabbit-1  | 2026-02-06 09:37:45.490494+00:00 [info] <0.254.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
rabbit-1  | 2026-02-06 09:37:45.490565+00:00 [info] <0.254.0> Successfully synced tables from a peer
rabbit-1  | 2026-02-06 09:37:45.490610+00:00 [info] <0.254.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
rabbit-1  | 2026-02-06 09:37:45.490649+00:00 [info] <0.254.0> Successfully synced tables from a peer
rabbit-1  | 2026-02-06 09:37:45.493609+00:00 [info] <0.254.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
rabbit-1  | 2026-02-06 09:37:45.493675+00:00 [info] <0.254.0> Successfully synced tables from a peer
rabbit-1  | 2026-02-06 09:37:45.493730+00:00 [info] <0.254.0> Running boot step tracking_metadata_store defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.493765+00:00 [info] <0.556.0> Setting up a table for connection tracking on this node: tracked_connection
rabbit-1  | 2026-02-06 09:37:45.493823+00:00 [info] <0.556.0> Setting up a table for per-vhost connection counting on this node: tracked_connection_per_vhost
rabbit-1  | 2026-02-06 09:37:45.493854+00:00 [info] <0.556.0> Setting up a table for per-user connection counting on this node: tracked_connection_per_user
rabbit-1  | 2026-02-06 09:37:45.493898+00:00 [info] <0.556.0> Setting up a table for channel tracking on this node: tracked_channel
rabbit-1  | 2026-02-06 09:37:45.493943+00:00 [info] <0.556.0> Setting up a table for channel tracking on this node: tracked_channel_per_user
rabbit-1  | 2026-02-06 09:37:45.494014+00:00 [info] <0.254.0> Running boot step networking_metadata_store defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494063+00:00 [info] <0.254.0> Running boot step feature_flags defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494328+00:00 [info] <0.254.0> Running boot step codec_correctness_check defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494362+00:00 [info] <0.254.0> Running boot step external_infrastructure defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494382+00:00 [info] <0.254.0> Running boot step rabbit_event defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494461+00:00 [info] <0.254.0> Running boot step rabbit_registry defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494503+00:00 [info] <0.254.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494560+00:00 [info] <0.254.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494592+00:00 [info] <0.254.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494620+00:00 [info] <0.254.0> Running boot step rabbit_exchange_type_direct defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494653+00:00 [info] <0.254.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494685+00:00 [info] <0.254.0> Running boot step rabbit_exchange_type_headers defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494713+00:00 [info] <0.254.0> Running boot step rabbit_exchange_type_topic defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494741+00:00 [info] <0.254.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494781+00:00 [info] <0.254.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494813+00:00 [info] <0.254.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494840+00:00 [info] <0.254.0> Running boot step rabbit_priority_queue defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494853+00:00 [info] <0.254.0> Priority queues enabled, real BQ is rabbit_variable_queue
rabbit-1  | 2026-02-06 09:37:45.494884+00:00 [info] <0.254.0> Running boot step rabbit_queue_location_client_local defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494911+00:00 [info] <0.254.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494947+00:00 [info] <0.254.0> Running boot step rabbit_queue_location_random defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494973+00:00 [info] <0.254.0> Running boot step kernel_ready defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.494983+00:00 [info] <0.254.0> Running boot step rabbit_sysmon_minder defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.495043+00:00 [info] <0.254.0> Running boot step rabbit_epmd_monitor defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.495617+00:00 [info] <0.564.0> epmd monitor knows us, inter-node communication (distribution) port: 25672
rabbit-1  | 2026-02-06 09:37:45.495654+00:00 [info] <0.254.0> Running boot step guid_generator defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.496684+00:00 [info] <0.254.0> Running boot step rabbit_node_monitor defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.496871+00:00 [info] <0.568.0> Starting rabbit_node_monitor (in ignore mode)
rabbit-1  | 2026-02-06 09:37:45.496919+00:00 [info] <0.254.0> Running boot step delegate_sup defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.497314+00:00 [info] <0.254.0> Running boot step rabbit_memory_monitor defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.497420+00:00 [info] <0.254.0> Running boot step rabbit_fifo_dlx_sup defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.497453+00:00 [info] <0.254.0> Running boot step core_initialized defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.497465+00:00 [info] <0.254.0> Running boot step rabbit_channel_tracking_handler defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.497504+00:00 [info] <0.254.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.497530+00:00 [info] <0.254.0> Running boot step rabbit_definitions_hashing defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.497561+00:00 [info] <0.254.0> Running boot step rabbit_exchange_parameters defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.518059+00:00 [info] <0.254.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.518306+00:00 [info] <0.254.0> Running boot step rabbit_policies defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.518532+00:00 [info] <0.254.0> Running boot step rabbit_policy defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.518570+00:00 [info] <0.254.0> Running boot step rabbit_queue_location_validator defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.518715+00:00 [info] <0.254.0> Running boot step rabbit_quorum_memory_manager defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.518742+00:00 [info] <0.254.0> Running boot step rabbit_quorum_queue defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.518844+00:00 [info] <0.254.0> Running boot step rabbit_stream_coordinator defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.518976+00:00 [info] <0.254.0> Running boot step rabbit_vhost_limit defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.519011+00:00 [info] <0.254.0> Running boot step rabbit_federation_parameters defined by app rabbitmq_federation
rabbit-1  | 2026-02-06 09:37:45.519066+00:00 [info] <0.254.0> Running boot step rabbit_federation_supervisor defined by app rabbitmq_federation
rabbit-1  | 2026-02-06 09:37:45.524818+00:00 [info] <0.254.0> Running boot step rabbit_federation_queue defined by app rabbitmq_federation
rabbit-1  | 2026-02-06 09:37:45.524897+00:00 [info] <0.254.0> Running boot step rabbit_federation_upstream_exchange defined by app rabbitmq_federation
rabbit-1  | 2026-02-06 09:37:45.524950+00:00 [info] <0.254.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management
rabbit-1  | 2026-02-06 09:37:45.524971+00:00 [info] <0.254.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent
rabbit-1  | 2026-02-06 09:37:45.524986+00:00 [info] <0.254.0> Management plugin: using rates mode 'basic'
rabbit-1  | 2026-02-06 09:37:45.525132+00:00 [info] <0.254.0> Running boot step recovery defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.532907+00:00 [info] <0.254.0> Running boot step empty_db_check defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.532937+00:00 [info] <0.254.0> Will seed default virtual host and user...
rabbit-1  | 2026-02-06 09:37:45.532996+00:00 [info] <0.254.0> Adding vhost '/' (description: 'Default virtual host', tags: [])
rabbit-1  | 2026-02-06 09:37:45.538828+00:00 [info] <0.628.0> Making sure data directory '/var/lib/rabbitmq/mnesia/rabbit@07a94984080a/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists
rabbit-1  | 2026-02-06 09:37:45.539475+00:00 [info] <0.628.0> Setting segment_entry_count for vhost '/' with 0 queues to '2048'
rabbit-1  | 2026-02-06 09:37:45.546012+00:00 [info] <0.628.0> Starting message stores for vhost '/'
rabbit-1  | 2026-02-06 09:37:45.546108+00:00 [info] <0.637.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index
rabbit-1  | 2026-02-06 09:37:45.547058+00:00 [info] <0.628.0> Started message store of type transient for vhost '/'
rabbit-1  | 2026-02-06 09:37:45.547151+00:00 [info] <0.641.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index
rabbit-1  | 2026-02-06 09:37:45.547500+00:00 [warning] <0.641.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch
rabbit-1  | 2026-02-06 09:37:45.548016+00:00 [info] <0.628.0> Started message store of type persistent for vhost '/'
rabbit-1  | 2026-02-06 09:37:45.548080+00:00 [info] <0.628.0> Recovering 0 queues of type rabbit_classic_queue took 8ms
rabbit-1  | 2026-02-06 09:37:45.548099+00:00 [info] <0.628.0> Recovering 0 queues of type rabbit_quorum_queue took 0ms
rabbit-1  | 2026-02-06 09:37:45.548123+00:00 [info] <0.628.0> Recovering 0 queues of type rabbit_stream_queue took 0ms
rabbit-1  | 2026-02-06 09:37:45.549396+00:00 [info] <0.254.0> Created user 'guest'
rabbit-1  | 2026-02-06 09:37:45.550057+00:00 [info] <0.254.0> Successfully set user tags for user 'guest' to [administrator]
rabbit-1  | 2026-02-06 09:37:45.551586+00:00 [info] <0.254.0> Successfully set permissions for user 'guest' in virtual host '/' to '.*', '.*', '.*'
rabbit-1  | 2026-02-06 09:37:45.551611+00:00 [info] <0.254.0> Running boot step rabbit_observer_cli defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.551655+00:00 [info] <0.254.0> Running boot step rabbit_looking_glass defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.551677+00:00 [info] <0.254.0> Running boot step rabbit_core_metrics_gc defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.551756+00:00 [info] <0.254.0> Running boot step background_gc defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.551817+00:00 [info] <0.254.0> Running boot step routing_ready defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.551831+00:00 [info] <0.254.0> Running boot step pre_flight defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.551848+00:00 [info] <0.254.0> Running boot step notify_cluster defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.551872+00:00 [info] <0.254.0> Running boot step networking defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.551888+00:00 [info] <0.254.0> Running boot step rabbit_quorum_queue_periodic_membership_reconciliation defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.551984+00:00 [info] <0.254.0> Running boot step definition_import_worker_pool defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.552093+00:00 [info] <0.315.0> Starting worker pool 'definition_import_pool' with 2 processes in it
rabbit-1  | 2026-02-06 09:37:45.552358+00:00 [info] <0.254.0> Running boot step cluster_name defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.552390+00:00 [info] <0.254.0> Initialising internal cluster ID to 'rabbitmq-cluster-id-uADiRTVut7dc3iu1-DfGAg'
rabbit-1  | 2026-02-06 09:37:45.553219+00:00 [info] <0.254.0> Running boot step virtual_host_reconciliation defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.553301+00:00 [info] <0.254.0> Running boot step direct_client defined by app rabbit
rabbit-1  | 2026-02-06 09:37:45.553363+00:00 [info] <0.254.0> Running boot step rabbit_federation_exchange defined by app rabbitmq_federation
rabbit-1  | 2026-02-06 09:37:45.553515+00:00 [info] <0.254.0> Running boot step rabbit_management_load_definitions defined by app rabbitmq_management
rabbit-1  | 2026-02-06 09:37:45.553565+00:00 [info] <0.676.0> Resetting node maintenance status
rabbit-1  | 2026-02-06 09:37:45.687887+00:00 [warning] <0.705.0> Deprecated features: `management_metrics_collection`: Feature `management_metrics_collection` is deprecated.
rabbit-1  | 2026-02-06 09:37:45.687887+00:00 [warning] <0.705.0> By default, this feature can still be used for now.
rabbit-1  | 2026-02-06 09:37:45.687887+00:00 [warning] <0.705.0> Its use will not be permitted by default in a future minor RabbitMQ version and the feature will be removed from a future major RabbitMQ version; actual versions to be determined.
rabbit-1  | 2026-02-06 09:37:45.687887+00:00 [warning] <0.705.0> To continue using this feature when it is not permitted by default, set the following parameter in your configuration:
rabbit-1  | 2026-02-06 09:37:45.687887+00:00 [warning] <0.705.0>     "deprecated_features.permit.management_metrics_collection = true"
rabbit-1  | 2026-02-06 09:37:45.687887+00:00 [warning] <0.705.0> To test RabbitMQ as if the feature was removed, set this in your configuration:
rabbit-1  | 2026-02-06 09:37:45.687887+00:00 [warning] <0.705.0>     "deprecated_features.permit.management_metrics_collection = false"
rabbit-1  | 2026-02-06 09:37:48.060254+00:00 [info] <0.742.0> Management plugin: HTTP (non-TLS) listener started on port 15672
rabbit-1  | 2026-02-06 09:37:48.060353+00:00 [info] <0.772.0> Statistics database started.
rabbit-1  | 2026-02-06 09:37:48.060387+00:00 [info] <0.771.0> Starting worker pool 'management_worker_pool' with 3 processes in it
rabbit-1  | 2026-02-06 09:37:48.064337+00:00 [info] <0.790.0> Prometheus metrics: HTTP (non-TLS) listener started on port 15692
rabbit-1  | 2026-02-06 09:37:48.064410+00:00 [info] <0.676.0> Ready to start client connection listeners
rabbit-1  | 2026-02-06 09:37:48.065438+00:00 [info] <0.834.0> started TCP listener on [::]:5672
rabbit-1  |  completed with 5 plugins.
rabbit-1  | 2026-02-06 09:37:48.161644+00:00 [info] <0.676.0> Server startup complete; 5 plugins started.
rabbit-1  | 2026-02-06 09:37:48.161644+00:00 [info] <0.676.0>  * rabbitmq_prometheus
rabbit-1  | 2026-02-06 09:37:48.161644+00:00 [info] <0.676.0>  * rabbitmq_federation
rabbit-1  | 2026-02-06 09:37:48.161644+00:00 [info] <0.676.0>  * rabbitmq_management
rabbit-1  | 2026-02-06 09:37:48.161644+00:00 [info] <0.676.0>  * rabbitmq_management_agent
rabbit-1  | 2026-02-06 09:37:48.161644+00:00 [info] <0.676.0>  * rabbitmq_web_dispatch
rabbit-1  | 2026-02-06 09:37:48.166123+00:00 [info] <0.9.0> Time to start RabbitMQ: 6162 ms
rabbit-1  | 2026-02-06 09:38:09.266481+00:00 [info] <0.839.0> accepting AMQP connection <0.839.0> (172.18.0.1:37300 -> 172.18.0.3:5672)
rabbit-1  | 2026-02-06 09:38:09.271547+00:00 [info] <0.839.0> connection <0.839.0> (172.18.0.1:37300 -> 172.18.0.3:5672): user 'guest' authenticated and granted access to vhost '/'
rabbit-1  | 2026-02-06 09:38:09.275988+00:00 [info] <0.839.0> closing AMQP connection <0.839.0> (172.18.0.1:37300 -> 172.18.0.3:5672, vhost: '/', user: 'guest')
rabbit-1  | 2026-02-06 09:38:09.295719+00:00 [info] <0.855.0> accepting AMQP connection <0.855.0> (172.18.0.1:37312 -> 172.18.0.3:5672)
rabbit-1  | 2026-02-06 09:38:09.299220+00:00 [info] <0.855.0> connection <0.855.0> (172.18.0.1:37312 -> 172.18.0.3:5672): user 'guest' authenticated and granted access to vhost '/'
rabbit-1  | 2026-02-06 09:38:09.314884+00:00 [notice] <0.871.0> queue 'test_cycle_queue' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.315019+00:00 [info] <0.864.0> ra: started cluster %2F_test_cycle_queue with 1 servers
rabbit-1  | 2026-02-06 09:38:09.315019+00:00 [info] <0.864.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.315019+00:00 [info] <0.864.0> Leader: {'%2F_test_cycle_queue',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.320747+00:00 [warning] <0.879.0> Deprecated features: `transient_nonexcl_queues`: Feature `transient_nonexcl_queues` is deprecated.
rabbit-1  | 2026-02-06 09:38:09.320747+00:00 [warning] <0.879.0> By default, this feature can still be used for now.
rabbit-1  | 2026-02-06 09:38:09.320747+00:00 [warning] <0.879.0> Its use will not be permitted by default in a future minor RabbitMQ version and the feature will be removed from a future major RabbitMQ version; actual versions to be determined.
rabbit-1  | 2026-02-06 09:38:09.320747+00:00 [warning] <0.879.0> To continue using this feature when it is not permitted by default, set the following parameter in your configuration:
rabbit-1  | 2026-02-06 09:38:09.320747+00:00 [warning] <0.879.0>     "deprecated_features.permit.transient_nonexcl_queues = true"
rabbit-1  | 2026-02-06 09:38:09.320747+00:00 [warning] <0.879.0> To test RabbitMQ as if the feature was removed, set this in your configuration:
rabbit-1  | 2026-02-06 09:38:09.320747+00:00 [warning] <0.879.0>     "deprecated_features.permit.transient_nonexcl_queues = false"
rabbit-1  | 2026-02-06 09:38:09.324985+00:00 [info] <0.890.0> accepting AMQP connection <0.890.0> (172.18.0.1:37326 -> 172.18.0.3:5672)
rabbit-1  | 2026-02-06 09:38:09.326814+00:00 [info] <0.890.0> connection <0.890.0> (172.18.0.1:37326 -> 172.18.0.3:5672): user 'guest' authenticated and granted access to vhost '/'
rabbit-1  | 2026-02-06 09:38:09.336570+00:00 [notice] <0.904.0> queue 'celery_delayed_27' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.336673+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_27 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.336673+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.336673+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_27',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.349988+00:00 [notice] <0.915.0> queue 'celery_delayed_26' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.350108+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_26 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.350108+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.350108+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_26',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.364954+00:00 [notice] <0.926.0> queue 'celery_delayed_25' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.365086+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_25 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.365086+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.365086+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_25',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.378323+00:00 [notice] <0.937.0> queue 'celery_delayed_24' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.378427+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_24 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.378427+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.378427+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_24',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.390712+00:00 [notice] <0.948.0> queue 'celery_delayed_23' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.390795+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_23 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.390795+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.390795+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_23',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.402949+00:00 [notice] <0.959.0> queue 'celery_delayed_22' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.403046+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_22 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.403046+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.403046+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_22',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.414633+00:00 [notice] <0.970.0> queue 'celery_delayed_21' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.414722+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_21 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.414722+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.414722+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_21',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.426152+00:00 [notice] <0.981.0> queue 'celery_delayed_20' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.426232+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_20 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.426232+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.426232+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_20',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.436924+00:00 [notice] <0.992.0> queue 'celery_delayed_19' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.437002+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_19 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.437002+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.437002+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_19',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.447788+00:00 [notice] <0.1003.0> queue 'celery_delayed_18' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.447868+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_18 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.447868+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.447868+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_18',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.464458+00:00 [notice] <0.1014.0> queue 'celery_delayed_17' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.464601+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_17 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.464601+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.464601+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_17',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.477185+00:00 [notice] <0.1025.0> queue 'celery_delayed_16' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.477304+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_16 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.477304+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.477304+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_16',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.493876+00:00 [notice] <0.1036.0> queue 'celery_delayed_15' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.494018+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_15 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.494018+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.494018+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_15',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.504620+00:00 [notice] <0.1047.0> queue 'celery_delayed_14' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.504726+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_14 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.504726+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.504726+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_14',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.516468+00:00 [notice] <0.1058.0> queue 'celery_delayed_13' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.516560+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_13 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.516560+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.516560+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_13',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.527655+00:00 [notice] <0.1069.0> queue 'celery_delayed_12' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.527735+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_12 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.527735+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.527735+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_12',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.539300+00:00 [notice] <0.1080.0> queue 'celery_delayed_11' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.539435+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_11 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.539435+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.539435+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_11',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.550886+00:00 [notice] <0.1091.0> queue 'celery_delayed_10' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.550987+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_10 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.550987+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.550987+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_10',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.562871+00:00 [notice] <0.1102.0> queue 'celery_delayed_9' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.562957+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_9 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.562957+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.562957+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_9',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.574910+00:00 [notice] <0.1113.0> queue 'celery_delayed_8' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.575004+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_8 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.575004+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.575004+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_8',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.586950+00:00 [notice] <0.1124.0> queue 'celery_delayed_7' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.587032+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_7 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.587032+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.587032+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_7',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.598368+00:00 [notice] <0.1135.0> queue 'celery_delayed_6' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.598444+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_6 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.598444+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.598444+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_6',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.610366+00:00 [notice] <0.1146.0> queue 'celery_delayed_5' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.610447+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_5 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.610447+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.610447+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_5',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.622883+00:00 [notice] <0.1157.0> queue 'celery_delayed_4' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.622976+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_4 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.622976+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.622976+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_4',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.634845+00:00 [notice] <0.1168.0> queue 'celery_delayed_3' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.634930+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_3 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.634930+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.634930+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_3',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.645804+00:00 [notice] <0.1179.0> queue 'celery_delayed_2' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.645886+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_2 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.645886+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.645886+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_2',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.656779+00:00 [notice] <0.1190.0> queue 'celery_delayed_1' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.656872+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_1 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.656872+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.656872+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_1',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.668842+00:00 [notice] <0.1201.0> queue 'celery_delayed_0' in vhost '/': candidate -> leader in term: 1 machine version: 3
rabbit-1  | 2026-02-06 09:38:09.668943+00:00 [info] <0.897.0> ra: started cluster %2F_celery_delayed_0 with 1 servers
rabbit-1  | 2026-02-06 09:38:09.668943+00:00 [info] <0.897.0> 0 servers failed to start: []
rabbit-1  | 2026-02-06 09:38:09.668943+00:00 [info] <0.897.0> Leader: {'%2F_celery_delayed_0',rabbit@07a94984080a}
rabbit-1  | 2026-02-06 09:38:09.718733+00:00 [info] <0.890.0> closing AMQP connection <0.890.0> (172.18.0.1:37326 -> 172.18.0.3:5672, vhost: '/', user: 'guest')
rabbit-1  | 2026-02-06 09:38:09.736445+00:00 [info] <0.1281.0> accepting AMQP connection <0.1281.0> (172.18.0.1:37330 -> 172.18.0.3:5672)
rabbit-1  | 2026-02-06 09:38:09.738255+00:00 [info] <0.1281.0> connection <0.1281.0> (172.18.0.1:37330 -> 172.18.0.3:5672): user 'guest' authenticated and granted access to vhost '/'
rabbit-1  | 2026-02-06 09:38:09.747418+00:00 [info] <0.1298.0> accepting AMQP connection <0.1298.0> (172.18.0.1:37338 -> 172.18.0.3:5672)
rabbit-1  | 2026-02-06 09:38:09.749512+00:00 [info] <0.1298.0> connection <0.1298.0> (172.18.0.1:37338 -> 172.18.0.3:5672): user 'guest' authenticated and granted access to vhost '/'
rabbit-1  | 2026-02-06 09:38:14.817755+00:00 [warning] <0.1193.0> Dead-letter queues cycle detected for source quorum queue 'celery_delayed_1' in vhost '/' with dead-letter exchange exchange 'celery_delayed_0' in vhost '/' and routing keys [undefined]: [<<"celery_delayed_0">>,
rabbit-1  | 2026-02-06 09:38:14.817755+00:00 [warning] <0.1193.0>                                                                                                                                                                                             <<"celery_delayed_1">>,
rabbit-1  | 2026-02-06 09:38:14.817755+00:00 [warning] <0.1193.0>                                                                                                                                                                                             <<"celery_delayed_0">>] This message will not be logged again.
rabbit-1  | 2026-02-06 09:38:14.818416+00:00 [warning] <0.1193.0> Cannot forward any dead-letter messages from source quorum queue 'celery_delayed_1' in vhost '/' with configured dead-letter-exchange exchange 'celery_delayed_0' in vhost '/' and configured dead-letter-routing-key 'undefined'. This can happen either if the dead-letter routing topology is misconfigured (for example no queue bound to dead-letter-exchange or wrong dead-letter-routing-key configured) or if non-mirrored classic queues are bound whose host node is down. Fix this issue to prevent dead-lettered messages from piling up in the source quorum queue. This message will not be logged again.
rabbit-1  | 2026-02-06 09:38:22.835397+00:00 [info] <0.855.0> closing AMQP connection <0.855.0> (172.18.0.1:37312 -> 172.18.0.3:5672, vhost: '/', user: 'guest')
rabbit-1  | 2026-02-06 09:38:22.928588+00:00 [info] <0.1380.0> accepting AMQP connection <0.1380.0> (172.18.0.1:51494 -> 172.18.0.3:5672)
rabbit-1  | 2026-02-06 09:38:22.933372+00:00 [info] <0.1380.0> connection <0.1380.0> (172.18.0.1:51494 -> 172.18.0.3:5672): user 'guest' authenticated and granted access to vhost '/'
rabbit-1  | 2026-02-06 09:38:22.936733+00:00 [info] <0.1380.0> closing AMQP connection <0.1380.0> (172.18.0.1:51494 -> 172.18.0.3:5672, vhost: '/', user: 'guest')
rabbit-1  | 2026-02-06 09:38:22.938862+00:00 [info] <0.1395.0> accepting AMQP connection <0.1395.0> (172.18.0.1:51500 -> 172.18.0.3:5672)
rabbit-1  | 2026-02-06 09:38:22.941585+00:00 [info] <0.1395.0> connection <0.1395.0> (172.18.0.1:51500 -> 172.18.0.3:5672): user 'guest' authenticated and granted access to vhost '/'
rabbit-1  | 2026-02-06 09:38:22.954749+00:00 [info] <0.1420.0> accepting AMQP connection <0.1420.0> (172.18.0.1:51508 -> 172.18.0.3:5672)
rabbit-1  | 2026-02-06 09:38:22.957052+00:00 [info] <0.1420.0> connection <0.1420.0> (172.18.0.1:51508 -> 172.18.0.3:5672): user 'guest' authenticated and granted access to vhost '/'
rabbit-1  | 2026-02-06 09:38:23.026444+00:00 [info] <0.1420.0> closing AMQP connection <0.1420.0> (172.18.0.1:51508 -> 172.18.0.3:5672, vhost: '/', user: 'guest')
rabbit-1  | 2026-02-06 09:38:23.041386+00:00 [info] <0.1505.0> accepting AMQP connection <0.1505.0> (172.18.0.1:51510 -> 172.18.0.3:5672)
rabbit-1  | 2026-02-06 09:38:23.043178+00:00 [info] <0.1505.0> connection <0.1505.0> (172.18.0.1:51510 -> 172.18.0.3:5672): user 'guest' authenticated and granted access to vhost '/'
rabbit-1  | 2026-02-06 09:38:36.119475+00:00 [info] <0.1395.0> closing AMQP connection <0.1395.0> (172.18.0.1:51500 -> 172.18.0.3:5672, vhost: '/', user: 'guest')
rabbit-1  | 2026-02-06 09:38:36.224116+00:00 [warning] <0.1505.0> closing AMQP connection <0.1505.0> (172.18.0.1:51510 -> 172.18.0.3:5672, vhost: '/', user: 'guest'):
rabbit-1  | 2026-02-06 09:38:36.224116+00:00 [warning] <0.1505.0> client unexpectedly closed TCP connection
rabbit-1  | 2026-02-06 09:38:36.224569+00:00 [warning] <0.1298.0> closing AMQP connection <0.1298.0> (172.18.0.1:37338 -> 172.18.0.3:5672, vhost: '/', user: 'guest'):
rabbit-1  | 2026-02-06 09:38:36.224569+00:00 [warning] <0.1298.0> client unexpectedly closed TCP connection
rabbit-1  | 2026-02-06 09:38:36.323387+00:00 [warning] <0.1281.0> closing AMQP connection <0.1281.0> (172.18.0.1:37330 -> 172.18.0.3:5672, vhost: '/', user: 'guest'):
rabbit-1  | 2026-02-06 09:38:36.323387+00:00 [warning] <0.1281.0> client unexpectedly closed TCP connection

The issue is not reproducable with RabbitMQ ≥4.0.1, as the RabbitMQ server will not interpret x-death headers being published by the client: see the release notes for RabbitMQ 4.0.1.

I'm also not able to reproduce the issue using a delay of 1 second, it seems only having the last death be the same queue as the current death prevents the cycle detection from occuring. Instead of a new death that would be appended to the x-death header, but the count for the existing death is incremented. I'm not sure, but I suspect it's not possible to setup dead-lettering directly to the same queue in RabbitMQ, a different kind of cycle detection probably occurs in that case. Using a delay of 3 seconds results in the message being routed to celery_delayed_1 than celery_delayed_0, after a retry it ends up back in celery_delayed_1 where the cycle is detected by RabbitMQ.

In order to run with RabbitMQ 3.13, this patch can be applied locally, and the docker container recreated:

Patch to use RabbitMQ 3.13
diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml
index b9a29b1dd..8a9c2c0b5 100644
--- a/docker/docker-compose.yml
+++ b/docker/docker-compose.yml
@@ -25,7 +25,7 @@ services:
       - azurite

   rabbit:
-    image: rabbitmq:management
+    image: rabbitmq:3.13-management
     ports:
       - "15672:15672"
       - "5672:5672"

The instrumentation to see the message headers was done by applying this patch:

Patch to log message headers
diff --git a/celery/app/amqp.py b/celery/app/amqp.py
index 6caedc5c5..66a321fef 100644
--- a/celery/app/amqp.py
+++ b/celery/app/amqp.py
@@ -556,6 +556,23 @@ class AMQP:
                     declare=declare, headers=headers2,
                     properties=properties, retry_policy=retry_policy,
                 )
+            # DEBUG: Log publishing details
+            import logging
+            import pprint
+            logger = logging.getLogger(__name__)
+            logger.info("=" * 80)
+            logger.info(f"PUBLISHING TASK: {name}")
+            logger.info(f"  Task ID: {headers2.get('id')}")
+            logger.info(f"  Exchange: {exchange}")
+            logger.info(f"  Routing Key: {routing_key}")
+            logger.info(f"  ETA: {headers2.get('eta')}")
+            logger.info(f"  Retries: {headers2.get('retries', 0)}")
+            if 'x-death' in headers2:
+                logger.info(f"  X-Death being sent in headers:")
+                logger.info(pprint.pformat(headers2['x-death']))
+            else:
+                logger.info("  NO X-Death headers being sent")
+            logger.info("=" * 80)
             ret = producer.publish(
                 body,
                 exchange=exchange,
diff --git a/celery/worker/strategy.py b/celery/worker/strategy.py
index 6a1c6225b..54952ee7d 100644
--- a/celery/worker/strategy.py
+++ b/celery/worker/strategy.py
@@ -142,6 +142,15 @@ def default(task, app, consumer,
             else:
                 body, headers, decoded, utc = proto1_to_proto2(message, body)
 
+        # DEBUG: Print message headers to see X-Death and other headers
+        import pprint
+        info("=" * 80)
+        info("MESSAGE HEADERS:")
+        info(pprint.pformat(dict(headers or {})))
+        info("MESSAGE PROPERTIES:")
+        info(pprint.pformat(getattr(message, 'properties', {})))
+        info("=" * 80)
+
         req = Req(
             message,
             on_ack=ack, on_reject=reject, app=app, hostname=hostname,

Despite the fact that this issue doesn't occur with non-EoL versions of RabbitMQ, it still affects us. We have a shared RabbitMQ server used by several dozen microservices. Historically we've relied on classic queue mirroring to provide high availability message brokering. We're moving each critical use-case from mirrored classic queues to quorum queues, but the scope is too large to be done in a day. We're unable to upgrade to RabbitMQ ≥4.0.1 until all services have been migrated. This means we're stuck with the behavior of RabbitMQ 3.13 for the coming months. Given that quorum queues support for Celery is relatively new, I doubt we are the only ones in this situation.

The integration test was importent to ensure we could reproduce the issue and resolve it when making a fix. However, I'm not sure how useful it is to keep the integration test as the issue can not be reproduced with the RabbitMQ being used in the CI. I propose either:

  • Keep the test as it is despite requiring to run manually integration tests against an older version of RabbitMQ in order to test the failure case.
  • Remove the integration test and rely solely on the unit test to ensure the correct header handling.
  • Add a second RabbitMQ 3.13 version on a different port to be able to test this case correctly.
  • Run the all the integration tests with a matrix of RabbitMQ versions.

Let me know which of these options you prefer.

@Izzette

Izzette commented Feb 6, 2026

Copy link
Copy Markdown
Contributor Author
Integration test error output before applying the fix when running with RabbitMQ 3.13
============================================================================================================================= test session starts ==============================================================================================================================
platform darwin -- Python 3.13.5, pytest-8.4.2, pluggy-1.6.0 -- /Users/izzi/src/celery/.tox/3.13-integration-rabbitmq/bin/python
cachedir: .tox/3.13-integration-rabbitmq/.pytest_cache
rootdir: /Users/izzi/src/celery
configfile: pyproject.toml
plugins: anyio-4.12.1, click-1.1.0, timeout-2.4.0, pytest_docker_tools-3.1.9, order-1.3.0, github-actions-annotate-failures-0.3.0, celery-1.2.1, subtests-0.15.0, rerunfailures-16.1, cov-7.0.0
collected 2 items

[2026-02-06 10:38:09,973: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2026-02-06 10:38:09,976: INFO/MainProcess] Global QoS is disabled. Prefetch count in now static.
t/integration/test_rabbitmq_quorum_queue_cycle_detection.py::test_countdown_task_with_retry_loses_message [2026-02-06 10:38:10,394: WARNING/MainProcess] No hostname was supplied. Reverting to default 'localhost'
[2026-02-06 10:38:10,415: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:10,415: INFO/MainProcess] PUBLISHING TASK: celery.ping
[2026-02-06 10:38:10,415: INFO/MainProcess]   Task ID: 191a129e-30fa-4da3-b0ff-b7b240d45813
[2026-02-06 10:38:10,415: INFO/MainProcess]   Exchange: celery.topic
[2026-02-06 10:38:10,415: INFO/MainProcess]   Routing Key: test_cycle_queue
[2026-02-06 10:38:10,415: INFO/MainProcess]   ETA: None
[2026-02-06 10:38:10,415: INFO/MainProcess]   Retries: 0
[2026-02-06 10:38:10,415: INFO/MainProcess]   NO X-Death headers being sent
[2026-02-06 10:38:10,415: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:10,420: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:10,420: INFO/MainProcess] MESSAGE HEADERS:
[2026-02-06 10:38:10,420: INFO/MainProcess] {'argsrepr': '()',
 'eta': None,
 'expires': None,
 'group': None,
 'group_index': None,
 'id': '191a129e-30fa-4da3-b0ff-b7b240d45813',
 'ignore_result': False,
 'kwargsrepr': '{}',
 'lang': 'py',
 'origin': 'gen89067@BM-003963',
 'parent_id': None,
 'replaced_task_nesting': 0,
 'retries': 0,
 'root_id': '191a129e-30fa-4da3-b0ff-b7b240d45813',
 'shadow': None,
 'stamped_headers': None,
 'stamps': {},
 'task': 'celery.ping',
 'timelimit': [None, None]}
[2026-02-06 10:38:10,420: INFO/MainProcess] MESSAGE PROPERTIES:
[2026-02-06 10:38:10,420: INFO/MainProcess] {'application_headers': {'argsrepr': '()',
                         'eta': None,
                         'expires': None,
                         'group': None,
                         'group_index': None,
                         'id': '191a129e-30fa-4da3-b0ff-b7b240d45813',
                         'ignore_result': False,
                         'kwargsrepr': '{}',
                         'lang': 'py',
                         'origin': 'gen89067@BM-003963',
                         'parent_id': None,
                         'replaced_task_nesting': 0,
                         'retries': 0,
                         'root_id': '191a129e-30fa-4da3-b0ff-b7b240d45813',
                         'shadow': None,
                         'stamped_headers': None,
                         'stamps': {},
                         'task': 'celery.ping',
                         'timelimit': [None, None]},
 'content_encoding': 'utf-8',
 'content_type': 'application/json',
 'correlation_id': '191a129e-30fa-4da3-b0ff-b7b240d45813',
 'delivery_mode': 2,
 'priority': 0,
 'reply_to': 'a0aa6a43-bb2e-3fd7-a29f-3fdf1c288626'}
[2026-02-06 10:38:10,420: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:10,420: INFO/MainProcess] Task celery.ping[191a129e-30fa-4da3-b0ff-b7b240d45813] received
[2026-02-06 10:38:10,428: INFO/MainProcess] Task celery.ping[191a129e-30fa-4da3-b0ff-b7b240d45813] succeeded in 0.007614125031977892s: 'pong'
[2026-02-06 10:38:10,429: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:10,429: INFO/MainProcess] PUBLISHING TASK: t.integration.test_rabbitmq_quorum_queue_cycle_detection.countdown_task_with_retry
[2026-02-06 10:38:10,429: INFO/MainProcess]   Task ID: 2dd261d0-02ce-4e7d-891e-5a7d103ee7ff
[2026-02-06 10:38:10,429: INFO/MainProcess]   Exchange: Exchange celery_delayed_27(topic)
[2026-02-06 10:38:10,429: INFO/MainProcess]   Routing Key: 0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.1.test_cycle_queue
[2026-02-06 10:38:10,429: INFO/MainProcess]   ETA: 2026-02-06T09:38:13.429020+00:00
[2026-02-06 10:38:10,429: INFO/MainProcess]   Retries: 0
[2026-02-06 10:38:10,429: INFO/MainProcess]   NO X-Death headers being sent
[2026-02-06 10:38:10,429: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:13,461: WARNING/MainProcess] /Users/izzi/src/celery/.tox/3.13-integration-rabbitmq/lib/python3.13/site-packages/amqp/serialization.py:135: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
  val = datetime.utcfromtimestamp(val)

[2026-02-06 10:38:13,461: WARNING/MainProcess] /Users/izzi/src/celery/.tox/3.13-integration-rabbitmq/lib/python3.13/site-packages/amqp/serialization.py:135: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
  val = datetime.utcfromtimestamp(val)

[2026-02-06 10:38:13,461: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:13,461: INFO/MainProcess] MESSAGE HEADERS:
[2026-02-06 10:38:13,462: INFO/MainProcess] {'argsrepr': '()',
 'eta': '2026-02-06T09:38:13.429020+00:00',
 'expires': None,
 'group': None,
 'group_index': None,
 'id': '2dd261d0-02ce-4e7d-891e-5a7d103ee7ff',
 'ignore_result': False,
 'kwargsrepr': '{}',
 'lang': 'py',
 'origin': 'gen89067@BM-003963',
 'parent_id': None,
 'replaced_task_nesting': 0,
 'retries': 0,
 'root_id': '2dd261d0-02ce-4e7d-891e-5a7d103ee7ff',
 'shadow': None,
 'stamped_headers': None,
 'stamps': {},
 'task': 't.integration.test_rabbitmq_quorum_queue_cycle_detection.countdown_task_with_retry',
 'timelimit': [None, None],
 'x-death': [{'count': 1,
              'exchange': 'celery_delayed_0',
              'queue': 'celery_delayed_0',
              'reason': 'expired',
              'routing-keys': ['0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.1.test_cycle_queue'],
              'time': datetime.datetime(2026, 2, 6, 9, 38, 12)},
             {'count': 1,
              'exchange': 'celery_delayed_27',
              'queue': 'celery_delayed_1',
              'reason': 'expired',
              'routing-keys': ['0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.1.test_cycle_queue'],
              'time': datetime.datetime(2026, 2, 6, 9, 38, 11)}],
 'x-first-death-exchange': 'celery_delayed_27',
 'x-first-death-queue': 'celery_delayed_1',
 'x-first-death-reason': 'expired',
 'x-last-death-exchange': 'celery_delayed_0',
 'x-last-death-queue': 'celery_delayed_0',
 'x-last-death-reason': 'expired'}
[2026-02-06 10:38:13,462: INFO/MainProcess] MESSAGE PROPERTIES:
[2026-02-06 10:38:13,463: INFO/MainProcess] {'application_headers': {'argsrepr': '()',
                         'eta': '2026-02-06T09:38:13.429020+00:00',
                         'expires': None,
                         'group': None,
                         'group_index': None,
                         'id': '2dd261d0-02ce-4e7d-891e-5a7d103ee7ff',
                         'ignore_result': False,
                         'kwargsrepr': '{}',
                         'lang': 'py',
                         'origin': 'gen89067@BM-003963',
                         'parent_id': None,
                         'replaced_task_nesting': 0,
                         'retries': 0,
                         'root_id': '2dd261d0-02ce-4e7d-891e-5a7d103ee7ff',
                         'shadow': None,
                         'stamped_headers': None,
                         'stamps': {},
                         'task': 't.integration.test_rabbitmq_quorum_queue_cycle_detection.countdown_task_with_retry',
                         'timelimit': [None, None],
                         'x-death': [{'count': 1,
                                      'exchange': 'celery_delayed_0',
                                      'queue': 'celery_delayed_0',
                                      'reason': 'expired',
                                      'routing-keys': ['0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.1.test_cycle_queue'],
                                      'time': datetime.datetime(2026, 2, 6, 9, 38, 12)},
                                     {'count': 1,
                                      'exchange': 'celery_delayed_27',
                                      'queue': 'celery_delayed_1',
                                      'reason': 'expired',
                                      'routing-keys': ['0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.1.test_cycle_queue'],
                                      'time': datetime.datetime(2026, 2, 6, 9, 38, 11)}],
                         'x-first-death-exchange': 'celery_delayed_27',
                         'x-first-death-queue': 'celery_delayed_1',
                         'x-first-death-reason': 'expired',
                         'x-last-death-exchange': 'celery_delayed_0',
                         'x-last-death-queue': 'celery_delayed_0',
                         'x-last-death-reason': 'expired'},
 'content_encoding': 'utf-8',
 'content_type': 'application/json',
 'correlation_id': '2dd261d0-02ce-4e7d-891e-5a7d103ee7ff',
 'delivery_mode': 2,
 'priority': 0,
 'reply_to': 'a0aa6a43-bb2e-3fd7-a29f-3fdf1c288626'}
[2026-02-06 10:38:13,464: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:13,464: INFO/MainProcess] Task t.integration.test_rabbitmq_quorum_queue_cycle_detection.countdown_task_with_retry[2dd261d0-02ce-4e7d-891e-5a7d103ee7ff] received
[2026-02-06 10:38:13,475: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:13,475: INFO/MainProcess] PUBLISHING TASK: t.integration.test_rabbitmq_quorum_queue_cycle_detection.countdown_task_with_retry
[2026-02-06 10:38:13,475: INFO/MainProcess]   Task ID: 2dd261d0-02ce-4e7d-891e-5a7d103ee7ff
[2026-02-06 10:38:13,475: INFO/MainProcess]   Exchange: Exchange celery_delayed_27(topic)
[2026-02-06 10:38:13,475: INFO/MainProcess]   Routing Key: 0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.1.test_cycle_queue
[2026-02-06 10:38:13,475: INFO/MainProcess]   ETA: 2026-02-06T09:38:16.468096+00:00
[2026-02-06 10:38:13,475: INFO/MainProcess]   Retries: 1
[2026-02-06 10:38:13,475: INFO/MainProcess]   X-Death being sent in headers:
[2026-02-06 10:38:13,476: INFO/MainProcess] [{'count': 1,
  'exchange': 'celery_delayed_0',
  'queue': 'celery_delayed_0',
  'reason': 'expired',
  'routing-keys': ['0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.1.test_cycle_queue'],
  'time': datetime.datetime(2026, 2, 6, 9, 38, 12)},
 {'count': 1,
  'exchange': 'celery_delayed_27',
  'queue': 'celery_delayed_1',
  'reason': 'expired',
  'routing-keys': ['0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.1.test_cycle_queue'],
  'time': datetime.datetime(2026, 2, 6, 9, 38, 11)}]
[2026-02-06 10:38:13,476: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:13,491: INFO/MainProcess] Task t.integration.test_rabbitmq_quorum_queue_cycle_detection.countdown_task_with_retry[2dd261d0-02ce-4e7d-891e-5a7d103ee7ff] retry: Retry in 3s: TaskFailedException('Simulated failure')
FAILED
[2026-02-06 10:38:23,615: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2026-02-06 10:38:23,617: INFO/MainProcess] Global QoS is disabled. Prefetch count in now static.
[2026-02-06 10:38:23,711: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:23,711: INFO/MainProcess] PUBLISHING TASK: celery.ping
[2026-02-06 10:38:23,711: INFO/MainProcess]   Task ID: b4aef817-436a-4718-8137-0f76fe563189
[2026-02-06 10:38:23,711: INFO/MainProcess]   Exchange: celery.topic
[2026-02-06 10:38:23,711: INFO/MainProcess]   Routing Key: test_cycle_queue
[2026-02-06 10:38:23,711: INFO/MainProcess]   ETA: None
[2026-02-06 10:38:23,711: INFO/MainProcess]   Retries: 0
[2026-02-06 10:38:23,711: INFO/MainProcess]   NO X-Death headers being sent
[2026-02-06 10:38:23,711: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:23,713: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:23,713: INFO/MainProcess] MESSAGE HEADERS:
[2026-02-06 10:38:23,713: INFO/MainProcess] {'argsrepr': '()',
 'eta': None,
 'expires': None,
 'group': None,
 'group_index': None,
 'id': 'b4aef817-436a-4718-8137-0f76fe563189',
 'ignore_result': False,
 'kwargsrepr': '{}',
 'lang': 'py',
 'origin': 'gen89067@BM-003963',
 'parent_id': None,
 'replaced_task_nesting': 0,
 'retries': 0,
 'root_id': 'b4aef817-436a-4718-8137-0f76fe563189',
 'shadow': None,
 'stamped_headers': None,
 'stamps': {},
 'task': 'celery.ping',
 'timelimit': [None, None]}
[2026-02-06 10:38:23,713: INFO/MainProcess] MESSAGE PROPERTIES:
[2026-02-06 10:38:23,713: INFO/MainProcess] {'application_headers': {'argsrepr': '()',
                         'eta': None,
                         'expires': None,
                         'group': None,
                         'group_index': None,
                         'id': 'b4aef817-436a-4718-8137-0f76fe563189',
                         'ignore_result': False,
                         'kwargsrepr': '{}',
                         'lang': 'py',
                         'origin': 'gen89067@BM-003963',
                         'parent_id': None,
                         'replaced_task_nesting': 0,
                         'retries': 0,
                         'root_id': 'b4aef817-436a-4718-8137-0f76fe563189',
                         'shadow': None,
                         'stamped_headers': None,
                         'stamps': {},
                         'task': 'celery.ping',
                         'timelimit': [None, None]},
 'content_encoding': 'utf-8',
 'content_type': 'application/json',
 'correlation_id': 'b4aef817-436a-4718-8137-0f76fe563189',
 'delivery_mode': 2,
 'priority': 0,
 'reply_to': '9a4ef4d7-7a5e-3162-8e29-220765d99fa6'}
[2026-02-06 10:38:23,714: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:23,714: INFO/MainProcess] Task celery.ping[b4aef817-436a-4718-8137-0f76fe563189] received
[2026-02-06 10:38:23,721: INFO/MainProcess] Task celery.ping[b4aef817-436a-4718-8137-0f76fe563189] succeeded in 0.0076374999480322s: 'pong'
[2026-02-06 10:38:23,722: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:23,722: INFO/MainProcess] PUBLISHING TASK: t.integration.test_rabbitmq_quorum_queue_cycle_detection.non_eta_task_with_retries
[2026-02-06 10:38:23,722: INFO/MainProcess]   Task ID: 8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb
[2026-02-06 10:38:23,722: INFO/MainProcess]   Exchange: celery.topic
[2026-02-06 10:38:23,722: INFO/MainProcess]   Routing Key: test_cycle_queue
[2026-02-06 10:38:23,722: INFO/MainProcess]   ETA: None
[2026-02-06 10:38:23,722: INFO/MainProcess]   Retries: 0
[2026-02-06 10:38:23,722: INFO/MainProcess]   NO X-Death headers being sent
[2026-02-06 10:38:23,722: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:23,725: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:23,725: INFO/MainProcess] MESSAGE HEADERS:
[2026-02-06 10:38:23,725: INFO/MainProcess] {'argsrepr': '()',
 'eta': None,
 'expires': None,
 'group': None,
 'group_index': None,
 'id': '8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb',
 'ignore_result': False,
 'kwargsrepr': '{}',
 'lang': 'py',
 'origin': 'gen89067@BM-003963',
 'parent_id': None,
 'replaced_task_nesting': 0,
 'retries': 0,
 'root_id': '8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb',
 'shadow': None,
 'stamped_headers': None,
 'stamps': {},
 'task': 't.integration.test_rabbitmq_quorum_queue_cycle_detection.non_eta_task_with_retries',
 'timelimit': [None, None]}
[2026-02-06 10:38:23,725: INFO/MainProcess] MESSAGE PROPERTIES:
[2026-02-06 10:38:23,725: INFO/MainProcess] {'application_headers': {'argsrepr': '()',
                         'eta': None,
                         'expires': None,
                         'group': None,
                         'group_index': None,
                         'id': '8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb',
                         'ignore_result': False,
                         'kwargsrepr': '{}',
                         'lang': 'py',
                         'origin': 'gen89067@BM-003963',
                         'parent_id': None,
                         'replaced_task_nesting': 0,
                         'retries': 0,
                         'root_id': '8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb',
                         'shadow': None,
                         'stamped_headers': None,
                         'stamps': {},
                         'task': 't.integration.test_rabbitmq_quorum_queue_cycle_detection.non_eta_task_with_retries',
                         'timelimit': [None, None]},
 'content_encoding': 'utf-8',
 'content_type': 'application/json',
 'correlation_id': '8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb',
 'delivery_mode': 2,
 'priority': 0,
 'reply_to': '9a4ef4d7-7a5e-3162-8e29-220765d99fa6'}
[2026-02-06 10:38:23,725: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:23,726: INFO/MainProcess] Task t.integration.test_rabbitmq_quorum_queue_cycle_detection.non_eta_task_with_retries[8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb] received
[2026-02-06 10:38:23,729: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:23,729: INFO/MainProcess] PUBLISHING TASK: t.integration.test_rabbitmq_quorum_queue_cycle_detection.non_eta_task_with_retries
[2026-02-06 10:38:23,729: INFO/MainProcess]   Task ID: 8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb
[2026-02-06 10:38:23,729: INFO/MainProcess]   Exchange: Exchange celery_delayed_27(topic)
[2026-02-06 10:38:23,729: INFO/MainProcess]   Routing Key: 0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.1.test_cycle_queue
[2026-02-06 10:38:23,729: INFO/MainProcess]   ETA: 2026-02-06T09:38:26.726629+00:00
[2026-02-06 10:38:23,729: INFO/MainProcess]   Retries: 1
[2026-02-06 10:38:23,729: INFO/MainProcess]   NO X-Death headers being sent
[2026-02-06 10:38:23,729: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:23,735: INFO/MainProcess] Task t.integration.test_rabbitmq_quorum_queue_cycle_detection.non_eta_task_with_retries[8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb] retry: Retry in 3s: TaskFailedException('Simulated failure')
t/integration/test_rabbitmq_quorum_queue_cycle_detection.py::test_non_eta_task_with_multiple_retries_loses_message [2026-02-06 10:38:26,752: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:26,752: INFO/MainProcess] MESSAGE HEADERS:
[2026-02-06 10:38:26,753: INFO/MainProcess] {'argsrepr': '()',
 'eta': '2026-02-06T09:38:26.726629+00:00',
 'expires': None,
 'group': None,
 'group_index': None,
 'id': '8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb',
 'ignore_result': False,
 'kwargsrepr': '{}',
 'lang': 'py',
 'origin': 'gen89067@BM-003963',
 'parent_id': '8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb',
 'replaced_task_nesting': 0,
 'retries': 1,
 'root_id': '8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb',
 'shadow': None,
 'stamped_headers': None,
 'stamps': {},
 'task': 't.integration.test_rabbitmq_quorum_queue_cycle_detection.non_eta_task_with_retries',
 'timelimit': [None, None],
 'x-death': [{'count': 1,
              'exchange': 'celery_delayed_0',
              'queue': 'celery_delayed_0',
              'reason': 'expired',
              'routing-keys': ['0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.1.test_cycle_queue'],
              'time': datetime.datetime(2026, 2, 6, 9, 38, 26)},
             {'count': 1,
              'exchange': 'celery_delayed_27',
              'queue': 'celery_delayed_1',
              'reason': 'expired',
              'routing-keys': ['0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.1.test_cycle_queue'],
              'time': datetime.datetime(2026, 2, 6, 9, 38, 25)}],
 'x-first-death-exchange': 'celery_delayed_27',
 'x-first-death-queue': 'celery_delayed_1',
 'x-first-death-reason': 'expired',
 'x-last-death-exchange': 'celery_delayed_0',
 'x-last-death-queue': 'celery_delayed_0',
 'x-last-death-reason': 'expired'}
[2026-02-06 10:38:26,753: INFO/MainProcess] MESSAGE PROPERTIES:
[2026-02-06 10:38:26,754: INFO/MainProcess] {'application_headers': {'argsrepr': '()',
                         'eta': '2026-02-06T09:38:26.726629+00:00',
                         'expires': None,
                         'group': None,
                         'group_index': None,
                         'id': '8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb',
                         'ignore_result': False,
                         'kwargsrepr': '{}',
                         'lang': 'py',
                         'origin': 'gen89067@BM-003963',
                         'parent_id': '8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb',
                         'replaced_task_nesting': 0,
                         'retries': 1,
                         'root_id': '8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb',
                         'shadow': None,
                         'stamped_headers': None,
                         'stamps': {},
                         'task': 't.integration.test_rabbitmq_quorum_queue_cycle_detection.non_eta_task_with_retries',
                         'timelimit': [None, None],
                         'x-death': [{'count': 1,
                                      'exchange': 'celery_delayed_0',
                                      'queue': 'celery_delayed_0',
                                      'reason': 'expired',
                                      'routing-keys': ['0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.1.test_cycle_queue'],
                                      'time': datetime.datetime(2026, 2, 6, 9, 38, 26)},
                                     {'count': 1,
                                      'exchange': 'celery_delayed_27',
                                      'queue': 'celery_delayed_1',
                                      'reason': 'expired',
                                      'routing-keys': ['0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.1.test_cycle_queue'],
                                      'time': datetime.datetime(2026, 2, 6, 9, 38, 25)}],
                         'x-first-death-exchange': 'celery_delayed_27',
                         'x-first-death-queue': 'celery_delayed_1',
                         'x-first-death-reason': 'expired',
                         'x-last-death-exchange': 'celery_delayed_0',
                         'x-last-death-queue': 'celery_delayed_0',
                         'x-last-death-reason': 'expired'},
 'content_encoding': 'utf-8',
 'content_type': 'application/json',
 'correlation_id': '8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb',
 'delivery_mode': 2,
 'priority': 0,
 'reply_to': '9a4ef4d7-7a5e-3162-8e29-220765d99fa6'}
[2026-02-06 10:38:26,754: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:26,755: INFO/MainProcess] Task t.integration.test_rabbitmq_quorum_queue_cycle_detection.non_eta_task_with_retries[8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb] received
[2026-02-06 10:38:26,760: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:26,760: INFO/MainProcess] PUBLISHING TASK: t.integration.test_rabbitmq_quorum_queue_cycle_detection.non_eta_task_with_retries
[2026-02-06 10:38:26,760: INFO/MainProcess]   Task ID: 8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb
[2026-02-06 10:38:26,760: INFO/MainProcess]   Exchange: Exchange celery_delayed_27(topic)
[2026-02-06 10:38:26,760: INFO/MainProcess]   Routing Key: 0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.1.test_cycle_queue
[2026-02-06 10:38:26,760: INFO/MainProcess]   ETA: 2026-02-06T09:38:29.758202+00:00
[2026-02-06 10:38:26,760: INFO/MainProcess]   Retries: 2
[2026-02-06 10:38:26,760: INFO/MainProcess]   X-Death being sent in headers:
[2026-02-06 10:38:26,761: INFO/MainProcess] [{'count': 1,
  'exchange': 'celery_delayed_0',
  'queue': 'celery_delayed_0',
  'reason': 'expired',
  'routing-keys': ['0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.1.test_cycle_queue'],
  'time': datetime.datetime(2026, 2, 6, 9, 38, 26)},
 {'count': 1,
  'exchange': 'celery_delayed_27',
  'queue': 'celery_delayed_1',
  'reason': 'expired',
  'routing-keys': ['0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.1.test_cycle_queue'],
  'time': datetime.datetime(2026, 2, 6, 9, 38, 25)}]
[2026-02-06 10:38:26,761: INFO/MainProcess] ================================================================================
[2026-02-06 10:38:26,774: INFO/MainProcess] Task t.integration.test_rabbitmq_quorum_queue_cycle_detection.non_eta_task_with_retries[8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb] retry: Retry in 3s: TaskFailedException('Simulated failure')
FAILED

=================================================================================================================================== FAILURES ===================================================================================================================================
_________________________________________________________________________________________________________________ test_countdown_task_with_retry_loses_message _________________________________________________________________________________________________________________

self = <celery.backends.rpc.ResultConsumer object at 0x1101706e0>, result = <AsyncResult: 2dd261d0-02ce-4e7d-891e-5a7d103ee7ff>, timeout = 8, on_interval = <promise@0x1101642d0>, on_message = None, kwargs = {'interval': 0.5, 'no_ack': True}, prev_on_m = None, _ = None

    def _wait_for_pending(self, result,
                          timeout=None, on_interval=None, on_message=None,
                          **kwargs):
        self.on_wait_for_pending(result, timeout=timeout, **kwargs)
        prev_on_m, self.on_message = self.on_message, on_message
        try:
>           for _ in self.drain_events_until(
                    result.on_ready, timeout=timeout,
                    on_interval=on_interval):

celery/backends/asynchronous.py:332:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <celery.backends.asynchronous.Drainer object at 0x110170590>, p = <promise@0x1101644b0 --> <weakref at 0x107df06d0; to 'celery.result.AsyncResult' at 0x1101a0050>>, timeout = 8, interval = 1, on_interval = <promise@0x1101642d0>
wait = <bound method ResultConsumer.drain_events of <celery.backends.rpc.ResultConsumer object at 0x1101706e0>>

    def drain_events_until(self, p, timeout=None, interval=1, on_interval=None, wait=None):
        wait = wait or self.result_consumer.drain_events
        time_start = time.monotonic()

        while 1:
            # Total time spent may exceed a single call to wait()
            if timeout and time.monotonic() - time_start >= timeout:
>               raise socket.timeout()
E               TimeoutError

celery/backends/asynchronous.py:77: TimeoutError

During handling of the above exception, another exception occurred:

    @pytest.mark.amqp
    @pytest.mark.timeout(35)
    def test_countdown_task_with_retry_loses_message():
        """Test that a countdown task with retry gets silently dropped by RabbitMQ.

        This test reproduces the first scenario from issue #9867:
        - Task is published with countdown=3
        - Task has default_retry_delay=3 (same delay to trigger cycle detection)
        - Task fails and attempts to retry
        - RabbitMQ detects a cycle via X-Death header and drops the message
        - The task is silently lost and never completes

        Expected: This test should FAIL until the bug is fixed.
        The task should complete after retrying, but instead it will timeout
        because the message is dropped by RabbitMQ.
        """
        app, queue_name = create_test_app_with_quorum_queues()

        @app.task(bind=True, default_retry_delay=3, max_retries=1)
        def countdown_task_with_retry(self):
            """Task that fails once then succeeds."""
            # First attempt: fail to trigger retry
            if self.request.retries < 1:
                raise self.retry(exc=TaskFailedException("Simulated failure"))

            return self.request.retries

        with start_worker(
            app,
            queues=[queue_name],
            loglevel="INFO",
            shutdown_timeout=15,
        ):
            # Execute task with countdown=3 (same as default_retry_delay)
            # We need to hit the same delayed buckets to trigger the cycle detection
            # in RabbitMQ.
            result = countdown_task_with_retry.apply_async(countdown=3)

            try:
                # Wait for the task to complete, we must wait at least long enough
                # for the countdown to be reached and the retry to occur (6s).
>               value = result.get(timeout=8)  # TODO: increase
                        ^^^^^^^^^^^^^^^^^^^^^

t/integration/test_rabbitmq_quorum_queue_cycle_detection.py:128:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
celery/result.py:253: in get
    return self.backend.wait_for_pending(
celery/backends/asynchronous.py:266: in wait_for_pending
    for _ in self._wait_for_pending(result, **kwargs):
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <celery.backends.rpc.ResultConsumer object at 0x1101706e0>, result = <AsyncResult: 2dd261d0-02ce-4e7d-891e-5a7d103ee7ff>, timeout = 8, on_interval = <promise@0x1101642d0>, on_message = None, kwargs = {'interval': 0.5, 'no_ack': True}, prev_on_m = None, _ = None

    def _wait_for_pending(self, result,
                          timeout=None, on_interval=None, on_message=None,
                          **kwargs):
        self.on_wait_for_pending(result, timeout=timeout, **kwargs)
        prev_on_m, self.on_message = self.on_message, on_message
        try:
            for _ in self.drain_events_until(
                    result.on_ready, timeout=timeout,
                    on_interval=on_interval):
                yield
                sleep(0)
        except socket.timeout:
>           raise TimeoutError('The operation timed out.')
E           celery.exceptions.TimeoutError: The operation timed out.

celery/backends/asynchronous.py:338: TimeoutError

During handling of the above exception, another exception occurred:

    @pytest.mark.amqp
    @pytest.mark.timeout(35)
    def test_countdown_task_with_retry_loses_message():
        """Test that a countdown task with retry gets silently dropped by RabbitMQ.

        This test reproduces the first scenario from issue #9867:
        - Task is published with countdown=3
        - Task has default_retry_delay=3 (same delay to trigger cycle detection)
        - Task fails and attempts to retry
        - RabbitMQ detects a cycle via X-Death header and drops the message
        - The task is silently lost and never completes

        Expected: This test should FAIL until the bug is fixed.
        The task should complete after retrying, but instead it will timeout
        because the message is dropped by RabbitMQ.
        """
        app, queue_name = create_test_app_with_quorum_queues()

        @app.task(bind=True, default_retry_delay=3, max_retries=1)
        def countdown_task_with_retry(self):
            """Task that fails once then succeeds."""
            # First attempt: fail to trigger retry
            if self.request.retries < 1:
                raise self.retry(exc=TaskFailedException("Simulated failure"))

            return self.request.retries

        with start_worker(
            app,
            queues=[queue_name],
            loglevel="INFO",
            shutdown_timeout=15,
        ):
            # Execute task with countdown=3 (same as default_retry_delay)
            # We need to hit the same delayed buckets to trigger the cycle detection
            # in RabbitMQ.
            result = countdown_task_with_retry.apply_async(countdown=3)

            try:
                # Wait for the task to complete, we must wait at least long enough
                # for the countdown to be reached and the retry to occur (6s).
                value = result.get(timeout=8)  # TODO: increase
            except exceptions.TimeoutError as e:
>               pytest.fail(
                    f"Task was silently dropped by RabbitMQ due to cycle detection. "
                    f"Exception: {e!r}."
                )
E               Failed: Task was silently dropped by RabbitMQ due to cycle detection. Exception: TimeoutError('The operation timed out.').

t/integration/test_rabbitmq_quorum_queue_cycle_detection.py:130: Failed
____________________________________________________________________________________________________________ test_non_eta_task_with_multiple_retries_loses_message _____________________________________________________________________________________________________________

self = <celery.backends.rpc.ResultConsumer object at 0x110aa1090>, result = <AsyncResult: 8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb>, timeout = 8, on_interval = <promise@0x110cfe170>, on_message = None, kwargs = {'interval': 0.5, 'no_ack': True}, prev_on_m = None, _ = None

    def _wait_for_pending(self, result,
                          timeout=None, on_interval=None, on_message=None,
                          **kwargs):
        self.on_wait_for_pending(result, timeout=timeout, **kwargs)
        prev_on_m, self.on_message = self.on_message, on_message
        try:
>           for _ in self.drain_events_until(
                    result.on_ready, timeout=timeout,
                    on_interval=on_interval):

celery/backends/asynchronous.py:332:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <celery.backends.asynchronous.Drainer object at 0x110aa11d0>, p = <promise@0x110cfceb0 --> <weakref at 0x110ae30d0; to 'celery.result.AsyncResult' at 0x110aa1310>>, timeout = 8, interval = 1, on_interval = <promise@0x110cfe170>
wait = <bound method ResultConsumer.drain_events of <celery.backends.rpc.ResultConsumer object at 0x110aa1090>>

    def drain_events_until(self, p, timeout=None, interval=1, on_interval=None, wait=None):
        wait = wait or self.result_consumer.drain_events
        time_start = time.monotonic()

        while 1:
            # Total time spent may exceed a single call to wait()
            if timeout and time.monotonic() - time_start >= timeout:
>               raise socket.timeout()
E               TimeoutError

celery/backends/asynchronous.py:77: TimeoutError

During handling of the above exception, another exception occurred:

    @pytest.mark.amqp
    @pytest.mark.timeout(35)
    def test_non_eta_task_with_multiple_retries_loses_message():
        """Test that a non-ETA task with multiple retries gets silently dropped.

        This test reproduces the second scenario from issue #9867:
        - Task is published without ETA/countdown (immediate execution)
        - Task fails and is retried with countdown=3
        - Task fails again and attempts second retry with countdown=3
        - RabbitMQ detects a cycle via X-Death header and drops the message
        - The task is silently lost after the first retry

        Expected: This test should FAIL until the bug is fixed.
        The task should complete after retrying twice, but instead it will timeout
        because the message is dropped by RabbitMQ after the first retry.
        """
        app, queue_name = create_test_app_with_quorum_queues()

        @app.task(bind=True, max_retries=2)
        def non_eta_task_with_retries(self):
            """Task that fails twice then succeeds."""
            # First and second attempts: fail to trigger retry
            if self.request.retries < 2:
                raise self.retry(exc=TaskFailedException("Simulated failure"), countdown=3)

            return self.request.retries

        with start_worker(
            app,
            queues=[queue_name],
            loglevel="INFO",
            shutdown_timeout=15,
        ):
            result = non_eta_task_with_retries.apply_async()

            try:
                # Wait for the task to complete, we must wait at least long enough
                # for the two retries to occur (6s).
>               value = result.get(timeout=8)  # TODO: increase
                        ^^^^^^^^^^^^^^^^^^^^^

t/integration/test_rabbitmq_quorum_queue_cycle_detection.py:177:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
celery/result.py:253: in get
    return self.backend.wait_for_pending(
celery/backends/asynchronous.py:266: in wait_for_pending
    for _ in self._wait_for_pending(result, **kwargs):
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <celery.backends.rpc.ResultConsumer object at 0x110aa1090>, result = <AsyncResult: 8a7e10b3-dfa4-466e-bb0c-eebaa8da15fb>, timeout = 8, on_interval = <promise@0x110cfe170>, on_message = None, kwargs = {'interval': 0.5, 'no_ack': True}, prev_on_m = None, _ = None

    def _wait_for_pending(self, result,
                          timeout=None, on_interval=None, on_message=None,
                          **kwargs):
        self.on_wait_for_pending(result, timeout=timeout, **kwargs)
        prev_on_m, self.on_message = self.on_message, on_message
        try:
            for _ in self.drain_events_until(
                    result.on_ready, timeout=timeout,
                    on_interval=on_interval):
                yield
                sleep(0)
        except socket.timeout:
>           raise TimeoutError('The operation timed out.')
E           celery.exceptions.TimeoutError: The operation timed out.

celery/backends/asynchronous.py:338: TimeoutError

During handling of the above exception, another exception occurred:

    @pytest.mark.amqp
    @pytest.mark.timeout(35)
    def test_non_eta_task_with_multiple_retries_loses_message():
        """Test that a non-ETA task with multiple retries gets silently dropped.

        This test reproduces the second scenario from issue #9867:
        - Task is published without ETA/countdown (immediate execution)
        - Task fails and is retried with countdown=3
        - Task fails again and attempts second retry with countdown=3
        - RabbitMQ detects a cycle via X-Death header and drops the message
        - The task is silently lost after the first retry

        Expected: This test should FAIL until the bug is fixed.
        The task should complete after retrying twice, but instead it will timeout
        because the message is dropped by RabbitMQ after the first retry.
        """
        app, queue_name = create_test_app_with_quorum_queues()

        @app.task(bind=True, max_retries=2)
        def non_eta_task_with_retries(self):
            """Task that fails twice then succeeds."""
            # First and second attempts: fail to trigger retry
            if self.request.retries < 2:
                raise self.retry(exc=TaskFailedException("Simulated failure"), countdown=3)

            return self.request.retries

        with start_worker(
            app,
            queues=[queue_name],
            loglevel="INFO",
            shutdown_timeout=15,
        ):
            result = non_eta_task_with_retries.apply_async()

            try:
                # Wait for the task to complete, we must wait at least long enough
                # for the two retries to occur (6s).
                value = result.get(timeout=8)  # TODO: increase
            except exceptions.TimeoutError as e:
>               pytest.fail(
                    f"Task was silently dropped by RabbitMQ due to cycle detection. "
                    f"Exception: {e!r}"
                )
E               Failed: Task was silently dropped by RabbitMQ due to cycle detection. Exception: TimeoutError('The operation timed out.')

t/integration/test_rabbitmq_quorum_queue_cycle_detection.py:179: Failed
=============================================================================================================================== warnings summary ===============================================================================================================================
t/integration/test_rabbitmq_quorum_queue_cycle_detection.py::test_non_eta_task_with_multiple_retries_loses_message
t/integration/test_rabbitmq_quorum_queue_cycle_detection.py::test_non_eta_task_with_multiple_retries_loses_message
  /Users/izzi/src/celery/.tox/3.13-integration-rabbitmq/lib/python3.13/site-packages/amqp/serialization.py:135: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
    val = datetime.utcfromtimestamp(val)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================================================================================================================== short test summary info ============================================================================================================================
FAILED t/integration/test_rabbitmq_quorum_queue_cycle_detection.py::test_countdown_task_with_retry_loses_message - Failed: Task was silently dropped by RabbitMQ due to cycle detection. Exception: TimeoutError('The operation timed out.').
FAILED t/integration/test_rabbitmq_quorum_queue_cycle_detection.py::test_non_eta_task_with_multiple_retries_loses_message - Failed: Task was silently dropped by RabbitMQ due to cycle detection. Exception: TimeoutError('The operation timed out.')
======================================================================================================================== 2 failed, 2 warnings in 26.99s ========================================================================================================================

@Izzette Izzette force-pushed the fix-rabbitmq-3.x-cycle-detection-native-delayed-delivery branch from b409f91 to 1acbaf5 Compare February 6, 2026 11:03
* Avoid RabbitMQ <4.0.1 dead-lettering cycle detection when retrying with native delayed-delivery.
@Izzette Izzette force-pushed the fix-rabbitmq-3.x-cycle-detection-native-delayed-delivery branch from 1acbaf5 to a5dcbc7 Compare February 6, 2026 11:03
@codecov

codecov Bot commented Feb 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.50%. Comparing base (203f8b8) to head (9abf598).
⚠️ Report is 110 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #10095   +/-   ##
=======================================
  Coverage   87.50%   87.50%           
=======================================
  Files         153      153           
  Lines       19374    19380    +6     
  Branches     2228     2229    +1     
=======================================
+ Hits        16953    16959    +6     
  Misses       2125     2125           
  Partials      296      296           
Flag Coverage Δ
unittests 87.49% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@auvipy auvipy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you please check the remaining failing integration tests?

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses RabbitMQ quorum-queue dead-letter cycle detection causing retried tasks using native delayed delivery to be dropped (fix for #9867) by stripping x-death-related headers before republishing retries.

Changes:

  • Filter RabbitMQ x-death/x-*-death-* headers from request execution options prior to republishing (retry path).
  • Add a unit test ensuring signature_from_request() does not propagate x-death headers.
  • Add an integration test module intended to exercise the quorum queue + delayed retry scenario.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

File Description
celery/app/task.py Filters x-death-related headers out of Context.as_execution_options() so retries won’t republish them.
t/unit/tasks/test_tasks.py Adds a unit test asserting x-death header filtering in signature_from_request().
t/integration/test_rabbitmq_quorum_queue_cycle_detection.py Adds integration coverage for the delayed retry + quorum queue cycle-detection scenario.

Comment thread t/integration/test_rabbitmq_quorum_queue_cycle_detection.py Outdated
Comment thread t/integration/test_rabbitmq_quorum_queue_cycle_detection.py Outdated
Comment thread t/integration/test_rabbitmq_quorum_queue_cycle_detection.py
Comment thread t/integration/test_rabbitmq_quorum_queue_cycle_detection.py Outdated
Comment thread t/integration/test_rabbitmq_quorum_queue_cycle_detection.py
Comment thread t/unit/tasks/test_tasks.py Outdated
@Izzette

Izzette commented Feb 9, 2026

Copy link
Copy Markdown
Contributor Author

@auvipy, the tests are failing due to Docker Hub rate limiting:

/usr/bin/docker pull rabbitmq:management
management: Pulling from library/rabbitmq
toomanyrequests: You have reached your unauthenticated pull rate limit. https://www.docker.com/increase-rate-limit
Warning: Docker pull failed with exit code 1, back off 2.488 seconds before retry.
/usr/bin/docker pull rabbitmq:management
Error response from daemon: toomanyrequests: You have reached your unauthenticated pull rate limit. https://www.docker.com/increase-rate-limit
Warning: Docker pull failed with exit code 1, back off 5.142 seconds before retry.
/usr/bin/docker pull rabbitmq:management
Error response from daemon: toomanyrequests: You have reached your unauthenticated pull rate limit. https://www.docker.com/increase-rate-limit
Error: Docker pull failed with exit code 1

@Izzette Izzette requested a review from auvipy February 9, 2026 06:59
@auvipy auvipy added this to the 5.7.0 milestone Feb 9, 2026
@auvipy

auvipy commented Feb 9, 2026

Copy link
Copy Markdown
Member

now surprisingly the unit tests are failing

@Izzette

Izzette commented Feb 9, 2026

Copy link
Copy Markdown
Contributor Author

The unit test failures are now some sort of pip or pypi issue:

Downloading https://pypi.python.org/packages/source/s/setuptools/setuptools-0.9.6.tar.gz
Extracting in /tmp/tmprwjxcr2_
Traceback (most recent call last):
  File "/tmp/pip-install-11ztlzcm/cassandra-driver_0b1b4bb3251947fb809d343b725947f5/ez_setup.py", line 115, in use_setuptools
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

I've never seen this one before, I'm not sure what to think about it. I don't see how it can be related to 6f9786c though.

@auvipy

auvipy commented Feb 9, 2026

Copy link
Copy Markdown
Member

lets wait

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

@auvipy auvipy merged commit 41bad6f into celery:main Feb 11, 2026
326 checks passed
@Izzette

Izzette commented Feb 11, 2026

Copy link
Copy Markdown
Contributor Author

@auvipy thank you <3

@auvipy

auvipy commented Feb 12, 2026

Copy link
Copy Markdown
Member

you are most welcome! feel free to come with any follow up PR if needed!

Izzette added a commit to Izzette/celery that referenced this pull request Feb 12, 2026
* fix: avoid cycle detection in native delayed delivery

* Avoid RabbitMQ <4.0.1 dead-lettering cycle detection when retrying with native delayed-delivery.

* chore: fix typo, rename, unused variable

---------

Co-authored-by: Asif Saif Uddin {"Auvi":"অভি"} <[email protected]>
@liran-cohen-hs

liran-cohen-hs commented Feb 16, 2026

Copy link
Copy Markdown

@auvipy can we add it to 5.6.x milestone for a closer release timeline ?
this is critical for us as we started the migration to quorum and use retries extensively in production
thanks !

@auvipy

auvipy commented Feb 16, 2026

Copy link
Copy Markdown
Member

this might be in v5.7a1

@liran-cohen-hs

Copy link
Copy Markdown

thanks @auvipy
does it mean it will be released only by June~ ? (looking at https://github.com/celery/celery/milestone/46)

@auvipy

auvipy commented Feb 22, 2026

Copy link
Copy Markdown
Member

alpha 1 should be by mid march

auvipy added a commit that referenced this pull request Mar 1, 2026
* fix: avoid cycle detection in native delayed delivery

* Avoid RabbitMQ <4.0.1 dead-lettering cycle detection when retrying with native delayed-delivery.

* chore: fix typo, rename, unused variable

---------

Co-authored-by: Asif Saif Uddin {"Auvi":"অভি"} <[email protected]>
Nusnus pushed a commit that referenced this pull request Mar 26, 2026
* fix: avoid cycle detection in native delayed delivery

* Avoid RabbitMQ <4.0.1 dead-lettering cycle detection when retrying with native delayed-delivery.

* chore: fix typo, rename, unused variable

---------

Co-authored-by: Asif Saif Uddin {"Auvi":"অভি"} <[email protected]>
Nusnus pushed a commit that referenced this pull request Mar 26, 2026
* fix: avoid cycle detection in native delayed delivery

* Avoid RabbitMQ <4.0.1 dead-lettering cycle detection when retrying with native delayed-delivery.

* chore: fix typo, rename, unused variable

---------

Co-authored-by: Asif Saif Uddin {"Auvi":"অভি"} <[email protected]>
Nusnus pushed a commit that referenced this pull request Mar 26, 2026
* fix: avoid cycle detection in native delayed delivery

* Avoid RabbitMQ <4.0.1 dead-lettering cycle detection when retrying with native delayed-delivery.

* chore: fix typo, rename, unused variable

---------

Co-authored-by: Asif Saif Uddin {"Auvi":"অভি"} <[email protected]>
736-c41-2c1-e464fc974 pushed a commit to Swiss-Armed-Forces/Loom that referenced this pull request Mar 29, 2026
This MR contains the following updates:

| Package | Type | Update | Change | OpenSSF |
|---|---|---|---|---|
| [celery](https://docs.celeryq.dev/) ([source](https://github.com/celery/celery), [changelog](https://docs.celeryq.dev/en/stable/changelog.html)) | dependencies | patch | `5.6.2` → `5.6.3` | [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/celery/celery/badge)](https://securityscorecards.dev/viewer/?uri=github.com/celery/celery) |

---

### Release Notes

<details>
<summary>celery/celery (celery)</summary>

### [`v5.6.3`](https://github.com/celery/celery/releases/tag/v5.6.3)

[Compare Source](celery/celery@v5.6.2...v5.6.3)

#### What's Changed

- Fix Django worker recursion bug + defensive checks for pool\_cls.**module** by [@&#8203;maycuatroi1](https://github.com/maycuatroi1) in [#&#8203;10048](celery/celery#10048)
- Docs: Update user\_preload\_options example to use click. by [@&#8203;jorsyk](https://github.com/jorsyk) in [#&#8203;10056](celery/celery#10056)
- Fix invalid configuration key "bootstrap\_servers" in Kafka demo by [@&#8203;jorsyk](https://github.com/jorsyk) in [#&#8203;10060](celery/celery#10060)
- Fix broken images on PyPI page by [@&#8203;Timour-Ilyas](https://github.com/Timour-Ilyas) in [#&#8203;10066](celery/celery#10066)
- Remove broken reference. by [@&#8203;sueannioanis](https://github.com/sueannioanis) in [#&#8203;10071](celery/celery#10071)
- Removed --dist=loadscope from smoke tests by [@&#8203;Nusnus](https://github.com/Nusnus) in [#&#8203;10073](celery/celery#10073)
- Docs: Clarify task\_retry signal args may be None by [@&#8203;GangEunzzang](https://github.com/GangEunzzang) in [#&#8203;10076](celery/celery#10076)
- Update example for Django by [@&#8203;sbc-khacnha](https://github.com/sbc-khacnha) in [#&#8203;10081](celery/celery#10081)
- Make tests compatible with pymongo >= 4.16 by [@&#8203;cjwatson](https://github.com/cjwatson) in [#&#8203;10074](celery/celery#10074)
- fix: source install of cassandra-driver by [@&#8203;Izzette](https://github.com/Izzette) in [#&#8203;10105](celery/celery#10105)
- fix: register task cross-reference role in Sphinx extension by [@&#8203;veeceey](https://github.com/veeceey) in [#&#8203;10100](celery/celery#10100)
- fix: avoid cycle detection in native delayed delivery by [@&#8203;Izzette](https://github.com/Izzette) in [#&#8203;10095](celery/celery#10095)
- fix(asynpool): avoid AttributeError when proc lacks \_sentinel\_poll by [@&#8203;mriddle](https://github.com/mriddle) in [#&#8203;10086](celery/celery#10086)
- fix dusk\_astronomical horizon sign (+18 -> -18) by [@&#8203;Mr-Neutr0n](https://github.com/Mr-Neutr0n) in [#&#8203;10121](celery/celery#10121)
- Fix/10106 onupdate col use lambda func by [@&#8203;ChickenBenny](https://github.com/ChickenBenny) in [#&#8203;10108](celery/celery#10108)
- Fix warm shutdown RuntimeError with eventlet>=0.37.0 ([#&#8203;10083](celery/celery#10083)) by [@&#8203;ChickenBenny](https://github.com/ChickenBenny) in [#&#8203;10123](celery/celery#10123)
- Fix 10109 db backend connection health by [@&#8203;ChickenBenny](https://github.com/ChickenBenny) in [#&#8203;10124](celery/celery#10124)
- Database Backend filter unsupport sql engine arguments with nullpool [#&#8203;7355](celery/celery#7355) by [@&#8203;ChickenBenny](https://github.com/ChickenBenny) in [#&#8203;10134](celery/celery#10134)
- fix(beat): correct argument order in Service.**reduce** by [@&#8203;bysiber](https://github.com/bysiber) in [#&#8203;10137](celery/celery#10137)
- ci: declare explicit read-only token permissions in workflow jobs by [@&#8203;Rohan5commit](https://github.com/Rohan5commit) in [#&#8203;10139](celery/celery#10139)
- chore: 'boto3to' to 'boto3 to' by [@&#8203;cuiweixie](https://github.com/cuiweixie) in [#&#8203;10133](celery/celery#10133)
- Database Backend: Add missing index on date\_done (Fixes [#&#8203;10097](celery/celery#10097)) by [@&#8203;ChickenBenny](https://github.com/ChickenBenny) in [#&#8203;10098](celery/celery#10098)
- docs: fix typo in CONTRIBUTING.rst by [@&#8203;Rohan5commit](https://github.com/Rohan5commit) in [#&#8203;10141](celery/celery#10141)
- Refer to Flower / Prometheus for monitoring by [@&#8203;WilliamDEdwards](https://github.com/WilliamDEdwards) in [#&#8203;10140](celery/celery#10140)
- docs: remove duplicated words in broker and routing docs by [@&#8203;Rohan5commit](https://github.com/Rohan5commit) in [#&#8203;10146](celery/celery#10146)
- docs: fix stale version reference and grammar in README by [@&#8203;kelsonbrito50](https://github.com/kelsonbrito50) in [#&#8203;10145](celery/celery#10145)
- docs: fix wording in Celery 5.3 worker pool notes by [@&#8203;Rohan5commit](https://github.com/Rohan5commit) in [#&#8203;10149](celery/celery#10149)
- docs: fix duplicated wording in 3.1 changelog entry by [@&#8203;Rohan5commit](https://github.com/Rohan5commit) in [#&#8203;10152](celery/celery#10152)
- docs: fix changelog typo in context manager wording by [@&#8203;Rohan5commit](https://github.com/Rohan5commit) in [#&#8203;10144](celery/celery#10144)
- Fix/10096 worker fails to reconnect after redis failover by [@&#8203;ChickenBenny](https://github.com/ChickenBenny) in [#&#8203;10151](celery/celery#10151)
- Improve on\_after\_finalize signal documentation by [@&#8203;Br1an67](https://github.com/Br1an67) in [#&#8203;10155](celery/celery#10155)
- Add non-commutative example to clarify partial arg ordering in canvas docs by [@&#8203;Br1an67](https://github.com/Br1an67) in [#&#8203;10157](celery/celery#10157)
- Remove redundant test\_isa\_mapping test (fixes [#&#8203;10077](celery/celery#10077)) by [@&#8203;daniel7an](https://github.com/daniel7an) in [#&#8203;10103](celery/celery#10103)
- Upgrade pytest-celery to >=1.3.0 and adopt PYTEST\_CELERY\_PKG build arg by [@&#8203;Nusnus](https://github.com/Nusnus) in [#&#8203;10162](celery/celery#10162)
- Remove deprecated args from redis get\_connection call by [@&#8203;JaeHyuckSa](https://github.com/JaeHyuckSa) in [#&#8203;10036](celery/celery#10036)
- Fix [#&#8203;6912](celery/celery#6912) rpc backend reconnection error by [@&#8203;ChickenBenny](https://github.com/ChickenBenny) in [#&#8203;10179](celery/celery#10179)
- Fix NameError with TYPE\_CHECKING annotations on Python 3.14+ (PEP 649) by [@&#8203;drichardson](https://github.com/drichardson) in [#&#8203;10165](celery/celery#10165)
- docs: Add elaboration on prefetch multiplier settings (worker\_prefetch\_multiplier) and worker\_eta\_task\_limit by [@&#8203;tsangwailam](https://github.com/tsangwailam) in [#&#8203;10181](celery/celery#10181)
- Fix O(K²) message bloat in a chain of chords by [@&#8203;Borzik](https://github.com/Borzik) in [#&#8203;10171](celery/celery#10171)
- Fix mock connection interfaces to prevent `TypeError` during exception handling by [@&#8203;ChickenBenny](https://github.com/ChickenBenny) in [#&#8203;10178](celery/celery#10178)
- fix(trace): dispatch chain/callbacks on dedup fast-path for redelivered tasks by [@&#8203;aurangzaib048](https://github.com/aurangzaib048) in [#&#8203;10159](celery/celery#10159)
- Extract `reconnect_on_error` to `BaseResultConsumer` by [@&#8203;ChickenBenny](https://github.com/ChickenBenny) in [#&#8203;10189](celery/celery#10189)
- pep 649 by [@&#8203;ericbuehl](https://github.com/ericbuehl) in [#&#8203;10187](celery/celery#10187)
- [Fix#9722](https://github.com/Fix/celery/issues/9722) friendly status errors for CLI by [@&#8203;ChickenBenny](https://github.com/ChickenBenny) in [#&#8203;10190](celery/celery#10190)
- docs: clarify after\_return behavior for retried tasks by [@&#8203;KianAnbarestani](https://github.com/KianAnbarestani) in [#&#8203;10192](celery/celery#10192)
- Add compression header to message protocol docs by [@&#8203;Br1an67](https://github.com/Br1an67) in [#&#8203;10156](celery/celery#10156)
- docs: fix duplicated word in bootsteps comment by [@&#8203;Rohan5commit](https://github.com/Rohan5commit) in [#&#8203;10153](celery/celery#10153)
- Remove outdated autoreloader section from extending docs by [@&#8203;Br1an67](https://github.com/Br1an67) in [#&#8203;10154](celery/celery#10154)
- Fix: prioritize request ignore\_result over task definition by [@&#8203;patri27826](https://github.com/patri27826) in [#&#8203;10184](celery/celery#10184)
- fix: clear the timer while catch the exception by [@&#8203;ChickenBenny](https://github.com/ChickenBenny) in [#&#8203;10218](celery/celery#10218)
- Prepare for release: v5.6.3 by [@&#8203;Nusnus](https://github.com/Nusnus) in [#&#8203;10221](celery/celery#10221)

#### New Contributors

- [@&#8203;maycuatroi1](https://github.com/maycuatroi1) made their first contribution in [#&#8203;10048](celery/celery#10048)
- [@&#8203;jorsyk](https://github.com/jorsyk) made their first contribution in [#&#8203;10056](celery/celery#10056)
- [@&#8203;Timour-Ilyas](https://github.com/Timour-Ilyas) made their first contribution in [#&#8203;10066](celery/celery#10066)
- [@&#8203;sueannioanis](https://github.com/sueannioanis) made their first contribution in [#&#8203;10071](celery/celery#10071)
- [@&#8203;GangEunzzang](https://github.com/GangEunzzang) made their first contribution in [#&#8203;10076](celery/celery#10076)
- [@&#8203;sbc-khacnha](https://github.com/sbc-khacnha) made their first contribution in [#&#8203;10081](celery/celery#10081)
- [@&#8203;veeceey](https://github.com/veeceey) made their first contribution in [#&#8203;10100](celery/celery#10100)
- [@&#8203;mriddle](https://github.com/mriddle) made their first contribution in [#&#8203;10086](celery/celery#10086)
- [@&#8203;Mr-Neutr0n](https://github.com/Mr-Neutr0n) made their first contribution in [#&#8203;10121](celery/celery#10121)
- [@&#8203;ChickenBenny](https://github.com/ChickenBenny) made their first contribution in [#&#8203;10108](celery/celery#10108)
- [@&#8203;bysiber](https://github.com/bysiber) made their first contribution in [#&#8203;10137](celery/celery#10137)
- [@&#8203;Rohan5commit](https://github.com/Rohan5commit) made their first contribution in [#&#8203;10139](celery/celery#10139)
- [@&#8203;cuiweixie](https://github.com/cuiweixie) made their first contribution in [#&#8203;10133](celery/celery#10133)
- [@&#8203;kelsonbrito50](https://github.com/kelsonbrito50) made their first contribution in [#&#8203;10145](celery/celery#10145)
- [@&#8203;Br1an67](https://github.com/Br1an67) made their first contribution in [#&#8203;10155](celery/celery#10155)
- [@&#8203;daniel7an](https://github.com/daniel7an) made their first contribution in [#&#8203;10103](celery/celery#10103)
- [@&#8203;drichardson](https://github.com/drichardson) made their first contribution in [#&#8203;10165](celery/celery#10165)
- [@&#8203;tsangwailam](https://github.com/tsangwailam) made their first contribution in [#&#8203;10181](celery/celery#10181)
- [@&#8203;Borzik](https://github.com/Borzik) made their first contribution in [#&#8203;10171](celery/celery#10171)
- [@&#8203;aurangzaib048](https://github.com/aurangzaib048) made their first contribution in [#&#8203;10159](celery/celery#10159)
- [@&#8203;ericbuehl](https://github.com/ericbuehl) made their first contribution in [#&#8203;10187](celery/celery#10187)
- [@&#8203;KianAnbarestani](https://github.com/KianAnbarestani) made their first contribution in [#&#8203;10192](celery/celery#10192)
- [@&#8203;patri27826](https://github.com/patri27826) made their first contribution in [#&#8203;10184](celery/celery#10184)

**Full Changelog**: <celery/celery@v5.6.2...v5.6.3>

</details>

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My45OS4wIiwidXBkYXRlZEluVmVyIjoiNDMuOTkuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiZGVwZW5kZW5jaWVzIiwicmVub3ZhdGUiXX0=-->

See merge request swiss-armed-forces/cyber-command/cea/loom!419
pchopinet pushed a commit to JaaJSoft/workspace that referenced this pull request Apr 4, 2026
Bumps [celery](https://github.com/celery/celery) from 5.6.2 to 5.6.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/celery/celery/releases">celery's
releases</a>.</em></p>
<blockquote>
<h2>v5.6.3</h2>
<h2>What's Changed</h2>
<ul>
<li>Fix Django worker recursion bug + defensive checks for
pool_cls.<strong>module</strong> by <a
href="https://github.com/maycuatroi1"><code>@​maycuatroi1</code></a> in
<a
href="https://redirect.github.com/celery/celery/pull/10048">celery/celery#10048</a></li>
<li>Docs: Update user_preload_options example to use click. by <a
href="https://github.com/jorsyk"><code>@​jorsyk</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10056">celery/celery#10056</a></li>
<li>Fix invalid configuration key &quot;bootstrap_servers&quot; in Kafka
demo by <a href="https://github.com/jorsyk"><code>@​jorsyk</code></a> in
<a
href="https://redirect.github.com/celery/celery/pull/10060">celery/celery#10060</a></li>
<li>Fix broken images on PyPI page by <a
href="https://github.com/Timour-Ilyas"><code>@​Timour-Ilyas</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10066">celery/celery#10066</a></li>
<li>Remove broken reference. by <a
href="https://github.com/sueannioanis"><code>@​sueannioanis</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10071">celery/celery#10071</a></li>
<li>Removed --dist=loadscope from smoke tests by <a
href="https://github.com/Nusnus"><code>@​Nusnus</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10073">celery/celery#10073</a></li>
<li>Docs: Clarify task_retry signal args may be None by <a
href="https://github.com/GangEunzzang"><code>@​GangEunzzang</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10076">celery/celery#10076</a></li>
<li>Update example for Django by <a
href="https://github.com/sbc-khacnha"><code>@​sbc-khacnha</code></a> in
<a
href="https://redirect.github.com/celery/celery/pull/10081">celery/celery#10081</a></li>
<li>Make tests compatible with pymongo &gt;= 4.16 by <a
href="https://github.com/cjwatson"><code>@​cjwatson</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10074">celery/celery#10074</a></li>
<li>fix: source install of cassandra-driver by <a
href="https://github.com/Izzette"><code>@​Izzette</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10105">celery/celery#10105</a></li>
<li>fix: register task cross-reference role in Sphinx extension by <a
href="https://github.com/veeceey"><code>@​veeceey</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10100">celery/celery#10100</a></li>
<li>fix: avoid cycle detection in native delayed delivery by <a
href="https://github.com/Izzette"><code>@​Izzette</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10095">celery/celery#10095</a></li>
<li>fix(asynpool): avoid AttributeError when proc lacks _sentinel_poll
by <a href="https://github.com/mriddle"><code>@​mriddle</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10086">celery/celery#10086</a></li>
<li>fix dusk_astronomical horizon sign (+18 -&gt; -18) by <a
href="https://github.com/Mr-Neutr0n"><code>@​Mr-Neutr0n</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10121">celery/celery#10121</a></li>
<li>Fix/10106 onupdate col use lambda func by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10108">celery/celery#10108</a></li>
<li>Fix warm shutdown RuntimeError with eventlet&gt;=0.37.0 (<a
href="https://redirect.github.com/celery/celery/issues/10083">#10083</a>)
by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10123">celery/celery#10123</a></li>
<li>Fix 10109 db backend connection health by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10124">celery/celery#10124</a></li>
<li>Database Backend filter unsupport sql engine arguments with nullpool
<a
href="https://redirect.github.com/celery/celery/issues/7355">#7355</a>
by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10134">celery/celery#10134</a></li>
<li>fix(beat): correct argument order in Service.<strong>reduce</strong>
by <a href="https://github.com/bysiber"><code>@​bysiber</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10137">celery/celery#10137</a></li>
<li>ci: declare explicit read-only token permissions in workflow jobs by
<a
href="https://github.com/Rohan5commit"><code>@​Rohan5commit</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10139">celery/celery#10139</a></li>
<li>chore: 'boto3to' to 'boto3 to' by <a
href="https://github.com/cuiweixie"><code>@​cuiweixie</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10133">celery/celery#10133</a></li>
<li>Database Backend: Add missing index on date_done (Fixes <a
href="https://redirect.github.com/celery/celery/issues/10097">#10097</a>)
by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10098">celery/celery#10098</a></li>
<li>docs: fix typo in CONTRIBUTING.rst by <a
href="https://github.com/Rohan5commit"><code>@​Rohan5commit</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10141">celery/celery#10141</a></li>
<li>Refer to Flower / Prometheus for monitoring by <a
href="https://github.com/WilliamDEdwards"><code>@​WilliamDEdwards</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10140">celery/celery#10140</a></li>
<li>docs: remove duplicated words in broker and routing docs by <a
href="https://github.com/Rohan5commit"><code>@​Rohan5commit</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10146">celery/celery#10146</a></li>
<li>docs: fix stale version reference and grammar in README by <a
href="https://github.com/kelsonbrito50"><code>@​kelsonbrito50</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10145">celery/celery#10145</a></li>
<li>docs: fix wording in Celery 5.3 worker pool notes by <a
href="https://github.com/Rohan5commit"><code>@​Rohan5commit</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10149">celery/celery#10149</a></li>
<li>docs: fix duplicated wording in 3.1 changelog entry by <a
href="https://github.com/Rohan5commit"><code>@​Rohan5commit</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10152">celery/celery#10152</a></li>
<li>docs: fix changelog typo in context manager wording by <a
href="https://github.com/Rohan5commit"><code>@​Rohan5commit</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10144">celery/celery#10144</a></li>
<li>Fix/10096 worker fails to reconnect after redis failover by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10151">celery/celery#10151</a></li>
<li>Improve on_after_finalize signal documentation by <a
href="https://github.com/Br1an67"><code>@​Br1an67</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10155">celery/celery#10155</a></li>
<li>Add non-commutative example to clarify partial arg ordering in
canvas docs by <a
href="https://github.com/Br1an67"><code>@​Br1an67</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10157">celery/celery#10157</a></li>
<li>Remove redundant test_isa_mapping test (fixes <a
href="https://redirect.github.com/celery/celery/issues/10077">#10077</a>)
by <a href="https://github.com/daniel7an"><code>@​daniel7an</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10103">celery/celery#10103</a></li>
<li>Upgrade pytest-celery to &gt;=1.3.0 and adopt PYTEST_CELERY_PKG
build arg by <a
href="https://github.com/Nusnus"><code>@​Nusnus</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10162">celery/celery#10162</a></li>
<li>Remove deprecated args from redis get_connection call by <a
href="https://github.com/JaeHyuckSa"><code>@​JaeHyuckSa</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10036">celery/celery#10036</a></li>
<li>Fix <a
href="https://redirect.github.com/celery/celery/issues/6912">#6912</a>
rpc backend reconnection error by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10179">celery/celery#10179</a></li>
<li>Fix NameError with TYPE_CHECKING annotations on Python 3.14+ (PEP
649) by <a
href="https://github.com/drichardson"><code>@​drichardson</code></a> in
<a
href="https://redirect.github.com/celery/celery/pull/10165">celery/celery#10165</a></li>
<li>docs: Add elaboration on prefetch multiplier settings
(worker_prefetch_multiplier) and worker_eta_task_limit by <a
href="https://github.com/tsangwailam"><code>@​tsangwailam</code></a> in
<a
href="https://redirect.github.com/celery/celery/pull/10181">celery/celery#10181</a></li>
<li>Fix O(K²) message bloat in a chain of chords by <a
href="https://github.com/Borzik"><code>@​Borzik</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10171">celery/celery#10171</a></li>
<li>Fix mock connection interfaces to prevent <code>TypeError</code>
during exception handling by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10178">celery/celery#10178</a></li>
<li>fix(trace): dispatch chain/callbacks on dedup fast-path for
redelivered tasks by <a
href="https://github.com/aurangzaib048"><code>@​aurangzaib048</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10159">celery/celery#10159</a></li>
<li>Extract <code>reconnect_on_error</code> to
<code>BaseResultConsumer</code> by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10189">celery/celery#10189</a></li>
<li>pep 649 by <a
href="https://github.com/ericbuehl"><code>@​ericbuehl</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10187">celery/celery#10187</a></li>
<li>Fix#9722 friendly status errors for CLI by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10190">celery/celery#10190</a></li>
<li>docs: clarify after_return behavior for retried tasks by <a
href="https://github.com/KianAnbarestani"><code>@​KianAnbarestani</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10192">celery/celery#10192</a></li>
<li>Add compression header to message protocol docs by <a
href="https://github.com/Br1an67"><code>@​Br1an67</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10156">celery/celery#10156</a></li>
<li>docs: fix duplicated word in bootsteps comment by <a
href="https://github.com/Rohan5commit"><code>@​Rohan5commit</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10153">celery/celery#10153</a></li>
<li>Remove outdated autoreloader section from extending docs by <a
href="https://github.com/Br1an67"><code>@​Br1an67</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10154">celery/celery#10154</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/celery/celery/blob/v5.6.3/Changelog.rst">celery's
changelog</a>.</em></p>
<blockquote>
<h1>5.6.3</h1>
<p>:release-date: 2026-03-26
:release-by: Tomer Nosrati</p>
<p>What's Changed</p>
<pre><code>
- Fix Django worker recursion bug + defensive checks for
pool_cls.__module__
([#10048](celery/celery#10048))
- Docs: Update user_preload_options example to use click.
([#10056](celery/celery#10056))
- Fix invalid configuration key &quot;bootstrap_servers&quot; in Kafka
demo ([#10060](celery/celery#10060))
- Fix broken images on PyPI page
([#10066](celery/celery#10066))
- Remove broken reference.
([#10071](celery/celery#10071))
- Removed --dist=loadscope from smoke tests
([#10073](celery/celery#10073))
- Docs: Clarify task_retry signal args may be None
([#10076](celery/celery#10076))
- Update example for Django
([#10081](celery/celery#10081))
- Make tests compatible with pymongo &gt;= 4.16
([#10074](celery/celery#10074))
- fix: source install of cassandra-driver
([#10105](celery/celery#10105))
- fix: register task cross-reference role in Sphinx extension
([#10100](celery/celery#10100))
- fix: avoid cycle detection in native delayed delivery
([#10095](celery/celery#10095))
- fix(asynpool): avoid AttributeError when proc lacks _sentinel_poll
([#10086](celery/celery#10086))
- fix dusk_astronomical horizon sign (+18 -&gt; -18)
([#10121](celery/celery#10121))
- Fix/10106 onupdate col use lambda func
([#10108](celery/celery#10108))
- Fix warm shutdown RuntimeError with eventlet&gt;=0.37.0
([#10083](celery/celery#10083))
([#10123](celery/celery#10123))
- Fix 10109 db backend connection health
([#10124](celery/celery#10124))
- Database Backend filter unsupport sql engine arguments with nullpool
[#7355](celery/celery#7355)
([#10134](celery/celery#10134))
- fix(beat): correct argument order in Service.__reduce__
([#10137](celery/celery#10137))
- ci: declare explicit read-only token permissions in workflow jobs
([#10139](celery/celery#10139))
- chore: 'boto3to' to 'boto3 to'
([#10133](celery/celery#10133))
- Database Backend: Add missing index on date_done (Fixes
[#10097](celery/celery#10097))
([#10098](celery/celery#10098))
- docs: fix typo in CONTRIBUTING.rst
([#10141](celery/celery#10141))
- Refer to Flower / Prometheus for monitoring
([#10140](celery/celery#10140))
- docs: remove duplicated words in broker and routing docs
([#10146](celery/celery#10146))
- docs: fix stale version reference and grammar in README
([#10145](celery/celery#10145))
- docs: fix wording in Celery 5.3 worker pool notes
([#10149](celery/celery#10149))
- docs: fix duplicated wording in 3.1 changelog entry
([#10152](celery/celery#10152))
- docs: fix changelog typo in context manager wording
([#10144](celery/celery#10144))
- Fix/10096 worker fails to reconnect after redis failover
([#10151](celery/celery#10151))
- Improve on_after_finalize signal documentation
([#10155](celery/celery#10155))
- Add non-commutative example to clarify partial arg ordering in canvas
docs ([#10157](celery/celery#10157))
- Remove redundant test_isa_mapping test (fixes
[#10077](celery/celery#10077))
([#10103](celery/celery#10103))
- Upgrade pytest-celery to &gt;=1.3.0 and adopt PYTEST_CELERY_PKG build
arg ([#10162](celery/celery#10162))
- Remove deprecated args from redis get_connection call
([#10036](celery/celery#10036))
- Fix [#6912](celery/celery#6912) rpc backend
reconnection error
([#10179](celery/celery#10179))
- Fix NameError with TYPE_CHECKING annotations on Python 3.14+ (PEP 649)
([#10165](celery/celery#10165))
- docs: Add elaboration on prefetch multiplier settings
(worker_prefetch_multiplier) and worker_eta_task_limit
([#10181](celery/celery#10181))
- Fix O(K²) message bloat in a chain of chords
([#10171](celery/celery#10171))
- Fix mock connection interfaces to prevent `TypeError` during exception
handling ([#10178](celery/celery#10178))
- fix(trace): dispatch chain/callbacks on dedup fast-path for
redelivered tasks
([#10159](celery/celery#10159))
&lt;/tr&gt;&lt;/table&gt; 
</code></pre>
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/celery/celery/commit/3f4d8d795ad128bd7430cc5dc174a802cded425c"><code>3f4d8d7</code></a>
Prepare for release: v5.6.3 (<a
href="https://redirect.github.com/celery/celery/issues/10221">#10221</a>)</li>
<li><a
href="https://github.com/celery/celery/commit/a989e8cf8876274b1f0612abffeeb2e9995ed321"><code>a989e8c</code></a>
fix: clear the timer while catch the exception (<a
href="https://redirect.github.com/celery/celery/issues/10218">#10218</a>)</li>
<li><a
href="https://github.com/celery/celery/commit/d06de5f047620b0ea2bdbdb3c0c56137b79ae9a1"><code>d06de5f</code></a>
Chore(deps): Bump nick-fields/retry from 3 to 4 (<a
href="https://redirect.github.com/celery/celery/issues/10213">#10213</a>)</li>
<li><a
href="https://github.com/celery/celery/commit/c3c19c31dc3e21f16d4d85a8ba8401a9223ace09"><code>c3c19c3</code></a>
Fix: prioritize request ignore_result over task definition (<a
href="https://redirect.github.com/celery/celery/issues/10184">#10184</a>)</li>
<li><a
href="https://github.com/celery/celery/commit/d23be53f6f3600d48df35a797c63eb1c7d4d4b97"><code>d23be53</code></a>
Remove outdated autoreloader section from extending docs (<a
href="https://redirect.github.com/celery/celery/issues/10154">#10154</a>)</li>
<li><a
href="https://github.com/celery/celery/commit/ada2da7475a5fa9f9ad079149a5d6864634abc28"><code>ada2da7</code></a>
docs: fix duplicated word in bootsteps comment\n\nSigned-off-by: Rohan
Santho...</li>
<li><a
href="https://github.com/celery/celery/commit/f45f62beb3b16ae960944f8c97de13ccf15f2d0a"><code>f45f62b</code></a>
Add compression header to message protocol docs (<a
href="https://redirect.github.com/celery/celery/issues/10156">#10156</a>)</li>
<li><a
href="https://github.com/celery/celery/commit/9a270925546ed9d0ca0303fb5006edc86b705fd9"><code>9a27092</code></a>
docs: clarify after_return behavior for retried tasks (<a
href="https://redirect.github.com/celery/celery/issues/10192">#10192</a>)</li>
<li><a
href="https://github.com/celery/celery/commit/6ee6230cd80ef6c3e7482e1f4cd970fbb0629b23"><code>6ee6230</code></a>
Fix#9722 friendly status errors for CLI (<a
href="https://redirect.github.com/celery/celery/issues/10190">#10190</a>)</li>
<li><a
href="https://github.com/celery/celery/commit/a9a2d4cecaf0e58c401ad6f68f022afa19770ac2"><code>a9a2d4c</code></a>
[pre-commit.ci] pre-commit autoupdate (<a
href="https://redirect.github.com/celery/celery/issues/10186">#10186</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/celery/celery/compare/v5.6.2...v5.6.3">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=celery&package-manager=uv&previous-version=5.6.2&new-version=5.6.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
HsiangNianian added a commit to retrofor/iamai that referenced this pull request Apr 15, 2026
Updates the requirements on [celery](https://github.com/celery/celery)
to permit the latest version.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/celery/celery/releases">celery's
releases</a>.</em></p>
<blockquote>
<h2>v5.6.3</h2>
<h2>What's Changed</h2>
<ul>
<li>Fix Django worker recursion bug + defensive checks for
pool_cls.<strong>module</strong> by <a
href="https://github.com/maycuatroi1"><code>@​maycuatroi1</code></a> in
<a
href="https://redirect.github.com/celery/celery/pull/10048">celery/celery#10048</a></li>
<li>Docs: Update user_preload_options example to use click. by <a
href="https://github.com/jorsyk"><code>@​jorsyk</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10056">celery/celery#10056</a></li>
<li>Fix invalid configuration key &quot;bootstrap_servers&quot; in Kafka
demo by <a href="https://github.com/jorsyk"><code>@​jorsyk</code></a> in
<a
href="https://redirect.github.com/celery/celery/pull/10060">celery/celery#10060</a></li>
<li>Fix broken images on PyPI page by <a
href="https://github.com/Timour-Ilyas"><code>@​Timour-Ilyas</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10066">celery/celery#10066</a></li>
<li>Remove broken reference. by <a
href="https://github.com/sueannioanis"><code>@​sueannioanis</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10071">celery/celery#10071</a></li>
<li>Removed --dist=loadscope from smoke tests by <a
href="https://github.com/Nusnus"><code>@​Nusnus</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10073">celery/celery#10073</a></li>
<li>Docs: Clarify task_retry signal args may be None by <a
href="https://github.com/GangEunzzang"><code>@​GangEunzzang</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10076">celery/celery#10076</a></li>
<li>Update example for Django by <a
href="https://github.com/sbc-khacnha"><code>@​sbc-khacnha</code></a> in
<a
href="https://redirect.github.com/celery/celery/pull/10081">celery/celery#10081</a></li>
<li>Make tests compatible with pymongo &gt;= 4.16 by <a
href="https://github.com/cjwatson"><code>@​cjwatson</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10074">celery/celery#10074</a></li>
<li>fix: source install of cassandra-driver by <a
href="https://github.com/Izzette"><code>@​Izzette</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10105">celery/celery#10105</a></li>
<li>fix: register task cross-reference role in Sphinx extension by <a
href="https://github.com/veeceey"><code>@​veeceey</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10100">celery/celery#10100</a></li>
<li>fix: avoid cycle detection in native delayed delivery by <a
href="https://github.com/Izzette"><code>@​Izzette</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10095">celery/celery#10095</a></li>
<li>fix(asynpool): avoid AttributeError when proc lacks _sentinel_poll
by <a href="https://github.com/mriddle"><code>@​mriddle</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10086">celery/celery#10086</a></li>
<li>fix dusk_astronomical horizon sign (+18 -&gt; -18) by <a
href="https://github.com/Mr-Neutr0n"><code>@​Mr-Neutr0n</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10121">celery/celery#10121</a></li>
<li>Fix/10106 onupdate col use lambda func by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10108">celery/celery#10108</a></li>
<li>Fix warm shutdown RuntimeError with eventlet&gt;=0.37.0 (<a
href="https://redirect.github.com/celery/celery/issues/10083">#10083</a>)
by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10123">celery/celery#10123</a></li>
<li>Fix 10109 db backend connection health by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10124">celery/celery#10124</a></li>
<li>Database Backend filter unsupport sql engine arguments with nullpool
<a
href="https://redirect.github.com/celery/celery/issues/7355">#7355</a>
by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10134">celery/celery#10134</a></li>
<li>fix(beat): correct argument order in Service.<strong>reduce</strong>
by <a href="https://github.com/bysiber"><code>@​bysiber</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10137">celery/celery#10137</a></li>
<li>ci: declare explicit read-only token permissions in workflow jobs by
<a
href="https://github.com/Rohan5commit"><code>@​Rohan5commit</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10139">celery/celery#10139</a></li>
<li>chore: 'boto3to' to 'boto3 to' by <a
href="https://github.com/cuiweixie"><code>@​cuiweixie</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10133">celery/celery#10133</a></li>
<li>Database Backend: Add missing index on date_done (Fixes <a
href="https://redirect.github.com/celery/celery/issues/10097">#10097</a>)
by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10098">celery/celery#10098</a></li>
<li>docs: fix typo in CONTRIBUTING.rst by <a
href="https://github.com/Rohan5commit"><code>@​Rohan5commit</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10141">celery/celery#10141</a></li>
<li>Refer to Flower / Prometheus for monitoring by <a
href="https://github.com/WilliamDEdwards"><code>@​WilliamDEdwards</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10140">celery/celery#10140</a></li>
<li>docs: remove duplicated words in broker and routing docs by <a
href="https://github.com/Rohan5commit"><code>@​Rohan5commit</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10146">celery/celery#10146</a></li>
<li>docs: fix stale version reference and grammar in README by <a
href="https://github.com/kelsonbrito50"><code>@​kelsonbrito50</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10145">celery/celery#10145</a></li>
<li>docs: fix wording in Celery 5.3 worker pool notes by <a
href="https://github.com/Rohan5commit"><code>@​Rohan5commit</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10149">celery/celery#10149</a></li>
<li>docs: fix duplicated wording in 3.1 changelog entry by <a
href="https://github.com/Rohan5commit"><code>@​Rohan5commit</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10152">celery/celery#10152</a></li>
<li>docs: fix changelog typo in context manager wording by <a
href="https://github.com/Rohan5commit"><code>@​Rohan5commit</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10144">celery/celery#10144</a></li>
<li>Fix/10096 worker fails to reconnect after redis failover by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10151">celery/celery#10151</a></li>
<li>Improve on_after_finalize signal documentation by <a
href="https://github.com/Br1an67"><code>@​Br1an67</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10155">celery/celery#10155</a></li>
<li>Add non-commutative example to clarify partial arg ordering in
canvas docs by <a
href="https://github.com/Br1an67"><code>@​Br1an67</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10157">celery/celery#10157</a></li>
<li>Remove redundant test_isa_mapping test (fixes <a
href="https://redirect.github.com/celery/celery/issues/10077">#10077</a>)
by <a href="https://github.com/daniel7an"><code>@​daniel7an</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10103">celery/celery#10103</a></li>
<li>Upgrade pytest-celery to &gt;=1.3.0 and adopt PYTEST_CELERY_PKG
build arg by <a
href="https://github.com/Nusnus"><code>@​Nusnus</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10162">celery/celery#10162</a></li>
<li>Remove deprecated args from redis get_connection call by <a
href="https://github.com/JaeHyuckSa"><code>@​JaeHyuckSa</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10036">celery/celery#10036</a></li>
<li>Fix <a
href="https://redirect.github.com/celery/celery/issues/6912">#6912</a>
rpc backend reconnection error by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10179">celery/celery#10179</a></li>
<li>Fix NameError with TYPE_CHECKING annotations on Python 3.14+ (PEP
649) by <a
href="https://github.com/drichardson"><code>@​drichardson</code></a> in
<a
href="https://redirect.github.com/celery/celery/pull/10165">celery/celery#10165</a></li>
<li>docs: Add elaboration on prefetch multiplier settings
(worker_prefetch_multiplier) and worker_eta_task_limit by <a
href="https://github.com/tsangwailam"><code>@​tsangwailam</code></a> in
<a
href="https://redirect.github.com/celery/celery/pull/10181">celery/celery#10181</a></li>
<li>Fix O(K²) message bloat in a chain of chords by <a
href="https://github.com/Borzik"><code>@​Borzik</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10171">celery/celery#10171</a></li>
<li>Fix mock connection interfaces to prevent <code>TypeError</code>
during exception handling by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10178">celery/celery#10178</a></li>
<li>fix(trace): dispatch chain/callbacks on dedup fast-path for
redelivered tasks by <a
href="https://github.com/aurangzaib048"><code>@​aurangzaib048</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10159">celery/celery#10159</a></li>
<li>Extract <code>reconnect_on_error</code> to
<code>BaseResultConsumer</code> by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10189">celery/celery#10189</a></li>
<li>pep 649 by <a
href="https://github.com/ericbuehl"><code>@​ericbuehl</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10187">celery/celery#10187</a></li>
<li>Fix#9722 friendly status errors for CLI by <a
href="https://github.com/ChickenBenny"><code>@​ChickenBenny</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10190">celery/celery#10190</a></li>
<li>docs: clarify after_return behavior for retried tasks by <a
href="https://github.com/KianAnbarestani"><code>@​KianAnbarestani</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10192">celery/celery#10192</a></li>
<li>Add compression header to message protocol docs by <a
href="https://github.com/Br1an67"><code>@​Br1an67</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10156">celery/celery#10156</a></li>
<li>docs: fix duplicated word in bootsteps comment by <a
href="https://github.com/Rohan5commit"><code>@​Rohan5commit</code></a>
in <a
href="https://redirect.github.com/celery/celery/pull/10153">celery/celery#10153</a></li>
<li>Remove outdated autoreloader section from extending docs by <a
href="https://github.com/Br1an67"><code>@​Br1an67</code></a> in <a
href="https://redirect.github.com/celery/celery/pull/10154">celery/celery#10154</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/celery/celery/blob/v5.6.3/Changelog.rst">celery's
changelog</a>.</em></p>
<blockquote>
<h1>5.6.3</h1>
<p>:release-date: 2026-03-26
:release-by: Tomer Nosrati</p>
<p>What's Changed</p>
<pre><code>
- Fix Django worker recursion bug + defensive checks for
pool_cls.__module__
([#10048](celery/celery#10048))
- Docs: Update user_preload_options example to use click.
([#10056](celery/celery#10056))
- Fix invalid configuration key &quot;bootstrap_servers&quot; in Kafka
demo ([#10060](celery/celery#10060))
- Fix broken images on PyPI page
([#10066](celery/celery#10066))
- Remove broken reference.
([#10071](celery/celery#10071))
- Removed --dist=loadscope from smoke tests
([#10073](celery/celery#10073))
- Docs: Clarify task_retry signal args may be None
([#10076](celery/celery#10076))
- Update example for Django
([#10081](celery/celery#10081))
- Make tests compatible with pymongo &gt;= 4.16
([#10074](celery/celery#10074))
- fix: source install of cassandra-driver
([#10105](celery/celery#10105))
- fix: register task cross-reference role in Sphinx extension
([#10100](celery/celery#10100))
- fix: avoid cycle detection in native delayed delivery
([#10095](celery/celery#10095))
- fix(asynpool): avoid AttributeError when proc lacks _sentinel_poll
([#10086](celery/celery#10086))
- fix dusk_astronomical horizon sign (+18 -&gt; -18)
([#10121](celery/celery#10121))
- Fix/10106 onupdate col use lambda func
([#10108](celery/celery#10108))
- Fix warm shutdown RuntimeError with eventlet&gt;=0.37.0
([#10083](celery/celery#10083))
([#10123](celery/celery#10123))
- Fix 10109 db backend connection health
([#10124](celery/celery#10124))
- Database Backend filter unsupport sql engine arguments with nullpool
[#7355](celery/celery#7355)
([#10134](celery/celery#10134))
- fix(beat): correct argument order in Service.__reduce__
([#10137](celery/celery#10137))
- ci: declare explicit read-only token permissions in workflow jobs
([#10139](celery/celery#10139))
- chore: 'boto3to' to 'boto3 to'
([#10133](celery/celery#10133))
- Database Backend: Add missing index on date_done (Fixes
[#10097](celery/celery#10097))
([#10098](celery/celery#10098))
- docs: fix typo in CONTRIBUTING.rst
([#10141](celery/celery#10141))
- Refer to Flower / Prometheus for monitoring
([#10140](celery/celery#10140))
- docs: remove duplicated words in broker and routing docs
([#10146](celery/celery#10146))
- docs: fix stale version reference and grammar in README
([#10145](celery/celery#10145))
- docs: fix wording in Celery 5.3 worker pool notes
([#10149](celery/celery#10149))
- docs: fix duplicated wording in 3.1 changelog entry
([#10152](celery/celery#10152))
- docs: fix changelog typo in context manager wording
([#10144](celery/celery#10144))
- Fix/10096 worker fails to reconnect after redis failover
([#10151](celery/celery#10151))
- Improve on_after_finalize signal documentation
([#10155](celery/celery#10155))
- Add non-commutative example to clarify partial arg ordering in canvas
docs ([#10157](celery/celery#10157))
- Remove redundant test_isa_mapping test (fixes
[#10077](celery/celery#10077))
([#10103](celery/celery#10103))
- Upgrade pytest-celery to &gt;=1.3.0 and adopt PYTEST_CELERY_PKG build
arg ([#10162](celery/celery#10162))
- Remove deprecated args from redis get_connection call
([#10036](celery/celery#10036))
- Fix [#6912](celery/celery#6912) rpc backend
reconnection error
([#10179](celery/celery#10179))
- Fix NameError with TYPE_CHECKING annotations on Python 3.14+ (PEP 649)
([#10165](celery/celery#10165))
- docs: Add elaboration on prefetch multiplier settings
(worker_prefetch_multiplier) and worker_eta_task_limit
([#10181](celery/celery#10181))
- Fix O(K²) message bloat in a chain of chords
([#10171](celery/celery#10171))
- Fix mock connection interfaces to prevent `TypeError` during exception
handling ([#10178](celery/celery#10178))
- fix(trace): dispatch chain/callbacks on dedup fast-path for
redelivered tasks
([#10159](celery/celery#10159))
&lt;/tr&gt;&lt;/table&gt; 
</code></pre>
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/celery/celery/commit/3f4d8d795ad128bd7430cc5dc174a802cded425c"><code>3f4d8d7</code></a>
Prepare for release: v5.6.3 (<a
href="https://redirect.github.com/celery/celery/issues/10221">#10221</a>)</li>
<li><a
href="https://github.com/celery/celery/commit/a989e8cf8876274b1f0612abffeeb2e9995ed321"><code>a989e8c</code></a>
fix: clear the timer while catch the exception (<a
href="https://redirect.github.com/celery/celery/issues/10218">#10218</a>)</li>
<li><a
href="https://github.com/celery/celery/commit/d06de5f047620b0ea2bdbdb3c0c56137b79ae9a1"><code>d06de5f</code></a>
Chore(deps): Bump nick-fields/retry from 3 to 4 (<a
href="https://redirect.github.com/celery/celery/issues/10213">#10213</a>)</li>
<li><a
href="https://github.com/celery/celery/commit/c3c19c31dc3e21f16d4d85a8ba8401a9223ace09"><code>c3c19c3</code></a>
Fix: prioritize request ignore_result over task definition (<a
href="https://redirect.github.com/celery/celery/issues/10184">#10184</a>)</li>
<li><a
href="https://github.com/celery/celery/commit/d23be53f6f3600d48df35a797c63eb1c7d4d4b97"><code>d23be53</code></a>
Remove outdated autoreloader section from extending docs (<a
href="https://redirect.github.com/celery/celery/issues/10154">#10154</a>)</li>
<li><a
href="https://github.com/celery/celery/commit/ada2da7475a5fa9f9ad079149a5d6864634abc28"><code>ada2da7</code></a>
docs: fix duplicated word in bootsteps comment\n\nSigned-off-by: Rohan
Santho...</li>
<li><a
href="https://github.com/celery/celery/commit/f45f62beb3b16ae960944f8c97de13ccf15f2d0a"><code>f45f62b</code></a>
Add compression header to message protocol docs (<a
href="https://redirect.github.com/celery/celery/issues/10156">#10156</a>)</li>
<li><a
href="https://github.com/celery/celery/commit/9a270925546ed9d0ca0303fb5006edc86b705fd9"><code>9a27092</code></a>
docs: clarify after_return behavior for retried tasks (<a
href="https://redirect.github.com/celery/celery/issues/10192">#10192</a>)</li>
<li><a
href="https://github.com/celery/celery/commit/6ee6230cd80ef6c3e7482e1f4cd970fbb0629b23"><code>6ee6230</code></a>
Fix#9722 friendly status errors for CLI (<a
href="https://redirect.github.com/celery/celery/issues/10190">#10190</a>)</li>
<li><a
href="https://github.com/celery/celery/commit/a9a2d4cecaf0e58c401ad6f68f022afa19770ac2"><code>a9a2d4c</code></a>
[pre-commit.ci] pre-commit autoupdate (<a
href="https://redirect.github.com/celery/celery/issues/10186">#10186</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/celery/celery/compare/v5.3.4...v5.6.3">compare
view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RabbitMQ detects cycle and drops message when using quorum queues with delayed tasks + retry

4 participants