fix: avoid cycle detection in native delayed delivery#10095
Conversation
Integration test error output before applying the fix when running with RabbitMQ 3.13 |
b409f91 to
1acbaf5
Compare
* Avoid RabbitMQ <4.0.1 dead-lettering cycle detection when retrying with native delayed-delivery.
1acbaf5 to
a5dcbc7
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #10095 +/- ##
=======================================
Coverage 87.50% 87.50%
=======================================
Files 153 153
Lines 19374 19380 +6
Branches 2228 2229 +1
=======================================
+ Hits 16953 16959 +6
Misses 2125 2125
Partials 296 296
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
auvipy
left a comment
There was a problem hiding this comment.
can you please check the remaining failing integration tests?
There was a problem hiding this comment.
Pull request overview
This PR addresses RabbitMQ quorum-queue dead-letter cycle detection causing retried tasks using native delayed delivery to be dropped (fix for #9867) by stripping x-death-related headers before republishing retries.
Changes:
- Filter RabbitMQ
x-death/x-*-death-*headers from request execution options prior to republishing (retry path). - Add a unit test ensuring
signature_from_request()does not propagatex-deathheaders. - Add an integration test module intended to exercise the quorum queue + delayed retry scenario.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
celery/app/task.py |
Filters x-death-related headers out of Context.as_execution_options() so retries won’t republish them. |
t/unit/tasks/test_tasks.py |
Adds a unit test asserting x-death header filtering in signature_from_request(). |
t/integration/test_rabbitmq_quorum_queue_cycle_detection.py |
Adds integration coverage for the delayed retry + quorum queue cycle-detection scenario. |
|
@auvipy, the tests are failing due to Docker Hub rate limiting:
|
|
now surprisingly the unit tests are failing |
|
The unit test failures are now some sort of pip or pypi issue:
I've never seen this one before, I'm not sure what to think about it. I don't see how it can be related to 6f9786c though. |
|
lets wait |
|
@auvipy thank you <3 |
|
you are most welcome! feel free to come with any follow up PR if needed! |
* fix: avoid cycle detection in native delayed delivery
* Avoid RabbitMQ <4.0.1 dead-lettering cycle detection when retrying with native delayed-delivery.
* chore: fix typo, rename, unused variable
---------
Co-authored-by: Asif Saif Uddin {"Auvi":"অভি"} <[email protected]>
|
@auvipy can we add it to 5.6.x milestone for a closer release timeline ? |
|
this might be in v5.7a1 |
|
thanks @auvipy |
|
alpha 1 should be by mid march |
* fix: avoid cycle detection in native delayed delivery
* Avoid RabbitMQ <4.0.1 dead-lettering cycle detection when retrying with native delayed-delivery.
* chore: fix typo, rename, unused variable
---------
Co-authored-by: Asif Saif Uddin {"Auvi":"অভি"} <[email protected]>
* fix: avoid cycle detection in native delayed delivery
* Avoid RabbitMQ <4.0.1 dead-lettering cycle detection when retrying with native delayed-delivery.
* chore: fix typo, rename, unused variable
---------
Co-authored-by: Asif Saif Uddin {"Auvi":"অভি"} <[email protected]>
* fix: avoid cycle detection in native delayed delivery
* Avoid RabbitMQ <4.0.1 dead-lettering cycle detection when retrying with native delayed-delivery.
* chore: fix typo, rename, unused variable
---------
Co-authored-by: Asif Saif Uddin {"Auvi":"অভি"} <[email protected]>
* fix: avoid cycle detection in native delayed delivery
* Avoid RabbitMQ <4.0.1 dead-lettering cycle detection when retrying with native delayed-delivery.
* chore: fix typo, rename, unused variable
---------
Co-authored-by: Asif Saif Uddin {"Auvi":"অভি"} <[email protected]>
This MR contains the following updates: | Package | Type | Update | Change | OpenSSF | |---|---|---|---|---| | [celery](https://docs.celeryq.dev/) ([source](https://github.com/celery/celery), [changelog](https://docs.celeryq.dev/en/stable/changelog.html)) | dependencies | patch | `5.6.2` → `5.6.3` | [](https://securityscorecards.dev/viewer/?uri=github.com/celery/celery) | --- ### Release Notes <details> <summary>celery/celery (celery)</summary> ### [`v5.6.3`](https://github.com/celery/celery/releases/tag/v5.6.3) [Compare Source](celery/celery@v5.6.2...v5.6.3) #### What's Changed - Fix Django worker recursion bug + defensive checks for pool\_cls.**module** by [@​maycuatroi1](https://github.com/maycuatroi1) in [#​10048](celery/celery#10048) - Docs: Update user\_preload\_options example to use click. by [@​jorsyk](https://github.com/jorsyk) in [#​10056](celery/celery#10056) - Fix invalid configuration key "bootstrap\_servers" in Kafka demo by [@​jorsyk](https://github.com/jorsyk) in [#​10060](celery/celery#10060) - Fix broken images on PyPI page by [@​Timour-Ilyas](https://github.com/Timour-Ilyas) in [#​10066](celery/celery#10066) - Remove broken reference. by [@​sueannioanis](https://github.com/sueannioanis) in [#​10071](celery/celery#10071) - Removed --dist=loadscope from smoke tests by [@​Nusnus](https://github.com/Nusnus) in [#​10073](celery/celery#10073) - Docs: Clarify task\_retry signal args may be None by [@​GangEunzzang](https://github.com/GangEunzzang) in [#​10076](celery/celery#10076) - Update example for Django by [@​sbc-khacnha](https://github.com/sbc-khacnha) in [#​10081](celery/celery#10081) - Make tests compatible with pymongo >= 4.16 by [@​cjwatson](https://github.com/cjwatson) in [#​10074](celery/celery#10074) - fix: source install of cassandra-driver by [@​Izzette](https://github.com/Izzette) in [#​10105](celery/celery#10105) - fix: register task cross-reference role in Sphinx extension by [@​veeceey](https://github.com/veeceey) in [#​10100](celery/celery#10100) - fix: avoid cycle detection in native delayed delivery by [@​Izzette](https://github.com/Izzette) in [#​10095](celery/celery#10095) - fix(asynpool): avoid AttributeError when proc lacks \_sentinel\_poll by [@​mriddle](https://github.com/mriddle) in [#​10086](celery/celery#10086) - fix dusk\_astronomical horizon sign (+18 -> -18) by [@​Mr-Neutr0n](https://github.com/Mr-Neutr0n) in [#​10121](celery/celery#10121) - Fix/10106 onupdate col use lambda func by [@​ChickenBenny](https://github.com/ChickenBenny) in [#​10108](celery/celery#10108) - Fix warm shutdown RuntimeError with eventlet>=0.37.0 ([#​10083](celery/celery#10083)) by [@​ChickenBenny](https://github.com/ChickenBenny) in [#​10123](celery/celery#10123) - Fix 10109 db backend connection health by [@​ChickenBenny](https://github.com/ChickenBenny) in [#​10124](celery/celery#10124) - Database Backend filter unsupport sql engine arguments with nullpool [#​7355](celery/celery#7355) by [@​ChickenBenny](https://github.com/ChickenBenny) in [#​10134](celery/celery#10134) - fix(beat): correct argument order in Service.**reduce** by [@​bysiber](https://github.com/bysiber) in [#​10137](celery/celery#10137) - ci: declare explicit read-only token permissions in workflow jobs by [@​Rohan5commit](https://github.com/Rohan5commit) in [#​10139](celery/celery#10139) - chore: 'boto3to' to 'boto3 to' by [@​cuiweixie](https://github.com/cuiweixie) in [#​10133](celery/celery#10133) - Database Backend: Add missing index on date\_done (Fixes [#​10097](celery/celery#10097)) by [@​ChickenBenny](https://github.com/ChickenBenny) in [#​10098](celery/celery#10098) - docs: fix typo in CONTRIBUTING.rst by [@​Rohan5commit](https://github.com/Rohan5commit) in [#​10141](celery/celery#10141) - Refer to Flower / Prometheus for monitoring by [@​WilliamDEdwards](https://github.com/WilliamDEdwards) in [#​10140](celery/celery#10140) - docs: remove duplicated words in broker and routing docs by [@​Rohan5commit](https://github.com/Rohan5commit) in [#​10146](celery/celery#10146) - docs: fix stale version reference and grammar in README by [@​kelsonbrito50](https://github.com/kelsonbrito50) in [#​10145](celery/celery#10145) - docs: fix wording in Celery 5.3 worker pool notes by [@​Rohan5commit](https://github.com/Rohan5commit) in [#​10149](celery/celery#10149) - docs: fix duplicated wording in 3.1 changelog entry by [@​Rohan5commit](https://github.com/Rohan5commit) in [#​10152](celery/celery#10152) - docs: fix changelog typo in context manager wording by [@​Rohan5commit](https://github.com/Rohan5commit) in [#​10144](celery/celery#10144) - Fix/10096 worker fails to reconnect after redis failover by [@​ChickenBenny](https://github.com/ChickenBenny) in [#​10151](celery/celery#10151) - Improve on\_after\_finalize signal documentation by [@​Br1an67](https://github.com/Br1an67) in [#​10155](celery/celery#10155) - Add non-commutative example to clarify partial arg ordering in canvas docs by [@​Br1an67](https://github.com/Br1an67) in [#​10157](celery/celery#10157) - Remove redundant test\_isa\_mapping test (fixes [#​10077](celery/celery#10077)) by [@​daniel7an](https://github.com/daniel7an) in [#​10103](celery/celery#10103) - Upgrade pytest-celery to >=1.3.0 and adopt PYTEST\_CELERY\_PKG build arg by [@​Nusnus](https://github.com/Nusnus) in [#​10162](celery/celery#10162) - Remove deprecated args from redis get\_connection call by [@​JaeHyuckSa](https://github.com/JaeHyuckSa) in [#​10036](celery/celery#10036) - Fix [#​6912](celery/celery#6912) rpc backend reconnection error by [@​ChickenBenny](https://github.com/ChickenBenny) in [#​10179](celery/celery#10179) - Fix NameError with TYPE\_CHECKING annotations on Python 3.14+ (PEP 649) by [@​drichardson](https://github.com/drichardson) in [#​10165](celery/celery#10165) - docs: Add elaboration on prefetch multiplier settings (worker\_prefetch\_multiplier) and worker\_eta\_task\_limit by [@​tsangwailam](https://github.com/tsangwailam) in [#​10181](celery/celery#10181) - Fix O(K²) message bloat in a chain of chords by [@​Borzik](https://github.com/Borzik) in [#​10171](celery/celery#10171) - Fix mock connection interfaces to prevent `TypeError` during exception handling by [@​ChickenBenny](https://github.com/ChickenBenny) in [#​10178](celery/celery#10178) - fix(trace): dispatch chain/callbacks on dedup fast-path for redelivered tasks by [@​aurangzaib048](https://github.com/aurangzaib048) in [#​10159](celery/celery#10159) - Extract `reconnect_on_error` to `BaseResultConsumer` by [@​ChickenBenny](https://github.com/ChickenBenny) in [#​10189](celery/celery#10189) - pep 649 by [@​ericbuehl](https://github.com/ericbuehl) in [#​10187](celery/celery#10187) - [Fix#9722](https://github.com/Fix/celery/issues/9722) friendly status errors for CLI by [@​ChickenBenny](https://github.com/ChickenBenny) in [#​10190](celery/celery#10190) - docs: clarify after\_return behavior for retried tasks by [@​KianAnbarestani](https://github.com/KianAnbarestani) in [#​10192](celery/celery#10192) - Add compression header to message protocol docs by [@​Br1an67](https://github.com/Br1an67) in [#​10156](celery/celery#10156) - docs: fix duplicated word in bootsteps comment by [@​Rohan5commit](https://github.com/Rohan5commit) in [#​10153](celery/celery#10153) - Remove outdated autoreloader section from extending docs by [@​Br1an67](https://github.com/Br1an67) in [#​10154](celery/celery#10154) - Fix: prioritize request ignore\_result over task definition by [@​patri27826](https://github.com/patri27826) in [#​10184](celery/celery#10184) - fix: clear the timer while catch the exception by [@​ChickenBenny](https://github.com/ChickenBenny) in [#​10218](celery/celery#10218) - Prepare for release: v5.6.3 by [@​Nusnus](https://github.com/Nusnus) in [#​10221](celery/celery#10221) #### New Contributors - [@​maycuatroi1](https://github.com/maycuatroi1) made their first contribution in [#​10048](celery/celery#10048) - [@​jorsyk](https://github.com/jorsyk) made their first contribution in [#​10056](celery/celery#10056) - [@​Timour-Ilyas](https://github.com/Timour-Ilyas) made their first contribution in [#​10066](celery/celery#10066) - [@​sueannioanis](https://github.com/sueannioanis) made their first contribution in [#​10071](celery/celery#10071) - [@​GangEunzzang](https://github.com/GangEunzzang) made their first contribution in [#​10076](celery/celery#10076) - [@​sbc-khacnha](https://github.com/sbc-khacnha) made their first contribution in [#​10081](celery/celery#10081) - [@​veeceey](https://github.com/veeceey) made their first contribution in [#​10100](celery/celery#10100) - [@​mriddle](https://github.com/mriddle) made their first contribution in [#​10086](celery/celery#10086) - [@​Mr-Neutr0n](https://github.com/Mr-Neutr0n) made their first contribution in [#​10121](celery/celery#10121) - [@​ChickenBenny](https://github.com/ChickenBenny) made their first contribution in [#​10108](celery/celery#10108) - [@​bysiber](https://github.com/bysiber) made their first contribution in [#​10137](celery/celery#10137) - [@​Rohan5commit](https://github.com/Rohan5commit) made their first contribution in [#​10139](celery/celery#10139) - [@​cuiweixie](https://github.com/cuiweixie) made their first contribution in [#​10133](celery/celery#10133) - [@​kelsonbrito50](https://github.com/kelsonbrito50) made their first contribution in [#​10145](celery/celery#10145) - [@​Br1an67](https://github.com/Br1an67) made their first contribution in [#​10155](celery/celery#10155) - [@​daniel7an](https://github.com/daniel7an) made their first contribution in [#​10103](celery/celery#10103) - [@​drichardson](https://github.com/drichardson) made their first contribution in [#​10165](celery/celery#10165) - [@​tsangwailam](https://github.com/tsangwailam) made their first contribution in [#​10181](celery/celery#10181) - [@​Borzik](https://github.com/Borzik) made their first contribution in [#​10171](celery/celery#10171) - [@​aurangzaib048](https://github.com/aurangzaib048) made their first contribution in [#​10159](celery/celery#10159) - [@​ericbuehl](https://github.com/ericbuehl) made their first contribution in [#​10187](celery/celery#10187) - [@​KianAnbarestani](https://github.com/KianAnbarestani) made their first contribution in [#​10192](celery/celery#10192) - [@​patri27826](https://github.com/patri27826) made their first contribution in [#​10184](celery/celery#10184) **Full Changelog**: <celery/celery@v5.6.2...v5.6.3> </details> --- - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My45OS4wIiwidXBkYXRlZEluVmVyIjoiNDMuOTkuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiZGVwZW5kZW5jaWVzIiwicmVub3ZhdGUiXX0=--> See merge request swiss-armed-forces/cyber-command/cea/loom!419
Bumps [celery](https://github.com/celery/celery) from 5.6.2 to 5.6.3. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/celery/celery/releases">celery's releases</a>.</em></p> <blockquote> <h2>v5.6.3</h2> <h2>What's Changed</h2> <ul> <li>Fix Django worker recursion bug + defensive checks for pool_cls.<strong>module</strong> by <a href="https://github.com/maycuatroi1"><code>@maycuatroi1</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10048">celery/celery#10048</a></li> <li>Docs: Update user_preload_options example to use click. by <a href="https://github.com/jorsyk"><code>@jorsyk</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10056">celery/celery#10056</a></li> <li>Fix invalid configuration key "bootstrap_servers" in Kafka demo by <a href="https://github.com/jorsyk"><code>@jorsyk</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10060">celery/celery#10060</a></li> <li>Fix broken images on PyPI page by <a href="https://github.com/Timour-Ilyas"><code>@Timour-Ilyas</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10066">celery/celery#10066</a></li> <li>Remove broken reference. by <a href="https://github.com/sueannioanis"><code>@sueannioanis</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10071">celery/celery#10071</a></li> <li>Removed --dist=loadscope from smoke tests by <a href="https://github.com/Nusnus"><code>@Nusnus</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10073">celery/celery#10073</a></li> <li>Docs: Clarify task_retry signal args may be None by <a href="https://github.com/GangEunzzang"><code>@GangEunzzang</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10076">celery/celery#10076</a></li> <li>Update example for Django by <a href="https://github.com/sbc-khacnha"><code>@sbc-khacnha</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10081">celery/celery#10081</a></li> <li>Make tests compatible with pymongo >= 4.16 by <a href="https://github.com/cjwatson"><code>@cjwatson</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10074">celery/celery#10074</a></li> <li>fix: source install of cassandra-driver by <a href="https://github.com/Izzette"><code>@Izzette</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10105">celery/celery#10105</a></li> <li>fix: register task cross-reference role in Sphinx extension by <a href="https://github.com/veeceey"><code>@veeceey</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10100">celery/celery#10100</a></li> <li>fix: avoid cycle detection in native delayed delivery by <a href="https://github.com/Izzette"><code>@Izzette</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10095">celery/celery#10095</a></li> <li>fix(asynpool): avoid AttributeError when proc lacks _sentinel_poll by <a href="https://github.com/mriddle"><code>@mriddle</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10086">celery/celery#10086</a></li> <li>fix dusk_astronomical horizon sign (+18 -> -18) by <a href="https://github.com/Mr-Neutr0n"><code>@Mr-Neutr0n</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10121">celery/celery#10121</a></li> <li>Fix/10106 onupdate col use lambda func by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10108">celery/celery#10108</a></li> <li>Fix warm shutdown RuntimeError with eventlet>=0.37.0 (<a href="https://redirect.github.com/celery/celery/issues/10083">#10083</a>) by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10123">celery/celery#10123</a></li> <li>Fix 10109 db backend connection health by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10124">celery/celery#10124</a></li> <li>Database Backend filter unsupport sql engine arguments with nullpool <a href="https://redirect.github.com/celery/celery/issues/7355">#7355</a> by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10134">celery/celery#10134</a></li> <li>fix(beat): correct argument order in Service.<strong>reduce</strong> by <a href="https://github.com/bysiber"><code>@bysiber</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10137">celery/celery#10137</a></li> <li>ci: declare explicit read-only token permissions in workflow jobs by <a href="https://github.com/Rohan5commit"><code>@Rohan5commit</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10139">celery/celery#10139</a></li> <li>chore: 'boto3to' to 'boto3 to' by <a href="https://github.com/cuiweixie"><code>@cuiweixie</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10133">celery/celery#10133</a></li> <li>Database Backend: Add missing index on date_done (Fixes <a href="https://redirect.github.com/celery/celery/issues/10097">#10097</a>) by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10098">celery/celery#10098</a></li> <li>docs: fix typo in CONTRIBUTING.rst by <a href="https://github.com/Rohan5commit"><code>@Rohan5commit</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10141">celery/celery#10141</a></li> <li>Refer to Flower / Prometheus for monitoring by <a href="https://github.com/WilliamDEdwards"><code>@WilliamDEdwards</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10140">celery/celery#10140</a></li> <li>docs: remove duplicated words in broker and routing docs by <a href="https://github.com/Rohan5commit"><code>@Rohan5commit</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10146">celery/celery#10146</a></li> <li>docs: fix stale version reference and grammar in README by <a href="https://github.com/kelsonbrito50"><code>@kelsonbrito50</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10145">celery/celery#10145</a></li> <li>docs: fix wording in Celery 5.3 worker pool notes by <a href="https://github.com/Rohan5commit"><code>@Rohan5commit</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10149">celery/celery#10149</a></li> <li>docs: fix duplicated wording in 3.1 changelog entry by <a href="https://github.com/Rohan5commit"><code>@Rohan5commit</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10152">celery/celery#10152</a></li> <li>docs: fix changelog typo in context manager wording by <a href="https://github.com/Rohan5commit"><code>@Rohan5commit</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10144">celery/celery#10144</a></li> <li>Fix/10096 worker fails to reconnect after redis failover by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10151">celery/celery#10151</a></li> <li>Improve on_after_finalize signal documentation by <a href="https://github.com/Br1an67"><code>@Br1an67</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10155">celery/celery#10155</a></li> <li>Add non-commutative example to clarify partial arg ordering in canvas docs by <a href="https://github.com/Br1an67"><code>@Br1an67</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10157">celery/celery#10157</a></li> <li>Remove redundant test_isa_mapping test (fixes <a href="https://redirect.github.com/celery/celery/issues/10077">#10077</a>) by <a href="https://github.com/daniel7an"><code>@daniel7an</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10103">celery/celery#10103</a></li> <li>Upgrade pytest-celery to >=1.3.0 and adopt PYTEST_CELERY_PKG build arg by <a href="https://github.com/Nusnus"><code>@Nusnus</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10162">celery/celery#10162</a></li> <li>Remove deprecated args from redis get_connection call by <a href="https://github.com/JaeHyuckSa"><code>@JaeHyuckSa</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10036">celery/celery#10036</a></li> <li>Fix <a href="https://redirect.github.com/celery/celery/issues/6912">#6912</a> rpc backend reconnection error by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10179">celery/celery#10179</a></li> <li>Fix NameError with TYPE_CHECKING annotations on Python 3.14+ (PEP 649) by <a href="https://github.com/drichardson"><code>@drichardson</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10165">celery/celery#10165</a></li> <li>docs: Add elaboration on prefetch multiplier settings (worker_prefetch_multiplier) and worker_eta_task_limit by <a href="https://github.com/tsangwailam"><code>@tsangwailam</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10181">celery/celery#10181</a></li> <li>Fix O(K²) message bloat in a chain of chords by <a href="https://github.com/Borzik"><code>@Borzik</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10171">celery/celery#10171</a></li> <li>Fix mock connection interfaces to prevent <code>TypeError</code> during exception handling by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10178">celery/celery#10178</a></li> <li>fix(trace): dispatch chain/callbacks on dedup fast-path for redelivered tasks by <a href="https://github.com/aurangzaib048"><code>@aurangzaib048</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10159">celery/celery#10159</a></li> <li>Extract <code>reconnect_on_error</code> to <code>BaseResultConsumer</code> by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10189">celery/celery#10189</a></li> <li>pep 649 by <a href="https://github.com/ericbuehl"><code>@ericbuehl</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10187">celery/celery#10187</a></li> <li>Fix#9722 friendly status errors for CLI by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10190">celery/celery#10190</a></li> <li>docs: clarify after_return behavior for retried tasks by <a href="https://github.com/KianAnbarestani"><code>@KianAnbarestani</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10192">celery/celery#10192</a></li> <li>Add compression header to message protocol docs by <a href="https://github.com/Br1an67"><code>@Br1an67</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10156">celery/celery#10156</a></li> <li>docs: fix duplicated word in bootsteps comment by <a href="https://github.com/Rohan5commit"><code>@Rohan5commit</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10153">celery/celery#10153</a></li> <li>Remove outdated autoreloader section from extending docs by <a href="https://github.com/Br1an67"><code>@Br1an67</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10154">celery/celery#10154</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/celery/celery/blob/v5.6.3/Changelog.rst">celery's changelog</a>.</em></p> <blockquote> <h1>5.6.3</h1> <p>:release-date: 2026-03-26 :release-by: Tomer Nosrati</p> <p>What's Changed</p> <pre><code> - Fix Django worker recursion bug + defensive checks for pool_cls.__module__ ([#10048](celery/celery#10048)) - Docs: Update user_preload_options example to use click. ([#10056](celery/celery#10056)) - Fix invalid configuration key "bootstrap_servers" in Kafka demo ([#10060](celery/celery#10060)) - Fix broken images on PyPI page ([#10066](celery/celery#10066)) - Remove broken reference. ([#10071](celery/celery#10071)) - Removed --dist=loadscope from smoke tests ([#10073](celery/celery#10073)) - Docs: Clarify task_retry signal args may be None ([#10076](celery/celery#10076)) - Update example for Django ([#10081](celery/celery#10081)) - Make tests compatible with pymongo >= 4.16 ([#10074](celery/celery#10074)) - fix: source install of cassandra-driver ([#10105](celery/celery#10105)) - fix: register task cross-reference role in Sphinx extension ([#10100](celery/celery#10100)) - fix: avoid cycle detection in native delayed delivery ([#10095](celery/celery#10095)) - fix(asynpool): avoid AttributeError when proc lacks _sentinel_poll ([#10086](celery/celery#10086)) - fix dusk_astronomical horizon sign (+18 -> -18) ([#10121](celery/celery#10121)) - Fix/10106 onupdate col use lambda func ([#10108](celery/celery#10108)) - Fix warm shutdown RuntimeError with eventlet>=0.37.0 ([#10083](celery/celery#10083)) ([#10123](celery/celery#10123)) - Fix 10109 db backend connection health ([#10124](celery/celery#10124)) - Database Backend filter unsupport sql engine arguments with nullpool [#7355](celery/celery#7355) ([#10134](celery/celery#10134)) - fix(beat): correct argument order in Service.__reduce__ ([#10137](celery/celery#10137)) - ci: declare explicit read-only token permissions in workflow jobs ([#10139](celery/celery#10139)) - chore: 'boto3to' to 'boto3 to' ([#10133](celery/celery#10133)) - Database Backend: Add missing index on date_done (Fixes [#10097](celery/celery#10097)) ([#10098](celery/celery#10098)) - docs: fix typo in CONTRIBUTING.rst ([#10141](celery/celery#10141)) - Refer to Flower / Prometheus for monitoring ([#10140](celery/celery#10140)) - docs: remove duplicated words in broker and routing docs ([#10146](celery/celery#10146)) - docs: fix stale version reference and grammar in README ([#10145](celery/celery#10145)) - docs: fix wording in Celery 5.3 worker pool notes ([#10149](celery/celery#10149)) - docs: fix duplicated wording in 3.1 changelog entry ([#10152](celery/celery#10152)) - docs: fix changelog typo in context manager wording ([#10144](celery/celery#10144)) - Fix/10096 worker fails to reconnect after redis failover ([#10151](celery/celery#10151)) - Improve on_after_finalize signal documentation ([#10155](celery/celery#10155)) - Add non-commutative example to clarify partial arg ordering in canvas docs ([#10157](celery/celery#10157)) - Remove redundant test_isa_mapping test (fixes [#10077](celery/celery#10077)) ([#10103](celery/celery#10103)) - Upgrade pytest-celery to >=1.3.0 and adopt PYTEST_CELERY_PKG build arg ([#10162](celery/celery#10162)) - Remove deprecated args from redis get_connection call ([#10036](celery/celery#10036)) - Fix [#6912](celery/celery#6912) rpc backend reconnection error ([#10179](celery/celery#10179)) - Fix NameError with TYPE_CHECKING annotations on Python 3.14+ (PEP 649) ([#10165](celery/celery#10165)) - docs: Add elaboration on prefetch multiplier settings (worker_prefetch_multiplier) and worker_eta_task_limit ([#10181](celery/celery#10181)) - Fix O(K²) message bloat in a chain of chords ([#10171](celery/celery#10171)) - Fix mock connection interfaces to prevent `TypeError` during exception handling ([#10178](celery/celery#10178)) - fix(trace): dispatch chain/callbacks on dedup fast-path for redelivered tasks ([#10159](celery/celery#10159)) </tr></table> </code></pre> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/celery/celery/commit/3f4d8d795ad128bd7430cc5dc174a802cded425c"><code>3f4d8d7</code></a> Prepare for release: v5.6.3 (<a href="https://redirect.github.com/celery/celery/issues/10221">#10221</a>)</li> <li><a href="https://github.com/celery/celery/commit/a989e8cf8876274b1f0612abffeeb2e9995ed321"><code>a989e8c</code></a> fix: clear the timer while catch the exception (<a href="https://redirect.github.com/celery/celery/issues/10218">#10218</a>)</li> <li><a href="https://github.com/celery/celery/commit/d06de5f047620b0ea2bdbdb3c0c56137b79ae9a1"><code>d06de5f</code></a> Chore(deps): Bump nick-fields/retry from 3 to 4 (<a href="https://redirect.github.com/celery/celery/issues/10213">#10213</a>)</li> <li><a href="https://github.com/celery/celery/commit/c3c19c31dc3e21f16d4d85a8ba8401a9223ace09"><code>c3c19c3</code></a> Fix: prioritize request ignore_result over task definition (<a href="https://redirect.github.com/celery/celery/issues/10184">#10184</a>)</li> <li><a href="https://github.com/celery/celery/commit/d23be53f6f3600d48df35a797c63eb1c7d4d4b97"><code>d23be53</code></a> Remove outdated autoreloader section from extending docs (<a href="https://redirect.github.com/celery/celery/issues/10154">#10154</a>)</li> <li><a href="https://github.com/celery/celery/commit/ada2da7475a5fa9f9ad079149a5d6864634abc28"><code>ada2da7</code></a> docs: fix duplicated word in bootsteps comment\n\nSigned-off-by: Rohan Santho...</li> <li><a href="https://github.com/celery/celery/commit/f45f62beb3b16ae960944f8c97de13ccf15f2d0a"><code>f45f62b</code></a> Add compression header to message protocol docs (<a href="https://redirect.github.com/celery/celery/issues/10156">#10156</a>)</li> <li><a href="https://github.com/celery/celery/commit/9a270925546ed9d0ca0303fb5006edc86b705fd9"><code>9a27092</code></a> docs: clarify after_return behavior for retried tasks (<a href="https://redirect.github.com/celery/celery/issues/10192">#10192</a>)</li> <li><a href="https://github.com/celery/celery/commit/6ee6230cd80ef6c3e7482e1f4cd970fbb0629b23"><code>6ee6230</code></a> Fix#9722 friendly status errors for CLI (<a href="https://redirect.github.com/celery/celery/issues/10190">#10190</a>)</li> <li><a href="https://github.com/celery/celery/commit/a9a2d4cecaf0e58c401ad6f68f022afa19770ac2"><code>a9a2d4c</code></a> [pre-commit.ci] pre-commit autoupdate (<a href="https://redirect.github.com/celery/celery/issues/10186">#10186</a>)</li> <li>Additional commits viewable in <a href="https://github.com/celery/celery/compare/v5.6.2...v5.6.3">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Updates the requirements on [celery](https://github.com/celery/celery) to permit the latest version. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/celery/celery/releases">celery's releases</a>.</em></p> <blockquote> <h2>v5.6.3</h2> <h2>What's Changed</h2> <ul> <li>Fix Django worker recursion bug + defensive checks for pool_cls.<strong>module</strong> by <a href="https://github.com/maycuatroi1"><code>@maycuatroi1</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10048">celery/celery#10048</a></li> <li>Docs: Update user_preload_options example to use click. by <a href="https://github.com/jorsyk"><code>@jorsyk</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10056">celery/celery#10056</a></li> <li>Fix invalid configuration key "bootstrap_servers" in Kafka demo by <a href="https://github.com/jorsyk"><code>@jorsyk</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10060">celery/celery#10060</a></li> <li>Fix broken images on PyPI page by <a href="https://github.com/Timour-Ilyas"><code>@Timour-Ilyas</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10066">celery/celery#10066</a></li> <li>Remove broken reference. by <a href="https://github.com/sueannioanis"><code>@sueannioanis</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10071">celery/celery#10071</a></li> <li>Removed --dist=loadscope from smoke tests by <a href="https://github.com/Nusnus"><code>@Nusnus</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10073">celery/celery#10073</a></li> <li>Docs: Clarify task_retry signal args may be None by <a href="https://github.com/GangEunzzang"><code>@GangEunzzang</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10076">celery/celery#10076</a></li> <li>Update example for Django by <a href="https://github.com/sbc-khacnha"><code>@sbc-khacnha</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10081">celery/celery#10081</a></li> <li>Make tests compatible with pymongo >= 4.16 by <a href="https://github.com/cjwatson"><code>@cjwatson</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10074">celery/celery#10074</a></li> <li>fix: source install of cassandra-driver by <a href="https://github.com/Izzette"><code>@Izzette</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10105">celery/celery#10105</a></li> <li>fix: register task cross-reference role in Sphinx extension by <a href="https://github.com/veeceey"><code>@veeceey</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10100">celery/celery#10100</a></li> <li>fix: avoid cycle detection in native delayed delivery by <a href="https://github.com/Izzette"><code>@Izzette</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10095">celery/celery#10095</a></li> <li>fix(asynpool): avoid AttributeError when proc lacks _sentinel_poll by <a href="https://github.com/mriddle"><code>@mriddle</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10086">celery/celery#10086</a></li> <li>fix dusk_astronomical horizon sign (+18 -> -18) by <a href="https://github.com/Mr-Neutr0n"><code>@Mr-Neutr0n</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10121">celery/celery#10121</a></li> <li>Fix/10106 onupdate col use lambda func by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10108">celery/celery#10108</a></li> <li>Fix warm shutdown RuntimeError with eventlet>=0.37.0 (<a href="https://redirect.github.com/celery/celery/issues/10083">#10083</a>) by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10123">celery/celery#10123</a></li> <li>Fix 10109 db backend connection health by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10124">celery/celery#10124</a></li> <li>Database Backend filter unsupport sql engine arguments with nullpool <a href="https://redirect.github.com/celery/celery/issues/7355">#7355</a> by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10134">celery/celery#10134</a></li> <li>fix(beat): correct argument order in Service.<strong>reduce</strong> by <a href="https://github.com/bysiber"><code>@bysiber</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10137">celery/celery#10137</a></li> <li>ci: declare explicit read-only token permissions in workflow jobs by <a href="https://github.com/Rohan5commit"><code>@Rohan5commit</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10139">celery/celery#10139</a></li> <li>chore: 'boto3to' to 'boto3 to' by <a href="https://github.com/cuiweixie"><code>@cuiweixie</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10133">celery/celery#10133</a></li> <li>Database Backend: Add missing index on date_done (Fixes <a href="https://redirect.github.com/celery/celery/issues/10097">#10097</a>) by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10098">celery/celery#10098</a></li> <li>docs: fix typo in CONTRIBUTING.rst by <a href="https://github.com/Rohan5commit"><code>@Rohan5commit</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10141">celery/celery#10141</a></li> <li>Refer to Flower / Prometheus for monitoring by <a href="https://github.com/WilliamDEdwards"><code>@WilliamDEdwards</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10140">celery/celery#10140</a></li> <li>docs: remove duplicated words in broker and routing docs by <a href="https://github.com/Rohan5commit"><code>@Rohan5commit</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10146">celery/celery#10146</a></li> <li>docs: fix stale version reference and grammar in README by <a href="https://github.com/kelsonbrito50"><code>@kelsonbrito50</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10145">celery/celery#10145</a></li> <li>docs: fix wording in Celery 5.3 worker pool notes by <a href="https://github.com/Rohan5commit"><code>@Rohan5commit</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10149">celery/celery#10149</a></li> <li>docs: fix duplicated wording in 3.1 changelog entry by <a href="https://github.com/Rohan5commit"><code>@Rohan5commit</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10152">celery/celery#10152</a></li> <li>docs: fix changelog typo in context manager wording by <a href="https://github.com/Rohan5commit"><code>@Rohan5commit</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10144">celery/celery#10144</a></li> <li>Fix/10096 worker fails to reconnect after redis failover by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10151">celery/celery#10151</a></li> <li>Improve on_after_finalize signal documentation by <a href="https://github.com/Br1an67"><code>@Br1an67</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10155">celery/celery#10155</a></li> <li>Add non-commutative example to clarify partial arg ordering in canvas docs by <a href="https://github.com/Br1an67"><code>@Br1an67</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10157">celery/celery#10157</a></li> <li>Remove redundant test_isa_mapping test (fixes <a href="https://redirect.github.com/celery/celery/issues/10077">#10077</a>) by <a href="https://github.com/daniel7an"><code>@daniel7an</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10103">celery/celery#10103</a></li> <li>Upgrade pytest-celery to >=1.3.0 and adopt PYTEST_CELERY_PKG build arg by <a href="https://github.com/Nusnus"><code>@Nusnus</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10162">celery/celery#10162</a></li> <li>Remove deprecated args from redis get_connection call by <a href="https://github.com/JaeHyuckSa"><code>@JaeHyuckSa</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10036">celery/celery#10036</a></li> <li>Fix <a href="https://redirect.github.com/celery/celery/issues/6912">#6912</a> rpc backend reconnection error by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10179">celery/celery#10179</a></li> <li>Fix NameError with TYPE_CHECKING annotations on Python 3.14+ (PEP 649) by <a href="https://github.com/drichardson"><code>@drichardson</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10165">celery/celery#10165</a></li> <li>docs: Add elaboration on prefetch multiplier settings (worker_prefetch_multiplier) and worker_eta_task_limit by <a href="https://github.com/tsangwailam"><code>@tsangwailam</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10181">celery/celery#10181</a></li> <li>Fix O(K²) message bloat in a chain of chords by <a href="https://github.com/Borzik"><code>@Borzik</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10171">celery/celery#10171</a></li> <li>Fix mock connection interfaces to prevent <code>TypeError</code> during exception handling by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10178">celery/celery#10178</a></li> <li>fix(trace): dispatch chain/callbacks on dedup fast-path for redelivered tasks by <a href="https://github.com/aurangzaib048"><code>@aurangzaib048</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10159">celery/celery#10159</a></li> <li>Extract <code>reconnect_on_error</code> to <code>BaseResultConsumer</code> by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10189">celery/celery#10189</a></li> <li>pep 649 by <a href="https://github.com/ericbuehl"><code>@ericbuehl</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10187">celery/celery#10187</a></li> <li>Fix#9722 friendly status errors for CLI by <a href="https://github.com/ChickenBenny"><code>@ChickenBenny</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10190">celery/celery#10190</a></li> <li>docs: clarify after_return behavior for retried tasks by <a href="https://github.com/KianAnbarestani"><code>@KianAnbarestani</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10192">celery/celery#10192</a></li> <li>Add compression header to message protocol docs by <a href="https://github.com/Br1an67"><code>@Br1an67</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10156">celery/celery#10156</a></li> <li>docs: fix duplicated word in bootsteps comment by <a href="https://github.com/Rohan5commit"><code>@Rohan5commit</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10153">celery/celery#10153</a></li> <li>Remove outdated autoreloader section from extending docs by <a href="https://github.com/Br1an67"><code>@Br1an67</code></a> in <a href="https://redirect.github.com/celery/celery/pull/10154">celery/celery#10154</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/celery/celery/blob/v5.6.3/Changelog.rst">celery's changelog</a>.</em></p> <blockquote> <h1>5.6.3</h1> <p>:release-date: 2026-03-26 :release-by: Tomer Nosrati</p> <p>What's Changed</p> <pre><code> - Fix Django worker recursion bug + defensive checks for pool_cls.__module__ ([#10048](celery/celery#10048)) - Docs: Update user_preload_options example to use click. ([#10056](celery/celery#10056)) - Fix invalid configuration key "bootstrap_servers" in Kafka demo ([#10060](celery/celery#10060)) - Fix broken images on PyPI page ([#10066](celery/celery#10066)) - Remove broken reference. ([#10071](celery/celery#10071)) - Removed --dist=loadscope from smoke tests ([#10073](celery/celery#10073)) - Docs: Clarify task_retry signal args may be None ([#10076](celery/celery#10076)) - Update example for Django ([#10081](celery/celery#10081)) - Make tests compatible with pymongo >= 4.16 ([#10074](celery/celery#10074)) - fix: source install of cassandra-driver ([#10105](celery/celery#10105)) - fix: register task cross-reference role in Sphinx extension ([#10100](celery/celery#10100)) - fix: avoid cycle detection in native delayed delivery ([#10095](celery/celery#10095)) - fix(asynpool): avoid AttributeError when proc lacks _sentinel_poll ([#10086](celery/celery#10086)) - fix dusk_astronomical horizon sign (+18 -> -18) ([#10121](celery/celery#10121)) - Fix/10106 onupdate col use lambda func ([#10108](celery/celery#10108)) - Fix warm shutdown RuntimeError with eventlet>=0.37.0 ([#10083](celery/celery#10083)) ([#10123](celery/celery#10123)) - Fix 10109 db backend connection health ([#10124](celery/celery#10124)) - Database Backend filter unsupport sql engine arguments with nullpool [#7355](celery/celery#7355) ([#10134](celery/celery#10134)) - fix(beat): correct argument order in Service.__reduce__ ([#10137](celery/celery#10137)) - ci: declare explicit read-only token permissions in workflow jobs ([#10139](celery/celery#10139)) - chore: 'boto3to' to 'boto3 to' ([#10133](celery/celery#10133)) - Database Backend: Add missing index on date_done (Fixes [#10097](celery/celery#10097)) ([#10098](celery/celery#10098)) - docs: fix typo in CONTRIBUTING.rst ([#10141](celery/celery#10141)) - Refer to Flower / Prometheus for monitoring ([#10140](celery/celery#10140)) - docs: remove duplicated words in broker and routing docs ([#10146](celery/celery#10146)) - docs: fix stale version reference and grammar in README ([#10145](celery/celery#10145)) - docs: fix wording in Celery 5.3 worker pool notes ([#10149](celery/celery#10149)) - docs: fix duplicated wording in 3.1 changelog entry ([#10152](celery/celery#10152)) - docs: fix changelog typo in context manager wording ([#10144](celery/celery#10144)) - Fix/10096 worker fails to reconnect after redis failover ([#10151](celery/celery#10151)) - Improve on_after_finalize signal documentation ([#10155](celery/celery#10155)) - Add non-commutative example to clarify partial arg ordering in canvas docs ([#10157](celery/celery#10157)) - Remove redundant test_isa_mapping test (fixes [#10077](celery/celery#10077)) ([#10103](celery/celery#10103)) - Upgrade pytest-celery to >=1.3.0 and adopt PYTEST_CELERY_PKG build arg ([#10162](celery/celery#10162)) - Remove deprecated args from redis get_connection call ([#10036](celery/celery#10036)) - Fix [#6912](celery/celery#6912) rpc backend reconnection error ([#10179](celery/celery#10179)) - Fix NameError with TYPE_CHECKING annotations on Python 3.14+ (PEP 649) ([#10165](celery/celery#10165)) - docs: Add elaboration on prefetch multiplier settings (worker_prefetch_multiplier) and worker_eta_task_limit ([#10181](celery/celery#10181)) - Fix O(K²) message bloat in a chain of chords ([#10171](celery/celery#10171)) - Fix mock connection interfaces to prevent `TypeError` during exception handling ([#10178](celery/celery#10178)) - fix(trace): dispatch chain/callbacks on dedup fast-path for redelivered tasks ([#10159](celery/celery#10159)) </tr></table> </code></pre> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/celery/celery/commit/3f4d8d795ad128bd7430cc5dc174a802cded425c"><code>3f4d8d7</code></a> Prepare for release: v5.6.3 (<a href="https://redirect.github.com/celery/celery/issues/10221">#10221</a>)</li> <li><a href="https://github.com/celery/celery/commit/a989e8cf8876274b1f0612abffeeb2e9995ed321"><code>a989e8c</code></a> fix: clear the timer while catch the exception (<a href="https://redirect.github.com/celery/celery/issues/10218">#10218</a>)</li> <li><a href="https://github.com/celery/celery/commit/d06de5f047620b0ea2bdbdb3c0c56137b79ae9a1"><code>d06de5f</code></a> Chore(deps): Bump nick-fields/retry from 3 to 4 (<a href="https://redirect.github.com/celery/celery/issues/10213">#10213</a>)</li> <li><a href="https://github.com/celery/celery/commit/c3c19c31dc3e21f16d4d85a8ba8401a9223ace09"><code>c3c19c3</code></a> Fix: prioritize request ignore_result over task definition (<a href="https://redirect.github.com/celery/celery/issues/10184">#10184</a>)</li> <li><a href="https://github.com/celery/celery/commit/d23be53f6f3600d48df35a797c63eb1c7d4d4b97"><code>d23be53</code></a> Remove outdated autoreloader section from extending docs (<a href="https://redirect.github.com/celery/celery/issues/10154">#10154</a>)</li> <li><a href="https://github.com/celery/celery/commit/ada2da7475a5fa9f9ad079149a5d6864634abc28"><code>ada2da7</code></a> docs: fix duplicated word in bootsteps comment\n\nSigned-off-by: Rohan Santho...</li> <li><a href="https://github.com/celery/celery/commit/f45f62beb3b16ae960944f8c97de13ccf15f2d0a"><code>f45f62b</code></a> Add compression header to message protocol docs (<a href="https://redirect.github.com/celery/celery/issues/10156">#10156</a>)</li> <li><a href="https://github.com/celery/celery/commit/9a270925546ed9d0ca0303fb5006edc86b705fd9"><code>9a27092</code></a> docs: clarify after_return behavior for retried tasks (<a href="https://redirect.github.com/celery/celery/issues/10192">#10192</a>)</li> <li><a href="https://github.com/celery/celery/commit/6ee6230cd80ef6c3e7482e1f4cd970fbb0629b23"><code>6ee6230</code></a> Fix#9722 friendly status errors for CLI (<a href="https://redirect.github.com/celery/celery/issues/10190">#10190</a>)</li> <li><a href="https://github.com/celery/celery/commit/a9a2d4cecaf0e58c401ad6f68f022afa19770ac2"><code>a9a2d4c</code></a> [pre-commit.ci] pre-commit autoupdate (<a href="https://redirect.github.com/celery/celery/issues/10186">#10186</a>)</li> <li>Additional commits viewable in <a href="https://github.com/celery/celery/compare/v5.3.4...v5.6.3">compare view</a></li> </ul> </details> <br /> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details>
Avoid RabbitMQ <4.0.1 dead-lettering cycle detection when retrying with native delayed-delivery.
This is achieved by stripping the
x-death-related headers before publishing tasks.Fixes #9867.
When running with RabbitMQ 3.13 before applying the fix we are able to see a reproduction of the issue in the integration testing environment.
Some breadcrumbs have been added to display the message headers and properties for the task being published (retried).
We can see that after the task is published for retry with the
x-deathheaders, the task is not received by the worker and instead is lost.A timeout is reached before the result of the task execution is available.
Note that the "retry: Retry in 3s: TaskFailedException('Simulated failure')" is printed just after the task is published for retry
The test error output is added in the following comment to avoid exceeding the pull request description limit.
We only see
Dead-letter queues cycle detected for source quorum queuelogged once by RabbitMQ, as it will not log twice for the same cycle without being restarted.RabbitMQ logs
The issue is not reproducable with RabbitMQ ≥4.0.1, as the RabbitMQ server will not interpret
x-deathheaders being published by the client: see the release notes for RabbitMQ 4.0.1.I'm also not able to reproduce the issue using a delay of 1 second, it seems only having the last death be the same queue as the current death prevents the cycle detection from occuring. Instead of a new death that would be appended to the x-death header, but the count for the existing death is incremented. I'm not sure, but I suspect it's not possible to setup dead-lettering directly to the same queue in RabbitMQ, a different kind of cycle detection probably occurs in that case. Using a delay of 3 seconds results in the message being routed to
celery_delayed_1thancelery_delayed_0, after a retry it ends up back incelery_delayed_1where the cycle is detected by RabbitMQ.In order to run with RabbitMQ 3.13, this patch can be applied locally, and the docker container recreated:
Patch to use RabbitMQ 3.13
The instrumentation to see the message headers was done by applying this patch:
Patch to log message headers
Despite the fact that this issue doesn't occur with non-EoL versions of RabbitMQ, it still affects us. We have a shared RabbitMQ server used by several dozen microservices. Historically we've relied on classic queue mirroring to provide high availability message brokering. We're moving each critical use-case from mirrored classic queues to quorum queues, but the scope is too large to be done in a day. We're unable to upgrade to RabbitMQ ≥4.0.1 until all services have been migrated. This means we're stuck with the behavior of RabbitMQ 3.13 for the coming months. Given that quorum queues support for Celery is relatively new, I doubt we are the only ones in this situation.
The integration test was importent to ensure we could reproduce the issue and resolve it when making a fix. However, I'm not sure how useful it is to keep the integration test as the issue can not be reproduced with the RabbitMQ being used in the CI. I propose either:
Let me know which of these options you prefer.