[{"data":1,"prerenderedAt":38},["ShallowReactive",2],{"article-celery-tasks-on-rabbitmq-40-classic-queues-vs-quorum-queues-4bie":3},{"type_of":4,"id":5,"title":6,"description":7,"readable_publish_date":8,"slug":9,"path":10,"url":11,"comments_count":12,"public_reactions_count":12,"collection_id":13,"published_timestamp":14,"language":15,"subforem_id":16,"positive_reactions_count":12,"cover_image":17,"social_image":18,"canonical_url":11,"created_at":19,"edited_at":13,"crossposted_at":13,"published_at":14,"last_comment_at":14,"reading_time_minutes":20,"tag_list":21,"tags":22,"body_html":27,"body_markdown":28,"user":29},"article",3300983,"Celery Tasks on RabbitMQ 4.0: Classic Queues vs Quorum Queues","Celery Tasks on RabbitMQ 4.0: Classic Queues vs Quorum Queues   When building distributed...","Jul 13","celery-tasks-on-rabbitmq-40-classic-queues-vs-quorum-queues-4bie","\u002Fashraful\u002Fcelery-tasks-on-rabbitmq-40-classic-queues-vs-quorum-queues-4bie","https:\u002F\u002Fdev.to\u002Fashraful\u002Fcelery-tasks-on-rabbitmq-40-classic-queues-vs-quorum-queues-4bie",0,null,"2026-07-13T04:59:50Z","en",1,"https:\u002F\u002Fmedia2.dev.to\u002Fdynamic\u002Fimage\u002Fwidth=1000,height=420,fit=cover,gravity=auto,format=auto\u002Fhttps%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F7r8fgcs0vqqr7tt6w809.png","https:\u002F\u002Fmedia2.dev.to\u002Fdynamic\u002Fimage\u002Fwidth=1200,height=627,fit=cover,gravity=auto,format=auto\u002Fhttps%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F7r8fgcs0vqqr7tt6w809.png","2026-03-02T07:01:58Z",4,"celery, rabbitmq, python, webdev",[23,24,25,26],"celery","rabbitmq","python","webdev","\u003Ch1>\n  \u003Ca name=\"celery-tasks-on-rabbitmq-40-classic-queues-vs-quorum-queues\" href=\"#celery-tasks-on-rabbitmq-40-classic-queues-vs-quorum-queues\">\n  \u003C\u002Fa>\n  Celery Tasks on RabbitMQ 4.0: Classic Queues vs Quorum Queues\n\u003C\u002Fh1>\n\n\u003Cp>When building distributed systems with \u003Cstrong>Celery\u003C\u002Fstrong> and \u003Cstrong>RabbitMQ\u003C\u002Fstrong>, one of the most common pitfalls is \u003Cstrong>tasks disappearing\u003C\u002Fstrong>. This usually happens in clustered setups when using \u003Cstrong>classic queues\u003C\u002Fstrong>. RabbitMQ 4.x introduced some changes and new queue types, which affects Celery users in production. In this post, we'll explore:\u003C\u002Fp>\n\n\u003Cul>\n\u003Cli>How RabbitMQ cluster connections work\n\u003C\u002Fli>\n\u003Cli>Differences between classic and quorum queues\n\u003C\u002Fli>\n\u003Cli>Limitations of each queue type\n\u003C\u002Fli>\n\u003Cli>Why tasks sometimes disappear on classic queues in a 3-node cluster\n\u003C\u002Fli>\n\u003Cli>How to fix these issues\n\u003C\u002Fli>\n\u003C\u002Ful>\n\n\n\u003Chr>\n\n\u003Ch2>\n  \u003Ca name=\"rabbitmq-cluster-architecture-n2-1\" href=\"#rabbitmq-cluster-architecture-n2-1\">\n  \u003C\u002Fa>\n  RabbitMQ Cluster Architecture (n\u002F2 + 1)\n\u003C\u002Fh2>\n\n\u003Cp>A \u003Cstrong>RabbitMQ cluster\u003C\u002Fstrong> is a peer-to-peer network of nodes where each node can accept connections and coordinate with other nodes. In production, cluster operations often rely on a \u003Cstrong>majority quorum rule\u003C\u002Fstrong>:\u003Cbr>\n\u003C\u002Fp>\n\n\u003Cdiv class=\"highlight js-code-highlight\">\n\u003Cpre class=\"highlight plaintext\">\u003Ccode>Minimum nodes to operate = floor(n\u002F2) + 1\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cdiv class=\"highlight__panel js-actions-panel\">\n\u003Cdiv class=\"highlight__panel-action js-fullscreen-code-action\">\n    \u003Csvg xmlns=\"http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\">\u003Ctitle>Enter fullscreen mode\u003C\u002Ftitle>\n    \u003Cpath d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\">\u003C\u002Fpath>\n\u003C\u002Fsvg>\n\n    \u003Csvg xmlns=\"http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\">\u003Ctitle>Exit fullscreen mode\u003C\u002Ftitle>\n    \u003Cpath d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\">\u003C\u002Fpath>\n\u003C\u002Fsvg>\n\n\u003C\u002Fdiv>\n\u003C\u002Fdiv>\n\u003C\u002Fdiv>\n\n\n\n\u003Cp>For example, in a \u003Cstrong>3-node cluster\u003C\u002Fstrong>:\u003C\u002Fp>\n\n\u003Cul>\n\u003Cli>\n\u003Ccode>n = 3\u003C\u002Fcode>\n\u003C\u002Fli>\n\u003Cli>\n\u003Ccode>(3 \u002F 2) + 1 = 2\u003C\u002Fcode> nodes\n\u003C\u002Fli>\n\u003Cli>At least \u003Cstrong>2 nodes must be up and reachable\u003C\u002Fstrong> for the cluster to accept writes or form a majority.\u003C\u002Fli>\n\u003C\u002Ful>\n\n\u003Cp>This ensures \u003Cstrong>consistency and reliability\u003C\u002Fstrong>, especially for replicated queues.\u003C\u002Fp>\n\n\u003Ch3>\n  \u003Ca name=\"rabbitmq-3node-cluster-diagram\" href=\"#rabbitmq-3node-cluster-diagram\">\n  \u003C\u002Fa>\n  RabbitMQ 3-node cluster diagram\n\u003C\u002Fh3>\n\n\n\n\u003Cdiv class=\"highlight js-code-highlight\">\n\u003Cpre class=\"highlight plaintext\">\u003Ccode>        +------------+\n        | Node A     |\n        | (rabbitmq1)|\n        +------------+\n           |    \\\n           |     \\\n           |      \\\n        +------------+        +------------+\n        | Node B     |--------| Node C     |\n        | (rabbitmq2)|        | (rabbitmq3)|\n        +------------+        +------------+\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cdiv class=\"highlight__panel js-actions-panel\">\n\u003Cdiv class=\"highlight__panel-action js-fullscreen-code-action\">\n    \u003Csvg xmlns=\"http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\">\u003Ctitle>Enter fullscreen mode\u003C\u002Ftitle>\n    \u003Cpath d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\">\u003C\u002Fpath>\n\u003C\u002Fsvg>\n\n    \u003Csvg xmlns=\"http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\">\u003Ctitle>Exit fullscreen mode\u003C\u002Ftitle>\n    \u003Cpath d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\">\u003C\u002Fpath>\n\u003C\u002Fsvg>\n\n\u003C\u002Fdiv>\n\u003C\u002Fdiv>\n\u003C\u002Fdiv>\n\n\n\n\u003Cul>\n\u003Cli>Each node is a \u003Cstrong>disc node\u003C\u002Fstrong> (stores persistent queues).\n\u003C\u002Fli>\n\u003Cli>Messages in \u003Cstrong>classic queues\u003C\u002Fstrong> are stored on the node where the queue is declared.\n\u003C\u002Fli>\n\u003Cli>Cluster nodes maintain peer-to-peer links; \u003Ccode>(n\u002F2)+1\u003C\u002Fcode> nodes are needed to maintain consistency in operations like leader election.\u003C\u002Fli>\n\u003C\u002Ful>\n\n\n\u003Chr>\n\n\u003Ch2>\n  \u003Ca name=\"classic-queues-in-rabbitmq-4x\" href=\"#classic-queues-in-rabbitmq-4x\">\n  \u003C\u002Fa>\n  Classic Queues in RabbitMQ 4.x\n\u003C\u002Fh2>\n\n\u003Cp>Classic queues are the \u003Cstrong>original queue type\u003C\u002Fstrong> in RabbitMQ:\u003C\u002Fp>\n\n\u003Cul>\n\u003Cli>Messages are \u003Cstrong>stored on the node where the queue is created\u003C\u002Fstrong>\n\u003C\u002Fli>\n\u003Cli>By default, \u003Cstrong>they are not replicated\u003C\u002Fstrong> across nodes (unless mirrored, but mirrored queues are deprecated and removed from 4.x ((\u003Ca href=\"https:\u002F\u002Fwww.rabbitmq.com\u002Fdocs\u002F3.13\u002Fha)%5BRead\" target=\"_blank\" rel=\"noopener noreferrer\">https:\u002F\u002Fwww.rabbitmq.com\u002Fdocs\u002F3.13\u002Fha)[Read\u003C\u002Fa> more]))\n\u003C\u002Fli>\n\u003Cli>Suitable for \u003Cstrong>single-node setups\u003C\u002Fstrong> or non-critical workloads\u003C\u002Fli>\n\u003C\u002Ful>\n\n\u003Ch3>\n  \u003Ca name=\"what-changed-in-rabbitmq-40\" href=\"#what-changed-in-rabbitmq-40\">\n  \u003C\u002Fa>\n  What changed in RabbitMQ 4.0\n\u003C\u002Fh3>\n\n\u003Cul>\n\u003Cli>\n\u003Ccode>classic_mirrored_queue_version\u003C\u002Fcode> feature is \u003Cstrong>removed\u003C\u002Fstrong> in 4.0\n\n\u003Cul>\n\u003Cli>This means old mirrored queues are deprecated\n\u003C\u002Fli>\n\u003Cli>Classic queues are now \u003Cstrong>always node-local unless manually mirrored via legacy options\u003C\u002Fstrong>\n\u003C\u002Fli>\n\u003C\u002Ful>\n\u003C\u002Fli>\n\u003C\u002Ful>\n\n\u003Ch3>\n  \u003Ca name=\"why-tasks-disappear-in-a-classic-queue\" href=\"#why-tasks-disappear-in-a-classic-queue\">\n  \u003C\u002Fa>\n  Why tasks disappear in a classic queue\n\u003C\u002Fh3>\n\n\u003Cp>In a \u003Cstrong>3-node cluster\u003C\u002Fstrong>, consider a task published to a \u003Cstrong>classic queue\u003C\u002Fstrong>:\u003C\u002Fp>\n\n\u003Cul>\n\u003Cli>Queue lives on \u003Cstrong>Node A\u003C\u002Fstrong>\n\u003C\u002Fli>\n\u003Cli>Cluster requires \u003Ccode>(n\u002F2)+1 = 2\u003C\u002Fcode> nodes to operate correctly\u003C\u002Fli>\n\u003Cli>If Node A becomes temporarily unreachable (network glitch, maintenance), Celery can still publish tasks to Node B\u003C\u002Fli>\n\u003Cli>Node B cannot deliver the task because \u003Cstrong>queue master is on Node A\u003C\u002Fstrong>\n\u003C\u002Fli>\n\u003Cli>As a result, tasks \u003Cstrong>appear lost\u003C\u002Fstrong>, especially in short outages\u003C\u002Fli>\n\u003C\u002Ful>\n\n\u003Ch4>\n  \u003Ca name=\"example-scenario\" href=\"#example-scenario\">\n  \u003C\u002Fa>\n  Example scenario\n\u003C\u002Fh4>\n\n\n\n\u003Cdiv class=\"highlight js-code-highlight\">\n\u003Cpre class=\"highlight plaintext\">\u003Ccode>Producer publishes task -> Node B\nQueue master       -> Node A\nCluster majority    -> 2 nodes required (A + B)\nNode A unavailable for short time\nResult: Task cannot be delivered, disappears\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cdiv class=\"highlight__panel js-actions-panel\">\n\u003Cdiv class=\"highlight__panel-action js-fullscreen-code-action\">\n    \u003Csvg xmlns=\"http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\">\u003Ctitle>Enter fullscreen mode\u003C\u002Ftitle>\n    \u003Cpath d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\">\u003C\u002Fpath>\n\u003C\u002Fsvg>\n\n    \u003Csvg xmlns=\"http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\">\u003Ctitle>Exit fullscreen mode\u003C\u002Ftitle>\n    \u003Cpath d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\">\u003C\u002Fpath>\n\u003C\u002Fsvg>\n\n\u003C\u002Fdiv>\n\u003C\u002Fdiv>\n\u003C\u002Fdiv>\n\n\n\n\u003Cblockquote>\n\u003Cp>The main reason for disappearing tasks is that \u003Cstrong>classic queues exist on a single node\u003C\u002Fstrong>, while the cluster is enforcing majority \u003Ccode>(n\u002F2)+1\u003C\u002Fcode>. If the node hosting the queue is unreachable, even briefly, the task cannot be routed to the master.\u003C\u002Fp>\n\u003C\u002Fblockquote>\n\n\n\u003Chr>\n\n\u003Ch2>\n  \u003Ca name=\"quorum-queues\" href=\"#quorum-queues\">\n  \u003C\u002Fa>\n  Quorum Queues\n\u003C\u002Fh2>\n\n\u003Cp>\u003Cstrong>Quorum queues\u003C\u002Fstrong> were introduced to replace mirrored queues and solve the reliability problems of classic queues in clusters.\u003C\u002Fp>\n\n\u003Ch3>\n  \u003Ca name=\"how-quorum-queues-work\" href=\"#how-quorum-queues-work\">\n  \u003C\u002Fa>\n  How quorum queues work\n\u003C\u002Fh3>\n\n\u003Cul>\n\u003Cli>Messages are \u003Cstrong>replicated across multiple nodes\u003C\u002Fstrong>\n\u003C\u002Fli>\n\u003Cli>Writes require \u003Cstrong>majority acknowledgment\u003C\u002Fstrong> (consensus) to be considered successful\n\u003C\u002Fli>\n\u003Cli>Safe against \u003Cstrong>node failures\u003C\u002Fstrong>, because tasks exist on multiple nodes\n\u003C\u002Fli>\n\u003C\u002Ful>\n\n\u003Ch3>\n  \u003Ca name=\"quorum-queue-diagram-in-3node-cluster\" href=\"#quorum-queue-diagram-in-3node-cluster\">\n  \u003C\u002Fa>\n  Quorum queue diagram in 3-node cluster\n\u003C\u002Fh3>\n\n\n\n\u003Cdiv class=\"highlight js-code-highlight\">\n\u003Cpre class=\"highlight plaintext\">\u003Ccode>             +----------------------+\n             |      Task Producer    |\n             +----------+-----------+\n                        |\n                        v\n             +----------------------+\n             | Quorum Queue Leader   |\n             |       Node A         |\n             +----------+-----------+\n                        |\n           -------------------------------\n           |                             |\n           v                             v\n+----------------------+       +----------------------+\n| Quorum Queue Follower |       | Quorum Queue Follower |\n|       Node B          |       |       Node C          |\n+----------------------+       +----------------------+\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cdiv class=\"highlight__panel js-actions-panel\">\n\u003Cdiv class=\"highlight__panel-action js-fullscreen-code-action\">\n    \u003Csvg xmlns=\"http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\">\u003Ctitle>Enter fullscreen mode\u003C\u002Ftitle>\n    \u003Cpath d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\">\u003C\u002Fpath>\n\u003C\u002Fsvg>\n\n    \u003Csvg xmlns=\"http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\">\u003Ctitle>Exit fullscreen mode\u003C\u002Ftitle>\n    \u003Cpath d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\">\u003C\u002Fpath>\n\u003C\u002Fsvg>\n\n\u003C\u002Fdiv>\n\u003C\u002Fdiv>\n\u003C\u002Fdiv>\n\n\n\n\u003Cul>\n\u003Cli>A task is only removed when \u003Cstrong>leader + majority followers confirm\u003C\u002Fstrong>\n\u003C\u002Fli>\n\u003Cli>Even if Node A goes down, Node B or C can continue serving the queue\u003C\u002Fli>\n\u003C\u002Ful>\n\n\n\u003Chr>\n\n\u003Ch3>\n  \u003Ca name=\"limitations-of-quorum-queues\" href=\"#limitations-of-quorum-queues\">\n  \u003C\u002Fa>\n  Limitations of quorum queues\n\u003C\u002Fh3>\n\n\u003Cdiv class=\"table-wrapper-paragraph\">\u003Ctable>\n\u003Cthead>\n\u003Ctr>\n\u003Cth>Limitation\u003C\u002Fth>\n\u003Cth>Explanation\u003C\u002Fth>\n\u003Cth>How to mitigate\u003C\u002Fth>\n\u003C\u002Ftr>\n\u003C\u002Fthead>\n\u003Ctbody>\n\u003Ctr>\n\u003Ctd>\u003Cstrong>Slower than classic queues\u003C\u002Fstrong>\u003C\u002Ftd>\n\u003Ctd>Each write requires majority acknowledgment\u003C\u002Ftd>\n\u003Ctd>Use quorum queues only for critical tasks\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\u003Cstrong>No priorities\u003C\u002Fstrong>\u003C\u002Ftd>\n\u003Ctd>Quorum queues do not support task priorities\u003C\u002Ftd>\n\u003Ctd>Use separate queues for priority handling\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\u003Cstrong>Memory pressure\u003C\u002Fstrong>\u003C\u002Ftd>\n\u003Ctd>Replication consumes memory on all nodes\u003C\u002Ftd>\n\u003Ctd>Monitor memory and adjust \u003Ccode>ram nodes\u003C\u002Fcode> if needed\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\u003Cstrong>Prefetching\u003C\u002Fstrong>\u003C\u002Ftd>\n\u003Ctd>Workers must fetch 1 task at a time for reliability\u003C\u002Ftd>\n\u003Ctd>\n\u003Ccode>worker_prefetch_multiplier = 1\u003C\u002Fcode> in Celery\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftbody>\n\u003C\u002Ftable>\u003C\u002Fdiv>\n\n\n\u003Chr>\n\n\u003Ch3>\n  \u003Ca name=\"celery-configuration-for-quorum-queues\" href=\"#celery-configuration-for-quorum-queues\">\n  \u003C\u002Fa>\n  Celery configuration for quorum queues\n\u003C\u002Fh3>\n\n\n\n\u003Cdiv class=\"highlight js-code-highlight\">\n\u003Cpre class=\"highlight python\">\u003Ccode>\u003Cspan class=\"n\">broker_url\u003C\u002Fspan> \u003Cspan class=\"o\">=\u003C\u002Fspan> \u003Cspan class=\"sh\">\"\u003C\u002Fspan>\u003Cspan class=\"s\">amqp:\u002F\u002Fadmin:admin@rabbitmq1:5672\u002F\u002F\u003C\u002Fspan>\u003Cspan class=\"sh\">\"\u003C\u002Fspan>\n\n\u003Cspan class=\"n\">task_default_queue\u003C\u002Fspan> \u003Cspan class=\"o\">=\u003C\u002Fspan> \u003Cspan class=\"sh\">\"\u003C\u002Fspan>\u003Cspan class=\"s\">celery\u003C\u002Fspan>\u003Cspan class=\"sh\">\"\u003C\u002Fspan>\n\u003Cspan class=\"n\">task_default_queue_type\u003C\u002Fspan> \u003Cspan class=\"o\">=\u003C\u002Fspan> \u003Cspan class=\"sh\">\"\u003C\u002Fspan>\u003Cspan class=\"s\">quorum\u003C\u002Fspan>\u003Cspan class=\"sh\">\"\u003C\u002Fspan>\n\n\u003Cspan class=\"n\">task_acks_late\u003C\u002Fspan> \u003Cspan class=\"o\">=\u003C\u002Fspan> \u003Cspan class=\"bp\">True\u003C\u002Fspan>\n\u003Cspan class=\"n\">task_reject_on_worker_lost\u003C\u002Fspan> \u003Cspan class=\"o\">=\u003C\u002Fspan> \u003Cspan class=\"bp\">True\u003C\u002Fspan>\n\u003Cspan class=\"n\">worker_prefetch_multiplier\u003C\u002Fspan> \u003Cspan class=\"o\">=\u003C\u002Fspan> \u003Cspan class=\"mi\">1\u003C\u002Fspan>\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cdiv class=\"highlight__panel js-actions-panel\">\n\u003Cdiv class=\"highlight__panel-action js-fullscreen-code-action\">\n    \u003Csvg xmlns=\"http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\">\u003Ctitle>Enter fullscreen mode\u003C\u002Ftitle>\n    \u003Cpath d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\">\u003C\u002Fpath>\n\u003C\u002Fsvg>\n\n    \u003Csvg xmlns=\"http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\">\u003Ctitle>Exit fullscreen mode\u003C\u002Ftitle>\n    \u003Cpath d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\">\u003C\u002Fpath>\n\u003C\u002Fsvg>\n\n\u003C\u002Fdiv>\n\u003C\u002Fdiv>\n\u003C\u002Fdiv>\n\n\n\n\u003Cul>\n\u003Cli>Ensures \u003Cstrong>tasks are not lost\u003C\u002Fstrong> even if nodes fail\n\u003C\u002Fli>\n\u003Cli>Compatible with a RabbitMQ cluster\n\u003C\u002Fli>\n\u003Cli>Works reliably with \u003Cstrong>countdown\u002FETA tasks\u003C\u002Fstrong> when paired with separate queues\u003C\u002Fli>\n\u003C\u002Ful>\n\n\n\u003Chr>\n\n\u003Ch2>\n  \u003Ca name=\"classic-vs-quorum-summary-table\" href=\"#classic-vs-quorum-summary-table\">\n  \u003C\u002Fa>\n  Classic vs Quorum: Summary Table\n\u003C\u002Fh2>\n\n\u003Cdiv class=\"table-wrapper-paragraph\">\u003Ctable>\n\u003Cthead>\n\u003Ctr>\n\u003Cth>Feature\u003C\u002Fth>\n\u003Cth>Classic Queue\u003C\u002Fth>\n\u003Cth>Quorum Queue\u003C\u002Fth>\n\u003C\u002Ftr>\n\u003C\u002Fthead>\n\u003Ctbody>\n\u003Ctr>\n\u003Ctd>Persistence\u003C\u002Ftd>\n\u003Ctd>Node-local, disk\u003C\u002Ftd>\n\u003Ctd>Node-local, replicated to majority\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>Reliability\u003C\u002Ftd>\n\u003Ctd>Low in clusters\u003C\u002Ftd>\n\u003Ctd>High, survives node failures\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>Supports ETA \u002F countdown\u003C\u002Ftd>\n\u003Ctd>Yes\u003C\u002Ftd>\n\u003Ctd>Yes\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>Priorities\u003C\u002Ftd>\n\u003Ctd>Yes\u003C\u002Ftd>\n\u003Ctd>No\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>Throughput\u003C\u002Ftd>\n\u003Ctd>High\u003C\u002Ftd>\n\u003Ctd>Lower than classic\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>Recommended for production\u003C\u002Ftd>\n\u003Ctd>❌ Only small setups\u003C\u002Ftd>\n\u003Ctd>✅ Critical tasks\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftbody>\n\u003C\u002Ftable>\u003C\u002Fdiv>\n\n\n\u003Chr>\n\n\u003Ch2>\n  \u003Ca name=\"key-takeaways\" href=\"#key-takeaways\">\n  \u003C\u002Fa>\n  Key Takeaways\n\u003C\u002Fh2>\n\n\u003Col>\n\u003Cli>\n\u003Cstrong>Classic queues are node-local in RabbitMQ 4.0\u003C\u002Fstrong>, so tasks can disappear if the master node is unavailable.\n\u003C\u002Fli>\n\u003Cli>RabbitMQ clusters enforce \u003Cstrong>\u003Ccode>(n\u002F2)+1\u003C\u002Fcode> majority\u003C\u002Fstrong> for operations, which interacts poorly with classic queues on a single node.\n\u003C\u002Fli>\n\u003Cli>\n\u003Cstrong>Quorum queues replicate tasks across nodes\u003C\u002Fstrong>, making task loss nearly impossible.\n\u003C\u002Fli>\n\u003Cli>Celery settings like \u003Ccode>task_acks_late\u003C\u002Fcode> and \u003Ccode>worker_prefetch_multiplier\u003C\u002Fcode> must be tuned with quorum queues.\n\u003C\u002Fli>\n\u003Cli>\n\u003Cstrong>Countdown \u002F ETA tasks\u003C\u002Fstrong> should be isolated in separate queues to avoid blocking main workers.\n\u003C\u002Fli>\n\u003C\u002Fol>\n\n\n\u003Chr>\n\n\u003Ch3>\n  \u003Ca name=\"recommended-production-setup-3node-cluster\" href=\"#recommended-production-setup-3node-cluster\">\n  \u003C\u002Fa>\n  Recommended Production Setup (3-node cluster)\n\u003C\u002Fh3>\n\n\n\n\u003Cdiv class=\"highlight js-code-highlight\">\n\u003Cpre class=\"highlight plaintext\">\u003Ccode>Celery workers -> connect to all 3 nodes\nDefault queue -> Quorum queue\nCountdown queue -> Quorum queue, separate worker\nClassic queues -> Only for dev or non-critical workloads\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cdiv class=\"highlight__panel js-actions-panel\">\n\u003Cdiv class=\"highlight__panel-action js-fullscreen-code-action\">\n    \u003Csvg xmlns=\"http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\">\u003Ctitle>Enter fullscreen mode\u003C\u002Ftitle>\n    \u003Cpath d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\">\u003C\u002Fpath>\n\u003C\u002Fsvg>\n\n    \u003Csvg xmlns=\"http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\">\u003Ctitle>Exit fullscreen mode\u003C\u002Ftitle>\n    \u003Cpath d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\">\u003C\u002Fpath>\n\u003C\u002Fsvg>\n\n\u003C\u002Fdiv>\n\u003C\u002Fdiv>\n\u003C\u002Fdiv>\n\n\n\n\u003Cblockquote>\n\u003Cp>This configuration ensures tasks are \u003Cstrong>never lost\u003C\u002Fstrong>, even during node failure or cluster partitions.\u003C\u002Fp>\n\u003C\u002Fblockquote>\n\n\u003Cp>This setup is \u003Cstrong>robust and production-ready\u003C\u002Fstrong> for Celery + RabbitMQ 4.0 clusters.\u003C\u002Fp>\n\n","# Celery Tasks on RabbitMQ 4.0: Classic Queues vs Quorum Queues\n\nWhen building distributed systems with **Celery** and **RabbitMQ**, one of the most common pitfalls is **tasks disappearing**. This usually happens in clustered setups when using **classic queues**. RabbitMQ 4.x introduced some changes and new queue types, which affects Celery users in production. In this post, we'll explore:\n\n- How RabbitMQ cluster connections work  \n- Differences between classic and quorum queues  \n- Limitations of each queue type  \n- Why tasks sometimes disappear on classic queues in a 3-node cluster  \n- How to fix these issues  \n\n---\n\n## RabbitMQ Cluster Architecture (n\u002F2 + 1)\n\nA **RabbitMQ cluster** is a peer-to-peer network of nodes where each node can accept connections and coordinate with other nodes. In production, cluster operations often rely on a **majority quorum rule**:\n\n```\nMinimum nodes to operate = floor(n\u002F2) + 1\n```\n\nFor example, in a **3-node cluster**:\n\n- `n = 3`  \n- `(3 \u002F 2) + 1 = 2` nodes  \n- At least **2 nodes must be up and reachable** for the cluster to accept writes or form a majority.\n\nThis ensures **consistency and reliability**, especially for replicated queues.\n\n### RabbitMQ 3-node cluster diagram\n\n```\n        +------------+\n        | Node A     |\n        | (rabbitmq1)|\n        +------------+\n           |    \\\n           |     \\\n           |      \\\n        +------------+        +------------+\n        | Node B     |--------| Node C     |\n        | (rabbitmq2)|        | (rabbitmq3)|\n        +------------+        +------------+\n```\n\n- Each node is a **disc node** (stores persistent queues).  \n- Messages in **classic queues** are stored on the node where the queue is declared.  \n- Cluster nodes maintain peer-to-peer links; `(n\u002F2)+1` nodes are needed to maintain consistency in operations like leader election.\n\n---\n\n## Classic Queues in RabbitMQ 4.x\n\nClassic queues are the **original queue type** in RabbitMQ:\n\n- Messages are **stored on the node where the queue is created**  \n- By default, **they are not replicated** across nodes (unless mirrored, but mirrored queues are deprecated and removed from 4.x ((https:\u002F\u002Fwww.rabbitmq.com\u002Fdocs\u002F3.13\u002Fha)[Read more]))  \n- Suitable for **single-node setups** or non-critical workloads\n\n### What changed in RabbitMQ 4.0\n\n- `classic_mirrored_queue_version` feature is **removed** in 4.0  \n  - This means old mirrored queues are deprecated  \n  - Classic queues are now **always node-local unless manually mirrored via legacy options**  \n\n### Why tasks disappear in a classic queue\n\nIn a **3-node cluster**, consider a task published to a **classic queue**:\n\n- Queue lives on **Node A**\n- Cluster requires `(n\u002F2)+1 = 2` nodes to operate correctly\n- If Node A becomes temporarily unreachable (network glitch, maintenance), Celery can still publish tasks to Node B\n- Node B cannot deliver the task because **queue master is on Node A**\n- As a result, tasks **appear lost**, especially in short outages\n\n#### Example scenario\n\n```\nProducer publishes task -> Node B\nQueue master       -> Node A\nCluster majority    -> 2 nodes required (A + B)\nNode A unavailable for short time\nResult: Task cannot be delivered, disappears\n```\n\n> The main reason for disappearing tasks is that **classic queues exist on a single node**, while the cluster is enforcing majority `(n\u002F2)+1`. If the node hosting the queue is unreachable, even briefly, the task cannot be routed to the master.\n\n---\n\n## Quorum Queues\n\n**Quorum queues** were introduced to replace mirrored queues and solve the reliability problems of classic queues in clusters.\n\n### How quorum queues work\n\n- Messages are **replicated across multiple nodes**  \n- Writes require **majority acknowledgment** (consensus) to be considered successful  \n- Safe against **node failures**, because tasks exist on multiple nodes  \n\n### Quorum queue diagram in 3-node cluster\n\n```\n             +----------------------+\n             |      Task Producer    |\n             +----------+-----------+\n                        |\n                        v\n             +----------------------+\n             | Quorum Queue Leader   |\n             |       Node A         |\n             +----------+-----------+\n                        |\n           -------------------------------\n           |                             |\n           v                             v\n+----------------------+       +----------------------+\n| Quorum Queue Follower |       | Quorum Queue Follower |\n|       Node B          |       |       Node C          |\n+----------------------+       +----------------------+\n```\n\n- A task is only removed when **leader + majority followers confirm**  \n- Even if Node A goes down, Node B or C can continue serving the queue\n\n---\n\n### Limitations of quorum queues\n\n| Limitation | Explanation | How to mitigate |\n|------------|-------------|----------------|\n| **Slower than classic queues** | Each write requires majority acknowledgment | Use quorum queues only for critical tasks |\n| **No priorities** | Quorum queues do not support task priorities | Use separate queues for priority handling |\n| **Memory pressure** | Replication consumes memory on all nodes | Monitor memory and adjust `ram nodes` if needed |\n| **Prefetching** | Workers must fetch 1 task at a time for reliability | `worker_prefetch_multiplier = 1` in Celery |\n\n---\n\n### Celery configuration for quorum queues\n\n```python\nbroker_url = \"amqp:\u002F\u002Fadmin:admin@rabbitmq1:5672\u002F\u002F\"\n\ntask_default_queue = \"celery\"\ntask_default_queue_type = \"quorum\"\n\ntask_acks_late = True\ntask_reject_on_worker_lost = True\nworker_prefetch_multiplier = 1\n```\n\n- Ensures **tasks are not lost** even if nodes fail  \n- Compatible with a RabbitMQ cluster  \n- Works reliably with **countdown\u002FETA tasks** when paired with separate queues\n\n---\n\n## Classic vs Quorum: Summary Table\n\n| Feature | Classic Queue | Quorum Queue |\n|---------|---------------|--------------|\n| Persistence | Node-local, disk | Node-local, replicated to majority |\n| Reliability | Low in clusters | High, survives node failures |\n| Supports ETA \u002F countdown | Yes | Yes |\n| Priorities | Yes | No |\n| Throughput | High | Lower than classic |\n| Recommended for production | ❌ Only small setups | ✅ Critical tasks |\n\n---\n\n## Key Takeaways\n\n1. **Classic queues are node-local in RabbitMQ 4.0**, so tasks can disappear if the master node is unavailable.  \n2. RabbitMQ clusters enforce **`(n\u002F2)+1` majority** for operations, which interacts poorly with classic queues on a single node.  \n3. **Quorum queues replicate tasks across nodes**, making task loss nearly impossible.  \n4. Celery settings like `task_acks_late` and `worker_prefetch_multiplier` must be tuned with quorum queues.  \n5. **Countdown \u002F ETA tasks** should be isolated in separate queues to avoid blocking main workers.  \n\n---\n\n### Recommended Production Setup (3-node cluster)\n\n```text\nCelery workers -> connect to all 3 nodes\nDefault queue -> Quorum queue\nCountdown queue -> Quorum queue, separate worker\nClassic queues -> Only for dev or non-critical workloads\n```\n\n> This configuration ensures tasks are **never lost**, even during node failure or cluster partitions.\n\nThis setup is **robust and production-ready** for Celery + RabbitMQ 4.0 clusters.",{"name":30,"username":31,"twitter_username":32,"github_username":33,"user_id":34,"website_url":35,"profile_image":36,"profile_image_90":37},"Mohamad Ashraful Islam","ashraful","_ashraful","iashraful",30490,"https:\u002F\u002Fashraful.dev","https:\u002F\u002Fmedia2.dev.to\u002Fdynamic\u002Fimage\u002Fwidth=640,height=640,fit=cover,gravity=auto,format=auto\u002Fhttps%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F30490%2F9dc1ef64-0233-43b5-9127-79affd83cc43.jpg","https:\u002F\u002Fmedia2.dev.to\u002Fdynamic\u002Fimage\u002Fwidth=90,height=90,fit=cover,gravity=auto,format=auto\u002Fhttps%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F30490%2F9dc1ef64-0233-43b5-9127-79affd83cc43.jpg",1784633123717]