Migrating a Live RabbitMQ Queue: No Loss, No Duplicates, No Downtime

You want to add a dead-letter exchange to a queue that has been running for a long time. You change the queue declaration and you deploy. The broker answers 406 PRECONDITION_FAILED.

This is not a bug. A queue's arguments are fixed for as long as the queue exists, and redeclaring it with different arguments is an error.

What happens next depends on your client. The service may refuse to start. Or it may log one error, stop declaring the rest of its topology, and carry on consuming the old queue with the old arguments. That is what Spring AMQP does by default, because mismatchedQueuesFatal is false, and it is the dangerous case: nothing looks broken. Either way, your change did not happen.

This is ordinary work. Queues acquire a dead-letter exchange, a length limit or a new queue type as a system grows, so every team running RabbitMQ meets this sooner or later. The awkward part turns out not to be RabbitMQ at all.

What follows is a sequence you can run by hand, on a system that is taking traffic, with no message lost and none delivered twice.

Replacing the queue is three commands on an idle system: declare the new queue, move the routing key over, retire the old one. Under live traffic, with a producer owned by another team, each of those three depends on knowing something you cannot reliably know.

Three words carry the whole article, and the third is the one that causes trouble. A queue holds messages and a consumer reads them. A routing key is a string the publisher attaches to each message. An exchange is the part most people picture wrongly: it stores nothing and it is not a pipe. It is a routing table. Publishers always publish to an exchange, never to a queue, and the exchange compares the message's routing key against its list of bindings — rules of the form "routing key rk goes to queue q" — to compute which queues get the message. Change a binding and you change where messages go, without touching the publisher or the consumer. That is the lever this whole migration pulls.

Here is the whole migration first, so the rest of the article has somewhere to land. Six steps, grouped into the parts below: inserting the exchange you own, the swap, and taking the scaffolding down.

One detour before the sequence, because it can save you the whole exercise. A policy can set the dead-letter exchange, the dead-letter routing key and the message TTL on a queue that is already running, with no redeclare and no downtime. If that covers your change, you do not need any of this.

Three things a policy cannot do. It cannot change the queue type, so classic to quorum always means a new queue. It cannot set the x-queue-* arguments that have no policy equivalent. And it cannot help when you specifically need the setting to be an argument rather than policy-managed — an argument on the queue overrides a policy, so teams that want the value pinned in code, immune to whoever edits policies next, have to put it in the declaration.

There is also a practical reason the policy route is not always available: policies are not part of the AMQP protocol. They live in the HTTP management API, so a client library that speaks AMQP cannot declare them. Spring AMQP is the example most readers will meet — RabbitAdmin declares exchanges, queues and bindings, and nothing else. A team whose topology is defined in application code has no way to express a policy there, and no @Bean that will create one at startup.

So from here on, assume the change really does need a new queue.

Part 1: The question you cannot answer

Your queue qOld is bound to routing key rk on exchange X. To replace it, you unbind rk → qOld and bind rk → qNew. Before you touch anything, one question decides whether that is safe.

Who else is bound to rk on X?

If you are the only consumer, the migration is easy. If other queues share that routing key, then you are changing a topology you do not fully understand, and any retry or dead-letter path you attach can reach consumers you did not know were there.

The list itself is easy to get. It is one management-API call, GET /api/exchanges/{vhost}/{name}/bindings/source, and it returns every binding on the exchange. Reading it is not the hard part, and two things make the reading less conclusive than it looks. The same exchange usually exists separately in every per-tenant vhost, so there is one list per vhost to check rather than one list. And on a topic exchange you are not looking for one routing key but for every pattern that matches it.

The real problem is worse than either. The answer expires. Somebody can bind a new queue to that routing key five minutes after you looked, while your deployment is still rolling out. A colleague can tell you they are not bound to your key, be completely right, and be out of date before you finish. A migration that is correct only because "we checked and nobody else is bound" will pass in test and surprise you later.

So stop asking the question.

Part 2: Interpose an exchange you own

Put an exchange that you control between exchange X and your queue.

---

Bootstrap — four steps, in this order:

1. Declare X2, with the same type as X, so that the binding you copy onto it means the same thing. Declaring it as an internal exchange is worth doing: publishers cannot publish to an internal exchange directly, so nothing can reach X2 except through the binding from X.
2. Bind X → X2 on rk. This is an exchange-to-exchange binding.
3. Bind rk → qOld on X2.
4. Remove the direct rk → qOld binding on X.

---

The order is the point. Add before remove, at every step. After step 3 and before step 4, queue qOld can be reached by two paths at the same time. That overlap is deliberate, and the next section explains why it is free.

The only change to exchange X was the removal of your own binding. No other team's binding was touched, and no policy was set on X. You never had to learn who the other consumers were.

The question now has an answer. You cannot know who is bound to rk on X. You can know exactly who is bound to rk on X2, because you created X2 thirty seconds ago and nobody else knows it exists.

Part 3: Two paths, one copy

Step 3 leaves qOld reachable through X → qOld and through X → X2 → qOld. The obvious worry is that the queue now receives every message twice.

It does not, and this is a documented guarantee rather than an accident:

We detect and eliminate cycles during message delivery, and ensure that transitively, over any routing topology, for every queue to which a given message is routed, each queue will receive exactly one copy of that message.

— RabbitMQ documentation, Exchange-to-Exchange Bindings

The word "deduplication" is misleading here, and the difference matters. RabbitMQ does not deliver two copies and then throw one away. Routing a message computes the set of destination queues, and a set cannot hold the same queue twice. Two paths to one queue collapse before any delivery happens.

This is a property of routing, and it is why add-before-remove works. Every intermediate state is safe because of how it is built, not because you pass through it quickly.

Part 4: The swap

Routing key rk now has exactly one destination on X2, so the swap itself is a local change. But this step cannot follow the add-before-remove rule, and it is worth being clear about why.

In Part 2 the two paths led to the same queue, so the exactly-one-copy guarantee made the overlap free. Here they would lead to two different queues, qOld and qNew. Binding both at once means every message is delivered to both, and both consumers process it. That is real duplication, not a routing artefact. So the old binding has to go before the new one arrives, and that leaves a moment where rk leads nowhere on X2. Publisher confirms do not save you here: unless the publisher sets the mandatory flag, the broker acknowledges a message it could not route to any queue.

Before going further: if your consumer is idempotent, you do not need any of what follows. Bind rk to both queues, let both consumers run for a moment, then unbind the old one and drain it. Every message in that window is processed twice, which is exactly what idempotence makes survivable, and it is the usual advice for this case. The rest of this part is for the case where a consumer is not idempotent and processing a message twice is not acceptable.

An alternate exchange covers that moment:

Whenever a message published to an exchange with a configured AE cannot be routed to any queue, the channel re-publishes the message to the specified AE instead.

— RabbitMQ documentation, Alternate Exchanges

The useful part is where you point it. The alternate exchange is bound directly to qNew, and it is put in place before anything is unbound:

---

The swap — six steps, in this order:

1. Declare the alternate exchange and bind it to qNew on rk. A message sent to an alternate exchange keeps the routing key it was published with, so give the alternate exchange the same type as X2 and bind on the same key. Bind it with an empty key instead and the alternate exchange cannot route the message either, which loses exactly the traffic this step exists to catch. (A fanout alternate exchange would also work, since it ignores keys, but then it is a second exchange type to reason about.)
2. Attach the alternate exchange to X2 with a policy. It has to be a policy, not an alternate-exchange argument: X2 already exists from the bootstrap, and redeclaring an existing exchange with a new argument fails with the same 406 this article opened with.
3. Unbind rk → qOld. From this moment, messages on rk are unroutable on X2, so they go to the alternate exchange and into qNew.
4. Wait for qOld to drain.
5. Bind rk → qNew directly on X2.
6. Detach the alternate exchange and delete it.

---

The catcher is armed first, so there is never an uncovered gap. There is also no holding queue and no replay step, because the catch and the correct delivery are the same binding. A message that arrives during step 3 lands in the new queue and its consumer processes it as normal traffic.

This depends on one narrow behaviour. An alternate exchange is used only when routing produces an empty set of destinations. A wildcard binding that also matches rk would keep that set non-empty, nothing would fall through, and the messages would quietly go somewhere else. It works here only because Part 2 made you the single binding for that routing key on X2.

During step 4, watch the depth of qOld instead of assuming it empties. Drained means two numbers are zero, not one: no messages ready and none unacknowledged. Poll for both, and watch for two different failures. Unacknowledged rising across consecutive polls means the consumer is taking messages faster than it can finish them, and you can abort on that immediately. The quieter failure is unacknowledged sitting flat and non-zero, bounded by the consumer's prefetch, because a consumer holds messages it will never acknowledge; nothing rises, so only a timeout catches it. When you finally delete the queue, delete it with if-empty and if-unused so the broker refuses if you were wrong.

Part 5: Take the scaffolding down

You can leave X2 in place. It costs one exchange and one binding for each migrated queue. That is clutter rather than a bug, and it accumulates: every queue you migrate this way leaves another permanent pair behind.

But the safety never came from X2 existing. It came from the order of the steps. X2 is scaffolding, and it can come down. The teardown mirrors the setup.

---

Teardown — four steps, in this order:

1. Bind rk → qNew directly on X.
2. Remove the X → X2 binding.
3. Remove rk → qNew on X2.
4. Delete X2, if it routes to nothing else.

---

Add before remove again. Step 1 gives qNew a direct path before either part of the old path disappears, and during that overlap the destination-set property keeps delivery at exactly one copy. The final binding graph has the same shape as a queue that was never migrated.

One condition applies from the moment qNew exists, not just at teardown, and it returns to the dead-letter exchange from the opening example. Do not point qNew's dead-letter exchange at X. Dead-lettering republishes the message into the exchange you configured, with the routing key it was originally published with — it does not travel back along the path the message arrived by. So a rejected message re-enters X on rk and fans out to every other queue bound to it, including the ones you could never enumerate. X2 does not shield you from that at any stage; interposing it changed where the queue is bound, not where its rejects are published. Binding qNew directly on X only adds one more recipient: qNew itself, which is a loop. Point the dead-letter exchange at the default exchange instead, and use x-dead-letter-routing-key to name the queue that should receive it.

What about messages sitting inside X2 when you delete it? There are none. Exchanges do not store messages. Routing is computed during basic.publish, against the binding table as it exists at that instant, and the message goes straight to the matching queues or nowhere.

Part 6: Doing this more than once

Everything above is a handful of rabbitmqadmin commands. You can run them by hand, and the first time you should.

The second time you will want to automate it. The steps themselves are easy to script; everything around them is not:

Concern	What it means
Concurrency	Several pods start at the same time. Exactly one must run the migration. The others must wait for its result, not skip past it.
Resumability	A pod can die in the middle of the sequence. Every step must be safe to repeat and safe to re-enter at any point.
Rollback	Most steps need an inverse that works while traffic is flowing, and you must know which step is the point of no return. Deleting the old queue is that step: after it, restoring the old topology is a manual job, and any message the new queue already caught does not go back.
Drain detection	"Is qOld empty?" needs a real answer, including the case where a consumer is stuck.
Rolling deployments	During a canary release, old consumers stay alive much longer than any sensible startup timeout. The old queue really is not ready to delete, and the correct behaviour is to stop and retry later.
Environments	The same sequence has to run in every environment and every vhost, without supervision, and leave a record of what ran where.

Four problems are worth naming, because they are not obvious.

Replacing a queue changes its failure behaviour. Suppose the old queue had no dead-letter exchange and the new one has one. That is often the whole reason for the migration. Your consumer rejects an unprocessable message the way it always has, with AmqpRejectAndDontRequeueException. On the old queue that dropped the message. On the new queue it dead-letters the message into your parking queue. Same code, same exception, opposite result. If "unprocessable" is a normal business outcome rather than a temporary failure, you can fill that parking queue before anybody notices. A queue's failure behaviour is part of its configuration, so replacing the queue silently changes what your existing error handling does.

Do not share one X2 between migrations. Creating one intermediate exchange per service instead of one per queue looks tidy, and it is not safe. The alternate exchange has to be attached with a policy, RabbitMQ applies at most one policy per exchange, and it chooses between two of equal priority, in the documentation's word, "undeterministically". Share the exchange and one crashed migration can leave a policy behind that drains a second migration's in-flight messages into the first migration's new queue. A preflight check that inspects exchange arguments will not see it, because the alternate exchange is not an argument. Use one X2 per queue, and delete it when you are done.

That also means you should refuse to run the swap if X2 carries an alternate-exchange argument that somebody added by hand. An argument overrides a policy, so the policy you set would be ignored and the fall-through would go somewhere you did not choose.

Never publish test messages into an exchange you do not own. A test message can land in another team's retry loop and bounce there until somebody notices. If you want to test the sequence, test it against an exchange that belongs to you.

Watch out for the client re-creating what you removed. In Spring AMQP, RabbitAdmin declares every Binding bean it finds in the application context at startup. If the bean for rk → qOld is still there, the next deployment puts that binding straight back on X, and your migration silently unwinds. The bean has to go in the same release as the migration.

This design does not preserve ordering across the swap. Order within a single queue holds, as long as it has one consumer and nothing is requeued. But during the overlap, the consumer of the old queue and the consumer of the new queue run at the same time, and AMQP gives no ordering guarantee across two different paths. If a consumer depends on strict ordering per key, that has to be solved separately.

Three cases are out of scope here. Check them before you copy this. A delayed-message exchange does not behave like a normal one and needs its own analysis. If X is federated, whether an exchange-to-exchange binding propagates upstream is not something this article verifies. And if qOld is declared auto-delete or with x-expires, it can disappear together with its messages when its last consumer goes away during a rolling deployment.

Takeaway

This is not a thought experiment. Four applications across three domains — the integration gateway, the webshop's provider service, and two last-mile logistics services — have moved their worker queues this way, roughly a hundred queues in total, in waves. Each queue was replaced while its traffic kept flowing, and each cutover was checked for loss and duplication before the next one started.

Nobody ran those steps by hand more than once. The sequence is driven by an orchestrator library, and it borrows its shape from database schema migration: each queue migration is a numbered file, V42__migrate_<queue-name>.yaml, the library runs the ones a service has not applied yet at startup, and it records the outcome per file in a history table so a restart does not repeat work. That is Flyway's versioned-migration pattern rather than Flyway itself — no dependency on it, just the same idea applied to broker topology instead of tables. Writing those files is itself automated: two agent skills generate the expand phase and the cleanup phase, including the new queue, the listener and the migration file, which is what made a hundred queues practical rather than a year of careful afternoons.

We are looking at releasing the library as open source. If that happens, it gets its own article.

The RabbitMQ details matter less than the shape of the problem.

One rule had to hold across the whole system: nobody else is bound to this routing key. It could not be verified, and no amount of checking would have made it dependable, because the state is unowned, unregistered, and changeable at runtime by people you cannot list. The answer was not better checking. It was to interpose something you control, so the rule became true locally and easy to verify.

The rest is discipline about order: add before remove, every time, so that every intermediate state is safe on its own rather than safe because the next step follows quickly. That is the expand and contract pattern, usually described for database columns and API fields. It works the same way when the interface is a routing key and the clients are messages already in flight.