Our Consumer Read the Order 12ms Before It Existed

A customer placed an order.

Our consumer picked it up in 40ms.

And said the order doesn’t exist.

The flow was simple:

POST /orders → save order → publish OrderCreatedEvent → consumer enriches it and writes back

It worked locally. It worked in staging.

In prod, it started throwing order not found for ~3% of orders. Not all of them. Just some. The worst kind of bug 😅

Checking one by one

• Message queue: fine.
• Consumer: fine.
• Database: the order was there when I queried it manually.

Then I put the logs side by side.

The consumer read the order 12ms before the transaction committed.

What was happening

The RabbitMQ publish was inside the @Transactional method.

So the event went out while the transaction was still open. The consumer picked it up, read from the DB on its own connection (where nothing was committed yet), and found nothing.

It only broke when the consumer beat our commit. That 3% was just the orders where the commit took a few milliseconds longer.

The fix 👇

1. Took the publish out of the transaction.
2. Fired it from @TransactionalEventListener(phase = AFTER_COMMIT).
3. Now the event goes out only after the commit lands, so whenever the consumer reads, the order is already visible to every connection. ✅

@TransactionalEventListener(phase = TransactionPhase.AFTER_COMMIT)
public void onOrderCreated(OrderCreatedEvent event) {
    rabbitTemplate.convertAndSend("orders", event);
}

Our event left before the commit did, and we never checked.