Are SOW topics that are replicated guaranteed to be in sync on all instances?

Yes, provided that all of the messages used to build the SOW are still present in the transaction logs, that both instances have processed all of the messages, and that the messages were processed on both instances in the same order.

AMPS does not directly replicate the state of a SOW. Instead, AMPS replicates the messages that created the SOW. The SOW itself is constructed by each local instance and persisted on that instance.

Under normal operation, when all messages have been replicated, the SOW databases for each instance will be in sync. While messages are actively flowing, of course, there may be discrepancies at any point in time as messages flow through replication.

However, if the SOW database of one instance is cleared, then AMPS must rebuild the state of that SOW from the messages available in the transaction logs.

Let's look at an example of how this works, starting with how AMPS rebuilds a SOW from transaction logs on a single instance.

SOW Care and Feeding

Imagine that on a single instance of AMPS, we have a SOW that contains the following three records:

SOW 1

orderId	Name	Order	Date Entered
1	Godzilla	12345	May 1, 2021
2	Mothra	12346	May 5, 2021
3	Gamora	12347	May 8, 2021

Now, imagine that today is May 9th, and the instance retains transaction logs for 7 days. In this case, maintenance on the instance removes the transaction log that contains orderId 1.

That's a normal operation, and does not cause a change in the SOW. AMPS keeps orderId 1 in the SOW -- there has been no change to the record, and no SOW delete command. The record is still perfectly valid, even though the state used to create the record is no longer present.

If the SOW file for the instance is deleted, though, AMPS has to rebuild the state of the SOW from the transaction log, producing the following records:

SOW 2

orderId	Name	Order	Date Entered
2	Mothra	12346	May 5, 2021
3	Gamora	12347	May 8, 2021

AMPS cannot recreate the record for orderId 1, because the transaction logs that contain the message no longer exist.

SOW and Replication

Now, let's add replication into the mix. What happens if the topic for this SOW is replicated, and on May 9th someone deletes the transaction logs and SOW from the downstream instance?

The results are exactly the same as if the SOW is rebuilt on the local instance. AMPS synchronizes the transaction logs and rebuilds the SOW, producing SOW 2 as before. Although the upstream instance still has orderId 1 in the SOW, the message that produced that record is no longer available to replicate, and so the downstream instance cannot populate the SOW with that record.