How much bandwidth will replication require?

AMPS replication provides the ability to provide messages from one instance of AMPS to another instance of AMPS. Starting in AMPS 4.3, AMPS offers replication compression to reduce the amount of bandwidth required for replication.

To ensure that you have enough bandwidth allocated for the instances, you can estimate the bandwidth needs for each replication destination using the following formula:

(number of messages * average message size) + (128 bytes * number of messages)

For example, if your application expects to replicate 10000 messages per second at peak capacity, with each message averaging 250 bytes, a good estimate is that outgoing replication will require approximately 30.24 Mbps at peak capacity per destination (3780000 bytes per second).

If the topic you will be replicating already has a transaction log with messages recorded, you can estimate the traffic using amps_journal_dump to inspect the msg len and timestamp of the messages in the log: if your application will continue the same pattern of activity, this can be a very accurate way to approximate the bandwidth required for replication.

If the topic you will be replicating does not have a transaction log configured, but has a SOW configured, you can use the AMPS admin interface to estimate the average message size by calculating the stored_bytes/valid_keys for the SOW. Using the current SOW state to estimate average message size is most useful when the size of the SOW records stays roughly the same over the lifetime of the message: if messages can differ in size during the record lifetime, then an estimate using these numbers at any given point in time may not be accurate for estimating replication traffic.

When replication compression is active, the size reduction depends on how well the data compresses. The method above is useful for estimating the uncompressed size, which should be larger than the amount of bandwidth required for a compressed connection. You can then test in your environment with representative data to understand the actual savings that compression provides.