LogoLogo
  • 60East Frequently Asked Questions
  • Release and Hotfix Policies
    • What is a "Preview" version of the AMPS Server?
    • What is the 60East Hotfix Policy?
  • Getting Support
    • What's in a minidump file? Does it include the data that's in AMPS?
    • How do I use WinSCP to upload files to the 60East SFTP support server?
    • Can everyone see things I put in support tickets?
    • How can I split large log files to upload them?
    • Is replication between different versions of AMPS supported?
  • AMPS Features and Capabilities
    • General Topics
      • What software/hardware is required to run AMPS?
      • Content filtering sounds cool, how does it work?
      • Is AMPS a point-to-point messaging product?
      • What database does AMPS use?
      • Can I publish and subscribe at the same time?
      • What CRC32 implementation does AMPS use?
      • How does AMPS support UTF-8 and Unicode?
    • Setting up Instances
      • AMPS on Apple Silicon: Running a Development Instance
      • Can I run AMPS on Windows?
      • How do I run AMPS in a container?
      • How can I have AMPS ignore SIGHUP?
    • State-of-the-World
      • What is the difference between a Topic in the SOW and the Transaction Log?
      • How does a messaging product support queries?
      • How do I subscribe to a view or an aggregate in my program?
      • How can I paginate a SOW query?
      • What happens if data changes while a SOW query is running?
      • Does AMPS return messages from the SOW in the order published?
      • How many columns/fields can I have in a SOW topic? In a view?
      • What is "Focus" in AMPS?
      • What are some ways to consume large SOW queries?
      • Do I need to list every field in a SOW topic configuration?
      • How can I get the number of records in a SOW topic?
      • How can I delete messages from a SOW topic?
      • Should I use the SOW delete action or message expiration to maintain my SOW topic?
      • How do I configure SOW storage parameters?
      • How do I get only the first 100 records from a SOW query?
      • What is the difference between historical SOW and bookmark replay?
      • Do I have to add configuration for every topic I want to publish to?
      • How does AMPS construct the SowKey for a message?
      • What happens if I change the key of a SOW topic?
    • Message Formats and Content Filtering
      • Why should I use AMPS content filtering?
      • What is the default message type for AMPS?
      • Does AMPS validate messages?
      • How does AMPS handle invalid FIX/NVFIX?
      • Can AMPS filters do case-insensitive comparisons?
      • Can I just send uninterpreted binary messages with AMPS?
      • What's BFlat?
    • Incremental Update and Change Set Delivery
      • Do I have to use delta publish to have a delta subscription?
    • Replication and High Availability
      • Does AMPS support replication?
      • Why does publishFlush() wait forever when I'm using replication?
      • Are SOW topics that are replicated guaranteed to be in sync on all instances?
      • How do I set up a failover group in my replication destination
      • How does AMPS report the specific failure in replication validation?
    • Messaging Paradigms
      • How can I do ad hoc message routing in AMPS?
      • What is a bookmark subscription?
    • Aggregation and Analytics
      • What is a View and how can it be used for aggregation?
    • Performance Questions
      • How fast is AMPS?
      • Which is faster, a regular subscription or a bookmark subscription that uses the "live" option?
  • Application Development with AMPS Clients
    • Unique Client Naming and AMPS
    • Which version of the client do I need for a specific version of AMPS?
    • How does an application pass credentials to AMPS?
    • What are the entries in a bookmark store file?
    • Is there a way to use AMPS itself as a bookmark store?
    • Should I use TCP_NODELAY for connections to AMPS?
    • Should I use Command or Message to send commands to AMPS?
    • How do I set the CorrelationId when I publish a message?
    • Why does publishFlush() wait forever when I'm using replication?
    • Can I send commands to AMPS from inside a message handler?
    • How do I use a custom Authenticator with HAClient?
    • What's the difference between CommandId, QueryId, and SubId?
    • Do I need to use an HAClient?
    • What character set is supported for CorrelationID?
    • Can I add a correlation token to the header of a message?
    • How do I change the filter on a subscription?
    • Can a client have more than one subscription at a time?
    • Can a subscriber tell who published a message?
    • Why do messages sometimes have strange data when I'm using asynchronous processing?
    • What character encoding is being used by the AMPS clients?
    • Why does my HAClient appear to hang when my server is down or the URI is incorrect?
    • How can I reconnect to my AMPS server when a disconnect occurs?
    • How can I use AMPS with a XAML application?
    • Can I enforce unique client naming without creating a transaction log file?
    • How can I tell if my program is not entitled to publish to a topic?
    • When using the Python client, will the background thread exit when the program exits?
    • Do I need more than one AMPS client object in my program?
    • Why does AMPS disconnect a client with a 'name in use' error?
    • In bookmark subscribes, why does EPOCH give me the same messages as MOST_RECENT?
  • Operations and Deployment
    • How can I manage complicated configuration files?
    • How can I find the version of my AMPS server?
    • Can I run multiple instances of AMPS on the same system?
    • What's in a minidump file? Does it include the data that's in AMPS?
    • What Does it Mean when dmesg Says AMPS Blocked for More than 120 Seconds
    • What is the 60East Hotfix Policy?
    • What software/hardware is required to run AMPS?
    • What does seconds_behind mean?
    • How do I run AMPS in a Docker container?
    • How do I use custom authentication with spark?
    • What is client reaping?
    • Is replication between different versions of AMPS supported?
    • How much bandwidth will replication require?
    • What value is normal for a specific performance counter?
    • Why is AMPS logging a memory allocation failure when the system has plenty of free memory?
    • Can I check a configuration file without restarting AMPS?
    • What's the difference between sync and async acknowledgement in AMPS replication?
    • Do I need to worry when a regular expression topic name can't be validated?
    • Can AMPS run as a User Service?
    • Can I dynamically choose which topics to replicate?
    • Should we have Hyper-threading enabled on AMPS servers?
    • Does AMPS correctly handle leap seconds?
    • How do I copy a SOW from one AMPS instance to another?
    • How can I control the size of the statistics database?
    • How can I configure AMPS to use a specific network adapter?
    • How can I limit the amount of memory AMPS consumes?
    • What if I need to move messages from one AMPS instance to another?
    • Is PCIe SSD a requirement when using a transaction log?
    • AMPS Log Messages
      • How do I get more information on an AMPS error?
      • How can I tell where a connection is coming from?
      • Does AMPS log current process limits?
      • What does it mean when AMPS has detected that it may not be running correctly?
      • How can I keep only the last week of error logs?
      • What is the difference between "execution time" and "elapsed time" in the logs?
      • What does a "potential stuck thread" message mean?
      • What does a "message cache is throttled" warning mean?
      • What does "Replication has detected an existing connection" mean? Why does AMPS close the connection?
      • Where is the list of all the event and error messages in AMPS?
      • What can cause an "unknown command received" or "parse format error"?
      • Why does AMPS disconnect a client with a 'name in use' error?
      • How do I exclude passwords in trace logging?
Powered by GitBook
On this page
  1. Operations and Deployment

How much bandwidth will replication require?

AMPS replication provides the ability to provide messages from one instance of AMPS to another instance of AMPS. Starting in AMPS 4.3, AMPS offers replication compression to reduce the amount of bandwidth required for replication.

To ensure that you have enough bandwidth allocated for the instances, you can estimate the bandwidth needs for each replication destination using the following formula:

(number of messages * average message size) + (128 bytes * number of messages)

For example, if your application expects to replicate 10000 messages per second at peak capacity, with each message averaging 250 bytes, a good estimate is that outgoing replication will require approximately 30.24 Mbps at peak capacity per destination (3780000 bytes per second).

If the topic you will be replicating already has a transaction log with messages recorded, you can estimate the traffic using amps_journal_dump to inspect the msg len and timestamp of the messages in the log: if your application will continue the same pattern of activity, this can be a very accurate way to approximate the bandwidth required for replication.

If the topic you will be replicating does not have a transaction log configured, but has a SOW configured, you can use the AMPS admin interface to estimate the average message size by calculating the stored_bytes/valid_keys for the SOW. Using the current SOW state to estimate average message size is most useful when the size of the SOW records stays roughly the same over the lifetime of the message: if messages can differ in size during the record lifetime, then an estimate using these numbers at any given point in time may not be accurate for estimating replication traffic.

When replication compression is active, the size reduction depends on how well the data compresses. The method above is useful for estimating the uncompressed size, which should be larger than the amount of bandwidth required for a compressed connection. You can then test in your environment with representative data to understand the actual savings that compression provides.

Last updated 8 months ago