Why is AMPS logging a memory allocation failure when the system has plenty of free memory?

The first thing to check when AMPS logs a memory allocation failure is whether the system has enough free memory to satisfy the request. You can do this either real time, by inspecting the system, or by using the administrative console or stats.db to check the amount of memory available at the time of the failure.

If AMPS is logging memory allocation failures while the system shows free memory, the next thing to check is the amount of maps consumed across the system. These maps are used for memory mapped files.

AMPS makes extensive use of memory mapped files. The Linux operating system sets a global limit for the number of maps (mapped regions) the kernel will allocate. When that limit is reached, attempts to map (or, in some cases unmap) regions will fail and report a memory allocation failure.

To check the number of maps available, compare the maximum maps the kernel will allow:

cat /proc/sys/vm/max_map_count

with the number of maps currently allocated:

sudo cat /proc/*/maps | wc -l

If the number of maps allocated by the system is close to the number of maps the kernel will allow, then 60East recommends increasing max_map_count. For current AMPS installations, 60East recommends setting max_map_count to 500000. If errors continue, increase the max_map_count to 1000000.

To edit this value, as root you can enter the following command:

    sysctl -w vm.max_map_count=500000

This command will update the value for /proc/sys/vm/max_map_count{.docutils .literal} and allow 500,000 maps to be created, but will only do so until the next time the machine is rebooted. To make a permanent change to this setting, as a root user, edit the /etc/sysctl.conf{.docutils .literal} file and either edit or append the following setting:


Keywords: memory allocation, out of memory, OOM, mmap, max_map_count, troubleshooting, munmap, ENOMEM

Last updated