之前系统出现过几次hung住的情况，没有oom，也没有其它内存相关的信息，而linux设计就是去尽量吃满内存，然后再回收清理的机制

目前这个参数还没有找到合适的处理这个预留的参数，一般也没有去调整的

系统是默认根据物理内存进行计算得到一个数值得

sysctl -a|grep min_free_kbytes
vm.min_free_kbytes = 45056

查看内核参数，这个小环境是保留的45M

Aerospike 的说法

https://discuss.aerospike.com/t/how-to-tune-the-linux-kernel-for-memory-performance/4195

The standard RedHat recommendation 204 is to keep min_free_kbytes at 1-3% of the total memory on the system, with Aerospike advising to keep at least 1.1GB, even if that is above the official recommended total memory percentage.

On a system with over 37GB of total RAM, you should leave no more than 3% of spare memory to min_free_kbytes in order to avoid the kernel spending too much time unnecessarily reclaiming memory. This would equal anywhere between 1.1GB and 3% of total RAM on such systems.

上面的说法是如果环境内存超过37G的情况下，按3%算就是1.1G，我们一般的环境也超过了40G，那么基本就是建议最少留个1.1G的，100G的可以保留到3G左右

内核参数

vm.min_free_kbytes = 1153434

vm.min_free_kbytes = 3145728

红帽的说法

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/performance_tuning_guide/sect-red_hat_enterprise_linux-performance_tuning_guide-configuration_tools-configuring_system_memory_capacity

Setting min_free_kbytes too low prevents the system from reclaiming memory. This can result in system hangs and OOM-killing multiple processes.

However, setting min_free_kbytes too high (for example, to 5–10% of total system memory) causes the system to enter an out-of-memory state immediately, resulting in the system spending too much time reclaiming memory.

红帽的说法是需要低于总内存的5%

ltp测试里面的参数控制

https://sourceforge.net/p/ltp/mailman/message/29738250/

Setting min_free_kbytes too high will cause system hangs,
especially in i386 arch, using less than 5% of total memory
can avoid it, so choose %5 of free memory or 2% of total memory.
Thanks Shuang pointed out it.


 * Description:
 *
 * The case is designed to test min_free_kbytes tunable.
 *
 * The tune is used to control free memory, and system always
 * reserve min_free_kbytes memory at least.
 *
 * Since the tune is not too large or too little, which will
 * lead to the system hang, so I choose two cases, and test them
 * on all overcommit_memory policy, at the same time, compare
 * the current free memory with the tunable value repeatedly.

 * a) default min_free_kbytes with all overcommit memory policy
 * b) 2x default value with all overcommit memory policy
 * c) 5% of MemFree or %2 MemTotal with all overcommit memory policy

测试用例里面测试内存过载情况下的几种参数，默认，两倍默认，5%空闲内存，或者总内存的2%，理论上，这几个都不会导致机器hung死

通过slabtop查看内核的缓存空间占用

[root@VM_0_17_centos ~]# slabtop -o|grep Total
 Active / Total Objects (% used)    : 550057 / 573695 (95.9%)
 Active / Total Slabs (% used)      : 22507 / 22507 (100.0%)
 Active / Total Caches (% used)     : 101 / 135 (74.8%)
 Active / Total Size (% used)       : 102508.62K / 106202.81K (96.5%)


[root@VM_0_17_centos ~]# grep Slab /proc/meminfo
Slab:             108676 kB


[root@VM_0_17_centos ~]# sysctl -a|grep min_free_kbytes
vm.min_free_kbytes = 45056

这个上面是腾讯云主机的，看到内核自身的占用应该在100M以上了，而我自己的vmware里面的虚拟机这个数值是47MB，这个数值可能跟不同的内核有关

系统还保留了一定的内存防止

Reserving 161MB of memory at 688MB for crashkernel (System RAM: 2047MB)

系统启动的时候看到的内存占用

[root@lab204 ~]# dmesg |grep Memory:
[    0.000000] Memory: 1841584k/2097152k available (7784k kernel code, 524k absent, 255044k reserved, 5958k data, 1980k init)

内核文档关于这个参数的解释

https://www.kernel.org/doc/Documentation/sysctl/vm.txt

min_free_kbytes:

This is used to force the Linux VM to keep a minimum number
of kilobytes free.  The VM uses this number to compute a
watermark[WMARK_MIN] value for each lowmem zone in the system.
Each lowmem zone gets a number of reserved free pages based
proportionally on its size.

Some minimal amount of memory is needed to satisfy PF_MEMALLOC
allocations; if you set this to lower than 1024KB, your system will
become subtly broken, and prone to deadlock under high loads.

Setting this too high will OOM your machine instantly.

建议能保留1G以上的空间

Why

Who

When

创建

武汉-运维-磨渣

2020-09-09