Thoughts on the loading order of the Galaxy Kirin v10 sysctl kernel parameters

background

Recently, many partners want to use Galaxy Kirin advanced server system v10 to deploy the latest version of the k8s cluster. They may have encountered various problems, so they are preparing to use kylinOS v10 to review the process of deploying the latest version of the k8s in kubeadm, which is also a pitfall for everyone.

When optimizing the server basic configuration, the step of modifying kernel parameters has triggered some new thinking.

process

When modifying kernel parameters, I was very skillful in executing the following command:

cat > /etc// << EOF
net.ipv4.ip_forward = 1
-nf-call-iptables = 1
-nf-call-ip6tables = 1
net..route_localnet = 1
vm.overcommit_memory=1
vm.panic_on_oom=0
 = 0
.max_user_watches=89100
-max=52706963
fs.nr_open=52706963
.nf_conntrack_max=2310720
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_probes = 3
net.ipv4.tcp_keepalive_intvl =15
net.ipv4.tcp_max_tw_buckets = 36000
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_max_orphans = 327680
net.ipv4.tcp_orphan_retries = 3
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_timestamps = 0
 = 16384
EOF

Then:

sysctl -p

Then I randomly verified the parameter of ip_forward and found that its value is still 0:

cat /proc/sys/net/ipv4/ip_forward
0

Suddenly I thought, I'll executesysctl -pWithout adding any parameters, it is synchronous/etc/ Files, while kylinOS v10 defaults/etc/In the file, the value of ip_forward is set to 0, so its parameter value has not been modified.

Reasonable thoughts

Since I'm executingsysctl -p Only synchronize/etc/File, if IrebootAfter that, which document will be finally effective?

I have never delved into the loading order of kernel parameters after system restart before, so I took this opportunity to learn the complete loading process.

System version description

# nkvers 
############## Kylin Linux Version #################
Release:
Kylin Linux Advanced Server Release V10 (Trading)

Kernel:
4.19.90-89.18.v2401.ky10.x86_64

Build:
Kylin Linux Advanced Server
Release V10 SP3 2403/(Trading)-x86_64-Build03/20240813
#################################################

Deep study on the loading order of kernel parameters

Through some official related documents and guidance from seniors, we have basically figured out the loading order after restart. The details are as follows.

Related services for kernel parameter loading

After the system restarts, there are two services loaded with kernel parameters, namelyand。

Check service status:

# systemctl status  
●  - Apply Kernel Variables
   Loaded: loaded (/usr/lib/systemd/system/; static; vendor preset: disabled)

# systemctl status  
●  - Dynamic System Tuning Daemon
   Loaded: loaded (/usr/lib/systemd/system/; enabled; vendor preset: enabled)

Generally, both services will take effect, sometimesWill be set todisable, prevent it from turning on and starting up automatically. Let’s first introduce the effective rules and configuration file scope of the following two service units.

Serve

This service is the system initialization stageImplicit calls will be automatically executed once when the system starts, and the loading configuration file path is:

/run//*.conf
/etc//*.conf
/usr/local/lib//*.conf
/usr/lib//*.conf
/lib//*.conf
/etc/

Serve

This service can be set to power on, or cancelled, freely controlled, and its loading configuration file path is:

/usr/lib/tuned/balanced/
/usr/lib/tuned/desktop/
/usr/lib/tuned/latency-performance/
/usr/lib/tuned/network-latency/
/usr/lib/tuned/network-throughput/
/usr/lib/tuned/powersave/
/usr/lib/tuned//
/usr/lib/tuned/throughput-performance/
/usr/lib/tuned/virtual-guest/
/usr/lib/tuned/virtual-host/

Configuration file loading order

Theoretically, the software package should install the included configuration file in the /usr/lib/ directory, and the /etc directory is only for system administrators.

Generally speaking, whenWhen the service is set to enable, the loading order is to load first.The corresponding configuration file for the service is loadedThe configuration file for the service.

When multiple files have the same kernel parameters configuration, the value loaded later will be overwritten by the value loaded first. This is a fixed rule!

Each service has so many configuration file effective paths, so what is their loading order?

Loading order

The configuration files must conform to the format. The configuration files with the same name in different directories shall only be subject to the configuration files in the directory with the highest priority. Directory priority order is as follows: /etc > /run > /usr/local/lib > /usr/lib > /lib
After reading the configuration in the above directory, finally read /etc/. Although /etc/ has the lowest reading priority, since the loading afterwards will overwrite the value loaded first under the same kernel parameters, the value of /etc/ file that will eventually take effect is the value of /etc/ file
Except for the configuration files with the same name, all other configuration files are independent of the directory and are handled in the dictionary order of the file names. For easy sorting, it is recommended to prefix the configuration files with two decimal numbers, so that the files with small numbers will be loaded first, and then the files with large numbers will be loaded. Similarly, loading the same parameters will overwrite the loaded first

Example:

There are several configuration files:

/usr/lib//
/usr/lib//
/etc//
/usr/lib//

The reading order must be:

/usr/lib// --> /usr/lib// --> /etc// --> /usr/lib// --> /etc/

Loading order

If it is enabled, it will be loaded later.
Tuned is divided into multiple modes, each mode corresponds to its configuration file.tuned-adm activeYou can see which mode the system is used.tuned-adm list profilesCheck the supported modes of the system.tuned-adm profile [name]Set the system to a certain mode, for example, this system is:
```
# tuned-adm activeCurrent active profile: virtual-guest
```
The corresponding loading configuration file is: /usr/lib/tuned/virtual-guest/
After the corresponding file is loaded, the tuned service will be executed.sysctl --systemCommand, if the configured parameters and the previously loaded configuration file have the same kernel parameters, it will also be overwritten.

sysctl --systemThe execution loading order is as follows:
```
# sysctl --system | grep Applying
* Applying /usr/lib// ...
* Applying /usr/lib// ...
* Applying /usr/lib// ...
* Applying /usr/lib//50-libkcapi-optmem_max.conf ...
* Applying /usr/lib// ...
* Applying /etc// ...
* Applying /etc// ...
* Applying /usr/lib// ...
* Applying /etc/ ...
```

Temporarily load the configuration file

It can also be used in the systemsysctl -pWhen the configuration file is loaded actively, if no files are specified, the default loading is loaded./etc/, if a file is specified, it means that the specified file is loaded, for example:

sysctl -p /etc//

Points to note in the Galaxy Kirin system

There is a default in the Galaxy Kirin system/usr/lib//The kernel parameter configuration file has been configured with many optimization parameters, and the naming rules are loaded in a relatively low order. Therefore, when configuring the same parameter items as its file, you need to pay attention to:

Modify the file name to load afterwards
You can also choose to modify the configuration directly
Configure directly into /etc/

Summarize

After loading
Different directory configuration files with the same name have high priority, and non-identical names are loaded in the dictionary order of file names.
The higher the priority, the more you load first, the lower the later, the more you load./etc/Last loaded
Don't forget/usr/lib//document
Don't forget the problem of overwriting the same parameters