• Linux

Linux

About

• Latest Kernel Docs

Kernel Boot-Parameter

Different distributions ship different boot-parameters. Look them up via:

   1 man 7 bootparam

Here are some important kernel command line parameters that should not be forgotten.

   1 GRUB_CMDLINE_LINUX_DEFAULT="quiet zswap.enabled=1 cgroup.enable=memory swapaccount=1 scsi_mod.use_blk_mq=1 nomodeset"

IPv6

Source of the hint: FreeIPA Deployment Recommendations

DO NOT use ipv6.disable=1 on the kernel commandline: It disables the whole IPv6 stack and breaks Samba.

If necessary, adding ipv6.disable_ipv6=1 will keep the IPv6 stack functional but will not assign IPv6 addresses to any of your network devices. This is recommended approach for cases when you don't use IPv6 networking.

You may also disable "all" or very specific interfaces.
/etc/sysctl.d/ipv6.conf

   1 net.ipv6.conf.all.disable_ipv6 = 1
   2 # Disabling "all" does not apply to interfaces
   3 # that are already "up" when sysctl settings are applied. 
   4 net.ipv6.conf.<interface0>.disable_ipv6 = 1

Virtual Memory

• https://www.kernel.org/doc/Documentation/sysctl/vm.txt

Swappiness

• path: /proc/sys/vm/swappiness

• sysctl key: vm.swappiness

• default: 60
• configuration:

  1. at boottime via sysctl

  2. at runtime via procfs

/proc/sys/vm/swappiness

   1 # cat /proc/sys/vm/swappiness
   2 60
   3 # echo 5 > /proc/sys/vm/swappiness
   4 # cat /proc/sys/vm/swappiness
   5 5

/etc/sysctl.d/vm.conf

   1 vm.swappiness = 5

Apply configuration via sysctl.

   1 # sysctl --system
   2 * Applying /etc/sysctl.d/30-baloo-inotify-limit.conf ...
   3 fs.inotify.max_user_watches = 524288
   4 * Applying /etc/sysctl.d/30-postgresql-shm.conf ...
   5 * Applying /etc/sysctl.d/30-tracker.conf ...
   6 fs.inotify.max_user_watches = 65536
   7 * Applying /usr/lib/sysctl.d/50-coredump.conf ...
   8 kernel.core_pattern = |/lib/systemd/systemd-coredump %P %u %g %s %t 9223372036854775808 %e
   9 * Applying /etc/sysctl.d/99-sysctl.conf ...
  10 * Applying /etc/sysctl.d/vm.conf ...
  11 vm.swappiness = 5
  12 vm.dirty_background_ratio = 8
  13 vm.dirty_expire_centisecs = 3000
  14 vm.dirty_ratio = 32
  15 vm.dirty_writeback_centisecs = 500
  16 * Applying /etc/sysctl.conf ...

Memory over-commitment

https://www.kernel.org/doc/html/latest/vm/overcommit-accounting.html

• sysctl key: vm.overcommit_memory

• path: /proc/sys/vm/overcommit_memory

• default: 0
• configuration:

  1. at boottime via sysctl

  2. at runtime via procfs

• usage:
  • redis demands it with

    WARNING overcommit_memory is set to 0!
    Background save may fail under low memory condition.
    To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and
    then reboot or run the command 'sysctl vm.overcommit_memory=1'
    for this to take effect.

/etc/sysctl.d/vm.conf

   1 vm.overcommit_memory = 1

Transparent hugepages

https://www.kernel.org/doc/html/latest/admin-guide/mm/transhuge.html

• path: /sys/kernel/mm/transparent_hugepage/

• default: 0
• configuration:

  1. at boottime via sysfsutils

  2. at runtime via sysfs

• usage:
  • redis demands it with

    WARNING you have Transparent Huge Pages (THP) support enabled in your kernel.
    This will create latency and memory usage issues with Redis.
    To fix this issue run the command 'echo madvise > /sys/kernel/mm/transparent_hugepage/enabled' as root,
    and add it to your /etc/rc.local in order to retain the setting after a reboot.
    Redis must be restarted after THP is disabled (set to 'madvise' or 'never').

/etc/sysfs.d/transparent_hugepage.conf

   1 kernel/mm/transparent_hugepage = madvise

zswap

• https://www.kernel.org/doc/html/latest/vm/zswap.html

• https://www.kernel.org/doc/Documentation/vm/zswap.txt

• https://en.wikipedia.org/wiki/Zswap

• https://lwn.net/Articles/537422/

• https://wiki.archlinux.org/index.php/Zswap

Zswap is a lightweight compressed cache for swap pages. It takes pages that are in the process of being swapped out and attempts to compress them into a dynamically allocated RAM-based memory pool. zswap basically trades CPU cycles for potentially reduced swap I/O. This trade-off can also result in a significant performance improvement if reads from the compressed cache are faster than reads from a swap device.

• path: /sys/module/zswap/parameters

• default: N
• configuration:

  1. at boottime via sysfsutils

  2. at runtime via sysfs

grep -R . /sys/module/zswap/parameters

   1 /sys/module/zswap/parameters/same_filled_pages_enabled:Y
   2 /sys/module/zswap/parameters/enabled:N
   3 /sys/module/zswap/parameters/max_pool_percent:20
   4 /sys/module/zswap/parameters/compressor:lzo
   5 /sys/module/zswap/parameters/zpool:zbud

/etc/sysfs.d/zswap.conf

   1 module/zswap/parameters/enabled = 1

IO-Scheduler

• On a hypervisor the scheduler bfq seems to be reasonable.

• On a VM with no disk or controller pass-through none should be used. This avoids optimizing the queues twice, which is inefficient and contra-productive. The hypervisor will optimize the io-request anyway.

Make alternative schedulers available

BLK-MQ is nowadays broadly available and enabled in distributions. Using multiple queues on multicore systems with fast storage promises some performance gains.

But when I took a look on available schedulers only "mq-deadline" and "none" were available.

   1 cat /sys/class/block/sda/queue/scheduler
   2 [mq-deadline] none

This is because these scheduler are shipped as a kernel module and need to be loaded first into the kernel via modprobe.

   1 ls /lib/modules/$(uname -r)/kernel/block
   2 bfq.ko  kyber-iosched.ko

Modules may be loaded manually:

   1 modprobe bfq
   2 modprobe kyber-iosched

Modules may also be loaded automatically at boot-time via /etc/modules.

   1 for SCHEDULER in bfq kyber-iosched; do
   2         if \! grep -q "$SCHEDULER" /etc/modules && \
   3            \! grep -qr "$SCHEDULER" /etc/modules-load.d; then
   4                 echo "$SCHEDULER" >> /etc/modules
   5         fi
   6 done

Set IO-Scheduler permanently

kernel-cmdline

• Method seems not to be working any longer.

• Service affecting.

/etc/default/grub

   1 GRUB_CMDLINE_LINUX_DEFAULT="quiet elevator=$SCHEDULER"

Refresh grub config and reboot.

   1 update-grub2
   2 systemctl reboot

udev-rule

• Works at run- and at boot-time!

• More selective because disks may be filtered with a regex.

/etc/udev/rules.d/60-persistent-storage-scheduler.rules

   1 # GUEST SHOULD NOT REORDER STORAGE REQUESTS -> NONE or NOOP
   2 ACTION=="add|change", KERNEL=="[sv]d[a-z]", ATTR{queue/scheduler}="none"
   3 # HYPERVISOR SHOULD NOT REORDER STORAGE REQUESTS IN PARALLEL QUEUES
   4 #ACTION=="add|change", KERNEL=="sd[ab]", ATTR{queue/scheduler}="bfq"
   5 

Reload udev-rules

Reload will probably happen automatically but the "trigger" is necessary.

   1 udevadm control --reload-rules && udevadm trigger

Drop FS Cache

   1 echo 3 | tee /proc/sys/vm/drop_caches

Hardening

Disable TCP Timestamping

• https://cloudshark.io/articles/tcp-timestamp-option/

• https://netsense.ch/blog/tcp-timestamps/

• https://www.scip.ch/?labs.20150305

• IETF RFC1323 TCP Extensions for High Performance (obsoleted)

• IETF RFC7323 TCP Extensions for High Performance

   1 hping3 -S -p 22 --tcp-timestamp $DESTINATION
   2 
   3 1 root@libertas /home/tobias/Downloads # hping3 -S -p 22 --tcp-timestamp www.rockstable.it
   4 HPING www.rockstable.it (bridge 178.63.149.226): S set, 40 headers + 0 data bytes
   5 len=56 ip=178.63.149.226 ttl=53 DF id=0 sport=22 flags=SA seq=0 win=65160 rtt=24.2 ms
   6   TCP timestamp: tcpts=2031225761
   7 
   8 len=56 ip=178.63.149.226 ttl=53 DF id=0 sport=22 flags=SA seq=1 win=65160 rtt=19.8 ms
   9   TCP timestamp: tcpts=2031226761
  10   HZ seems hz=1000
  11   System uptime seems: 23 days, 12 hours, 13 minutes, 46 seconds

Disable temporarily

   1 echo 0 > /proc/sys/net/ipv4/tcp_timestamps

Disable persitent
{{{/etc/sysctl.d/tcp.conf

   1 net.ipv4.tcp_timestamps=0