Debian

Contents

Debian

About

https://www.debian.org

Releases

Base system from scratch

Remote system is a dedicated root server in the Hetzner datacenter. Only a live linux amd64 is running and it's reachable by ssh with pubkey auth. We got the option to restart/reset the live linux or boot into the new system. So the logs of the new system is all information we can get (besides the status: running/stopped).

Partitioning

Debian partman units

Offtopic (debian-installer) but nice to know.

Hint: Debian partman units

Debian partman does not support binary units like !KiB, !MiB or !GiB. All units are SI units based on 10. Default unit (no unit) is "1m" aligned to the next cylinder (1Mib).

The sizes of the partitions will not be what you expect.

To achieve correct sizes to may:

partition manually with parted in a separate shell and enter partman when done.
enter the sizes in byte with unit "b" or "B" (don't leave it away) Some examples:
```
   1 ~ # echo $((4*2**30))b ### 4GiB
   2 4294967296b
   3 ~ # echo $((256*2**20))b ### 256MiB
   4 268435456b
```

Create the partitions on the first disk

Aline it ot the full 1 MiB/Cylinder (MegaByte binary), because

this allows aligned read/writes. Otherwise it will degrade performance significantly.
this is a power of 2 (1, 2, 4(MEMORY PAGESIZE), 8, ...2^n)KiB. This means if you want to stripe or change some cluster sizes, it will always match (if less).

The boot-partition should be at least 256MiB, initial ram-disks can be large.

(vmlinuz 6MiB + initrd 50MiB + system-map 4MiB) = 60MiB/Kernel So max 3 Kernels …

   1 parted
   2 GNU Parted 3.2
   3 Using /dev/sda
   4 Welcome to GNU Parted! Type 'help' to view a list of commands.
   5 (parted) unit MiB
   6 (parted) print free
   7 Model: ATA HGST HUS726060AL (scsi)
   8 Disk /dev/sda: 5723167MiB
   9 Sector size (logical/physical): 512B/4096B
  10 Partition Table: gpt
  11 Disk Flags: 
  12 
  13 Number  Start       End         Size        File system  Name           Flags
  14         0.02MiB     1.00MiB     0.98MiB     Free Space
  15  1      1.00MiB     2.00MiB     1.00MiB                  bios_grub_sda  bios_grub
  16  2      2.00MiB     256MiB      254MiB      fat16        EFI_sda        boot, esp
  17  3      256MiB      131072MiB   130816MiB                swap1_sda      raid
  18  4      131072MiB   5723166MiB  5592094MiB  btrfs        root_sda
  19         5723166MiB  5723167MiB  0.98MiB     Free Space

Clone GPT to other disk

   1 sgdisk /dev/sda -R /dev/sdb

Create a multidisk RAID1 as swap

   1 mdadm -v --create md_swap1 --level=1 --symlinks yes --raid-devices=2 /dev/sda3 /dev/sdb3
   2 mdadm --detail --scan
   3 mkswap /dev/md/md_swap1 --label md_swap1
   4 swapon /dev/md/md_swap1

Filesystems

   1 ### CREATE FILESYSTEM
   2 mkfs.btrfs -L rootfs --data raid1 --metadata raid1 /dev/sda4 /dev/sdb4
   3 
   4 ### CREATE SUBVOLUMES
   5 mount -o noatime,compress=lzo,space_cache,autodefrag /dev/sdb4 /mnt
   6 btrfs subvolume create /mnt/root
   7 btrfs subvolume create /mnt/home
   8 
   9 btrfs subvolume list /mnt
  10 ID 258 gen 8 top level 5 path root
  11 
  12 btrfs subvolume get-default /mnt/
  13 ID 5 (FS_TREE)
  14 btrfs subvolume set-default 258 /mnt
  15 
  16 umount /mnt
  17 mount -o noatime,compress=lzo,space_cache,autodefrag /dev/sdb4 /mnt
  18 mount -t btrfs
  19 /dev/sda4 on /mnt type btrfs (rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=258,subvol=/root)
  20 
  21 debootstrap buster /mnt http://ftp.de.debian.org/debian

Migrate data to subvolumes

   1 umount /mnt
   2 mount -o noatime,compress=lzo,space_cache,autodefrag,subvol=/ /dev/sdb4 /mnt
   3 
   4 btrfs subvolume create /mnt/home
   5 
   6 btrfs subvolume snapshot /mnt/root/ /mnt/var_log
   7 find /mnt/var_log -mindepth 1 -maxdepth 1 \! -name var | xargs rm -r --
   8 mv /mnt/var_log/var/log/* /mnt/var_log/; rm -r /mnt/var_log/var/
   9 find /mnt/root/var/log -mindepth 1 -maxdepth 1 | xargs rm -r --
  10 
  11 mount -o noatime,compress=lzo,space_cache,autodefrag /dev/sdb4 /mnt
  12 mount -o noatime,compress=lzo,space_cache,autodefrag,subvol=/home /dev/sdb4 /mnt/home
  13 mount -o noatime,compress=lzo,space_cache,autodefrag,subvol=/var_log /dev/sdb4 /mnt/var/log/
  14 
  15 mount -t btrfs
  16 /dev/sda4 on /mnt type btrfs (rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=258,subvol=/root)
  17 /dev/sda4 on /mnt/home type btrfs (rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=261,subvol=/home)
  18 /dev/sda4 on /mnt/var/log type btrfs (rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=262,subvol=/var_log)

Prepare EFI filesystems

Unnecessary - just skip EFI.

   1 ### USE UPPERCASE FS-LABELS
   2 mkfs.vfat -n EFI_SDA /dev/sda2
   3 mkfs.vfat -n EFI_SDB /dev/sdb2
   4 mkdir /mnt/boot/EFI /mnt/boot/EFI_SDB
   5 mount /mnt/boot/EFI
   6 mount /mnt/boot/EFI_SDB
   7 rsync -a EFI ../EFI_SDB/

fstab

Make sure to use FS-UUIDS and no devices in fstab

   1 blkid
   2 /dev/sdb4: LABEL="rootfs" UUID="0cc274fd-4cb8-4cc7-9f60-f59de41f6891" UUID_SUB="d65eac71-86ca-45e0-b59a-7c872de54e59" TYPE="btrfs" PARTLABEL="root_sdb" PARTUUID="20199b69-b0a7-4151-9dd1-83c741d580a0"
   3 /dev/loop0: UUID="40c4ea95-0ecc-4c51-9f3e-e49d8f62f160" TYPE="ext2"
   4 /dev/sda1: PARTLABEL="bios_grub_sda" PARTUUID="1db5c109-b489-4466-8d51-23ef189253b7"
   5 /dev/sda2: SEC_TYPE="msdos" LABEL_FATBOOT="EFI" LABEL="EFI" UUID="8AC4-4574" TYPE="vfat" PARTLABEL="EFI_sda" PARTUUID="14827a48-5a73-4012-bd28-176d365bd903"
   6 /dev/sda3: UUID="4b81ee7b-00bb-3ce6-e83d-c0c00c449861" UUID_SUB="9e9e8cf9-9b01-6128-e2b6-acf8e04e7e9e" LABEL="rescue:md_swap1" TYPE="linux_raid_member" PARTLABEL="swap1_sda" PARTUUID="95586f2b-45ac-478a-8481-fd176f16cbb5"
   7 /dev/sda4: LABEL="rootfs" UUID="0cc274fd-4cb8-4cc7-9f60-f59de41f6891" UUID_SUB="6c72fb67-6c5d-40bc-a4b4-dd34199a1d2b" TYPE="btrfs" PARTLABEL="root_sda" PARTUUID="20199b69-b0a7-4151-9dd1-83c741d580a0"
   8 /dev/sdb1: PARTLABEL="bios_grub_sdb" PARTUUID="1db5c109-b489-4466-8d51-23ef189253b7"
   9 /dev/sdb2: PARTLABEL="EFI_sdb" PARTUUID="14827a48-5a73-4012-bd28-176d365bd903"
  10 /dev/sdb3: UUID="4b81ee7b-00bb-3ce6-e83d-c0c00c449861" UUID_SUB="981dfa0d-6584-2576-a51a-02102935a87b" LABEL="rescue:md_swap1" TYPE="linux_raid_member" PARTLABEL="swap1_sdb" PARTUUID="95586f2b-45ac-478a-8481-fd176f16cbb5"
  11 /dev/md127: LABEL="md_swap1" UUID="87294740-52c4-4557-b838-ddc44ba8aa4b" TYPE="swap"

Edit fstab to reflect new structure of filesystem

   1 cat /etc/fstab
   2 # /etc/fstab: static file system information.
   3 #
   4 # Use 'blkid' to print the universally unique identifier for a
   5 # device; this may be used with UUID= as a more robust way to name devices
   6 # that works even if disks are added and removed. See fstab(5).
   7 
   8 #<file_system>                             <mount_point>  <type>  <options>                                                                    <dump>  <pass>
   9 UUID=87294740-52c4-4557-b838-ddc44ba8aa4b  none           swap    sw                                                                           0       0
  10 UUID=0cc274fd-4cb8-4cc7-9f60-f59de41f6891  /              btrfs   rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=258,subvol=/root     0       1
  11 UUID=8AC4-4574                             /boot/EFI      vfat    utf8                                                                         0       0
  12 UUID=B3B5-67FA                             /boot/EFI_SDB  vfat    utf8                                                                         0       0
  13 UUID=0cc274fd-4cb8-4cc7-9f60-f59de41f6891  /home          btrfs   rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=261,subvol=/home     0       0
  14 UUID=0cc274fd-4cb8-4cc7-9f60-f59de41f6891  /var/log       btrfs   rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=262,subvol=/var_log  0       2

Validate fstab

   1 findmnt --verify
   2 Success, no errors or warnings detected

Chroot into new system

   1 mount -t proc none /mnt/proc
   2 mount -t tmpfs none /mnt/tmp
   3 
   4 for MNT in /dev /dev/pts /run /sys;do
   5         mount --bind $MNT /mnt/${MNT%/}
   6 done
   7 chroot /mnt /bin/bash

Basic Configuration

Adjust hostname and Mailname

   1 echo 'kvm2' > /etc/hostname
   2 ### DEFINED $myorigin
   3 echo 'kvm2.rockstable.org' > /etc/mailname

Please see also Names#hostnames on this topic.

Reconfigure the hostname and exec a new shell

   1 hostname -F /etc/hostname
   2 exec "$SHELL"

Adjust hosts

/etc/hosts

   1 127.0.0.1       localhost
   2 ::1             localhost ip6-localhost ip6-loopback
   3 ff02::1         ip6-allnodes
   4 ff02::2         ip6-allrouters
   5 
   6 195.201.246.253      kvm2.rockstable.org. kvm2
   7 2a01:4f8:231:702::2  kvm2.rockstable.org. kvm2

Please make sure the canonical name/FQDN is listed in front of any additional aliases or the server may not be able to determine its own domain. Further a canonical name/FQDN should not point to the loopback (lo) interface of the server.

Adjust timezone

   1 dpkg-reconfigure tzdata                                                                                                                                     
   2 
   3 Current default time zone: 'Europe/Berlin'
   4 Local time is now:      Wed May 22 14:22:32 CEST 2019.
   5 Universal Time is now:  Wed May 22 12:22:32 UTC 2019.

Configure apt-sources

   1 cat /etc/apt/sources.list
   2 deb       http://ftp.de.debian.org/debian              buster            main  contrib  non-free
   3 deb-src   http://ftp.de.debian.org/debian/             buster            main  contrib  non-free
   4 deb       http://ftp.de.debian.org/debian/             buster-updates    main  contrib  non-free
   5 deb-src   http://ftp.de.debian.org/debian/             buster-updates    main  contrib  non-free
   6 deb       http://security.debian.org/debian-security/  buster/updates    main  contrib  non-free
   7 deb-src   http://security.debian.org/debian-security/  buster/updates    main  contrib  non-free
   8 #deb      http://ftp.de.debian.org/debian/             buster-backports  main  contrib  non-free
   9 #deb-src  http://ftp.de.debian.org/debian/             buster-backports  main  contrib  non-free
  10

Pinning: apt_preferences

If you create a preferences file like /etc/apt/preferences{,.d/filename{,.pref}} make sure the Pinning blocks are separated by a line, which must not contain any whitespace characters or apt will not respect your preference.

This example has cause (invisible) problems with the ^I Tab-characters between the blocks!

   1 # Debian preferences^I^I$
   2 Package:^I*$    
   3 Pin:^I^Irelease n=jessie$
   4 Pin-Priority:^I500$
   5 ^I^I$
   6 Package:^I*$    
   7 Pin:^I^Irelease n=jessie-updates$
   8 Pin-Priority:^I500$

Check preferences:

   1 apt-cache policy|grep -C1 release

Some notes regarding preferences

If you are using multiple codenames on one system at once (e.g. Buster: 500, Bullseye: 400) you should change the priority of the respective backports to be slightly less than their corresponding codename (e.g. Buster: 490, Bullseye: 390).

Background: If you left them at 100 (default) and you install a package from backports explicitly like an updated kernel, it will be upgraded to the codename with the higher priority when a package of the same name but a higher version is available there. This may not be what you intended.

Debian codenames with suffixes or from debian security should have the same priority as the main codename. This ensures you have the most recent and secure version installed.

If you are switching between packages from different codenames, make sure to mark dependencies as "Installed Automatically" A ("M" in aptitude). Or manually installed packages that are not required by any other manually installed package will end up as garbage. Garbage can be identified using:

   1 aptitude search '~g'

Add some essential packages

   1 apt install \
   2         apt-file aptitude bash-completion byobu btrfs-progs ca-certificates curl \
   3         dmidecode dosfstools git gpm htop iftop info iotop jq libcrack2 locales \
   4         lsb-release lsof man-db mc mlocate openssl parted pigz psmisc pv \
   5         pwgen python-apt python3-apt rsync screen sqlite3 ssl-cert strace sudo \
   6         sysstat tmux vim wget zsh

Configure locales

   1 dpkg-reconfigure locales

Configure vim

Adjust it to your needs like in vim

zsh grml-flavoured

I strongly recommend this config! It's simply awesome. Thanks for this!

   1 git clone git://git.grml.org/grml-etc-core.git /opt/grml-etc-core
   2 mv /etc/zsh{,_dist}
   3 ln -s /opt/grml-etc-core/etc/zsh /etc/zsh
   4 chsh -s /bin/zsh
   5 exec zsh

Networking

ifupdown vs. ifupdown2

Cons

ifupdown2 currently does not support the interfaces-keyword metric provided by ifmetric, which is still required as a package. If you get multiple default routes e.g. via dhcp-client and you don't have access to ifupdown >= 1.2.7-1, you should stick with ifupdown.
- 1.2.5-1 supports metric but has Bug 930839
- 1.2.7-1 Fixed, working.
Changing is service interrupting

Pros

New command ifreload which can change

status of interfaces without taking them down.

Install ifupdown2

You better do it in a tmux session, because ssh-session will break and won't return.

   1 tmux
   2 PRIMARY_INTERFACE="enp1s0"
   3 aptitude install \
   4         ifupdown- \
   5         bridge-utils dnsutils ethtool fail2ban ifupdown2 iputils-tracepath \
   6         isc-dhcp-client net-tools pciutils python-gvgen python-mako \
   7         python-pkg-resources traceroute; \
   8         ifup "$PRIMARY_INTERFACE"

Use Enter ~ . to terminate frozen ssh-clients. Help about ssh escape commands: Enter ~ ?

Predictable device names

www.freedesktop.org: PredictableNetworkInterfaceNames

Let's predict some names.

   1 lspci| grep -i eth
   2 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (2) I219-LM

Okay, so PCI-Address is 00:1f.6

Bus:Device.Function (BDF)

BUS: 0
Device/Slot: 31 = 1*16¹ + 15*16⁰
Function: 6

access.redhat.com: Understanding the predictable network interface device names

So we predict the interface name to be: enp0s31f6 = "en" + "p" + "0" + s + "31" + "f" + "6"

On Upgrade to buster

If the freshly upgraded system still has old interface naming scheme, you may wish to upgrade. To achieve this, you have to remove /etc/udev/rules.d/80-net-setup-link.rules as well as /etc/systemd/network/50-virtio-kernel-names.link and rebuild your initial-ramdisks.

Please also read: /usr/share/doc/udev/README.Debian.gz

   1 rm /etc/udev/rules.d/80-net-setup-link.rules
   2 rm /etc/systemd/network/50-virtio-kernel-names.link
   3 update-initramfs -u

Configure interfaces

   1 cat /etc/network/interfaces
   2 # interfaces(5) file used by ifup(8) and ifdown(8)
   3 
   4 auto lo
   5 iface lo inet loopback
   6 
   7 auto enp0s31f6
   8 iface enp0s31f6 inet static
   9         address         195.201.246.253/26
  10         address         2a01:4f8:231:702::2/64
  11         gateway         195.201.246.193
  12 
  13 # Include files from /etc/network/interfaces.d:
  14 source-directory /etc/network/interfaces.d

Check the configuration of the DNS-resolver

   1 ### CONTROL CONFIGURATION DNS-RESOLVER
   2 cat /etc/resolv.conf

Install openssh-server

   1 aptitude install openssh-server ssh-askpass openssh-client

If you skipped the step of setting up the hostname your should renew the hostkeys or edit the wrong comment.

   1 rm /etc/ssh/ssh_host_*
   2 dpkg-reconfigure openssh-server

Make sure ssh-server starts on boot

   1 systemctl enable ssh
   2 Synchronizing state of ssh.service with SysV service script with /lib/systemd/systemd-sysv-install.
   3 Executing: /lib/systemd/systemd-sysv-install enable ssh

Embed pubkey for authentication

   1 ls /mnt/root/.ssh/authorized_keys
   2 /mnt/root/.ssh/authorized_keys

Make sure root can login via pubkey-auth (default is fine)

   1 #PermitRootLogin prohibit-password
   2

Prepare boot

   1 aptitude install \
   2         acl apparmor-profiles-extra apparmor-utils firmware-linux \
   3         firmware-linux-free grub-pc initramfs-tools \
   4         linux-headers-amd64 linux-image-amd64 mdadm nfs-common

Just forget about EFI-boot in situations where you can't control that machines UEFI/BIOS and stick with grub-pc.

Last safetys infront of reboot

   1 grub-install /dev/sda
   2 i386-pc wird für Ihre Plattform installiert.
   3 installation beendet. Keine Fehler aufgetreten.
   4 grub-install /dev/sdb
   5 i386-pc wird für Ihre Plattform installiert.
   6 installation beendet. Keine Fehler aufgetreten.

   1 update-grub2
   2 Generating grub configuration file ...
   3 Found linux image: /boot/vmlinuz-4.19.0-5-amd64
   4 Found initrd image: /boot/initrd.img-4.19.0-5-amd64
   5 done

   1 update-initramfs -k all -u 
   2 update-initramfs: Generating /boot/initrd.img-4.19.0-5-amd64

Make sure MD-RAID reassebles on next boot

If you assembled the RAID array earlier in the live system, you will have to change the name configuration from the live systems hostname to your new one.

   1 cat /etc/mdadm/mdadm.conf                                                                                                                                                                                                                                                                                                                
   2 # mdadm.conf
   3 #
   4 # !NB! Run update-initramfs -u after updating this file.
   5 # !NB! This will ensure that initramfs has an uptodate copy.
   6 #
   7 # Please refer to mdadm.conf(5) for information about this file.
   8 #
   9 
  10 # by default (built-in), scan all partitions (/proc/partitions) and all
  11 # containers for MD superblocks. alternatively, specify devices to scan, using
  12 # wildcards if desired.
  13 #DEVICE partitions containers
  14 
  15 # automatically tag new arrays as belonging to the local system
  16 HOMEHOST <system>
  17 
  18 # instruct the monitoring daemon where to send mail alerts
  19 MAILADDR root
  20 
  21 # definitions of existing MD arrays
  22 ARRAY /dev/md/md_swap1  metadata=1.2 UUID=4b81ee7b:00bb3ce6:e83dc0c0:0c449861 name=kvm2:md_swap1
  23 
  24 # This configuration was auto-generated on Wed, 22 May 2019 09:56:16 +0000 by mkconf
  25

Create new mdadm.conf

If there is no file your can create a new one which at least assemles your md-arrays on boot.

   1 mdadm --detail --scan >> /etc/mdadm/mdadm.conf

WELL, GOOD LUCK

   1 shutdown -r now

Extending to a hypervisor

libvirt

Some more filesystem

Moving libvirt to own subvolume

   1 ### CREATE A MOUNT POINT
   2 mkdir /media/btrfs5
   3 ### MOUNT ROOT SUBVOLUME TO THIS MOUNT POINT
   4 mount -o rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=5,subvol=/ \
   5         /dev/sda4 /media/btrfs5
   6 ### STOP LIBVIRTD
   7 systemctl stop libvirtd.service
   8 ### CHECK FOR OPEN INODES
   9 lsof /var/lib/libvirt
  10 ### CREATE A SNAPSHOT OF THE FILESYSTEM "root"
  11 ### AND NAME IT "var_lib_libvirt"
  12 btrfs subvolume snapshot \
  13         /media/btrfs5/root \
  14         /media/btrfs5/var_lib_libvirt
  15 ### LIST SUBVOLUMES
  16 btrfs subvol list /media/btrfs5/
  17 ID 258 gen 4933 top level 5 path root
  18 ID 261 gen 1520 top level 5 path home
  19 ID 262 gen 4933 top level 5 path var_log
  20 ID 265 gen 4933 top level 5 path var_lib_libvirt
  21 ### MOUNT FRESHLY CREATED SUBVOLUME TO /var/lib/libvirt
  22 mount -o rw,noatime,compress=lzo,space_cache,autodefrag,subvol=/var_lib_libvirt \
  23         /dev/sda4 /var/lib/libvirt
  24 ### DELETE EVERYTHING BELOW THE NEW MOUNT POINT
  25 ### WHOSE NAME IS NOT "var" RECURSIVELY
  26 find /media/btrfs5/var_lib_libvirt \
  27         -mindepth 1 -maxdepth 1 \! -name var \
  28         |xargs rm -r --
  29 ### MOVE CONTENTS OF THE SUBDIR libvirt
  30 ### TO TOP-LEVEL OF THE SUBVOLUME
  31 mv /media/btrfs5/var_lib_libvirt/var/lib/libvirt/* \
  32         /media/btrfs5/var_lib_libvirt
  33 ### DELETE var IN SUBVOLUME RECURSIVELY
  34 rm -r /media/btrfs5/var_lib_libvirt/var
  35 ### DELETE CONTENT OF libvirt IN THE ROOT-SUBVOLUME RECURSIVELY
  36 rm -r /media/btrfs5/root/var/lib/libvirt/*
  37 ### CHECK OLD DIRECTORY
  38 ll /media/btrfs5/root/var/lib/libvirt/
  39 insgesamt 0
  40 ### CHECK NEW DIRECTORY
  41 ll /media/btrfs5/var_lib_libvirt/
  42 insgesamt 0
  43 drwx--x--x 1 root         root          0 Apr  7 12:36 boot
  44 drwx--x--x 1 root         root          0 Apr  7 12:36 images
  45 drwxr-x--- 1 libvirt-qemu libvirt-qemu 62 Mai 23 10:16 qemu
  46 drwx------ 1 root         root          0 Apr  7 12:36 sanlock
  47 ### MOUNT THE NEW SUBVOLUME TO THE DESTINATION
  48 mount -o rw,noatime,compress=lzo,space_cache,autodefrag,subvol=/var_lib_libvirt \
  49         /dev/sda4 /var/lib/libvirt
  50 ### START LIBVIRTD
  51 systemctl start libvirtd.service

Adjust fstab

   1 # /etc/fstab: static file system information.
   2 #
   3 # Use 'blkid' to print the universally unique identifier for a
   4 # device; this may be used with UUID= as a more robust way to name devices
   5 # that works even if disks are added and removed. See fstab(5).
   6 
   7 #<file_system>                             <mount_point>     <type>  <options>                                                                            <dump>  <pass>
   8 UUID=87294740-52c4-4557-b838-ddc44ba8aa4b  none              swap    sw                                                                                   0       0
   9 UUID=0cc274fd-4cb8-4cc7-9f60-f59de41f6891  /                 btrfs   rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=258,subvol=/root             0       1
  10 #UUID=8AC4-4574                            /boot/EFI         vfat    utf8                                                                                 0       0
  11 #UUID=B3B5-67FA                            /boot/EFI_SDB     vfat    utf8                                                                                 0       0
  12 UUID=0cc274fd-4cb8-4cc7-9f60-f59de41f6891  /var/log          btrfs   rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=262,subvol=/var_log          0       2
  13 UUID=0cc274fd-4cb8-4cc7-9f60-f59de41f6891  /var/lib/libvirt  btrfs   rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=266,subvol=/var_lib_libvirt  0       2
  14 UUID=0cc274fd-4cb8-4cc7-9f60-f59de41f6891  /home             btrfs   rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=261,subvol=/home             0       0
  15 UUID=0cc274fd-4cb8-4cc7-9f60-f59de41f6891  /media/btrfs5     btrfs   rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=5,subvol=/                   0       0

Filesystem maintenance

Please see filesystems/btrfs#Maintenance

ksmtuned

All machines idle? Trade in some CPU and SAVE A WHOLE BUNCH OF MEM!

Redhat: KSM

   1 aptitude install ksmtuned

Information can be aquired under /sys/kernel/mm/ksm

   1 grep -rH "" /sys/kernel/mm/ksm*
   2 echo "Saved mem: $(($(cat /sys/kernel/mm/ksm/pages_sharing)*4/2^1024)) MiB"
   3 
   4 ### Or as watch
   5 watch -n 1 -- '\
   6         grep -rH "" /sys/kernel/mm/ksm*; \
   7         echo "Saved mem: $(($(cat /sys/kernel/mm/ksm/pages_sharing)*4/(2^1024))) MiB"'

Run ksm earlier by lowering KSM_THRES_COEF in
/etc/ksmtuned.conf

   1 # Configuration file for ksmtuned.
   2 
   3 # How long ksmtuned should sleep between tuning adjustments
   4 # KSM_MONITOR_INTERVAL=60
   5 
   6 # Millisecond sleep between ksm scans for 16Gb server.
   7 # Smaller servers sleep more, bigger sleep less.
   8 # KSM_SLEEP_MSEC=10
   9 
  10 # KSM_NPAGES_BOOST=300
  11 # KSM_NPAGES_DECAY=-50
  12 # KSM_NPAGES_MIN=64
  13 # KSM_NPAGES_MAX=1250
  14 
  15 #KSM_THRES_COEF=20
  16 KSM_THRES_COEF=50
  17 # KSM_THRES_CONST=2048
  18 
  19 # uncomment the following if you want ksmtuned debug info
  20 
  21 # LOGFILE=/var/log/ksmtuned
  22 # DEBUG=1

Example - Debian Buster monoculture in production:

   1 Every 1,0s: grep -rH "" /sys/kernel/mm/ksm*;echo "Saved mem: $(($(cat /sys/kernel/mm/ksm/pages_sharing)*4/(2^1024))) MiB"                                                                                                                                              kvm2: Fri Jan 31 14:03:57 2020
   2 
   3 /sys/kernel/mm/ksm/stable_node_dups:5938
   4 /sys/kernel/mm/ksm/max_page_sharing:256
   5 /sys/kernel/mm/ksm/pages_volatile:271474
   6 /sys/kernel/mm/ksm/stable_node_chains_prune_millisecs:2000
   7 /sys/kernel/mm/ksm/merge_across_nodes:1
   8 /sys/kernel/mm/ksm/pages_unshared:5277423
   9 /sys/kernel/mm/ksm/stable_node_chains:497
  10 /sys/kernel/mm/ksm/pages_shared:1049263
  11 /sys/kernel/mm/ksm/use_zero_pages:0
  12 /sys/kernel/mm/ksm/pages_to_scan:1250
  13 /sys/kernel/mm/ksm/sleep_millisecs:10
  14 /sys/kernel/mm/ksm/run:1
  15 /sys/kernel/mm/ksm/full_scans:153
  16 /sys/kernel/mm/ksm/pages_sharing:5635769
  17 Saved mem: 21971 MiB

Well, ~22GiB of 64GiB saved! WTF?!1!! I dare to claim, this is a must have!

I assume KSM doesn't search already swapped pages to save expensive disk io. Maybe it's a good idea to gain some memory in a first run and to move content of the swap space back to the main memory in a second step by umounting and mounting swap-space.

   1 swapoff -a; swapon -a

In addition it may be a good idea to reduce vm.swappiness to once more profit from KSM.

OpenvSwitch

To attach out VMs to the network we can use serveral approaches:

Shared hostbridge
1. classical linux bridge
2. openvswitch
network device passthrough
1. physical device
2. virtual device funtion
macvtap direct connection
1. with vepa
2. without vepa
libvirt networks
1. NAT based network
2. Routed network config
3. Isolated network config
4. Isolated IPv6 network config
5. Network config with no gateway addresses

Since i have no contraints or requirements, i like to use openvswitch as a bridge (as "OpenStack would probably probably use it).

Install

   1 aptitude install openvswitch-switch

Add vSwitch

Create bridge

   1 ovs-vsctl add-br ovs-virt

Add fakebridges

Fake bridges are just virtual sub bridges of a parent bridge, that assign an attached port to a specific vlan in access mode.

   1 ovs-vsctl add-br ovs-1a ovs-virt 1000   #public dmz
   2 ovs-vsctl add-br ovs-1n ovs-virt 1500   #extranet
   3 ovs-vsctl add-br ovs-2a ovs-virt 2000   #intranet
   4 ovs-vsctl add-br ovs-2n ovs-virt 2500   #secure zone
   5 ovs-vsctl add-br ovs-mon1 ovs-virt 3000 #monitoring
   6

Configure new interfaces

Conventional networks

One public interface used:

for general conectivity
in routing the public network to the host and
as a NAT-address for the internal private networks.

/etc/network/interfaces

   1 auto lo
   2 iface lo inet loopback
   3 
   4 ### OUTSIDE
   5 auto enp0s31f6
   6 iface enp0s31f6 inet static
   7         address         195.201.246.253/26
   8         address         2a01:4f8:231:702::2/64
   9         gateway         195.201.246.193
  10         alias           "OUTSIDE"
  11 
  12 ### FAKE BRIDGES
  13 ### VLANS 500-999
  14 auto ovs-pub1
  15 iface ovs-pub1 inet static
  16   address       178.63.149.225/28
  17   alias         "fake-bridge: public dmz - public network"
  18 
  19 ### VLANS 1000-1499
  20 auto ovs-1a
  21 iface ovs-1a inet static
  22   address       172.18.0.1/24
  23   alias         "fake-bridge: public dmz - private network"
  24 
  25 ### VLANS 1500-1999
  26 auto ovs-1n
  27 iface ovs-1n inet static
  28   address       172.18.64.1/24
  29   alias         "fake-bridge: extranet dmz"
  30 
  31 ### VLANS 2000-2499
  32 auto ovs-2a
  33 iface ovs-2a inet static
  34   address       172.18.128.1/24
  35   alias         "fake-bridge: intranet dmz"
  36 
  37 ### VLANS 2500-2999
  38 auto ovs-2n
  39 iface ovs-2n inet static
  40   address       172.18.192.1/24
  41   alias         "fake-bridge: secure zone"
  42 
  43 ### VLANS 3000-3499
  44 auto ovs-mon1
  45 iface ovs-mon1 inet static
  46   address       172.19.255.1/24
  47   alias         "fake-bridge: monitoring"

PPP Guest-Host

With point-to-point connection from guest to host. This is useful:

for single IPs or
to reduce overhead in small networks (/30) originating from network and broadcast addresses.

Please don't get irritated by the last octet of the IP-addresses.

Create a fake-brigde on the host and assign a private IP-address, set a route (for the public address) to the guest and optionally set a static arp entry.

   1 ### external1
   2 auto ovs-ext1
   3 iface ovs-ext1 inet static
   4   address       172.31.255.252
   5   netmask       255.255.255.255
   6   up            ip r add 176.9.99.252/32 dev ovs-ext1
   7   #up           arp -v -H ether -i ovs-ext1 -s 176.9.99.252 52:54:00:aa:bb:cc
   8

In the guest assign the public IP-address and create a point-to-point connection to the (private address of the) host.

   1 auto ens9
   2 iface ens9 inet static
   3         address         176.9.99.252
   4         netmask         255.255.255.255
   5         #broadcast      172.31.255.252
   6         #Note the spelling of pointopoint
   7         pointopoint     172.31.255.252
   8         gateway         172.31.255.252
   9         dns-nameserver  172.31.255.252

Firewalling

   1 aptitude install iptables-persistent ipset iptables fwbuilder

Configure fwbuilder

   1 addgroup firewall
   2 adduser --ingroup firewall --home /etc/fwbuilder --gecos 'remote firewall admin,,,' --system firefighter

   1 cat /etc/sudoers.d/firewall
   2 firefighter     ALL= (root:root) NOPASSWD: /etc/init.d/firewall, /usr/sbin/ipset, /sbin/modprobe, /sbin/iptables

Performance

Laptop

To save power on your laptop and therefore increase time that may be spend on battery, install the laptop-mode-tools. They will e.g. set the cpu-frequency scaling_governor to ondemand and other more conservative options. In my specific case battery usage was reduced by 1/3 and time on battery increased by factor 3/2, which is significant.

   1 aptitude install laptop-mode-tools

They can be configured in /etc/laptop-mode/.

With powertop settings consuming too much power may be configured.

CPU govenour

Another way to adjust cpu-frequency scaling_governor is via cpufrequtils.

/etc/default/cpufrequtils

   1 ENABLE="true"
   2 GOVERNOR="ondemand"
   3 MAX_SPEED="0"
   4 MIN_SPEED="0"

IO-Scheduler

Linux#IO-Scheduler

Security

CPU microcode

Microcode updates are ephemeral: they will be lost after a processor hard reset or after the processor is powered off. They must be reapplied at every boot and after the system wakes up from suspend to RAM or disk.

Depending on your CPU vendor install either of the following:

   1 aptitude install amd64-microcode
   2 aptitude install intel-microcode

To force a microcode update at runtime (on your own risk) run as root.

   1 echo 1 > /sys/devices/system/cpu/microcode/reload

To omit loading of the microcode at boot time add dis_ucode_ldr to your kernel command line in grub menu editor.

   1 linux   /boot/vmlinuz-5.4.0-4-amd64 root=UUID=75258d3e-37f9-42f7-9187-444be692f85d ro quiet dis_ucode_ldr

You may configure tthe microcode packages in

/etc/default/amd64-microcode
/etc/default/intel-microcode

Early loading microcode maybe blacklisted by

/etc/modprobe.d/amd64-microcode
/etc/modprobe.d/intel-microcode

You can get the running microcode revision from /proc/cpuinfo

   1 grep -E 'stepping|model|microcode' /proc/cpuinfo

Compare the latest manufacturer microcode update guidance document.

Automatic security upgrades

   1 aptitude install unattended-upgrades

Check ephemeral pinning with

   1 unattended-upgrades --dry-run --debug --verbose

You may run unattended-upgrades with debugging output in foreground, to check what it is doing.

   1 unattended-upgrades -d

If it is slow

check if you have packages on hold apt-mark showhold or broken packages

Too solve the problem use aptitude, it tends to be more powerful in untangling dependency problems.

Check OS Integrity

   1 dpkg -V

debsecan

   1 aptitude install debsecan

Is run automatically via a cronjob, but you should add the parameter --suite

   1 # cron entry for debsecan
   2 MAILTO=root
   3 
   4 4 * * * * daemon test -x /usr/bin/debsecan \
   5         && /usr/bin/debsecan --cron --suite "$(lsb_release -cs)"
   6 # (Note: debsecan delays actual processing past 2:00 AM, and runs only
   7 # once per day.)
   8

Mobile security

Goal is to protect as much as possible as early and strong as possible.

Requirements:

basics:
- bios-password is set
- hdd-password is set
mandatory:
- be maintainable
- perform well
- flexible sizing like in LVM2/btrfs
- include swap, root, …
- protect kernel/initrd
optionally:
- include /boot, means to enter the password a 3rd time in grub2 to unlock /boot
- use trusted platform module (tpm)

The inspiration:

This ultimatively leads to multi-layer full disk encryption (OPAL+LUKS).

ATA hard drive password

ATA Security Feature Set or ATA Security (since ATA-3 (1996–2002, ANSI X3.298-1997))

Tools

hdparm

About

Available on disks with AT Attachment (ATA) storage interface, this includes SATA, SCSI/SAS?, NVMe

"first comer” ownership model
- first that sets the password owns the device
access control mechanism only
32byte master and user keys (NULL-Byte padded)
high or maximum security mode

Mode high

In High security mode, the device can be unlocked with either the User or Master password, using the "SECURITY UNLOCK DEVICE" ATA command. There is an attempt limit, normally set to 5, after which the disk must be power cycled or hard-reset before unlocking can be attempted again. Also in High security mode, the SECURITY ERASE UNIT command can be used with either the User or Master password.

Mode maximum

In Maximum security mode, the device can be unlocked only with the User password. If the User password is not available, the only remaining way to get at least the bare hardware back to a usable state is to issue the SECURITY ERASE PREPARE command, immediately followed by SECURITY ERASE UNIT. In Maximum security mode, the SECURITY ERASE UNIT command requires the Master password and will completely erase all data on the disk. Word 89 in the IDENTIFY response indicates how long the operation will take.

Opal

Before building check if your drive supports Opal.

   1 lspci -vvv

Tools

msed (manage self-encrypting devices) and OpalTool, the two known Open Source code bases available for self-encrypting drives support on Linux, have both been retired, and their development efforts officially merged to form sedutil, under the umbrella of The Drive Trust Alliance (DTA). sedutil is "an Open Source (GPLv3) effort to make Self Encrypting Drive technology freely available to everyone."

https://github.com/Drive-Trust-Alliance/sedutil

Build sedutil

   1 mkdir ~/workspace/
   2 cd ~/workspace
   3 git clone https://github.com/Drive-Trust-Alliance/sedutil.git
   4 cd sedutil
   5 sudo aptitude install build-essential autoconf
   6 autoreconf -i
   7 autoconf
   8 ./configure --prefix=/usr/local
   9 make
  10 #make clean
  11

Scan for drives

   1 % sudo ./sedutil-cli --scan 
   2 [sudo] Passwort für tobias: 
   3 Scanning for Opal compliant disks
   4 /dev/nvme0 No  HFM512GDHTNG-8710B                       80010C00
   5 Unable to verify Kernel flag libata.allow_tpm 
   6 /dev/sda   No  SanDisk pSSD                            6EB 1030
   7 No more disks present ending scan

Yeah, it works and
as the no in column states my current NVMe does not support Opal. :-/ I'll come back once i got a Opal drive.

About

Opal “Family” of specifications:

Opal
Opalite
Pyrite

Found on Arch-Wiki - Self-Encrypting_Drives: Self-encrypting drives adhering to the TCG OPAL 2.0 standard specification (almost all modern self-encrypting drives) implement key management via an authentication key, and a 2nd-level data encryption key, both stored in the disk controller. The data encryption key is the key against which the data is actually encrypted/decrypeted. The authentication key is the user-facing 1st-level password/passphrase which decrypts the data encryption key (which in turn decrypts the data). Data writen to the disk is always encrypted. This approach has specific advantages:

Allows the user to change the passphrase without losing the existing encrypted data on the disk.
- This improves security, as it is fast and easy to respond to security threats and revoke a compromised passphrase
Facilitates near-instant and cryptographically secure full disk erasure.

For those who are familiar; this concept is similar to the LUKS key management layer often used in a dm-crypt deployment. Using LUKS, the user can effectively have up to 8 different key-files / passphrases to decrypt the encryption key, which in turn decrypts the underlying data. This approach allows the user to revoke / change these key-files / passphrases as required without needing to re-encrypt the data, as the 2nd-level encryption key is unchanged (itself being re-encrypted by the new passphrase).

In fact, in drives featuring full-disk encryption, data is always encrypted with the data encryption key when stored to disk, even if there is no password set (e.g. a new drive). Manufacturers do this to make it easier for users who are not able to, or do not wish to enable the security features of the self-encrypting drive. This can be thought of as all drives by default having a zero-length password that transparently encrypts/decrypts the data always (similar to how passwordless SSH keys provide (somewhat) secure access without user intervention).

The key point to note is that if at a later stage the user wishes to "enable" encryption, they can configure the passphrase (authentication key), which will then be used to encrypt the existing data encryption key (thus prompting for passphrase before decrypting the data encryption key in future). However, as the existing data encryption key will not be changed (regenerated), this in effect locks the drive, while preserving the existing encrypted data on the disk.

Advantages:

Easier to setup (compared to software-based encryption)
Notably transparent to the user, except for initial bootup authentication
Data-at-Rest protection
Increased performance (CPU is freed up from encryption/decryption calculations)
The main CPU and RAM are eliminated as possible attack targets
Optimally fast and #Secure disk erasure (sanitation) (regardless of disk size)
Protection from alternative boot methods due to the possibility to encrypt the MBR, rendering the drive inaccessible before pre-boot authentication

Disadvantages:

Constant-power exploits:
Typical self-encrypting drives, once unlocked, will remain unlocked as long as power is provided. This vulnerability can be exploited by means of altering the environment external to the drive, without cutting power, in effect keeping the drive in an unlocked state. For example, it has been shown (by researchers at University of Erlangen-Nuremberg) that it is possible to reboot the computer into an attacker-controlled operating system without cutting power to the drive. The researchers have also demonstrated moving the drive to another computer without cutting power.[1]
Key-in-memory exploits:
When the system is powered down into S3 ("sleep") mode, the drive is powered down, but the drive keeps access to the encryption key in its internal memory (NVRAM) to allow for a resume ("wake"). This is necessary because for system booted with an arbitrary operating system there is no standard mechanism to prompt the user to re-enter the pre-boot decryption passphrase again. An attacker (with physical access to the drive) can leverage this to access the drive. Taking together known exploits the researchers summarize "we were able to break hardware-based full-disk encryption on eleven [of twelve] of those systems provided they were running or in standby mode".[2] Note, however, S3 ("sleep") is not currently supported by sedutil (the current available toolset for managing a TCG OPAL 2.0 self-encrypting drives via Linux)
Compromised firmware:
The firmware of the drive may be compromised (backdoor) and data sent to it thus potentially compromised (decryptable by the malicious third party in question, provided access to physical drive is achievable). A study demonstrated methods for compromising device firmware, as well as applying invalid passwords to access data on OPAL devices.[3] If data is encrypted by the operating system (e.g. dm-crypt), the encryption key is unknown to the compromised drive, thus circumventing this attack vector entirely.

Found in White Paper - Storage Opal and NVMe

requires AES-128 or AES-256
hardware-based encrpytion that may be scaled to meet the bandwidth of the storage device.
credentials
- 1-4 admin for provisioning, configuration or erasure
- 2-8 user to perform various actions
subdivision of the storage device into multiple locking ranges of contiguous LBAs
each locking range
- has different media encryption key (MEK)
- is unlocked independently
- is erased independently (by destruction of media encryption key and generation of a new one)
n>=0 users may
- unlock locking ranges
- erase locking ranges
fast and reliable erasure of locking ranges
supports MBR-shadowding, through which a host-application can store and execute a “Pre-Boot Authentication Environment”. Such a mechanism is necessary to allow unlock of the range in which the OS is stored, in order to allow the OS to boot.

Storage glossary

advanced encryption standard (AES)
authentication key (AK)
advanced technology attachment (ATA) -> P-ATA
AT attachment packet interface (ATAPI)
data encryption key (DEK)
full disk encryption (FDE)
full encryption disks (FED), self-encrypting HDD
hard drive disk (HDD)
integrated drive electronics (IDE)
logical block addressing (LBA), number of blocks starting with zero (size with 512byte blocks)
- 28bit LBA (128GiB)
- 48bit LBA (128PiB)
- SCSI Command Descriptor Block (CDB)
  - 10Byte CDB with 4byte (32bit) LBA (2TiB)
  - 16/32Byte CDB with 8byte (64bit) Long-LBA (16EiB)
non-volatile memory express (NVMe)
opal security subsystem class (SSC)
Self-Monitoring, Analysis and Reporting Technology (S.M.A.R.T.)
storage device (SD)
self-encrypting drive (SED)
small computer system interface (SCSI)
trusted computing group (TCG)

LUKS

Tools

   1 apt install gdisk parted cryptsetup cryptsetup-initramfs dosfstools xfsprogs

About

The Linux Unified Key Setup (LUKS) is a disk encryption specification created by Clemens Fruhwirth in 2004 and was originally intended for Linux.

Why LUKS?

compatibility via standardization,
secure against low entropy attacks,
support for multiple keys,
effective passphrase revocation,
free.

Transform

Target configuration

   1 NAME                    FSTYPE      LABEL  FSAVAIL FSUSE% MOUNTPOINT
   2 nvme0n1
   3 ├─nvme0n1p1             vfat               
   4 ├─nvme0n1p2             vfat        EFI    213.8M    16% /mnt/boot/efi
   5 ├─nvme0n1p3
   6 └─nvme0n1p4             crypto_LUKS
   7   └─crypt1              LVM2_member
   8     ├─vg_crypt-lv_swap1 swap        swap1  
   9     └─vg_crypt-lv_root  xfs         rootfs 355.7G    14% /mnt

FDE walkthrough

Perform a backup Preparation

   1 ### MOUNT A REMOTE STORAGE (e.g. NFS4)
   2 mount libertas:/media/space/tmp /media/external1
   3 
   4 ### AND BACKUP PARTITIONING TABLE (JUST INCASE)
   5 mkdir /media/external1/backup
   6 sgdisk --backup=/media/external1/backup/nvme0n1.sgdisk /dev/nvme0n1
   7 The operation has completed successfully.
   8 ### TO RESTORE SIMPLY
   9 # sgdisk -l /media/external1/backup/nvme0n1 /dev/nvme0n1
  10

Partitioning

   1 ### watch -n1 -- 'lsblk -f; echo; blkid'
   2 
   3 parted /dev/nvme0n1
   4 unit MiB
   5 print free
   6 mktable gpt
   7 mkpart bios_grub 1 2
   8 mkpart EFI 2 256
   9 mkpart boot1 256 1024
  10 mkpart crypt1 4 -1
  11 set 1 bios_grub on
  12 set 2 esp on
  13 set 2 boot on
  14 print free
  15 quit

Grub2 disk: Implement support for LUKS2:

With cryptsetup 2.0, a new version of LUKS was introduced that breaks compatibility with the previous version due to various reasons. GRUB currently lacks any support for LUKS2, making it impossible to decrypt disks encrypted with that version. This commit implements support for this new format.

This commit has not landed in Debian, yet. So we need to downgrade header to LUKS version 1 and the password based key derivation functions (PBKDF) to PBKDF2 (from Argon2i or Argon2id).

See also the documentation of the cryptsetup-team of Debian.net

Fresh cryptsetup

   1 cryptsetup luksFormat --type luks1 /dev/nvme0n1p4
   2 cryptsetup open /dev/nvme0n1p4 crypt1

To downgrade manually from LUKS2 to version 1

   1 cryptsetup luksDump /dev/nvme0n1p4 \
   2         | grep -i -e version -e PBKDF *
   3 cryptsetup luksConvertKey --pbkdf pbkdf2 /dev/nvme0n1p4
   4 Enter passphrase for keyslot to be converted: 
   5 cryptsetup convert --type luks1 /dev/nvme0n1p4

LVM2

   1 pvcreate /dev/mapper/crypt1
   2 vgcreate vg_crypt /dev/mapper/crypt1
   3 lvcreate -L 64g -n lv_swap1 vg_crypt
   4 lvcreate -l 100%FREE -n lv_root vg_crypt
   5 lvrename vg_crypt lv_root1 lv_root

Formating filesystems and swap

   1 mkfs.vfat -n EFI /dev/nvme0n1p2
   2 mkfs.xfs -L rootfs /dev/vg_crypt/lv_root
   3 mkswap -L swap1 /dev/vg_crypt/lv_swap1
   4 mount /dev/vg_crypt/lv_root /mnt
   5 mkdir /mnt/boot/efi
   6 mount /dev/nvme0n1p2 /mnt/boot/efi

Restore the data from backup.
I created the tar-archive without changing the directory, so i have to strip "mnt/" away at extraction-time using --strip-components=1.

   1 ### RESTORE ROOTFS
   2 MOUNT_POINT="/mnt"
   3 ssh user@target.host \
   4         "cat path/to/archive.tar.gz" \
   5         |pigz -dc \
   6         |tar -xf - --strip-components=1 -C "$MOUNT_POINT"
   7 ### RESTORE EFIFS
   8 MOUNT_POINT="/mnt/boot/efi"
   9 ssh user@target.host \
  10         "cat path/to/archive_efi.tar.gz" \
  11         |pigz -dc \
  12         |tar -xf - --strip-components=1 -C "$MOUNT_POINT"

   1 grml-chroot /mnt
   2 blkid |sed 's/^/#/' >> /etc/fstab
   3 ### EDIT /etc/fstab
   4 mount -a

   1 aptitude install cryptsetup cryptsetup-initramfs

Change swap uuid to new value

   1 blkid /dev/nvme0n1p2 \
   2         |sed 's/^/#/' >> /etc/initramfs-tools/conf.d/resume

/etc/initramfs-tools/conf.d/resume

   1 RESUME=UUID=3c8d7d58-c524-4a74-94f6-ec66a3bb07af

Adjust crypttab

   1 blkid /dev/nvme0n1p4 |sed 's/^/#/' >> /etc/crypttab

/etc/crypttab

   1 #<target_name>  <source_device>                            <key_file>  <options>
   2 crypt1          UUID=cb58fb88-fc1e-4102-bfdb-9d943b1f01db  none        luks,discard

/etc/default/grub

   1 # If you change this file, run 'update-grub' afterwards to update
   2 # /boot/grub/grub.cfg.
   3 # For full documentation of the options in this file, see:
   4 #   info -f grub -n 'Simple configuration'
   5 
   6 GRUB_DEFAULT=0
   7 GRUB_TIMEOUT=5
   8 GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
   9 GRUB_CMDLINE_LINUX_DEFAULT="quiet"
  10 GRUB_CMDLINE_LINUX=""
  11 
  12 # Uncomment to enable BadRAM filtering, modify to suit your needs
  13 # This works with Linux (no patch required) and with any kernel that obtains
  14 # the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
  15 #GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"
  16 
  17 # Uncomment to disable graphical terminal (grub-pc only)
  18 #GRUB_TERMINAL=console
  19 
  20 # The resolution used on graphical terminal
  21 # note that you can use only modes which your graphic card supports via VBE
  22 # you can see them in real GRUB with the command `vbeinfo'
  23 #GRUB_GFXMODE=640x480
  24 
  25 # Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
  26 #GRUB_DISABLE_LINUX_UUID=true
  27 
  28 # Uncomment to disable generation of recovery mode menu entries
  29 #GRUB_DISABLE_RECOVERY="true"
  30 
  31 # Uncomment to get a beep at grub start
  32 #GRUB_INIT_TUNE="480 440 1"
  33 
  34 ### GRUB-INSTALL DEMANDS IT
  35 GRUB_ENABLE_CRYPTODISK=y
  36 #GRUB_PRELOAD_MODULES="lvm cryptodisk mdraid1x luks"
  37

   1 update-initramfs -k all -u
   2 grub-install /dev/nvme0n1
   3 update-grub2

Unmount remote storage
Reboot

Plymouth

Install bootsplash "plymouth"

   1 aptitude install plymouth plymouth-themes plymouth-x11

Add splash to GRUB_CMDLINE_LINUX_DEFAULT /etc/default/grub

   1 GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"

   1 update-initramfs -k all -u
   2 update-grub2

List and set available themes

   1 plymouth-set-default-theme --list

Set other theme and update grub and initramfs

   1 sudo plymouth-set-default-theme -R lines

Get new themes from store.kde.org.
Just unpack them to /usr/share/plymouth/themes

Preview plymouth themes in X11

Inspired by this blogpost

plymouth_preview.sh

   1 #!/bin/bash
   2 
   3 ## Preview Plymouth Splash ##
   4 ##      by _khAttAm_       ##
   5 ##    www.khattam.info     ##
   6 ##    License: GPL v3      ##
   7 
   8 chk_root () {
   9   if [ ! $( id -u ) -eq 0 ]; then
  10     echo "Must be run as root"
  11     exit
  12   fi
  13 }
  14 
  15 chk_root
  16 
  17 DURATION=$1
  18 if [ $# -ne 1 ]; then
  19   DURATION=5
  20 fi
  21 
  22 #CURRENT_THEME=
  23 THEMES="$(plymouth-set-default-theme --list \
  24         |sort |uniq)"
  25 
  26 while read THEME; do
  27         plymouth-set-default-theme "$THEME"
  28         echo "$THEME"
  29         sleep 1.5
  30         plymouthd
  31         plymouth --show-splash
  32         for ((I=0; I<$DURATION; I++)); do
  33                 plymouth --update=test$I;
  34                 sleep 1;
  35         done
  36         plymouth quit
  37 done <<< "$THEMES"

   1 chmod a+x ./plymouth_preview.sh
   2 sudo ./plymouth_preview.sh

Debian CD-Image with jigdo

Jigsaw Download, or short jigdo, is a tool designed to ease the distribution of very large files over the internet, for example CD or DVD images. Its aim is to make downloading the images as easy for users as a click on a direct download link in a browser, while avoiding all the problems that server administrators have with hosting such large files.

   1 aptitude install jigdo-file jigit

Example: Create a CD-image of Debian Jessie

Create a configuration for jigdo-lite and to reliefe primary Debian mirrors. ~/.jigdo-lite

   1 debianMirror='http://debian.inf.tu-dresden.de/debian/'
   2 nonusMirror='http://debian.inf.tu-dresden.de/debian/'

   1 ---- /!\ '''Edit conflict - your version:''' ----
   2 
   3 ---- /!\ '''End of edit conflict''' ----
   4 mkdir jessie
   5 cd jessie
   6 jigdo-lite --noask 'https://cdimage.debian.org/cdimage/archive/8.0.0/amd64/jigdo-cd/debian-8.0.0-amd64-netinst.jigdo'

Done:

   1 jigdo-lite --noask 'https://cdimage.debian.org/cdimage/archive/8.0.0/amd64/jigdo-cd/debian-8.0.0-amd64-netinst.jigdo'
   2 
   3 Jigsaw Download "lite"
   4 Copyright (C) 2001-2005  |  jigdo@
   5 Richard Atterer          |  atterer.org
   6 Loading settings from `/home/tobias/.jigdo-lite'
   7 
   8 Downloading .jigdo file
   9 --2019-11-05 11:47:20--  https://cdimage.debian.org/cdimage/archive/8.0.0/amd64/jigdo-cd/debian-8.0.0-amd64-netinst.jigdo
  10 Auflösen des Hostnamens cdimage.debian.org (cdimage.debian.org)… 194.71.11.173, 194.71.11.165, 2001:6b0:19::165, ...
  11 Verbindungsaufbau zu cdimage.debian.org (cdimage.debian.org)|194.71.11.173|:443 … verbunden.
  12 HTTP-Anforderung gesendet, auf Antwort wird gewartet … 200 OK
  13 Länge: 24660 (24K)
  14 Wird in »debian-8.0.0-amd64-netinst.jigdo« gespeichert.
  15 
  16 debian-8.0.0-amd64-netinst.jigdo                                          100%[==================================================================================================================================================================================>]  24.08K  --.-KB/s    in 0.06s   
  17 
  18 2019-11-05 11:47:20 (428 KB/s) - »debian-8.0.0-amd64-netinst.jigdo« gespeichert [24660/24660]
  19 
  20 
  21 -----------------------------------------------------------------
  22 Images offered by `https://cdimage.debian.org/cdimage/archive/8.0.0/amd64/jigdo-cd/debian-8.0.0-amd64-netinst.jigdo':
  23   1: 'Debian GNU/Linux 8.0.0 "Jessie" - Official amd64 NETINST Binary-1 20150425-12:50 (20150425)' (debian-8.0.0-amd64-netinst.iso)
  24 
  25 Further information about `debian-8.0.0-amd64-netinst.iso':
  26 Generated on Sat, 25 Apr 2015 12:53:05 +0000
  27 
  28 -----------------------------------------------------------------
  29 If you already have a previous version of the CD you are
  30 downloading, jigdo can re-use files on the old CD that are also
  31 present in the new image, and you do not need to download them
  32 again. Mount the old CD ROM and enter the path it is mounted under
  33 (e.g. `/mnt/cdrom').
  34 Alternatively, just press enter if you want to start downloading
  35 the remaining files.
  36 Files to scan: 
  37 
  38 -----------------------------------------------------------------
  39 The jigdo file refers to files stored on Debian mirrors. Please
  40 choose a Debian mirror as follows: Either enter a complete URL
  41 pointing to a mirror (in the form
  42 `ftp://ftp.debian.org/debian/'), or enter any regular expression
  43 for searching through the list of mirrors: Try a two-letter
  44 country code such as `de', or a country name like `United
  45 States', or a server name like `sunsite'.
  46 Debian mirror [http://debian.inf.tu-dresden.de/debian/]: 
  47 
  48 Downloading .template file
  49 --2019-11-05 11:47:20--  https://cdimage.debian.org/cdimage/archive/8.0.0/amd64/jigdo-cd/debian-8.0.0-amd64-netinst.template
  50 Auflösen des Hostnamens cdimage.debian.org (cdimage.debian.org)… 194.71.11.165, 194.71.11.173, 2001:6b0:19::173, ...
  51 Verbindungsaufbau zu cdimage.debian.org (cdimage.debian.org)|194.71.11.165|:443 … verbunden.
  52 HTTP-Anforderung gesendet, auf Antwort wird gewartet … 302 Found
  53 Platz: https://saimei.ftp.acc.umu.se/cdimage/archive/8.0.0/amd64/jigdo-cd/debian-8.0.0-amd64-netinst.template [folgend]
  54 --2019-11-05 11:47:20--  https://saimei.ftp.acc.umu.se/cdimage/archive/8.0.0/amd64/jigdo-cd/debian-8.0.0-amd64-netinst.template
  55 Auflösen des Hostnamens saimei.ftp.acc.umu.se (saimei.ftp.acc.umu.se)… 194.71.11.138, 2001:6b0:19::138
  56 Verbindungsaufbau zu saimei.ftp.acc.umu.se (saimei.ftp.acc.umu.se)|194.71.11.138|:443 … verbunden.
  57 HTTP-Anforderung gesendet, auf Antwort wird gewartet … 200 OK
  58 Länge: 8641689 (8.2M)
  59 Wird in »debian-8.0.0-amd64-netinst.template« gespeichert.
  60 
  61 debian-8.0.0-amd64-netinst.template                                       100%[=========================================================>]   8.24M  1.11MB/s    in 7.5s    
  62 
  63 2019-11-05 11:47:28 (1.10 MB/s) - »debian-8.0.0-amd64-netinst.template« gespeichert [8641689/8641689]
  64 
  65 
  66 -----------------------------------------------------------------
  67 Merging parts from `file:' URIs, if any...
  68 0 der 813 vom Template benötigten Dateien gefunden
  69 Es wird keine Image-Datei oder temporäre Datei erzeugt - versuchen Sie es mit anderen Eingabedateien
  70 --2019-11-05 11:47:28--  http://debian.inf.tu-dresden.de/debian/pool/main/s/systemd/libpam-systemd_215-17_amd64.deb
  71 Auflösen des Hostnamens debian.inf.tu-dresden.de (debian.inf.tu-dresden.de)… 141.76.2.4
  72 Verbindungsaufbau zu debian.inf.tu-dresden.de (debian.inf.tu-dresden.de)|141.76.2.4|:80 … verbunden.
  73 HTTP-Anforderung gesendet, auf Antwort wird gewartet … 404 Not Found
  74 2019-11-05 11:47:28 FEHLER 404: Not Found.
  75 
  76 --2019-11-05 11:47:28--  http://debian.inf.tu-dresden.de/debian/pool/main/j/jfsutils/jfsutils_1.1.15-2.1_amd64.deb
  77 Wiederverwendung der bestehenden Verbindung zu debian.inf.tu-dresden.de:80.
  78 HTTP-Anforderung gesendet, auf Antwort wird gewartet … 200 OK
  79 Länge: 285760 (279K) [application/x-debian-package]
  80 Wird in »./debian-8.0.0-amd64-netinst.iso.tmpdir/debian.inf.tu-dresden.de/debian/pool/main/j/jfsutils/jfsutils_1.1.15-2.1_amd64.deb« gespeichert.
  81 
  82 debian.inf.tu-dresden.de/debian/pool/main/j/jfsutils/jfsutils_1.1.15-2.1_ 100%[=========================================================>] 279.06K   894KB/s    in 0.3s    
  83 
  84 2019-11-05 11:47:29 (894 KB/s) - »./debian-8.0.0-amd64-netinst.iso.tmpdir/debian.inf.tu-dresden.de/debian/pool/main/j/jfsutils/jfsutils_1.1.15-2.1_amd64.deb« gespeichert [285760/285760]
  85 
  86 --2019-11-05 11:47:29--  http://debian.inf.tu-dresden.de/debian/pool/main/l/less/less_458-3_amd64.deb
  87 Wiederverwendung der bestehenden Verbindung zu debian.inf.tu-dresden.de:80.
  88 HTTP-Anforderung gesendet, auf Antwort wird gewartet … 200 OK
  89 Länge: 124466 (122K) [application/x-debian-package]
  90 Wird in »./debian-8.0.0-amd64-netinst.iso.tmpdir/debian.inf.tu-dresden.de/debian/pool/main/l/less/less_458-3_amd64.deb« gespeichert.
  91 
  92 debian.inf.tu-dresden.de/debian/pool/main/l/less/less_458-3_amd64.deb     100%[=========================================================>] 121.55K  --.-KB/s    in 0.1s    
  93 
  94 2019-11-05 11:47:29 (1.13 MB/s) - »./debian-8.0.0-amd64-netinst.iso.tmpdir/debian.inf.tu-dresden.de/debian/pool/main/l/less/less_458-3_amd64.deb« gespeichert [124466/124466]
  95 
  96 
  97 <… OUTPUT OMITED …>
  98 
  99 
 100 --2019-11-05 11:52:06--  http://snapshot.debian.org/archive/debian/20150425T124850Z/pool/main/l/linux/loop-modules-3.16.0-4-amd64-di_3.16.7-ckt9-2_amd64.udeb
 101 Verbindungsaufbau zu snapshot.debian.org (snapshot.debian.org)|193.62.202.27|:80 … verbunden.
 102 HTTP-Anforderung gesendet, auf Antwort wird gewartet … 200 OK
 103 Länge: 13294 (13K)
 104 Wird in »./debian-8.0.0-amd64-netinst.iso.tmpdir/snapshot.debian.org/archive/debian/20150425T124850Z/pool/main/l/linux/loop-modules-3.16.0-4-amd64-di_3.16.7-ckt9-2_amd64.udeb« gespeichert.
 105 
 106 snapshot.debian.org/archive/debian/20150425T124850Z/pool/main/l/linux/loo 100%[=========================================================>]  12.98K  --.-KB/s    in 0.01s   
 107 
 108 2019-11-05 11:52:06 (1.23 MB/s) - »./debian-8.0.0-amd64-netinst.iso.tmpdir/snapshot.debian.org/archive/debian/20150425T124850Z/pool/main/l/linux/loop-modules-3.16.0-4-amd64-di_3.16.7-ckt9-2_amd64.udeb« gespeichert [13294/13294]
 109 
 110 BEENDET --2019-11-05 11:52:06--
 111 Verstrichene Zeit: 6.3s
 112 Geholt: 10 Dateien, 5.0M in 4.7s (1.07 MB/s)
 113 10 der 11 vom Template benötigten Dateien gefunden                                                                                                                      '                                    
 114 Eingabedateien wurden in temporäre Datei »debian-8.0.0-amd64-netinst.iso.tmp« geschrieben - wiederholen Sie das Kommando mit weiteren Dateien
 115 --2019-11-05 11:52:06--  http://snapshot.debian.org/archive/debian/20150425T124850Z/pool/main/l/linux/linux-headers-3.16.0-4-common_3.16.7-ckt9-2_amd64.deb
 116 Auflösen des Hostnamens snapshot.debian.org (snapshot.debian.org)… 185.17.185.185, 193.62.202.27, 2001:630:206:4000:1a1a:0:c13e:ca1b, ...
 117 Verbindungsaufbau zu snapshot.debian.org (snapshot.debian.org)|185.17.185.185|:80 … verbunden.
 118 HTTP-Anforderung gesendet, auf Antwort wird gewartet … 200 OK
 119 Länge: 4480432 (4.3M)
 120 Wird in »./debian-8.0.0-amd64-netinst.iso.tmpdir/snapshot.debian.org/archive/debian/20150425T124850Z/pool/main/l/linux/linux-headers-3.16.0-4-common_3.16.7-ckt9-2_amd64.deb« gespeichert.
 121 
 122 snapshot.debian.org/archive/debian/20150425T124850Z/pool/main/l/linux/lin 100%[=========================================================>]   4.27M  1.11MB/s    in 3.9s    
 123 
 124 2019-11-05 11:52:10 (1.11 MB/s) - »./debian-8.0.0-amd64-netinst.iso.tmpdir/snapshot.debian.org/archive/debian/20150425T124850Z/pool/main/l/linux/linux-headers-3.16.0-4-common_3.16.7-ckt9-2_amd64.deb« gespeichert [4480432/4480432]
 125 
 126 1 der 1 vom Template benötigten Dateien gefunden                                                                                                                                                  '
 127 »debian-8.0.0-amd64-netinst.iso« wurde erfolgreich erzeugt
 128 
 129 -----------------------------------------------------------------
 130 Finished!
 131 The fact that you got this far is a strong indication that `debian-8.0.0-amd64-netinst.iso'
 132 was generated correctly. I will perform an additional, final check,
 133 which you can interrupt safely with Ctrl-C if you do not want to wait.
 134 
 135 OK: Prüfsummen stimmen überein, Image-Datei ist in Ordnung!
 136 jigdo-lite --noask   3.90s user 7.87s system 4% cpu 4:50.47 total

This worked out fine. Now i can test upgrading "oldoldstable" to "stable". Installation succeded.

Aptitude

This cli and gui interface adds in my option some essential features to the apt ecosystem.

It offers search patterns that are very useful.

https://www.debian.org/doc/manuals/aptitude/ch02s04s05.en.html#tableSearchTermQuickGuide

Example: identify installed packages from "Debian Backports". Using the shorthands for:

?installed, ~i

    Matches package versions which are currently installed.

    Since all versions are tested by default, this normally
    matches packages which are currently installed.

?narrow(filter, pattern), ~S filter pattern

    This term “narrows” the search to package versions matching filter.
    In particular, it matches any package version which matches both filter and pattern.
    The string value of the match is the string value of pattern.

?origin(origin), ~Oorigin

    Matches package versions whose origin matches the regular expression origin.
    For instance, “!?origin(debian)” will find any unofficial packages on your system
    (packages not from the Debian archive).

   1 aptitude search '~S ~i ~O"Debian Backports"'
   2 aptitude search '~S ~i ~O"Debian Backports"' \
   3         |sed 's/^i [A ]\+//;s/ - .*//'

To force a downgrade, set the priority of the stable codename above 990 (to force install). Example preference for downgrade:

   1 Package:        *
   2 Pin:            release n=buster
   3 Pin-Priority:   1000

Open aptitude and search for ~Vbpo to identify once again all packages from backports and select the packages to be downgraded by pressing +. They will be marked as to be downgraded i W in organge.

Don't forget to remove the preferences afterwards.

Preseeding

https://www.debian.org/releases/stable/amd64/apb.en.html

Create preseed file from manually installed Debian.

   1 apt install debconf-utils
   2 
   3 PRESEED="preseed.cfg"
   4 echo "#_preseed_V1" > "$PRESEED"
   5 debconf-get-selections --installer >> "$PRESEED"
   6 debconf-get-selections >> "$PRESEED"

LVM

https://www.tldp.org/LDP/sag/html/partitions.html

Debian

About

Releases

Base system from scratch

Partitioning

Debian partman units

Create the partitions on the first disk

Clone GPT to other disk

Create a multidisk RAID1 as swap

Filesystems

Migrate data to subvolumes

Prepare EFI filesystems

fstab

Make sure to use FS-UUIDS and no devices in fstab

Edit fstab to reflect new structure of filesystem

Validate fstab

Chroot into new system

Basic Configuration

Adjust hostname and Mailname

Adjust hosts

Adjust timezone

Configure apt-sources

Pinning: apt_preferences

Some notes regarding preferences

Add some essential packages

Configure locales

Configure vim

zsh grml-flavoured

Networking

ifupdown vs. ifupdown2

Cons

Pros

Install ifupdown2

Predictable device names

On Upgrade to buster

Configure interfaces

Check the configuration of the DNS-resolver

Install openssh-server

Make sure ssh-server starts on boot

Embed pubkey for authentication

Make sure root can login via pubkey-auth (default is fine)

Prepare boot

Last safetys infront of reboot

Make sure MD-RAID reassebles on next boot

Create new mdadm.conf

WELL, GOOD LUCK

Extending to a hypervisor

Some more filesystem

Moving libvirt to own subvolume

Adjust fstab

Filesystem maintenance

ksmtuned

OpenvSwitch

Add vSwitch

Add fakebridges

Configure new interfaces

Conventional networks

PPP Guest-Host

Firewalling

Configure fwbuilder

Performance

Laptop

CPU govenour

IO-Scheduler

Security

CPU microcode

Automatic security upgrades

Check OS Integrity

debsecan

Mobile security

ATA hard drive password

Tools

Links

About

Mode high

Mode maximum

Opal

Tools

Build sedutil

Links