Debian

Contents

Debian

Base system from scratch

Remote system is a dedicated root server in the Hetzner datacenter. Only a live linux amd64 is running and it's reachable by ssh with pubkey auth. We got the option to restart/reset the live linux or boot into the new system. So the logs of the new system is all information we can get (besides the status: running/stopped).

Partitioning

Create the partitions on the first disk

Aline it ot the full 1 MiB/Cylinder (MegaByte binary), because

this allows aligned read/writes. Otherwise it will degrade performance significantly.
this is a power of 2 (1, 2, 4(MEMORY PAGESIZE), 8, ...2^n)KiB. This means if you want to stripe or change some cluster sizes, it will always match (if less).

   1 parted
   2 GNU Parted 3.2
   3 Using /dev/sda
   4 Welcome to GNU Parted! Type 'help' to view a list of commands.
   5 (parted) unit MiB
   6 (parted) print free
   7 Model: ATA HGST HUS726060AL (scsi)
   8 Disk /dev/sda: 5723167MiB
   9 Sector size (logical/physical): 512B/4096B
  10 Partition Table: gpt
  11 Disk Flags: 
  12 
  13 Number  Start       End         Size        File system  Name           Flags
  14         0.02MiB     1.00MiB     0.98MiB     Free Space
  15  1      1.00MiB     2.00MiB     1.00MiB                  bios_grub_sda  bios_grub
  16  2      2.00MiB     256MiB      254MiB      fat16        EFI_sda        boot, esp
  17  3      256MiB      131072MiB   130816MiB                swap1_sda      raid
  18  4      131072MiB   5723166MiB  5592094MiB  btrfs        root_sda
  19         5723166MiB  5723167MiB  0.98MiB     Free Space

Clone GPT to other disk

   1 sgdisk /dev/sda -R /dev/sdb

Create a multidisk RAID1 as swap

   1 mdadm -v --create md_swap1 --level=1 --symlinks yes --raid-devices=2 /dev/sda3 /dev/sdb3
   2 mdadm --detail --scan
   3 mkswap /dev/md/md_swap1 --label md_swap1
   4 swapon /dev/md/md_swap1

Filesystems

   1 ### CREATE FILESYSTEM
   2 mkfs.btrfs -L rootfs --data raid1 --metadata raid1 /dev/sda4 /dev/sdb4
   3 
   4 ### CREATE SUBVOLUMES
   5 mount -o noatime,compress=lzo,space_cache,autodefrag /dev/sdb4 /mnt
   6 btrfs subvolume create /mnt/root
   7 btrfs subvolume create /mnt/home
   8 
   9 btrfs subvolume list /mnt
  10 ID 258 gen 8 top level 5 path root
  11 
  12 btrfs subvolume get-default /mnt/
  13 ID 5 (FS_TREE)
  14 btrfs subvolume set-default 258 /mnt
  15 
  16 umount /mnt
  17 mount -o noatime,compress=lzo,space_cache,autodefrag /dev/sdb4 /mnt
  18 mount -t btrfs
  19 /dev/sda4 on /mnt type btrfs (rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=258,subvol=/root)
  20 
  21 debootstrap buster /mnt http://ftp.de.debian.org/debian

Migrate data to subvolumes

   1 umount /mnt
   2 mount -o noatime,compress=lzo,space_cache,autodefrag,subvol=/ /dev/sdb4 /mnt
   3 
   4 btrfs subvolume create /mnt/home
   5 
   6 btrfs subvolume snapshot /mnt/root/ /mnt/var_log
   7 find /mnt/var_log -mindepth 1 -maxdepth 1 \! -name var | xargs rm -r --
   8 mv /mnt/var_log/var/log/* /mnt/var_log/; rm -r /mnt/var_log/var/
   9 find /mnt/root/var/log -mindepth 1 -maxdepth 1 | xargs rm -r --
  10 
  11 mount -o noatime,compress=lzo,space_cache,autodefrag /dev/sdb4 /mnt
  12 mount -o noatime,compress=lzo,space_cache,autodefrag,subvol=/home /dev/sdb4 /mnt/home
  13 mount -o noatime,compress=lzo,space_cache,autodefrag,subvol=/var_log /dev/sdb4 /mnt/var/log/
  14 
  15 mount -t btrfs
  16 /dev/sda4 on /mnt type btrfs (rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=258,subvol=/root)
  17 /dev/sda4 on /mnt/home type btrfs (rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=261,subvol=/home)
  18 /dev/sda4 on /mnt/var/log type btrfs (rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=262,subvol=/var_log)

Prepare EFI filesystems

Unnecessary - just skip EFI.

   1 ### USE UPPERCASE FS-LABELS
   2 mkfs.vfat -n EFI_SDA /dev/sda2
   3 mkfs.vfat -n EFI_SDB /dev/sdb2
   4 mkdir /mnt/boot/EFI /mnt/boot/EFI_SDB
   5 mount /mnt/boot/EFI
   6 mount /mnt/boot/EFI_SDB
   7 rsync -a EFI ../EFI_SDB/

fstab

Make sure to use FS-UUIDS and no devices in fstab

   1 blkid
   2 /dev/sdb4: LABEL="rootfs" UUID="0cc274fd-4cb8-4cc7-9f60-f59de41f6891" UUID_SUB="d65eac71-86ca-45e0-b59a-7c872de54e59" TYPE="btrfs" PARTLABEL="root_sdb" PARTUUID="20199b69-b0a7-4151-9dd1-83c741d580a0"
   3 /dev/loop0: UUID="40c4ea95-0ecc-4c51-9f3e-e49d8f62f160" TYPE="ext2"
   4 /dev/sda1: PARTLABEL="bios_grub_sda" PARTUUID="1db5c109-b489-4466-8d51-23ef189253b7"
   5 /dev/sda2: SEC_TYPE="msdos" LABEL_FATBOOT="EFI" LABEL="EFI" UUID="8AC4-4574" TYPE="vfat" PARTLABEL="EFI_sda" PARTUUID="14827a48-5a73-4012-bd28-176d365bd903"
   6 /dev/sda3: UUID="4b81ee7b-00bb-3ce6-e83d-c0c00c449861" UUID_SUB="9e9e8cf9-9b01-6128-e2b6-acf8e04e7e9e" LABEL="rescue:md_swap1" TYPE="linux_raid_member" PARTLABEL="swap1_sda" PARTUUID="95586f2b-45ac-478a-8481-fd176f16cbb5"
   7 /dev/sda4: LABEL="rootfs" UUID="0cc274fd-4cb8-4cc7-9f60-f59de41f6891" UUID_SUB="6c72fb67-6c5d-40bc-a4b4-dd34199a1d2b" TYPE="btrfs" PARTLABEL="root_sda" PARTUUID="20199b69-b0a7-4151-9dd1-83c741d580a0"
   8 /dev/sdb1: PARTLABEL="bios_grub_sdb" PARTUUID="1db5c109-b489-4466-8d51-23ef189253b7"
   9 /dev/sdb2: PARTLABEL="EFI_sdb" PARTUUID="14827a48-5a73-4012-bd28-176d365bd903"
  10 /dev/sdb3: UUID="4b81ee7b-00bb-3ce6-e83d-c0c00c449861" UUID_SUB="981dfa0d-6584-2576-a51a-02102935a87b" LABEL="rescue:md_swap1" TYPE="linux_raid_member" PARTLABEL="swap1_sdb" PARTUUID="95586f2b-45ac-478a-8481-fd176f16cbb5"
  11 /dev/md127: LABEL="md_swap1" UUID="87294740-52c4-4557-b838-ddc44ba8aa4b" TYPE="swap"

Edit fstab to reflect new structure of filesystem

   1 cat /etc/fstab
   2 # /etc/fstab: static file system information.
   3 #
   4 # Use 'blkid' to print the universally unique identifier for a
   5 # device; this may be used with UUID= as a more robust way to name devices
   6 # that works even if disks are added and removed. See fstab(5).
   7 
   8 #<file_system>                             <mount_point>  <type>  <options>                                                                    <dump>  <pass>
   9 UUID=87294740-52c4-4557-b838-ddc44ba8aa4b  none           swap    sw                                                                           0       0
  10 UUID=0cc274fd-4cb8-4cc7-9f60-f59de41f6891  /              btrfs   rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=258,subvol=/root     0       1
  11 UUID=8AC4-4574                             /boot/EFI      vfat    utf8                                                                         0       0
  12 UUID=B3B5-67FA                             /boot/EFI_SDB  vfat    utf8                                                                         0       0
  13 UUID=0cc274fd-4cb8-4cc7-9f60-f59de41f6891  /home          btrfs   rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=261,subvol=/home     0       0
  14 UUID=0cc274fd-4cb8-4cc7-9f60-f59de41f6891  /var/log       btrfs   rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=262,subvol=/var_log  0       2

Validate fstab

   1 findmnt --verify
   2 Success, no errors or warnings detected

Chroot into new system

   1 mount -t proc none /mnt/proc
   2 mount -t tmpfs /none /mnt/tmp
   3 mount --bind /dev/ /mnt/dev/
   4 mount --bind /sys /mnt/sys/
   5 chroot /mnt /bin/bash

Basic Configuration

Adjust hostname and Mailname

   1 echo 'kvm2' > /etc/hostname
   2 echo 'kvm2.rockstable.org' > /etc/mailname

Adjust hosts

   1 root@kvm2:/etc# cat /etc/hosts
   2 127.0.0.1       localhost
   3 ::1             localhost ip6-localhost ip6-loopback
   4 ff02::1         ip6-allnodes
   5 ff02::2         ip6-allrouters
   6 
   7 195.201.246.253      kvm2.rockstable.org.
   8 2a01:4f8:231:702::2  kvm2.rockstable.org.
   9 
  10 hostname -F /etc/hostname
  11 exec "$SHELL"

Adjust timezone

   1 dpkg-reconfigure tzdata                                                                                                                                     
   2 
   3 Current default time zone: 'Europe/Berlin'
   4 Local time is now:      Wed May 22 14:22:32 CEST 2019.
   5 Universal Time is now:  Wed May 22 12:22:32 UTC 2019.

Configure apt-sources

   1 cat /etc/apt/sources.list
   2 deb       http://ftp.de.debian.org/debian              buster            main  contrib  non-free
   3 deb-src   http://ftp.de.debian.org/debian/             buster            main  contrib  non-free
   4 deb       http://ftp.de.debian.org/debian/             buster-updates    main  contrib  non-free
   5 deb-src   http://ftp.de.debian.org/debian/             buster-updates    main  contrib  non-free
   6 deb       http://security.debian.org/debian-security/  buster/updates    main  contrib  non-free
   7 deb-src   http://security.debian.org/debian-security/  buster/updates    main  contrib  non-free
   8 #deb      http://ftp.de.debian.org/debian/             buster-backports  main  contrib  non-free
   9 #deb-src  http://ftp.de.debian.org/debian/             buster-backports  main  contrib  non-free
  10

Add some essential packages

   1 apt install \
   2         apt-file aptitude bash-completion btrfs-progs ca-certificates curl \
   3         dmidecode dosfstools git htop iftop info iotop libcrack2 locales \
   4         lsb-release lsof man-db mc mlocate openssl parted pigz psmisc pv \
   5         pwgen rsync strace sudo sysstat tmux vim wget zsh

Configure locales

   1 dpkg-reconfigure locales

Configure vim

   1 ### OVERRIDE /etc/vim/vimrc WITH /etc/vim/vimrc.local
   2 ### YOU CAN "COPY" THE FILE BUT
   3 ### DO NOT CREATE A SOURCE-RECURSION
   4 root@rescue:/etc/network# cat /etc/vim/vimrc.local 
   5 " All system-wide defaults are set in $VIMRUNTIME/debian.vim and sourced by
   6 " the call to :runtime you can find below.  If you wish to change any of those
   7 " settings, you should do it in this file (/etc/vim/vimrc), since debian.vim
   8 " will be overwritten everytime an upgrade of the vim packages is performed.
   9 " It is recommended to make changes after sourcing debian.vim since it alters
  10 " the value of the 'compatible' option.
  11 
  12 " This line should not be removed as it ensures that various options are
  13 " properly set to work with the Vim-related packages available in Debian.
  14 runtime! debian.vim
  15 
  16 " Vim will load $VIMRUNTIME/defaults.vim if the user does not have a vimrc.
  17 " This happens after /etc/vim/vimrc(.local) are loaded, so it will override
  18 " any settings in these files.
  19 " If you don't want that to happen, uncomment the below line to prevent
  20 " defaults.vim from being loaded.
  21 let g:skip_defaults_vim = 1
  22 
  23 " Uncomment the next line to make Vim more Vi-compatible
  24 " NOTE: debian.vim sets 'nocompatible'.  Setting 'compatible' changes numerous
  25 " options, so any other options should be set AFTER setting 'compatible'.
  26 "set compatible
  27 
  28 " Vim5 and later versions support syntax highlighting. Uncommenting the next
  29 " line enables syntax highlighting by default.
  30 syntax on
  31 
  32 " If using a dark background within the editing area and syntax highlighting
  33 " turn on this option as well
  34 set background=dark
  35 
  36 " Uncomment the following to have Vim jump to the last position when
  37 " reopening a file
  38 au BufReadPost * if line("'\"") > 1 && line("'\"") <= line("$") | exe "normal! g'\"" | endif
  39 
  40 " Uncomment the following to have Vim load indentation rules and plugins
  41 " according to the detected filetype.
  42 filetype plugin indent on
  43 
  44 " The following are commented out as they cause vim to behave a lot
  45 " differently from regular Vi. They are highly recommended though.
  46 set showcmd             " Show (partial) command in status line.
  47 set showmatch           " Show matching brackets.
  48 "set ignorecase         " Do case insensitive matching
  49 "set smartcase          " Do smart case matching
  50 "set incsearch          " Incremental search
  51 "set autowrite          " Automatically save before commands like :next and :make
  52 "set hidden             " Hide buffers when they are abandoned
  53 "set mouse=a            " Enable mouse usage (all modes)
  54

zsh grml-flavoured

   1 git clone git://git.grml.org/grml-etc-core.git /opt/grml-etc-core
   2 mv /etc/zsh{,_dist}
   3 ln -s /opt/grml-etc-core/etc/zsh /etc/zsh
   4 chsh -s /bin/zsh
   5 exec zsh

Networking

Add ifupdown2 and remove ifupdown

You better do it in a tmux session, because ssh-session will break and won't return. Don't forget to change

   1 tmux
   2 PRIMARY_INTERFACE="enp1s0"
   3 aptitude install \
   4         ifupdown- \
   5         bridge-utils dnsutils ethtool fail2ban ifupdown2 iputils-tracepath \
   6         isc-dhcp-client net-tools pciutils python-gvgen python-mako \
   7         python-pkg-resources traceroute; \
   8         ifup "$PRIMARY_INTERFACE"

Use Enter ~ . to terminate frozen ssh-clients. Help about ssh escape commands: Enter ~ ?

Predictable device names

https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/

Let's predict some names.

   1 lspci| grep -i eth
   2 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (2) I219-LM

Okay, so PCI-Address is 00:1f.6

   1 Bus:Device.Function (BDF)
   2 BUS: 0
   3 Device/Slot: 31 = 1*16¹ + 15*16⁰
   4 Function: 6

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/networking_guide/sec-understanding_the_predictable_network_interface_device_names

   1 So we predict interface name to be:
   2 enp0s31f6 = "en" + "p" + "0" + s + "31" + "f" + "6"

On Upgrade to buster

If the freshly upgraded system still has old interface naming scheme, you may wish to upgrade. To achieve this, you have to remove /etc/udev/rules.d/80-net-setup-link.rules as well as /etc/systemd/network/50-virtio-kernel-names.link and rebuild your initial-ramdisks.

Please also read: /usr/share/doc/udev/README.Debian.gz

   1 rm /etc/udev/rules.d/80-net-setup-link.rules
   2 rm /etc/systemd/network/50-virtio-kernel-names.link
   3 update-initramfs -u

Configure interfaces

   1 cat /etc/network/interfaces
   2 # interfaces(5) file used by ifup(8) and ifdown(8)
   3 
   4 auto lo
   5 iface lo inet loopback
   6 
   7 auto enp0s31f6
   8 iface enp0s31f6 inet static
   9         address         195.201.246.253/26
  10         address         2a01:4f8:231:702::2/64
  11         gateway         195.201.246.193
  12 
  13 # Include files from /etc/network/interfaces.d:
  14 source-directory /etc/network/interfaces.d

Check the configuration of the DNS-resolver

   1 ### CONTROL CONFIGURATION DNS-RESOLVER
   2 cat /etc/resolv.conf

Install openssh-server

   1 aptitude install openssh-server ssh-askpass openssh-client

If you skipped the step of setting up the hostname your should renew the hostkeys or edit the wrong comment.

   1 rm /etc/ssh/ssh_host_*
   2 dpkg-reconfigure openssh-server

Make sure ssh-server starts on boot

   1 systemctl enable ssh
   2 Synchronizing state of ssh.service with SysV service script with /lib/systemd/systemd-sysv-install.
   3 Executing: /lib/systemd/systemd-sysv-install enable ssh

Embed pubkey for authentication

   1 ls /mnt/root/.ssh/authorized_keys
   2 /mnt/root/.ssh/authorized_keys

Make sure root can login via pubkey-auth (default is fine)

   1 #PermitRootLogin prohibit-password
   2

Prepare boot

   1 aptitude install \
   2         acl apparmor-profiles-extra apparmor-utils firmware-linux \
   3         firmware-linux-free grub-pc initramfs-tools intel-microcode \
   4         linux-headers-amd64 linux-image-amd64 mdadm nfs-common

Just forget about EFI-boot in situations where you can't control that machines UEFI/BIOS and stick with grub-pc.

Last safetys infront of reboot

   1 grub-install /dev/sda
   2 i386-pc wird für Ihre Plattform installiert.
   3 installation beendet. Keine Fehler aufgetreten.
   4 grub-install /dev/sdb
   5 i386-pc wird für Ihre Plattform installiert.
   6 installation beendet. Keine Fehler aufgetreten.

   1 update-grub2
   2 Generating grub configuration file ...
   3 Found linux image: /boot/vmlinuz-4.19.0-5-amd64
   4 Found initrd image: /boot/initrd.img-4.19.0-5-amd64
   5 done

   1 update-initramfs -k all -u 
   2 update-initramfs: Generating /boot/initrd.img-4.19.0-5-amd64

Make sure MD-RAID reassebles on next boot

If you assembled the RAID array earlier in the live system, you will have to change the name configuration from the live systems hostname to your new one.

   1 cat /etc/mdadm/mdadm.conf                                                                                                                                                                                                                                                                                                                
   2 # mdadm.conf
   3 #
   4 # !NB! Run update-initramfs -u after updating this file.
   5 # !NB! This will ensure that initramfs has an uptodate copy.
   6 #
   7 # Please refer to mdadm.conf(5) for information about this file.
   8 #
   9 
  10 # by default (built-in), scan all partitions (/proc/partitions) and all
  11 # containers for MD superblocks. alternatively, specify devices to scan, using
  12 # wildcards if desired.
  13 #DEVICE partitions containers
  14 
  15 # automatically tag new arrays as belonging to the local system
  16 HOMEHOST <system>
  17 
  18 # instruct the monitoring daemon where to send mail alerts
  19 MAILADDR root
  20 
  21 # definitions of existing MD arrays
  22 ARRAY /dev/md/md_swap1  metadata=1.2 UUID=4b81ee7b:00bb3ce6:e83dc0c0:0c449861 name=kvm2:md_swap1
  23 
  24 # This configuration was auto-generated on Wed, 22 May 2019 09:56:16 +0000 by mkconf
  25

Create new mdadm.conf

If there is no file your can create a new one which at least assemles your md-arrays on boot.

   1 mdadm --detail --scan >> /etc/mdadm/mdadm.conf

WELL, GOOD LUCK

   1 shutdown -r now

Extending to a hypervisor

Install libvirt

   1 aptitude install libvirt-daemon-systen virt-manager virt-top virt-what ksmtuned

Moving libvirt to own subvolume

   1 mkdir /media/btrfs5
   2 mount -o rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=5,subvol=/ /dev/sda4 /media/btrfs5
   3 systemctl stop libvirtd.service
   4 lsof /var/lib/libvirt
   5 btrfs subvolume snapshot /media/btrfs5/root /media/btrfs5/var_lib_libvirt
   6 btrfs subvol list /media/btrfs5/
   7 ID 258 gen 4933 top level 5 path root
   8 ID 261 gen 1520 top level 5 path home
   9 ID 262 gen 4933 top level 5 path var_log
  10 ID 265 gen 4933 top level 5 path var_lib_libvirt
  11 
  12 mount -o rw,noatime,compress=lzo,space_cache,autodefrag,subvol=/var_lib_libvirt /dev/sda4 /var/lib/libvirt
  13 
  14 find /media/btrfs5/var_lib_libvirt -mindepth 1 -maxdepth 1 \! -name var | xargs rm -r --
  15 mv /media/btrfs5/var_lib_libvirt/var/lib/libvirt/* /media/btrfs5/var_lib_libvirt
  16 rm -r /media/btrfs5/var_lib_libvirt/var
  17 
  18 rm -r /media/btrfs5/root/var/lib/libvirt/*
  19 
  20 ll /media/btrfs5/root/var/lib/libvirt/
  21 insgesamt 0
  22 ll /media/btrfs5/var_lib_libvirt/
  23 insgesamt 0
  24 drwx--x--x 1 root         root          0 Apr  7 12:36 boot
  25 drwx--x--x 1 root         root          0 Apr  7 12:36 images
  26 drwxr-x--- 1 libvirt-qemu libvirt-qemu 62 Mai 23 10:16 qemu
  27 drwx------ 1 root         root          0 Apr  7 12:36 sanlock
  28 
  29 mount -o rw,noatime,compress=lzo,space_cache,autodefrag,subvol=/var_lib_libvirt /dev/sda4 /var/lib/libvirt
  30 systemctl start libvirtd.service

Adjust fstab

   1 # /etc/fstab: static file system information.
   2 #
   3 # Use 'blkid' to print the universally unique identifier for a
   4 # device; this may be used with UUID= as a more robust way to name devices
   5 # that works even if disks are added and removed. See fstab(5).
   6 
   7 #<file_system>                             <mount_point>     <type>  <options>                                                                            <dump>  <pass>
   8 UUID=87294740-52c4-4557-b838-ddc44ba8aa4b  none              swap    sw                                                                                   0       0
   9 UUID=0cc274fd-4cb8-4cc7-9f60-f59de41f6891  /                 btrfs   rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=258,subvol=/root             0       1
  10 #UUID=8AC4-4574                            /boot/EFI         vfat    utf8                                                                                 0       0
  11 #UUID=B3B5-67FA                            /boot/EFI_SDB     vfat    utf8                                                                                 0       0
  12 UUID=0cc274fd-4cb8-4cc7-9f60-f59de41f6891  /var/log          btrfs   rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=262,subvol=/var_log          0       2
  13 UUID=0cc274fd-4cb8-4cc7-9f60-f59de41f6891  /var/lib/libvirt  btrfs   rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=266,subvol=/var_lib_libvirt  0       2
  14 UUID=0cc274fd-4cb8-4cc7-9f60-f59de41f6891  /home             btrfs   rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=261,subvol=/home             0       0
  15 UUID=0cc274fd-4cb8-4cc7-9f60-f59de41f6891  /media/btrfs5     btrfs   rw,noatime,compress=lzo,space_cache,autodefrag,subvolid=5,subvol=/                   0       0

Filesystem maintenance

   1 aptitude install btrfsmaintenance btrfs-heatmap duperemove

Configure btrfsmaintenance

Change the balance and scrub path from "/" to "auto"

   1 kvm2:~# cat /etc/default/btrfsmaintenance 
   2 ## Path:        System/File systems/btrfs
   3 ## Type:        string(none,stdout,journal,syslog)
   4 ## Default:     "stdout"
   5 #
   6 # Output target for messages. Journal and syslog messages are tagged by the task name like
   7 # 'btrfs-scrub' etc.
   8 BTRFS_LOG_OUTPUT="stdout"
   9 
  10 ## Path:        System/File systems/btrfs
  11 ## Type:        string
  12 ## Default:     ""
  13 #
  14 # Run periodic defrag on selected paths. The files from a given path do not
  15 # cross mount points or other subvolumes/snapshots. If you want to defragment
  16 # nested subvolumes, all have to be listed in this variable.
  17 # (Colon separated paths)
  18 BTRFS_DEFRAG_PATHS=""
  19 
  20 ## Path:           System/File systems/btrfs
  21 ## Type:           string(none,daily,weekly,monthly)
  22 ## Default:        "none"
  23 ## ServiceRestart: btrfsmaintenance-refresh
  24 #
  25 # Frequency of defrag.
  26 BTRFS_DEFRAG_PERIOD="none"
  27 
  28 ## Path:        System/File systems/btrfs
  29 ## Type:        string
  30 ## Default:     "+1M"
  31 #
  32 # Minimal file size to consider for defragmentation
  33 BTRFS_DEFRAG_MIN_SIZE="+1M"
  34 
  35 ## Path:        System/File systems/btrfs
  36 ## Type:        string
  37 ## Default:     "/"
  38 #
  39 # Which mountpoints/filesystems to balance periodically. This may reclaim unused
  40 # portions of the filesystem and make the rest more compact.
  41 # (Colon separated paths)
  42 # The special word/mountpoint "auto" will evaluate all mounted btrfs
  43 # filesystems
  44 BTRFS_BALANCE_MOUNTPOINTS="auto"
  45 
  46 ## Path:           System/File systems/btrfs
  47 ## Type:           string(none,daily,weekly,monthly)
  48 ## Default:        "weekly"
  49 ## ServiceRestart: btrfsmaintenance-refresh
  50 #
  51 # Frequency of periodic balance.
  52 BTRFS_BALANCE_PERIOD="weekly"
  53 
  54 ## Path:        System/File systems/btrfs
  55 ## Type:        string
  56 ## Default:     "1 5 10 20 30 40 50"
  57 #
  58 # The usage percent for balancing data block groups.
  59 #
  60 # Note: default values should not disturb normal work but may not reclaim
  61 # enough block groups. If you observe that, add higher values but beware that
  62 # this will increase IO load on the system.
  63 BTRFS_BALANCE_DUSAGE="1 5 10 20 30 40 50"
  64 
  65 ## Path:        System/File systems/btrfs
  66 ## Type:        string
  67 ## Default:     "1 5 10 20 30"
  68 #
  69 # The usage percent for balancing metadata block groups. The values are also
  70 # used in case the filesystem has mixed blockgroups.
  71 #
  72 # Note: default values should not disturb normal work but may not reclaim
  73 # enough block groups. If you observe that, add higher values but beware that
  74 # this will increase IO load on the system.
  75 BTRFS_BALANCE_MUSAGE="1 5 10 20 30"
  76 
  77 ## Path:        System/File systems/btrfs
  78 ## Type:        string
  79 ## Default:     "/"
  80 #
  81 # Which mountpoints/filesystems to scrub periodically.
  82 # (Colon separated paths)
  83 # The special word/mountpoint "auto" will evaluate all mounted btrfs
  84 # filesystems
  85 BTRFS_SCRUB_MOUNTPOINTS="auto"
  86 
  87 ## Path:        System/File systems/btrfs
  88 ## Type:        string(none,weekly,monthly)
  89 ## Default:     "monthly"
  90 ## ServiceRestart: btrfsmaintenance-refresh
  91 #
  92 # Frequency of periodic scrub.
  93 BTRFS_SCRUB_PERIOD="monthly"
  94 
  95 ## Path:        System/File systems/btrfs
  96 ## Type:        string(idle,normal)
  97 ## Default:     "idle"
  98 #
  99 # Priority of IO at which the scrub process will run. Idle should not degrade
 100 # performance but may take longer to finish.
 101 BTRFS_SCRUB_PRIORITY="idle"
 102 
 103 ## Path:        System/File systems/btrfs
 104 ## Type:        boolean
 105 ## Default:     "false"
 106 #
 107 # Do read-only scrub and don't try to repair anything.
 108 BTRFS_SCRUB_READ_ONLY="false"
 109 
 110 ## Path:           System/File systems/btrfs
 111 ## Description:    Configuration for periodic fstrim
 112 ## Type:           string(none,daily,weekly,monthly)
 113 ## Default:        "none"
 114 ## ServiceRestart: btrfsmaintenance-refresh
 115 #
 116 # Frequency of periodic trim. Off by default so it does not collide with
 117 # fstrim.timer . If you do not use the timer, turn it on here. The recommended
 118 # period is 'weekly'.
 119 BTRFS_TRIM_PERIOD="none"
 120 
 121 ## Path:        System/File systems/btrfs
 122 ## Description: Configuration for periodic fstrim - mountpoints
 123 ## Type:        string
 124 ## Default:     "/"
 125 #
 126 # Which mountpoints/filesystems to trim periodically.
 127 # (Colon separated paths)
 128 # The special word/mountpoint "auto" will evaluate all mounted btrfs
 129 # filesystems
 130 BTRFS_TRIM_MOUNTPOINTS="/"

Deduplication

The last duperemove command seems to be suffient, since it traverses its subvolumes.

   1 mkdir /media/btrfs5/duperemove
   2 VERYNICE="nice -n 19 ionice -c idle"
   3 #$VERYNICE duperemove -rdA --hashfile /media/btrfs5/duperemove/duperemove-home.hash /media/btrfs5/home
   4 #$VERYNICE duperemove -rdA --hashfile /media/btrfs5/duperemove/duperemove-root.hash /media/btrfs5/root
   5 #$VERYNICE duperemove -rdA --hashfile /media/btrfs5/duperemove/duperemove-var_lib_libvirt.hash /media/btrfs5/var_lib_libvirt
   6 #$VERYNICE duperemove -rdA --hashfile /media/btrfs5/duperemove/duperemove-var_log.hash /media/btrfs5/var_log
   7 $VERYNICE duperemove -rdA --hashfile /media/btrfs5/duperemove/duperemove-btrfs5.hash /media/btrfs5/

So we are useing 1.7GB of 5,4TB.

OpenvSwitch

To attach out VMs to the network we can use serveral approaches:

Shared hostbridge
1. classical linux bridge
2. openvswitch
network device passthrough
1. physical device
2. virtual device funtion
macvtap direct connection
1. with vepa
2. without vepa
libvirt networks
1. NAT based network
2. Routed network config
3. Isolated network config
4. Isolated IPv6 network config
5. Network config with no gateway addresses

Since i have no contraints or requirements, i like to use openvswitch as a bridge (as OpenStack would probably probabluse it).

   1 aptitude install openvswitch-switch

Add vSwitch

   1 ovs-vsctl add-br ovs-virt

Add fakebridges

Fake bridges are just virtual sub bridges of a parent bridge that assign an attached port to a specific vlan in access mode.

   1 ovs-vsctl add-br ovs-1a ovs-virt 1000 #public dmz
   2 ovs-vsctl add-br ovs-1n ovs-virt 1500 #extranet
   3 ovs-vsctl add-br ovs-2a ovs-virt 2000 #intranet
   4 ovs-vsctl add-br ovs-2n ovs-virt 2500 #secure zone
   5

Configure new interfaces

Firewalling

   1 aptitude install iptables-persistent ipset iptables fwbuilder

Configure fwbuilder

   1 addgroup firewall
   2 adduser --ingroup firewall --home /etc/fwbuilder --gecos 'remote firewall admin,,,' --system firefighter

   1 cat /etc/sudoers.d/firewall
   2 firefighter     ALL= (root:root) NOPASSWD: /etc/init.d/firewall, /usr/sbin/ipset, /sbin/modprobe, /sbin/iptables

Performance

IO-Scheduler

On hypervisor: bfq

   1 if \! grep -q bfq /etc/modules && \
   2    \! grep -qr bfq /etc/modules-load.d; then
   3         echo bfq >> /etc/modules
   4 fi

   1 cat /etc/udev/rules.d/50-io-scheduler.rules
   2 ACTION=="add|change", KERNEL=="sda", ATTR{queue/scheduler}="bfq"
   3 ACTION=="add|change", KERNEL=="sdb", ATTR{queue/scheduler}="bfq"

In VM: none (if no passthrough)

udev rule:

   1 ACTION=="add|change", KERNEL=="[sv]d[a-z]", ATTR{queue/scheduler}="none"

Method with /etc/default/grub seems not to be working.

   1 GRUB_CMDLINE_LINUX_DEFAULT="quiet elevator=none"

   1 update-grub2
   2 systemctl reboot

Security

Automatic security upgrades

   1 aptitude install unattended-upgrades

debsecan

   1 aptitude install debsecan

   1 debsecan