Bareos

Contents

Bareos

About

Backup Archiving Recovery Open Sourced

Backed by a German corporation Bareos GmbH & Co. KG that offers support.
Please take a look at Bareos Pricing

Source code:

   1 git clone https://github.com/bareos/bareos.git

What is Bareos?

Bareos (Backup Archiving Recovery Open Sourced) is a reliable, cross-network open source software for backup, archiving and recovery of data for all well-established operating systems. Emerged from the Bacula Project in 2010, Bareos was and is actively developed as a fork and enriched with lots of new features.

Thus Bareos today offers among other features LTO hardware encryption, bandwidth limitation and new practical console commands. The source code of Bareos is available on https://github.com/bareos/ under the AGPL v3 license. Additionally Bareos offers binary package repositories for the most important Linux distributions as well as for Windows.

Source: bareos.com - What is Bareos

Ports

tcp/9101 bareos-dir
tcp/9102 bareos-fd
- must be open on client for a backup to be initiated by the director
tcp/9103 bareos-sd

There are predefined services in firewalld "bacula" and "bacula-client".

Domain Names

Always use FQDNs instead relying on the search domains. This saves some queries and helps Bareos to choose the correct address.

Preface

Make sure you have some disk-resources when experimenting. In my little lab I gathered 1TiB in just 5 days of active usage.

VM Setup

I'm running bareos in a VM running on Libvirt/KVM (2Cores, 4GiB Ram, 20GiB Storage, 1Nic).

Nothing special besides a filesystem residing on a btrfs raid1 made of 2 8TiB CMR HDDs on the host, which is mapped to the guest. This is where backups are written to.

9p

This worked quite well for a some time, but with the appearance of #virtio-fs and the general availablity of support in the tools, i definetly recommend to go with the new approach.

9p is slower
9p had problems during boot,
when mounted via /etc/fstab and the vm stuck in on the password prompt for the single user target.

Example domain xml

   1 <filesystem type="mount" accessmode="mapped">
   2   <driver type="path" wrpolicy="immediate"/>
   3   <source dir="/media/space/backup/backup1"/>
   4   <target dir="bareos_storage"/>
   5   <alias name="fs0"/>
   6   <address type="pci" domain="0x0000" bus="0x07" slot="0x00" function="0x0"/>
   7 </filesystem>

/etc/fstab

   1 #<file_system>  <mount_point>            <type>  <options>                         <dump>  <pass>
   2 bareos_storage  /var/lib/bareos/storage  9p      trans=virtio,version=9p2000.L,rw  0       0

mount -t 9p

   1 bareos_storage on /var/lib/bareos/storage type 9p (rw,relatime,sync,dirsync,access=client,trans=virtio)

The driver will be exchanged with #virtio-fs when Linux 5.4 becomes stable and decent support is available in the tooling.

Fio test

To see the longish results, enable the comments.

Just a naive test.

   1 mkdir /var/lib/bareos/storage_9p/fio
   2 cd /var/lib/bareos/storage_9p/fio
   3 fio /usr/share/doc/fio/examples/fio-seq-RW.fio
   4 file1: (g=0): rw=rw, bs=(R) 256KiB-256KiB, (W) 256KiB-256KiB, (T) 256KiB-256KiB, ioengine=libaio, iodepth=16
   5 ...
   6 fio-3.12
   7 Starting 4 processes
   8 file1: Laying out IO file (1 file / 10240MiB)
   9 Jobs: 4 (f=4): [M(4)][100.0%][r=59.3MiB/s,w=39.8MiB/s][r=237,w=159 IOPS][eta 00m:00s]
  10 file1: (groupid=0, jobs=1): err= 0: pid=17725: Thu Jan 14 12:21:36 2021
  11   read: IOPS=24, BW=6324KiB/s (6476kB/s)(5559MiB/900030msec)
  12     slat (usec): min=2851, max=14972, avg=5514.29, stdev=1014.60
  13     clat (msec): min=64, max=3525, avg=360.71, stdev=304.31
  14      lat (msec): min=68, max=3529, avg=366.23, stdev=304.39
  15     clat percentiles (msec):
  16      |  1.00th=[   77],  5.00th=[   85], 10.00th=[   97], 20.00th=[  144],
  17      | 30.00th=[  186], 40.00th=[  228], 50.00th=[  271], 60.00th=[  326],
  18      | 70.00th=[  409], 80.00th=[  518], 90.00th=[  709], 95.00th=[  911],
  19      | 99.00th=[ 1552], 99.50th=[ 1821], 99.90th=[ 2702], 99.95th=[ 2937],
  20      | 99.99th=[ 3473]
  21    bw (  KiB/s): min=  510, max=34304, per=25.99%, avg=6531.75, stdev=4966.58, samples=1742
  22    iops        : min=    1, max=  134, avg=25.47, stdev=19.41, samples=1742
  23   write: IOPS=16, BW=4203KiB/s (4303kB/s)(3694MiB/900030msec); 0 zone resets
  24     slat (msec): min=4, max=2373, avg=52.60, stdev=76.79
  25     clat (usec): min=5, max=3472.5k, avg=370781.79, stdev=315624.33
  26      lat (msec): min=72, max=3577, avg=423.38, stdev=348.86
  27     clat percentiles (msec):
  28      |  1.00th=[   75],  5.00th=[   84], 10.00th=[   96], 20.00th=[  146],
  29      | 30.00th=[  190], 40.00th=[  234], 50.00th=[  279], 60.00th=[  338],
  30      | 70.00th=[  422], 80.00th=[  535], 90.00th=[  735], 95.00th=[  944],
  31      | 99.00th=[ 1620], 99.50th=[ 1921], 99.90th=[ 2903], 99.95th=[ 3138],
  32      | 99.99th=[ 3339]
  33    bw (  KiB/s): min=  510, max=24064, per=26.19%, avg=4406.25, stdev=3253.58, samples=1716
  34    iops        : min=    1, max=   94, avg=17.17, stdev=12.72, samples=1716
  35   lat (usec)   : 10=0.01%
  36   lat (msec)   : 100=10.91%, 250=34.10%, 500=32.98%, 750=13.00%, 1000=5.04%
  37   cpu          : usr=0.08%, sys=2.91%, ctx=2374272, majf=0, minf=12
  38   IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
  39      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  40      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
  41      issued rwts: total=22235,14775,0,0 short=0,0,0,0 dropped=0,0,0,0
  42      latency   : target=0, window=0, percentile=100.00%, depth=16
  43 file1: (groupid=0, jobs=1): err= 0: pid=17726: Thu Jan 14 12:21:36 2021
  44   read: IOPS=24, BW=6339KiB/s (6491kB/s)(5571MiB/900030msec)
  45     slat (usec): min=2584, max=11855, avg=5517.96, stdev=1015.75
  46     clat (msec): min=62, max=3945, avg=359.68, stdev=306.16
  47      lat (msec): min=67, max=3950, avg=365.20, stdev=306.24
  48     clat percentiles (msec):
  49      |  1.00th=[   75],  5.00th=[   83], 10.00th=[   93], 20.00th=[  142],
  50      | 30.00th=[  184], 40.00th=[  228], 50.00th=[  275], 60.00th=[  330],
  51      | 70.00th=[  405], 80.00th=[  518], 90.00th=[  709], 95.00th=[  902],
  52      | 99.00th=[ 1603], 99.50th=[ 1821], 99.90th=[ 2635], 99.95th=[ 3004],
  53      | 99.99th=[ 3876]
  54    bw (  KiB/s): min=  510, max=33280, per=26.02%, avg=6540.27, stdev=5022.02, samples=1743
  55    iops        : min=    1, max=  130, avg=25.43, stdev=19.63, samples=1743
  56   write: IOPS=16, BW=4211KiB/s (4312kB/s)(3701MiB/900030msec); 0 zone resets
  57     slat (msec): min=4, max=2370, avg=52.47, stdev=76.13
  58     clat (usec): min=13, max=3766.0k, avg=370292.61, stdev=320940.52
  59      lat (msec): min=70, max=4049, avg=422.76, stdev=354.82
  60     clat percentiles (msec):
  61      |  1.00th=[   75],  5.00th=[   84], 10.00th=[   94], 20.00th=[  146],
  62      | 30.00th=[  190], 40.00th=[  234], 50.00th=[  279], 60.00th=[  338],
  63      | 70.00th=[  418], 80.00th=[  535], 90.00th=[  726], 95.00th=[  919],
  64      | 99.00th=[ 1653], 99.50th=[ 2022], 99.90th=[ 2903], 99.95th=[ 3272],
  65      | 99.99th=[ 3675]
  66    bw (  KiB/s): min=  510, max=23504, per=26.16%, avg=4400.15, stdev=3193.90, samples=1722
  67    iops        : min=    1, max=   91, avg=17.07, stdev=12.48, samples=1722
  68   lat (usec)   : 20=0.01%
  69   lat (msec)   : 100=11.49%, 250=32.92%, 500=33.83%, 750=13.13%, 1000=4.83%
  70   cpu          : usr=0.08%, sys=2.90%, ctx=2379462, majf=0, minf=15                                                                                                                                                                                                                                 
  71   IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
  72      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  73      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
  74      issued rwts: total=22285,14805,0,0 short=0,0,0,0 dropped=0,0,0,0
  75      latency   : target=0, window=0, percentile=100.00%, depth=16
  76 file1: (groupid=0, jobs=1): err= 0: pid=17727: Thu Jan 14 12:21:36 2021
  77   read: IOPS=24, BW=6207KiB/s (6356kB/s)(5455MiB/900030msec)
  78     slat (usec): min=2971, max=18847, avg=5537.71, stdev=1016.32
  79     clat (msec): min=61, max=3428, avg=366.17, stdev=307.00
  80      lat (msec): min=66, max=3433, avg=371.71, stdev=307.07
  81     clat percentiles (msec):
  82      |  1.00th=[   78],  5.00th=[   86], 10.00th=[  100], 20.00th=[  148],
  83      | 30.00th=[  190], 40.00th=[  234], 50.00th=[  279], 60.00th=[  338],
  84      | 70.00th=[  418], 80.00th=[  531], 90.00th=[  709], 95.00th=[  911],
  85      | 99.00th=[ 1569], 99.50th=[ 1871], 99.90th=[ 2836], 99.95th=[ 3205],
  86      | 99.99th=[ 3339]
  87    bw (  KiB/s): min=  510, max=33792, per=25.53%, avg=6416.51, stdev=4803.03, samples=1740
  88    iops        : min=    1, max=  132, avg=24.94, stdev=18.77, samples=1740
  89   write: IOPS=16, BW=4204KiB/s (4305kB/s)(3695MiB/900030msec); 0 zone resets
  90     slat (msec): min=4, max=2442, avg=52.70, stdev=77.42
  91     clat (usec): min=9, max=3406.3k, avg=372761.24, stdev=322041.57
  92      lat (msec): min=71, max=3548, avg=425.46, stdev=355.36
  93     clat percentiles (msec):
  94      |  1.00th=[   78],  5.00th=[   86], 10.00th=[   97], 20.00th=[  148],
  95      | 30.00th=[  194], 40.00th=[  239], 50.00th=[  284], 60.00th=[  342],
  96      | 70.00th=[  422], 80.00th=[  535], 90.00th=[  718], 95.00th=[  927],
  97      | 99.00th=[ 1653], 99.50th=[ 2039], 99.90th=[ 3104], 99.95th=[ 3306],
  98      | 99.99th=[ 3373]
  99    bw (  KiB/s): min=  510, max=22528, per=26.13%, avg=4396.33, stdev=3237.55, samples=1720
 100    iops        : min=    1, max=   88, avg=17.05, stdev=12.66, samples=1720
 101   lat (usec)   : 10=0.01%
 102   lat (msec)   : 100=10.55%, 250=32.81%, 500=34.00%, 750=13.96%, 1000=4.75%
 103   cpu          : usr=0.08%, sys=2.95%, ctx=2347348, majf=0, minf=14
 104   IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
 105      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
 106      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
 107      issued rwts: total=21821,14780,0,0 short=0,0,0,0 dropped=0,0,0,0
 108      latency   : target=0, window=0, percentile=100.00%, depth=16
 109 file1: (groupid=0, jobs=1): err= 0: pid=17728: Thu Jan 14 12:21:36 2021
 110   read: IOPS=24, BW=6263KiB/s (6413kB/s)(5505MiB/900029msec)
 111     slat (usec): min=2843, max=13630, avg=5508.24, stdev=1017.93
 112     clat (msec): min=64, max=3615, avg=363.82, stdev=310.11
 113      lat (msec): min=67, max=3619, avg=369.33, stdev=310.19
 114     clat percentiles (msec):
 115      |  1.00th=[   75],  5.00th=[   85], 10.00th=[   95], 20.00th=[  144],
 116      | 30.00th=[  186], 40.00th=[  230], 50.00th=[  275], 60.00th=[  334],
 117      | 70.00th=[  418], 80.00th=[  523], 90.00th=[  709], 95.00th=[  911],
 118      | 99.00th=[ 1603], 99.50th=[ 1905], 99.90th=[ 2769], 99.95th=[ 3138],
 119      | 99.99th=[ 3540]
 120    bw (  KiB/s): min=  510, max=34304, per=25.70%, avg=6457.68, stdev=4939.59, samples=1745
 121    iops        : min=    1, max=  134, avg=25.16, stdev=19.30, samples=1745
 122   write: IOPS=16, BW=4206KiB/s (4307kB/s)(3697MiB/900029msec); 0 zone resets
 123     slat (msec): min=4, max=2436, avg=52.64, stdev=77.04
 124     clat (usec): min=5, max=3420.2k, avg=371105.32, stdev=316602.38
 125      lat (msec): min=69, max=3621, avg=423.75, stdev=350.07
 126     clat percentiles (msec):
 127      |  1.00th=[   75],  5.00th=[   85], 10.00th=[   96], 20.00th=[  146],
 128      | 30.00th=[  190], 40.00th=[  236], 50.00th=[  288], 60.00th=[  347],
 129      | 70.00th=[  422], 80.00th=[  531], 90.00th=[  718], 95.00th=[  927],
 130      | 99.00th=[ 1670], 99.50th=[ 1989], 99.90th=[ 2836], 99.95th=[ 3071],
 131      | 99.99th=[ 3406]
 132    bw (  KiB/s): min=  510, max=24576, per=26.31%, avg=4426.71, stdev=3227.28, samples=1709
 133    iops        : min=    1, max=   96, avg=17.23, stdev=12.61, samples=1709
 134   lat (usec)   : 10=0.01%
 135   lat (msec)   : 100=11.49%, 250=32.57%, 500=33.66%, 750=13.50%, 1000=4.83%
 136   cpu          : usr=0.08%, sys=2.90%, ctx=2360901, majf=0, minf=13
 137   IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
 138      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
 139      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
 140      issued rwts: total=22018,14788,0,0 short=0,0,0,0 dropped=0,0,0,0
 141      latency   : target=0, window=0, percentile=100.00%, depth=16
 142 
 143 Run status group 0 (all jobs):
 144    READ: bw=24.5MiB/s (25.7MB/s), 6207KiB/s-6339KiB/s (6356kB/s-6491kB/s), io=21.6GiB (23.2GB), run=900029-900030msec
 145   WRITE: bw=16.4MiB/s (17.2MB/s), 4203KiB/s-4211KiB/s (4303kB/s-4312kB/s), io=14.4GiB (15.5GB), run=900029-900030msec
 146 fio /usr/share/doc/fio/examples/fio-seq-RW.fio  7,62s user 125,55s system 10% cpu 21:30,52 total

Cleanup

   1 cd ..
   2 rm -r fio

virtio-fs

You'll need

virtiofs support in the guest kernel (Linux v5.4 or later)
Libvirt 6.2.0+, better 6.9+
qemu 5.0.0

With shared memory (tmpfs)

I personally fapour the tmpfs approach, because it does not set upper boundaries to the amount of hugepages. However by default tmpfs is also limited to half of the RAM.

/etc/libvirt/qemu.conf

   1 # This directory is used for memoryBacking source if configured as file.
   2 # NOTE: big files will be stored here
   3 #memory_backing_dir = "/var/lib/libvirt/qemu/ram"
   4 memory_backing_dir = "/dev/shm"

Restart libvirtd

   1 systemctl restart libvirtd.service

Example domain xml

   1 <domain type='kvm'>
   2   …
   3   <memory unit='>MiB'>4096</memory>
   4   <currentMemory unit='KiB'>2097152</currentMemory>
   5   <memoryBacking>
   6     <access mode='shared'/>
   7   </memoryBacking>
   8   …
   9   <cpu mode='host-model' check='partial'>
  10     <numa>
  11       <cell id='0' cpus='0-3' memory='4096' unit='MiB' memAccess='shared'/>
  12     </numa>
  13   </cpu>
  14   …
  15   <devices>
  16     …
  17     <filesystem type='mount' accessmode='passthrough'>
  18       <driver type='virtiofs'/>
  19       <source dir='/media/space/backup/backup1'/>
  20       <target dir='bareos_storage_virtiofs'/>
  21       <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
  22     </filesystem>
  23     …
  24   </devices>
  25 </domain>

With explicit hugepages

The VM won't start, unless you allocated the full amount of RAM in hugepages. Preallocate hugepages to back the memory of the VM.

   1 virsh allocpages 2M 2048

Please see Linux#Hugepages for a persistent configuration.

Example domain xml (necessary)

   1 <domain type='kvm'>
   2   …
   3   <memory unit='MiB'>4096</memory>
   4   <currentMemory unit='MiB'>2048</currentMemory>
   5   <memoryBacking>
   6     <hugepages>
   7       <page size='2048' unit='KiB'/>
   8     </hugepages>
   9     <access mode='shared'/>
  10   </memoryBacking>
  11   …
  12   <cpu mode='host-model' check='partial'>
  13     <numa>
  14       <cell id='0' cpus='0-3' memory='4096' unit='MiB' memAccess='shared'/>
  15     </numa>
  16   </cpu>
  17   …
  18   <devices>
  19     …
  20     <filesystem type='mount' accessmode='passthrough'>
  21       <driver type='virtiofs'/>
  22       <source dir='/media/space/backup/backup1'/>
  23       <target dir='bareos_storage_virtiofs'/>
  24       <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
  25     </filesystem>
  26     …
  27   </devices>
  28 </domain>

Fio test

To see the longish results, enable the comments.

Just a naive test

   1 mkdir /var/lib/bareos/storage/fio
   2 cd /var/lib/bareos/storage/fio
   3 fio /usr/share/doc/fio/examples/fio-seq-RW.fio
   4 
   5 file1: (g=0): rw=rw, bs=(R) 256KiB-256KiB, (W) 256KiB-256KiB, (T) 256KiB-256KiB, ioengine=libaio, iodepth=16
   6 ...
   7 fio-3.12
   8 Starting 4 processes
   9 file1: Laying out IO file (1 file / 10240MiB)
  10 Jobs: 4 (f=4): [M(4)][100.0%][r=1069MiB/s,w=727MiB/s][r=4277,w=2906 IOPS][eta 00m:00s]
  11 file1: (groupid=0, jobs=1): err= 0: pid=13103: Thu Jan 14 11:43:20 2021
  12   read: IOPS=612, BW=153MiB/s (161MB/s)(135GiB/900001msec)
  13     slat (usec): min=43, max=853169, avg=393.69, stdev=1263.28
  14     clat (usec): min=1602, max=37792k, avg=14890.06, stdev=436567.44
  15      lat (usec): min=1717, max=37793k, avg=15284.17, stdev=436569.72
  16     clat percentiles (msec):
  17      |  1.00th=[    5],  5.00th=[    6], 10.00th=[    6], 20.00th=[    7],
  18      | 30.00th=[    8], 40.00th=[    8], 50.00th=[    9], 60.00th=[    9],
  19      | 70.00th=[   10], 80.00th=[   11], 90.00th=[   12], 95.00th=[   13],
  20      | 99.00th=[   16], 99.50th=[   17], 99.90th=[   25], 99.95th=[   30],
  21      | 99.99th=[17113]
  22    bw (  KiB/s): min=  512, max=353792, per=41.39%, avg=259549.94, stdev=43036.90, samples=1086
  23    iops        : min=    2, max= 1382, avg=1013.84, stdev=168.11, samples=1086
  24   write: IOPS=408, BW=102MiB/s (107MB/s)(89.8GiB/900001msec); 0 zone resets
  25     slat (usec): min=143, max=37780k, avg=1848.00, stdev=173203.68
  26     clat (usec): min=2, max=37793k, avg=14390.07, stdev=406456.91
  27      lat (usec): min=1080, max=37793k, avg=16238.60, stdev=441873.69
  28     clat percentiles (msec):
  29      |  1.00th=[    5],  5.00th=[    6], 10.00th=[    7], 20.00th=[    7],
  30      | 30.00th=[    8], 40.00th=[    9], 50.00th=[    9], 60.00th=[   10],
  31      | 70.00th=[   10], 80.00th=[   11], 90.00th=[   12], 95.00th=[   13],
  32      | 99.00th=[   16], 99.50th=[   17], 99.90th=[   24], 99.95th=[   30],
  33      | 99.99th=[17113]
  34    bw (  KiB/s): min=  512, max=225280, per=41.37%, avg=173290.20, stdev=27300.81, samples=1086
  35    iops        : min=    2, max=  880, avg=676.88, stdev=106.65, samples=1086
  36   lat (usec)   : 4=0.01%
  37   lat (msec)   : 2=0.01%, 4=0.55%, 10=74.15%, 20=25.09%, 50=0.16%
  38   lat (msec)   : 250=0.01%, 750=0.01%, 1000=0.01%
  39   cpu          : usr=0.80%, sys=27.88%, ctx=1953111, majf=0, minf=12
  40   IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
  41      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  42      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
  43      issued rwts: total=551075,368000,0,0 short=0,0,0,0 dropped=0,0,0,0
  44      latency   : target=0, window=0, percentile=100.00%, depth=16
  45 file1: (groupid=0, jobs=1): err= 0: pid=13104: Thu Jan 14 11:43:20 2021
  46   read: IOPS=611, BW=153MiB/s (160MB/s)(134GiB/900001msec)
  47     slat (usec): min=42, max=35260k, avg=455.56, stdev=47539.00
  48     clat (usec): min=2, max=37788k, avg=15216.77, stdev=447333.20
  49      lat (usec): min=243, max=37788k, avg=15672.74, stdev=449851.87
  50     clat percentiles (msec):
  51      |  1.00th=[    5],  5.00th=[    6], 10.00th=[    6], 20.00th=[    7],
  52      | 30.00th=[    8], 40.00th=[    8], 50.00th=[    9], 60.00th=[    9],
  53      | 70.00th=[   10], 80.00th=[   11], 90.00th=[   12], 95.00th=[   13],
  54      | 99.00th=[   16], 99.50th=[   17], 99.90th=[   24], 99.95th=[   31],
  55      | 99.99th=[17113]
  56    bw (  KiB/s): min= 1532, max=344910, per=41.32%, avg=259077.80, stdev=42335.00, samples=1086
  57    iops        : min=    5, max= 1347, avg=1011.96, stdev=165.36, samples=1086
  58   write: IOPS=409, BW=102MiB/s (107MB/s)(89.9GiB/900001msec); 0 zone resets
  59     slat (usec): min=148, max=37779k, avg=1755.64, stdev=163159.15
  60     clat (usec): min=245, max=37785k, avg=13931.32, stdev=389051.86
  61      lat (usec): min=503, max=37789k, avg=15687.48, stdev=421922.18
  62     clat percentiles (msec):
  63      |  1.00th=[    5],  5.00th=[    6], 10.00th=[    7], 20.00th=[    7],
  64      | 30.00th=[    8], 40.00th=[    9], 50.00th=[    9], 60.00th=[   10],
  65      | 70.00th=[   10], 80.00th=[   11], 90.00th=[   12], 95.00th=[   13],
  66      | 99.00th=[   16], 99.50th=[   17], 99.90th=[   24], 99.95th=[   29],
  67      | 99.99th=[17113]
  68    bw (  KiB/s): min=  510, max=220672, per=41.39%, avg=173362.11, stdev=27355.21, samples=1086
  69    iops        : min=    1, max=  862, avg=677.13, stdev=106.86, samples=1086
  70   lat (usec)   : 4=0.01%, 250=0.01%, 750=0.01%
  71   lat (msec)   : 2=0.01%, 4=0.55%, 10=74.03%, 20=25.21%, 50=0.17%
  72   lat (msec)   : 250=0.01%, 750=0.01%, 1000=0.01%
  73   cpu          : usr=0.84%, sys=27.71%, ctx=1963887, majf=0, minf=15
  74   IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
  75      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%                                                                                                                                                                                                                     
  76      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
  77      issued rwts: total=550190,368159,0,0 short=0,0,0,0 dropped=0,0,0,0
  78      latency   : target=0, window=0, percentile=100.00%, depth=16
  79 file1: (groupid=0, jobs=1): err= 0: pid=13105: Thu Jan 14 11:43:20 2021
  80   read: IOPS=612, BW=153MiB/s (161MB/s)(135GiB/900001msec)
  81     slat (usec): min=44, max=7437.3k, avg=412.49, stdev=11059.38
  82     clat (usec): min=742, max=37792k, avg=14685.10, stdev=427440.19
  83      lat (usec): min=1247, max=37793k, avg=15098.00, stdev=427598.94
  84     clat percentiles (msec):
  85      |  1.00th=[    5],  5.00th=[    6], 10.00th=[    6], 20.00th=[    7],
  86      | 30.00th=[    8], 40.00th=[    8], 50.00th=[    9], 60.00th=[    9],
  87      | 70.00th=[   10], 80.00th=[   11], 90.00th=[   12], 95.00th=[   13],
  88      | 99.00th=[   16], 99.50th=[   17], 99.90th=[   24], 99.95th=[   30],
  89      | 99.99th=[17113]
  90    bw (  KiB/s): min=  510, max=362496, per=41.35%, avg=259291.17, stdev=43366.78, samples=1088
  91    iops        : min=    1, max= 1416, avg=1012.81, stdev=169.40, samples=1088
  92   write: IOPS=409, BW=102MiB/s (107MB/s)(89.9GiB/900001msec); 0 zone resets
  93     slat (usec): min=112, max=37779k, avg=1818.40, stdev=172556.40
  94     clat (usec): min=4, max=37794k, avg=14684.07, stdev=420477.17
  95      lat (usec): min=740, max=37795k, avg=16503.00, stdev=454556.31
  96     clat percentiles (msec):
  97      |  1.00th=[    5],  5.00th=[    6], 10.00th=[    7], 20.00th=[    7],
  98      | 30.00th=[    8], 40.00th=[    9], 50.00th=[    9], 60.00th=[   10],
  99      | 70.00th=[   10], 80.00th=[   11], 90.00th=[   12], 95.00th=[   13],
 100      | 99.00th=[   16], 99.50th=[   17], 99.90th=[   23], 99.95th=[   28],
 101      | 99.99th=[17113]
 102    bw (  KiB/s): min=  510, max=236544, per=41.37%, avg=173290.47, stdev=27898.89, samples=1088
 103    iops        : min=    1, max=  924, avg=676.87, stdev=108.99, samples=1088
 104   lat (usec)   : 10=0.01%, 750=0.01%
 105   lat (msec)   : 2=0.01%, 4=0.50%, 10=74.25%, 20=25.05%, 50=0.15%
 106   lat (msec)   : 250=0.01%, 750=0.01%, 1000=0.01%
 107   cpu          : usr=0.80%, sys=27.80%, ctx=1954852, majf=0, minf=14
 108   IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
 109      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
 110      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
 111      issued rwts: total=551089,368308,0,0 short=0,0,0,0 dropped=0,0,0,0
 112      latency   : target=0, window=0, percentile=100.00%, depth=16
 113 file1: (groupid=0, jobs=1): err= 0: pid=13106: Thu Jan 14 11:43:20 2021
 114   read: IOPS=613, BW=153MiB/s (161MB/s)(135GiB/900001msec)
 115     slat (usec): min=41, max=3805.1k, avg=397.25, stdev=5198.37
 116     clat (usec): min=1265, max=37792k, avg=14655.43, stdev=422874.96
 117      lat (usec): min=1535, max=37792k, avg=15053.09, stdev=422908.42
 118     clat percentiles (msec):
 119      |  1.00th=[    5],  5.00th=[    6], 10.00th=[    6], 20.00th=[    7],
 120      | 30.00th=[    8], 40.00th=[    8], 50.00th=[    9], 60.00th=[    9],
 121      | 70.00th=[   10], 80.00th=[   11], 90.00th=[   12], 95.00th=[   13],
 122      | 99.00th=[   16], 99.50th=[   17], 99.90th=[   24], 99.95th=[   31],
 123      | 99.99th=[17113]
 124    bw (  KiB/s): min=  512, max=351232, per=41.43%, avg=259803.88, stdev=43436.51, samples=1088
 125    iops        : min=    2, max= 1372, avg=1014.84, stdev=169.67, samples=1088
 126   write: IOPS=409, BW=102MiB/s (107MB/s)(89.9GiB/900001msec); 0 zone resets
 127     slat (usec): min=143, max=37779k, avg=1840.99, stdev=173080.88
 128     clat (usec): min=3, max=37791k, avg=14694.66, stdev=426993.13
 129      lat (usec): min=776, max=37795k, avg=16536.17, stdev=460779.24
 130     clat percentiles (msec):
 131      |  1.00th=[    5],  5.00th=[    6], 10.00th=[    7], 20.00th=[    7],
 132      | 30.00th=[    8], 40.00th=[    9], 50.00th=[    9], 60.00th=[   10],
 133      | 70.00th=[   10], 80.00th=[   11], 90.00th=[   12], 95.00th=[   13],
 134      | 99.00th=[   16], 99.50th=[   17], 99.90th=[   24], 99.95th=[   30],
 135      | 99.99th=[17113]
 136    bw (  KiB/s): min=  512, max=225280, per=41.39%, avg=173373.41, stdev=27446.78, samples=1087
 137    iops        : min=    2, max=  880, avg=677.22, stdev=107.21, samples=1087
 138   lat (usec)   : 4=0.01%, 1000=0.01%
 139   lat (msec)   : 2=0.01%, 4=0.55%, 10=74.24%, 20=25.02%, 50=0.15%
 140   lat (msec)   : 250=0.01%, 750=0.01%, 1000=0.01%
 141   cpu          : usr=0.80%, sys=27.74%, ctx=1881579, majf=0, minf=14
 142   IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
 143      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
 144      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
 145      issued rwts: total=552137,368114,0,0 short=0,0,0,0 dropped=0,0,0,0
 146      latency   : target=0, window=0, percentile=100.00%, depth=16
 147 
 148 Run status group 0 (all jobs):
 149    READ: bw=612MiB/s (642MB/s), 153MiB/s-153MiB/s (160MB/s-161MB/s), io=538GiB (578GB), run=900001-900001msec
 150   WRITE: bw=409MiB/s (429MB/s), 102MiB/s-102MiB/s (107MB/s-107MB/s), io=360GiB (386GB), run=900001-900001msec
 151 fio /usr/share/doc/fio/examples/fio-seq-RW.fio  33,09s user 1006,59s system 106% cpu 16:14,10 total

For easier comparision

   1 Run status group 0 (all jobs):
   2    READ: bw=24.5MiB/s (25.7MB/s), 6207KiB/s-6339KiB/s (6356kB/s-6491kB/s), io=21.6GiB (23.2GB), run=900029-900030msec
   3   WRITE: bw=16.4MiB/s (17.2MB/s), 4203KiB/s-4211KiB/s (4303kB/s-4312kB/s), io=14.4GiB (15.5GB), run=900029-900030msec
   4 fio /usr/share/doc/fio/examples/fio-seq-RW.fio  7,62s user 125,55s system 10% cpu 21:30,52 total

The difference is simply tremendous. Therefor my recommendation use virtiofs instead of 9p.

Cleanup

   1 cd ..
   2 rm -r fio

Virtiofs additional options

There are some additional options that may be passed via xml please see

libvirt.org - Domain XML format Filesystems
man 1 virtiofsd
/usr/share/libvirt/schemas/domain.rng

Here is a example configuration

   1     <filesystem type='mount' accessmode='passthrough'>
   2       <driver type='virtiofs' queue='1024'/>
   3       <!--<binary path='/usr/libexec/virtiofsd' xattr='on'>-->
   4       <binary path='/usr/lib/qemu/virtiofsd' xattr='on'>
   5         <cache mode='always'/>
   6         <lock posix='on' flock='on'/>
   7       </binary>
   8       <source dir='/media/space/backup/backup1'/>
   9       <target dir='bareos_storage_virtiofs'/>
  10       <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
  11     </filesystem>

Installation

3rd party repo

Debian packages are quite outdated even in Sid. With Bareos the new packages have to be used as the quality improved tremendously.

Every year a new repo for a new release is spawned. So keep your lists current.

/etc/apt/sources.list.d/bareos.list

   1 #deb http://download.bareos.org/bareos/release/17.2//Debian_9.0 /
   2 #deb http://download.bareos.org/bareos/release/18.2//Debian_9.0 /
   3 ### The usage of the link "latest" got deprecated. :-/
   4 #deb http://download.bareos.org/bareos/release/latest/Debian_9.0/ /
   5 #deb http://download.bareos.org/bareos/release/latest/Debian_10/ /
   6 #deb http://download.bareos.org/bareos/release/20/Debian_10/ /
   7 ### There are nightly builds for development
   8 #deb http://download.bareos.org/bareos/experimental/nightly/Debian_10/ /
   9

/etc/apt/sources.list.d/bareos.list

   1 deb http://download.bareos.org/bareos/release/21/Debian_11/ /
   2 #deb http://download.bareos.org/bareos/experimental/nightly/Debian_11/ /
   3

Import the public key to proof the signature.

   1 #apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F93C028C093BFBA2
   2 #apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 118283D9A7862CEE
   3 apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E5D882B28657AE28
   4 #apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 42DA24A6DFEF9127
   5

The public key for Bareos 20 has not yet been uploaded to the pgp infrastructure. so i recommend running some checks on the key.

   1 LANG=C apt update 2>&1 | grep -E '^(E|W): '
   2 W: GPG error: http://download.bareos.org/bareos/release/20/Debian_10  InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0CCBA537DBE083A6
   3 E: The repository 'http://download.bareos.org/bareos/release/20/Debian_10  InRelease' is not signed.
   4 
   5 wget -qO /tmp/bareos_release.key \
   6         'https://download.bareos.org/bareos/release/20/Debian_10/Release.key'
   7 gpg --keyid-format LONG --show-keys < /tmp/bareos_release.key
   8 pub   rsa4096 2020-12-11 [SC]
   9       C68B001F74D2F20243D0B7A20CCBA537DBE083A6
  10 uid                      Bareos 20 Signing Key <signing@bareos.com
  11 apt-key add /tmp/bareos_release.key
  12 apt update

Please take a look here
gpg#New way with own file/keyring

Install bareos

Package "bareos" pulls in all dependencies. As a base OS I suggest Debian.

   1 aptitude install bareos bareos-tools

And keep them uptodate.

Install bareos-tools

   1 aptitude install bareos bareos-tools

Binaries of package
dpkg -L bareos-tools| grep bin/

   1 /usr/bin/bregex
   2 /usr/bin/bwild
   3 /usr/sbin/bcopy
   4 /usr/sbin/bextract
   5 /usr/sbin/bls
   6 /usr/sbin/bpluginfo
   7 /usr/sbin/bregex
   8 /usr/sbin/bwild

Configure

Bareos Director-Daemon

Bareos Docs - Director Daemon

Resource Director (self)

/etc/bareos/bareos-dir.d/director/bareos-dir.conf

   1 Director {                            # define myself
   2   Name = bareos-dir
   3   QueryFile = "/usr/lib/bareos/scripts/query.sql"
   4   Maximum Concurrent Jobs = 10
   5   Password = "very_log_director_password"         # Console password
   6   Messages = Daemon
   7   Auditing = yes
   8 
   9   # Enable the Heartbeat if you experience connection losses
  10   # (eg. because of your router or firewall configuration).
  11   # Additionally the Heartbeat can be enabled in bareos-sd and bareos-fd.
  12   #
  13   # Heartbeat Interval = 1 min
  14 
  15   # remove comment in next line to load dynamic backends from specified directory
  16   # Backend Directory = /usr/lib/bareos/backends
  17 
  18   # remove comment from "Plugin Directory" to load plugins from specified directory.
  19   # if "Plugin Names" is defined, only the specified plugins will be loaded,
  20   # otherwise all director plugins (*-dir.so) from the "Plugin Directory".
  21   #
  22   # Plugin Directory = /usr/lib/bareos/plugins
  23   # Plugin Names = ""
  24 
  25   ### Collect and keep statistics
  26   ### This enabled by default in the director resource,
  27   ### but must be enabled manually in the storage resource.
  28   Statistics Collect Interval = 150             ### DEFAULT: 150
  29   Statistics Retention = 160704000              ### Defaults: 160704000 (5y)
  30 }

Filesets

LinuxBase

/etc/bareos/bareos-dir.d/fileset/LinuxBase.conf

   1 FileSet {
   2   Name = "LinuxBase"
   3   Description = "Backup a minimal set of a Linux Server."
   4   Enable VSS = false
   5 
   6   Include {
   7     Options {
   8       Compression       = GZIP
   9       Signature         = SHA256
  10       One FS            = yes
  11       noatime           = yes
  12       Sparse            = yes
  13       IgnoreCase        = no
  14       Wildfile = /boot/config-*
  15     }
  16     Options {
  17       exclude = yes
  18       Regexdir = "/home/[^/]+/.steam"
  19       Regexdir = "/home/[^/]+/.local/share/Trash/files"
  20       Regexdir = "/home/[^/]+/Downloads"
  21     }
  22     Exclude Dir Containing = .nobackup
  23     File = /etc
  24     File = /home
  25     File = /opt
  26     File = /root
  27     File = /var/lib
  28   }
  29 
  30   Include {
  31      Options {
  32         Wildfile = /boot/config-*
  33      }
  34      Options {
  35         Exclude = yes
  36         RegexFile = ".*"
  37      }
  38      File = /boot
  39   }
  40 
  41   Exclude {
  42   }
  43 }

Windows10AllMedia

Gather all media files of every user.

Windows backups are by default created with Wiki EN Volume Shadow Copy Service or VSS. So files can be backed up even if they are openend by other programms. This service can probably only be utilized with NTFS or ReFS.

/etc/bareos/bareos-dir.d/fileset/Windows10AllMedia.conf

   1 FileSet {
   2 FileSet {
   3   Name = "Windows10AllMedia"
   4   Description = "Backup all media files of all users on a Windows Server."
   5   Enable VSS = yes
   6 
   7   Include {
   8 
   9     File  = "C:/Users"
  10 
  11     Options {
  12       Signature = SHA256
  13       Verify = s1
  14       IgnoreCase = yes
  15 
  16       # Include all users’ directories so we reach the inner ones.  Unlike a
  17       # WildDir pattern ending in *, this RegExDir only matches the top-level
  18       # directories and not any inner ones.
  19       RegExDir = "^C:/Users/[^/]+$"
  20 
  21       # Ditto all users’ Media directories.
  22       WildDir = "C:/Users/*/Bibliothek"
  23       WildDir = "C:/Users/*/Documents"
  24       WildDir = "C:/Users/*/Music"
  25       WildDir = "C:/Users/*/Pictures"
  26       WildDir = "C:/Users/*/Videos"
  27 
  28       # Include the contents of the Media directories and any subdirectories.
  29       Wild = "C:/Users/*/Bibliothek/*"
  30       Wild = "C:/Users/*/Documents/*"
  31       Wild = "C:/Users/*/Music/*"
  32       Wild = "C:/Users/*/Pictures/*"
  33       Wild = "C:/Users/*/Videos/*"
  34     }
  35 
  36     Options {
  37       Exclude = yes
  38       IgnoreCase = yes
  39 
  40       # Exclude everything else, in particular any files at the top level and
  41       # any other directories or files in the users’ directories.
  42       Wild = "C:/Users/*"
  43     }
  44   }
  45 }

Clients

A classical connection is initiated by the director and a basic client looks like this.

On the server /etc/bareos/bareos-dir.d/client/fuxtop-fd.conf

   1 Client {
   2   Name = fuxtop-fd
   3   Address = fuxtop.dungeon.rockstable.org
   4   Address = fuxtop-wlan.dungeon.rockstable.org
   5   Description = Fux's laptop
   6   Catalog = MyCatalog
   7   Password = ezoh9EeGhiquahping5eihaayadohr5Ei
   8 }

Job Defaults (JobDefs)

Tox simplify configuration of a job, sets of defaults for jobs can be created, which than may be referenced during the creation of a job with jobdef="DefaultJob-Name"

/etc/bareos/bareos-dir.d/jobdefs/DefaultJob-Linux.conf

   1 JobDefs {
   2   Name = "DefaultJob-Linux"
   3   Type = Backup
   4   Level = Incremental
   5   Messages = "Standard"
   6   Storage = "File"
   7   Pool = "Incremental"
   8   FullBackupPool = "Full"
   9   IncrementalBackupPool = "Incremental"
  10   DifferentialBackupPool = "Differential"
  11   Client = "bareos-fd"
  12   FileSet = "LinuxBase"
  13   Schedule = "WeeklyCycle"
  14   WriteBootstrap = "/var/lib/bareos/%c.bsr"
  15   runscript {
  16     command = "/usr/local/lib/bareos/scripts/bareos-run_script.sh before"
  17     runswhen = before
  18     failonerror = No
  19   }
  20   runscript {
  21     command = "/usr/local/lib/bareos/scripts/bareos-run_script.sh after"
  22     runswhen = after
  23     failonerror = No
  24   }
  25 }

/etc/bareos/bareos-dir.d/jobdefs/DefaultJob-Windows10.conf

   1 JobDefs {
   2   Name = "DefaultJob-Windows10"
   3   Type = Backup
   4   Level = Incremental
   5   Messages = "Standard"
   6   Storage = "File"
   7   Pool = "Incremental"
   8   FullBackupPool = "Full"
   9   IncrementalBackupPool = "Incremental"
  10   DifferentialBackupPool = "Differential"
  11   Client = "muckie-pc-fd"
  12   FileSet = "Windows10AllMedia"
  13   Schedule = "WeeklyCycle"
  14   WriteBootstrap = "/var/lib/bareos/%c.bsr"
  15 }

Pre/Post Backup Scripts

Create a directory for custom scripts on the client

   1 install -o root -g root -d /usr/local/lib/bareos/scripts

A little script to be used to run some tasks before and after the backup with some sensitivity of the distribution.

/usr/local/lib/bareos/scripts/bareos-run_script.sh

   1 #!/bin/bash
   2 
   3 APPEND_DATE=false
   4 
   5 usage () {
   6 cat <<-EOL
   7 $(basename $0) [-d||--date] [--] [before||after]
   8 EOL
   9 }
  10 
  11 # Note that we use "$@" to let each command-line parameter expand to a
  12 # separate word. The quotes around "$@" are essential!
  13 # We need TEMP as the 'eval set --' would nuke the return value of getopt.
  14 TEMP=$(getopt -o 'd' --long 'date' -n "$(basename $0)" -- "$@")
  15 
  16 if [ $? -ne 0 ]; then
  17         echo 'Terminating...' >&2
  18         exit 1
  19 fi
  20 
  21 # Note the quotes around "$TEMP": they are essential!
  22 eval set -- "$TEMP"
  23 unset TEMP
  24 
  25 while true; do
  26         case "$1" in
  27                 '-d'|'--date')
  28                         APPEND_DATE=true
  29                         shift
  30                         continue
  31                 ;;
  32                 #'-b'|'--b-long')
  33                 #       echo "Option b, argument '$2'"
  34                 #       shift 2
  35                 #       continue
  36                 #;;
  37                 #'-c'|'--c-long')
  38                 #       # c has an optional argument. As we are in quoted mode,
  39                 #       # an empty parameter will be generated if its optional
  40                 #       # argument is not found.
  41                 #       case "$2" in
  42                 #               '')
  43                 #                       echo 'Option c, no argument'
  44                 #               ;;
  45                 #               *)
  46                 #                       echo "Option c, argument '$2'"
  47                 #               ;;
  48                 #       esac
  49                 #       shift 2
  50                 #       continue
  51                 #;;
  52                 '--')
  53                         shift
  54                         break
  55                 ;;
  56                 *)
  57                         echo 'Internal error!' >&2
  58                         exit 1
  59                 ;;
  60         esac
  61 done
  62 
  63 ### SANITIZE
  64 if   [ ${#@} -eq 0 ]; then
  65         WHEN="before"
  66 elif [ ${#@} -gt 1 ]; then
  67         cat<<-EOL
  68                 Please decide between "before" and "after".
  69                 Default is before.
  70         EOL
  71         usage
  72         exit 2
  73 elif [ $1 == "before" ] || [ $1 == "after" ]; then
  74         WHEN="$1"
  75 else
  76         echo "Unknown execution timepoint '$1'."
  77         echo "Exiting…"
  78         usage
  79         exit 2
  80 fi
  81 
  82 FILE_SUFFIX="txt"
  83 if $APPEND_DATE; then
  84         DATE="$(date +%F)"
  85         FILE_SUFFIX="_$DATE.$FILE_SUFFIX"
  86 else
  87         FILE_SUFFIX=".$FILE_SUFFIX"
  88 fi
  89 
  90 ISSUE="$(cat /etc/issue)"
  91 if egrep -q '(Debian|Ubuntu)' <<< "$ISSUE"; then
  92         OS="Debian"
  93 fi
  94 
  95 ### MAIN
  96 
  97 
  98 if [ "$OS" == "Debian" ]; then
  99         BACKUP_DIR="/var/backups/bareos"
 100         FILE_SELECTIONS="dpkg-selections$FILE_SUFFIX"
 101         FILE_LIST="dpkg-list$FILE_SUFFIX"
 102 else
 103         echo "Unsupported OS. Exiting…"
 104         exit 1
 105 fi
 106 
 107 ### COMMAND LIST: AFTER
 108 if [ "$WHEN" == "before" ]; then
 109         [ -d "$BACKUP_DIR" ] || mkdir "$BACKUP_DIR"
 110         dpkg --get-selections > "$BACKUP_DIR/$FILE_SELECTIONS"
 111         dpkg -l > "$BACKUP_DIR/$FILE_LIST"
 112 
 113 ### COMMAND LIST: AFTER
 114 elif [ "$WHEN" == "after" ]; then
 115         rm "$BACKUP_DIR/$FILE_SELECTIONS"
 116         rm "$BACKUP_DIR/$FILE_LIST"
 117 
 118 fi
 119 
 120 echo "Script executed successfully $WHEN backup."
 121 exit 0

Storage

There are some statistics that should come in handy if you want to estimate how long you backup will take or if e.g. you want to account and bill the backup storage.

/etc/bareos/bareos-dir.d/storage/File.conf

   1 Storage {
   2   Name = File
   3   Address = backup1                # N.B. Use a fully qualified name here (do not use "localhost" here).
   4   Password = "very_long_storage_daemon_password"
   5   Device = FileStorage
   6   Media Type = File
   7   Collect Statistics = yes      ### DEFAULT: no
   8 }

Pools

In Maximum Volume Bytes it makes no difference if you write a capital "G" or a lowercase "g".

/etc/bareos/bareos-dir.d/pool/Differential.conf

   1 Pool {
   2   Name = Differential
   3   Pool Type = Backup
   4   Recycle = yes                       # Bareos can automatically recycle Volumes
   5   AutoPrune = yes                     # Prune expired volumes
   6   Volume Retention = 90 days          # How long should the Differential Backups be kept? (#09)
   7   Maximum Volume Bytes = 10G          # Limit Volume size to something reasonable
   8   Maximum Volumes = 100               # Limit number of Volumes in Pool
   9   Label Format = "Differential-"      # Volumes will be labeled "Differential-<volume-id>"
  10 }

/etc/bareos/bareos-dir.d/pool/Full.conf

   1 Pool {
   2   Name = Full
   3   Pool Type = Backup
   4   Recycle = yes                       # Bareos can automatically recycle Volumes
   5   AutoPrune = yes                     # Prune expired volumes
   6   Volume Retention = 365 days         # How long should the Full Backups be kept? (#06)
   7   Maximum Volume Bytes = 50G          # Limit Volume size to something reasonable
   8   Maximum Volumes = 100               # Limit number of Volumes in Pool
   9   Label Format = "Full-"              # Volumes will be labeled "Full-<volume-id>"
  10 }

/etc/bareos/bareos-dir.d/pool/Incremental.conf

   1 Pool {
   2   Name = Incremental
   3   Pool Type = Backup
   4   Recycle = yes                       # Bareos can automatically recycle Volumes
   5   AutoPrune = yes                     # Prune expired volumes
   6   Volume Retention = 30 days          # How long should the Incremental Backups be kept?  (#12)
   7   Maximum Volume Bytes = 5g           # Limit Volume size to something reasonable
   8   Maximum Volumes = 200               # Limit number of Volumes in Pool
   9   Label Format = "Incremental-"       # Volumes will be labeled "Incremental-<volume-id>"
  10 }

/etc/bareos/bareos-dir.d/pool/Scratch.conf

   1 Pool {
   2   Name = Scratch
   3   Pool Type = Scratch
   4 }

Trouble Shooting

Wrong Unix Permissions in Config Directories

bareos-director does not start if it can't read a config-file. So make sure not to create files that bareos can't read. This can easily happen if you copy a file with unix permission o-rwx as root, which changes owner.

It will definetely complain in the logs.

Okt 16 09:12:46 backup1 systemd[1]: Starting Bareos Director Daemon service...
Okt 16 09:12:46 backup1 bareos-dir[1573]: 0bareos-dir: ERROR TERMINATION at parse_conf.c:198
Okt 16 09:12:46 backup1 bareos-dir[1573]: Config error: Cannot open config file "/etc/bareos/bareos-dir.d/*/*.conf": Keine Berechtigung
Okt 16 09:12:46 backup1 bareos-dir[1574]: [55B blob data]
Okt 16 09:12:46 backup1 bareos-dir[1574]: Config error: Cannot open config file "/etc/bareos/bareos-dir.d/*/*.conf": Keine Berechtigung
Okt 16 09:12:46 backup1 systemd[1]: bareos-director.service: Can't open PID file /var/lib/bareos/bareos-dir.9101.pid (yet?) after start: No such file or directory
Okt 16 09:12:46 backup1 systemd[1]: bareos-director.service: Failed with result 'protocol'.

To identify the file you can also use the following command:

   1 find /etc/bareos -type f \! -user bareos \! -group bareos -exec ls -l {} \;

bconsole

Reload configuration

Atfer changes to the configuration files, bareos config can be reloaded without restarting the service.

   1 echo reload |bconsole

Add client

Create a new password and hash it to md5:

   1 PASSWORD="$(pwgen 32 1)"
   2 echo -en "$PASSWORD\t";echo -n "$PASSWORD"|md5sum| sed -r 's/\s+.*$//;s/^/[md5]/'
   3 very_long_P455W0rD      [md5]1e83f0211757b383fd4715ba90e229a9

On bareos server configure director:

   1 bconsole
   2 # PLAIN PASSWORD DISCOURAGED
   3 configure add client name="fuxtop" address="fuxtop.dungeon.rockstable.org" description="Fux's laptop" catalog="MyCatalog" password="very_long_P455W0rD"

Add job

Add a job with bandwidth limit that does not exceed 54MBit/s WiFi (max. 6.75MB/s -> 4096KiB/s)

   1 # bconsole
   2 configure add job name=muckie-pc-job jobdefs=DefaultJob-Windows10 client=muckie-pc-fd maximumbandwidth=4096

Label mediums

Edit pool to hold more and eventually larger mediums

/etc/bareos/bareos-dir.d/pool/Incremental.conf

   1 Pool {
   2   Name = Incremental
   3   Pool Type = Backup
   4   Recycle = yes                       # Bareos can automatically recycle Volumes
   5   AutoPrune = yes                     # Prune expired volumes
   6   Volume Retention = 30 days          # How long should the Incremental Backups be kept?  (#12)
   7   Maximum Volume Bytes = 4G           # Limit Volume size to something reasonable
   8   Maximum Volumes = 150               # Limit number of Volumes in Pool
   9   Label Format = "Incremental-"       # Volumes will be labeled "Incremental-<volume-id>"
  10 }

Label

   1 # bconsole
   2 Connecting to Director localhost:9101
   3 1000 OK: bareos-dir Version: 17.2.7 (16 Jul 2018)
   4 Enter a period to cancel a command.
   5 *reload
   6 reloaded
   7 *
   8 *list volumes pool=Incremental
   9 *
  10 *label pool=Incremental storage=File volume=Incremental-0139
  11 Connecting to Storage daemon File at backup1:9103 ...
  12 Sending label command for Volume "Incremental-0139" Slot 0 ...
  13 3000 OK label. VolBytes=204 Volume="Incremental-0139" Device="FileStorage" (/var/lib/bareos/storage)
  14 Catalog record for Volume "Incremental-0139", Slot 0 successfully created.
  15 Requesting to mount FileStorage ...
  16 3001 OK mount requested. Device="FileStorage" (/var/lib/bareos/storage)

Job rescued.

Limit bandwidth

The limit is in KiB/s.

For example a 54Mbit/s WiFi can only transport 6912KiB/s and the networking gets close unusable. In this case it's a good idea to limit the backup to 4096KiB/s.

Reduce bandwidth to 4MiB/s

   1 # bconsole
   2 ### DETERMINE THE RUNNING JOB ON A CLIENT
   3 *status client=muckie-pc-fd
   4 ### CHANGE BWLIMIT
   5 *setbandwidth jobid=436 limit=4096
   6 Connecting to Client muckie-pc-fd at muckie-pc.dungeon.rockstable.org:9102
   7 OK Limiting bandwidth to 4096kb/s muckie-pc-job.2021-01-15_09.40.00_11

The maximum-bandwidth may be specified when adding the job.

A persistent configuration looks like this
/etc/bareos/bareos-dir.d/job/muckie-pc-job.conf

   1 Job {
   2   Name = muckie-pc-job
   3   JobDefs = DefaultJob-Windows10
   4   Client = muckie-pc-fd
   5   Maximum Bandwidth = 4096
   6 }

Check DNS resolution on client

Check on client how domain name is resolved.

   1 bconsole
   2 *resolve client=client-fd DNS.domain.tld

Enable debugging

Enable debugging on the component

   1 bconsole
   2 *setdebug level=200 trace=1 client=client-fd
   3 *setdebug level=200 trace=1 dir

Database queries

By Querying the database you may gain some flexibility.

List all media that is

in pool 'Incremental'
smaller than 4GiB and
not in state append or purged
not used by a job

   1 SELECT
   2         p.name "poolname",
   3         m.volumename,
   4         m.volstatus,
   5         m.maxvolbytes / POWER(2, 30) "Size [GiB]"
   6 FROM media AS m
   7 INNER JOIN pool AS p ON m.poolid = p.poolid
   8 WHERE
   9         p.name='Incremental'
  10         AND m.maxvolbytes < 4 * POWER(2,30)
  11         AND m.volstatus NOT IN ('Append','Purged')
  12         AND NOT EXISTS(
  13                 SELECT 1 FROM jobmedia jm
  14                 WHERE jm.mediaid = m.mediaid
  15         )
  16 ORDER BY m.volumename ASC;

The above query in a slightly modified way, can be used as input for scripting. I determined some small files to be deleted.

   1 sudo -u bareos -- /usr/bin/psql -qt <<EOF \
   2         |sed '/^$/d' > /tmp/small_volumes.txt
   3 SELECT
   4         m.volumename
   5 FROM media AS m
   6 INNER JOIN pool AS p ON m.poolid = p.poolid
   7 WHERE
   8         p.name='Incremental'
   9         AND m.maxvolbytes < 4 * POWER(2,30)
  10         AND m.volstatus NOT IN ('Append','Purged')
  11 ORDER BY m.volumename ASC;
  12 EOF

Now use it as a input for a shell loop to perform various actions.

   1 while read FILE; do
   2          echo "list volume=$FILE"
   3 done \
   4 < /tmp/small_volumes.txt \
   5 |bconsole

You might also take only 5

   1 while read FILE; do
   2          echo "list volume=$FILE"
   3 done \
   4 <<< "$(head -n5 /tmp/small_volumes.txt)" \
   5 |bconsole

I don't want this files to be recycled, so i can delete them later from the catalog and reclaim their space.

   1 while read FILE; do
   2          echo "update volume=$FILE recycle=no"
   3 done \
   4 < /tmp/small_volumes.txt \
   5 |bconsole

Bareos Storage-Daemon

Bareos Docs - Storage Daemon

/etc/bareos/bareos-sd.d/device/FileStorage.conf

   1 Device {
   2   Name = FileStorage
   3   Media Type = File
   4   Archive Device = /var/lib/bareos/storage
   5   LabelMedia = yes;                   # lets Bareos label unlabeled media
   6   Random Access = yes;
   7   AutomaticMount = yes;               # when device opened, read it
   8   RemovableMedia = no;
   9   AlwaysOpen = no;
  10   Description = "File device. A connecting Director must have the same Name and MediaType."
  11 }

As always the passwords of the storage daemon and the director have to match.

/etc/bareos/bareos-sd.d/director/bareos-dir.conf

   1 Director {
   2   Name = bareos-dir
   3   Password = "vaeQuaiG7aeGhee7moothijah6shieY1"
   4   Description = "Director, who is permitted to contact this storage daemon."
   5 }

An for comparision the director counterpart
/etc/bareos/bareos-dir.d/storage/File.conf

   1 Storage {
   2   Name = File
   3   # N.B. Use a fully qualified name here (do not use "localhost" here).
   4   Address = backup1.dungeon.rockstable.org.                
   5   Password = "vaeQuaiG7aeGhee7moothijah6shieY1"
   6   Device = FileStorage
   7   Media Type = File
   8   Collect Statistics = yes      ### DEFAULT: no
   9 }

/etc/bareos/bareos-sd.d/messages/Standard.conf

   1 Messages {
   2   Name = Standard
   3   Director = bareos-dir = all
   4   Description = "Send all messages to the Director."
   5 }

/etc/bareos/bareos-sd.d/storage/bareos-sd.conf

   1 Storage {
   2   Name = bareos-sd
   3   Maximum Concurrent Jobs = 20
   4 
   5   # remove comment from "Plugin Directory" to load plugins from specified directory.
   6   # if "Plugin Names" is defined, only the specified plugins will be loaded,
   7   # otherwise all storage plugins (*-sd.so) from the "Plugin Directory".
   8   #
   9   # Plugin Directory = /usr/lib/bareos/plugins
  10   # Plugin Names = ""
  11 }

Bareos Filesystem-Daemon

Bareos Docs - Filesystem Daemon

/etc/bareos/bareos-fd.d/client/myself.conf

   1 Client {
   2   Name = fuxtop-fd
   3   Maximum Concurrent Jobs = 20
   4 
   5   # remove comment from "Plugin Directory" to load plugins from specified directory.
   6   # if "Plugin Names" is defined, only the specified plugins will be loaded,
   7   # otherwise all storage plugins (*-fd.so) from the "Plugin Directory".
   8   #
   9   # Plugin Directory = /usr/lib/bareos/plugins
  10   # Plugin Names = ""
  11 
  12   # if compatible is set to yes, we are compatible with bacula
  13   # if set to no, new bareos features are enabled which is the default
  14   # compatible = yes
  15 }

Connection from client to director

It's very useful for a mobile device to try a backup when you are actually @home.

Classical connections from the director to the filedaemon are still allowed unless their have been explicitly disabled.

# TODO - Inititiate not only the connection, do the backup …

Some inspiration

Dejan Lukan on infosec - Data Backups with Bacula: Mobile Devices

In the client's file daemon the director configuration needs to be changed to be able to initiate the connection from the client side.
/etc/bareos/bareos-fd.d/director/bareos-dir.conf C:\ProgramData\Bareos\bareos-fd.d\director\bareos-dir.conf

   1 Director {
   2   Name = bareos-dir
   3   #Password = "ezoh9EeGhiquahping5eihaayadohr5Ei"
   4   Password = "[md5]615b12ba593fe1a31d4e75c0b86e9aaa"
   5   Description = "Allow the configured Director to access this file daemon."
   6 
   7   # Allow connection from client to director
   8   Connection From Client To Director = yes
   9   # Director Network Address
  10   Address = backup1.dungeon.rockstable.org
  11   #Address = 192.168.182.201
  12 }

On the server this the Client configuration needs to be changed to be able to initiate the connection from the client side.
/etc/bareos/bareos-dir.d/client/fuxtop-fd.conf

   1 Client {
   2   Name = fuxtop-fd
   3   Address = fuxtop.dungeon.rockstable.org
   4   Address = fuxtop-wlan.dungeon.rockstable.org
   5   Description = Fux's laptop
   6   Catalog = MyCatalog
   7   Password = ezoh9EeGhiquahping5eihaayadohr5Ei
   8 
   9   Connection From Client To Director = true
  10   ### KEEPALIVE MAY BE ENABLED TO IMPROVE STABILITY
  11   #Heartbeat Interval = 60
  12 }

Tail the log-buffer and restart the daemon

   1 journalctl -f
   2 systemctl restart bareos-fd.service
   3 systemctl status bareos-fd.service

Check the connection on the client

   1 netstat -tuep| grep 'backup'
   2 Proto Recv-Q Send-Q Local Address           Foreign Address         State       Benutzer   Inode      PID/Program name    
   3 tcp        0      0 fuxtop.dungeon.ro:33920 backup1.dung:bacula-dir VERBUNDEN   root       95072      10758/bareos-fd

Caveats

bareos-fd.service has dependencies on

nss-lookup.target network.target remote-fs.target time-sync.target

It won't start successfully if any of these targets can't be reached. So make sure that e.g. any remote-mount (like nfs, smb, glusterfs, …) in /etc/fstab works under any condition. In doubt use autofs.

Bareos-Webui

http://backup1/bareos-webui/dashboard/

   1 aptitude install bareos-webui

Redirect to https

/etc/apache2/sites-available/redirect_80.conf

   1 <VirtualHost *:80>
   2     # The ServerName directive sets the request scheme, hostname and port that
   3     # the server uses to identify itself. This is used when creating
   4     # redirection URLs. In the context of virtual hosts, the ServerName
   5     # specifies what hostname must appear in the request's Host: header to
   6     # match this virtual host. For the default virtual host (this file) this
   7     # value is not decisive as it is used as a last resort host regardless.
   8     # However, you must set it for any further virtual host explicitly.
   9     Define      SERVER_NAME backup1.dungeon.rockstable.org
  10     ServerName  ${SERVER_NAME}
  11     ServerAlias backup1
  12 
  13     DocumentRoot /var/www/html
  14 
  15     # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
  16     # error, crit, alert, emerg.
  17     # It is also possible to configure the loglevel for particular
  18     # modules, e.g.
  19     #LogLevel info ssl:warn
  20 
  21     ErrorLog    ${APACHE_LOG_DIR}/${SERVER_NAME}_error.log
  22     CustomLog   ${APACHE_LOG_DIR}/${SERVER_NAME}_access.log combined
  23 
  24     # For most configuration files from conf-available/, which are
  25     # enabled or disabled at a global level, it is possible to
  26     # include a line for only one particular virtual host. For example the
  27     # following line enables the CGI configuration for this host only
  28     # after it has been globally disabled with "a2disconf".
  29     #Include conf-available/serve-cgi-bin.conf
  30 
  31     ### Redirect anything up to the encrypted vHost
  32     ### (breaks /server-status if certificate is invalid)
  33     #RedirectMatch  ^/(.*)$                 https://${SERVER_NAME}/$1
  34 
  35     ### Finally nothing leads around mod_rewrite
  36     RewriteEngine on
  37     RewriteCond %{REQUEST_URI}      !=/server-status
  38     RewriteRule ^/(.*)$             https://%{HTTP_HOST}/$1 [R=301,L]
  39 </VirtualHost>
  40 
  41 # vim: syntax=apache ts=4 sw=4 sts=4 sr noet

   1 a2ensite redirect_80
   2 systemctl reload apache2.service

Bareos vHost

/etc/apache2/sites-available/bareos_443.conf

   1 <IfModule mod_ssl.c>
   2         <VirtualHost _default_:443>
   3                 ServerAdmin webmaster@rockstable.it
   4 
   5                 DocumentRoot /var/www/html
   6 
   7                 # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
   8                 # error, crit, alert, emerg.
   9                 # It is also possible to configure the loglevel for particular
  10                 # modules, e.g.
  11                 #LogLevel info ssl:warn
  12 
  13                 ErrorLog ${APACHE_LOG_DIR}/error.log
  14                 CustomLog ${APACHE_LOG_DIR}/access.log combined
  15 
  16                 # For most configuration files from conf-available/, which are
  17                 # enabled or disabled at a global level, it is possible to
  18                 # include a line for only one particular virtual host. For example the
  19                 # following line enables the CGI configuration for this host only
  20                 # after it has been globally disabled with "a2disconf".
  21                 #Include conf-available/serve-cgi-bin.conf
  22 
  23                 #   SSL Engine Switch:
  24                 #   Enable/Disable SSL for this virtual host.
  25                 SSLEngine on
  26 
  27                 #   A self-signed (snakeoil) certificate can be created by installing
  28                 #   the ssl-cert package. See
  29                 #   /usr/share/doc/apache2/README.Debian.gz for more info.
  30                 #   If both key and certificate are stored in the same file, only the
  31                 #   SSLCertificateFile directive is needed.
  32                 SSLCertificateFile      /etc/ssl/certs/ssl-cert-snakeoil.pem
  33                 SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
  34 
  35                 #   Server Certificate Chain:
  36                 #   Point SSLCertificateChainFile at a file containing the
  37                 #   concatenation of PEM encoded CA certificates which form the
  38                 #   certificate chain for the server certificate. Alternatively
  39                 #   the referenced file can be the same as SSLCertificateFile
  40                 #   when the CA certificates are directly appended to the server
  41                 #   certificate for convinience.
  42                 #SSLCertificateChainFile /etc/apache2/ssl.crt/server-ca.crt
  43 
  44                 #   Certificate Authority (CA):
  45                 #   Set the CA certificate verification path where to find CA
  46                 #   certificates for client authentication or alternatively one
  47                 #   huge file containing all of them (file must be PEM encoded)
  48                 #   Note: Inside SSLCACertificatePath you need hash symlinks
  49                 #                to point to the certificate files. Use the provided
  50                 #                Makefile to update the hash symlinks after changes.
  51                 #SSLCACertificatePath /etc/ssl/certs/
  52                 #SSLCACertificateFile /etc/apache2/ssl.crt/ca-bundle.crt
  53 
  54                 #   Certificate Revocation Lists (CRL):
  55                 #   Set the CA revocation path where to find CA CRLs for client
  56                 #   authentication or alternatively one huge file containing all
  57                 #   of them (file must be PEM encoded)
  58                 #   Note: Inside SSLCARevocationPath you need hash symlinks
  59                 #                to point to the certificate files. Use the provided
  60                 #                Makefile to update the hash symlinks after changes.
  61                 #SSLCARevocationPath /etc/apache2/ssl.crl/
  62                 #SSLCARevocationFile /etc/apache2/ssl.crl/ca-bundle.crl
  63 
  64                 #   Client Authentication (Type):
  65                 #   Client certificate verification type and depth.  Types are
  66                 #   none, optional, require and optional_no_ca.  Depth is a
  67                 #   number which specifies how deeply to verify the certificate
  68                 #   issuer chain before deciding the certificate is not valid.
  69                 #SSLVerifyClient require
  70                 #SSLVerifyDepth  10
  71 
  72                 #   SSL Engine Options:
  73                 #   Set various options for the SSL engine.
  74                 #   o FakeBasicAuth:
  75                 #        Translate the client X.509 into a Basic Authorisation.  This means that
  76                 #        the standard Auth/DBMAuth methods can be used for access control.  The
  77                 #        user name is the `one line' version of the client's X.509 certificate.
  78                 #        Note that no password is obtained from the user. Every entry in the user
  79                 #        file needs this password: `xxj31ZMTZzkVA'.
  80                 #   o ExportCertData:
  81                 #        This exports two additional environment variables: SSL_CLIENT_CERT and
  82                 #        SSL_SERVER_CERT. These contain the PEM-encoded certificates of the
  83                 #        server (always existing) and the client (only existing when client
  84                 #        authentication is used). This can be used to import the certificates
  85                 #        into CGI scripts.
  86                 #   o StdEnvVars:
  87                 #        This exports the standard SSL/TLS related `SSL_*' environment variables.
  88                 #        Per default this exportation is switched off for performance reasons,
  89                 #        because the extraction step is an expensive operation and is usually
  90                 #        useless for serving static content. So one usually enables the
  91                 #        exportation for CGI and SSI requests only.
  92                 #   o OptRenegotiate:
  93                 #        This enables optimized SSL connection renegotiation handling when SSL
  94                 #        directives are used in per-directory context.
  95                 #SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire
  96                 <FilesMatch "\.(cgi|shtml|phtml|php)$">
  97                                 SSLOptions +StdEnvVars
  98                 </FilesMatch>
  99                 <Directory /usr/lib/cgi-bin>
 100                                 SSLOptions +StdEnvVars
 101                 </Directory>
 102 
 103                 #   SSL Protocol Adjustments:
 104                 #   The safe and default but still SSL/TLS standard compliant shutdown
 105                 #   approach is that mod_ssl sends the close notify alert but doesn't wait for
 106                 #   the close notify alert from client. When you need a different shutdown
 107                 #   approach you can use one of the following variables:
 108                 #   o ssl-unclean-shutdown:
 109                 #        This forces an unclean shutdown when the connection is closed, i.e. no
 110                 #        SSL close notify alert is send or allowed to received.  This violates
 111                 #        the SSL/TLS standard but is needed for some brain-dead browsers. Use
 112                 #        this when you receive I/O errors because of the standard approach where
 113                 #        mod_ssl sends the close notify alert.
 114                 #   o ssl-accurate-shutdown:
 115                 #        This forces an accurate shutdown when the connection is closed, i.e. a
 116                 #        SSL close notify alert is send and mod_ssl waits for the close notify
 117                 #        alert of the client. This is 100% SSL/TLS standard compliant, but in
 118                 #        practice often causes hanging connections with brain-dead browsers. Use
 119                 #        this only for browsers where you know that their SSL implementation
 120                 #        works correctly.
 121                 #   Notice: Most problems of broken clients are also related to the HTTP
 122                 #   keep-alive facility, so you usually additionally want to disable
 123                 #   keep-alive for those clients, too. Use variable "nokeepalive" for this.
 124                 #   Similarly, one has to force some clients to use HTTP/1.0 to workaround
 125                 #   their broken HTTP/1.1 implementation. Use variables "downgrade-1.0" and
 126                 #   "force-response-1.0" for this.
 127                 # BrowserMatch "MSIE [2-6]" \
 128                 #               nokeepalive ssl-unclean-shutdown \
 129                 #               downgrade-1.0 force-response-1.0
 130 
 131                 Include conf-available/bareos-webui.conf
 132                 RedirectMatch "^/$" "/bareos-webui"
 133         </VirtualHost>
 134 </IfModule>
 135 
 136 # vim: syntax=apache ts=4 sw=4 sts=4 sr noet

   1 a2ensite redirect_80
   2 systemctl reload apache2.service

Http2 support

Apache2 mod_http2 does not support mod_mpm_prefork. So we switch to php-fpm, mod_proxy_fcgi and mod_mpm_event.

   1 aptitude install php-fpm
   2 a2dismod php7.0
   3 a2enconf php7.3-fpm
   4 a2enmod ssl
   5 a2enmod proxy_fcgi
   6 a2dismod mpm_prefork
   7 a2enmod mpm_event
   8 a2enmod http2
   9 systemctl restart apache2

Trouble Shooting

I really recommend upgrading to version 21 - a great release. All problems with the UI are basically gone. I works like a charm! Thanks!

Uninitialized variable "extras"

Bareos Bug-Report #1120

mod_proxy_fcgi throws an error with php7.3+, regarding a undefined variable "extras".

[Mon Oct 14 12:54:19.526596 2019] [proxy_fcgi:error] [pid 842:tid 139696600672000] [client 192.168.182.13:40176] AH01071: Got error 'PHP message: PHP Notice:  compact(): Undefined variable: extras in /usr/share/bareos-webui/vendor/zendframework/zend-view/src/Helper/HeadLink.php on line 403PHP message: PHP Notice:  compact(): Undefined variable:
extras in /usr/share/bareos-webui/vendor/zendframework/zend-view/src/Helper/HeadLink.php on line 403'

So i applied a little patch to /usr/share/bareos-webui/vendor/zendframework/zend-view/src/Helper/HeadLink.php which initializes extras to empty string "" if (0 < count($args) && is_array($args[0])

HeadLink.php.patch

   1 375a376
   2 >         $extras                = "";
   3 400a402,403
   4 >       } else {
   5 >             $extras = (string) $extras;

   1 patch /usr/share/bareos-webui/vendor/zendframework/zend-view/src/Helper/HeadLink.php \
   2 < HeadLink.php.patch

   1     /**
   2      * Create item for stylesheet link item
   3      *
   4      * @param  array $args
   5      * @return stdClass|false Returns false if stylesheet is a duplicate
   6      */
   7     public function createDataStylesheet(array $args)
   8     {
   9         $rel                   = 'stylesheet';
  10         $type                  = 'text/css';
  11         $media                 = 'screen';
  12         $conditionalStylesheet = false;
  13         $href                  = array_shift($args);
  14         $extras                = "";
  15 
  16         if ($this->isDuplicateStylesheet($href)) {
  17             return false;
  18         }
  19 
  20         if (0 < count($args)) {
  21             $media = array_shift($args);
  22             if (is_array($media)) {
  23                 $media = implode(',', $media);
  24             } else {
  25                 $media = (string) $media;
  26             }
  27         }
  28         if (0 < count($args)) {
  29             $conditionalStylesheet = array_shift($args);
  30             if (!empty($conditionalStylesheet) && is_string($conditionalStylesheet)) {
  31                 $conditionalStylesheet = (string) $conditionalStylesheet;
  32             } else {
  33                 $conditionalStylesheet = null;
  34             }
  35         }
  36 
  37         if (0 < count($args) && is_array($args[0])) {
  38             $extras = array_shift($args);
  39             $extras = (array) $extras;
  40         } else {
  41             $extras = (array) $extras;
  42         }
  43 
  44         $attributes = compact('rel', 'type', 'href', 'media', 'conditionalStylesheet', 'extras');
  45 
  46         return $this->createData($attributes);
  47     }

I'm not a programmer per definition - but works, so far.

"continue" targeting switch is equivalent to "break"

https://www.php.net/manual/de/control-structures.continue.php

Error is in:
/usr/share/bareos-webui/vendor/zendframework/zend-stdlib/src/ArrayObject.php

Create a patch file
ArrayObject.php.patch

   1 426c426
   2 <                     continue;
   3 ---
   4 >                     continue 2;

   1 patch /usr/share/bareos-webui/vendor/zendframework/zend-stdlib/src/ArrayObject.php \
   2 < ArrayObject.php.patch

fwrite(): send of 26 bytes failed with errno=104

Notice: fwrite(): send of 26 bytes failed with errno=104 Connection reset by peer in /usr/share/bareos-webui/vendor/Bareos/library/Bareos/BSock/BareosBSock.php on line 219

Apply both solutions:

BareosBSock.php.patch

   1 538c538,548
   2 <          $crypto_method = STREAM_CRYPTO_METHOD_TLSv1_2_CLIENT;
   3 ---
   4 >          /*
   5 >          * STREAM_CRYPTO_METHOD_TLSv1_2_CLIENT was introduced with PHP version
   6 >          * 5.6.0. We need to care that calling stream_socket_enable_crypto method
   7 >          * works with versions < 5.6.0 as well.
   8 >          */
   9 >          $crypto_method = STREAM_CRYPTO_METHOD_TLS_CLIENT;
  10 > 
  11 >          if (defined('STREAM_CRYPTO_METHOD_TLSv1_2_CLIENT')) {
  12 >             $crypto_method |= STREAM_CRYPTO_METHOD_TLSv1_2_CLIENT;
  13 >          }
  14 >

   1 patch /usr/share/bareos-webui/vendor/Bareos/library/Bareos/BSock/BareosBSock.php \
   2 < BareosBSock.php.patch

Disable TLS in
/etc/bareos/bareos-dir.d/console/admin.conf

   1 #
   2 # Restricted console used by bareos-webui
   3 #
   4 Console {
   5   Name = admin
   6   Password = "jekoo2OdaiQuaeng"
   7   Profile = "webui-admin"
   8   TLS Enable = No
   9 }

BConsole webview to small

Bconsole width and height are limited to 1080x650 but can be adjusted.

Adjust bconsole size /usr/share/bareos-webui/module/Director/view/director/director/console.phtml

   1 <pre class="bconsole" id="bconsole" style="font-size: 9pt; padding-left: 5px; overflow: scroll; width: 98vw; height: 80vh; background: #000; color: #fff;" onclick="focusScreen();">
   2    <p><?php echo $this->translate("bconsole (batch-mode), please handle with care."); ?><br /><?php echo $this->translate("All commands have to be a one liner, dialogs are not working."); ?><br /><?php echo $this->translate("Type help for a list of commands."); ?></p>
   3    <div class="prompt"><label> * </label><input type="text" class="cli" id="cli" value ="" size=120 style="display: inline; background: #000; color: #fff; border-color: #000; border-style: hidden; outline: none;"/></div>
   4 </pre>

console.phtml.patch

   1 41c41
   2 < <pre class="bconsole" id="bconsole" style="font-size: 9pt; padding-left: 5px; overflow: scroll; width: 1080px; height: 650px; background: #000; color: #fff;" onclick="focusScreen();">
   3 ---
   4 > <pre class="bconsole" id="bconsole" style="font-size: 9pt; padding-left: 5px; overflow: scroll; width: 98vw; height: 80vh; background: #000; color: #fff;" onclick="focusScreen();">

   1 patch /usr/share/bareos-webui/module/Director/view/director/director/console.phtml \
   2 < console.phtml.patch

Bareos Tray Monitor

Does not start correctly, when it is called from application menu, becasue it is called with a configuration file, that does not exist.

   1 % /usr/bin/bareos-tray-monitor -c /etc/bareos/tray-monitor.conf
   2 14-May 15:59 tray-monitor JobId 0: Error: lib/parse_conf.cc:543 Failed to read config file "/etc/bareos/tray-monitor.conf"
   3 tray-monitor ERROR TERMINATION
   4 lib/parse_conf.cc:205 Failed to find config filename.

A strace told me, that bareos tray monitor tries to read this files by default.

   1 …
   2 stat("/etc/bareos", {st_mode=S_IFDIR|0755, st_size=126, ...}) = 0
   3 stat("/etc/bareos/tray-monitor.conf", 0x7fffd58d6b20) = -1 ENOENT (Datei oder Verzeichnis nicht gefunden)
   4 stat("/etc/bareos/tray-monitor.d", {st_mode=S_IFDIR|0755, st_size=26, ...}) = 0
   5 …

So we can just leave the option -c and its argument away. Just make a copy of the entry in the application launcher and change the exec call to /usr/bin/bareos-tray-monitor. You may now start this with your GUI-Session.