docker
Contents
About
Installation
Install Docker according to:
Clean up the old stuff. Variable data and configuration is preserved.
1 sudo apt-get remove docker docker-engine docker.io containerd runc
Install docker-ce
1 aptitude install \
2 apt-transport-https \
3 ca-certificates \
4 curl \
5 gnupg-agent gpg \
6 software-properties-common
7 curl -fsSL "https://download.docker.com/linux/debian/gpg" \
8 |sudo sh -c 'gpg --dearmor \
9 > /usr/share/keyrings/docker-keyring.gpg'
10 ln -s /usr/share/keyrings/docker-keyring.gpg \
11 /etc/apt/trusted.gpg.d
12 echo "deb https://download.docker.com/linux/debian " \
13 "$(lsb_release -cs) stable" \
14 > /etc/apt/sources.list.d/docker.list
15 aptitude update
16 apt-cache madison docker-ce
17 aptitude install docker-ce docker-ce-cli containerd.io docker-compose
18 docker run --rm hello-world
Configure
Default network "bridge"
https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-configuration-file
https://docs.docker.com/engine/reference/commandline/dockerd/
List all networks available
docker network ls
docker network inspect bridge
1 [
2 {
3 "Name": "bridge",
4 "Id": "8ab5cdf06024e78917d269837d62ce9c0e814acf95acc5baab7a9b7e5077ee90",
5 "Created": "2020-08-26T18:02:07.498361623+02:00",
6 "Scope": "local",
7 "Driver": "bridge",
8 "EnableIPv6": false,
9 "IPAM": {
10 "Driver": "default",
11 "Options": null,
12 "Config": [
13 {
14 "Subnet": "172.18.0.0/16",
15 "Gateway": "172.18.0.1"
16 }
17 ]
18 },
19 "Internal": false,
20 "Attachable": false,
21 "Ingress": false,
22 "ConfigFrom": {
23 "Network": ""
24 },
25 "ConfigOnly": false,
26 "Containers": {
27 "35219be17ee106bd94f7b010e8a56d83c428c842208939c273cdad8bcab1fb7e": {
28 "Name": "happy_chatterjee",
29 "EndpointID": "9fcb38ea8d1522fcd6d8835409adf0f7bb41835c7aed02695c3ca77dd19d80c6",
30 "MacAddress": "02:42:ac:12:00:02",
31 "IPv4Address": "172.18.0.2/16",
32 "IPv6Address": ""
33 }
34 },
35 "Options": {
36 "com.docker.network.bridge.default_bridge": "true",
37 "com.docker.network.bridge.enable_icc": "true",
38 "com.docker.network.bridge.enable_ip_masquerade": "true",
39 "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
40 "com.docker.network.bridge.name": "docker0",
41 "com.docker.network.driver.mtu": "1500"
42 },
43 "Labels": {}
44 }
45 ]
In my specific case the network 172.18.0.0/16 collides with the host network and must be changed.
/etc/docker/daemon.json
The default-gateway has to be with in fixed-cidr. If you specify a default-gateway make sure you don't reuse the bridge ip or it will collide and the daemon does not start up. So if you specify it it must differ, otherwise just leave it away.
1 systemctl restart docker.service
Docker nftables
Gather some information
System information
Validate your configuration
There is a script to support configuration validation at
/usr/share/docker.io/contrib/check-config.sh
1 warning: /proc/config.gz does not exist, searching other paths for kernel config ...
2 info: reading kernel config from /boot/config-5.9.0-1-amd64 ...
3
4 Generally Necessary:
5 - cgroup hierarchy: properly mounted [/sys/fs/cgroup]
6 - apparmor: enabled and tools installed
7 - CONFIG_NAMESPACES: enabled
8 - CONFIG_NET_NS: enabled
9 - CONFIG_PID_NS: enabled
10 - CONFIG_IPC_NS: enabled
11 - CONFIG_UTS_NS: enabled
12 - CONFIG_CGROUPS: enabled
13 - CONFIG_CGROUP_CPUACCT: enabled
14 - CONFIG_CGROUP_DEVICE: enabled
15 - CONFIG_CGROUP_FREEZER: enabled
16 - CONFIG_CGROUP_SCHED: enabled
17 - CONFIG_CPUSETS: enabled
18 - CONFIG_MEMCG: enabled
19 - CONFIG_KEYS: enabled
20 - CONFIG_VETH: enabled (as module)
21 - CONFIG_BRIDGE: enabled (as module)
22 - CONFIG_BRIDGE_NETFILTER: enabled (as module)
23 - CONFIG_NF_NAT_IPV4: missing
24 - CONFIG_IP_NF_FILTER: enabled (as module)
25 - CONFIG_IP_NF_TARGET_MASQUERADE: enabled (as module)
26 - CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: enabled (as module)
27 - CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled (as module)
28 - CONFIG_NETFILTER_XT_MATCH_IPVS: enabled (as module)
29 - CONFIG_IP_NF_NAT: enabled (as module)
30 - CONFIG_NF_NAT: enabled (as module)
31 - CONFIG_NF_NAT_NEEDED: missing
32 - CONFIG_POSIX_MQUEUE: enabled
33
34 Optional Features:
35 - CONFIG_USER_NS: enabled
36 - CONFIG_SECCOMP: enabled
37 - CONFIG_CGROUP_PIDS: enabled
38 - CONFIG_MEMCG_SWAP: enabled
39 - CONFIG_MEMCG_SWAP_ENABLED: missing
40 (cgroup swap accounting is currently enabled)
41 - CONFIG_LEGACY_VSYSCALL_NONE: enabled
42 (containers using eglibc <= 2.13 will not work. Switch to
43 "CONFIG_VSYSCALL_[NATIVE|EMULATE]" or use "vsyscall=[native|emulate]"
44 on kernel command line. Note that this will disable ASLR for the,
45 VDSO which may assist in exploiting security vulnerabilities.)
46 - CONFIG_BLK_CGROUP: enabled
47 - CONFIG_BLK_DEV_THROTTLING: enabled
48 - CONFIG_IOSCHED_CFQ: missing
49 - CONFIG_CFQ_GROUP_IOSCHED: missing
50 - CONFIG_CGROUP_PERF: enabled
51 - CONFIG_CGROUP_HUGETLB: missing
52 - CONFIG_NET_CLS_CGROUP: enabled (as module)
53 - CONFIG_CGROUP_NET_PRIO: enabled
54 - CONFIG_CFS_BANDWIDTH: enabled
55 - CONFIG_FAIR_GROUP_SCHED: enabled
56 - CONFIG_RT_GROUP_SCHED: missing
57 - CONFIG_IP_NF_TARGET_REDIRECT: enabled (as module)
58 - CONFIG_IP_VS: enabled (as module)
59 - CONFIG_IP_VS_NFCT: enabled
60 - CONFIG_IP_VS_PROTO_TCP: enabled
61 - CONFIG_IP_VS_PROTO_UDP: enabled
62 - CONFIG_IP_VS_RR: enabled (as module)
63 - CONFIG_EXT4_FS: enabled (as module)
64 - CONFIG_EXT4_FS_POSIX_ACL: enabled
65 - CONFIG_EXT4_FS_SECURITY: enabled
66 - Network Drivers:
67 - "overlay":
68 - CONFIG_VXLAN: enabled (as module)
69 Optional (for encrypted networks):
70 - CONFIG_CRYPTO: enabled
71 - CONFIG_CRYPTO_AEAD: enabled (as module)
72 - CONFIG_CRYPTO_GCM: enabled (as module)
73 - CONFIG_CRYPTO_SEQIV: enabled (as module)
74 - CONFIG_CRYPTO_GHASH: enabled (as module)
75 - CONFIG_XFRM: enabled
76 - CONFIG_XFRM_USER: enabled (as module)
77 - CONFIG_XFRM_ALGO: enabled (as module)
78 - CONFIG_INET_ESP: enabled (as module)
79 - CONFIG_INET_XFRM_MODE_TRANSPORT: missing
80 - "ipvlan":
81 - CONFIG_IPVLAN: enabled (as module)
82 - "macvlan":
83 - CONFIG_MACVLAN: enabled (as module)
84 - CONFIG_DUMMY: enabled (as module)
85 - "ftp,tftp client in container":
86 - CONFIG_NF_NAT_FTP: enabled (as module)
87 - CONFIG_NF_CONNTRACK_FTP: enabled (as module)
88 - CONFIG_NF_NAT_TFTP: enabled (as module)
89 - CONFIG_NF_CONNTRACK_TFTP: enabled (as module)
90 - Storage Drivers:
91 - "aufs":
92 - CONFIG_AUFS_FS: missing
93 - "btrfs":
94 - CONFIG_BTRFS_FS: enabled (as module)
95 - CONFIG_BTRFS_FS_POSIX_ACL: enabled
96 - "devicemapper":
97 - CONFIG_BLK_DEV_DM: enabled (as module)
98 - CONFIG_DM_THIN_PROVISIONING: enabled (as module)
99 - "overlay":
100 - CONFIG_OVERLAY_FS: enabled (as module)
101 - "zfs":
102 - /dev/zfs: missing
103 - zfs command: missing
104 - zpool command: missing
105
106 Limits:
107 - /proc/sys/kernel/keys/root_maxkeys: 1000000
Maintenance
Docker container, images and volumes can take up some space in on the filesystem after some time.
Remove dangling images
Please see:
man 1 docker-image-ls
man 1 docker-image-prune
List dangling images
1 docker image ls -f dangling=true
Prune dangling images
1 $ docker image prune
2 WARNING! This will remove all dangling images.
3 Are you sure you want to continue? [y/N] y
4 Deleted Images:
5 deleted: sha256:75cb633f04e62e30fe8403a65266f1e3101cd51f364428538829563091204f1e
6 deleted: sha256:0dc256478b8dc35e0c116761f05917628f15313b8ef0770a84ebf23943fcce19
7 deleted: sha256:9391e2662ef789223b524fd1b4c236c1d7d6172a61955c316a815b11f82bcd81
8 …
9 deleted: sha256:02c0bf5d62a2fbf6cd11a27d829f7c20b2f94c2fe863939b05b236d52297b051
10 deleted: sha256:fce68388e832420876d39418fa36312bc660368f56f5f695f5a06c961d49ea10
11 deleted: sha256:7edbc2a3032d4caf9592758803106967428556a813b9d7ede3520b000ba303eb
12
13 Total reclaimed space: 5.165GB
Remove dangling containers
Please see:
man 1 docker-container-ls
man 1 docker-container-prune
List stopped containers
1 docker container ls --filter status=exited
2 CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3 29c14afd7f3f … "docker-entrypoint.s…" 4 weeks ago Exited (1) 4 weeks ago …
4 bbef6da2d86c … "docker-entrypoint.s…" 4 weeks ago Exited (1) 4 weeks ago …
5 92e946506980 ff545529608b "/bin/sh -c 'npm i -…" 2 years ago Exited (1) 2 years ago serene_kepler
Prune stopped containers
1 docker container prune
2 WARNING! This will remove all stopped containers.
3 Are you sure you want to continue? [y/N] y
4 Deleted Containers:
5 29c14afd7f3fc40a7ab8e2849528567f772d0e89829df54c6580b9cfcf4b993d
6 bbef6da2d86c0aeba28b8c270c250bf3ad26b2475b40e7b4993b687c8e8f5664
7 92e946506980d4241f9eb95a33482ac350afb2414f0b4b400c76f92f03c1e4c1
8
9 Total reclaimed space: 180kB
Yeah …
Remove dangling volumes
Please see:
man 1 docker-volume-ls
man 1 docker-volume-prune
List dangling volumes
1 docker volume ls --filter dangling=true
2 DRIVER VOLUME NAME
3 local 0cfd96e05e4e6ee7d4e1b151fd62a2556a84a8dea1dc72eaff93c3f3d0be40ae
4 local 1b71a24e918dfe33710e336dafc57622535dd4d69a3d514c8024820f633b8ef1
5 local 1c0d4b21c4934c26c2f26e08d69dbcf13e1665664e45c0b3a1769a5f1e2352e2
6 local 1d0b28010382498be1fe50b5518e188fd7d9fa282b0490793ba52d700cebfab5
7 …
8 local f3a40d83dbc36cda240c32e5099f7b9e92a7c922d623461e9db6bb607cf2c5d1
9 local f701d4aa8fcb5d0a0552596a8902d5c599190a72abf6c1582d0d0ccd8e0ef575
10 local fb125f6ecc06922c11a9981c6bb7f6d6e84656015264f4843b8a17d6916abde7
Prune dangling volumes
1 docker volume prune
2 WARNING! This will remove all local volumes not used by at least one container.
3 Are you sure you want to continue? [y/N] y
4 Deleted Volumes:
5 07d1d6705c38fd1d20ad0e4d1e5a4ecc665d0b13f7d3e9cb3a772b65cb1fc051
6 e0c880d3f408e19e130dc2210535d64a73a9ece39fd66220b36bf92f4f2e1516
7 7bce8fadce14a659fb717d0121a1b2e4a393c8f9097fc2fa754d17d984b6377a
8 …
9 1d0b28010382498be1fe50b5518e188fd7d9fa282b0490793ba52d700cebfab5
10 f3a40d83dbc36cda240c32e5099f7b9e92a7c922d623461e9db6bb607cf2c5d1
11 85fb8bb39d7ac80733418e464055a901337fb9f01f0ef4825b3d8e14be9129d8
12
13 Total reclaimed space: 0B
Even more yeah…
Remove containers
Please see:
man 1 docker-image-ls
man 1 docker-image-rm
Remove hand-selected images
Get all tags of an image from dockerhub
docker_list-tags.sh
1 #!/bin/bash
2
3 if [ -z "$1" ]; then
4 cat <<-EOF
5 Please provide a name as parameter like . Exiting …
6 Like: $0 'postgis/postgis'
7 EOF
8 exit 1
9 fi
10
11 NAME="$1"
12 SCHEMA='https'
13 HOST="registry.hub.docker.com"
14 URI_PREFIX="v2/repositories"
15 LAST=false
16
17 NEXT="$SCHEMA://$HOST/$URI_PREFIX/$NAME/tags/"
18 while ! "$LAST"; do
19 PAGE="$(curl -s "$NEXT")"
20 RESULT+="$PAGE"
21 NEXT="$(jq -r '.next' <<< "$PAGE")"
22 if [ ! "$NEXT" ]; then
23 LAST=true
24 fi
25 done
26
27 jq -r '."results"[]["name"]' <<< "$RESULT" \
28 |sort -V
Privileged ports
To allow the usage of privileged ports lower (<=1024), the kernel namespace the unprivileged container is run in, can be configured to allow this binding.
Please see
man -P "less -p '--sysctl=SYSCTL'" 1 docker-run
--sysctl=SYSCTL
- Configure namespaced kernel parameters at runtime
- Sysctls beginning with fs.mqueue.*
If you use the --ipc=host option these sysctls will not be allowed. Network Namespace - current sysctls allowed:
- Sysctls beginning with net.*
If you use the --network=host option these sysctls will not be allowed.
Run the container
Reload the configuration
1 docker kill -s HUP my-running-haproxy
Dockerfile
* Docker docs - Dockerfile reference
docker-compose
Environment variables
Docker docs - compose environment
Starting with +v1.28, .env file is placed at the base of the project directory
- Project directory can be explicitly defined
with the --file option or COMPOSE_FILE environment variable. Otherwise, it is the current working directory where the docker compose command is executed (+1.28).
- You can override the environment file path
using a command line argument --env-file
- When you set the same environment variable in multiple files,
here’s the priority used by compose to choose which value to use:
- Compose file
- Shell environment variables
- Environment file
- Dockerfile
- Variable is not defined
Docker compose file
Environment example
IMHO hidden files are a pitfalls in most scenarios. Therefor I decided to use explicitly defined environment files.
The current configuration and the variable substitution may be checked with
docker-compose config
Now adjust the environment to your needs
$CONFIG/env.prod
1 TAG="1.0"
$CONFIG/env.dev
1 TAG="1.1-rc1"
$CONFIG/docker-compose.yml
The current configuration and the variable substitution may be checked with
docker-compose config
1 # cd "$PROJECT"
2 # ls -la . etc
3 .:
4 total 580
5 drwxr-xr-x 7 root root 4096 Oct 27 12:14 .
6 drwxr-xr-x 3 root root 4096 Oct 27 11:46 ..
7 lrwxrwxrwx 1 root root 19 Oct 27 12:11 .env -> etc/env.dev
8 lrwxrwxrwx 1 root root 30 Oct 27 12:14 docker-compose.yml -> etc/docker-compose.yml
9 drwxr-xr-x 3 root root 4096 Oct 27 12:14 etc
10
11 etc:
12 total 36
13 drwxr-xr-x 3 root root 4096 Oct 27 12:14 .
14 drwxr-xr-x 7 root root 4096 Oct 27 12:14 ..
15 -rw-r--r-- 1 root root 55 Oct 27 12:10 docker-compose.yml
16 -rw-r--r-- 1 root root 767 Oct 27 11:39 env.dev
17 -rw-r--r-- 1 root root 767 Oct 27 10:38 env.prod
18
19 # docker-compose \
20 --file etc/docker-compose.yml \
21 --env-file etc/env.dev \
22 config
23 services:
24 web:
25 image: webapp:"1.1-rc1"
26 version: '3.0'
27
28 # docker-compose \
29 --file etc/docker-compose.yml \
30 --env-file etc/env.prod \
31 config
32 services:
33 web:
34 image: webapp:"1.0"
35 version: '3.0'
36
37 # docker-compose \
38 --file etc/docker-compose.yml \
39 config
40 services:
41 web:
42 image: webapp:"1.1-rc1"
43 version: '3.0'
44 # docker-compose config
45 services:
46 web:
47 image: webapp:"1.1-rc1"
48 version: '3.0'
As convenient as before, but much more clear.
Startup compose
Startup in detached more writing a colorful log and stop it again.
Some actions