ssh

Contents

About

OpenSSH is the premier connectivity tool for remote login with the SSH protocol. It encrypts all traffic to eliminate eavesdropping, connection hijacking, and other attacks. In addition, OpenSSH provides a large suite of secure tunneling capabilities, several authentication methods, and sophisticated configuration options.

directory structure

openssh creates a directory $HOME/.ssh in your home-directory typically with the following files

ls -d1 .ssh/*

   1 .ssh/authorized_keys
   2 .ssh/config
   3 .ssh/id_rsa
   4 .ssh/id_rsa.pub
   5 .ssh/known_hosts

authorized_keys contains public keys for public key authentication
config user's configuration file for OpenSSH client
id_rsa PRIVATE KEY: Contains the DSA, ECDSA, authenticator-hosted ECDSA, Ed25519, authenticator-hosted Ed25519 or RSA authentication identity of the user.
id_rsa.pub PUBLIC KEY: Contains the DSA, ECDSA, authenticator-hosted ECDSA, Ed25519, authenticator-hosted Ed25519 or RSA public key for authentication.
known_hosts: Contains host public keys for all known hosts.

Modularity using Include

I recommend using Include statements to keep your configs clean, modular, shareable, flexible, …

Create directory structure with

local includes with overrides not to be shared
private include shared between your devices
shared includes shared with colleagues.

Either fully local

   1 SSH="$HOME/.ssh"
   2 DIRS=(
   3   "config.local.d"
   4   "config.private.d"
   5   "config.d"
   6 )
   7 for DIR in "${DIRS[@]}" ; do
   8         install -m 700 -d "$SSH/$DIR"
   9 done

Or distributed with e.g. Nextcloud

   1 SSH="$HOME/.ssh"
   2 SSH_NC="$HOME/Nextcloud/.config/ssh"
   3 install -m 700 -d "$SSH/config.local.d"
   4 ITEMS=(
   5   "config.private.d"
   6   "config.d"
   7   "config"
   8 )
   9 for ITEM in "${ITEMS[@]}" ; do
  10         ln -s "$SSH_NC/$ITEM" "$SSH"
  11 done

Include all files in this directory, that end up on .ssh_config
~.ssh/config

   1 Include config.local.d/*.ssh_config
   2 Include config.private.d/*.ssh_config
   3 Include config.d/*.ssh_config

Now do your usual configuration in the files of the sub-directories. You should prefix the files with a number, because order matters.
~/.ssh/config.d/00_first.ssh_config

   1 ### YOUR SSH_CONFIG
   2 host alice
   3         username eve

Enjoy your config everywhere.

shell aliases

   1 alias insecscp='command scp -o "StrictHostKeyChecking=no" -o "UserKnownHostsFile=/dev/null"'
   2 alias insecssh='command ssh -o "StrictHostKeyChecking=no" -o "UserKnownHostsFile=/dev/null"'
   3 alias s='command ssh'

Log in as root

Logging in as root over ssh by password is disabled by the default configuration. Because it is

not accountable and therefore a security issue.
target to brute-force attacks.
using a shared secret (password of root).
probably unnecessary.

Instead it's recommended at this point to log in to your personal account and elevate privileges using sudo.

Check SSH-HostKey

Please also see the man page

   1 man -P "less -p '^VERIFYING HOST KEYS'" ssh

To check a new method make sure to remove all entries from your known_hosts file. This can (even with a hashed filed) easily be achieved with

   1 ssh-keygen -R "$FQDN" #-f ~/.ssh/known_hosts
   2

TOFU

Usually SSH host keys are validated using the paradigm
Wiki EN - Trust On First Use (TOFU).

The TOFU aspect of this application forces a sysadmin (or other trusted user) to validate the remote server's identity upon first connection.

This manual approach scales quickly beyond the capabilities of the users for

large computer networks
often changing host keys
networks with hardly reachable administrators
…

This may lead to

alarm fatique and
blind trust and

thus leaves man-in-the-middle attacks undetected in place.

Check SSH-HostKey using the DNS SSHFP RR

Having these SSHFP resource records in place is a really nice feature for e.g.

#SFTP servers
bastion hosts
actually hosts

Please see the man page

   1 man -P "less -p '^VerifyHostKeyDNS'" ssh_config

Check SSHFP record in DNS (if published)

   1 ssh -o "VerifyHostKeyDNS ask"

For the creation of the resource records please see
DNS#SSHFP

Persist config in
~/.ssh/config

   1 VerifyHostKeyDNS        ask

Check SSH-HostKey using a small ASCII graphic

Please see the man page

   1 man -P "less -p '^VisualHostKey'" ssh_config

   1 ssh-keygen -l -v -f /etc/ssh/ssh_host_rsa_key.pub

Check the host key using a small ASCII graphic

   1 ssh -o "VisualHostKey yes" "$REMOTE_HOST"

Persist config in
~/.ssh/config

   1 VisualHostKey           yes

HostKey changed

Probably the ssh hostkey did not change, but the

Routing is different (still in a VPN?)
DNS resolution is different
Resolved address is very common like 192.168.1.1

Disable Hostkey checking

When you are frequently configuring the same addresses in different networks you will always have to delete the old entry from known_hosts. This is annoying.

To suppress the ssh hostkey check of a session

   1 ssh -o "StrictHostKeyChecking=no" -o "UserKnownHostsFile=/dev/null
   2 scp -o "StrictHostKeyChecking=no" -o "UserKnownHostsFile=/dev/null

You can also create some #shell aliases for quick access.

To permanently suppress the ssh hostkey check for a remote server, just configure your users
~/.ssh/config

   1 host openwrt
   2      hostname 192.168.1.1
   3      StrictHostKeyChecking no
   4      UserKnownHostsFile /dev/null

pubkey auth

The private key

FOR YOUR EYES ONLY!

Safety Rules

Make sure storage of your private key meets all of the following criteria.

The file

owner is set to yourself
group is set to your own
only readable by yourself (only 0600 )
and is always encrypted
is stored on a non-public accessible storage
has a backup which is stored secure to
is never in the hands of other people (no matter how trustworthy they are).

Be careful to not mix up private and public key. The public key simply adds the {{{.pub

extension.So make sure you are working with the *.pub files. }}}

Formating

Encoding formats

The openssh tool ssh-keygen will create new keypairs in its own openssh format. But also supports outputing and converting the different formats into each other with the option -m. There are 3 arguments 3rd-party

RFC4716 / OpenSSH new
- -----BEGIN OPENSSH PRIVATE KEY-----
- ---- BEGIN SSH2 PUBLIC KEY ----
PKCS8
- -----BEGIN PRIVATE KEY-----
- -----BEGIN PUBLIC KEY-----
PEM / OpenSSH old
- deprecated for OpenSSH
- Converted by puttygen to private-openssh
- -----BEGIN RSA PRIVATE KEY-----
- -----BEGIN RSA PUBLIC KEY-----

OpenSSH public keys

are on a single line and
mostly have 3 parts separated by spaces
1. Type (like ssh-rsa or ecdsa-sha2-nistp521 or …)
2. Key
3. Comment

Transcode formats

Import 3rd-party key from source format (default: -m rfc4716) and output it to the OpenSSH-compatible (PKCS8)

   1 ssh-keygen -i -f id_rsa_source.ppk \
   2        |tee id_rsa_imported

Export key from OpenSSH-compatible private/public key and output a public key in 3rd-party format (default: -m rfc4716)

   1 ssh-keygen -e -f id_rsa_source \
   2         |tee id_rsa_rfc4716.pub
   3 ssh-keygen -e -f id_rsa_source.pub \
   4         |tee id_rsa_rfc4716.pub
   5 ssh-keygen -e -f id_rsa_source -m pkcs8 \
   6         |tee id_rsa_pkcs8.pub

Putty format

Install putty-tools

   1 apt install putty-tools

Conversion

   1 ### CONVERT PRIVATE KEY TO PUTTY FORMAT
   2 #puttygen -O output-type -o output-file source/keyfile
   3 puttygen -O private -o id_rsa.ppk id_rsa
   4 ### CONVERT PUTTY PRIVATE KEY TO OPENSSH-FORMAT private-openssh (PEM)
   5 puttygen -O private-openssh -o id_rsa_openssh id_rsa.ppk
   6 ### CONVERT PUTTY PRIVATE KEY TO OPENSSH-FORMAT private-openssh-new (RFC4716)
   7 puttygen -O private-openssh-new -o id_rsa_openssh_new id_rsa.ppk

Reformat white-spaces in key file

Add newlines back to a missformated private key

   1 sed -r 's/ (-----)/\n\1/' < id_rsa_test \
   2         |fold -s -w 71 > id_rsa_test_reformated

Special operations on the keypair

Create public key from private key

Read private key in openssh format and output public key in openssh format

   1 ssh-keygen -y -f id_rsa_source \
   2         |tee id_rsa_source.pub

Remove passphrase from private key

Change/remove password

   1 ### CREATE BACKUP
   2 cp -p id_rsa_source id_rsa.bak
   3 ### CHANGE PASSWORD WITH PROMPT
   4 ssh-keygen -p -N "" -f id_rsa_destination
   5 ### CHANGE PASSWORD NON-INTERACTIVELY
   6 ssh-keygen -p -N "" -f id_rsa_destination
   7 ### CONVERT PRIVATE KEY TO PKCS8
   8 ssh-keygen -p -N "" -f id_rsa_destination

PubKey auth on Linux and Unix

Generate a key pair for pubkey auth

RSA or
- length >= 4096 bit
ECDSA
- you may specify the curve with -b (256, 384 or 521).

Make sure to not accidently overwrite a key-pair in the default location when creating a new one.

Check with
ls -l $HOME/.ssh/id_*

The key comment may be useful to help identify the key. The comment is initialized to "user@host" when the key is created, but can be changed using the -c option. An email-address is usually a good choice for a comment.

Use ssh-keygen to generate a key-pair

   1 ### CREATE ~/.ssh/id_rsa*
   2 ssh-keygen -t rsa -b 4096 -C "user@domain.tld"
   3 ### CREATE ~/.ssh/id_ecdsa*
   4 ssh-keygen -t ecdsa -b 521 -C "user@domain.tld"
   5 ### THE COMMENT MAY ALSO BE LEFT OUT
   6 ### TO CREATE USER@HOST STYLE COMMENTS
   7 ssh-keygen -t ecdsa -b 521

ssh-agent on *nix

If you are using the key a lot, it is more convenient to unlock/decrypt your private key and keep it to RAM using ssh-agent.

Add the following lines to your respective shells configuration file to start a session.

With shell auto-detection

   1 eval $(ssh-agent)

Explicit Bourne Shell commands ~/.bashrc

   1 eval $(ssh-agent -s)

Explicit C-Shell commands ~/.cshrc

   1 eval $(ssh-agent -c)

To add all keys from the default location ~/.ssh/id_*, just type ssh-add and you will be asked to provide a pass-phrase to decrypt the keys.

It's quite convenient to have all keys encrypted with the same pass-phrase.

You may also add a single key, which e.g. may not reside in the default location, to the agent with
ssh-add -x $HOME/path/to/id_file

The ssh-agent may
be locked with ssh-add -x
and locked with ssh-add -X

You may delete all keys from the ssh-agent with
ssh-add -D.

PubKey auth on Win10

There are already a number of very good tutorials

docs.microsoft.com OpenSSH key management

Install the powershell module for openssh (from an elevated shell)

   1 ### INSTALL THE OPENSSHUTILS MODULE TO THE SERVER.
   2 ### THIS WILL BE VALUABLE WHEN DEPLOYING USER KEYS.
   3 Install-Module -Force OpenSSHUtils -Scope AllUsers

Generate a key pair for pubkey auth - Windows

Please the the *nix variant as it no longer differs since the integration of ssh into powershell
#Generate a key pair for pubkey auth and return here later.

ssh-agent on Windows

Start the ssh-service (probably for this session only)

   1 ### SET THE STARTUP-TYPE OF THE SSH-AGENT TO MANUAL
   2 Get-Service -Name ssh-agent | Set-Service -StartupType Manual
   3 
   4 ### START THE SSH-AGENT SERVICE TO PRESERVE THE SERVER KEYS
   5 Start-Service ssh-agent
   6 
   7 ### LIKE THIS YOU MAY ALSO START THE SSHD SERVICE
   8 # Start-Service sshd
   9

Check agent status and import the private keys

   1 ### THIS SHOULD RETURN A STATUS OF RUNNING
   2 Get-Service ssh-agent
   3 
   4 ### NOW LOAD YOUR KEY FILES INTO SSH-AGENT
   5 ssh-add
   6 ### ADD A SPECIFIC KEY TO THE AGENT
   7 ssh-add ~/path/to/id_file
   8 ### LOCK THE AGENT WITH A PASSWORD
   9 ssh-add -x
  10 ### UNLOCK THE AGENT
  11 ssh-add -X
  12 ### DELETE ALL IDENTITIES FROM THE AGENT
  13 ssh-add -D

Now log in to your favorite host… :-D

Test pubkey auth on localhost

Now you can prepare pubkey auth. Therefore the pubkey needs to be appended to the list of authorized keys on the target system. Be careful not to leak you private key!
Example on localhost

   1 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

You can test it by logging in to localhost via ssh.

   1 ssh localhost

When encountering problems please take a look at:
tail -f /var/log/auth.log or on RedHat
tail -f /var/log/secure

Distribute your pubkeys

You also may copy your pubkey to systems, that reachable via ssh and you are able to login.

   1 ssh-copy-id $REMOTE_USER@$REMOTE-SYSTEM

Or append content of you public keys files to the remote authorized_keys, when you are rotating you key-pairs.

   1 ssh -i ~/.ssh/old/id_rsa \
   2         tobias@git.rockstable.it -- \
   3                 'echo "'"$(cat ~/.ssh/id_{rsa,ecdsa}.pub)"'" \
   4                         >> ~/.ssh/authorized_keys'

You now may login without password prompt by the remote system.

   1 ssh $REMOTE_USER@$REMOTE-SYSTEM

Certificate authentication

WIP

RedHat Docs - Using OpenSSH Certificate Authentication
Added in OpenSSH release 5.4
- Add support for certificate authentication of users and hosts using a new, minimal OpenSSH certificate format (not X.509). Certificates contain a public key, identity information and some validity constraints and are signed with a standard SSH public key using ssh-keygen(1). CA keys may be marked as trusted in authorized_keys or via a TrustedUserCAKeys option in sshd_config(5) (for user
  authentication), or in known_hosts (for host authentication).
- Documentation for certificate support may be found in ssh-keygen(1), sshd(8) and ssh(1) and a description of the protocol extensions in PROTOCOL.certkeys.

Generate a CA

It is recommended to

use different CAs to sign user and host certificates.
store the CA signing keys securely on an offline machine.

Generate CAs for signing

   1 DIR_CA="/root/.ssh/CA"
   2 mkdir -pv "$DIR_CA"
   3 cd "$DIR_CA"
   4 ssh-keygen -t rsa -f "$DIR_CA/ca_host_key"
   5 ssh-keygen -t rsa -f "$DIR_CA/ca_user_key"

Distribute and configure CAs

Show certificate information

Show information about a certificate

   1 ssh-keygen -L -f input-cert

Bugs in OpenSSH for Win32

If you are using ssh with a JumpHost you might run in to a bug with older ssh versions (<8.1). https://github.com/PowerShell/Win32-OpenSSH/issues/1172

   1 posix_spawn: No such file or directory”

Please update to the latest Version of Windows first! But this is not fixed in Windows 10 v2004…

Here are some links to a installation advisory on

You'll probably need the latest ssh-version OpenSSH-Win64.zip from https://github.com/PowerShell/Win32-OpenSSH/releases

Please use the script install-sshd.ps1 with administrative permissions. powershell.exe -ExecutionPolicy Bypass -File install-sshd.ps1

Check it with ssh -V

You may be forced to list the new openssh-directory in the PATH variable in front of %SYSTEMROOT%\System32\OpenSSH\.

ssh escape characters

A little, often unknown, but very useful feature of openssh!

   1 ESCAPE CHARACTERS
   2      When a pseudo-terminal has been requested, ssh supports a number of func‐
   3      tions through the use of an escape character.
   4 
   5      A single tilde character can be sent as ~~ or by following the tilde by a
   6      character other than those described below.  The escape character must
   7      always follow a newline to be interpreted as special.  The escape charac‐
   8      ter can be changed in configuration files using the EscapeChar configura‐
   9      tion directive or on the command line by the -e option.
  10 
  11      The supported escapes (assuming the default ‘~’) are:
  12 
  13      ~.      Disconnect.
  14 
  15      ~^Z     Background ssh.
  16 
  17      ~#      List forwarded connections.
  18 
  19      ~&      Background ssh at logout when waiting for forwarded connection /
  20              X11 sessions to terminate.
  21 
  22      ~?      Display a list of escape characters.
  23 
  24      ~B      Send a BREAK to the remote system (only useful if the peer sup‐
  25              ports it).
  26 
  27      ~C      Open command line.  Currently this allows the addition of port
  28              forwardings using the -L, -R and -D options (see above).  It also
  29              allows the cancellation of existing port-forwardings with
  30              -KL[bind_address:]port for local, -KR[bind_address:]port for re‐
  31              mote and -KD[bind_address:]port for dynamic port-forwardings.
  32              !command allows the user to execute a local command if the
  33              PermitLocalCommand option is enabled in ssh_config(5).  Basic
  34              help is available, using the -h option.
  35 
  36      ~R      Request rekeying of the connection (only useful if the peer sup‐
  37              ports it).
  38 
  39      ~V      Decrease the verbosity (LogLevel) when errors are being written
  40              to stderr.
  41 
  42      ~v      Increase the verbosity (LogLevel) when errors are being written
  43              to stderr.

keyboard-interactive authentication

sshpass

Please try pubkey-authentication first, before trying keyboard-interactive auth with sshpass.
Never use option -ppassword, because everybody can read it the password ps or top (using a race condition before obfuscation by sshpass).

Install ssh-pass

   1 aptitude install sshpass

via file descriptor

Source: Serverfault: How to automate ssh login with password

Example

   1 #!/bin/bash
   2 # Generate a name for a pipe (-u|--dry-run)
   3 PIPE="$(mktemp -u)"
   4 # Create FIFO pipe
   5 mkfifo -m 600 "$PIPE"
   6 # Opened pipe for both reading and writing on file descriptor 3
   7 exec 3<>"$PIPE"
   8 # Delete the directory entry
   9 rm "$PIPE"
  10 UIDNAME="user"
  11 HOST="host"
  12 FILE="path/to/file"
  13 # Write your password to the pipe.
  14 # You may even use gpg at this point.
  15 echo 'my_secret_password' >&3
  16 # Read password with sshpass from file descriptor 3 and
  17 # connect via sftp
  18 sshpass -d3 sftp "$UIDNAME"@"$HOST":"$FILE"
  19 # Close the pipe when done
  20 exec 3>&-

environment variable

   1 #!/bin/bash
   2 # MAKE VARIABLE SSHPASS AVAILABLE
   3 # IN THE ENVIRONMENT OF ANY SUBSEQUENT COMMAND
   4 export SSHPASS="my_secret_password"
   5 # SSHPASS ENVIRONMENT VARIABLE MAY BE READ
   6 # FROM "/proc/$PID/environ"
   7 # BY THE INVOKING USER AND ROOT
   8 sshpass -e sftp user@host:path/to/file
   9 unset SSHPASS

password file

Make sure Unix-permissions are set correctly (replace "$OWNER")!

   1 install -m 600 -o "$OWNER" path/to/password.file
   2 ### WRITE PASSWORD IN "password.file"
   3

   1 #!/bin/bash
   2 export PWFILE="path/to/password.file"
   3 sshpass -f"PWFILE" sftp user@host:path/to/file
   4 unset PWFILE

Query features locally

Query help

   1 ssh -Q help
   2 cipher
   3 cipher-auth
   4 compression
   5 kex
   6 kex-gss
   7 key
   8 key-cert
   9 key-plain
  10 key-sig
  11 mac
  12 protocol-version
  13 sig

Client Alive Checks

/etc/ssh/sshd_config

   1      ClientAliveCountMax
   2              Sets the number of client alive messages which may be sent with-
   3              out sshd(8) receiving any messages back from the client.  If this
   4              threshold is reached while client alive messages are being sent,
   5              sshd will disconnect the client, terminating the session.  It is
   6              important to note that the use of client alive messages is very
   7              different from TCPKeepAlive.  The client alive messages are sent
   8              through the encrypted channel and therefore will not be spoofa-
   9              ble.  The TCP keepalive option enabled by TCPKeepAlive is spoofa-
  10              ble.  The client alive mechanism is valuable when the client or
  11              server depend on knowing when a connection has become unrespon-
  12              sive.
  13 
  14              The default value is 3.  If ClientAliveInterval is set to 15, and
  15              ClientAliveCountMax is left at the default, unresponsive SSH
  16              clients will be disconnected after approximately 45 seconds.
  17              Setting a zero ClientAliveCountMax disables connection termina-
  18              tion.
  19 
  20      ClientAliveInterval
  21              Sets a timeout interval in seconds after which if no data has
  22              been received from the client, sshd(8) will send a message
  23              through the encrypted channel to request a response from the
  24              client.  The default is 0, indicating that these messages will
  25              not be sent to the client.

Old remote ssh-server

The cure is to upgrade the remote software, but if there is no other choice…

there is maybe some legacy support in OpenSSH
http://www.openssh.com/legacy.html

Please check the OpenSSH release notes

no matching cipher

The error

   1 Unable to negotiate with UNKNOWN port 65535:
   2 no matching cipher found.
   3 Their offer: aes128-cbc,3des-cbc,des-cbc

The man page

   1 man -P "less -p '^\s*Ciphers'" 5 ssh_config

Specify the options manually on the cmdline

   1 ssh -o "ciphers +3des-cbc" remote-host

You may prefer a permanent solution in
~/.ssh/config

   1 host    remote-host
   2         hostname        192.168.255.11
   3         ciphers         +3des-cbc

no matching key exchange method

The error

   1 Unable to negotiate with UNKNOWN port 65535:
   2 no matching key exchange method found.
   3 Their offer:
   4 diffie-hellman-group-exchange-sha1,
   5 diffie-hellman-group14-sha1,
   6 diffie-hellman-group1-sha1

The OpenSSH release notes confirm the deactivation of these algorithms.

KEx-method	Removed in OpenSSH	Notes
diffie-hellman-group14-sha1	OpenSSH 8.2/8.2p1 (2020-02-14)	2048-bit
diffie-hellman-group1-sha1	OpenSSH 7.0/7.0p1 (2015-08-11)	1024-bit

diffie-hellman-group-exchange-sha1 needs a minimum modulus of 2048-bit since OpenSSH 7.2/7.2p1 (2016-02-29).

The man page

   1 man -P "less -p '^\s*KexAlgorithms'" 5 ssh_config

Specify the options manually on the cmdline

   1 ssh -o "KexAlgorithms +diffie-hellman-group14-sha1" remote-host

You may prefer a permanent solution in
~/.ssh/config

   1 host    remote-host
   2         hostname        192.168.255.11
   3         KexAlgorithms   +diffie-hellman-group14-sha1

no matching host key type

The error

   1 Unable to negotiate with UNKNOWN port 22:
   2 no matching host key type found.
   3 Their offer: ssh-dss

Specify the options manually on the cmdline

   1 ssh -o "HostKeyAlgorithms +ssh-dss" remote-host

You may prefer a permanent solution in
~/.ssh/config

   1 host    remote-host
   2         hostname            192.168.255.11
   3         HostKeyAlgorithms   +ssh-dss

no mutual signature algorithm

OpenSSH 8.8 release disables RSA signatures using the SHA-1 hash algorithm by default. This change has been made as the SHA-1 hash algorithm is cryptographically broken, and it is possible to create chosen-prefix hash collisions for <USD$50K

You'll see this error message with ssh -vvv

   1 debug1: send_pubkey_test: no mutual signature algorithm

To connect use

   1 ssh     -o HostKeyAlgorithms=+ssh-rsa \
   2         -o PubkeyAcceptedKeyTypes=+ssh-rsa \
   3          old-host

~/.ssh/config

   1     Host old-host
   2         HostkeyAlgorithms +ssh-rsa
   3         PubkeyAcceptedAlgorithms +ssh-rsa

sftp-server: not found

This release switches scp(1) from using the legacy scp/rcp protocol to using the SFTP protocol by default.

You may see this error message with scp

   1 ash: /usr/libexec/sftp-server: not found

To connect anyway use

   1 scp -O remote-host:/root/backup/backup-\*.tar.gz .

Port Forwarding

There are some ultimatively useful methods to forward ports

You may create a simple ssh-connection that detaches

   1 nohup ssh -anNqTx -L localhost:2222:localhost:2222 usr@host
   2 ### YOU MAY ALSO REDIRECT THE STDOUT TO A LOGFILE
   3 #OUT="$HOME/nohup.out"
   4 #touch "$OUT"
   5 #chmod 0600 "$OUT"
   6 #nohup ssh -anNqTx -L localhost:2222:localhost:2222 usr@host > "$OUT"
   7 ### WHERE OPTIONS ARE:
   8 # -a      Disables forwarding of the authentication agent connection.
   9 # -n      Redirects stdin from /dev/null (actually, prevents reading from stdin).
  10 #         This must be used when ssh is run in the background.
  11 #         A common trick is to use this to run X11 programs on a remote machine.
  12 #         For example, ssh -n shadows.cs.hut.fi emacs &
  13 #         will start an emacs on shadows.cs.hut.fi,
  14 #         and the X11 connection will be automatically forwarded over an encrypted channel.
  15 #         The ssh program will be put in the background.
  16 #         (This does not work if ssh needs to ask for a password or passphrase;
  17 #         see also the -f option.) 
  18 #         Refer to the description of StdinNull in ssh_config(5) for details.
  19 # -N      Do not execute a remote command.
  20 #         This is useful for just forwarding ports.
  21 #         Refer to the description of SessionType in ssh_config(5) for details.
  22 # -q      Quiet mode.  Causes most warning and diagnostic messages to be suppressed.
  23 # -T      Disable pseudo-terminal allocation.
  24 # -x      Disables X11 forwarding.
  25

X11 Forwarding

Add options -X or -Y to you ssh cmdline. If you are on a low bandwidth connection, it's a good idea to add -C to enable compression, But it's generally a good idea on todays high performance machines.

   1 ssh -XC remote-host
   2 % xauth list
   3 % ### START YOUR GUI PROGRAMM
   4 % virt-manager

Or make it permanent in
~/.ssh/config

   1 host    remote-host
   2         ForwardX11              yes
   3         Compression             yes

cannot open display

Please check that X11Forwarding yes is enabled on the server side, too. Default is no.

grep '^[^#]' /etc/ssh/sshd_config

   1 Include /etc/ssh/sshd_config.d/*.conf
   2 ChallengeResponseAuthentication no
   3 UsePAM yes
   4 X11Forwarding yes
   5 PrintMotd no
   6 AcceptEnv LANG LC_*
   7 Subsystem       sftp    /usr/lib/openssh/sftp-server

The package xauth has to be installed to set the magic cookie.

   1 dpkg -l xauth 
   2 apt install xauth
   3 ### LOG OUT, IN AND RETRY
   4

Hardening

Positron Security - SSH Hardening Guides

ssh-audit

About ssh-audit

ssh-audit is a tool for ssh server & client configuration auditing.

github.com jtesta/ssh-audit
Please also see #Hardening

Install ssh-audit

Install sshuttle

   1 apt install ssh-audit

Use ssh-audit

Taking a look on a current Debian Trixie system. (Usually the output is colored beautifully.)

   1 % ssh-audit localhost
   2 # general
   3 (gen) banner: SSH-2.0-OpenSSH_9.7p1 Debian-7
   4 (gen) software: OpenSSH 9.7p1
   5 (gen) compatibility: OpenSSH 8.5+, Dropbear SSH 2018.76+
   6 (gen) compression: enabled (zlib@openssh.com)
   7 
   8 # key exchange algorithms
   9 (kex) sntrup761x25519-sha512@openssh.com    -- [info] available since OpenSSH 8.5
  10 (kex) curve25519-sha256                     -- [info] available since OpenSSH 7.4, Dropbear SSH 2018.76
  11                                             `- [info] default key exchange since OpenSSH 6.4
  12 (kex) curve25519-sha256@libssh.org          -- [info] available since OpenSSH 6.4, Dropbear SSH 2013.62
  13                                             `- [info] default key exchange since OpenSSH 6.4
  14 (kex) ecdh-sha2-nistp256                    -- [fail] using elliptic curves that are suspected as being backdoored by the U.S. National Security Agency
  15                                             `- [info] available since OpenSSH 5.7, Dropbear SSH 2013.62
  16 (kex) ecdh-sha2-nistp384                    -- [fail] using elliptic curves that are suspected as being backdoored by the U.S. National Security Agency
  17                                             `- [info] available since OpenSSH 5.7, Dropbear SSH 2013.62
  18 (kex) ecdh-sha2-nistp521                    -- [fail] using elliptic curves that are suspected as being backdoored by the U.S. National Security Agency
  19                                             `- [info] available since OpenSSH 5.7, Dropbear SSH 2013.62
  20 (kex) diffie-hellman-group-exchange-sha256 (3072-bit) -- [info] available since OpenSSH 4.4
  21                                                       `- [info] OpenSSH's GEX fallback mechanism was triggered during testing. Very old SSH clients will still be able to create connections using a 2048-bit modulus, though modern clients will use 3072. This can only be disabled by recompiling the code (see https://github.com/openssh/openssh-portable/blob/V_9_4/dh.c#L477).
  22 (kex) diffie-hellman-group16-sha512         -- [info] available since OpenSSH 7.3, Dropbear SSH 2016.73
  23 (kex) diffie-hellman-group18-sha512         -- [info] available since OpenSSH 7.3
  24 (kex) diffie-hellman-group14-sha256         -- [warn] 2048-bit modulus only provides 112-bits of symmetric strength
  25                                             `- [info] available since OpenSSH 7.3, Dropbear SSH 2016.73
  26 (kex) ext-info-s                            -- [info] pseudo-algorithm that denotes the peer supports RFC8308 extensions
  27 (kex) kex-strict-s-v00@openssh.com          -- [info] pseudo-algorithm that denotes the peer supports a stricter key exchange method as a counter-measure to the Terrapin attack (CVE-2023-48795)
  28 
  29 # host-key algorithms
  30 (key) rsa-sha2-512 (2048-bit)               -- [warn] 2048-bit modulus only provides 112-bits of symmetric strength
  31                                             `- [info] available since OpenSSH 7.2
  32 (key) rsa-sha2-256 (2048-bit)               -- [warn] 2048-bit modulus only provides 112-bits of symmetric strength
  33                                             `- [info] available since OpenSSH 7.2
  34 (key) ecdsa-sha2-nistp256                   -- [fail] using elliptic curves that are suspected as being backdoored by the U.S. National Security Agency
  35                                             `- [warn] using weak random number generator could reveal the key
  36                                             `- [info] available since OpenSSH 5.7, Dropbear SSH 2013.62
  37 (key) ssh-ed25519                           -- [info] available since OpenSSH 6.5
  38 
  39 # encryption algorithms (ciphers)
  40 (enc) chacha20-poly1305@openssh.com         -- [info] available since OpenSSH 6.5
  41                                             `- [info] default cipher since OpenSSH 6.9
  42 (enc) aes128-ctr                            -- [info] available since OpenSSH 3.7, Dropbear SSH 0.52
  43 (enc) aes192-ctr                            -- [info] available since OpenSSH 3.7
  44 (enc) aes256-ctr                            -- [info] available since OpenSSH 3.7, Dropbear SSH 0.52
  45 (enc) aes128-gcm@openssh.com                -- [info] available since OpenSSH 6.2
  46 (enc) aes256-gcm@openssh.com                -- [info] available since OpenSSH 6.2
  47 
  48 # message authentication code algorithms
  49 (mac) umac-64-etm@openssh.com               -- [warn] using small 64-bit tag size
  50                                             `- [info] available since OpenSSH 6.2
  51 (mac) umac-128-etm@openssh.com              -- [info] available since OpenSSH 6.2
  52 (mac) hmac-sha2-256-etm@openssh.com         -- [info] available since OpenSSH 6.2
  53 (mac) hmac-sha2-512-etm@openssh.com         -- [info] available since OpenSSH 6.2
  54 (mac) hmac-sha1-etm@openssh.com             -- [fail] using broken SHA-1 hash algorithm
  55                                             `- [info] available since OpenSSH 6.2
  56 (mac) umac-64@openssh.com                   -- [warn] using encrypt-and-MAC mode
  57                                             `- [warn] using small 64-bit tag size
  58                                             `- [info] available since OpenSSH 4.7
  59 (mac) umac-128@openssh.com                  -- [warn] using encrypt-and-MAC mode
  60                                             `- [info] available since OpenSSH 6.2
  61 (mac) hmac-sha2-256                         -- [warn] using encrypt-and-MAC mode
  62                                             `- [info] available since OpenSSH 5.9, Dropbear SSH 2013.56
  63 (mac) hmac-sha2-512                         -- [warn] using encrypt-and-MAC mode
  64                                             `- [info] available since OpenSSH 5.9, Dropbear SSH 2013.56
  65 (mac) hmac-sha1                             -- [fail] using broken SHA-1 hash algorithm
  66                                             `- [warn] using encrypt-and-MAC mode
  67                                             `- [info] available since OpenSSH 2.1.0, Dropbear SSH 0.28
  68 
  69 # fingerprints
  70 (fin) ssh-ed25519: SHA256:1BKHx1zOUXI0NLZVRqwA9LczkNK32UjGwiewxnMC+40
  71 (fin) ssh-rsa: SHA256:bsePYli+5ohF+AQsOXZph6fJsRxERdX+Fu1pVQ0vYgs
  72 
  73 # algorithm recommendations (for OpenSSH 9.7)
  74 (rec) -ecdh-sha2-nistp256                   -- kex algorithm to remove 
  75 (rec) -ecdh-sha2-nistp384                   -- kex algorithm to remove 
  76 (rec) -ecdh-sha2-nistp521                   -- kex algorithm to remove 
  77 (rec) -ecdsa-sha2-nistp256                  -- key algorithm to remove 
  78 (rec) -hmac-sha1                            -- mac algorithm to remove 
  79 (rec) -hmac-sha1-etm@openssh.com            -- mac algorithm to remove 
  80 (rec) !rsa-sha2-256                         -- key algorithm to change (increase modulus size to 3072 bits or larger) 
  81 (rec) !rsa-sha2-512                         -- key algorithm to change (increase modulus size to 3072 bits or larger) 
  82 (rec) -diffie-hellman-group14-sha256        -- kex algorithm to remove 
  83 (rec) -hmac-sha2-256                        -- mac algorithm to remove 
  84 (rec) -hmac-sha2-512                        -- mac algorithm to remove 
  85 (rec) -umac-128@openssh.com                 -- mac algorithm to remove 
  86 (rec) -umac-64-etm@openssh.com              -- mac algorithm to remove 
  87 (rec) -umac-64@openssh.com                  -- mac algorithm to remove 
  88 
  89 # additional info
  90 (nfo) For hardening guides on common OSes, please see: <https://www.ssh-audit.com/hardening_guides.html>
  91 (nfo) Be aware that, while this target properly supports the strict key exchange method (via the kex-strict-?-v00@openssh.com marker) needed to protect against the Terrapin vulnerability (CVE-2023-48795), all peers must also support this feature as well, otherwise the vulnerability will still be present.  The following algorithms would allow an unpatched peer to create vulnerable SSH channels with this target: chacha20-poly1305@openssh.com.  If any CBC ciphers are in this list, you may remove them while leaving the *-etm@openssh.com MACs in place; these MACs are fine while paired with non-CBC cipher types.

Well, initially I just was curious, now, I just have got lots to do.

sshuttle

Sshuttle makes it possible to access remote networks using SSH. It creates a transparent proxy server, using iptables, that will forward all the traffic through an SSH tunnel to a remote copy of sshuttle.

Install sshuttle

   1 apt install sshuttle

Optionally create a sudoers.d file

   1 sshuttle --sudoers
   2 [sudo] Passwort für tobias: 
   3 Success, sudoers file update.

/etc/sudoers.d/sshuttle_auto

   1 Cmnd_Alias SSHUTTLEEA0 = /usr/bin/env PYTHONPATH=/usr/lib/python3/dist-packages /usr/bin/python3 /usr/bin/sshuttle *
   2 
   3 tobias ALL=NOPASSWD: SSHUTTLEEA0

Usage

Forward anything incl. DNS requests.

   1 sshuttle --dns -r username@sshserver 0/0

Forward RFC1918 networks, networks automatically detected (from remote routing table), DNS requests to the remote host and honor remote /etc/hosts.

   1 sshuttle --dns -NHr username@sshserver \
   2         10.0.0.0/8 172.16.0.0/12 192.168.0.0/16

Configuration files sshuttle1.conf

   1 --dns
   2 --auto-nets
   3 --auto-hosts
   4 -r
   5 username@sshserver
   6 10.0.0.0/8
   7 172.16.0.0/12
   8 192.168.0.0/16

Use the config with @.
The configuration file may be overidden on the cli.

   1 sshuttle @sshuttle1.conf

Daemonizing

There is a init-script, which may be used with the configuration files in /etc/sshuttle to startup sshuttle as a daemon (e.g. on boot or manually) at
/usr/share/doc/sshuttle/sshuttle.conf

SFTP

SFTP Server

SFTP server subsystems

SFTP is realized as a subsystem to openssh. By default no subsystems are configured.

/etc/ssh/sshd_config

   1 # override default of no subsystems
   2 Subsystem      sftp    /usr/lib/openssh/sftp-server
   3 #Subsystem       sftp    internal-sftp
   4

There are two sftp subsystems in openssh, which are functionally on par.

The external binary /usr/lib/openssh/sftp-server
- Standalone process
- Needs to be accessible in the ChrootDirectory
- Is said to consume less memory with increasing number of sessions
- First/Older implementation
The internal-sftp server code
- Compiled in sftp-server code
- Easier to be used with ChrootDirectory
- Second/Newer implementation

The group sftponly

Create group

   1 SFTP_GROUP="sftponly"
   2 addgroup --system "$SFTP_GROUP"

Create user and add to group

   1 PREFIX="/srv/sftp"
   2 SFTP_USER="sftp1"
   3 adduser --no-create-home \
   4         --home "$PREFIX/$SFTP_USER" \
   5         --shell /bin/nologin \
   6         "$SFTP_USER"
   7 adduser "$SFTP_USER" "$SFTP_GROUP"

To limit a group to only use SFTP
/etc/ssh/sshd_config

   1 Match group sftponly
   2      ChrootDirectory /srv/sftp/%u
   3      X11Forwarding no
   4      AllowTcpForwarding no
   5      AllowAgentForwarding no
   6      ForceCommand internal-sftp

ChrootDirectory

The documentation states
man -P "less -p 'ChrootDirectory'" 5 sshd_config

   1      ChrootDirectory
   2              Specifies the pathname of a directory to chroot(2) to after
   3              authentication.  At session startup sshd(8) checks that all
   4              components of the pathname are root-owned directories which
   5              are not writable by any other user or group.  After the ch‐
   6              root, sshd(8) changes the working directory to the user's home
   7              directory.  Arguments to ChrootDirectory accept the tokens de‐
   8              scribed in the TOKENS section.
   9 
  10              The ChrootDirectory must contain the necessary files and di‐
  11              rectories to support the user's session.  For an interactive
  12              session this requires at least a shell, typically sh(1), and
  13              basic /dev nodes such as null(4), zero(4), stdin(4),
  14              stdout(4), stderr(4), and tty(4) devices.  For file transfer
  15              sessions using SFTP no additional configuration of the envi‐
  16              ronment is necessary if the in-process sftp-server is used,
  17              though sessions which use logging may require /dev/log inside
  18              the chroot directory on some operating systems (see
  19              sftp-server(8) for details).
  20 
  21              For safety, it is very important that the directory hierarchy
  22              be prevented from modification by other processes on the sys‐
  23              tem (especially those outside the jail).  Misconfiguration can
  24              lead to unsafe environments which sshd(8) cannot detect.
  25 
  26              The default is none, indicating not to chroot(2).

To chroot directory may be created like this:

   1 PREFIX="/srv/sftp"
   2 SFTP_USER=sftp1
   3 install -o root -g sftp1 -m 0750 \
   4         -d "$PREFIX/$SFTP_USER"
   5 install -o sftp1 -g sftp1 -m 0750 \
   6         -d "$PREFIX/$SFTP_USER/writeable"

No containing directory (closer to the filesystem root) may be writable by the user that tries to login, because this would allow link substitution attacks. To simplify checking the tree, you may use
Check directory permissions

To resolution of uids and gids you need a filtered version of /etc/passwd and /etc/groups in the jail.

For this reason I previously used a package is no longer part of Debian. :-/
http://www.floc.net/makejail/

I need to find something similar. There are some candidates like:

firejail
jailkit

In combination with autofs a very flexible setup can be created!

ssh

About

Links

directory structure

Modularity using Include

shell aliases

Log in as root

Check SSH-HostKey

TOFU

Check SSH-HostKey using the DNS SSHFP RR

Check SSH-HostKey using a small ASCII graphic

HostKey changed

Disable Hostkey checking

pubkey auth

Links

The private key

Formating

Encoding formats

Transcode formats

Putty format

Reformat white-spaces in key file

Special operations on the keypair

Create public key from private key

Remove passphrase from private key

PubKey auth on Linux and Unix

Generate a key pair for pubkey auth

ssh-agent on *nix

PubKey auth on Win10

Generate a key pair for pubkey auth - Windows

ssh-agent on Windows

Test pubkey auth on localhost

Distribute your pubkeys

Certificate authentication

Generate a CA

Distribute and configure CAs

Show certificate information

Bugs in OpenSSH for Win32

ssh escape characters

keyboard-interactive authentication

sshpass

via file descriptor

environment variable

password file

Query features locally

Client Alive Checks

Old remote ssh-server

no matching cipher

no matching key exchange method

no matching host key type

no mutual signature algorithm

sftp-server: not found

Port Forwarding

X11 Forwarding

cannot open display

Hardening

ssh-audit

About ssh-audit

Install ssh-audit

Use ssh-audit

sshuttle

Install sshuttle

Usage

Daemonizing

SFTP

SFTP Server

SFTP server subsystems

The group sftponly

ChrootDirectory

SFTP Client