> ## Documentation Index
> Fetch the complete documentation index at: https://docs.americ.io.vn/llms.txt
> Use this file to discover all available pages before exploring further.

# ssh

> SSH personal notes.

#### SSH checklist

```shell theme={null}
# What port is SSH listening on?
sudo ss -tlnp | grep sshd

# Review critical sshd config settings
grep -E "^(Port|PermitRootLogin|PasswordAuthentication|MaxAuthTries|AllowUsers)" /etc/ssh/sshd_config

# Quick count of failed attempts
sudo grep "Failed password" /var/log/auth.log | wc -l

# Top offending IPs
sudo grep "Failed password" /var/log/auth.log | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn | head -20

# Recent 50 failures with timestamp
sudo grep "Failed password" /var/log/auth.log | tail -50

# Who is currently logged in
who

# More detail: IP, login time, idle
w

# All active SSH connections via socket
ss -tnp | grep :22

# SSH login history (recent)
last | grep "pts\|ssh" | head -20

# Check if installed
fail2ban-client status 2>/dev/null || echo "fail2ban NOT installed"

# Install it
sudo apt install fail2ban -y

# After install — check SSH jail status
sudo fail2ban-client status sshd

# List currently banned IPs
sudo fail2ban-client get sshd banip
```

⚡ One-liner Full Snapshot

```shell theme={null}
# refs: https://gist.github.com/OliverKain/e784e9754d47ccafc3be1d8061e6f1ba
echo "=== SSH PORT ===" && sudo ss -tlnp | grep sshd && \
echo "" && \
echo "=== FAILED LOGINS (top IPs) ===" && sudo grep "Failed password" /var/log/auth.log | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn | head -10 && \
echo "" && \
echo "=== ACTIVE SESSIONS ===" && w && \
echo "" && \
echo "=== FAIL2BAN ===" && sudo fail2ban-client status sshd 2>/dev/null || echo "fail2ban not running"
```

#### SSH via private keys

```shell theme={null}
# Generate key pair (anywhere/at client)
ssh-keygen -t ed25519 -C "comment (email/action...)"

# Validate by ssh to host using private key file
ssh -i $PRIVATE_KEY_PATH $USER@$HOST -p $PORT

# View public key (for copying)
cat ~/.ssh/id_ed25519.pub
# Windows
type $env:USERPROFILE\.ssh\id_ed25519.pub
# Initialize .ssh folder
mkdir -p ~/.ssh
# Copy
echo "PASTE YOUR PUBLIC KEY HERE" >> ~/.ssh/authorized_keys
# Set permission (just in case)
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

# One-liner - Add public key to the authorized_keys of host's user, create path if not exist (process at client)
cat $PUBLIC_KEY_PATH | ssh $USER@$HOST -p $PORT "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"
```

#### Change SSH port

* Prerequisites:

```shell theme={null}
# Check ssh status
which sshd
# Install sshd
sudo systemctl status ssh
# Install
sudo apt update && sudo apt install openssh-server -y
```

* Modified `sshd_config`

```bash theme={null}
# Find/Check location
find / -name "sshd_config" 2>/dev/null
# Edit
sudo vi /etc/ssh/sshd_config
```

* Search & uncomment `Port 22`, then change to the desired port.
* Restart `ssh` service

```shell theme={null}
# Ubuntu
# Disable ssh.socket
sudo systemctl disable --now ssh.socket
sudo systemctl enable --now ssh.service
#sudo service ssh restart
sudo systemctl restart ssh

# AlmaLinux
systemctl restart sshd
```

* Allow port via [ufw](/linux/awesome-tools/system/ufw) or [firewalld](/linux/awesome-tools/system/firewalld)
* Apply fail2ban

#### Quick setup script

<CodeGroup>
  ```shell ubuntu.sh expandable theme={null}
  #!/usr/bin/env bash
  set -e

  ### ===== BASIC OPTIONS (always run) =====
  CUSTOM_PORT=1993
  ALLOW_PASSWORD_LOGIN=true
  ALLOW_ROOT_LOGIN=true        # set to false to block root SSH login

  NEWUSER="americio"
  SSH_PUBLIC_KEY="ssh-ed25519 AAAA...your_key_here"
  PASSWORDLESS_SUDO=false

  BANTIME="1h"
  FINDTIME="10m"
  MAXRETRY=3

  ### ===== ADVANCED OPTIONS (only run if ADVANCED_SETUP=true) =====
  # Gates: 2FA (Google Authenticator TOTP), auditd hardening rules, unattended-upgrades
  ADVANCED_SETUP=false
  ### ========================================================

  # ─── helpers ──────────────────────────────────────────────────────────────────
  section() { echo; echo "===================================================================="; echo "  $*"; echo "===================================================================="; }
  info()    { echo "  ℹ️  $*"; }
  ok()      { echo "  ✅ $*"; }
  warn()    { echo "  ⚠️  $*"; }

  # Restart a service, validate it came up, exit on failure
  restart_service() {
      local svc="$1"
      echo "🔄 Restarting $svc..."
      if sudo systemctl restart "$svc"; then
          sleep 1
          if sudo systemctl is-active --quiet "$svc"; then
              ok "$svc is running."
          else
              warn "$svc failed to start — check: journalctl -u $svc -n 30"
              exit 1
          fi
      else
          warn "systemctl restart $svc failed."
          exit 1
      fi
  }

  # ─── 1. SSH HARDENING (always) ────────────────────────────────────────────────
  section "SSH HARDENING"

  FILE=/etc/ssh/sshd_config

  echo "💾 Backing up original sshd_config"
  sudo cp "$FILE" "$FILE.backup.$(date +%F-%H%M%S)"

  echo "✒️  Writing secure sshd_config..."
  sudo tee "$FILE" >/dev/null <<EOF
  # ===== SECURE SSH CONFIG =====

  Port ${CUSTOM_PORT}
  # Uncomment to disable IPv6: AddressFamily inet

  PermitRootLogin $( [ "$ALLOW_ROOT_LOGIN" = true ] && echo "yes" || echo "no" )
  PasswordAuthentication $( [ "$ALLOW_PASSWORD_LOGIN" = true ] && echo "yes" || echo "no" )
  PermitEmptyPasswords no
  KbdInteractiveAuthentication no
  PubkeyAuthentication yes
  AuthorizedKeysFile .ssh/authorized_keys

  MaxAuthTries 3
  LoginGraceTime 20
  MaxSessions 2

  ClientAliveInterval 120
  ClientAliveCountMax 2

  LogLevel VERBOSE

  Include /etc/ssh/sshd_config.d/*.conf
  EOF

  echo "🤔 Validating SSH configuration..."
  # Ubuntu 24.04 uses ssh.service; older distros use sshd.service
  SSH_SERVICE=$(systemctl list-units --type=service --all | grep -oE 'ssh(d)?\.service' | head -1)
  if sudo sshd -t; then
      restart_service "$SSH_SERVICE"
      ok "SSH hardened successfully (service: $SSH_SERVICE)."
  else
      warn "SSH config ERROR — restoring backup..."
      sudo cp "$FILE.backup"* "$FILE"
      exit 1
  fi

  # ─── 2. USER SETUP (always) ───────────────────────────────────────────────────
  section "USER SETUP"

  echo "🪄 Creating user: $NEWUSER"
  if id "$NEWUSER" >/dev/null 2>&1; then
      info "User already exists, skipping creation."
  else
      sudo adduser --disabled-password --gecos "" "$NEWUSER"
      GENPASS=$(openssl rand -base64 16)
      echo "$NEWUSER:$GENPASS" | sudo chpasswd
      echo "🔐 Temporary password for $NEWUSER: $GENPASS"
  fi

  sudo usermod -aG sudo "$NEWUSER"
  sudo usermod -aG adm  "$NEWUSER"
  sudo usermod -s /bin/bash "$NEWUSER"

  echo "🗃️  Setting up SSH directory"
  sudo mkdir -p /home/$NEWUSER/.ssh
  echo "$SSH_PUBLIC_KEY" | sudo tee /home/$NEWUSER/.ssh/authorized_keys >/dev/null
  sudo chmod 700  /home/$NEWUSER/.ssh
  sudo chmod 600  /home/$NEWUSER/.ssh/authorized_keys
  sudo chown -R "$NEWUSER:$NEWUSER" /home/$NEWUSER/.ssh

  if [ "$PASSWORDLESS_SUDO" = true ]; then
      echo "🔓 Enabling passwordless sudo for $NEWUSER"
      echo "$NEWUSER ALL=(ALL) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/90-"$NEWUSER" >/dev/null
      sudo chmod 440 /etc/sudoers.d/90-"$NEWUSER"
  fi

  ok "User setup completed."

  # ─── 3. UFW FIREWALL (always) ─────────────────────────────────────────────────
  section "UFW FIREWALL"

  sudo ufw --force reset
  sudo ufw default deny incoming
  sudo ufw default allow outgoing

  sudo ufw limit "${CUSTOM_PORT}"/tcp
  sudo ufw allow http
  sudo ufw allow https

  sudo ufw --force enable
  sudo ufw status verbose
  ok "UFW firewall configured."

  # ─── 4. FAIL2BAN (always) ─────────────────────────────────────────────────────
  section "FAIL2BAN"

  sudo apt update -q
  sudo apt install -y fail2ban

  sudo tee /etc/fail2ban/jail.local >/dev/null <<EOF
  [DEFAULT]
  bantime  = ${BANTIME}
  findtime = ${FINDTIME}
  maxretry = ${MAXRETRY}
  backend  = systemd
  banaction = ufw
  ignoreip = 127.0.0.1/8 ::1

  [sshd]
  enabled = true
  port    = ${CUSTOM_PORT}
  filter  = sshd
  backend = systemd
  logpath = journal
  EOF

  sudo systemctl enable --now fail2ban
  restart_service fail2ban
  sleep 2
  sudo fail2ban-client status sshd
  ok "Fail2ban installed & configured."

  # ─── 5. ADVANCED SETUP (opt-in) ───────────────────────────────────────────────
  if [ "$ADVANCED_SETUP" != true ]; then
      echo
      info "ADVANCED_SETUP=false — skipping 2FA, auditd, and unattended-upgrades."
      echo "  Set ADVANCED_SETUP=true at the top of this script to enable them."
      echo
      echo "💯 Basic VPS hardening complete. Goodbye!"
      exit 0
  fi

  section "ADVANCED SETUP (ADVANCED_SETUP=true)"

  # ── 5a. Lock root password ────────────────────────────────────────────────────
  echo "🔒 Locking root password"
  sudo passwd -l root
  ok "Root password locked."

  # ── 5b. Unattended Security Updates ──────────────────────────────────────────
  echo "📦 Unattended Security Updates..."
  sudo apt install -y unattended-upgrades
  sudo dpkg-reconfigure -plow unattended-upgrades
  ok "Unattended upgrades enabled."

  # ── 5b. SSH 2FA (Google Authenticator TOTP) ───────────────────────────────────
  section "SSH 2FA — Google Authenticator TOTP"

  echo "📦 Installing TOTP PAM module..."
  sudo apt install -y libpam-google-authenticator

  echo "🪪 Generating TOTP secret for user: $NEWUSER"
  sudo -u "$NEWUSER" bash -c "google-authenticator -t -d -f -r 3 -R 30 -W --quiet"

  echo "📁 Backing up SSH PAM config..."
  sudo cp /etc/pam.d/sshd /etc/pam.d/sshd.bak."$(date +%F-%H%M%S)"

  echo "✒️  Updating PAM rules for TOTP"
  sudo sed -i '1i auth required pam_google_authenticator.so' /etc/pam.d/sshd

  echo "📁 Backing up sshd_config..."
  sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak."$(date +%F-%H%M%S)"

  echo "⚙️  Configuring SSH to require publickey + TOTP"
  sudo sed -i 's/^#\?ChallengeResponseAuthentication.*/ChallengeResponseAuthentication yes/' /etc/ssh/sshd_config
  sudo sed -i 's/^#\?KbdInteractiveAuthentication.*/KbdInteractiveAuthentication yes/'       /etc/ssh/sshd_config

  if grep -q "^AuthenticationMethods" /etc/ssh/sshd_config; then
      sudo sed -i 's/^AuthenticationMethods.*/AuthenticationMethods publickey,keyboard-interactive/' /etc/ssh/sshd_config
  else
      echo "AuthenticationMethods publickey,keyboard-interactive" | sudo tee -a /etc/ssh/sshd_config >/dev/null
  fi

  echo "🤔 Validating SSH configuration with 2FA..."
  if sudo sshd -t; then
      restart_service "$SSH_SERVICE"
      ok "SSH 2FA enabled successfully."
  else
      warn "ERROR: Invalid SSH config — rolling back 2FA changes..."
      sudo cp /etc/ssh/sshd_config.bak.* /etc/ssh/sshd_config
      sudo cp /etc/pam.d/sshd.bak.*      /etc/pam.d/sshd
      restart_service "$SSH_SERVICE"
      exit 1
  fi

  echo
  echo "  👉 IMPORTANT: Open a NEW terminal and test login before closing this session."
  echo "  SSH now requires:"
  echo "    1) SSH private key"
  echo "    2) Google Authenticator 6-digit TOTP code"

  # ── 5c. auditd Hardening Rules ────────────────────────────────────────────────
  section "AUDITD HARDENING RULES"

  echo "📜 Installing hardened auditd rules..."
  sudo apt install -y auditd audispd-plugins

  sudo tee /etc/audit/rules.d/hardening.rules >/dev/null <<'EOF'
  ## ===============================
  ## HARDENED AUDITD SECURITY RULES
  ## ===============================

  # Track sudo log changes
  -w /var/log/sudo.log -p wa -k sudo_logs

  # Track privileged binary changes
  -w /usr/bin  -p wa -k privileged-bins
  -w /usr/sbin -p wa -k privileged-bins

  # Track essential system config changes
  -w /etc/passwd   -p wa -k passwd_changes
  -w /etc/group    -p wa -k group_changes
  -w /etc/shadow   -p wa -k shadow_changes
  -w /etc/sudoers  -p wa -k sudoers_changes
  -w /etc/sudoers.d/ -p wa -k sudoers_changes

  # Track SSH config changes
  -w /etc/ssh/sshd_config      -p wa -k ssh_config
  -w /etc/ssh/sshd_config.d/   -p wa -k ssh_config

  # Track login/auth events
  -w /var/log/faillog  -p wa -k auth_failures
  -w /var/log/lastlog  -p wa -k auth_changes
  -w /var/log/tallylog -p wa -k auth_failures

  # Track systemd service changes
  -w /etc/systemd/system/      -p wa -k systemd_changes
  -w /usr/lib/systemd/system/  -p wa -k systemd_changes

  # Track kernel module loading
  -w /sbin/modprobe     -p x -k kernel_modules
  -w /usr/sbin/modprobe -p x -k kernel_modules

  # Monitor root escalation
  -w /bin/su -p x -k root_escalation

  # File permission/ownership changes
  -a always,exit -F arch=b64 -S chmod,chown,chgrp,fchmod,fchown -k perms
  -a always,exit -F arch=b32 -S chmod,chown,chgrp,fchmod,fchown -k perms

  # Root shell spawning
  -a always,exit -F arch=b64 -S execve -F uid=0 -k root_shell
  -a always,exit -F arch=b32 -S execve -F uid=0 -k root_shell

  # Firewall rule changes
  -w /etc/ufw/       -p wa -k ufw_changes
  -w /etc/firewalld/ -p wa -k firewalld_changes

  # Cron persistence tracking
  -w /etc/crontab       -p wa -k cron_changes
  -w /etc/cron.d/       -p wa -k cron_changes
  -w /etc/cron.daily/   -p wa -k cron_changes
  -w /etc/cron.hourly/  -p wa -k cron_changes

  # Make audit config immutable (requires reboot to change rules after this)
  -e 2
  EOF

  sudo augenrules --load
  sudo systemctl enable --now auditd
  restart_service auditd
  ok "auditd hardening rules installed."

  # ─── DONE ─────────────────────────────────────────────────────────────────────
  echo
  echo "💯 Full VPS hardening complete (basic + advanced). Goodbye!"
  ```

  ```shell almalinux.sh theme={null}
  #!/usr/bin/env bash
  set -e

  ### ===== BASIC OPTIONS (always run) =====
  CUSTOM_PORT=1993
  ALLOW_PASSWORD_LOGIN=true
  ALLOW_ROOT_LOGIN=true        # set to false to block root SSH login

  NEWUSER="americio"
  SSH_PUBLIC_KEY="ssh-ed25519 AAAA...your_key_here"
  PASSWORDLESS_SUDO=false

  BANTIME="1h"
  FINDTIME="10m"
  MAXRETRY=3

  ### ===== ADVANCED OPTIONS (only run if ADVANCED_SETUP=true) =====
  # Gates: 2FA (Google Authenticator TOTP), auditd hardening rules, dnf-automatic
  ADVANCED_SETUP=false
  ### ========================================================

  # ─── helpers ──────────────────────────────────────────────────────────────────
  section() { echo; echo "===================================================================="; echo "  $*"; echo "===================================================================="; }
  info()    { echo "  ℹ️  $*"; }
  ok()      { echo "  ✅ $*"; }
  warn()    { echo "  ⚠️  $*"; }

  restart_service() {
      local svc="$1"
      echo "🔄 Restarting $svc..."
      if sudo systemctl restart "$svc"; then
          sleep 1
          if sudo systemctl is-active --quiet "$svc"; then
              ok "$svc is running."
          else
              warn "$svc failed to start — check: journalctl -u $svc -n 30"
              exit 1
          fi
      else
          warn "systemctl restart $svc failed."
          exit 1
      fi
  }

  # ─── 1. SSH HARDENING (always) ────────────────────────────────────────────────
  section "SSH HARDENING"

  FILE=/etc/ssh/sshd_config

  echo "💾 Backing up original sshd_config"
  sudo cp "$FILE" "$FILE.backup.$(date +%F-%H%M%S)"

  echo "✒️  Writing secure sshd_config..."
  sudo tee "$FILE" >/dev/null <<EOF
  # ===== SECURE SSH CONFIG =====

  Port ${CUSTOM_PORT}
  # Uncomment to disable IPv6: AddressFamily inet

  PermitRootLogin $( [ "$ALLOW_ROOT_LOGIN" = true ] && echo "yes" || echo "no" )
  PasswordAuthentication $( [ "$ALLOW_PASSWORD_LOGIN" = true ] && echo "yes" || echo "no" )
  PermitEmptyPasswords no
  KbdInteractiveAuthentication no
  PubkeyAuthentication yes
  AuthorizedKeysFile .ssh/authorized_keys

  MaxAuthTries 3
  LoginGraceTime 20
  MaxSessions 2

  ClientAliveInterval 120
  ClientAliveCountMax 2

  LogLevel VERBOSE

  Include /etc/ssh/sshd_config.d/*.conf
  EOF

  echo "🤔 Validating SSH configuration..."
  # AlmaLinux uses sshd.service
  SSH_SERVICE="sshd.service"
  if sudo sshd -t; then
      restart_service "$SSH_SERVICE"
      ok "SSH hardened successfully."
  else
      warn "SSH config ERROR — restoring backup..."
      sudo cp "$FILE.backup"* "$FILE"
      exit 1
  fi

  # ─── 2. USER SETUP (always) ───────────────────────────────────────────────────
  section "USER SETUP"

  echo "🪄 Creating user: $NEWUSER"
  if id "$NEWUSER" >/dev/null 2>&1; then
      info "User already exists, skipping creation."
  else
      # Mirrors Ubuntu `adduser --disabled-password --gecos ""`
      sudo useradd -m -s /bin/bash "$NEWUSER"
      sudo passwd -l "$NEWUSER"                    # lock password (no login via password)
      GENPASS=$(openssl rand -base64 16)
      echo "$NEWUSER:$GENPASS" | sudo chpasswd     # set a temp password (unlocks the account)
      echo "🔐 Temporary password for $NEWUSER: $GENPASS"
  fi

  # AlmaLinux: sudo group is `wheel`, adm group exists too
  sudo usermod -aG wheel "$NEWUSER"
  sudo usermod -aG adm   "$NEWUSER"
  sudo usermod -s /bin/bash "$NEWUSER"

  echo "🗃️  Setting up SSH directory"
  sudo mkdir -p /home/$NEWUSER/.ssh
  echo "$SSH_PUBLIC_KEY" | sudo tee /home/$NEWUSER/.ssh/authorized_keys >/dev/null
  sudo chmod 700  /home/$NEWUSER/.ssh
  sudo chmod 600  /home/$NEWUSER/.ssh/authorized_keys
  sudo chown -R "$NEWUSER:$NEWUSER" /home/$NEWUSER/.ssh

  if [ "$PASSWORDLESS_SUDO" = true ]; then
      echo "🔓 Enabling passwordless sudo for $NEWUSER"
      echo "$NEWUSER ALL=(ALL) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/90-"$NEWUSER" >/dev/null
      sudo chmod 440 /etc/sudoers.d/90-"$NEWUSER"
  fi

  ok "User setup completed."

  # ─── 3. FIREWALLD (always) ────────────────────────────────────────────────────
  section "FIREWALLD"

  # firewalld is installed by default on AlmaLinux; ensure it's running
  sudo systemctl enable --now firewalld

  echo "🔧 Registering custom SSH port with SELinux"
  sudo dnf install -y policycoreutils-python-utils
  sudo semanage port -a -t ssh_port_t -p tcp "$CUSTOM_PORT" 2>/dev/null || \
  sudo semanage port -m -t ssh_port_t -p tcp "$CUSTOM_PORT"

  echo "🛡️  Applying firewall rules..."
  sudo firewall-cmd --permanent --add-port="${CUSTOM_PORT}"/tcp
  sudo firewall-cmd --permanent --add-service=http
  sudo firewall-cmd --permanent --add-service=https

  # Remove default SSH port (22) if custom port differs
  if [ "$CUSTOM_PORT" != "22" ]; then
      sudo firewall-cmd --permanent --remove-service=ssh 2>/dev/null || true
  fi

  sudo firewall-cmd --reload
  sudo firewall-cmd --list-all
  ok "firewalld configured."

  # ─── 4. FAIL2BAN (always) ─────────────────────────────────────────────────────
  section "FAIL2BAN"

  # AlmaLinux: fail2ban lives in EPEL
  sudo dnf install -y epel-release
  sudo dnf install -y fail2ban

  sudo tee /etc/fail2ban/jail.local >/dev/null <<EOF
  [DEFAULT]
  bantime  = ${BANTIME}
  findtime = ${FINDTIME}
  maxretry = ${MAXRETRY}
  backend  = systemd
  banaction = firewallcmd-ipset
  ignoreip = 127.0.0.1/8 ::1

  [sshd]
  enabled = true
  port    = ${CUSTOM_PORT}
  filter  = sshd
  backend = systemd
  logpath = journal
  EOF

  sudo systemctl enable --now fail2ban
  restart_service fail2ban
  sleep 2
  sudo fail2ban-client status sshd
  ok "Fail2ban installed & configured."

  # ─── 5. ADVANCED SETUP (opt-in) ───────────────────────────────────────────────
  if [ "$ADVANCED_SETUP" != true ]; then
      echo
      info "ADVANCED_SETUP=false — skipping 2FA, auditd, and dnf-automatic."
      echo "  Set ADVANCED_SETUP=true at the top of this script to enable them."
      echo
      echo "💯 Basic VPS hardening complete. Goodbye!"
      exit 0
  fi

  section "ADVANCED SETUP (ADVANCED_SETUP=true)"

  # ── 5a. Lock root password ────────────────────────────────────────────────────
  echo "🔒 Locking root password"
  sudo passwd -l root
  ok "Root password locked."

  # ── 5b. Automatic Security Updates ───────────────────────────────────────────
  # AlmaLinux equivalent of unattended-upgrades: dnf-automatic
  echo "📦 Enabling automatic security updates (dnf-automatic)..."
  sudo dnf install -y dnf-automatic
  # Set to security-only updates
  sudo sed -i 's/^upgrade_type\s*=.*/upgrade_type = security/' /etc/dnf/automatic.conf
  sudo sed -i 's/^apply_updates\s*=.*/apply_updates = yes/'    /etc/dnf/automatic.conf
  sudo systemctl enable --now dnf-automatic.timer
  ok "dnf-automatic (security updates) enabled."

  # ── 5c. SSH 2FA (Google Authenticator TOTP) ───────────────────────────────────
  section "SSH 2FA — Google Authenticator TOTP"

  echo "📦 Installing TOTP PAM module..."
  # AlmaLinux: package is google-authenticator (not libpam-google-authenticator)
  sudo dnf install -y google-authenticator

  echo "🪪 Generating TOTP secret for user: $NEWUSER"
  sudo -u "$NEWUSER" bash -c "google-authenticator -t -d -f -r 3 -R 30 -W --quiet"

  echo "📁 Backing up SSH PAM config..."
  sudo cp /etc/pam.d/sshd /etc/pam.d/sshd.bak."$(date +%F-%H%M%S)"

  echo "✒️  Updating PAM rules for TOTP"
  sudo sed -i '1i auth required pam_google_authenticator.so' /etc/pam.d/sshd

  echo "📁 Backing up sshd_config..."
  sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak."$(date +%F-%H%M%S)"

  echo "⚙️  Configuring SSH to require publickey + TOTP"
  sudo sed -i 's/^#\?ChallengeResponseAuthentication.*/ChallengeResponseAuthentication yes/' /etc/ssh/sshd_config
  sudo sed -i 's/^#\?KbdInteractiveAuthentication.*/KbdInteractiveAuthentication yes/'       /etc/ssh/sshd_config

  if grep -q "^AuthenticationMethods" /etc/ssh/sshd_config; then
      sudo sed -i 's/^AuthenticationMethods.*/AuthenticationMethods publickey,keyboard-interactive/' /etc/ssh/sshd_config
  else
      echo "AuthenticationMethods publickey,keyboard-interactive" | sudo tee -a /etc/ssh/sshd_config >/dev/null
  fi

  echo "🤔 Validating SSH configuration with 2FA..."
  if sudo sshd -t; then
      restart_service "$SSH_SERVICE"
      ok "SSH 2FA enabled successfully."
  else
      warn "ERROR: Invalid SSH config — rolling back 2FA changes..."
      sudo cp /etc/ssh/sshd_config.bak.* /etc/ssh/sshd_config
      sudo cp /etc/pam.d/sshd.bak.*      /etc/pam.d/sshd
      restart_service "$SSH_SERVICE"
      exit 1
  fi

  echo
  echo "  👉 IMPORTANT: Open a NEW terminal and test login before closing this session."
  echo "  SSH now requires:"
  echo "    1) SSH private key"
  echo "    2) Google Authenticator 6-digit TOTP code"

  # ── 5d. auditd Hardening Rules ────────────────────────────────────────────────
  section "AUDITD HARDENING RULES"

  echo "📜 Installing hardened auditd rules..."
  # AlmaLinux: audit + audispd-plugins
  sudo dnf install -y audit audispd-plugins

  sudo tee /etc/audit/rules.d/hardening.rules >/dev/null <<'EOF'
  ## ===============================
  ## HARDENED AUDITD SECURITY RULES
  ## ===============================

  # Track sudo log changes
  -w /var/log/sudo.log -p wa -k sudo_logs

  # Track privileged binary changes
  -w /usr/bin  -p wa -k privileged-bins
  -w /usr/sbin -p wa -k privileged-bins

  # Track essential system config changes
  -w /etc/passwd    -p wa -k passwd_changes
  -w /etc/group     -p wa -k group_changes
  -w /etc/shadow    -p wa -k shadow_changes
  -w /etc/sudoers   -p wa -k sudoers_changes
  -w /etc/sudoers.d/ -p wa -k sudoers_changes

  # Track SSH config changes
  -w /etc/ssh/sshd_config      -p wa -k ssh_config
  -w /etc/ssh/sshd_config.d/   -p wa -k ssh_config

  # Track login/auth events
  -w /var/log/faillog  -p wa -k auth_failures
  -w /var/log/lastlog  -p wa -k auth_changes
  -w /var/log/tallylog -p wa -k auth_failures

  # Track systemd service changes
  -w /etc/systemd/system/      -p wa -k systemd_changes
  -w /usr/lib/systemd/system/  -p wa -k systemd_changes

  # Track kernel module loading
  -w /sbin/modprobe     -p x -k kernel_modules
  -w /usr/sbin/modprobe -p x -k kernel_modules

  # Monitor root escalation
  -w /bin/su -p x -k root_escalation

  # File permission/ownership changes
  -a always,exit -F arch=b64 -S chmod,chown,chgrp,fchmod,fchown -k perms
  -a always,exit -F arch=b32 -S chmod,chown,chgrp,fchmod,fchown -k perms

  # Root shell spawning
  -a always,exit -F arch=b64 -S execve -F uid=0 -k root_shell
  -a always,exit -F arch=b32 -S execve -F uid=0 -k root_shell

  # Firewall rule changes
  -w /etc/firewalld/ -p wa -k firewalld_changes

  # Cron persistence tracking
  -w /etc/crontab       -p wa -k cron_changes
  -w /etc/cron.d/       -p wa -k cron_changes
  -w /etc/cron.daily/   -p wa -k cron_changes
  -w /etc/cron.hourly/  -p wa -k cron_changes

  # Make audit config immutable (requires reboot to change rules after this)
  -e 2
  EOF

  sudo augenrules --load
  sudo systemctl enable --now auditd
  restart_service auditd
  ok "auditd hardening rules installed."

  # ─── DONE ─────────────────────────────────────────────────────────────────────
  echo
  echo "💯 Full VPS hardening complete (basic + advanced). Goodbye!"
  echo "Login using: ssh $NEWUSER@server -p $CUSTOM_PORT"
  echo "Keep your existing SSH session open while testing."
  ```
</CodeGroup>

* Run:

```shell theme={null}
curl -O https://gist.githubusercontent.com/username/abcdef1234567890/raw/harden.sh
chmod +x ./harden.sh
sudo ./harden.sh
# One-liner, no download
curl -sSL https://gist.githubusercontent.com/username/abcdef1234567890/raw/harden.sh | sudo bash
```

#### 🛡️ SSH Protection Layers — Basic to Advanced

##### Layer 1 — Immediate Basics (Do These First) - `sshd_config`

| No. | Action                    | Command/File                  |
| --- | ------------------------- | ----------------------------- |
| 1   | Change default SSH port   | `Port XXXX`                   |
| 2   | Disable root login        | `PermitRootLogin no`          |
| 3   | Limit max auth tries      | `MaxAuthTries 3`              |
| 4   | Reduce login grace        | `LoginGraceTime 30`           |
| 5   | Restrict to specific user | `AllowUsers <<yourusername>>` |

##### Layer 2 — Key-Based Auth (Eliminates Brute Force)

| No. | Action                   | Command/File                |
| --- | ------------------------ | --------------------------- |
| 6   | Generate ed25519 keypair | `ssh-keygen -t ed25519`     |
| 7   | Copy key to server       | ssh-copy-id                 |
| 8   | Disable password auth    | `PasswordAuthentication no` |
| 9   | Disable empty passwords  | `PermitEmptyPasswords no`   |

##### Layer 3 — Firewall (UFW)

| No. | Action                    | Command/File                                       |
| --- | ------------------------- | -------------------------------------------------- |
| 10  | Default deny all incoming | `ufw default deny incoming`                        |
| 11  | Allow only your SSH port  | `ufw allow [YOUR_PORT]/tcp`                        |
| 12  | Whitelist your IP for SSH | `ufw allow from [YOUR_IP] to any port [YOUR_PORT]` |
| 13  | Enable UFW                | `ufw enable`                                       |

##### Layer 4 — Auto-Blocking (fail2ban)

| No. | Action                           | Command/File                 |
| --- | -------------------------------- | ---------------------------- |
| 14  | Install fail2ban                 | `sudo apt install fail2ban`  |
| 15  | Configure \[sshd]                | `jailmaxretry=5, bantime=1h` |
| 16  | Recidive jail (repeat offenders) | `bantime=-1 permanent ban`   |
| 17  | Email alerts on bans             | `action = %(action_mwl)`     |

##### Layer 5 — Log Monitoring (Visibility)

| No. | Action                   | Command/File                                                                                                         |   |   |   |
| --- | ------------------------ | -------------------------------------------------------------------------------------------------------------------- | - | - | - |
| 18  | Check failed logins      | `grep "Failed password" /var/log/auth.log`                                                                           |   |   |   |
| 19  | Find top attacker IPs    | `sudo grep "Failed password" /var/log/auth.log \| awk '{print \$(NF-3)}' \| sort \| uniq -c \| sort -rn \| head -20` |   |   |   |
| 20  | Verify successful logins | grep "Accepted" /var/log/auth.log                                                                                    |   |   |   |
| 21  | Watch live auth log      | `tail -f /var/log/auth.log`                                                                                          |   |   |   |

##### Layer 6 — Advanced Hardening

| No. | Action                        | Command/File                              |
| --- | ----------------------------- | ----------------------------------------- |
| 22  | Fix Docker bypassing UFW      | `daemon.json → "iptables": false`         |
| 23  | Bind services to localhost    | `docker-compose.yml → 127.0.0.1:PORT`     |
| 24  | Block Postgres publicly       | ufw deny 5432                             |
| 25  | Nginx attack detection custom | `fail2ban filter jail`                    |
| 26  | Block bot scanner             | `probesbotsearch-common`, `fail2ban jail` |

##### Layer 7 — Expert Level

| No. | Action                     | Command/File                |
| --- | -------------------------- | --------------------------- |
| 27  | Two-factor auth for SSH    | libpam-google-authenticator |
| 28  | Port knocking              | knockd                      |
| 29  | Geo-blocking by country    | `geoip-shell` or `nftables` |
| 30  | Intrusion detection system | OSSEC or Wazuh              |
| 31  | Centralized log monitoring | Grafana + Loki + Promtail   |