Securing SSH access takes more than replacing a password with a key. A sound setup controls who can connect, which credentials they can use, where connections can originate, and what each account may do after login. It also leaves you a recovery route when a configuration change goes wrong.
For a small Linux server, the practical starting point is a named administrator account, a protected SSH key, restricted network access, and no direct root login. Apply those controls in stages. If you are still preparing the machine, complete the account and recovery steps in setting up your first Linux server before tightening authentication.
Prepare a recovery route before changing SSH
The first security control is surprisingly unglamorous: make sure you can recover the server. Confirm that your hosting provider’s console, serial console, or physical keyboard access works. Opening a control panel is not the same as proving that its recovery console lets you sign in.
Keep your current SSH session open throughout the change. Use a second terminal to test new connections. An existing session proves that you authenticated earlier; it does not prove that the new configuration accepts fresh logins.
Before editing anything, save a copy of the main SSH server configuration and any included configuration files. Record the current listening port, administrator username, and firewall rules. On a managed server, also check whether provisioning software controls these settings. Otherwise, the next deployment may helpfully undo your work.
Use this order:
- Confirm console access and a working administrator account.
- Install and test the administrator’s public key.
- Change authentication and account restrictions.
- Validate the configuration and apply it through the correct service.
- Test a fresh connection before closing the original session.
Change firewall restrictions separately from authentication where possible. If you alter the allowed address, listening port, and authentication methods together, a failed connection gives you three suspects instead of one.
Create and protect your SSH keys
An SSH key pair has two parts. The public key belongs on the server. The private key stays on the client device that initiates the connection. Public key authentication proves possession of the private key without sending that private key to the server.
Generate the pair on your own computer using OpenSSH’s ssh-keygen utility. Ed25519 is a practical choice for compatible systems; select it with the utility’s -t ed25519 option. Choose a recognisable filename and enter a long passphrase when prompted. Do not overwrite an existing private key unless you have checked where it is already used.
A passphrase encrypts the private key file. OpenSSH also supports hardware authenticator backed keys through the ed25519-sk and ecdsa-sk types, which generally require interaction with the authenticator. These options and their requirements are documented in the OpenSSH key generation manual.
If your organisation requires a particular cryptographic policy, choose a permitted key type rather than bypassing that policy to follow a generic tutorial.
Give each administrator their own key, and preferably use a separate pair for each client device. Sharing one private key across a team makes departures, lost laptops, and access reviews harder to handle. Separate keys let you remove one device’s access without replacing everyone’s credentials.
An SSH agent can reduce repeated passphrase entry, but an unlocked credential still needs protection. Lock your workstation when unattended, protect its storage, and avoid leaving administrative credentials available in unnecessary sessions. Do not enable agent forwarding as a routine convenience.
For recovery, keep either a securely stored backup of your private key or a separately protected recovery key. A hardware authenticator also needs a loss plan, such as a second enrolled device. The spare should not live in the same laptop bag.
Install the public key and prove it works
Add the public key to the intended server account’s ~/.ssh/authorized_keys file. The tilde refers to that account’s home directory. Installing a key for root does not grant access to a named administrator account, or vice versa.
Use ssh-copy-id where available, or append the public key through an existing trusted session or provider console. Preserve existing authorised keys unless you intend to revoke them. Each public key entry should remain on one line; a wrapped key pasted from an email can fail without looking obviously broken.
Check ownership and permissions before troubleshooting cryptography. A conservative arrangement is mode 700 for the account’s .ssh directory and 600 for authorized_keys, with both owned by the account. Private key files on the client should also be protected from other users. These settings match the Oracle Linux guidance on OpenSSH file permissions.
The home directory and its parent path must not allow untrusted users to replace the SSH files. Do not make the directory broadly writable to silence an access error. If the permission numbers are unfamiliar, review Linux file permissions and ownership before changing them recursively.
Now open a separate connection using the intended username and private key. For this test, configure the client to offer that identity and permit only public key authentication. A successful login followed by a server password prompt is not proof that the key worked.
A private key passphrase prompt is different: it unlocks the local key, rather than authenticating with the server account’s password. Check which credential the prompt requests.
Also verify the server’s host key fingerprint through a trusted route, such as its console or an administrator supplied record. Host keys identify the server; user keys identify the person connecting. Do not dismiss an unexpected host key change by deleting the saved entry and hoping for the best.
Set a clear authentication policy
OpenSSH server configuration normally starts in /etc/ssh/sshd_config. Many distributions also load snippets from /etc/ssh/sshd_config.d/, but the directory only matters if an active Include directive reads it.
File order matters. For most OpenSSH settings, the first obtained value wins, rather than the last. Ubuntu places its snippet include near the beginning of the main file, so an included setting can take precedence over one farther down. Check the actual layout rather than assuming a file named 99-hardening.conf wins; the Ubuntu OpenSSH configuration documentation covers this behaviour.
After testing key access, the following settings form a reasonable policy for an administrator account that should authenticate only with a public key. They are configuration entries, not terminal commands.
| Setting | Purpose |
|---|---|
PubkeyAuthentication yes |
Allows public key authentication. |
AuthenticationMethods publickey |
Requires public key authentication for login. |
PasswordAuthentication no |
Disables the SSH password authentication method. |
KbdInteractiveAuthentication no |
Disables keyboard interactive authentication, which can otherwise provide another password based route. |
PermitRootLogin no |
Blocks direct SSH login as root. |
X11Forwarding no |
Disables SSH’s X11 forwarding feature when it is not required. |
Disabling password authentication alone does not define a complete key only policy. Review other enabled methods and any conditional exceptions. The distinction between authentication methods and root login controls is set out in the OpenSSH server configuration manual.
Leave distribution supplied PAM integration intact unless you have a reason to change it. PAM can still perform account and session checks after successful key authentication. Switching it off is not a necessary shortcut to disabling SSH passwords.
These SSH settings do not remove the account’s local password or decide how sudo authenticates. Keep those decisions separate. Before blocking root login, prove that the named administrator can perform the administrative tasks needed to maintain and recover the server.
Restrict which accounts may log in
A valid key should not be the only access rule. Decide which accounts need SSH at all. A web application account, database service account, and human administrator have different jobs; they should not inherit identical remote access simply because they exist on the same machine.
For a small server, AllowUsers can name the permitted accounts. For a team, AllowGroups can make membership in a dedicated SSH access group the requirement. Use a recognisable group name and document who approves membership.
Create the group and add the intended administrators before activating the restriction. Confirm membership through the server’s account tools, then test a fresh login. A group policy with no eligible administrator is a very effective access control, just not a useful one.
Allow and deny rules interact. When several account restrictions are present, verify that the intended user satisfies all applicable checks. Keep the policy simple enough that another administrator can review it without reconstructing months of exceptions.
SSH admission and administrative authority are separate. Membership in an SSH access group should not automatically mean unrestricted sudo access. Define those permissions independently through Linux users, groups, and sudo controls.
Use named accounts rather than a shared administrator login. Individual accounts make access removal clearer and give logs a more useful identity. They do not replace auditing, but they avoid starting every investigation with “which person was admin?”
Use multifactor authentication deliberately
A passphrase protected software key is useful, but its passphrase is checked locally. The server does not independently verify that passphrase, so do not describe the arrangement as server enforced multifactor authentication.
Where a separate server side factor is required, OpenSSH can require a public key followed by keyboard interactive authentication. The setting AuthenticationMethods publickey,keyboard-interactive:pam expresses that sequence on a compatible PAM enabled build. The comma requires both methods; space separated method lists represent alternatives. The Debian OpenSSH authentication configuration manual documents these rules and PAM integration.
This is an alternative to the key only policy above, not an extra line to paste beneath it. Keyboard interactive authentication must be enabled, and PAM must actually enforce the intended second factor. A prompt is not evidence of a second factor if it simply asks for another ordinary password.
Plan enrolment, lost device recovery, and emergency access before enforcement. Test both success and failure: a valid key without the required second factor must not be enough. Keep automation accounts out of human authentication workflows unless the automation has been designed for them.
Reduce who can reach the SSH listener
Authentication protects entry to an account. Network restrictions reduce who can attempt that entry. Where practical, expose SSH only through a management VPN, a controlled bastion host, or known administrator address ranges.
Apply the policy at every relevant layer. A cloud security group and the server’s own firewall are separate controls. Check IPv4 and IPv6 exposure; restricting one while leaving the other open does not produce the intended result.
Address allowlists work best when administrator addresses are stable. A residential connection or mobile hotspot may change address, leaving a legitimate user outside yesterday’s rule. A management VPN can offer a more predictable access route, provided its own authentication and recovery arrangements are maintained.
Add the new permitted route, test it, and only then remove the broader rule. The mechanics depend on whether the server uses UFW, firewalld, or nftables; keep those changes within one managed policy using the Linux firewall configuration guide.
Changing SSH from TCP port 22 may reduce routine scan noise, but it does not replace authentication or network controls. It also adds operational details to remember in clients, monitoring, firewall rules, and recovery notes. Choose it for an operational reason, not as the centrepiece of server security.
Give automation less access than a person
Backups, deployments, and monitoring should use dedicated accounts and credentials. Do not copy your personal administrator key onto a build runner simply because a deployment needs to transfer files.
For an automated task, define the permitted operation before granting access. A backup key might need to invoke one approved backup entry point, not open an unrestricted shell. A deployment account might need write access to one application directory, not the authority to alter SSH configuration.
OpenSSH’s authorised key options can help express those restrictions. The restrict option disables forwarding features, terminal allocation, and the user SSH startup file, but it does not by itself forbid arbitrary commands. Pair it with a carefully designed forced command when the key must perform only one operation.
A from= restriction can also tie a key to an expected network origin. Use that as another condition, not a replacement for protecting the credential. These controls are detailed in the OpenSSH daemon manual and authorised key options.
Forced commands need care. A wrapper that accepts unrestricted shell input can undo the restriction it was meant to provide. Prefer a narrow interface with fixed permitted actions and avoid giving the account broad sudo permissions.
Disable forwarding features that an account does not need, but do not mistake those switches for a sandbox. An account with unrestricted shell access can run other networking tools. Strong restrictions require an account whose filesystem permissions and allowed operations match its purpose.
Validate the effective settings before applying them
Check syntax with the SSH daemon’s -t test mode before reloading or restarting anything. Run the check with sufficient privileges to read the server configuration and host keys. Resolve every reported error first.
Then inspect the effective configuration with its -T mode. This matters when package defaults, included snippets, or provisioning files set values elsewhere. If you use Match blocks, supply representative connection details through the daemon’s -C option so the test evaluates the intended user and address conditions.
Validation is necessary, but not sufficient. A configuration can be syntactically valid while excluding every administrator. Check the resulting authentication methods, root policy, account restrictions, and any conditional exceptions against your intended design.
Apply the change through the service unit supplied by the distribution. Unit names and reload support differ, so inspect the installed unit rather than copying an instruction for another system. The distinction between application reloads, restarts, and unit changes is covered in managing Linux services with systemd.
Also check whether a socket unit owns the listening address. Ubuntu supports socket activated SSH, which can change how port and address edits take effect. Listener changes need the appropriate socket and service procedure, not a blind service restart; consult the Ubuntu SSH socket activation documentation for that distinction.
After applying the change, start a genuinely new connection rather than reusing an existing multiplexed SSH connection. Confirm that your key works, sudo behaves as intended, and prohibited login methods fail. Only then close the original session.
Troubleshoot without weakening the policy
When access fails, identify the stage of failure before changing settings. A timeout points to a different problem from a rejected key or an account denied by policy.
For a timeout, check routing, firewall rules, the destination address, and the listening port. For a refused connection, check whether the expected listener is active. For authentication failures, check the username, offered identity, authorised key location, ownership, permissions, and account admission rules.
Use the client’s verbose diagnostic mode to see which identity it offers. Too many loaded agent identities can complicate testing, so select the intended key explicitly. Never solve that confusion by uploading the private key to the server.
Review server logs alongside the failed attempt. The relevant unit name depends on the distribution. Follow troubleshooting services with journalctl to narrow the time window and correlate the client attempt with the server’s reason for rejection.
If ordinary permissions look correct, investigate mandatory access controls such as SELinux rather than disabling them. After a lockout, use the recovery console to correct the last change, validate again, and retest. Broadly enabling password and root login may restore access, but it also removes the policy you were trying to establish.
Maintain access controls after setup
SSH security needs maintenance whenever people, devices, or server roles change. Keep an inventory of authorised keys with an owner, purpose, and identifiable fingerprint. Comments help humans recognise entries, but they are not proof of ownership.
When a laptop is lost or an administrator leaves, remove the relevant access everywhere it was granted. Removing a public key prevents future authentication through that entry; it does not terminate an already authenticated session. Investigate active sessions separately when compromise is suspected.
If a private key may have been stolen, replace it rather than changing only its passphrase. The old credential remains a concern wherever its public key is still trusted.
Keep the SSH packages and operating system supported and patched through a documented Linux security update process. Treat configuration backups and emergency credentials as sensitive material, with recovery procedures included in offsite backup and recovery testing.
A well secured SSH setup has a clear answer to four questions: who may connect, what credential they need, what they may do, and how access can be recovered or revoked. Keep those answers simple, testable, and current.