If you have a fresh Linux server, two changes will do more for its security than almost anything else. Swap password logins for SSH keys, and put a firewall in front of it that only opens the ports you actually use. This guide walks through both, step by step, and shows you how to do it without accidentally locking yourself out, which is the thing everyone is secretly worried about.
Why password logins are the weak spot
The moment your server is online, bots start trying to log in. They guess common usernames like root, admin and ubuntu, and they throw millions of passwords at the SSH port. Most of it is automated and constant. You will see it the first time you read your auth logs and it can be a bit alarming.
A password is one secret. If it leaks, gets guessed, or you reused it somewhere that got breached, that is the whole door. An SSH key is a different and much stronger setup. So let's start there.
How SSH keys actually work
An SSH key comes in two halves that are made together as a pair. There is a private key, which stays on your own computer and never leaves it, and a public key, which you copy onto any server you want to log into.
The public key is fine to share. It cannot be used to log in on its own. When you connect, the server uses the public key to send your machine a little challenge that only the matching private key can answer. Your computer answers it quietly in the background, the server is satisfied, and you are in. No password travels across the network at any point.
Think of the public key as a lock you bolt onto the server's door, and the private key as the only key that opens it. You can hand out copies of the lock all day. As long as you are the only one holding the key, you are the only one getting in.
A quick warning: protect that private key. If someone copies it off your laptop, they get the same access you have. This is why you put a passphrase on it, which we will do in a second.
Generating a key with ssh-keygen
On your own machine, not the server, open a terminal and run this. Mac and Linux have it built in, and Windows 10 and 11 ship with it too.
ssh-keygen -t ed25519 -C "[email protected]"The -t ed25519 part picks a modern key type that is small and strong. If you are on something old that does not support it, use ssh-keygen -t rsa -b 4096 instead. The -C bit is just a comment so you can tell your keys apart later.
It will ask where to save the key. Press Enter to accept the default, which puts it in your .ssh folder. Then it asks for a passphrase. Set one. I know it is one more thing to type, but if your laptop ever gets stolen, that passphrase is the difference between a minor headache and a stranger logging into all your servers. You can use ssh-agent so you only type it once per session.
When it finishes you will have two files. The private key has no extension, and the public key ends in .pub. The one ending in .pub is the one that goes on the server. Never share or upload the other one.
Getting the public key onto the server
The easy way is ssh-copy-id. From your machine, while you can still log in with a password, run this with your own user and server address.
ssh-copy-id [email protected]It logs in, creates the right folder, and adds your public key to a file called authorized_keys with the correct permissions. That is the whole job.
If ssh-copy-id is not available, you can do it by hand. Print your public key, copy the whole line, then on the server paste it into ~/.ssh/authorized_keys. The manual version looks like this.
mkdir -p ~/.ssh
chmod 700 ~/.ssh
echo "paste-your-public-key-line-here" >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keysThose permission numbers matter more than they look. SSH is fussy on purpose. If the .ssh folder or the authorized_keys file is readable by other users, SSH will quietly refuse to use the key and fall back to asking for a password. So if your key is not working, check the permissions first.
Test the key before you change anything else
Do not skip this. Open a new terminal and try to log in.
ssh [email protected]If it lets you in without asking for your account password, and only asks for your key passphrase, the key works. Good. Keep that terminal open and logged in. We are about to turn off password logins, and having a working session already open is your safety net.
Turning off password logins safely
Once your key works, you can tell SSH to stop accepting passwords at all. The bots can keep guessing forever and it will not matter, because there is no password to guess anymore.
On the server, open the SSH config file with a text editor. You will need sudo.
sudo nano /etc/ssh/sshd_configFind these lines, uncomment them if they have a # in front, and set them like this.
PasswordAuthentication no
PubkeyAuthentication yes
PermitRootLogin prohibit-passwordThat last one means root can still be reached with a key if you really need it, but never with a password. Honestly, on most setups you should be logging in as a normal user and using sudo, not as root at all, but that is a topic for another day.
Save the file, then reload SSH so it picks up the change.
sudo systemctl restart sshNow here is the careful part. Do not close that first terminal. Open a brand new one and try to connect again. If the new connection works with your key, you are golden. If something is wrong, your old session is still open so you can fix the config and try again. Only once the fresh login works should you relax. This habit of testing in a second terminal has saved us more times than I can count.
Adding a firewall with ufw
Keys handle who can log in. A firewall handles what can be reached at all. The cleanest approach is simple. Block everything by default, then open only the few ports you genuinely use.
On Ubuntu and Debian the friendly tool for this is ufw, which stands for uncomplicated firewall. It is a nicer front end over the messy rules underneath.
Before you do anything, allow SSH. This is the one rule you absolutely cannot forget, because if you enable the firewall without it, you will lock yourself out of your own server. If your SSH runs on the standard port 22, this works.
sudo ufw allow OpenSSHIf you moved SSH to a custom port, allow that number instead, for example sudo ufw allow 2222/tcp. Then set the default behaviour and turn it on.
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw enableIt will warn you that enabling may disrupt existing connections. Since you allowed SSH first, your session stays up. Check what is allowed with sudo ufw status and you will see a short list of open ports.
Only open ports you actually use
This is where people go wrong. They open a pile of ports just in case, and every open port is one more thing an attacker can poke at. Keep the list short and honest. Open a port only when something real is listening on it.
Here are the common ones, just so you know what you are looking at.
| Port | What it is for |
|---|---|
| 22 | SSH, your login |
| 80 | Web traffic over HTTP |
| 443 | Web traffic over HTTPS |
| 25565 | A Minecraft server |
So if you are running a website, you would add the web ports like this.
sudo ufw allow 80/tcp
sudo ufw allow 443/tcpIf you run a Discord bot with discord.py or discord.js, you usually do not need to open anything at all, because the bot connects out to Discord rather than waiting for connections in. Outgoing traffic is already allowed, so it just works. When you stop using a service, remove its rule with sudo ufw delete allow 80/tcp so the door does not sit open for no reason.
One thing worth saying plainly. A firewall on the server is not the same as DDoS protection. A firewall decides which ports answer. It does not stop someone from flooding your line with junk traffic. That is a separate layer, and on our own game and VPS plans the network level protection sits in front of the box for exactly that reason. Your ufw rules and that protection are doing different jobs, and you want both.
Putting it together
So the whole flow, start to finish, is short. Make a key pair on your own machine. Copy the public half to the server. Test that the key logs you in. Turn off password logins. Add ufw, allow SSH first, deny the rest, and open only the ports you really need. Test once more from a fresh terminal.
None of these steps are hard on their own. The thing that trips people up is rushing the order and locking themselves out, which is why testing in a second terminal keeps coming up. Take it slow the first time. Do it once carefully and it becomes muscle memory, and every server you set up after this will be tighter from the very first hour.
