How to protect Linux from Hackers // My server security strategy!

Hey everyone christian here and today we are protecting your linux server against hackers malware and other bad things that could potentially infect your system and we are talking about some security strategies that you can use for most common attacks out there like how to secure authentication on your server

But also how to find out what’s currently running on it and how you could optimize your deployments to lower down the attack surface especially when you’re running some administrative web interfaces and darker containers which most of you guys probably will do so i hope that will be interesting for you

And just to let you know i have one more security video and two more devops videos coming out the next weeks so if you don’t want to miss that make sure you’re hitting that subscribe button and of course like the videos okay so let’s jump right into it and first i want to

Address or emphasize how important it is to come up with your own strategy on all your linux servers because when you search on the web about how to protect a linux server or how to harden a linux server whatever you want to call it you will already find many resources with

Best practices that people are just throwing out and you might just want to grab some of these commands you can easily run on your server and then assume you’re done your server is now secured against all attacks and you don’t need to worry about it anymore that’s probably the worst thing you can

Do because security is much more complicated than a simple checklist there are so many aspects about server security that you would need to consider which are of course also always changing and even when you went through all this stuff there might be certain areas or some edge cases you haven’t really

Thought about and of course that’s very challenging but it’s still a great idea to have a solid setup that’s protected against most common and most critical threats on the web and my biggest wish is that you carefully think about it so instead of giving you a simple checklist

To copy paste i want to explain how i’m thinking about my security strategy and some things might apply to your scenarios some might not so i want you to develop your own strategy by giving you some useful tips and ideas and some practical examples and we’ll start with the most basic and

Simple thing you should always do which is also by the way the most effective one and that is always update your software always update the operating system in all the applications that you are running on your server because whenever there is a new security vulnerability disclosure often there’s

Already a patch for it available so you might just want to upgrade the software as soon as possible especially when something new is made public you can assume that it’s getting much bigger attention and people are always trying to take advantage of it only luxury software is updated by the package

Manager that your distribution is using so on my servers i mostly use ubuntu and there i use the unintended upgrade package and configure it to install any security updates automatically so on ubuntu the unintended upgrades package should be already installed but if it is not you can simply do that by executing

The sudo app install unattended upgrades and hit enter so you can see that is already installed on my server and now i want to configure it to just enable the automatic updates for all the security patches in the system there’s just one command you need to execute and this is sudo dpkg reconfigure

Priority low unattended upgrades and hit enter so now you simply just need to confirm this and this should write two different configuration files in the etc dot conf dot d so let’s cd into this folder and let’s take a look at these configuration files so the first one is

The 20 output upgrades configuration file and they are just two lines in here that will automatically execute the unintended upgrade script and what if this script should exactly do is configured in this file here so let’s take a look at the 50 unintended upgrades configuration files so i won’t change this default

Configuration but if you really want you could technically do this you could also configure the server to automatically reboot or send you an email notification about the unintended upgrade packages and if you want to test if your configuration settings have been applied correctly you simply can execute sudo unattended upgrade dry run

And then use the debug mode and this will present you an output what would happen if the server would run the daily script and you can see i currently have everything updated so there are no packages found that can be upgraded automatically and there are no pending

Auto removal so everything is fine and you should also see in the log file that this script now should run daily on your server okay so now all linux server will automatically update all packages whenever there is a new security patch out there everything is fine right

Well on my system not because i also run many applications in darker containers and the problem is that the package manager on the host operating system don’t update any of them and that’s by the way not the only issue we have with docker containers and security updates because whenever you’re using

Containerization as every new technology it has some very specific challenges you need to think about so there are two major problems with docker the first one is that all the docker images are not always updated as they should be because the maintainer of the docker image hasn’t updated their packages and

Released a new image yet so i would always be very careful when i deploy a darker container on my server who is creating that image and is it still an actively maintained project and that’s why i mostly just use docker images that are created by the owner of the application or

Use some good resources that i generally trust a little bit more like the linux server.io for example they have a great collection of many customized docker images to deploy home server applications or services and they are updated frequently and well maintained and whenever there is a new docker image

Available that includes a security patch you also need to redeploy your container because docker containers are immutable they are not persistently storing any updates so the correct way to update them is to destroy the container and redeploy it with a new version so i use a tool that is called watchtower to

Automatically update all my running docker containers at midnight where a quick restart doesn’t hurt me you might apply your own strategy for updating your containers but if you are interested in watchtower i also made a separate video about it and how to configure so i will put your link in the

Description down below check it out it’s a great way to automatically update all your containers and this is really something you should never forget on your server the next big section is about ssh authentication which is a secure protocol to remotely authenticate to a linux shell therefore it’s called a

Secure shell i guess here you will generally find tons of so-called best practices on the web trying to tell you what you should do but i want you to understand first that ssh is by itself a very secure protocol there’s nothing in particular that makes it by default insecure that you would

Need to change so most problems with ssh are just a result of bad habits people have so whenever you’re using credentials to authenticate via ssh make sure you’re always using a strong password and if you want to avoid using passwords completely it’s even better and more comfortable to use private and

Public ssh keys so that’s what i usually do because then i don’t need to remember any credentials and i also avoid you reusing my password accidentally i usually also create a separate user for my server so in most linux distributions that is also created by default during the installation procedure but if you

Don’t have a separate user and you’re still using the root then simply just create a new user by executing the user add command and then give it a name like question for example i also want to automatically create a new home folder for it and set the default shell to bin

Bash you could also give this a comment for example with c administrative administrative user but this is optional of course so let’s hit enter and now i want to add this user to any administrative groups because otherwise this user wouldn’t be able to execute any commands with root user privileges

And i couldn’t use that to administrate my server so let’s execute user mod a capital g and add this user to the groups sudo adm that’s responsible for reading some specific log files and also docker because then my user is also able to execute any docker commands without

Using the sudo command and of course specify the user’s name and i also want to give this user a strong password because if you execute a sudo command you also need to enter your password every time so let’s also execute passwd and enter the user’s name and now i want

To add a new password and that’s now updated successfully so now i could simply just authenticate with that user and this password to my server but as i just said i also want to add a private and public ssh keys to get rid of this password authentication and to do that

You can simply execute this on a powershell or on your linux client from wherever you want to authenticate to your linux server and just create a new private and public keypair so on the client you need to execute this command here the ssh keygen and i want to create this with

This length here to create a strong private and public key pair and you can also add a comment here because when you have multiple ssh keys and you cannot really differentiate them from each other you can always place a comment at the end of this key to easily

Identify who is this key for for example enter my email address and if we hit now enter we also can place this somewhere in a different location but this is a default location where your private key and also the public key is stored later and now you could also add a passphrase

So what’s the deal with passphrases on ssh keys and i usually treat it this way because if i can make sure that i can protect this file the same way as i could protect my root password i simply can just create an ssh key without the passphrase but if the client where the

Key is stored is not really secured maybe you are you don’t have a device encryption and someone could just break into your file system and copy this file away this user could be able to authenticate to your linux server without any pass rates in this case so this potentially could be insecure but

It’s just depending on how secure can you store your private and public key pair so in my case i have a device encryption i have a user authentication on my client machine so usually just create private and public ssh key pairs without the passphrase but just in case

You cannot protect this the same way like root password and make sure you’re setting up a strong passphrase on your ssh keys as well okay so now the files have been created and let’s ziggy into the dot ssh folder that’s the default folder where all the ssh keys are stored

And you can see there are two files created the public key and the private key so the private key as i said you never should share with anyone this is like your root user’s password yeah so protect it the best way you can but the public key you can simply share with

Anyone and you can distribute on all your servers and you can use that corresponding private key to authenticate to all the users where you have distributed your public key to so let’s do that and copy the public key onto my server and to do that we first

Of all need to create a dot ssh folder at our home users directory so cd into home question and let’s create a new folder dot ssh so let’s go back to our client and copy the public key and i want to copy this to my home server in this location here home christian

Dot ssh and i want to put this into my authorized piece file so if you want to use multiple ssh keys for example from different clients just copy them all into this one text file the authorized key files let’s hit enter and if i could remember my root password that should

Also work so that’s the reason why i usually just don’t use credentials but i use ssh keys to authenticate to my servers and now we also need to change the permissions on our server because otherwise the remote user is not able to read that authorized key files on the server so also execute

Ch own and set this to christian our newly created user to the dot ssh and i also want to do that recursively freely for all the files that are included in this folder let’s hit enter and now we can test if we now can authenticate to our server without using

Any password so let’s try to do that and let’s open a new ssh connection ssh question at bomb server and you can see i’m now logged in with my user and i’ve used this private and public keypad that i’ve generated to authenticate and now we can configure the ssh server to only

Accept authentication via private and public keypairs which is generally a good idea and we can also disable the root users login so that no one can actually try to authenticate with the root user so to do that you simply just need to change a configuration file so execute pseudo-vim

Etc ssh sshd underscore config so let’s enter the password for this user and this is our default configuration file of our ssh server and there are two different lines i want you to change the first one is the permit root login so you can change this line to

No and there’s also another line that is password authentication yes so if you don’t want anyone to authenticate without a private and public keypad anymore you simply can set this to no write and exit this file and now we need to restart our ssh server so enter sudo system ctl restart

Ssh by doing these simple steps to protect your ssh server you can stop the most automated malware and bots out there because they are just trying to connect to your ssh port with some common username and password combinations using ssh is great if you are just the only one whose

Administrating the server and it’s very straightforward and secure but if you want to manage shell access in a team and have a better audit logging what’s happening on a server i also would recommend you look at an access proxy and two-factor authentication for example i just recently made a video

About the software teleport which i’ll link you in the description down below and teleport is also the sponsor of this video so thanks teleport by the way and this is a free and open source access proxy so teleport allows you to authenticate to a linux shell through a

Centralized proxy server that you can completely self-host in your own environment or run it in the cloud and you can protect everything with two-factor authentication which is extremely useful because it adds another verification layer so for example you can authenticate with username and password but you still need to have a

Second factor like an authentication app on your phone and every time you want to log into your server you need to scan a new access token that’s only valid for a very short amount of time and once you’re authenticated you can get access to all the servers that are

Connected in the teleport access plane so you can add other servers are reverse tunnels as well so for example if your server is behind in that firewall which is very useful in some cases and that makes the management of linux servers very easy and secure and you can also

Share it easily in a team in the dashboard you can see who is locked into your server at which time and how long the session was you can also see what’s what’s done in the session because teleport records all the sessions to your server and you can

Easily play them back so it’s worth noting that the latest teleport version also manage the access to web interfaces databases and even kubernetes clusters which you should also not forget because usually you are not only managing your server in the shell most of the time you will also have

Administrative web interfaces and those interfaces can be protected by teleport with two-factor authentication as well so as i said teleport is completely free and open source as a community edition so you can just go to the home page download it there’s no sign up or login needed anywhere but there’s also an

Enterprise version existing which adds 24 7 support by their engineers and you can also use other features like signal sign-on to integrate that with other authentication services like an active directory for example so if you want to use that in bigger corporate environments with your development team

Or in an administrator team for example just reach out to the teleport team and you will find a link to their product in the description down below okay so let’s come to a topic that is probably also my favorite one and this is network security so here we’re

Talking about all traffic that’s coming in or going out of your server and my first tip here is that you should only expose services that you really need so first get an overview of all services and ports that your server is listening on because every application that wants

To receive data from a client needs to listen on a separate port most of them are defined for example port 80 for http for for free for https and 22 for ssh and so on you can easily do that by executing the ssl vpn command in the

Shell and then you’ll get a list of all applications that are currently listening on network ports and you should go through all of them and find out if you really need them what are they for and what exactly are they doing for example you can see that i have an

Ssh or server that is running and listening on port 22 and all ip addresses with 000 are applications that are listening on all incoming interfaces and this is the first question you should ask yourself for every service so do you really need that service and any services you don’t know you should

Always double check because if you’re running some old legacy applications that are using unsecured protocols or have serious security vulnerabilities you don’t want to make them accessible on your server and the best way is to really just go through all these services that are running and check if

You need them to listen on a part or not in case you find something that you should stop from listening the best way is to configure the application itself to stop listening on the port so most services have a configuration file in the ad etc directory of your linux

Server just check the documentation of the application and how to stop it from listening on the pod or if you really don’t need that application at all just remove it from the system but you could also enable a firewall and that’s a great way to control the traffic to your

Linux server as well so a firewall that is based on the ip table stack on your link server is the uncomplicated firewall and you can easily enable that on your server and it will drop all traffic that’s coming in and only allow what you explicitly describe in a rule

So before activating it make also sure that you’re not locking out yourself so always add a new rule for your primary management tool first so in my case this is sh so let’s allow ssh traffic to my server and we can do this with sudo ufw

Allow 22 and the port 22 stands for the ssh protocol so this will create two rules here first for ipv4 and the second for ipv6 and allow all traffic on that particular port and to demonstrate how that works i want to connect to a simple web server that is running on my home

System so you can see that an nginx web server is running on port 80 and listening for all incoming connections and when we open the browser window and hit enter you can see that it now connects to the nginx web server so now when we enable the firewall this

Shouldn’t be possible anymore because we only have the port 22 enabled and not the port 80. so let’s enable the uncomplicated fireball so let’s execute ufw enable and let’s confirm this and you can see the firewall is now active and running on our system you can see which rules are currently defined by

Executing sudo ufw status and you can see on the port 22 on ipv4 and ipv6 is allowed here so let’s go back to our browser window because we shouldn’t be able to access the nginx server anymore and you can see that it’s loading but we don’t get a response sometimes you need

To wait a few minutes until this gets active because if you already have an open tcp connection you need to wait a few seconds before this tcp connection will reset but if we now add a new rule to allow the port 80 in our firewall you can see the zulu ufw allow

Port 80 and we now reload the page you can see that the connection immediately is established so you can see with a simple firewall you can also control the access to your server but it’s very important that you understand that the ufw only helps you to control which

Traffic is allowed to come in and go out for any services that are allowed in the firewall rules it doesn’t add any protection at all so it does not scan the traffic or detect any attacks there’s also another problem with the uncomplicated firewall that i noticed when i’m running darker containers and i

Think you should know this since many of you are also running darker containers especially for administrative web interfaces so let’s go back here and let’s see which services i also have running on my server and you can see that i for example have an administrative web interface on this ip

Address here running on port 9000 and because i haven’t added a firewall rule to allow this service i shouldn’t be able to access it right so let’s just copy this and try to access it on our server but why am i still able to access the portina web interface because i haven’t

Really allowed this traffic in the uncomplicated firewall so this is because i have deployed this application in a darker container and the uncomplicated firewall by default doesn’t really care about what’s exposed in a container technically you can see the reason when taking a closer look at the ip table chains because the

Uncomplicated firewall operates in the default chains and docker creates a separate chains for its network that’s ignored by the uncomplicated firewall you can see it here and so that’s why i said in the beginning it’s not enough to just copy paste some commands you really need to carefully think about

Does this really apply to my setup or is there anything else that i haven’t considered so when you’re going through all the network ports and the applications that are running on your server especially take attention to the darker containers and how they are connected to the network and always test

Your configuration check if the rules really apply and so on and in the end you might be able to close down some unnecessary services that could cause some problems but you still will have many services that you would need to allow access to anyway because you might

Want to run a web server or mail server and of course those services need to listen on a part and they need to be accessible through a firewall so how can you protect those services from being attacked first you should avoid using any unsecured protocols for example if

You have a web application that’s running on port 80 you don’t want to authenticate with a password here because port 80 is http and the traffic is unencrypted so anyone in between could just steal your password a great way to expose any unsecured applications via https and trusted ssl certs is to

Use a reverse proxy a reverse proxy is a web server that sits in front of your application and it forwards any client requests to them so for example if you want to deploy a web application like bit warden or portainer that doesn’t come with an https server itself i

Usually just use a reverse proxy to expose this with trusted ssl search securely so i recently made some tutorials about the nginx proxy manager which is a reverse proxy based on the nginx web server i’m using this in most of my hobby projects because it’s very easy and straightforward to set up it

Also has a very intuitive web interface and it’s even able to block common exploits so this system is based on the nginx security settings and allows you to enable a very basic protection layer for your critical web applications however you should consider that this is still a hobby project so there are of

Course also other reverse proxies existing that will offer you even better protection and mostly these are commercial products and called web application firewalls i might do a separate video on those of things but again for a very basic and simple way to protect legacy http protocols with https

A reverse proxy like the nginx proxy manager is still a good idea second you should avoid exposing any administrative interfaces to the public internet in general so with this method you can for example deny access to an administrative web interface from an untrusted network so limit the listening ip of the servers

To a specific interface which is protected by a network like vpn or dmz dmz stands for demilitarized zone and it is a separate network that’s sitting in between the public internet and your local area network so usually this is an area where you can put your web applications and better control which

Traffic is allowed to come in or go out of the network i’ve done that on some services in my home network that are protected with a separate firewall appliance and on my cloud instances i usually configure all administrative web interfaces to only listen to an internal

Cloud network and i later use a vpn to establish a secured connection between those machines and access the cloud network so if you’re interested in learning about vpn connections i’ve made several videos about wireguard which is still a fairly new protocol but very powerful in my opinion and that’s also

Very easy to set up i also made a video about a managed wire guard vpn server that’s called tail scale and this allows you to deploy a very small service on all of your machines and then all the machines are automatically connected through a mesh vpn network and this is

Even working in public networks where you have nut firewalls in between and so on so this is really a great way to do it but if you know using tail scale or deploy your own wire guard server or use another axis gateway is completely up to you i don’t want to recommend any

Specific service to you just as a general advice you should find a way to somehow control the network access to your administrative applications through a network security concept and again you need to find your own strategy that works for you using vpns dmz’s or other gateways are just some ideas on how you

Could approach this another strategy is to use an intrusion prevention system and sometimes you will also hear the term intrusion detection system but it’s only detecting that something bad is happening of course we also want to prevent that so there are common open source ips systems available the most common one is

Fail to ban financial ban is very simple and easy to set up as this service just goes through your log files and rejects ip addresses that have tried accessing your server and made too many password failures so this is great for any services like ssh or web servers to

Block brute force attempts so let me show you an example how to protect an ssh server with fail to ban so first of all we need to install that via the sudo app install failed to ban and once this is done you can simply enable this we are the command sudo

System ctl enable fail to ban no so this will automatically enable and also start the service so now we can check if the service is running with sudo systemctl status failed to ban and you can see that the service is now active and running you can also get detailed information of what is

Happening in the background with a pseudo fail to ban dash client and now we want to call this status here so you can see that i currently have a j list and the j list is just a collection of configuration files where you want to block specific ip addresses for services

And currently that is only listening on the sshd log files you can also get the details about those services with the pseudo fail to ban client status sshd for example and now you can get more information about what is currently failed and if there are any banned ip

Addresses so usually on my local server there’s not much going on because this is protected from the public internet and you’re not able to access this without a vpn right just i’ve described in the previous section but maybe on my cloud instance which is publicly available i should be able to see

Something else so maybe just check this fail to ban blind status sshd so you can see on my cloud instance there’s much more going on even though i just deployed this service a few hours ago you can see that’s already 13 ips that are completely banned and two are currently

In the banned list so you can see that’s not banned for ever this is just for a short period of time but it’s a great way to block those brute force attempts on your server and you can even configure this on other services as well so you cannot just configure this for

Ssh also for ftp or for web services and so on it’s worth noting that there are much more versatile intrusion prevention systems out there but to block any automated tools in simple brute force attacks fail to ban is still a great option for home lab server or a small environment

Okay so now we have the case that something is somehow still able to infect our application even though we have limited the access to our applications we have configured an ips we have a reverse proxy but still something is going through and exploits our web server how can we protect from

This scenario that the attacker or the malware can’t spread out in fact the rest of the system so one effective method is to isolate the application with app armor and docker for example so limit what the application could do on the system app armor is installed and

Loaded by default on ubuntu it uses profiles for every application to determine which files and permissions the application requires so with the sudo app armor underscore status you can see which profiles are currently running on your applications so you can see i have eight processors that are currently

In the enforce mode so that are protected by app armor and buy those profiles so that’s working for containers and also for applications that are usually installed on a server as long as there are app armor profiles existing for it another way to isolate applications of course is to use docker

And docker is absolutely my favorite way of deploying applications on a server because it’s not just isolating the application in its own contained environment it’s also making it easier to maintain applications and you can see that i’ve deployed most applications and services on my server in darker containers like my web servers my

Managed interfaces my bit warden instance and so on so i’ve talked about docker in earlier videos so much and i’ve made several videos about it and if you just want to get started with basics i’ve put your link to my videos in the description down below of course nothing

Is 100 secure and systems like docker and app armor doesn’t really help you to protect the application itself but it’s helping you in case of a privilege escalation and it makes it much harder for an attacker or malware to break out of the container in fact the rest of the

System okay so this was how i would approach a basic security strategy for my linux servers i bet i i probably missed something and there are tons of other solutions and strategies you could think about maybe i’ll do separate videos on them and link them in the

Description below or make a playlist but this should and i can say this with confidence give you a good starting point when you’re seriously thinking about protecting your linux server so i hope you enjoyed this video don’t forget to give it a thumbs up and share it with

Your friends by the way i’m also doing a live streaming every week on thursday here on youtube so if you have any questions for me or you want to continue the discussion with me just jump into some of my live streams okay so that’s definitely enough for today i need a

Cold drink now and i hope i see you in the next video take care everyone i see you soon bye-bye yeah that’s absolutely true first you need to just um get familiar with the concept of containerization once you get familiar with it it’s actually very easy it’s it’s easier than traditional application

Deployment i would say but it’s like there is a hurdle that people just need to pass because there is something completely new that’s weird and you don’t know about it so you’re afraid of trying it out and if you get past that horror you see that it’s actually not that

Difficult the same experience i had with kubernetes