Being a noob and all I was wondering whats the real benefit of having a monolithic lets say proxmox instance with router, DNS, VPN but also home asssistant and NAS functionalitiy all in one server? I always thought dedicated devices are simpler to maintain or replace and some services are also more critical than others I guess?
Services that can utilize the full power of a single machine are quite rare. I have about 15 docker containers in total taking up about 800mb of ram on one of my servers. In reality having multiple can be more complex and harder to maintain, not to mention power efficiency and cost.
Pretty much the tradeoff that you said. Harder to maintain an all in one box since things conflict with each other. That said, it’s also harder to maintain 10 devices instead of 2. Usually, you want to segregate your services based on maintenance schedule. Something that you reboot once a year like your router probably shouldn’t be on the same device as something that you might reboot every day, like home assistant, if you value your sanity.
Also, virtualization is pretty much dead-end now and will just make your life harder.
In terms of the easiest software available for self hosting, I would use a dedicated router and a dedicated nas, as those are fairly standalone and can be purchased as appliances. Then I would use a single machine with Debian or NixOS, and use it as a Kubernetes or Docker host. (Kubernetes is super easy with k3s and easier to maintain than Docker, but there’s a higher barrier to entry as you’d have to write your services with Pod files instead of docker-compose files)
I wouldn’t recommend something that tries to do everything, like Unraid, TrueNAS, or Proxmox, as they honestly obfuscate things and make things harder to maintain. Though they can be nice for DIY NASes.
If you’re interested in high availability and clustering for a DIY NAS, you could even look into ceph/rook, which is what I’m using for my NAS, but it’s like 20x the effort of just having a standard NFS appliance.
Kubernetes is super easy with k3s and easier to maintain than Docker
I don’t think I’ve ever heard anyone say this… Kubernetes is a massive pain in the ass to learn, maintain and troubleshoot. If you find it easy that’s great, but it’s not for everyone.
I mean that with k3s you can get a kubernetes cluster running with 0 effort on a single machine. It is easier to maintain, because it handles restarting containers, updating containers, managing ports, provisioning storage, creating databases, etc for you. I’ve found the logs and events system to be super useful for troubleshooting compared to Dockerd, but maybe it can be tricky if it does something you don’t expect it to.
Obviously you need to learn how to use that automation to take advantage of it, and stuff like networking and persistent volumes can be confusing if you don’t have a good guide on it. The fact that there are different drivers for networking, storage, database management, etc can also take a bit of time. That said, networking and storage can be confusing on Docker too if you don’t have a good guide, and Docker-compose also has a learning curve, so I honestly don’t think Kubernetes is that much more effort. The main thing is that most guides are written for Docker, but the Kubernetes documentation is really good too.
If you just want to just run containers for jellyfin and home-assistant, Docker compose will be good enough. But if you want databases, reverse proxy, certificates, dns, self-healing, etc, for running bigger stuff like nextcloud and lemmy, then I would spend the extra 50% effort and do it on Kubernetes, it’ll save you time and headaches in the long run.
Asking an LLM like Lllama or ChatGPT might be a good way to learn the basics with Kubernetes, but things move fast once you start getting into the newest operators like CNPG and Gateway API.
I do all that with docker… I fail to see what Kubernetes adds to that on a single machine.
Kubernetes does it a lot better. No more messing with caddy config files, or docker sockets, you get the real deal, production stuff.
A new high-availability postgres cluster with automatic backups is just a Cluster, a firewall rule is just a NetworkPolicy, a new subdomain is just an HTTPRoute, a new proxy container is just a Gateway, a new auto-renewed Let’s Encrypt certificate is just a Certificate, and DNS is set up automatically with the domain name from the HTTPRoute without me touching anything. Everything is high-availability and self-healing, I’ve never had anything go down or crash.
The other thing is ArgoCD, which automatically syncs your cluster with git. If I edit any of my config files in git, it is instantly updated on the cluster itself.
Here is my configuration for my 200+ containers, even my Lemmy instance is running here: https://codeberg.org/jlh/h5b/src/branch/main/argo/custom_applications
Docker and the Docker ecosystem copies a lot of features from Kubernetes, because they’re essentially the same thing, but Kubernetes does it in a production-ready, maintainable way. Kubernetes is an automation tool that lets 1 engineer do the work of 10.
Right, right, you just have to reinvent a dozen wheels, use only software that Kubernetes knows how to work with, and learn a bunch of new names for everything.
deleted by creator
I’m no expert; only been dipping my toes in the selfhosted water for a few years.
But my thought process would be all the main stuff on your main server and the redundant instances on a little backup
I have a Raspberry Pi 4B as my load bearing Mac mini.
