Time for Some Changes to My Primary Server

Published on 2024-08-27

Categories: tech
Tags: selfhosting

Time for changes in my current XCP-NG / Xen orchestra and docker swarm configuration. I'm migrating back to Proxmox and will try Kubernetes with K3S and k3sup.

My selfhosting Journey

My journey to self hosting started in the 90s and evolved (good and bad) over the past 25 or so years:

1. Created a Linux router using a 486, some ISA BNC network cards, and a dialup connection with a remote dialup controller.
2. Self hosting email
3. HTPC
4. Self hosting Plex, Kodi, RSS, Music collection, pictures, books, Nextcloud and more
5. home made NAS
6. Freenas (The old TrueNAS)
7. Openmediavault
8. Truenas Core
9. Truenas Scale
10. Docker servers
11. Docker swarm orchestration
12. Proxmox
13. XCP-NG / Xen Orchestra

My current setup is:

1. TrueNAS Scale
2. XCP-NG / Xen Orchestra
3. 3 Docker swarm clusters (private, public, infra)
4. 48 ports layer 3 swich, 2 access points
5. VLAN to separate my networks (dmz, private, infra, untrusted)

VM platform

User of both Proxmox and XCP-NG / Xen Orchestra for couple of years.

Running XCP-NG / Xen orchestra (XOA) at home means:

• XCP-NG is rock solid
• XCP-NG does not come with a web management console and the TUI console is quite limited
• you have two separate components to manage (XCP-NG nodes and XOA), but you can re-use XOA for multiple XCP-NG nodes.
• You can migrate VMs from one node to another without downtime. I tried this once, but never used since.
• The community edition of XOA has some limitations
• You can get Complete edition of XOA by spawning a VM and compiling it using the awesome XenOrchestraInstallerUpdater script
• You need to manually upgrade Complete edition of XOA
• You get a bit more complex cloud init config, but at the benefit of allowing to supply any of the user, meta, provider cloud init files directly in the web UI
• Dashboard is minimal.
• KVM hypervisor

Running Proxmox at home means:

• One component to manage
• automated updates
• LXC containers
• Nicer default Dashboard
• Simple cloud init configuration, but more limited. You need to do some CLI do extend some of the files.
• Simpler to manage
• As far as I remember, seamless migration from one node to another is not supported
• Has more complete off the shelf features
• KVM hypervisor
• A lot more YouTube tutorials for Proxmox
• quite good CLI

My main decider for going back to Proxmox: automated updates, a better web console and I don’t need seamless VM migration.

Otherwise they are quite similar.

Container orchestration platform

I’m running two docker swarm clusters to distribute my apps. It’s working OK, but Docker swarm seems to be quite dead (or stable).

The other main Container Orchestration platform is Kubernetes.

Why I’m switching to Kubernetes:

• Active community
• Many lightweight Kubernetes platform, like K0S, K3S, Minikube and more.
• I use it at work as a developer
• I want to learn more of it’s components and administration
• I also want to learn Helm, the Kubernetes “package manager”

For comments, use email or Mastodon

Don't forget to subscribe to my RSS feed!