Budget and powerful k8s cluster.
Last year became interested in Kubernetes. Administration side of this tool is more interesting for me, however I have a lot of fun using k8s cluster for self-hosted applications.
A few months ago I tried to use some managed services (mainly from Scaleway and Digital Ocean) to check how my setup will work in production environment. On my homelab cluster I have 8 powerful workers (but they are VMs on one machine), 3 masters and metallb. I'm able to run a lot of containers there without any performance issues. I wanted to duplicate this experience in a production (somehow) environment.
I decided to rent some cheap dedicated servers and set up a cluster myself. With load balancer for ingress controller, persistent storage on nodes' drives and at least 4 worker nodes.
Project is in progress. First updates soon.
10th December 2024
I have finally finished the first phase of the project. I have created a set of Ansible roles that define:
- Base server configuration - installation of all tools needed for manual interventions and configuration changes,
- Basic Kubernetes node - common for masters and workers,
- Kubernetes master node
- Kubernetes worker node
and Ansible playbook, which initializes the cluster with locally signed certificates. This was a very important part and it took me some time to debug issues with certificates common names, kubeconfigs etc.
I'm preparing a blog post about this part, but first I have to finish tidying up the tasks files in the roles. I have separate files for different aspects (e.g. generating private keys and certificate signing requests, installing additional software etc.), but not everything is so well organised.
Next steps for this stage of the project:
- Preparing kubernetes-storage role to set up Ceph Rook. I have been testing a few storage solutions for Kubernetes and this looks the most promising to me.
- Adapting my kubernetes-master role for multi-master configuration.
- Roles for load balancers. Currently I'm using HAProxy for multi-master load balancer, so this role will probably just install that tool, fill the configuration and apply it to the LB server.