Managing kubernetes with flux [K3s, part 2]

January 29, 2025

Raspberry on motherboard

GitOps with Fluxcd

Instead of managing kubernetes resources manually (who would do that in 2025?), I'm using a GitOps tool to declare the resources in a git repository and let the tool do the sync for me.

I've already have experience with ArgoCD, but I went with FluxCD for performance reasons:

ArgoCD requires to run its server within the Kubernetes cluster, consuming a lot of resources, and as I'm running on Raspberry Pis, FluxCD is more efficient with its modular design, only a small number of components are installed.

Be careful, as Fluxcd is lightweight, there is no built-in UI, it's designed to work with Kubernetes-native tools (kubectl) or Grafana/Prometheus for observability.

K3s and Fluxcd

For the K3s cluster, I've used the single-server architecture to start.
There is no high availability yet, I fixed that later on and will discuss it in a follow up blog post.

In the git repository, I've slightly changed the recommanded structure to make it simpler for my use case:

├── apps
│   ├── podinfo
├── modules
│   ├── monitoring
│   ├── secrets 
│   └── traefik
└── clusters
    └── production
        ├── infrastructure
        └── apps.yaml

This way, I'm building reusable infrastructure modules, that can be used in any cluster, and multiple times if needed thanks to them being Kustomization resources, with substitution variables.

There is no mention of environment (production, staging) in the apps because I've no need for it yet, I'm also using substitution variables only when needed.

Here's a snapshot of my kubernetes as code with flux.

Running Fluxd

I'm running this command on each node, using ansible:

$ flux bootstrap github \
    --token-auth \
    --owner=CHANGE_ME \
    --repository=CHANGE_ME \
    --branch=master \
    --path=clusters/production \
    --reconcile=false \
    --personal

Then, I can lookup /etc/rancher/k3s/k3s.yaml on the server to find the credentials to authenticate on the cluster. I will simply add them to my ~/.kube/config.

Once the flux components are installed, I can check the status of the components deployed by flux:

$ flux get ks
NAME                REVISION                SUSPENDED   READY   MESSAGE                                
apps                master@sha1:6f75ce50    False       True    Applied revision: master@sha1:6f75ce50      
flux-system         master@sha1:6f75ce50    False       True    Applied revision: master@sha1:6f75ce50  
monitoring          master@sha1:6f75ce50    False       True    Applied revision: master@sha1:6f75ce50  
secrets             master@sha1:6f75ce50    False       True    Applied revision: master@sha1:6f75ce50  
traefik             master@sha1:6f75ce50    False       True    Applied revision: master@sha1:6f75ce50

Everything's running fine.