k3s is now the most-downloaded Kubernetes distribution — 33,100+ GitHub stars and tens of thousands of new installations per week (SUSE, May 2024). But here's the thing most homelab k3s guides won't tell you: a production HA cluster needs SIX components deployed in a specific order, and getting that order wrong means you spend hours chasing error messages that point at the symptom instead of the cause.
I know because I got it wrong. Multiple times. On my own cluster.
Most guides cover two, maybe three components — kube-vip and MetalLB, or MetalLB and Traefik. Nobody covers the full chain: kube-vip → secrets → certificates → load balancer → reverse proxy → gitops orchestrator. And the dependency chain between them? The part where each step exists because the NEXT component needs something the PREVIOUS one provides? That's the part nobody explains. They tell you what to type. They don't tell you why the order is non-negotiable.
This guide is different. It's architecture, not a tutorial. By the end, you'll understand what each component does, why it must be deployed in a specific order, and how they fit together as a system. Every architectural decision here is backed by a real, running cluster — 18+ applications, public GitHub repo, production traffic. For step-by-step instructions with actual commands, each component has a dedicated deep-dive post. This is the map. The spokes are the turn-by-turn directions.
Key Takeaways
- kube-vip and MetalLB solve DIFFERENT problems at different network layers — confusing them is the #1 community support question, and running both with overlapping modes causes ARP conflicts that make services intermittently unreachable
- Our production k3s HA homelab cluster requires six components in a hard dependency chain: kube-vip → Sealed Secrets → cert-manager → MetalLB → Traefik → ArgoCD — deploy out of order and you get cascading failures
- In 2026, 82% of Kubernetes users run it in production (CNCF Annual Survey, January 2026) — the bootstrap discipline described here is what separates production deployments from toy clusters
- A 2-node k3s HA cluster offers ZERO control plane fault tolerance — this isn't a bug, it's etcd math — but it's the launchpad for a 3-node upgrade that's a single Git commit away
Table of Contents
- Load Balancing & High Availability — kube-vip and MetalLB
- Security Foundations — Sealed Secrets and cert-manager
- Networking Architecture — IP Planning, the Three Layers, and Traefik
- The Dependency Chain — Bootstrap Order and the GitOps Boundary
- The 2-Node HA Paradox (and the 3-Node Upgrade Path)
- Frequently Asked Questions
- Where to Go From Here
Load Balancing & High Availability — kube-vip and MetalLB
k3s is deployed across more than 2,300 U.S. Home Depot stores running production workloads at the edge (Data Center Knowledge, June 2022), and every one of those deployments had to solve the same architecture problem you're facing: how do you separate control plane availability from workload service exposure? kube-vip and MetalLB solve different problems at different layers of the network stack, and confusing them is one of the top community support questions I see on r/homelab and r/kubernetes. kube-vip provides a stable virtual IP for the control plane API server so your kubectl doesn't die when a node goes down. MetalLB assigns real IPs from your LAN to workload services so your apps are actually reachable. They are both required, they are NOT interchangeable, and — here's the gotcha that cost me an evening of debugging — running kube-vip's --services flag alongside MetalLB causes ARP conflicts that make services intermittently unreachable.
kube-vip — Control Plane Stability
Here's the problem kube-vip solves: without a virtual IP, your kubeconfig points at a specific node's IP address. When that node goes down — and it will, this is a homelab, not a data center with five-nines uptime — your kubectl stops working. The API server is still running on the surviving node, but you can't reach it because you're pointed at a dead IP.
kube-vip creates a virtual IP that floats between your control plane nodes. If the node holding the VIP goes down, another node takes it over — usually within seconds. Your kubeconfig points at the VIP, not any individual node, so it keeps working regardless of which node is the current leader.
DaemonSet mode vs static pod: I use DaemonSet mode. Static pod mode requires you to SSH into every control plane node and place a manifest in /etc/kubernetes/manifests/ — that's fine for three nodes, but it's a manual step you'll forget during a rebuild at 2 AM. DaemonSet mode runs kube-vip as a Kubernetes resource inside the cluster it protects. The key value is vip_address — the IP that floats between your nodes. kube-vip auto-detects the network interface when vip_interface is left blank, so you usually don't need to set it unless you have multiple interfaces and want to pin it to a specific one.
Why kube-vip must be deployed BEFORE additional control plane nodes join: When k3s generates its API server TLS certificate, it includes SANs (Subject Alternative Names) for every address you pass via --tls-san. If the VIP isn't in that list when the first node is installed, the certificate won't include it — and you CANNOT add SANs to an existing certificate without reinstalling. Join a second control plane node without the VIP in the cert, and it'll fail with TLS errors that look like network problems but are actually certificate problems.
MetalLB — Workload Service IPs
Without MetalLB, every LoadBalancer-type service in your cluster stays <pending> forever. Cloud providers like AWS and GCP have built-in load balancer controllers that auto-assign external IPs. Bare metal doesn't. MetalLB fills that gap.
In L2/ARP mode (the right choice for most homelabs), MetalLB elects one node as the "speaker" for each service IP. That node responds to ARP requests for the IP, and traffic flows through it to the service. If that node goes down, another node takes over — automatic failover, usually within a few seconds. It's not true load distribution across nodes, but for a homelab running a handful of services, it's exactly what you need.
GOTCHA: --disable servicelb is NOT optional. k3s ships with its own load balancer — Klipper, aka ServiceLB — enabled by default. If you install MetalLB without disabling ServiceLB first, both daemonsets compete to answer ARP requests for the same IPs. The result? Services that work for 30 seconds, then don't, then work again. Maddening to debug because it looks like a network flapping problem. The fix is simple but it MUST happen at install time: --disable servicelb in your k3s install command or config file. You cannot disable ServiceLB after installation.
CRD ordering matters: Apply the main MetalLB manifest first and WAIT for the controller and speaker pods to be ready BEFORE applying your IPAddressPool and L2Advertisement resources. The CRDs those resources reference don't exist until the main manifest is applied. Apply them in the wrong order and you'll get "no matches for kind" errors.
The ARP Conflict Nobody Documents
This one deserves its own section because it burned me and I see it in GitHub issues constantly.
kube-vip has a --services flag that makes it act as a ServiceLB — it'll assign IPs to LoadBalancer services, just like MetalLB does. If you enable this flag AND run MetalLB, you now have TWO daemonsets competing to answer ARP for the same IPs. One says "that IP is mine!", the other says "no, MINE!", and your services flap between reachable and unreachable in a pattern that looks exactly like a hardware problem.
The fix: kube-vip for control plane ONLY (no --services flag). MetalLB for workload services ONLY. Clean separation. Two different layers. Never the two shall overlap.

k3s is deployed in some genuinely impressive places — more than 2,300 U.S. Home Depot stores run on it (Data Center Knowledge, June 2022). If k3s can handle point-of-sale infrastructure across thousands of retail locations, the architecture scales to your homelab.
Security Foundations — Sealed Secrets and cert-manager
In 2025, 77% of organizations have adopted GitOps to some degree (CNCF Annual Survey, April 2025), and the #1 blocker for public-repo GitOps is secrets management. Sealed Secrets and cert-manager solve the two security problems every homelab hits within the first week: how do I commit my configs to a public Git repo without leaking API keys and passwords, and how do I get valid TLS certificates for every service without manually renewing them every 90 days? They're deployed back-to-back because cert-manager's own credentials are stored as a SealedSecret — deploy cert-manager first and it can't decrypt its own configuration.
Sealed Secrets — Encrypted Secrets in Git
Here's a fun moment from my own repo: GASP there are secrets in a public GitHub repository! Except they're not really secrets — they're SealedSecrets. Encrypted blobs that only the controller in my cluster can decrypt. The private key never leaves the cluster. The encrypted blob is safe to commit, push, and share.
The architecture is straightforward: the Sealed Secrets controller holds a private key inside your cluster. The kubeseal CLI tool encrypts your Secret using the controller's public key. The resulting encrypted blob — a SealedSecret resource — is safe to commit to a public repo because without the private key (which lives in the cluster, not in git), it's meaningless bytes.
The private key backup is NON-NEGOTIABLE. Lose the Sealed Secrets private key and every SealedSecret in your repo becomes permanently unreadable. There is no recovery. No reset flow. No "I forgot my password" link. Back up the key immediately after deploying the controller — I store mine in Bitwarden. The backup command is kubectl get secret -n kube-system -l sealedsecrets.bitnami.com/sealed-secrets-key -o yaml > main.key. Store that file somewhere that isn't your cluster, then delete the local copy.
Sealed Secrets is the thing that makes public-repo GitOps possible — without it, you're choosing between private repos (which break the "clone and learn" model) or plaintext secrets (which break everything).
There's a third path — external secret managers like HashiCorp Vault or AWS Secrets Manager with something like External Secrets Operator. These work, but they add significant complexity (you're now running and backing up a Vault cluster) and they break the truly self-contained GitOps model where a git clone + kubectl apply is all you need to reconstruct the cluster. For a homelab, Sealed Secrets hits the sweet spot: secrets are encrypted in git, the decryption key lives in the cluster, and disaster recovery is "restore the private key backup, apply the repo."
cert-manager — TLS Automation
Let's Encrypt gives you free TLS certificates. cert-manager automates the entire lifecycle — request, renewal, rotation — so you never have to think about certificate expiry again. For a homelab running 18+ services, that's the difference between "TLS everywhere" and "I'll set up TLS later, I promise."
I use DNS-01 challenges instead of HTTP-01. Two reasons: (1) DNS-01 gives you wildcard certificates (*.home.diceninjagaming.com covers every service with one cert), and (2) DNS-01 works without opening port 80 to the internet — your cluster sits behind NAT, Let's Encrypt verifies domain ownership via a DNS TXT record instead. The trade-off is you need a DNS provider with an API (I use AWS Route 53), but that's a one-time setup.
The staging → production promotion path matters. Let's Encrypt has rate limits. If you point a new ClusterIssuer directly at production and something is misconfigured, you'll burn through your weekly quota in minutes and be locked out for seven days. Always test with the staging issuer first (letsencrypt-staging), verify certificates show READY=True, then switch to production.
Why cert-manager before Traefik: TLS certificates are Kubernetes Secrets. Traefik reads those Secrets at startup to terminate TLS. If Traefik starts before the certificates exist, you get self-signed cert warnings — and browsers HATE those. cert-manager must be running and have issued certificates BEFORE Traefik tries to read them.
Wildcard certificates live in the traefik namespace and cover all services under your domain. Per-app certificates (for things that need their own TLS, like Authentik with OIDC) live in the app's own namespace. This separation keeps certificate management clean as your cluster grows.
The Credential Bootstrapping Problem
This is the first real-world example of the dependency chain this entire guide is about, and it's subtle enough that I missed it on my first build:
cert-manager needs Route 53 API credentials to solve DNS-01 challenges. Those credentials are a Secret. To commit them to a public repo, they must be a SealedSecret. The Sealed Secrets controller must already be running to decrypt that SealedSecret. Therefore: Sealed Secrets MUST come before cert-manager.
Deploy them in reverse and cert-manager sits there unable to decrypt its own credentials, waiting for a controller that hasn't been installed yet. The error message won't say "hey, you deployed these in the wrong order." It'll say something about DNS propagation or authentication failures, and you'll spend an hour debugging Route 53 permissions that are perfectly fine.
Networking Architecture — IP Planning, the Three Layers, and Traefik
82% of Kubernetes users now run it in production, up from 66% in 2023 (CNCF Annual Survey, January 2026), and the networking architecture is what separates "it works on my machine" from "it works." The k3s networking stack has three layers: control plane access (kube-vip VIP), service exposure (MetalLB IPs), and HTTP/S routing (Traefik ingress). They form a pipeline — external traffic flows through Traefik's LoadBalancer IP (assigned by MetalLB) to the Traefik pod, which routes it to the right service and pod. Control plane traffic flows through the kube-vip VIP to whichever node is the current API server leader.
IP planning is the foundation. If your VIP collides with your DHCP range, or your MetalLB pool overlaps with your router's static assignments, you get intermittent IP conflicts that look exactly like random cluster failures. No amount of YAML debugging fixes an IP conflict. I learned this one the hard way — my first build required reinstalling the initial control plane node because I forgot to include the VIP in --tls-san and you cannot add SANs to an existing k3s API server certificate. Can't be done.
IP Planning — The Step Every Guide Skips
Before installing a single component, carve out three reserved blocks from your LAN subnet:
| Component | IPs Needed | Type | Example |
|---|---|---|---|
| kube-vip VIP | 1 | Single IP — control plane virtual address | 192.168.5.200 |
| MetalLB Pool | Range | Multiple IPs — workload LoadBalancer services | 192.168.5.201–192.168.5.220 |
| Traefik IP | 1 | First address in MetalLB pool — reserved for ingress | 192.168.5.201 |
MetalLB's pool is a range of reserved IPs allocated as-needed to Kubernetes LoadBalancer services. Traefik gets the first one. Future services (like an MQTT broker or a game server that needs its own external IP) get the next available address from the pool.
The clean way to handle this: pick a block near the top of your subnet (I use 192.168.5.200–220), split it between the three consumers, and configure your router's DHCP server to exclude the entire block from dynamic assignment. On pfSense/OPNsense, this is under Services → DHCP Server → LAN → "Address Pool Range" — set the range to exclude your reserved block. On consumer routers, look for "Address Reservation" or "DHCP Reservation" and exclude the range.
The --tls-san value in your k3s install command MUST include the VIP. This value gets baked into the API server's TLS certificate at install time — it's the list of addresses the certificate is valid for. If you forget it — and I did — the API server's certificate won't include the VIP as a valid address, and kubectl connections through the VIP will fail with TLS errors. This is a certificate problem that looks like a network problem. And you cannot add SANs to an existing certificate without reinstalling the first control plane node. Not the end of the world, but an hour of your life you won't get back.
kube-vip auto-detects the network interface when vip_interface is left blank in the manifest, so you usually don't need to set it unless you have multiple interfaces and want to pin it to a specific one.
Why Three Layers?
Each layer handles a different part of the traffic path. Here's the full picture:
| Layer | Component | What It Handles | Traffic Type |
|---|---|---|---|
| 1 — Control Plane Access | kube-vip | Stable VIP for k3s API server | kubectl commands, node registration, controller communication |
| 2 — Service Exposure | MetalLB | External IPs for LoadBalancer services | Incoming traffic to your cluster from your LAN |
| 3 — HTTP/S Routing | Traefik | TLS termination, path-based routing, middleware | HTTP/S traffic to your applications |
The pipeline works like this: an external request hits your Traefik LoadBalancer IP (assigned by MetalLB from its IP pool). MetalLB's speaker node answers the ARP request and routes the traffic to the Traefik pod. Traefik terminates TLS (using certificates from cert-manager), applies any middlewares (auth, rate limiting, headers), and routes to the correct service and pod. Control plane traffic is separate: kubectl connects to the kube-vip VIP, which forwards to whichever node is the current API server leader.

Traefik — Why Not NGINX?
This is a hot take! But there are three reasons specific to k3s homelabs that make Traefik the right choice over NGINX Ingress Controller, and neither of them is "because k3s ships with Traefik."
Reason 1 — ExternalName resolution: Traefik resolves ExternalName services through the Kubernetes API, not CoreDNS. This means it can proxy to raw IP addresses without DNS resolution. Here's a real example from my cluster: I have Docker containers running on my Unraid NAS at 192.168.5.20 — Immich, Paperless-NGX, and a few other things that haven't been migrated to Kubernetes yet. I expose them through Traefik via an ExternalName service pointing at that raw IP. NGINX Ingress Controller resolves ExternalName through CoreDNS, which has no A record for a bare IP address. Traefik resolves it through the Kubernetes API and connects directly. This architectural detail is something you only learn by running it.
Reason 2 — Dynamic config reload (no pod restarts): Traefik watches the Kubernetes API and reloads routing configuration live — add a new IngressRoute, change a middleware, rotate a TLS certificate, and Traefik picks it up without restarting a single pod. NGINX requires a configuration reload on changes, which can drop active connections. In a homelab where you're constantly iterating — adding services, tweaking routes, experimenting with middleware — not having to restart your ingress controller every time you change something is a genuine quality-of-life improvement. Over the lifetime of a cluster, that's hundreds of reloads you don't think about because Traefik handled them silently.
Reason 3 — IngressRoute CRD with middleware chaining: Instead of NGINX-style annotations and ConfigMaps, Traefik uses a first-class Kubernetes CRD called `IngressRoute`. Middlewares are also CRDs — you define them once, then chain them together declaratively. A typical route in my cluster chains three middlewares: Default security headers → internal IP whitelist → Authentik forward-auth → the service. Each middleware is a separate resource, version-controlled in git, and reusable across any number of routes. Add a new service that needs the same auth? Reference the same middleware. Want to add rate limiting to one specific route? Insert a rate-limit middleware in that route's chain without touching anything else. With NGINX, this logic lives in annotations on the Ingress resource or in a ConfigMap — harder to compose, harder to version, harder to reuse. The IngressRoute CRD makes your routing configuration as declarative and auditable as the rest of your cluster.
NGINX is faster in raw throughput benchmarks — purpose-built ingress controllers consistently outperform general-purpose proxies on requests-per-second. But for a homelab running a dozen or two dozen services, the difference is academic. Traefik's architectural fit — ExternalName resolution for hybrid Docker/Kubernetes setups, dynamic reload for constant iteration, and the IngressRoute CRD for declarative middleware chaining — matters far more than a few percentage points of throughput you'll never miss.
The networking architecture described here — three layers, clean separation, IP planning as foundation — is what "production" actually means for a homelab.
The Dependency Chain — Bootstrap Order and the GitOps Boundary
The average organization now runs 2,341 containers, more than double the 1,140 in 2023 (CNCF Annual Survey, April 2025). At that scale, you can't manage infrastructure with kubectl apply — you need a dependency chain that works. The six bootstrap components have hard dependencies: kube-vip → Sealed Secrets → cert-manager → MetalLB → Traefik → ArgoCD. Deploy out of order and you'll spend hours debugging failures where the error messages point at the symptom, not the cause. This is the part no existing guide covers — every tutorial says "do this" without explaining why the order can't be changed.
The Full Dependency Table
Every step exists because the next component needs something the previous one provides. Break the chain and the failure cascades:
| Step | Component | Depends On | Why It Must Be First | Failure Mode If Skipped |
|---|---|---|---|---|
| 0 | kube-vip | (none) | VIP must exist before additional nodes join — TLS SANs are baked into the API server certificate at install time | Additional control plane nodes fail to join with TLS errors that look like network problems |
| 1 | Sealed Secrets | kube-vip (cluster must be stable) | Controller must be running before any SealedSecret resource is applied — it needs to exist to decrypt them | SealedSecrets applied before the controller exists are never seen by the controller and remain permanently undecryptable |
| 2 | cert-manager | Sealed Secrets (Route 53 credentials are a SealedSecret) | TLS certificates must be issued before Traefik starts — Traefik reads them at startup | Traefik starts with no certificates; browsers show self-signed cert warnings |
| 3 | MetalLB | cert-manager (certs exist; next is service IPs) | LoadBalancer IPs must be available before Traefik requests one | Traefik service stays <pending> forever — no external IP assigned |
| 4 | Traefik | MetalLB (needs LoadBalancer IP) + cert-manager (needs TLS certs) | Ingress must exist before any IngressRoute resource is created, including ArgoCD's own UI | ArgoCD UI has no ingress; apps unreachable; entire cluster is a black box |
| 5 | ArgoCD | Traefik (needs ingress for its UI) | GitOps controller is the last manual step — after this, everything is GitOps | You keep kubectl apply-ing forever; no version history; no self-healing; no drift detection |
This table is the core deliverable of this guide. Print it. Bookmark it. When something breaks during your bootstrap, start here — the error message will point at the symptom, but the dependency table points at the cause.
The GitOps Boundary — What Lives Outside ArgoCD and Why
kube-vip is the bootstrap paradox: ArgoCD cannot manage the thing ArgoCD needs to run. ArgoCD requires a stable cluster with a reachable API server. kube-vip provides that stable API server address. Therefore, kube-vip can never be managed by ArgoCD — it must exist before GitOps is possible.
Everything else — Sealed Secrets, cert-manager, MetalLB, Traefik — is deployed manually during bootstrap but ADOPTED by ArgoCD when you apply app-of-apps.yaml. ArgoCD sees these resources already exist in the cluster, recognizes them as matching the desired state in git, and adopts them rather than reinstalling. They go from "manually managed" to "GitOps managed" without a redeploy.
The handoff moment: After kubectl apply -f app-of-apps.yaml, every future change is a git push. Never a kubectl apply. New app? Create apps/<name>/ and apps/manifests/<name>.yaml, push, done. Config change? Edit the manifest, push, ArgoCD detects drift and syncs. This is the moment your cluster stops being a collection of manual commands and becomes a system.
Once ArgoCD takes over, sync waves replicate the bootstrap ordering declaratively.
- Sync wave -3 runs SealedSecrets
- Sync wave -2 runs operators (cert-manager, CNPG)
- Sync wave -1 runs platform services (Sealed Secrets, MetalLB)
- Sync wave 0 runs workloads (Traefik, ArgoCD, applications).
The manual bootstrap order becomes the automated sync order — same logic, different mechanism.
The bootstrap discipline that manages six components today scales to managing thousands of containers tomorrow. The principles don't change — only the scale does.
The 2-Node HA Paradox (and the 3-Node Upgrade Path)
A 2-node embedded etcd cluster requires 2 out of 2 nodes for quorum — that's zero fault tolerance by mathematical definition (Sidero Labs, September 2022). A 2-node embedded etcd k3s cluster offers ZERO control plane fault tolerance. Both nodes must be available for quorum — that's 2 out of 2. Lose one node and the API server is unreachable. This isn't a bug or a misconfiguration. It's etcd math, and it's the thing every "HA" guide either doesn't know or doesn't want to tell you.
I'm telling you because knowing the limitation is better than discovering it at 11 PM when a node goes down and you can't figure out why kubectl stopped working even though "the other node is still running!" (Not a fun time, let me tell you!)
The Quorum Math
The etcd quorum formula is floor((n-1)/2) failed nodes tolerated. Here's what that means in practice:
| Nodes | Quorum Required | Failed Nodes Tolerated | HA? | Recommendation |
|---|---|---|---|---|
| 1 | 1/1 | 0 | No | Dev/testing only |
| 2 | 2/2 | 0 | No | Valid starting point; the VIP is stable even if fault tolerance is not |
| 3 | 2/3 | 1 | Yes | Minimum for production HA |
| 5 | 3/5 | 2 | Yes | Overkill for homelab unless you have specific uptime requirements |
Source: Sidero Labs, "Why should a Kubernetes control plane be three nodes?" (September 2022) — the definitive article on etcd quorum sizing.
What Actually Happens When a Node Goes Down in 2-Node
The API server becomes unreachable. kubectl stops working. You can't deploy, can't scale, can't check logs through the API. It FEELS like the cluster is dead.
But here's what's still running: all your workloads. kubelet doesn't need the API server to keep pods alive — it already has the pod specs. Your applications continue serving traffic. DNS continues working (CoreDNS pods on the surviving node). MetalLB may stutter during leader re-election — ARP responses can take a few seconds to converge — but services come back. The cluster isn't dead. You just can't control it until the second node recovers or you add a third.
The 3-Node Upgrade Path
Adding a third node changes everything. Quorum becomes 2/3 — you can lose any one node and maintain full control plane operation. The upgrade path, once your cluster is running:
- Add the third node (
k3s joinwith the VIP and token) - Adjust ArgoCD replica count (Redis quorum needs 3 nodes for HA mode)
- Increase CloudNativePG instances from 2→3 (database HA)
- Increase Longhorn replicas from 2→3 (storage HA)
All via Git commits. No downtime. No manual intervention beyond the initial k3s join, and even that can(should!) be in an Ansible playbook or Terraform module. This is the production way — the 2-node cluster was never the destination. It was the launchpad.
k3s HA vs Single Node: Best Architecture for Your Homelab?
Frequently Asked Questions
What's the difference between kube-vip and MetalLB in a k3s cluster?
kube-vip provides a virtual IP for the k3s control plane API server, keeping kubectl connected to a stable address regardless of which control plane node is the leader. MetalLB assigns IPs from your local subnet to LoadBalancer-type workload services, like Traefik's ingress. They operate at different network layers and are both required — confusing them is the #1 community question in r/homelab.
Do I need to disable ServiceLB to use MetalLB with k3s?
Yes. k3s ships with Klipper ServiceLB enabled by default. If MetalLB and ServiceLB run simultaneously, both attempt to assign external IPs via ARP and services flap between reachable and unreachable. Disable ServiceLB at install time with --disable servicelb in your k3s install command. This cannot be changed after installation — you'd need to reinstall k3s.
Why does MetalLB show <pending> for my Traefik external IP?
Three common causes: (1) MetalLB CRDs applied after IPAddressPool/L2Advertisement — apply the main manifest and wait for controller/speaker rollout first. (2) ServiceLB still running alongside MetalLB — disable with --disable servicelb. (3) IP pool doesn't overlap your LAN subnet — MetalLB only advertises IPs via L2/ARP on the same subnet as your nodes. The assigned IP must be in your router's subnet.
Does my 2-node k3s HA cluster survive a node failure?
For workloads, yes — kubelet keeps pods running without the API server. For the control plane, no — embedded etcd requires 2/2 nodes for quorum. The API server becomes unavailable until the failed node recovers. This is expected etcd behavior, not a misconfiguration. Adding a third node changes quorum to 2/3 and provides one-node fault tolerance.
Why use Traefik instead of NGINX Ingress Controller with k3s?
Traefik resolves ExternalName services via the Kubernetes API rather than CoreDNS, so it can proxy to raw IP addresses (NAS, Docker containers) without DNS resolution failures. Dynamic configuration reloading means Traefik doesn't need to restart when applying new IngressRoutes or Middleware. The IngressRoute CRDs that Traefik uses are **chefs kiss** amazing: Easy to read, easy to manage, and easy to store in source control.
Can I manage everything with ArgoCD from day one?
No. kube-vip must exist before the cluster is stable enough for ArgoCD to run — it's the bootstrap paradox. Sealed Secrets, cert-manager, MetalLB, and Traefik are all required for ArgoCD when app-of-apps.yaml is applied. ArgoCD detects existing resources and manages them going forward without reinstalling. After that handoff, every change is git push — never kubectl apply.
Where to Go From Here
This pillar covers the architecture. Here's the thing about architecture though — it's only useful if you actually build something. Remember: you can have excuses or results. YOU CAN'T HAVE BOTH.
The six components are a system. The dependency chain is non-negotiable. The GitOps boundary is real — kube-vip lives outside by necessity, everything else gets adopted. And your 2-node cluster has zero control plane fault tolerance by design, and that's OK, because the 3-node upgrade is a Git commit away.
Every architectural decision in this guide is backed by working configs in a public repo. Not hypothetical YAML. Not "hello world" examples. Real configs running 18+ applications in a real cluster, right now. Go look at the repo: github.com/Taegost/homelab-k8s. Clone it. Compare it to what you're building. Submit a PR if you find a better way — I mean that.
What's your setup look like? Running 2 nodes or 3? Hit me up in the comments — I read every one, and the troubleshooting stories are always the best part.
Sources:
- GitHub, k3s-io/k3s repository, retrieved 2026-05-31, https://github.com/k3s-io/k3s
- SUSE Communities Blog, "K3s: The World's Most Downloaded and Beloved Kubernetes Distribution," May 2024, retrieved 2026-05-31, https://www.suse.com/c/k3s-most-downloaded-kubernetes-distribution/
- Data Center Knowledge, "Home Depot Upgrades 2,300+ Retail Edge Locations Using SUSE Rancher, K3s," June 2022, retrieved 2026-05-31, https://www.datacenterknowledge.com/data-center-site-selection/home-depot-upgrades-2-300-retail-edge-locations-using-suse-rancher-k3s
- CNCF, "Annual Cloud Native Survey 2024," published April 1, 2025, retrieved 2026-05-31, https://www.cncf.io/announcements/2025/04/01/cncf-research-reveals-how-cloud-native-technology-is-reshaping-global-business-and-innovation/
- CNCF, "Annual Cloud Native Survey 2025," published January 20, 2026, retrieved 2026-05-31, https://www.cncf.io/announcements/2026/01/20/kubernetes-established-as-the-de-facto-operating-system-for-ai-as-production-use-hits-82-in-2025-cncf-annual-cloud-native-survey/
- Sidero Labs, "Why should a Kubernetes control plane be three nodes?" September 2022, retrieved 2026-05-31, https://www.siderolabs.com/blog/why-should-a-kubernetes-control-plane-be-three-nodes/
- kube-vip Documentation, "DaemonSet Installation," retrieved 2026-05-31, https://kube-vip.io/docs/installation/daemonset/
- MetalLB Documentation, "L2 Mode," retrieved 2026-05-31, https://metallb.universe.tf/
- Traefik Documentation, "IngressRoute CRD," retrieved 2026-05-31, https://doc.traefik.io/traefik/