[![Discord](https://img.shields.io/discord/673534664354430999?label&logo=discord&logoColor=white&color=blue)](https://discord.gg/home-operations)   [![Talos](https://kromgo.nasenov.dev/badges/talos_version)](https://talos.dev)   [![Kubernetes](https://kromgo.nasenov.dev/badges/kubernetes_version)](https://kubernetes.io)   [![Flux](https://kromgo.nasenov.dev/badges/flux_version)](https://fluxcd.io)   [![Renovate](https://img.shields.io/github/actions/workflow/status/nasenov/homelab/renovate.yaml?branch=main&label&logo=renovate&color=blue)](https://github.com/nasenov/homelab/actions/workflows/renovate.yaml)
[![Age](https://kromgo.nasenov.dev/badges/cluster_birth_age)](https://github.com/home-operations/kromgo)   [![Uptime](https://kromgo.nasenov.dev/badges/cluster_uptime_age)](https://github.com/home-operations/kromgo)   [![Nodes](https://kromgo.nasenov.dev/badges/cluster_node_count)](https://github.com/home-operations/kromgo)   [![Pods](https://kromgo.nasenov.dev/badges/cluster_pod_count)](https://github.com/home-operations/kromgo)   [![CPU](https://kromgo.nasenov.dev/badges/cluster_cpu_usage)](https://github.com/home-operations/kromgo)   [![Memory](https://kromgo.nasenov.dev/badges/cluster_memory_usage)](https://github.com/home-operations/kromgo)   [![Power](https://kromgo.nasenov.dev/badges/cluster_power_usage)](https://github.com/home-operations/kromgo)   [![Alerts](https://kromgo.nasenov.dev/badges/cluster_alert_count)](https://github.com/home-operations/kromgo)
--- ## 💡 Overview Welcome to my home infrastructure and Kubernetes cluster repository! This project embraces Infrastructure as Code (IaC) and GitOps principles, leveraging [Kubernetes](https://github.com/kubernetes/kubernetes), [Flux](https://github.com/fluxcd/flux2), [OpenTofu](https://opentofu.org), [Renovate](https://github.com/renovatebot/renovate), and [GitHub Actions](https://github.com/features/actions) to maintain a fully automated, declarative homelab environment. --- ## 🌱 Kubernetes My semi-hyperconverged cluster runs [Talos Linux](https://github.com/siderolabs/talos)-an immutable, minimal Linux distribution purpose-built for Kubernetes-on three bare-metal machines. Storage is handled by [Rook](https://github.com/rook/rook), providing persistent block, object, and file storage directly within the cluster, complemented by a dedicated NAS for media files. The entire cluster is architected for complete reproducibility: I can tear it down and rebuild from scratch without losing any data. ### Core Components - [cert-manager](https://github.com/cert-manager/cert-manager): Creates TLS certificates for services in my cluster. - [cilium](https://github.com/cilium/cilium): High-performance container networking powered by [eBPF](https://ebpf.io). - [cloudflared](https://github.com/cloudflare/cloudflared): Secure tunnel providing Cloudflare-protected access to cluster services. - [envoy-gateway](https://github.com/envoyproxy/gateway): Modern ingress controller for cluster traffic management. - [external-dns](https://github.com/kubernetes-sigs/external-dns): Automated DNS record synchronization for ingress resources. - [external-secrets](https://github.com/external-secrets/external-secrets): Kubernetes secrets management integrated with [Bitwarden](https://bitwarden.com/). - [rook](https://github.com/rook/rook): Cloud-native distributed storage orchestrator for persistent storage. - [spegel](https://github.com/spegel-org/spegel): Stateless cluster local OCI registry mirror. - [volsync](https://github.com/backube/volsync): Backup and recovery of persistent volume claims. ### GitOps [Flux](https://github.com/fluxcd/flux2) continuously monitors the [kubernetes](./kubernetes) folder and reconciles my cluster state with whatever is defined in this Git repository-Git is the single source of truth. Here's how it works: Flux recursively scans the [kubernetes/apps](./kubernetes/apps) directory, discovering the top-level `kustomization.yaml` in each subdirectory. These files typically define a namespace and one or more Flux `Kustomization` resources (`ks.yaml`). Each Flux `Kustomization` then manages a `HelmRelease` or other Kubernetes resources for that application. Meanwhile, [Renovate](https://github.com/renovatebot/renovate) continuously scans the **entire** repository for dependency updates, automatically opening pull requests when new versions are available. Once merged, Flux picks up the changes and updates the cluster automatically. ### Directories This Git repository contains the following directories under [Kubernetes](./kubernetes/). ```sh 📁 kubernetes ├── 📁 apps # applications ├── 📁 bootstrap # bootstrap configuration ├── 📁 components # re-useable kustomize components ``` ### Cluster layout Here's how Flux orchestrates application deployments with dependencies. Most applications are deployed as `HelmRelease` resources that depend on other `HelmRelease`'s, while some `Kustomization`'s depend on other `Kustomization`'s. Occasionally, an application may have dependencies on both types. The diagram below illustrates this: `actual` won't deploy or upgrade until `rook-ceph-cluster` is successfully installed and healthy.
Click to see a high-level architecture diagram ```mermaid graph LR classDef kustom fill:#43A047,stroke:#2E7D32,stroke-width:3px,color:#fff,font-weight:bold,rx:10,ry:10 classDef helm fill:#1976D2,stroke:#0D47A1,stroke-width:3px,color:#fff,font-weight:bold,rx:10,ry:10 A["📦 Kustomization
rook-ceph"]:::kustom B["📦 Kustomization
rook-ceph-cluster"]:::kustom C["🎯 HelmRelease
rook-ceph"]:::helm D["🎯 HelmRelease
rook-ceph-cluster"]:::helm E["📦 Kustomization
actual"]:::kustom F["🎯 HelmRelease
actual"]:::helm A -->|Creates| C B -->|Creates| D B -.->|Depends on| A E -->|Creates| F E -.->|Depends on| B ```
--- ## 😶 Cloud Dependencies While most of my infrastructure and workloads are self-hosted I do rely upon the cloud for certain key parts of my setup. | Service | Use | Cost | | ----------------------------------------- | ------------------------------------------------------------- | ------------- | | [Bitwarden](https://bitwarden.com/) | Secrets with [External Secrets](https://external-secrets.io/) | Free | | [Cloudflare](https://cloudflare.com/) | Domain, S3 and Tunnel | ~$12/yr | | [Discord](https://discord.com/) | Alerts and Notifications | Free | | [Github](https://github.com/) | Github Actions | Free | | [Let's Encrypt](https://letsencrypt.org/) | TLS certificates | Free | | | | Total: ~$1/mo | --- ## 🌎 DNS I run two instances of [ExternalDNS](https://github.com/kubernetes-sigs/external-dns) to handle DNS automation: - **Private DNS**: Syncs records to my `UniFi Cloud Gateway Ultra` - **Public DNS**: Syncs records to `Cloudflare` This is achieved by defining routes with two specific gateways: `internal` for private DNS and `external` for public DNS. Each ExternalDNS instance watches for routes using its assigned gateway and syncs the appropriate DNS records to the corresponding platform. --- ## ⚙ Hardware | Device | Num | OS Disk Size | Data Disk Size | RAM | OS | Function | | ------------------------- | --- | ------------ | ---------------------------------------- | ---- | ------- | ---------- | | Lenovo m90q i5-10500T | 3 | 256GB NVMe | 500GB SSD (local) / 1TB NVMe (rook-ceph) | 32GB | Talos | Kubernetes | | Aoostar WTR PRO 5825U | 1 | 512GB NVMe | 2x2TB SSD | 16GB | TrueNAS | NAS | | UniFi Cloud Gateway Ultra | 1 | - | - | - | - | Router | | UniFi Switch Lite 8 PoE | 1 | - | - | - | - | Switch | | UniFi U7 Lite | 1 | - | - | - | - | AP |