# A Modern Terraform Reference Architecture for Amazon EKS Source: https://devopsdreams.blogspot.com/2026/03/a-modern-terraform-reference_0947887689.html Published: 2026-03-16T10:00:00.000-07:00 Updated: 2026-10-13T00:08:16.496-07:00 Author: Unknown Topics: aws, devops, eks, karpenter, platform-engineering, terraform Most EKS failures are not Kubernetes failures. They are boundary failures between [Terraform state](https://www.google.com/search?ved=1t:260882&q=define+Terraform+state&bbid=3735948330054827072&bpid=9215514641697070146), VPC capacity, node provisioning, and workload identity. This guide lays out a production-ready reference architecture that keeps those seams explicit. ## TL;DR [Section link](https://devopsdreams.blogspot.com/2026/03/a-modern-terraform-reference_0947887689.html#dd-section-1) A modern [Terraform reference architecture](https://www.google.com/search?ved=1t:260882&q=Terraform+reference+architecture+best+practices&bbid=3735948330054827072&bpid=9215514641697070146) for Amazon EKS should separate network, cluster, and add-on state; reserve [private subnet capacity](https://www.google.com/search?ved=1t:260882&q=AWS+private+subnet+capacity+requirements&bbid=3735948330054827072&bpid=9215514641697070146) for [control-plane ENIs](https://www.google.com/search?ved=1t:260882&q=Amazon+EKS+control-plane+ENIs+networking&bbid=3735948330054827072&bpid=9215514641697070146) and pods; keep a small stable baseline of managed nodes; use Karpenter for bursty or heterogeneous workloads; and choose workload identity deliberately instead of treating [IRSA](https://www.google.com/search?ved=1t:260882&q=define+IRSA+AWS&bbid=3735948330054827072&bpid=9215514641697070146) and EKS Pod Identity as interchangeable. The goal is not just to create a cluster, but to make upgrades, add-on lifecycle, IAM boundaries, and node replacement predictable. If you design those boundaries early, EKS gets much easier to operate. ![Generated diagram showing network, cluster, add-on, and compute boundaries in a modern Terraform reference architecture for Amazon EKS.](data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAxMjAwIDYzMCIgd2lkdGg9IjEyMDAiIGhlaWdodD0iNjMwIiByb2xlPSJpbWciIGFyaWEtbGFiZWxsZWRieT0idGl0bGUgZGVzYyIgc3R5bGU9ImRpc3BsYXk6YmxvY2s7d2lkdGg6MTAwJTtoZWlnaHQ6YXV0bzsiPgogIDx0aXRsZSBpZD0idGl0bGUiPk1vZGVybiBUZXJyYWZvcm0gcmVmZXJlbmNlIGFyY2hpdGVjdHVyZSBmb3IgQW1hem9uIEVLUzwvdGl0bGU+CiAgPGRlc2MgaWQ9ImRlc2MiPkEgZ2VuZXJhdGVkIGhlcm8gZ3JhcGhpYyBzaG93aW5nIHN0YXRlIGJvdW5kYXJpZXMsIFZQQywgRUtTIGNsdXN0ZXIsIG1hbmFnZWQgbm9kZSBncm91cHMsIEthcnBlbnRlciwgYW5kIGFkZC1vbnMgaW4gYSBtb2Rlcm4gRUtTIFRlcnJhZm9ybSBhcmNoaXRlY3R1cmUuPC9kZXNjPgogIDxkZWZzPgogICAgPGxpbmVhckdyYWRpZW50IGlkPSJiZyIgeDE9IjAiIHgyPSIxIiB5MT0iMCIgeTI9IjEiPjxzdG9wIG9mZnNldD0iMCUiIHN0b3AtY29sb3I9IiMxNzI1NTQiLz48c3RvcCBvZmZzZXQ9IjUwJSIgc3RvcC1jb2xvcj0iIzBmNzY2ZSIvPjxzdG9wIG9mZnNldD0iMTAwJSIgc3RvcC1jb2xvcj0iIzFlNDBhZiIvPjwvbGluZWFyR3JhZGllbnQ+CiAgICA8ZmlsdGVyIGlkPSJzaGFkb3ciIHg9Ii0yMCUiIHk9Ii0yMCUiIHdpZHRoPSIxNDAlIiBoZWlnaHQ9IjE0MCUiPjxmZURyb3BTaGFkb3cgZHg9IjAiIGR5PSIxNiIgc3RkRGV2aWF0aW9uPSIxOCIgZmxvb2QtY29sb3I9IiMwMjA2MTciIGZsb29kLW9wYWNpdHk9IjAuMjIiLz48L2ZpbHRlcj4KICA8L2RlZnM+CiAgPHJlY3Qgd2lkdGg9IjEyMDAiIGhlaWdodD0iNjMwIiByeD0iMjgiIGZpbGw9InVybCgjYmcpIi8+CiAgPHRleHQgeD0iNzQiIHk9IjkyIiBmaWxsPSIjYmZkYmZlIiBmb250LWZhbWlseT0iU2Vnb2UgVUksIEFyaWFsLCBzYW5zLXNlcmlmIiBmb250LXNpemU9IjIyIiBmb250LXdlaWdodD0iNzAwIiBsZXR0ZXItc3BhY2luZz0iNCI+QU1BWk9OIEVLUyBXSVRIIFRFUlJBRk9STTwvdGV4dD4KICA8dGV4dCB4PSI3NCIgeT0iMTUyIiBmaWxsPSIjZmZmZmZmIiBmb250LWZhbWlseT0iU2Vnb2UgVUksIEFyaWFsLCBzYW5zLXNlcmlmIiBmb250LXNpemU9IjUwIiBmb250LXdlaWdodD0iODAwIj5SZWZlcmVuY2UgQXJjaGl0ZWN0dXJlPC90ZXh0PgogIDx0ZXh0IHg9Ijc0IiB5PSIyMDYiIGZpbGw9IiNkYmVhZmUiIGZvbnQtZmFtaWx5PSJTZWdvZSBVSSwgQXJpYWwsIHNhbnMtc2VyaWYiIGZvbnQtc2l6ZT0iMjQiPlNlcGFyYXRlIHN0YXRlLCBwcmVzZXJ2ZSBzdWJuZXQgY2FwYWNpdHksIG1peCBzdGFibGUgbm9kZXMgd2l0aCBLYXJwZW50ZXIsIGFuZCBjaG9vc2Ugd29ya2xvYWQgaWRlbnRpdHkgZGVsaWJlcmF0ZWx5LjwvdGV4dD4KICA8ZyBmaWx0ZXI9InVybCgjc2hhZG93KSI+CiAgICA8cmVjdCB4PSI3MiIgeT0iMjc2IiB3aWR0aD0iMjM0IiBoZWlnaHQ9IjIzOCIgcng9IjI0IiBmaWxsPSIjZWZmNmZmIi8+CiAgICA8cmVjdCB4PSIzMzIiIHk9IjI3NiIgd2lkdGg9IjI1NCIgaGVpZ2h0PSIyMzgiIHJ4PSIyNCIgZmlsbD0iI2RiZWFmZSIvPgogICAgPHJlY3QgeD0iNjEyIiB5PSIyNzYiIHdpZHRoPSIyNDAiIGhlaWdodD0iMjM4IiByeD0iMjQiIGZpbGw9IiNkY2ZjZTciLz4KICAgIDxyZWN0IHg9Ijg3OCIgeT0iMjc2IiB3aWR0aD0iMjUwIiBoZWlnaHQ9IjIzOCIgcng9IjI0IiBmaWxsPSIjZmVmM2M3Ii8+CiAgPC9nPgogIDx0ZXh0IHg9IjEwMiIgeT0iMzI0IiBmaWxsPSIjMWUzYThhIiBmb250LWZhbWlseT0iU2Vnb2UgVUksIEFyaWFsLCBzYW5zLXNlcmlmIiBmb250LXNpemU9IjI4IiBmb250LXdlaWdodD0iODAwIj5uZXR3b3JrIHN0YXRlPC90ZXh0PgogIDx0ZXh0IHg9IjEwMiIgeT0iMzYwIiBmaWxsPSIjMWU0MGFmIiBmb250LWZhbWlseT0iU2Vnb2UgVUksIEFyaWFsLCBzYW5zLXNlcmlmIiBmb250LXNpemU9IjE5Ij5WUEMsIHN1Ym5ldHMsIHJvdXRlczwvdGV4dD4KICA8dGV4dCB4PSIxMDIiIHk9IjM5NCIgZmlsbD0iIzFlNDBhZiIgZm9udC1mYW1pbHk9IlNlZ29lIFVJLCBBcmlhbCwgc2Fucy1zZXJpZiIgZm9udC1zaXplPSIxOSI+c3BhcmUgSVBzIGZvciBFTklzPC90ZXh0PgogIDxyZWN0IHg9IjExOCIgeT0iNDMwIiB3aWR0aD0iMTQ0IiBoZWlnaHQ9IjUyIiByeD0iMTIiIGZpbGw9IiM5M2M1ZmQiLz4KICA8dGV4dCB4PSIxOTAiIHk9IjQ2MiIgdGV4dC1hbmNob3I9Im1pZGRsZSIgZmlsbD0iIzFlM2E4YSIgZm9udC1mYW1pbHk9IlNlZ29lIFVJLCBBcmlhbCwgc2Fucy1zZXJpZiIgZm9udC1zaXplPSIyMCIgZm9udC13ZWlnaHQ9IjgwMCI+cHJpdmF0ZSBzdWJuZXRzPC90ZXh0PgoKICA8dGV4dCB4PSIzNjIiIHk9IjMyNCIgZmlsbD0iIzFlM2E4YSIgZm9udC1mYW1pbHk9IlNlZ29lIFVJLCBBcmlhbCwgc2Fucy1zZXJpZiIgZm9udC1zaXplPSIyOCIgZm9udC13ZWlnaHQ9IjgwMCI+Y2x1c3RlciBzdGF0ZTwvdGV4dD4KICA8dGV4dCB4PSIzNjIiIHk9IjM2MCIgZmlsbD0iIzFlNDBhZiIgZm9udC1mYW1pbHk9IlNlZ29lIFVJLCBBcmlhbCwgc2Fucy1zZXJpZiIgZm9udC1zaXplPSIxOSI+RUtTIGNvbnRyb2wgcGxhbmU8L3RleHQ+CiAgPHRleHQgeD0iMzYyIiB5PSIzOTQiIGZpbGw9IiMxZTQwYWYiIGZvbnQtZmFtaWx5PSJTZWdvZSBVSSwgQXJpYWwsIHNhbnMtc2VyaWYiIGZvbnQtc2l6ZT0iMTkiPlBvZCBJZGVudGl0eSAvIElSU0E8L3RleHQ+CiAgPHJlY3QgeD0iMzk0IiB5PSI0MjYiIHdpZHRoPSIxMzIiIGhlaWdodD0iNjgiIHJ4PSIxNiIgZmlsbD0iIzFkNGVkOCIvPgogIDx0ZXh0IHg9IjQ2MCIgeT0iNDY4IiB0ZXh0LWFuY2hvcj0ibWlkZGxlIiBmaWxsPSIjZWZmNmZmIiBmb250LWZhbWlseT0iU2Vnb2UgVUksIEFyaWFsLCBzYW5zLXNlcmlmIiBmb250LXNpemU9IjIwIiBmb250LXdlaWdodD0iODAwIj5FS1M8L3RleHQ+CgogIDx0ZXh0IHg9IjY0MiIgeT0iMzI0IiBmaWxsPSIjMTY2NTM0IiBmb250LWZhbWlseT0iU2Vnb2UgVUksIEFyaWFsLCBzYW5zLXNlcmlmIiBmb250LXNpemU9IjI4IiBmb250LXdlaWdodD0iODAwIj5jb21wdXRlIGJvdW5kYXJ5PC90ZXh0PgogIDx0ZXh0IHg9IjY0MiIgeT0iMzYwIiBmaWxsPSIjMTY2NTM0IiBmb250LWZhbWlseT0iU2Vnb2UgVUksIEFyaWFsLCBzYW5zLXNlcmlmIiBmb250LXNpemU9IjE5Ij5tYW5hZ2VkIG5vZGVzIGZvciBiYXNlbGluZTwvdGV4dD4KICA8dGV4dCB4PSI2NDIiIHk9IjM5NCIgZmlsbD0iIzE2NjUzNCIgZm9udC1mYW1pbHk9IlNlZ29lIFVJLCBBcmlhbCwgc2Fucy1zZXJpZiIgZm9udC1zaXplPSIxOSI+S2FycGVudGVyIGZvciBidXJzdDwvdGV4dD4KICA8cmVjdCB4PSI2NDQiIHk9IjQzMCIgd2lkdGg9Ijk2IiBoZWlnaHQ9IjU0IiByeD0iMTIiIGZpbGw9IiMzNGQzOTkiLz4KICA8dGV4dCB4PSI2OTIiIHk9IjQ2MyIgdGV4dC1hbmNob3I9Im1pZGRsZSIgZmlsbD0iIzA2NGUzYiIgZm9udC1mYW1pbHk9IlNlZ29lIFVJLCBBcmlhbCwgc2Fucy1zZXJpZiIgZm9udC1zaXplPSIxOCIgZm9udC13ZWlnaHQ9IjgwMCI+TU5HPC90ZXh0PgogIDxyZWN0IHg9Ijc1NiIgeT0iNDMwIiB3aWR0aD0iNzYiIGhlaWdodD0iNTQiIHJ4PSIxMiIgZmlsbD0iIzZlZTdiNyIvPgogIDx0ZXh0IHg9Ijc5NCIgeT0iNDYzIiB0ZXh0LWFuY2hvcj0ibWlkZGxlIiBmaWxsPSIjMDY0ZTNiIiBmb250LWZhbWlseT0iU2Vnb2UgVUksIEFyaWFsLCBzYW5zLXNlcmlmIiBmb250LXNpemU9IjE3IiBmb250LXdlaWdodD0iODAwIj5LYXJwZW50ZXI8L3RleHQ+CgogIDx0ZXh0IHg9IjkwOCIgeT0iMzI0IiBmaWxsPSIjOTI0MDBlIiBmb250LWZhbWlseT0iU2Vnb2UgVUksIEFyaWFsLCBzYW5zLXNlcmlmIiBmb250LXNpemU9IjI4IiBmb250LXdlaWdodD0iODAwIj5hZGRvbnMgc3RhdGU8L3RleHQ+CiAgPHRleHQgeD0iOTA4IiB5PSIzNjAiIGZpbGw9IiM5MjQwMGUiIGZvbnQtZmFtaWx5PSJTZWdvZSBVSSwgQXJpYWwsIHNhbnMtc2VyaWYiIGZvbnQtc2l6ZT0iMTkiPlZQQyBDTkksIENvcmVETlMsIEVCUyBDU0k8L3RleHQ+CiAgPHRleHQgeD0iOTA4IiB5PSIzOTQiIGZpbGw9IiM5MjQwMGUiIGZvbnQtZmFtaWx5PSJTZWdvZSBVSSwgQXJpYWwsIHNhbnMtc2VyaWYiIGZvbnQtc2l6ZT0iMTkiPmRheS0yIHBsYXRmb3JtIHNlcnZpY2VzPC90ZXh0PgogIDxjaXJjbGUgY3g9IjEwMDYiIGN5PSI0NTIiIHI9IjQyIiBmaWxsPSIjZjU5ZTBiIi8+CiAgPHRleHQgeD0iMTAwNiIgeT0iNDYwIiB0ZXh0LWFuY2hvcj0ibWlkZGxlIiBmaWxsPSIjZmZmYmViIiBmb250LWZhbWlseT0iU2Vnb2UgVUksIEFyaWFsLCBzYW5zLXNlcmlmIiBmb250LXNpemU9IjE4IiBmb250LXdlaWdodD0iODAwIj5hZGQtb25zPC90ZXh0Pgo8L3N2Zz4=) A production EKS architecture works better when Terraform state, networking, compute, identity, and add-on ownership are treated as explicit seams. ## The Real Design Problem Is Not Creating EKS. It Is Owning the Seams. [Section link](https://devopsdreams.blogspot.com/2026/03/a-modern-terraform-reference_0947887689.html#dd-section-2) You can stand up Amazon EKS with Terraform in a weekend. The harder part is keeping the platform understandable six months later when you are upgrading Kubernetes, rotating AMIs, adding a storage driver, or trying to explain why a workload can reach S3 from one namespace but not another. A modern reference architecture has to make those boundaries explicit: - Terraform state boundaries - VPC and subnet capacity boundaries - compute ownership boundaries between managed node groups and Karpenter - identity boundaries between node roles, IRSA, and EKS Pod Identity - add-on ownership boundaries between Amazon EKS and your own day-2 tooling If those seams are vague, EKS feels random. If those seams are designed up front, the platform behaves like an engineered system. **Three AWS details should shape the design immediately:** - An EKS cluster must be created with at least two subnets in different Availability Zones, and EKS creates 2-4 control-plane ENIs in the subnets you provide. Those subnets need spare IP capacity, not just enough room for nodes. [AWS docs](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html) - Cluster subnets need at least six available IPs each, and AWS recommends at least sixteen. That recommendation matters during upgrades, when control-plane ENIs are replaced. [AWS docs](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html) - The S3 backend supports native lockfiles with `use_lockfile = true`, while DynamoDB locking is now documented as deprecated for that backend. Bucket versioning is strongly recommended for state recovery. [HashiCorp docs](https://developer.hashicorp.com/terraform/language/backend/s3) That is the baseline for the architecture below. ## 1. Start With State Boundaries, Not Cluster Resources [Section link](https://devopsdreams.blogspot.com/2026/03/a-modern-terraform-reference_0947887689.html#dd-section-3) Terraform state is part of the architecture. If you collapse VPCs, clusters, add-ons, and application extras into one state file, every routine change gets a wider blast radius than it needs. The cleaner split for EKS platforms is usually: | State | Owns | Why it should stay separate | | --- | --- | --- | | `network` | VPC, subnets, route tables, NAT, endpoints | Long-lived and shared by more than one cluster or stack | | `cluster` | EKS control plane, access config, encryption, baseline node groups | Cluster lifecycle changes are high-impact and deserve narrow plans | | `addons` | EKS managed add-ons, workload IAM roles, Pod Identity associations | Changes often, but should not re-plan the VPC or cluster role chain | | `platform-services` | ingress, external-dns, cert-manager, [GitOps](https://www.google.com/search?ved=1t:260882&q=what+is+GitOps&bbid=3735948330054827072&bpid=9215514641697070146) bootstrap | Often shifts toward [Helm](https://www.google.com/search?ved=1t:260882&q=what+is+Helm+Kubernetes&bbid=3735948330054827072&bpid=9215514641697070146) or GitOps ownership later | Use S3 remote state with versioning and lockfiles. If you operate in multiple AWS accounts, use an administrative execution role pattern and keep state keys environment-specific rather than stuffing everything into workspaces and hoping naming discipline will save you. S3 `use_lockfile` requires Terraform 1.10 or later. Configure lock-file permissions and bucket versioning separately, and validate the complete configuration with your selected Terraform release. [S3 backend requirements](https://developer.hashicorp.com/terraform/language/backend/s3). ```hcl terraform { required_version = ">= 1.10.0" backend "s3" { bucket = "company-terraform-state" key = "prod/eu-west-1/eks-platform/cluster.tfstate" region = "eu-west-1" use_lockfile = true } required_providers { aws = { source = "hashicorp/aws" version = "~> 6.36" } } } provider "aws" { region = var.region assume_role { role_arn = var.terraform_execution_role_arn } default_tags { tags = { Environment = var.environment Platform = "eks" ManagedBy = "terraform" } } } ``` Why this holds up better: - `network` changes stay rare and deliberate. - `cluster` plans stay readable during Kubernetes upgrades. - `addons` can move faster without re-planning the whole platform. - `platform-services` can later hand off to GitOps without a state migration crisis. This is one of the biggest differences between a demo and a reference architecture. ## 2. Design the VPC for Control-Plane ENIs and Pod Density Before You Design Nodes [Section link](https://devopsdreams.blogspot.com/2026/03/a-modern-terraform-reference_0947887689.html#dd-section-4) Most EKS networking mistakes come from thinking only about worker nodes. AWS is explicit that the cluster subnets are also where the control plane creates ENIs, and those ENIs are replaced during cluster version updates. That is why tiny, overused subnets create upgrade pain before you ever hit an application outage. [AWS docs](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html) The baseline I would use for a production EKS VPC is: - private subnets for nodes and control-plane ENIs - tightly scoped public endpoint CIDRs, or private-only API access if your operator path supports it - dedicated endpoint strategy for private clusters - conscious pod-density tuning on the [VPC CNI](https://www.google.com/search?ved=1t:260882&q=Amazon+EKS+VPC+CNI+configuration&bbid=3735948330054827072&bpid=9215514641697070146) instead of accepting default IPv4 exhaustion behavior For EC2-backed clusters, Amazon EKS add-ons are the clean default for networking components. AWS now recommends EKS add-ons over self-managed versions because updates and centralized configuration are simpler through AWS APIs. [AWS docs](https://docs.aws.amazon.com/eks/latest/userguide/eks-networking-add-ons.html) ```hcl resource "aws_eks_cluster" "this" { name = var.cluster_name role_arn = aws_iam_role.eks_cluster.arn version = var.kubernetes_version access_config { authentication_mode = "API_AND_CONFIG_MAP" } enabled_cluster_log_types = [ "api", "audit", "authenticator", "controllerManager", "scheduler", ] encryption_config { resources = ["secrets"] provider { key_arn = aws_kms_key.eks_secrets.arn } } kubernetes_network_config { service_ipv4_cidr = "172.20.0.0/16" } vpc_config { subnet_ids = var.cluster_subnet_ids endpoint_private_access = true endpoint_public_access = true public_access_cidrs = var.admin_cidrs } } ``` A few practical rules matter more than they look: - If you put managed node groups in public subnets, `MapPublicIpOnLaunch` must be enabled or nodes will fail to join. [AWS docs](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html) - If you keep nodes in private subnets, they still need image and API reachability. That usually means NAT plus selected endpoints, or a more complete PrivateLink design. [AWS docs](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html) - If you are tight on IPv4, enable prefix delegation on the VPC CNI and design for contiguous `/28` availability in the subnet. AWS notes that prefix attachment is typically faster than attaching a new ENI and is the main lever for higher pod density on EC2 nodes. [AWS docs](https://docs.aws.amazon.com/eks/latest/best-practices/prefix-mode-linux.html) The trap is easy to spot in hindsight: teams scale nodes to fix pending Pods when the real bottleneck is subnet address economics. ## 3. Keep a Stable Baseline With Managed Node Groups, Then Add Karpenter Where It Actually Helps [Section link](https://devopsdreams.blogspot.com/2026/03/a-modern-terraform-reference_0947887689.html#dd-section-5) The cleanest modern EKS compute pattern is not "managed node groups or Karpenter." It is "managed node groups first, Karpenter where the workload shape justifies it." Managed node groups are still the right default for: - system-critical baseline capacity - ingress and DNS paths you do not want tied to burst autoscaling - clusters that need predictable rolling update semantics - launch-template-heavy customization AWS-managed node groups automatically drain nodes during updates and terminations, support node auto repair, and let you keep the EC2 mechanics inside your AWS account without manually wiring ASGs. [AWS docs](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html) ```hcl resource "aws_eks_node_group" "system" { cluster_name = aws_eks_cluster.this.name node_group_name = "${var.cluster_name}-system" node_role_arn = aws_iam_role.node.arn subnet_ids = var.private_node_subnet_ids capacity_type = "ON_DEMAND" instance_types = ["m7i.large"] ami_type = "AL2023_x86_64_STANDARD" scaling_config { min_size = 2 desired_size = 3 max_size = 6 } update_config { max_unavailable_percentage = 25 } node_repair_config { enabled = true } labels = { "node-role" = "system" } taint { key = "CriticalAddonsOnly" value = "true" effect = "NO_SCHEDULE" } } ``` Karpenter is strongest when capacity is variable, instance diversity matters, or Pod scheduling constraints are too awkward to model with a pile of node groups. AWS says it best: Karpenter fits workloads with changing capacity needs, while managed node groups and ASGs remain a good fit for more static, consistent workloads. [AWS docs](https://docs.aws.amazon.com/eks/latest/best-practices/karpenter.html) That architecture choice has two important implications: - keep at least one small non-Karpenter baseline pool so Karpenter itself is not responsible for hosting its own controller - let Karpenter handle bursty, specialized, or Spot-heavy workloads rather than replacing every node group on day one Karpenter’s modern APIs also change how you think about compute. `NodePool` defines the scheduling envelope and disruption policy; `EC2NodeClass` defines AWS launch details such as subnet selectors, security-group selectors, role, and AMI selection. [Karpenter nodeclass](https://karpenter.sh/docs/concepts/nodeclasses/) [Karpenter docs](https://karpenter.sh/docs/concepts/nodepools/) Check NodePool behavior against your installed Karpenter release. The linked current documentation does not establish compatibility with every earlier controller version. ```yaml apiVersion: karpenter.k8s.aws/v1 kind: EC2NodeClass metadata: name: apps spec: role: eks-node-role-prod subnetSelectorTerms: - tags: karpenter.sh/discovery: prod-eks tier: private securityGroupSelectorTerms: - tags: karpenter.sh/discovery: prod-eks amiSelectorTerms: - alias: al2023@v20240807 --- apiVersion: karpenter.sh/v1 kind: NodePool metadata: name: apps spec: template: spec: nodeClassRef: group: karpenter.k8s.aws kind: EC2NodeClass name: apps expireAfter: 336h requirements: - key: kubernetes.io/arch operator: In values: ["amd64"] - key: karpenter.sh/capacity-type operator: In values: ["spot", "on-demand"] - key: karpenter.k8s.aws/instance-category operator: In values: ["c", "m", "r"] disruption: consolidationPolicy: WhenEmptyOrUnderutilized consolidateAfter: 5m budgets: - nodes: "10%" ``` Two trade-offs here are easy to miss: - Pin the AMI in production. Karpenter explicitly warns against `@latest` in production because a new AMI can drift nodes into an untested image. [Karpenter docs](https://karpenter.sh/docs/tasks/managing-amis/) - Treat `expireAfter` as an upper bound, not a promise. Karpenter can still replace nodes earlier due to consolidation or drift if budgets allow it. [Karpenter docs](https://karpenter.sh/docs/concepts/nodepools/) [Karpenter docs](https://karpenter.sh/v1.0/concepts/disruption/) That is why managed baseline plus Karpenter burst capacity is such a durable design: you get fast Pod-driven scaling without making every system component depend on Karpenter’s disruption logic. ## 4. EKS Pod Identity and IRSA Solve Similar Problems, but They Change Different Parts of the System [Section link](https://devopsdreams.blogspot.com/2026/03/a-modern-terraform-reference_0947887689.html#dd-section-6) Teams often talk about IRSA and EKS Pod Identity as if one is just a newer spelling of the other. They are not the same operational model. **IRSA** - requires a cluster-specific OIDC provider - uses `AssumeRoleWithWebIdentity` - stays useful where OIDC-based service account federation is already part of the pattern - creates extra setup friction in private environments because the cluster OIDC issuer is outside the VPC endpoint path **EKS Pod Identity** - does not require an IAM OIDC provider - uses a simpler trust principal, `pods.eks.amazonaws.com` - relies on the EKS Pod Identity Agent running on nodes - is easier to reuse across clusters because the role trust is not tied to a cluster-specific OIDC issuer AWS explicitly recommends Pod Identity as the preferred way to grant IAM permissions to EKS add-ons and positions it as the simpler method for workload credentials on supported EC2-backed EKS clusters. [AWS add-ons-iam](https://docs.aws.amazon.com/eks/latest/userguide/add-ons-iam.html) [AWS docs](https://docs.aws.amazon.com/eks/latest/userguide/pod-identities.html) The catch is that support boundaries still matter: - Pod Identity requires the Pod Identity Agent. [AWS docs](https://docs.aws.amazon.com/eks/latest/userguide/pod-id-agent-setup.html) - The agent needs node-role permission for `eks-auth:AssumeRoleForPodIdentity`. [AWS docs](https://docs.aws.amazon.com/eks/latest/userguide/pod-id-agent-setup.html) - In private clusters, nodes need the `eks-auth` PrivateLink endpoint for Pod Identity to work. [AWS docs](https://docs.aws.amazon.com/eks/latest/userguide/pod-id-agent-setup.html) - Pod Identity is not available for all compute combinations, including Fargate and Windows pods. [AWS docs](https://docs.aws.amazon.com/eks/latest/userguide/pod-identities.html) That means the reference architecture decision is usually: - default to Pod Identity for new EC2-backed workloads and supported add-ons - keep IRSA for controllers or environments where Pod Identity is not the right fit yet ```hcl data "aws_iam_policy_document" "pod_identity_trust" { statement { effect = "Allow" principals { type = "Service" identifiers = ["pods.eks.amazonaws.com"] } actions = [ "sts:AssumeRole", "sts:TagSession", ] } } resource "aws_iam_role" "external_dns" { name = "${var.cluster_name}-external-dns" assume_role_policy = data.aws_iam_policy_document.pod_identity_trust.json } resource "aws_eks_pod_identity_association" "external_dns" { cluster_name = aws_eks_cluster.this.name namespace = "dns" service_account = "external-dns" role_arn = aws_iam_role.external_dns.arn } ``` The most important architectural payoff is not fewer lines of IAM. It is cleaner separation of responsibilities between the platform team managing EKS associations and the IAM team managing reusable role trust and policies. ## 5. Prefer Managed Add-ons for the Baseline, and Make IAM Explicit for the Ones That Need It [Section link](https://devopsdreams.blogspot.com/2026/03/a-modern-terraform-reference_0947887689.html#dd-section-7) If your cluster baseline still installs core components as random Helm releases because "that is how we always did it," you are leaving operational clarity on the table. AWS recommends EKS add-ons over self-managed networking add-ons, and the same general logic applies to baseline platform components that AWS actively manages through the EKS add-on API. [AWS docs](https://docs.aws.amazon.com/eks/latest/userguide/eks-networking-add-ons.html) For a conventional EC2-backed baseline, I would usually manage these in Terraform: - `vpc-cni` - `coredns` - `kube-proxy` - `eks-pod-identity-agent` - `aws-ebs-csi-driver` Whether every one of those belongs in your exact baseline depends on your storage and compute model, but the general principle is stable: if the add-on is foundational, versioned by AWS, and tied to cluster compatibility, treat it as infrastructure. AWS also notes that some add-ons need IAM permissions, and add-ons can now manage Pod Identity associations through the add-on APIs themselves. If both IRSA and Pod Identity settings are supplied and the Pod Identity Agent is installed, EKS prefers Pod Identity and ignores the IRSA role for that add-on. [AWS docs](https://docs.aws.amazon.com/eks/latest/userguide/add-ons-iam.html) ```hcl resource "aws_eks_addon" "pod_identity_agent" { cluster_name = aws_eks_cluster.this.name addon_name = "eks-pod-identity-agent" } resource "aws_eks_addon" "vpc_cni" { cluster_name = aws_eks_cluster.this.name addon_name = "vpc-cni" resolve_conflicts_on_create = "OVERWRITE" resolve_conflicts_on_update = "PRESERVE" pod_identity_association { service_account = "aws-node" role_arn = aws_iam_role.vpc_cni.arn } depends_on = [aws_eks_addon.pod_identity_agent] } resource "aws_eks_addon" "coredns" { cluster_name = aws_eks_cluster.this.name addon_name = "coredns" resolve_conflicts_on_create = "OVERWRITE" resolve_conflicts_on_update = "PRESERVE" } ``` This is the operational trade-off: - managed add-ons reduce upgrade ambiguity and centralize compatibility - self-managed installs can still be justified when you need a configuration surface or release cadence that AWS does not expose yet That is a real trade-off, not a purity test. But for the reference architecture, managed first is the cleaner default. ## Why This Architecture Holds Up Better in Production [Section link](https://devopsdreams.blogspot.com/2026/03/a-modern-terraform-reference_0947887689.html#dd-section-8) The point of a reference architecture is not that it is the only correct design. The point is that it remains legible under stress. This one does because each major concern has a clear owner: | Concern | Default owner | Why | | --- | --- | --- | | VPC shape, subnet math, endpoints | Terraform `network` state | Slow-changing and shared | | Cluster version, access mode, encryption, logging | Terraform `cluster` state | Needs explicit plans and review | | Stable system capacity | managed node groups | Predictable lifecycle and repair | | Bursty or mixed instance capacity | Karpenter | Pod-driven placement and diversity | | Workload AWS credentials | Pod Identity first, IRSA where needed | Least privilege without overloading node roles | | Baseline compatibility add-ons | Terraform `addons` state | Tied to cluster lifecycle, not app deploys | That separation is what keeps common EKS incidents boring: - cluster upgrades do not become subnet surprises - add-on updates do not become IAM archaeology - AMI rotation does not become hidden drift - scaling does not collapse into "just add another node group" The architecture is modern because it respects what the platform has become: EKS is no longer only a Kubernetes API endpoint. It is a set of interacting control planes for networking, identity, storage, and compute, and Terraform is useful only when it makes those interactions easier to reason about. ## Frequently Asked Questions [Section link](https://devopsdreams.blogspot.com/2026/03/a-modern-terraform-reference_0947887689.html#dd-section-9) **Q: What is the smallest production-safe compute pattern for EKS?** A: A small On-Demand managed node group for system-critical components plus Karpenter for bursty application capacity is a strong default. It gives you a stable place for baseline add-ons and a safe rollback path if Karpenter policy needs adjustment. **Q: Should I use only private endpoints for the Kubernetes API?** A: Use private-only access if your operator path, CI runners, and emergency access model are already inside the VPC or connected network. If not, keep private access enabled and restrict public access to known admin CIDRs until you can remove the public endpoint safely. **Q: Is EKS Pod Identity a full replacement for IRSA?** A: Not universally. Pod Identity is simpler for many EC2-backed workloads and is the AWS-recommended path for add-on IAM, but IRSA still matters for support gaps and existing OIDC-based patterns. The right choice depends on workload support and network constraints, not on fashion. **Q: When is Karpenter the wrong first move?** A: Karpenter is the wrong first move when your real problem is weak subnet design, poor resource requests, or add-on sprawl. It improves node provisioning, but it does not rescue a cluster with bad IP economics, unclear ownership, or missing workload constraints. **Q: What should I review before every EKS minor upgrade?** A: Review subnet headroom, add-on compatibility, node AMI strategy, Pod Identity or IRSA dependencies, and disruption policies for both managed node groups and Karpenter. Most upgrade pain comes from those dependencies, not from the version bump itself. ## Resources [Section link](https://devopsdreams.blogspot.com/2026/03/a-modern-terraform-reference_0947887689.html#dd-section-10) - [Primary documentation](https://github.com/hashicorp/terraform/releases/tag/v1.10.0) - [Terraform S3 backend](https://developer.hashicorp.com/terraform/language/backend/s3) - [Terraform remote state backends](https://developer.hashicorp.com/terraform/language/state/backends) - [Amazon EKS networking requirements](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html) - [Amazon EKS managed node groups](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html) - [Amazon EKS add-ons](https://docs.aws.amazon.com/eks/latest/userguide/eks-add-ons.html) - [IAM roles for Amazon EKS add-ons](https://docs.aws.amazon.com/eks/latest/userguide/add-ons-iam.html) - [EKS Pod Identity](https://docs.aws.amazon.com/eks/latest/userguide/pod-identities.html) - [IAM roles for service accounts](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) - [Amazon EKS Pod Identity Agent](https://docs.aws.amazon.com/eks/latest/userguide/pod-id-agent-setup.html) - [Amazon EKS prefix mode for Linux](https://docs.aws.amazon.com/eks/latest/best-practices/prefix-mode-linux.html) - [Amazon EKS best practices for Karpenter](https://docs.aws.amazon.com/eks/latest/best-practices/karpenter.html) - [Karpenter NodePools](https://karpenter.sh/docs/concepts/nodepools/) - [Karpenter EC2NodeClasses](https://karpenter.sh/docs/concepts/nodeclasses/) - [Karpenter Managing AMIs](https://karpenter.sh/docs/tasks/managing-amis/)