--- AWSTemplateFormatVersion: '2010-09-09' Description: 'Amazon EKS - BottleRocket SPOT EC2 Instance' Parameters: KeyName: Description: The EC2 Key Pair to allow SSH access to the instances Type: AWS::EC2::KeyPair::KeyName NodeImageId: Type: String Default: "" Description: (Optional) Specify your own custom image ID. This value overrides any AWS Systems Manager Parameter Store value specified above. NodeImageIdSSMParam: Type: "AWS::SSM::Parameter::Value" Default: /aws/service/bottlerocket/aws-k8s-1.18/x86_64/latest/image_id Description: AWS Systems Manager Parameter Store parameter of the AMI ID for the worker node instances. NodeInstanceType: Description: EC2 instance type for the node instances Type: String Default: t3.medium ConstraintDescription: Must be a valid EC2 instance type AllowedValues: - a1.medium - a1.large - a1.xlarge - a1.2xlarge - a1.4xlarge - c1.medium - c1.xlarge - c3.large - c3.xlarge - c3.2xlarge - c3.4xlarge - c3.8xlarge - c4.large - c4.xlarge - c4.2xlarge - c4.4xlarge - c4.8xlarge - c5.large - c5.xlarge - c5.2xlarge - c5.4xlarge - c5.9xlarge - c5.12xlarge - c5.18xlarge - c5.24xlarge - c5.metal - c5d.large - c5d.xlarge - c5d.2xlarge - c5d.4xlarge - c5d.9xlarge - c5d.18xlarge - c5n.large - c5n.xlarge - c5n.2xlarge - c5n.4xlarge - c5n.9xlarge - c5n.18xlarge - c6g.large - c6g.xlarge - c6g.2xlarge - c6g.4xlarge - c6g.9xlarge - c6g.18xlarge - c6g.large - c6g.xlarge - c6g.2xlarge - c6g.4xlarge - c6g.9xlarge - c6g.18xlarge - c6g.8xlarge - c6g.8xlarge - d2.xlarge - d2.2xlarge - d2.4xlarge - d2.8xlarge - f1.2xlarge - f1.4xlarge - f1.16xlarge - g2.2xlarge - g2.8xlarge - g3s.xlarge - g3.4xlarge - g3.8xlarge - g3.16xlarge - h1.2xlarge - h1.4xlarge - h1.8xlarge - h1.16xlarge - hs1.8xlarge - i2.xlarge - i2.2xlarge - i2.4xlarge - i2.8xlarge - i3.large - i3.xlarge - i3.2xlarge - i3.4xlarge - i3.8xlarge - i3.16xlarge - i3.metal - i3en.large - i3en.xlarge - i3en.2xlarge - i3en.3xlarge - i3en.6xlarge - i3en.12xlarge - i3en.24xlarge - m1.small - m1.medium - m1.large - m1.xlarge - m2.xlarge - m2.2xlarge - m2.4xlarge - m3.medium - m3.large - m3.xlarge - m3.2xlarge - m4.large - m4.xlarge - m4.2xlarge - m4.4xlarge - m4.10xlarge - m4.16xlarge - m5.large - m5.xlarge - m5.2xlarge - m5.4xlarge - m5.8xlarge - m5.12xlarge - m5.16xlarge - m5.24xlarge - m5.metal - m5a.large - m5a.xlarge - m5a.2xlarge - m5a.4xlarge - m5a.8xlarge - m5a.12xlarge - m5a.16xlarge - m5a.24xlarge - m5ad.large - m5ad.xlarge - m5ad.2xlarge - m5ad.4xlarge - m5ad.12xlarge - m5ad.24xlarge - m5d.large - m5d.xlarge - m5d.2xlarge - m5d.4xlarge - m5d.8xlarge - m5d.12xlarge - m5d.16xlarge - m5d.24xlarge - m5d.metal - p2.xlarge - p2.8xlarge - p2.16xlarge - p3.2xlarge - p3.8xlarge - p3.16xlarge - p3dn.24xlarge - g4dn.xlarge - g4dn.2xlarge - g4dn.4xlarge - g4dn.8xlarge - g4dn.12xlarge - g4dn.16xlarge - g4dn.metal - r3.large - r3.xlarge - r3.2xlarge - r3.4xlarge - r3.8xlarge - r4.large - r4.xlarge - r4.2xlarge - r4.4xlarge - r4.8xlarge - r4.16xlarge - r5.large - r5.xlarge - r5.2xlarge - r5.4xlarge - r5.8xlarge - r5.12xlarge - r5.16xlarge - r5.24xlarge - r5.metal - r5a.large - r5a.xlarge - r5a.2xlarge - r5a.4xlarge - r5a.8xlarge - r5a.12xlarge - r5a.16xlarge - r5a.24xlarge - r5ad.large - r5ad.xlarge - r5ad.2xlarge - r5ad.4xlarge - r5ad.12xlarge - r5ad.24xlarge - r5d.large - r5d.xlarge - r5d.2xlarge - r5d.4xlarge - r5d.8xlarge - r5d.12xlarge - r5d.16xlarge - r5d.24xlarge - r5d.metal - t1.micro - t2.nano - t2.micro - t2.small - t2.medium - t2.large - t2.xlarge - t2.2xlarge - t3.nano - t3.micro - t3.small - t3.medium - t3.large - t3.xlarge - t3.2xlarge - t3a.nano - t3a.micro - t3a.small - t3a.medium - t3a.large - t3a.xlarge - t3a.2xlarge - u-6tb1.metal - u-9tb1.metal - u-12tb1.metal - x1.16xlarge - x1.32xlarge - x1e.xlarge - x1e.2xlarge - x1e.4xlarge - x1e.8xlarge - x1e.16xlarge - x1e.32xlarge - z1d.large - z1d.xlarge - z1d.2xlarge - z1d.3xlarge - z1d.6xlarge - z1d.12xlarge - z1d.metal NodeAutoScalingGroupMaxSize: Type: Number Description: Maximum size of Node Group ASG. Default: 3 NodeVolumeSize: Type: Number Default: 20 Description: Node volume size (in GB) NodeDesiredSize: Type: Number Description: Desired Capacity of the Node Group ASG Default: 2 ClusterName: Description: The cluster name provided when the cluster was created. If it is incorrect, nodes will not be able to join the cluster. Type: String NodeGroupName: Description: Unique identifier for the Node Group. Type: String ClusterName: Description: The cluster name provided when the cluster was created. If it is incorrect, nodes will not be able to join the cluster. Type: String APIServer: Description: Refer to https://console.aws.amazon.com/eks/home Type: String ClusterCertificate: Description: Refer to https://console.aws.amazon.com/eks/home Type: String NodeLabel: Description: Addition node labels (key = "Value") Type: String Default: 'bottlerocket = "1.0.2"' ClusterControlPlaneSecurityGroup: Description: The security group of the cluster control plane. Type: AWS::EC2::SecurityGroup::Id WorkerNodeSecurityGroup: Description: The additional security group of the workernodes. Type: AWS::EC2::SecurityGroup::Id VpcId: Description: The VPC of the worker instances Type: AWS::EC2::VPC::Id Subnets: Description: The subnets where workers can be created. Type: List SpotPrice: Description: The spot price for this Auto Scaling group Type: String Conditions: HasNodeImageId: !Not - "Fn::Equals": - Ref: NodeImageId - "" Metadata: AWS::CloudFormation::Interface: ParameterGroups: - Label: default: "EKS Cluster" Parameters: - ClusterName - ClusterControlPlaneSecurityGroup - APIServer - ClusterCertificate - NodeLabel - Label: default: "Worker Node Configuration" Parameters: - NodeGroupName - NodeAutoScalingGroupMaxSize - NodeDesiredSize - NodeInstanceType - SpotPrice - NodeImageId - NodeVolumeSize - KeyName - WorkerNodeSecurityGroup - Label: default: "Worker Network Configuration" Parameters: - VpcId - Subnets Resources: NodeInstanceProfile: Type: AWS::IAM::InstanceProfile Properties: Path: "/" Roles: - !Ref NodeInstanceRole NodeInstanceRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: - ec2.amazonaws.com Action: - sts:AssumeRole Path: "/" ManagedPolicyArns: - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore NodeSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Security group for all nodes in the cluster VpcId: !Ref VpcId Tags: - Key: !Sub "kubernetes.io/cluster/${ClusterName}" Value: 'owned' NodeSecurityGroupIngress: Type: AWS::EC2::SecurityGroupIngress DependsOn: NodeSecurityGroup Properties: Description: Allow node to communicate with each other GroupId: !Ref NodeSecurityGroup SourceSecurityGroupId: !Ref NodeSecurityGroup IpProtocol: '-1' FromPort: 0 ToPort: 65535 NodeSecurityGroupFromControlPlaneIngress: Type: AWS::EC2::SecurityGroupIngress DependsOn: NodeSecurityGroup Properties: Description: Allow worker Kubelets and pods to receive communication from the cluster control plane GroupId: !Ref NodeSecurityGroup SourceSecurityGroupId: !Ref ClusterControlPlaneSecurityGroup IpProtocol: tcp FromPort: 1025 ToPort: 65535 ControlPlaneEgressToNodeSecurityGroup: Type: AWS::EC2::SecurityGroupEgress DependsOn: NodeSecurityGroup Properties: Description: Allow the cluster control plane to communicate with worker Kubelet and pods GroupId: !Ref ClusterControlPlaneSecurityGroup DestinationSecurityGroupId: !Ref NodeSecurityGroup IpProtocol: tcp FromPort: 1025 ToPort: 65535 ClusterControlPlaneSecurityGroupIngress: Type: AWS::EC2::SecurityGroupIngress DependsOn: NodeSecurityGroup Properties: Description: Allow pods to communicate with the cluster API Server GroupId: !Ref ClusterControlPlaneSecurityGroup SourceSecurityGroupId: !Ref NodeSecurityGroup IpProtocol: tcp ToPort: 443 FromPort: 443 NodeSecurityGroupFromControlPlaneOn443Ingress: Type: AWS::EC2::SecurityGroupIngress DependsOn: NodeSecurityGroup Properties: Description: Allow pods running extension API servers on port 443 to receive communication from cluster control plane GroupId: !Ref NodeSecurityGroup SourceSecurityGroupId: !Ref ClusterControlPlaneSecurityGroup IpProtocol: tcp FromPort: 443 ToPort: 443 ControlPlaneEgressToNodeSecurityGroupOn443: Type: AWS::EC2::SecurityGroupEgress DependsOn: NodeSecurityGroup Properties: Description: Allow the cluster control plane to communicate with pods running extension API servers on port 443 GroupId: !Ref ClusterControlPlaneSecurityGroup DestinationSecurityGroupId: !Ref NodeSecurityGroup IpProtocol: tcp FromPort: 443 ToPort: 443 NodeGroup: Type: AWS::AutoScaling::AutoScalingGroup Properties: DesiredCapacity: !Ref NodeDesiredSize LaunchConfigurationName: !Ref NodeLaunchConfig MinSize: 0 MaxSize: !Ref NodeAutoScalingGroupMaxSize VPCZoneIdentifier: !Ref Subnets Tags: - Key: Name Value: !Sub "${ClusterName}-${NodeGroupName}-Node" PropagateAtLaunch: 'true' - Key: !Sub 'kubernetes.io/cluster/${ClusterName}' Value: 'owned' PropagateAtLaunch: 'true' UpdatePolicy: AutoScalingRollingUpdate: MinInstancesInService: '0' MaxBatchSize: '1' PauseTime: PT5M NodeLaunchConfig: Type: AWS::AutoScaling::LaunchConfiguration Properties: AssociatePublicIpAddress: 'false' BlockDeviceMappings: - DeviceName: /dev/xvdb Ebs: DeleteOnTermination: true VolumeSize: !Ref NodeVolumeSize VolumeType: gp2 IamInstanceProfile: !Ref NodeInstanceProfile ImageId: !If - HasNodeImageId - Ref: NodeImageId - Ref: NodeImageIdSSMParam InstanceType: !Ref NodeInstanceType KeyName: !Ref KeyName SpotPrice: !Ref SpotPrice SecurityGroups: - !Ref NodeSecurityGroup - !Ref WorkerNodeSecurityGroup UserData: Fn::Base64: !Sub | [settings.kubernetes] api-server = "${APIServer}" cluster-certificate = "${ClusterCertificate}" cluster-name = "${ClusterName}" [settings.kubernetes.node-labels] lifecycle = "spot" ${NodeLabel} [settings.host-containers.admin] enabled = true superpowered = true Outputs: NodeInstanceRole: Description: The node instance role Value: !GetAtt NodeInstanceRole.Arn NodeInstanceRole: Description: The node instance role Value: !GetAtt NodeInstanceRole.Arn NodeSecurityGroup: Description: The security group for the node group Value: !Ref NodeSecurityGroup