Amazon Elastic Kubernetes Service (EKS)

EKS is Amazon Web Services' Managed Kubernetes service that allows customers to run their containerized applications on Kubernetes clusters running in the AWS cloud. Nirmata significantly simplifies EKS cluster deployment and management and eliminates any undifferentiated heavy lifting required to operate AWS EKS clusters at scale. Nirmata enhances EKS clusters and helps customer leverage EKS deployments to their fullest.

Prerequisites

EKS Deployment requires a few things to be in-place before you deploy the cluster through Nirmata:

  1. VPC: Create IP Address block for your EC2 instances.

  2. Subnet: Any AWS setup requires you to setup a Subnet with subnet policy which will be used to provide connectivity across your EC2 instances.It is recommended to Create minimum 2 Availability Zone

  3. Security Group: Security group defines that security policy for access into and between your EC2 instances. Security policy is core construct needed to setup your AWS services. For an EKS cluster, two security groups are recommended - control-plane security group for cluster operations and worker node security group for application traffic. For control plane security group, for inbound traffic, ports 443 from all nodes in worker security group are recommended. For outbound traffic, ports 1025-65535 are recommended to be open. Port 10250 is minimum requirement. For worker security group, port 443 and port-range 1025-65535 is recommended to be open. For outbound traffic, all ports can be open. Click here for more information.

  4. Cloud Credentials: In Nirmata UI, configure the Cloud Credentials. You can use the ‘Assume Role’ option, which is the most secure option for third party access to your AWS account or use the Access Key option.

To access the Add Cloud Credentials options, select Cloud Credentials from the sidebar menu. Then click on the Add Cloud Credentials icon. Click Here to Create Cloud Credentials

Note: Assume Role will be use only when you are using Nirmata Saas platform

  1. EKS Role: AWS requires an additional role to create and manage the EKS cluster resources. Create a role with AmazonEKSClusterPolicy and AmazonEKSServicePolicy and use that in the Cluster Role ARN. Click here for a step-by-step introduction on creating an IAM role for EKS.

  2. Node IAM Role: A node instance IAM role is required to allow kubelet running on nodes to make calls to AWS APIs. Ensure that the AmazonEKSWorkerNodePolicy, AmazonEKS_CNI_Policy, and AmazonEC2ContainerRegistryReadOnly managed policies are attached to the role. You can refer to EKS documentation for more details.

EKS Cluster Type Configuration Steps

To create an EKS cluster type, select Clusters from the sidebar menu and then click +Cluster Types and Add Cluster Type button. Choose EKS from the Add Cluster Type panel.

  1. Cloud Credentials- Select the Cloud Credentials you would like to use for this cluster type.

  2. Kubernetes Version - Select the Kubernetes version.

  3. Region - Select the region where your cluster will be deployed.

  4. VPC - Select virtual private cloud (VPC) in which your cluster will be deployed.

  5. Subnet - Select at least two Subnets for your cluster. Elastic Network Interfaces (ENI) on the cluster nodes will be able to communicate on these networks.

  6. Security Groups - Select the security groups to apply to the EKS-managed Elastic Network Interfaces (ENI) that are created in your worker node subnets.

  7. Private Endpoint Access - Select this option if you want the API server endpoint for your cluster to be private.

  8. Cluster Role ARN - This is the EKS cluster role you created in Step-5 for Nirmata to deploy EKS. Again, with cloud provider integration, you should see it in the drop down menu.

  9. Enable Envelop Encryption - Select this option to provide an additional layer of encryption for your Kubernetes cluster.

Then select the configuration for your node pools:

  1. Instance Type: Select the EC2 instance type you would like to use for EKS cluster.

  2. SSH Key- Select the SSH key ID to be configured on the nodes.

  3. Disk Size- Enter the disk size for the nodes.

  4. Node Instance IAM Role - Select the IAM Role that will be used by the nodes. You can create a new role, in the AWS IAM console. See instructions in Step 6 for EKS: Deployment Prerequisites

  5. Security Group - Select the security groups to apply to the EKS-managed Elastic Network Interfaces (ENI) that are created in your worker node subnets.

  6. Use Custom Image - Select to use custom image. By default this option is not selected so that you can select the standard EKS-optimized Amazon Machine Images.

    Image ID: You can provide an EKS compliant image (AMI ID) for the instance type or you can select one of the available image types (e.g. Bottlerocket).

    Bottlerocket is a Linux-based open-source operating system that is purpose-built by Amazon Web Services for running containers. To learn more about Amazon Bottlerocket you can go here .

    When selecting this option, you can also provide the user data. To learn more about using custom images refer to the section below: Using Custom AMIs.

    AMI Type - Select the EKS-optimized Amazon Machine Image for nodes.

  7. Node Labels - Specify the labels for your nodes.

Finally, configure advanced settings for your EKS cluster type:

  1. Logging -Configure logging for your EKS cluster control plane.

  2. Fargate - Enable Fargate for your cluster and configure default settings to be used when a Fargate profile is automatically created for Fargate-enabled namespace. You can find more details regarding how Nirmata automates the creation for Fargate profiles here .

  3. Overrides - Select fields that you would like to allow to be modified when a new cluster is created. This lets you hide complex configuration in the cluster type while allowing the user to modify configuration such as machine type, networks etc. when the cluster is created.

  4. System Metadata - Specify any metadata that you would like to add to the cluster. The metadata will be added to the ‘system-metadata-map’ ConfigMap in the ‘nirmata’ namespace in your cluster.

  5. Add-Ons - Select the add-ons that you would like to deploy to the cluster. The selected add-ons will automatically be deployed once the cluster is created and ‘Ready’.

After completing the configuration steps, click Create. Your EKS cluster type will be created. Now, you can use this cluster type to create an EKS

Using Custom AMIs

To use custom images when creating EKS clusters, you need to specify the custom image in the cluster type.

  1. During cluster type creation, select ‘Use Custom AMI’ option

  2. Next, specify the image ID for your custom AMI. In case you are interested in using the Bottlerocket AMI, you need to add AmazonSSMFullAccess policy to the IAM role being used for the AWS cloud credentials

  3. Next, you will need to add ‘User Data’. User data is specific to your AMI and will be run when the image is initialized. In your user data, you will need the information to connect to the EKS control plane. You can get that information by using the following variables in the User Data

    • ${API_SERVER} : The URL of the api server
    • ${CA_CERTIFICATE_DATA} : the CA certificate
    • ${CLUSTER_NAME} : the name of the cluster

    For example, if you select ‘Amazon EKS-optimized Bottlerocket AMI’, the user data is automatically set to:

[settings.kubernetes]
api-server = "${API_SERVER}"
cluster-certificate = "${CA_CERTIFICATE_DATA}"
cluster-name = "${CLUSTER_NAME}"

Now, when you create you cluster using this cluster type, the custom AMI will be used and if the user data is configured correctly, the nodes will be connected to the EKS control plane.

NOTE: If your custom AMI is not configured to enable the kubelet to connect to the EKS control plane, the cluster creation will timeout and fail.

Creating an EKS Cluster

To create an EKS cluster, select Clusters from side menu and clicking on the Add Cluster button.

  1. Select EKS in the Add Cluster screen

  2. Select the cluster type that you previously created

  3. Provide the node count and alternately enable autoscaling for your node pool.

Once you click on Create cluster button, the cluster will be created in 10-15 minutes.

NOTE: If you are creating a cluster in private address space and do not have a routed connection back to Nirmata you will be able to create the cluster but the Nirmata controller will not connect to Nirmata. Your cluster will display “Pending controller connect” state and subsequently fail.