Install AWS Load Balancer Controller add-on
$ export EKS_CLUSTER_NAME=blog
$ export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output text)
$ export LB_Controller_IAMPolicy=AWSLoadBalancerControllerIAMPolicy
Set service-account-name.
$ export SERVICE_ACCOUNT_NAME=aws-load-balancer-controller
$ export SERVICE_ACCOUNT_NAMESPACE=kube-system
$ export IAM_ROLE_NAME_0=AmazonEKSLoadBalancerControllerRole
Create an IAM policy.
Download an IAM policy for the AWS Load Balancer Controller that allows it to make calls to AWS APIs on your behalf.
Refer to https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.4.5/docs/install/iam_policy.json
$ vim iam-policy.json
下面这个不用
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "acm:DescribeCertificate", "acm:ListCertificates", "acm:GetCertificate" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "ec2:AuthorizeSecurityGroupIngress", "ec2:CreateSecurityGroup", "ec2:CreateTags", "ec2:DeleteTags", "ec2:DeleteSecurityGroup", "ec2:DescribeAccountAttributes", "ec2:DescribeAddresses", "ec2:DescribeInstances", "ec2:DescribeInstanceStatus", "ec2:DescribeInternetGateways", "ec2:DescribeNetworkInterfaces", "ec2:DescribeSecurityGroups", "ec2:DescribeSubnets", "ec2:DescribeTags", "ec2:DescribeVpcs", "ec2:ModifyInstanceAttribute", "ec2:ModifyNetworkInterfaceAttribute", "ec2:RevokeSecurityGroupIngress" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "elasticloadbalancing:AddListenerCertificates", "elasticloadbalancing:AddTags", "elasticloadbalancing:CreateListener", "elasticloadbalancing:CreateLoadBalancer", "elasticloadbalancing:CreateRule", "elasticloadbalancing:CreateTargetGroup", "elasticloadbalancing:DeleteListener", "elasticloadbalancing:DeleteLoadBalancer", "elasticloadbalancing:DeleteRule", "elasticloadbalancing:DeleteTargetGroup", "elasticloadbalancing:DeregisterTargets", "elasticloadbalancing:DescribeListenerCertificates", "elasticloadbalancing:DescribeListeners", "elasticloadbalancing:DescribeLoadBalancers", "elasticloadbalancing:DescribeLoadBalancerAttributes", "elasticloadbalancing:DescribeRules", "elasticloadbalancing:DescribeSSLPolicies", "elasticloadbalancing:DescribeTags", "elasticloadbalancing:DescribeTargetGroups", "elasticloadbalancing:DescribeTargetGroupAttributes", "elasticloadbalancing:DescribeTargetHealth", "elasticloadbalancing:ModifyListener", "elasticloadbalancing:ModifyLoadBalancerAttributes", "elasticloadbalancing:ModifyRule", "elasticloadbalancing:ModifyTargetGroup", "elasticloadbalancing:ModifyTargetGroupAttributes", "elasticloadbalancing:RegisterTargets", "elasticloadbalancing:RemoveListenerCertificates", "elasticloadbalancing:RemoveTags", "elasticloadbalancing:SetIpAddressType", "elasticloadbalancing:SetSecurityGroups", "elasticloadbalancing:SetSubnets", "elasticloadbalancing:SetWebACL" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "iam:CreateServiceLinkedRole", "iam:GetServerCertificate", "iam:ListServerCertificates" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "cognito-idp:DescribeUserPoolClient" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "waf-regional:GetWebACLForResource", "waf-regional:GetWebACL", "waf-regional:AssociateWebACL", "waf-regional:DisassociateWebACL" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "tag:GetResources", "tag:TagResources" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "waf:GetWebACL" ], "Resource": "*" } ] }用这个
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "iam:CreateServiceLinkedRole" ], "Resource": "*", "Condition": { "StringEquals": { "iam:AWSServiceName": "elasticloadbalancing.amazonaws.com" } } }, { "Effect": "Allow", "Action": [ "ec2:DescribeAccountAttributes", "ec2:DescribeAddresses", "ec2:DescribeAvailabilityZones", "ec2:DescribeInternetGateways", "ec2:DescribeVpcs", "ec2:DescribeVpcPeeringConnections", "ec2:DescribeSubnets", "ec2:DescribeSecurityGroups", "ec2:DescribeInstances", "ec2:DescribeNetworkInterfaces", "ec2:DescribeTags", "ec2:GetCoipPoolUsage", "ec2:DescribeCoipPools", "elasticloadbalancing:DescribeLoadBalancers", "elasticloadbalancing:DescribeLoadBalancerAttributes", "elasticloadbalancing:DescribeListeners", "elasticloadbalancing:DescribeListenerCertificates", "elasticloadbalancing:DescribeSSLPolicies", "elasticloadbalancing:DescribeRules", "elasticloadbalancing:DescribeTargetGroups", "elasticloadbalancing:DescribeTargetGroupAttributes", "elasticloadbalancing:DescribeTargetHealth", "elasticloadbalancing:DescribeTags" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "cognito-idp:DescribeUserPoolClient", "acm:ListCertificates", "acm:DescribeCertificate", "iam:ListServerCertificates", "iam:GetServerCertificate", "waf-regional:GetWebACL", "waf-regional:GetWebACLForResource", "waf-regional:AssociateWebACL", "waf-regional:DisassociateWebACL", "wafv2:GetWebACL", "wafv2:GetWebACLForResource", "wafv2:AssociateWebACL", "wafv2:DisassociateWebACL", "shield:GetSubscriptionState", "shield:DescribeProtection", "shield:CreateProtection", "shield:DeleteProtection" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "ec2:AuthorizeSecurityGroupIngress", "ec2:RevokeSecurityGroupIngress" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "ec2:CreateSecurityGroup" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "ec2:CreateTags" ], "Resource": "arn:aws:ec2:*:*:security-group/*", "Condition": { "StringEquals": { "ec2:CreateAction": "CreateSecurityGroup" }, "Null": { "aws:RequestTag/elbv2.k8s.aws/cluster": "false" } } }, { "Effect": "Allow", "Action": [ "ec2:CreateTags", "ec2:DeleteTags" ], "Resource": "arn:aws:ec2:*:*:security-group/*", "Condition": { "Null": { "aws:RequestTag/elbv2.k8s.aws/cluster": "true", "aws:ResourceTag/elbv2.k8s.aws/cluster": "false" } } }, { "Effect": "Allow", "Action": [ "ec2:AuthorizeSecurityGroupIngress", "ec2:RevokeSecurityGroupIngress", "ec2:DeleteSecurityGroup" ], "Resource": "*", "Condition": { "Null": { "aws:ResourceTag/elbv2.k8s.aws/cluster": "false" } } }, { "Effect": "Allow", "Action": [ "elasticloadbalancing:CreateLoadBalancer", "elasticloadbalancing:CreateTargetGroup" ], "Resource": "*", "Condition": { "Null": { "aws:RequestTag/elbv2.k8s.aws/cluster": "false" } } }, { "Effect": "Allow", "Action": [ "elasticloadbalancing:CreateListener", "elasticloadbalancing:DeleteListener", "elasticloadbalancing:CreateRule", "elasticloadbalancing:DeleteRule" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "elasticloadbalancing:AddTags", "elasticloadbalancing:RemoveTags" ], "Resource": [ "arn:aws:elasticloadbalancing:*:*:targetgroup/*/*", "arn:aws:elasticloadbalancing:*:*:loadbalancer/net/*/*", "arn:aws:elasticloadbalancing:*:*:loadbalancer/app/*/*" ], "Condition": { "Null": { "aws:RequestTag/elbv2.k8s.aws/cluster": "true", "aws:ResourceTag/elbv2.k8s.aws/cluster": "false" } } }, { "Effect": "Allow", "Action": [ "elasticloadbalancing:AddTags", "elasticloadbalancing:RemoveTags" ], "Resource": [ "arn:aws:elasticloadbalancing:*:*:listener/net/*/*/*", "arn:aws:elasticloadbalancing:*:*:listener/app/*/*/*", "arn:aws:elasticloadbalancing:*:*:listener-rule/net/*/*/*", "arn:aws:elasticloadbalancing:*:*:listener-rule/app/*/*/*" ] }, { "Effect": "Allow", "Action": [ "elasticloadbalancing:ModifyLoadBalancerAttributes", "elasticloadbalancing:SetIpAddressType", "elasticloadbalancing:SetSecurityGroups", "elasticloadbalancing:SetSubnets", "elasticloadbalancing:DeleteLoadBalancer", "elasticloadbalancing:ModifyTargetGroup", "elasticloadbalancing:ModifyTargetGroupAttributes", "elasticloadbalancing:DeleteTargetGroup" ], "Resource": "*", "Condition": { "Null": { "aws:ResourceTag/elbv2.k8s.aws/cluster": "false" } } }, { "Effect": "Allow", "Action": [ "elasticloadbalancing:RegisterTargets", "elasticloadbalancing:DeregisterTargets" ], "Resource": "arn:aws:elasticloadbalancing:*:*:targetgroup/*/*" }, { "Effect": "Allow", "Action": [ "elasticloadbalancing:SetWebAcl", "elasticloadbalancing:ModifyListener", "elasticloadbalancing:AddListenerCertificates", "elasticloadbalancing:RemoveListenerCertificates", "elasticloadbalancing:ModifyRule" ], "Resource": "*" } ] }
Create an IAM policy using the policy downloaded in the previous step.
aws iam create-policy \ --policy-name $LB_Controller_IAMPolicy \ --policy-document file://iam-policy.json$ aws iam create-policy --policy-name $ALB_Ingress_Controller_IAMPolicy --policy-document file://iam-policy.json
{ "Policy": { "PolicyName": "AWSLoadBalancerControllerIAMPolicy", "PolicyId": "ANPA***VQ4O", "Arn": "arn:aws:iam::111122223333:policy/AWSLoadBalancerControllerIAMPolicy", "Path": "/", "DefaultVersionId": "v1", "AttachmentCount": 0, "PermissionsBoundaryUsageCount": 0, "IsAttachable": true, "CreateDate": "2023-01-03T09:30:34+00:00", "UpdateDate": "2023-01-03T09:30:34+00:00" } }
Create an IAM role. Create a Kubernetes service account named
aws-load-balancer-controller
in the kube-system
namespace for the AWS Load Balancer Controller and annotate the Kubernetes service account with the name of the IAM role.You can use
eksctl
or the AWS CLI and kubectl
to create the IAM role and Kubernetes service account.You must use at least version 1.18.17 of the AWS CLI to receive the proper output from this command!
$ OIDC_PROVIDER=$(aws eks describe-cluster --name $EKS_CLUSTER_NAME --query "cluster.identity.oidc.issuer" --output text | sed -e "s/^https:\/\///")
$ echo $OIDC_PROVIDER
oidc.eks.us-west-2.amazonaws.com/id/BFDB***D49F
Retrieve your cluster's OIDC provider ID and store it in a variable.
oidc_id=$(aws eks describe-cluster --name $EKS_CLUSTER_NAME --query "cluster.identity.oidc.issuer" --output text | cut -d '/' -f 5)
echo $oidc_id
BFDB7****8D49F
Determine whether an IAM OIDC provider with your cluster's ID is already in your account.
aws iam list-open-id-connect-providers | grep $oidc_id | cut -d "/" -f4
BFDB7***8D49F"If output is returned, then you already have an IAM OIDC provider for your cluster. If no output is returned, then you must create an IAM OIDC provider for your cluster.
Copy the following contents to your device.
After replacing the text, run the modified command to create the
trust.json
file.read -r -d '' TRUST_RELATIONSHIP <<EOF { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::${AWS_ACCOUNT_ID}:oidc-provider/${OIDC_PROVIDER}" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "${OIDC_PROVIDER}:aud": "sts.amazonaws.com", "${OIDC_PROVIDER}:sub": "system:serviceaccount:${SERVICE_ACCOUNT_NAMESPACE}:${SERVICE_ACCOUNT_NAME}" } } } ] } EOF
$ echo "${TRUST_RELATIONSHIP}" > trust.json
$ cat trust.json
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::111122223333:oidc-provider/oidc.eks.us-west-2.amazonaws.com/id/BFDB7***8D49F" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "oidc.eks.us-west-2.amazonaws.com/id/BFDB7***8D49F:aud": "sts.amazonaws.com", "oidc.eks.us-west-2.amazonaws.com/id/BFDB7***8D49F:sub": "system:serviceaccount:kube-system:aws-load-balancer-controller" } } } ] }
Create an IAM Role
$ aws iam create-role --role-name $IAM_ROLE_NAME_0 --assume-role-policy-document file://trust.json --description "IAM ROLE"
{ "Role": { "Path": "/", "RoleName": "AmazonEKSLoadBalancerControllerRole", "RoleId": "AROA***4YDF", "Arn": "arn:aws:iam::111122223333:role/AmazonEKSLoadBalancerControllerRole", "CreateDate": "2023-01-03T12:34:45+00:00", "AssumeRolePolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::111122223333:oidc-provider/oidc.eks.us-west-2.amazonaws.com/id/BFDB7***8D49F" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "oidc.eks.us-west-2.amazonaws.com/id/BFDB7***8D49F:aud": "sts.amazonaws.com", "oidc.eks.us-west-2.amazonaws.com/id/BFDB7***8D49F:sub": "system:serviceaccount:kube-system:aws-load-balancer-controller" } } } ] } } }
Attach the required Amazon EKS managed IAM policy "AWSLoadBalancerControllerIAMPolicy" to the IAM role "AmazonEKSLoadBalancerControllerRole".
$ aws iam attach-role-policy --role-name $IAM_ROLE_NAME_0 --policy-arn=arn:aws:iam::${AWS_ACCOUNT_ID}:policy/${LB_Controller_IAMPolicy}
Create the Kubernetes service account on your cluster. The Kubernetes service account named "aws-load-balancer-controller" is annotated with the IAM role that you created named "AmazonEKSLoadBalancerControllerRole".
aws-load-balancer-controller-service-account.yaml
read -r -d '' SERVICE_ACCOUNT <<EOF apiVersion: v1 kind: ServiceAccount metadata: labels: app.kubernetes.io/component: controller app.kubernetes.io/name: ${SERVICE_ACCOUNT_NAME} name: ${SERVICE_ACCOUNT_NAME} namespace: ${SERVICE_ACCOUNT_NAMESPACE} annotations: eks.amazonaws.com/role-arn: arn:aws:iam::${AWS_ACCOUNT_ID}:role/${IAM_ROLE_NAME_0} EOF
echo "${SERVICE_ACCOUNT}" > aws-load-balancer-controller-service-account.yaml
cat aws-load-balancer-controller-service-account.yaml
apiVersion: v1 kind: ServiceAccount metadata: labels: app.kubernetes.io/component: controller app.kubernetes.io/name: aws-load-balancer-controller name: aws-load-balancer-controller namespace: kube-system annotations: eks.amazonaws.com/role-arn: arn:aws:iam::111122223333:role/AmazonEKSLoadBalancerControllerRole
kubectl apply -f aws-load-balancer-controller-service-account.yaml
serviceaccount/aws-load-balancer-controller created
Annotate service account with the IAM role.
kubectl annotate serviceaccount -n kube-system $SERVICE_ACCOUNT_NAME \ eks.amazonaws.com/role-arn=arn:aws:iam::${AWS_ACCOUNT_ID}:role/${IAM_ROLE_NAME_0}$ kubectl annotate serviceaccount -n $SERVICE_ACCOUNT_NAMESPACE $SERVICE_ACCOUNT_NAME \
eks.amazonaws.com/role-arn=arn:aws:iam::${AWS_ACCOUNT_ID}:role/${IAM_ROLE_NAME_0}
serviceaccount/alb-ingress-controller annotated-
Install the AWS Load Balancer Controller using Helm V3 or later or by applying a Kubernetes manifest. If you want to deploy the controller on Fargate, use the Helm procedure. The Helm procedure doesn't depend on cert-manager because it generates a self-signed certificate.
Add the
eks-charts
repository.helm repo add eks https://aws.github.io/eks-charts
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/ec2-user/.kube/config WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/ec2-user/.kube/config "eks" has been added to your repositories
Update your local repo to make sure that you have the most recent charts.
helm repo update
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/ec2-user/.kube/config WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/ec2-user/.kube/config Hang tight while we grab the latest from your chart repositories... ...Successfully got an update from the "eks" chart repository ...Successfully got an update from the "grafana" chart repository ...Successfully got an update from the "prometheus-community" chart repository ...Successfully got an update from the "stable" chart repository Update Complete. ⎈Happy Helming!⎈
Install the AWS Load Balancer Controller.
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \ -n $SERVICE_ACCOUNT_NAMESPACE \ --set clusterName=$EKS_CLUSTER_NAME \ --set serviceAccount.create=false \ --set serviceAccount.name=$SERVICE_ACCOUNT_NAME-
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/ec2-user/.kube/config WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/ec2-user/.kube/config NAME: aws-load-balancer-controller LAST DEPLOYED: Tue Jan 3 13:01:43 2023 NAMESPACE: kube-system STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: AWS Load Balancer controller installed!
Move the Pod to the work nodes based on ARM64.
kubectl patch deployment -n kube-system aws-load-balancer-controller --patch '{"spec": {"template": {"spec": {"nodeSelector": {"kubernetes.io/arch": "arm64"}}}}}'
deployment.apps/aws-load-balancer-controller patched
Important
The deployed chart doesn't receive security updates automatically. You need to manually upgrade to a newer chart when it becomes available. When upgrading, change install to upgrade in the previous command, but run the following command to install the TargetGroupBinding custom resource definitions before running the previous command.
kubectl apply -k "github.com/aws/eks-charts/stable/aws-load-balancer-controller/crds?ref=master"
Verify that the controller is installed.
kubectl get deployment -n kube-system aws-load-balancer-controller
NAME READY UP-TO-DATE AVAILABLE AGE aws-load-balancer-controller 2/2 2 2 16m
Refer to https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.1.4/docs/examples/alb-ingress-controller.yaml
$ wget https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.1.4/docs/examples/alb-ingress-controller.yaml
Content:
# Application Load Balancer (ALB) Ingress Controller Deployment Manifest. # This manifest details sensible defaults for deploying an ALB Ingress Controller. # GitHub: https://github.com/kubernetes-sigs/aws-alb-ingress-controller apiVersion: apps/v1 kind: Deployment metadata: labels: app.kubernetes.io/name: alb-ingress-controller name: alb-ingress-controller # Namespace the ALB Ingress Controller should run in. Does not impact which # namespaces it's able to resolve ingress resource for. For limiting ingress # namespace scope, see --watch-namespace. namespace: kube-system spec: selector: matchLabels: app.kubernetes.io/name: alb-ingress-controller template: metadata: labels: app.kubernetes.io/name: alb-ingress-controller spec: containers: - name: alb-ingress-controller args: # Limit the namespace where this ALB Ingress Controller deployment will # resolve ingress resources. If left commented, all namespaces are used. # - --watch-namespace=your-k8s-namespace # Setting the ingress-class flag below ensures that only ingress resources with the # annotation kubernetes.io/ingress.class: "alb" are respected by the controller. You may # choose any class you'd like for this controller to respect. - --ingress-class=alb # REQUIRED # Name of your cluster. Used when naming resources created # by the ALB Ingress Controller, providing distinction between # clusters. # - --cluster-name=devCluster # AWS VPC ID this ingress controller will use to create AWS resources. # If unspecified, it will be discovered from ec2metadata. # - --aws-vpc-id=vpc-xxxxxx # AWS region this ingress controller will operate in. # If unspecified, it will be discovered from ec2metadata. # List of regions: http://docs.aws.amazon.com/general/latest/gr/rande.html#vpc_region # - --aws-region=us-west-1 # Enables logging on all outbound requests sent to the AWS API. # If logging is desired, set to true. # - --aws-api-debug # Maximum number of times to retry the aws calls. # defaults to 10. # - --aws-max-retries=10 # env: # AWS key id for authenticating with the AWS API. # This is only here for examples. It's recommended you instead use # a project like kube2iam for granting access. #- name: AWS_ACCESS_KEY_ID # value: KEYVALUE # AWS key secret for authenticating with the AWS API. # This is only here for examples. It's recommended you instead use # a project like kube2iam for granting access. #- name: AWS_SECRET_ACCESS_KEY # value: SECRETVALUE # Repository location of the ALB Ingress Controller. image: docker.io/amazon/aws-alb-ingress-controller:v1.1.4 serviceAccountName: alb-ingress-controller
$ kubectl apply -f alb-ingress-controller.yaml
deployment.apps/alb-ingress-controller created
$ kubectl edit deployment.apps/${SERVICE_ACCOUNT_NAME} -n kube-system
Add a line for the cluster name after the
--ingress-class=alb
line. If you're running the ALB ingress controller on Fargate, then you must also add the lines for the VPC ID, and AWS Region name of your cluster. Once you've added the appropriate lines, save and close the file.spec: ... template: ... spec: containers: - args: - --ingress-class=alb - --cluster-name=blog
$ kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE alb-ingress-controller-544cdfbb74-mjkrz 1/1 Running 0 27s aws-node-k682g 1/1 Running 0 5m39s coredns-84549585c-bmjdg 1/1 Running 0 24m coredns-84549585c-df4dd 1/1 Running 0 24m kube-proxy-tll66 1/1 Running 0 5m39s
References
Installing the AWS Load Balancer Controller add-on
https://github.com/kubernetes-sigs/aws-load-balancer-controller
-