Sky Cone

The post Notes of Site Upgrade - Y2023 Q4 holds the catalog of the whole upgrade's note. 


This post focuses on the update of AWS Load Balancer Controller, from 2.4.5 to v2.6.2.


% k get deploy -n kube-system aws-load-balancer-controller -o yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "2"
    meta.helm.sh/release-name: aws-load-balancer-controller
    meta.helm.sh/release-namespace: kube-system
  creationTimestamp: "2023-01-03T13:01:44Z"
  generation: 2
  labels:
    app.kubernetes.io/instance: aws-load-balancer-controller
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: aws-load-balancer-controller
    app.kubernetes.io/version: v2.4.5
    helm.sh/chart: aws-load-balancer-controller-1.4.6
  name: aws-load-balancer-controller
  namespace: kube-system
  resourceVersion: "1066800016"
  uid: 3111c84c-405c-492a-821c-240911473c30
spec:
  progressDeadlineSeconds: 600
  replicas: 2
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/instance: aws-load-balancer-controller
      app.kubernetes.io/name: aws-load-balancer-controller
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      annotations:
        prometheus.io/port: "8080"
        prometheus.io/scrape: "true"
      creationTimestamp: null
      labels:
        app.kubernetes.io/instance: aws-load-balancer-controller
        app.kubernetes.io/name: aws-load-balancer-controller
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app.kubernetes.io/name
                  operator: In
                  values:
                  - aws-load-balancer-controller
              topologyKey: kubernetes.io/hostname
            weight: 100
      containers:
      - args:
        - --cluster-name=blog
        - --ingress-class=alb
        command:
        - /controller
        image: 602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon/aws-load-balancer-controller:v2.4.5
        ...

Deploy the AWS Load Balancer Controller to an Amazon EKS cluster
1. Create an IAM policy.
a. Download an IAM policy for the AWS Load Balancer Controller that allows it to make calls to AWS APIs on your behalf.
% curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.6.2/docs/install/iam_policy.json

b. Create / update an IAM policy using the policy downloaded in the previous step. If you downloaded iam_policy_us-gov.json, change iam_policy.json to iam_policy_us-gov.json before running the command.

aws iam create-policy \
    --policy-name AWSLoadBalancerControllerIAMPolicy \
    --policy-document file://iam_policy.json

To update the policy document for an existing IAM policy using the AWS Command Line Interface (CLI), you need to follow these steps
1. Retrieve the Current Policy Document:

• First, you need to get the current policy document of the IAM policy that you want to update. This is important because when you update a policy, you replace the entire policy document. You can't modify just a part of it.
• Use the aws iam get-policy-version command to retrieve the current policy document. You'll need to know the policy ARN and the policy version ID.

% aws iam get-policy --policy-arn arn:aws:iam::<111122223333>:policy/AWSLoadBalancerControllerIAMPolicy --query 'Policy.DefaultVersionId' --output text

v1

% aws iam get-policy-version --policy-arn arn:aws:iam::<111122223333>:policy/AWSLoadBalancerControllerIAMPolicy --version-id v1

Create a New Policy Version:
Once you have modified the policy document, create a new version of the policy with the updated document.
Use the aws iam create-policy-version command to create a new version. You can either upload the modified policy document directly in the command line or save it as a file and reference the file in the command.

% aws iam create-policy-version --policy-arn arn:aws:iam::<111122223333>:policy/AWSLoadBalancerControllerIAMPolicy --policy-document file://iam_policy.json --set-as-default

{
    "PolicyVersion": {
        "VersionId": "v2",
        "IsDefaultVersion": true,
        "CreateDate": "2023-12-15T12:36:43+00:00"
    }
}

Create an IAM role. Create a Kubernetes service account named aws-load-balancer-controller in the kube-system namespace for the AWS Load Balancer Controller and annotate the Kubernetes service account with the name of the IAM role.

(Optional) Configure the AWS Security Token Service endpoint type used by your Kubernetes service account. For more information, see Configuring the AWS Security Token Service endpoint for a service account.

AWS recommends using the regional AWS STS endpoints instead of the global endpoint.

% kubectl describe pod aws-load-balancer-controller-7874b86679-8pkf5 -n kube-system | grep AWS_STS_REGIONAL_ENDPOINTS

      AWS_STS_REGIONAL_ENDPOINTS:   regional

Install the AWS Load Balancer Controller using Helm V3 or later or by applying a Kubernetes manifest. If you want to deploy the controller on Fargate, use the Helm procedure. The Helm procedure doesn't depend on cert-manager because it generates a self-signed certificate.
Add the eks-charts repository.
$ helm repo add eks https://aws.github.io/eks-charts

"eks" has been added to your repositories

Update your local repo to make sure that you have the most recent charts.
$ helm repo update eks

Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "eks" chart repository
Update Complete. ⎈Happy Helming!⎈

Install the AWS Load Balancer Controller. If you're deploying the controller to Amazon EC2 nodes that have restricted access to the Amazon EC2 instance metadata service (IMDS), or if you're deploying to Fargate, then add the following flags to the helm command that follows:
--set region=region-code
--set vpcId=vpc-xxxxxxxx
Replace my-cluster with the name of your cluster. In the following command, aws-load-balancer-controller is the Kubernetes service account that you created in a previous step.

% k get po -n kube-system
NAME                                            READY   STATUS    RESTARTS       AGE
aws-load-balancer-controller-7874b86679-8pkf5   1/1     Running   0              157d
aws-load-balancer-controller-7874b86679-hp2x5   1/1     Running   0              157d
...

helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=blog \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller \
  --set region=us-west-2 \
  --set vpcId=vpc-0b***30

Update exising

helm upgrade aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=blog \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller \
  --set region=us-west-2 \
  --set vpcId=vpc-0b***30

% k get po -n kube-system

NAME                                           READY   STATUS    RESTARTS       AGE
aws-load-balancer-controller-b6b8b8b55-55wht   1/1     Running   0              19s
aws-load-balancer-controller-b6b8b8b55-c5xhh   1/1     Running   0              16s
...

% k get deploy -n kube-system aws-load-balancer-controller -o yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "3"
    meta.helm.sh/release-name: aws-load-balancer-controller
    meta.helm.sh/release-namespace: kube-system
  creationTimestamp: "2023-01-03T13:01:44Z"
  generation: 3
  labels:
    app.kubernetes.io/instance: aws-load-balancer-controller
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: aws-load-balancer-controller
    app.kubernetes.io/version: v2.6.2
    helm.sh/chart: aws-load-balancer-controller-1.6.2
  name: aws-load-balancer-controller
  namespace: kube-system
  resourceVersion: "1179133939"
  uid: 3111c84c-405c-492a-821c-240911473c30
spec:
  progressDeadlineSeconds: 600
  replicas: 2
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/instance: aws-load-balancer-controller
      app.kubernetes.io/name: aws-load-balancer-controller
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      annotations:
        prometheus.io/port: "8080"
        prometheus.io/scrape: "true"
      creationTimestamp: null
      labels:
        app.kubernetes.io/instance: aws-load-balancer-controller
        app.kubernetes.io/name: aws-load-balancer-controller
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app.kubernetes.io/name
                  operator: In
                  values:
                  - aws-load-balancer-controller
              topologyKey: kubernetes.io/hostname
            weight: 100
      containers:
      - args:
        - --cluster-name=blog
        - --ingress-class=alb
        - --aws-region=us-west-2
        - --aws-vpc-id=vpc-0b05f131ef6058b30
        image: public.ecr.aws/eks/aws-load-balancer-controller:v2.6.2
...

References

Installing the AWS Load Balancer Controller add-on

Restrict access to the instance profile assigned to the worker node