Deploy Flink to EKS - Part 1 - Install kubectl, create kubeconfig - [Description]
2020年07月16日
Cluster provisioning usually takes less than 10 minutes. You can query the status of your cluster with the following command. When your cluster status is ACTIVE, you can proceed.
~]$ aws eks describe-cluster --name flink --query cluster.status
When your cluster provisioning is complete, retrieve the endpoint and certificateAuthority.data values with the following commands. These must be added to your kubectl configuration so that you can communicate with your cluster.
Retrieve the endpoint:
~]$ aws eks describe-cluster --name flink --query cluster.endpoint
Retrieve the certificateAuthority.data:
~]$ aws eks describe-cluster --name flink --query cluster.certificateAuthority.data
Install kubectl
Download the Amazon EKS-vended kubectl binary for your cluster's Kubernetes version.
~]$ curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.17.7/2020-07-08/bin/linux/amd64/kubectl
(Optional) Verify the downloaded binary with the SHA-256 sum for your binary. Download the SHA-256 sum for your cluster's Kubernetes version for Linux. Kubernetes 1.17:
~]$ curl -o kubectl.sha256 https://amazon-eks.s3.us-west-2.amazonaws.com/1.17.7/2020-07-08/bin/linux/amd64/kubectl.sha256
Check the SHA-256 sum for your downloaded binary.
~]$ openssl sha1 -sha256 kubectl
Compare the generated SHA-256 sum in the command output against your downloaded SHA-256 file. The two should match.
~]$ cat kubectl.sha256
Apply execute permissions to the binary.
~]$ chmod +x ./kubectl
Copy the binary to a folder in your PATH. If you have already installed a version of kubectl, then we recommend creating a $HOME/bin/kubectl and ensuring that $HOME/bin comes first in your $PATH.
~]$ mkdir -p $HOME/bin && cp ./kubectl $HOME/bin/kubectl && export PATH=$PATH:$HOME/bin
(Optional) Add the $HOME/bin path to your shell initialization file so that it is configured when you open a shell.
~]$ echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
After you install kubectl, you can verify its version with the following command:
~]$ kubectl version --short --client
Install aws-iam-authenticator
If you're running the AWS CLI version 1.16.156 or later, then you don't need to install the authenticator. Instead, you can use the aws eks get-token command.
Create kubeconfig file manually
Create the default ~/.kube directory if it does not already exist.
~]$ mkdir -p ~/.kube
Open your favorite text editor and copy the kubeconfig code blocks below into it。
To use the AWS CLI aws eks get-token command (requires version 1.16.156 or later of the AWS CLI):
~]$ vim ~/.kube/config-flink
Replace the <endpoint-url> with the endpoint URL that was created for your cluster.
Replace the
Replace the <cluster-name> with your cluster name.
(Optional) To assume an IAM role to perform cluster operations instead of the default AWS credential provider chain, uncomment the -r or --role and <role-arn> lines and substitute an IAM role ARN to use with your user.
(Optional) To always use a specific named AWS credential profile (instead of the default AWS credential provider chain), uncomment the env lines and substitute <aws-profile> with the profile name to use.
Save the file to the default kubectl folder, with your cluster name in the file name. For example, if your cluster name is devel, save the file to ~/.kube/config-devel.
Add that file path to your KUBECONFIG environment variable so that kubectl knows where to look for your cluster configuration.
~]$ export KUBECONFIG=$KUBECONFIG:~/.kube/config-flink
(Optional) Add the configuration to your shell initialization file so that it is configured when you open a shell.
~]$ echo 'export KUBECONFIG=$KUBECONFIG:~/.kube/config-flink' >> ~/.bashrc
-
~]$ aws eks update-kubeconfig --name flink
Test your configuration.
~]$ kubectl get svc
References
Installing kubectl
-
-
This environment is based on Amazon Linux 2.
The Kubernetes version is 1.17.
~]$ aws eks create-cluster --name flink --role-arn arn:aws:iam::123456789012:role/eksServiceRole --resources-vpc-config subnetIds=subnet-0fe6###14c2,subnet-0e31###dddd,subnet-0173###7750,subnet-0caa###e73d,securityGroupIds=sg-0dc2###8bb8,endpointPublicAccess=true,endpointPrivateAccess=true --kubernetes-version 1.17
{
"cluster": {
"status": "CREATING",
"logging": {
"clusterLogging": [
{
"enabled": false,
"types": [
"api",
"audit",
"authenticator",
"controllerManager",
"scheduler"
]
}
]
},
"name": "flink",
"tags": {},
"certificateAuthority": {},
"roleArn": "arn:aws:iam::123456789012:role/eksServiceRole",
"resourcesVpcConfig": {
"subnetIds": [
"subnet-0fe6###14c2",
"subnet-0e31###dddd",
"subnet-0173###7750",
"subnet-0caa###e73d"
],
"vpcId": "vpc-0b05###8b30",
"endpointPrivateAccess": true,
"endpointPublicAccess": true,
"securityGroupIds": [
"sg-0dc2###8bb8"
]
},
"version": "1.16",
"arn": "arn:aws:eks:us-west-2:123456789012:cluster/flink",
"platformVersion": "eks.2",
"createdAt": 1594904041.626
}
}
Cluster provisioning usually takes less than 10 minutes. You can query the status of your cluster with the following command. When your cluster status is ACTIVE, you can proceed.
~]$ aws eks describe-cluster --name flink --query cluster.status
"ACTIVE"
When your cluster provisioning is complete, retrieve the endpoint and certificateAuthority.data values with the following commands. These must be added to your kubectl configuration so that you can communicate with your cluster.
Retrieve the endpoint:
~]$ aws eks describe-cluster --name flink --query cluster.endpoint
"https://C4BC1A03718A5FEB16D91191FD61A1C5.yl4.us-west-2.eks.amazonaws.com"
Retrieve the certificateAuthority.data:
~]$ aws eks describe-cluster --name flink --query cluster.certificateAuthority.data
"LS0t####tLQo="
Install kubectl
Download the Amazon EKS-vended kubectl binary for your cluster's Kubernetes version.
~]$ curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.17.7/2020-07-08/bin/linux/amd64/kubectl
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 56.6M 100 56.6M 0 0 72.0M 0 --:--:-- --:--:-- --:--:-- 72.0M
(Optional) Verify the downloaded binary with the SHA-256 sum for your binary. Download the SHA-256 sum for your cluster's Kubernetes version for Linux. Kubernetes 1.17:
~]$ curl -o kubectl.sha256 https://amazon-eks.s3.us-west-2.amazonaws.com/1.17.7/2020-07-08/bin/linux/amd64/kubectl.sha256
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 73 100 73 0 0 424 0 --:--:-- --:--:-- --:--:-- 424
Check the SHA-256 sum for your downloaded binary.
~]$ openssl sha1 -sha256 kubectl
SHA256(kubectl)= bba88f1###
Compare the generated SHA-256 sum in the command output against your downloaded SHA-256 file. The two should match.
~]$ cat kubectl.sha256
bba88f1### kubectl
Apply execute permissions to the binary.
~]$ chmod +x ./kubectl
Copy the binary to a folder in your PATH. If you have already installed a version of kubectl, then we recommend creating a $HOME/bin/kubectl and ensuring that $HOME/bin comes first in your $PATH.
~]$ mkdir -p $HOME/bin && cp ./kubectl $HOME/bin/kubectl && export PATH=$PATH:$HOME/bin
(Optional) Add the $HOME/bin path to your shell initialization file so that it is configured when you open a shell.
~]$ echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
After you install kubectl, you can verify its version with the following command:
~]$ kubectl version --short --client
Client Version: v1.17.7-eks-bffbac
Install aws-iam-authenticator
If you're running the AWS CLI version 1.16.156 or later, then you don't need to install the authenticator. Instead, you can use the aws eks get-token command.
Create kubeconfig file manually
Create the default ~/.kube directory if it does not already exist.
~]$ mkdir -p ~/.kube
Open your favorite text editor and copy the kubeconfig code blocks below into it。
To use the AWS CLI aws eks get-token command (requires version 1.16.156 or later of the AWS CLI):
~]$ vim ~/.kube/config-flink
apiVersion: v1
clusters:
- cluster:
server: https://9FE4###5969.gr7.us-west-2.eks.amazonaws.com
certificate-authority-data: LS0tLS1CR###
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: aws
name: aws
current-context: aws
kind: Config
preferences: {}
users:
- name: aws
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
command: aws
args:
- "eks"
- "get-token"
- "--cluster-name"
- "flink"
# - "--role"
# - "<role-arn>"
# env:
# - name: AWS_PROFILE
# value: "<aws-profile>"
Replace the <endpoint-url> with the endpoint URL that was created for your cluster.
Replace the
<base64-encoded-ca-cert> with the certificateAuthority.data that was created for your cluster.Replace the <cluster-name> with your cluster name.
(Optional) To assume an IAM role to perform cluster operations instead of the default AWS credential provider chain, uncomment the -r or --role and <role-arn> lines and substitute an IAM role ARN to use with your user.
(Optional) To always use a specific named AWS credential profile (instead of the default AWS credential provider chain), uncomment the env lines and substitute <aws-profile> with the profile name to use.
Save the file to the default kubectl folder, with your cluster name in the file name. For example, if your cluster name is devel, save the file to ~/.kube/config-devel.
Add that file path to your KUBECONFIG environment variable so that kubectl knows where to look for your cluster configuration.
~]$ export KUBECONFIG=$KUBECONFIG:~/.kube/config-flink
(Optional) Add the configuration to your shell initialization file so that it is configured when you open a shell.
~]$ echo 'export KUBECONFIG=$KUBECONFIG:~/.kube/config-flink' >> ~/.bashrc
-
~]$ aws eks update-kubeconfig --name flink
Updated context arn:aws:eks:us-west-2:123456789012:cluster/flink in /home/ec2-user/.kube/config-flink-
Test your configuration.
~]$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 172.20.0.1 <none> 443/TCP 8m
References
Installing kubectl
-