Use AWS Controllers for Kubernetes To Deploy a Resolution Consisting of Lambda, DynamoDB, and API Gateway
On this weblog submit, you’ll be utilizing AWS Controllers for Kubernetes on an Amazon EKS cluster to place collectively an answer the place HTTP requests despatched to a REST endpoint uncovered by Amazon API Gateway are processed by a Lambda perform and continued to a DynamoDB desk.
AWS Controllers for Kubernetes (also called ACK) leverage Kubernetes Custom Resource and Custom Resource Definitions and provide the skill to handle and use AWS companies instantly from Kubernetes with no need to outline assets exterior of the cluster. The thought behind ACK is to allow Kubernetes customers to explain the specified state of AWS assets utilizing the Kubernetes API and configuration language. ACK will then care for provisioning and managing the AWS assets to match the specified state. That is achieved through the use of Service controllers which can be accountable for managing the lifecycle of a selected AWS service. Every ACK service controller is packaged right into a separate container picture that’s printed in a public repository equivalent to a person ACK service controller.
There isn’t a single ACK container picture. As a substitute, there are container photos for every particular person ACK service controller that manages assets for a selected AWS API.
This weblog submit will stroll you thru use the API Gateway, DynamoDB, and Lambda service controllers for ACK.
Stipulations
To observe alongside step-by-step, along with an AWS account, you will have to have AWS CLI, kubectl, and Helm put in.
There are a number of the way in which you’ll be able to create an Amazon EKS cluster. I desire utilizing the eksctl CLI due to the comfort it gives. Creating an EKS cluster utilizing eksctl will be as straightforward as this:
eksctl create cluster --name <my-cluster> --region <region-code>
For particulars, discuss with the Getting Started with Amazon EKS – eksctl documentation.
Clone this GitHub repository and alter to the proper listing:
git clone https://github.com/abhirockzz/k8s-ack-apigw-lambda
cd k8s-ack-apigw-lambda
Okay, let’s get began!
Setup the ACK Service Controllers for AWS Lambda, API Gateway, and DynamoDB
Set up ACK Controllers
Log into the Helm registry that shops the ACK charts:
aws ecr-public get-login-password --region us-east-1 | helm registry login --username AWS --password-stdin public.ecr.aws
Deploy the ACK service controller for Amazon Lambda utilizing the lambda-chart Helm chart:
RELEASE_VERSION_LAMBDA_ACK=$(curl -sL "https://api.github.com/repos/aws-controllers-k8s/lambda-controller/releases/newest" | grep '"tag_name":' | reduce -d'"' -f4)
helm set up --create-namespace -n ack-system oci://public.ecr.aws/aws-controllers-k8s/lambda-chart "--version=$RELEASE_VERSION_LAMBDA_ACK" --generate-name --set=aws.area=us-east-1
Deploy the ACK service controller for API Gateway utilizing the apigatewayv2-chart Helm chart:
RELEASE_VERSION_APIGWV2_ACK=$(curl -sL "https://api.github.com/repos/aws-controllers-k8s/apigatewayv2-controller/releases/newest" | grep '"tag_name":' | reduce -d'"' -f4)
helm set up --create-namespace -n ack-system oci://public.ecr.aws/aws-controllers-k8s/apigatewayv2-chart "--version=$RELEASE_VERSION_APIGWV2_ACK" --generate-name --set=aws.area=us-east-1
Deploy the ACK service controller for DynamoDB utilizing the dynamodb-chart Helm chart:
RELEASE_VERSION_DYNAMODB_ACK=$(curl -sL "https://api.github.com/repos/aws-controllers-k8s/dynamodb-controller/releases/newest" | grep '"tag_name":' | reduce -d'"' -f4)
helm set up --create-namespace -n ack-system oci://public.ecr.aws/aws-controllers-k8s/dynamodb-chart "--version=$RELEASE_VERSION_DYNAMODB_ACK" --generate-name --set=aws.area=us-east-1
Now, it is time to configure the IAM permissions for the controller to invoke Lambda, DynamoDB, and API Gateway.
Configure IAM Permissions
Create an OIDC Identification Supplier for Your Cluster
For the steps under, exchange the EKS_CLUSTER_NAME and AWS_REGION variables along with your cluster title and area.
export EKS_CLUSTER_NAME=demo-eks-cluster
export AWS_REGION=us-east-1
eksctl utils associate-iam-oidc-provider --cluster $EKS_CLUSTER_NAME --region $AWS_REGION --approve
OIDC_PROVIDER=$(aws eks describe-cluster --name $EKS_CLUSTER_NAME --query "cluster.id.oidc.issuer" --output textual content | reduce -d "https://dzone.com/" -f2- | reduce -d "https://dzone.com/" -f2-)
Create IAM Roles for Your Lambda, API Gateway, and DynamoDB ACK Service Controllers
ACK Lambda Controller
Set the next setting variables:
ACK_K8S_SERVICE_ACCOUNT_NAME=ack-lambda-controller
ACK_K8S_NAMESPACE=ack-system
AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output textual content)
Create the belief coverage for the IAM function:
learn -r -d '' TRUST_RELATIONSHIP <<EOF
{
"Model": "2012-10-17",
"Assertion": [
"Effect": "Allow",
"Principal":
"Federated": "arn:aws:iam::$AWS_ACCOUNT_ID:oidc-provider/$OIDC_PROVIDER"
,
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition":
"StringEquals":
"$OIDC_PROVIDER:sub": "system:serviceaccount:$ACK_K8S_NAMESPACE:$ACK_K8S_SERVICE_ACCOUNT_NAME"
]
}
EOF
echo "$TRUST_RELATIONSHIP" > trust_lambda.json
Create the IAM function:
ACK_CONTROLLER_IAM_ROLE="ack-lambda-controller"
ACK_CONTROLLER_IAM_ROLE_DESCRIPTION="IRSA function for ACK lambda controller deployment on EKS cluster utilizing Helm charts"
aws iam create-role --role-name "$ACK_CONTROLLER_IAM_ROLE" --assume-role-policy-document file://trust_lambda.json --description "$ACK_CONTROLLER_IAM_ROLE_DESCRIPTION"
Connect IAM coverage to the IAM function:
# we're getting the coverage instantly from the ACK repo
INLINE_POLICY="$(curl https://uncooked.githubusercontent.com/aws-controllers-k8s/lambda-controller/essential/config/iam/recommended-inline-policy)"
aws iam put-role-policy
--role-name "$ACK_CONTROLLER_IAM_ROLE"
--policy-name "ack-recommended-policy"
--policy-document "$INLINE_POLICY"
Connect ECR permissions to the controller IAM function: these are required since Lambda features might be pulling photos from ECR. Be certain to replace the ecr-permissions.json file with the AWS_ACCOUNT_ID and AWS_REGION variables.
aws iam put-role-policy
--role-name "$ACK_CONTROLLER_IAM_ROLE"
--policy-name "ecr-permissions"
--policy-document file://ecr-permissions.json
Affiliate the IAM function to a Kubernetes service account:
ACK_CONTROLLER_IAM_ROLE_ARN=$(aws iam get-role --role-name=$ACK_CONTROLLER_IAM_ROLE --query Position.Arn --output textual content)
export IRSA_ROLE_ARN=eks.amazonaws.com/role-arn=$ACK_CONTROLLER_IAM_ROLE_ARN
kubectl annotate serviceaccount -n $ACK_K8S_NAMESPACE $ACK_K8S_SERVICE_ACCOUNT_NAME $IRSA_ROLE_ARN
Repeat the steps for the API Gateway controller.
ACK API Gateway Controller
Set the next setting variables:
ACK_K8S_SERVICE_ACCOUNT_NAME=ack-apigatewayv2-controller
ACK_K8S_NAMESPACE=ack-system
AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output textual content)
Create the belief coverage for the IAM function:
learn -r -d '' TRUST_RELATIONSHIP <<EOF
{
"Model": "2012-10-17",
"Assertion": [
"Effect": "Allow",
"Principal":
"Federated": "arn:aws:iam::$AWS_ACCOUNT_ID:oidc-provider/$OIDC_PROVIDER"
,
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition":
"StringEquals":
"$OIDC_PROVIDER:sub": "system:serviceaccount:$ACK_K8S_NAMESPACE:$ACK_K8S_SERVICE_ACCOUNT_NAME"
]
}
EOF
echo "$TRUST_RELATIONSHIP" > trust_apigatewayv2.json
Create the IAM function:
ACK_CONTROLLER_IAM_ROLE="ack-apigatewayv2-controller"
ACK_CONTROLLER_IAM_ROLE_DESCRIPTION="IRSA function for ACK apigatewayv2 controller deployment on EKS cluster utilizing Helm charts"
aws iam create-role --role-name "$ACK_CONTROLLER_IAM_ROLE" --assume-role-policy-document file://trust_apigatewayv2.json --description "$ACK_CONTROLLER_IAM_ROLE_DESCRIPTION"
Connect managed IAM insurance policies to the IAM function:
aws iam attach-role-policy --role-name "$ACK_CONTROLLER_IAM_ROLE" --policy-arn arn:aws:iam::aws:coverage/AmazonAPIGatewayAdministrator
aws iam attach-role-policy --role-name "$ACK_CONTROLLER_IAM_ROLE" --policy-arn arn:aws:iam::aws:coverage/AmazonAPIGatewayInvokeFullAccess
Affiliate the IAM function to a Kubernetes service account:
ACK_CONTROLLER_IAM_ROLE_ARN=$(aws iam get-role --role-name=$ACK_CONTROLLER_IAM_ROLE --query Position.Arn --output textual content)
export IRSA_ROLE_ARN=eks.amazonaws.com/role-arn=$ACK_CONTROLLER_IAM_ROLE_ARN
kubectl annotate serviceaccount -n $ACK_K8S_NAMESPACE $ACK_K8S_SERVICE_ACCOUNT_NAME $IRSA_ROLE_ARN
Repeat the steps for the DynamoDB controller.
ACK DynamoDB Controller
Set the next setting variables:
ACK_K8S_SERVICE_ACCOUNT_NAME=ack-dynamodb-controller
ACK_K8S_NAMESPACE=ack-system
AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output textual content)
Create the belief coverage for the IAM function:
learn -r -d '' TRUST_RELATIONSHIP <<EOF
{
"Model": "2012-10-17",
"Assertion": [
"Effect": "Allow",
"Principal":
"Federated": "arn:aws:iam::$AWS_ACCOUNT_ID:oidc-provider/$OIDC_PROVIDER"
,
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition":
"StringEquals":
"$OIDC_PROVIDER:sub": "system:serviceaccount:$ACK_K8S_NAMESPACE:$ACK_K8S_SERVICE_ACCOUNT_NAME"
]
}
EOF
echo "$TRUST_RELATIONSHIP" > trust_dynamodb.json
Create the IAM function:
ACK_CONTROLLER_IAM_ROLE="ack-dynamodb-controller"
ACK_CONTROLLER_IAM_ROLE_DESCRIPTION="IRSA function for ACK dynamodb controller deployment on EKS cluster utilizing Helm charts"
aws iam create-role --role-name "$ACK_CONTROLLER_IAM_ROLE" --assume-role-policy-document file://trust_dynamodb.json --description "$ACK_CONTROLLER_IAM_ROLE_DESCRIPTION"
Connect IAM coverage to the IAM function:
# for dynamodb controller, we use the managed coverage ARN as an alternative of the inline coverage (like we did for Lambda controller)
POLICY_ARN="$(curl https://uncooked.githubusercontent.com/aws-controllers-k8s/dynamodb-controller/essential/config/iam/recommended-policy-arn)"
aws iam attach-role-policy --role-name "$ACK_CONTROLLER_IAM_ROLE" --policy-arn "$POLICY_ARN"
Affiliate the IAM function to a Kubernetes service account:
ACK_CONTROLLER_IAM_ROLE_ARN=$(aws iam get-role --role-name=$ACK_CONTROLLER_IAM_ROLE --query Position.Arn --output textual content)
export IRSA_ROLE_ARN=eks.amazonaws.com/role-arn=$ACK_CONTROLLER_IAM_ROLE_ARN
kubectl annotate serviceaccount -n $ACK_K8S_NAMESPACE $ACK_K8S_SERVICE_ACCOUNT_NAME $IRSA_ROLE_ARN
Restart ACK Controller Deployments and Confirm the Setup
Restart the ACK service controller Deployment utilizing the next instructions. It would replace the service controller Pods with IRSA setting variables. Get the record of ACK service controller deployments:
export ACK_K8S_NAMESPACE=ack-system
kubectl get deployments -n $ACK_K8S_NAMESPACE
Restart Lambda, API Gateway, and DynamoDB Deployments:
DEPLOYMENT_NAME_LAMBDA=<enter deployment title for lambda controller>
kubectl -n $ACK_K8S_NAMESPACE rollout restart deployment $DEPLOYMENT_NAME_LAMBDA
DEPLOYMENT_NAME_APIGW=<enter deployment title for apigw controller>
kubectl -n $ACK_K8S_NAMESPACE rollout restart deployment $DEPLOYMENT_NAME_APIGW
DEPLOYMENT_NAME_DYNAMODB=<enter deployment title for dynamodb controller>
kubectl -n $ACK_K8S_NAMESPACE rollout restart deployment $DEPLOYMENT_NAME_DYNAMODB
Checklist Pods for these Deployments. Confirm that the AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN setting variables exist to your Kubernetes Pod utilizing the next instructions:
kubectl get pods -n $ACK_K8S_NAMESPACE
LAMBDA_POD_NAME=<enter Pod title for lambda controller>
kubectl describe pod -n $ACK_K8S_NAMESPACE $LAMBDA_POD_NAME | grep "^s*AWS_"
APIGW_POD_NAME=<enter Pod title for apigw controller>
kubectl describe pod -n $ACK_K8S_NAMESPACE $APIGW_POD_NAME | grep "^s*AWS_"
DYNAMODB_POD_NAME=<enter Pod title for dynamodb controller>
kubectl describe pod -n $ACK_K8S_NAMESPACE $DYNAMODB_POD_NAME | grep "^s*AWS_"
Now that the ACK service controller has been arrange and configured, you’ll be able to create AWS assets!
Create API Gateway Assets, DynamoDB desk, and Deploy the Lambda Operate
Create API Gateway Assets
Within the file apigw-resources.yaml, exchange the AWS account ID within the integrationURI attribute for the Integration useful resource. That is what the ACK manifest for API Gateway assets (API, Integration, and Stage) seems like:
apiVersion: apigatewayv2.companies.k8s.aws/v1alpha1
sort: API
metadata:
title: ack-demo-apigw-httpapi
spec:
title: ack-demo-apigw-httpapi
protocolType: HTTP
---
apiVersion: apigatewayv2.companies.k8s.aws/v1alpha1
sort: Integration
metadata:
title: ack-demo-apigw-integration
spec:
apiRef:
from:
title: ack-demo-apigw-httpapi
integrationType: AWS_PROXY
integrationMethod: POST
integrationURI: arn:aws:lambda:us-east-1:AWS_ACCOUNT_ID:perform:demo-dynamodb-func-ack
payloadFormatVersion: "2.0"
---
apiVersion: apigatewayv2.companies.k8s.aws/v1alpha1
sort: Stage
metadata:
title: demo-stage
spec:
apiRef:
from:
title: ack-demo-apigw-httpapi
stageName: demo-stage
autoDeploy: true
description: "demo stage for http api"
Create the API Gateway assets (API, Integration, and Stage) utilizing the next command:
kubectl apply -f apigw-resources.yaml
Create DynamoDB Desk
That is what the ACK manifest for the DynamoDB desk seems like:
apiVersion: dynamodb.companies.k8s.aws/v1alpha1
sort: Desk
metadata:
title: consumer
annotations:
companies.k8s.aws/area: us-east-1
spec:
attributeDefinitions:
- attributeName: e-mail
attributeType: S
billingMode: PAY_PER_REQUEST
keySchema:
- attributeName: e-mail
keyType: HASH
tableName: consumer
You’ll be able to exchange the us-east-1 area along with your most popular area.
Create a desk (named consumer) utilizing the next command:
kubectl apply -f dynamodb-table.yaml
# record the tables
kubectl get tables
Construct Operate Binary and Create Docker Picture
GOARCH=amd64 GOOS=linux go construct -o essential essential.go
aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws
docker construct -t demo-apigw-dynamodb-func-ack .
Create a personal ECR repository, tag, and push the Docker picture to ECR:
AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output textual content)
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com
aws ecr create-repository --repository-name demo-apigw-dynamodb-func-ack --region us-east-1
docker tag demo-apigw-dynamodb-func-ack:newest $AWS_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/demo-apigw-dynamodb-func-ack:newest
docker push $AWS_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/demo-apigw-dynamodb-func-ack:newest
Create an IAM execution Position for the Lambda perform and fix the required insurance policies:
export ROLE_NAME=demo-apigw-dynamodb-func-ack-role
ROLE_ARN=$(aws iam create-role
--role-name $ROLE_NAME
--assume-role-policy-document '"Model": "2012-10-17","Assertion": [ "Effect": "Allow", "Principal": "Service": "lambda.amazonaws.com", "Action": "sts:AssumeRole"]'
--query 'Position.[Arn]' --output textual content)
aws iam attach-role-policy --role-name $ROLE_NAME --policy-arn arn:aws:iam::aws:coverage/service-role/AWSLambdaBasicExecutionRole
Because the Lambda perform wants to put in writing information to DynamoDB, let’s add the next coverage to the IAM function:
aws iam put-role-policy
--role-name "$ROLE_NAME"
--policy-name "dynamodb-put"
--policy-document file://dynamodb-put.json
Create the Lambda Operate
Replace perform.yaml file with the next information:
imageURI– The URI of the Docker picture that you just pushed to ECR; e.g.,<AWS_ACCOUNT_ID>.dkr.ecr.us-east-1.amazonaws.com/demo-apigw-dynamodb-func-ack:newestfunction– The ARN of the IAM function that you just created for the Lambda perform; e.g.,arn:aws:iam::<AWS_ACCOUNT_ID>:function/demo-apigw-dynamodb-func-ack-role
That is what the ACK manifest for the Lambda perform seems like:
apiVersion: lambda.companies.k8s.aws/v1alpha1
sort: Operate
metadata:
title: demo-apigw-dynamodb-func-ack
annotations:
companies.k8s.aws/area: us-east-1
spec:
architectures:
- x86_64
title: demo-apigw-dynamodb-func-ack
packageType: Picture
code:
imageURI: AWS_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/demo-apigw-dynamodb-func-ack:newest
setting:
variables:
TABLE_NAME: consumer
function: arn:aws:iam::AWS_ACCOUNT_ID:function/demo-apigw-dynamodb-func-ack-role
description: A perform created by ACK lambda-controller
To create the Lambda perform, run the next command:
kubectl create -f perform.yaml
# record the perform
kubectl get features
Add API Gateway Set off Configuration
Right here is an instance utilizing AWS Console. Open the Lambda perform within the AWS Console and click on on the Add set off button. Choose API Gateway because the set off supply, choose the prevailing API, and click on on the Add button.
Now you might be able to check out the end-to-end answer!
Take a look at the Software
Get the API Gateway endpoint:
export API_NAME=ack-demo-apigw-httpapi
export STAGE_NAME=demo-stage
export URL=$(kubectl get api/"$API_NAME" -o=jsonpath=".standing.apiEndpoint")/"$STAGE_NAME"/demo-apigw-dynamodb-func-ack"
Invoke the API Gateway endpoint:
curl -i -X POST -H 'Content material-Sort: utility/json' -d '"e-mail":"[email protected]","title":"user1"' $URL
curl -i -X POST -H 'Content material-Sort: utility/json' -d '"e-mail":"[email protected]","title":"user2"' $URL
curl -i -X POST -H 'Content material-Sort: utility/json' -d '"e-mail":"[email protected]","title":"user3"' $URL
curl -i -X POST -H 'Content material-Sort: utility/json' -d '"e-mail":"[email protected]","title":"user4"' $URL
The Lambda perform needs to be invoked and the info needs to be written to the DynamoDB desk. Examine the DynamoDB desk utilizing the CLI (or AWS console):
aws dynamodb scan --table-name consumer
Clear Up
After you’ve got explored the answer, you’ll be able to clear up the assets by operating the next instructions:
Delete API Gateway assets, DynamoDB desk, and the Lambda perform:
kubectl delete -f apigw-resources.yaml
kubectl delete -f perform.yaml
kubectl delete -f dynamodb-table.yaml
To uninstall the ACK service controllers, run the next instructions:
export ACK_SYSTEM_NAMESPACE=ack-system
helm ls -n $ACK_SYSTEM_NAMESPACE
helm uninstall -n $ACK_SYSTEM_NAMESPACE <enter title of the apigw chart>
helm uninstall -n $ACK_SYSTEM_NAMESPACE <enter title of the lambda chart>
helm uninstall -n $ACK_SYSTEM_NAMESPACE <enter title of the dynamodb chart>
Conclusion and Subsequent Steps
On this submit, we’ve seen use AWS Controllers for Kubernetes to create a Lambda perform, API Gateway integration, and DynamoDB desk and wire them collectively to deploy an answer. All of this (nearly) was executed utilizing Kubernetes! I encourage you to check out different AWS companies supported by ACK : here is a complete list.
Comfortable constructing!
