ALB Controller

aws alb controller (application load balance controller)

์ฟ ๋ฒ„๋„คํ‹ฐ์Šค๋ฅผ ์ง์ ‘ ์„ค์น˜ํ•˜๊ณ  ์‚ฌ์šฉํ• ๋•Œ๋Š” ingress-nginx๋ฅผ ์‚ฌ์šฉํ•˜์˜€์œผ๋‚˜ eks์—๋Š” aws (alb) application load balance๋ฅผ ์‚ฌ์šฉํ• ์ˆ˜ ์žˆ๋Š” ๋ฐฉ๋ฒ•์ด ์žˆ๋‹ค.

์ฒ˜์Œ ๊ณ ๋ฏผ์ด aws์—์„œ ๋กœ๋“œ๋ฐœ๋ž€์Šค๋ฅผ ์„ธํŒ…ํ•˜๋Š”๊ฒŒ ๋ฒˆ๊ฑฐ๋กญ๋‹ค๋Š” ๊ณ ๋ฏผ์ด ์žˆ์—‡๋Š”๋ฐ ๊ทธ๊ฑธ aws์—์„œ ์•Œ๊ณ  ์žˆ์—‡๋Š”์ง€ kubernete ์„ค์ •ํŒŒ์ผ์— ์ ์–ด๋งŒ ์ฃผ๋ฉด ์ž๋™์œผ๋กœ alb๊ฐ€ ์ƒ์„ฑ์ด ๋œ๋‹ค.

ALB Controller๋ฅผ ์„ค์น˜๋ฅผ ํ•ด๋‘๋ฉด ์ฟ ๋ฒ„๋„คํ‹ฐ์Šค์— ์„ค์ •์„ ํ•˜๋ฉด ์ด ์ปจํŠธ๋กค๋Ÿฌ๊ฐ€ ALB๋ฅผ ์ž๋™์œผ๋กœ ๋“ฑ๋กํ•ด์ฃผ๋Š” ๊ฒƒ์ด๋‹ค.

์ด ์ปจํŠธ๋กค๋Ÿฌ๊ฐ€ alb์— ์ ‘์†์ด ๊ฐ€๋Šฅํ•˜๊ฒŒ ๋˜์•ผ๋Š”๋ฐ ์ด๊ฑธ oidc๋ฅผ ์‚ฌ์šฉํ•œ๋‹ค.(์ธ์ฆ)

ALB๋Š” nodeport ๋‚˜ loadbalance๋งŒ ์ง€์›์„ ํ•œ๋‹ค.(์ค‘์š”)

๋งํฌ์— ์žˆ๋Š” ๋‚ด์šฉ์„ ํ•ด์ฃผ๋ฉด ๋œ๋‹ค. ๊ฐ„๋‹จํ•˜๊ฒŒ ์š”์•ฝํ•ด๋ณด๋ฉด

OIDC

https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html

OIDC Issuer

๊ธฐ๋ณธ์ ์œผ๋กœ eksctl๋กœ ์ƒ์„ฑํ•˜๋ฉด ์ด๊ฑด ๊ธฐ๋ณธ๊ฐ’์ด ๊ฐ™์ด ์ƒ๊น€

aws eks describe-cluster --name cluster01 --query "cluster.identity.oidc.issuer" --output text
> https://oidc.eks.us-west-1.amazonaws.com/id/295F23831974F59E6DF049E7284078A6

OIDC Provider

aws iam list-open-id-connect-providers | grep 295F23831974F59E6DF049E7284078A6

์•„๋ฌด๊ฒƒ๋„ ์•ˆ๋‚˜์˜จ๋‹ค. ์—†๋‹ค๋Š”๊ฑฐ๋‹ค ๊ทธ๋Ÿฌ๋ฉด ์ƒ์„ฑ ํ•ด์ค˜์•ผ ํ•œ๋‹ค. ์žˆ์œผ๋ฉด ์ƒ์„ฑ ๋ถ€๋ถ„์„ ๋„˜์–ด๊ฐ€๋ฉด ๋œ๋‹ค.

์—†์œผ๋ฉด ์ƒ์„ฑ

Create an IAM OIDC provider for your cluster

eksctl utils associate-iam-oidc-provider \
    --region us-west-1 \
    --cluster cluster01 \
    --approve

๋‚ด์šฉ ํ™•์ธ

aws iam list-open-id-connect-providers | grep 295F231974F59E6DF049E7284078A6
> - Arn: arn:aws:iam::530310009353:oidc-provider/oidc.eks.us-west-1.amazonaws.com/id/295F23831974F59E6DF049E7284078A6

๋‚ด์šฉ์ด ์žˆ๋‹ค. oidc provider๋Š” ๋งŒ๋“ค์–ด์กŒ๋‹ค.

์›น์‚ฌ์ดํŠธ์—์„œ๋„ ์ƒ์„ฑ ํ™•์ธ ๊ฐ€๋Šฅ

https://console.aws.amazon.com/iamv2/home#/identity_providers

ALB Controller Install

Create an IAM policy

# download
curl -o iam_policy.json https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.2.0/docs/install/iam_policy.json

aws iam create-policy \
  --policy-name AWSLoadBalancerControllerIAMPolicy \
  --policy-document file://iam_policy.json

arn์„ ๋ณต์‚ฌํ•ด์„œ ๋ณด๊ด€ํ•ด๋‘”๋‹ค.

Policy:
  Arn: arn:aws:iam::530310009353:policy/AWSLoadBalancerControllerIAMPolicy
  AttachmentCount: 0
  CreateDate: '2021-06-02T22:27:30+00:00'
  DefaultVersionId: v1
  IsAttachable: true
  Path: /
  PermissionsBoundaryUsageCount: 0
  PolicyId: ANPAXW6HU27ETIAOLPJGG
  PolicyName: AWSLoadBalancerControllerIAMPolicy
  UpdateDate: '2021-06-02T22:27:30+00:00'

์›น์‚ฌ์ดํŠธ์—์„œ ํ™•์ธ

https://console.aws.amazon.com/iam/home#/policies

AWSLoadBalancerControllerIAMPolicy๋กœ ๊ฒ€์ƒ‰ํ•ด๋ณด๋ฉด ์ƒ์„ฑ๋œ ๊ฒƒ์„ ์•Œ์ˆ˜ ์žˆ๋‹ค.

create Role

  • Open the IAM console at https://console.aws.amazon.com/iam/

  • role > create role

  • trusted entity > Web identity

  • permissions

  • Attach Policy section > AWSLoadBalancerControllerIAMPolicy

  • tags > review >

  • Role Name : AmazonEKSLoadBalancerControllerRole > create role

    ์ƒ์„ฑ๋œ๊ฑฐ ํ™•์ธ

  • After the role is created, choose the role in the console to open it for editing

  • Trust relationships > Edit trust relationship

  • ๋‹ค์Œ ๋ถ€๋ถ„์„ ์ˆ˜์ •

  • ๋‹ค์Œ ์ฝ”๋“œ๋กœ ๋ณ€๊ฒฝ

    sub": "system:serviceaccount:kube-system:aws-load-balancer-controller"

  • Update Trust Policy

  • role arn์„ ๋ณต์‚ฌํ•ด๋‘”๋‹ค. arn:aws:iam::530310009353:role/AmazonEKSLoadBalancerControllerRole

aws-load-balancer-controller-service-account.yaml
```yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/name: aws-load-balancer-controller
  name: aws-load-balancer-controller
  namespace: kube-system
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::530310009353:role/AmazonEKSLoadBalancerControllerRole
```
role-arn ์„ ๋ณต์‚ฌํ•ด๋‘”๊ฑธ๋กœ ๋ฎ์–ด์“ด๋‹ค.
  • create service account kubectl apply -f aws-load-balancer-controller-service-account.yaml

controller ์„ค์น˜

ํ˜„์žฌ alb controller๊ฐ€ ์žˆ๋Š”์ง€ ํ™•์ธํ•œ๋‹ค. ์—†์–ด์•ผ ํ•œ๋‹ค. ์žˆ์œผ๋ฉด ์ง€์šด๋‹ค.

kubectl get deployment -n kube-system alb-ingress-controller
> Error from server (NotFound): deployments.apps "alb-ingress-controller" not found

https://github.com/kubernetes-sigs/aws-load-balancer-controller ์—์„œ ์ตœ์‹  ๋ฆด๋ฆฌ์ฆˆ๋ฅผ ํ™•์ธํ•œ์ˆ˜ ๋ฒ„์ „๋“ฑ์€ ์ˆ˜์ •ํ•ด๋ผ.

cert-manager๊ฐ€ ๋””ํŽœ๋˜์‹œ๊ฐ€ ๊ฑธ๋ ค์žˆ๋‹ค. ๊ฐ™์ด ์„ค์น˜ํ•˜์ž.

kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.3.1/cert-manager.yaml
curl -o v2_2_0_full.yaml https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.2.0/docs/install/v2_2_0_full.yaml

ํŒŒ์ผ์„ ์ˆ˜์ •ํ•˜์ž.

ServiceAccount ์‚ญ์ œ

cluster name๋ณ€๊ฒฝ

kubectl apply -f v2_2_0_full.yaml

ํ™•์ธ

kubectl get deployment -n kube-system aws-load-balancer-controller
kubectl logs deploy/aws-load-balancer-controller -n kube-system

์•„์›ƒํ’‹์ด ๋‚˜์˜ค๋ฉด ์ž˜ ๋œ๊ฒƒ์ด๋‹ค.

์—๋Ÿฌ๊ฐ€ ๋‚˜์˜ค๋ฉด ๊ถŒํ•œ๋ถ€๋ถ„์„ ๋‹ค์‹œ ํ•ด๋ณด๋„๋ก ํ•˜์ž.

๋กœ๊ทธ ํ™•์ธ

์ž˜ ์•ˆ๋˜๋ฉด ๋กœ๊ทธ๋ฅผ ํ™•์ธํ•ด๋ด์•ผํ•œ๋‹ค.

kubectl logs  aws-load-balancer-controller-7d7f98596-rg8wf -n kube-system
> {"level":"error","ts":1622646021.3727376,"logger":"controller","msg":"Reconciler error","controller":"ingress","name":"www","namespace":"default","error":"couldn't auto-discover subnets: UnauthorizedOperation: You are not authorized to perform this operation.\n\tstatus code: 403, request id: 73f7cb4e-c285-4a5a-9068-13e4e6c94f6a"}

์ด๋Ÿฌ๋ฉด Oidc๊ฐ€ ์ž˜ ๋™์ž‘ํ•˜์ง€ ์•Š๋Š” ๊ฒƒ์ด๋‹ค.

version์ด ์•ˆ๋งž์•˜๋‹ค. policy๋Š” 2.1.3 controller๋Š” 2.2.2 ๋กœ ๋˜๋ฒ„๋ ธ๋‹ค.

2.1.3์œผ๋กœ ํ•˜๋ฉด ์ž˜๋œ๋‹ค.

alb controller ์‚ญ์ œ

kubectl delete -f test-deploy.yml
kubectl delete -f aws-load-balancer-controller-service-account.yaml
kubectl delete -f v2_2_0_full.yaml

delete role : AmazonEKSLoadBalancerControllerRole

delete policy : AWSLoadBalancerControllerIAMPolicy

๊ธฐ๋ณธ ingress ์‚ฌ์šฉ๋ฒ•

test-deploy.yml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: www
  namespace: default
  labels:
    app: www
spec:
  replicas: 1
  selector:
    matchLabels:
      app: www
  template:
    metadata:
      labels:
        app: www
    spec:
      containers:
        - name: www
          image: nginx:latest
          ports:
            - containerPort: 80

---
apiVersion: v1
kind: Service
metadata:
  name: www
  namespace: default
  labels:
    app: www
spec:
  type: NodePort
  selector:
    app: www
  ports:
    - name: http
      port: 80
      targetPort: 80

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: www
  namespace: default
  annotations:
    kubernetes.io/ingress.class: 'alb'
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}]'
    alb.ingress.kubernetes.io/scheme: internet-facing # ์ธํ„ฐ๋„ท(public)์—์„œ ์ ‘์†์ด ๋˜๊ฒŒ ํ•œ๋‹ค.
spec:
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: www
                port:
                  number: 80

์ด๊ฑธ ์‚ฌ์šฉํ•˜๋ฉด ์ž๋™์œผ๋กœ aws application load balance๋„ ๋งŒ๋“ค์–ด ์ค€๋‹ค.

kubectl apply -f test-deploy.yml

ec2 -> load balance

dns์— ์ถ”๊ฐ€

route53

http๋ฅผ https๋กœ redirect

์œ„์— ์ƒ˜ํ”Œ์— ์ ํ˜€๋Š” ์žˆ์œผ๋‚˜ ํŠน๋ณ„ํžˆ ๋”ฐ๋กœ ์„ค๋ช…ํ•œ๋‹ค.

anotation์— ๋‹ค์Œ ์ถ”๊ฐ€

alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'

๊ทธ๋ฆฌ๊ณ  path์— ๋‹ค์Œ ์ถ”๊ฐ€

- path: /
  backend:
    service:
      name: ssl-redirect
      port:
        name: use-annotation

์ด๋Ÿฌ๋ฉด http๋กœ ์ ‘๊ทผํ•˜๋ฉด https๋กœ ๋ฆฌ๋‹ค์ด๋ ‰ํŠธ๋ฅผ ์‹œ์ผœ์ค€๋‹ค. ๊ผญ ์ด์„ค์ •์ด ๋งจ์œ„์— ์™€์•ผํ•œ๋‹ค.

๊ด€๋ จ ๋‚ด์šฉ์€ ์—ฌ๊ธฐ๋ฅผ ์ฐธ๊ณ ํ•˜์ž. https://github.com/kubernetes-sigs/aws-load-balancer-controller/blob/main/docs/guide/tasks/ssl_redirect.md

ssl backend

ํŠน์ • pod๋Š” ํ”„๋กœ๊ทธ๋žจ ์ž์ฒด์—์„œ ssl๋กœ ์ ‘๊ทผ์„ ๋ฐ›์•„์•ผํ•  ํ•„์š”๊ฐ€ ์žˆ์„๋•Œ alb controller์—์„œ๋Š” ๋‹ค์Œ์ฒ˜๋Ÿผ ์ฒ˜๋ฆฌํ•œ๋‹ค.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: auth
  namespace: auth-staging
  annotations:
    kubernetes.io/ingress.class: 'alb'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-west-2:xxxx:certificate/199b8e95-fe5b-43e6-b499-061e4f133011
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80},{"HTTPS":443}]'
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
    alb.ingress.kubernetes.io/backend-protocol: HTTPS # ์—ฌ๊ธฐ ์ถ”๊ฐ€
spec:
  rules:
    - host: www.aaa.com
      http:
        paths:
          - path: /
            backend:
              service:
                name: ssl-redirect
                port:
                  name: use-annotation
          - path: /
            backend:
              service:
                name: auth
                port:
                number: 80

anotation์— ๋‹ค์Œ ์ถ”๊ฐ€๋ฅผ ๋ณผ์ˆ˜ ์žˆ๋‹ค.

alb.ingress.kubernetes.io/backend-protocol: HTTPS # ์—ฌ๊ธฐ ์ถ”๊ฐ€

๋งŒ์•ฝ ์ด๊ฑธ ์ถ”๊ฐ€ํ•˜์ง€ ์•Š์œผ๋ฉด ์ด๋Ÿฐ ์—๋Ÿฌ๋ฅผ ๋ณผ์ˆ˜๊ฐ€ ์žˆ๋‹ค.

Getting Handshake failedโ€ฆunexpected packet format

alb๊ฐ€ ๊ธฐ๋ณธ์ ์œผ๋กœ http๋กœ ํ†ต์‹ ์„ ์‹œ๋„ํ•˜๋ฏ€๋กœ ํฌํŠธ๋Š” 443์„ ์“ฐ๋ฉด์„œ http๋ฅผ ๋ณด๋‚ด๊ฒŒ ๋˜๋‹ค๋ณด๋‹ˆ ์ด๋Ÿฐ ์—๋Ÿฌ๊ฐ€ ๋‚˜์˜จ๋‹ค.

health check

๋กœ๋“œ ๋ฐœ๋ž€์Šค๊ฐ€ ๊ธฐ๋ณธ์ ์œผ๋กœ pod๋ฅผ ๋‹ค ์ฒดํฌํ•ด์„œ ์„œ๋น„์Šค๋ฅผ ์œ ์ง€ํ•ด์ค€๋‹ค. ํŠน๋ณ„ํžˆ health check ๊ฒฝ๋กœ๋ฅผ ์ˆ˜์ •ํ•˜๋ ค๋ฉด ๋‹ค์Œ์ฒ˜๋Ÿผ ํ•˜์ž.

alb.ingress.kubernetes.io/healthcheck-protocol: HTTPS #๊ธฐ๋ณธ๊ฐ’ http
alb.ingress.kubernetes.io/healthcheck-path: /health
alb.ingress.kubernetes.io/healthcheck-interval-seconds: '60'

pod๊ฐ€ ssl์„ ๊ธฐ๋Œ€ํ•˜๊ณ  ์žˆ์œผ๋ฉด healthcheck-protocol๋„ ๋งž๋Š”๊ฐ’์„ ๋„ฃ์–ด์ค˜์•ผํ•œ๋‹ค.

ELB target group์— ๊ฐ€๋ฉด ์œ„ ๋‚ด์šฉ์„ ์•ˆ๋„ฃ๋”๋ผ๋„ ๊ธฐ๋ณธ์œผ๋กœ healthcheck๊ฐ€ ์ƒ์„ฑ์ด ๋œ๋‹ค. ๊ธฐ๋ณธ๊ฐ’์ด ์žˆ๊ธฐ ๋•Œ๋ฌธ์ด๋‹ค.

nginx app์„ alb์— ์˜คํ”ˆ

ํ•œ๋ฒˆ๋” ๋ชจ๋“ ๊ฑธ ์ ์šฉํ•ด์„œ alb๋ฅผ ์‚ฌ์šฉํ•ด๋ณด์ž

---
apiVersion: v1
kind: Service
metadata:
  name: www
  namespace: default
  labels:
    app: www
spec:
  type: NodePort
  selector:
    app: www
  ports:
    - name: http
      port: 80
      targetPort: 80

์ ์šฉํ•˜๋ฉด alb๊ฐ€ ์ƒ๊ธฐ๋Š”๊ฒƒ์„ aws console ์—์„œ ๋ณผ ์ˆ˜ ์žˆ๋‹ค.

  • ssl๋„ ์ ์šฉํ–‡๋‹ค. cert-arn์€ certificate-manager์— ๊ฐ€์„œ ๋งŒ๋“ค๋ฉด ์ƒ๊ธด๋‹ค. ๊ทธ๊ฑธ ์‚ฌ์šฉ

  • ssl redirect ์ ์šฉ ์™„๋ฃŒ

  • internet-facing : ํ•„์ˆ˜์ด๋‹ค.

  • ํฌํŠธ๋Š” 80 443์€ ๋‘˜๋‹ค ์—ด์–ด์ฃผ๋ฉด ์ข‹๋‹ค.

multiple domain and ssl ์ ์šฉ

rules์— ์—ฌ๋Ÿฌ๊ฐœ์˜ ๋„๋ฉ”์ธ์„ ์ถ”๊ฐ€ํ•œ๋‹ค. ๊ทธ๋ฆฌ๊ณ  ssl์„ ์ปด๋งˆ๋กœ ์—ฐ๊ฒฐํ•œ๋‹ค.

multi-ssl
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: www
  namespace: default
  annotations:
    ...
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-west-2:YOURACCOUNT:certificate/a2eb12f7-7e36-4d50-811c-8bxxxxx7,arn:aws:acm:us-west-2:YOURACCOUNT:certificate/a2eb12f7-7e36-4d50-811c-8xxxxxxx #comma๋กœ ์—ฐ๊ฒฐ
spec:
  rules:
    - host: www.aaa.com
      http:
        paths:
          - path: /*
            backend:
              serviceName: aaa
              servicePort: 80
    - host: www.bbb.com
      http:
        paths:
          - path: /*
            backend:
              serviceName: bbb
              servicePort: 80

์—ฌ๋Ÿฌ๊ฐœ์˜ ์ธ๊ทธ๋ ˆ์Šค์—์„œ albํ•˜๋‚˜๋ฅผ ๊ณต์œ ํ•˜๊ธฐ

alb.ingress.kubernetes.io/group.name: shared-ingress

Ingress๊ฐ€ ๋‹ค ๊ฐ๊ฐ์˜ name space์— ์ƒ๊ธฐ๋Š”๊ฒƒ์€ ๋งž๋‹ค.

๊ทธ๋ฆฌ๊ณ  ๊ทธ๊ฒƒ๋“ค์ด ํ•˜๋‚˜์˜ ๋กœ๋“œ๋ฐœ๋ž€์Šค๋ฅผ ์‚ฌ์šฉํ•œ๋‹ค.

Last updated

Was this helpful?