๐Ÿ“—
smiley book
  • Smiley Books
  • AI
    • Readme
    • openai-whisper
      • ์ƒ˜ํ”Œ ์‹คํ–‰ํ•ด๋ณด๊ธฐ
      • GPU ์„œ๋ฒ„ ์ค€๋น„ํ•˜๊ธฐ
      • API๋กœ whisper๋ฅผ ์™ธ๋ถ€์— ์˜คํ”ˆํ•˜๊ธฐ
      • ํ”„๋กฌํ”„ํŠธ ์ง€์›
      • ์‹ค์‹œ๊ฐ„ message chat
      • ํ™”๋ฉด ์ด์˜๊ฒŒ ๋งŒ๋“ค๊ธฐ์™€ ๋กœ๊ทธ์ธ
      • ํŒŒ์ด์ฌ ๊ฐ€์ƒํ™˜๊ฒฝ
      • ์‹ค์‹œ๊ฐ„ voice chat
      • fine tunning(๋ฏธ์„ธ ์กฐ์ •) ์œผ๋กœ ์„ฑ๋Šฅ ์˜ฌ๋ฆฌ๊ธฐ
      • app์—์„œ api๋ฅผ ํ˜ธ์ถœํ•˜์—ฌ ์‹ค์‹œ๊ฐ„์œผ๋กœ ํ…์ŠคํŠธ๋กœ ๋ฐ”๊ฟ”๋ณด๊ธฐ
    • ollama - llm์„ ์‰ฝ๊ฒŒ ๋‚ด์ปด์—์„œ ์‹คํ–‰
      • ollama webui
      • ollama docker
    • stable diffusion
      • SDXL - text to image
      • SD-webui
    • ChatGPT
      • ๋‹ต๋ณ€์ด ๋Š๊ธธ๋•Œ
      • ์—ญํ• ์„ ์ •ํ•˜์ž
      • ๊ตฌ์ฒด์ ์ธ ์งˆ๋ฌธ
      • ๊ฒฐ๊ณผํ˜•ํƒœ๋ฅผ ์ง€์ •
      • ํ”„๋กฌํ”„ํŠธ๋ฅผ ์—ฌ๋Ÿฌ์ค„๋กœ ์‚ฌ์šฉํ•˜์ž.
      • ๋งˆํ‹ด ํŒŒ์šธ๋Ÿฌ ๊ธ€ ๋ฒˆ์—ญ๋ณธ
    • Prompt Engineering
    • Auto-GPT
    • Gemini
      • google ai studio
      • gemini-api
      • embedding guide
    • Huggingface
      • huggingface ์‚ฌ์šฉ๋ฒ•
      • huggingface nlp ๊ณต๋ถ€์ค‘
    • kaggle
      • download dataset
    • langchain
      • langchain์„ ๊ณต๋ถ€ํ•˜๋ฉฐ ์ •๋ฆฌ
      • basic
      • slackbot
      • rag
      • document-loader
      • website-loader
      • confluence
      • memory
      • function-call
      • langsmith
      • agent-toolkit
  • Ansible
    • templates vs files and jinja2
    • dynamic inventory
    • limit ์˜ต์…˜ ๊ฐ•์ œํ•˜๊ธฐ
    • limit ์‚ฌ์šฉํ›„ gather_fact ๋ฌธ์ œ
  • AWS
    • AWS CLI
    • EKS
      • cluster manage
      • ALB Controller
      • external-dns
      • fargate
    • ECR
    • S3
    • Certificate Manager
  • Azure
    • Azure AD OAuth Client Flow
  • Container
    • Registry
    • ๋นŒ๋“œ์‹œ์— env๊ฐ’ ์„ค์ •ํ•˜๊ธฐ
  • DB
    • PXC
      • Operator
      • PMM
      • ์‚ญ์ œ
      • GTID
      • Cross Site Replication
    • Mssql
    • Mysql
  • dotnet
    • Thread Pool
    • Connection Pool
    • Thread Pool2
  • Devops
    • Recommendation
  • GIT
    • Basic
    • Submodule
  • GitHub
    • Repository
    • GitHub Action
    • GitHub PR
    • Self Hosted Runner
    • GitHub Webhook
  • GitLab
    • CI/CD
    • CI/CD Advance
    • Ssl renew
    • CI/CD Pass env to other job
  • Go Lang
    • ๊ฐœ๋ฐœ ํ™˜๊ฒฝ ๊ตฌ์ถ•
    • multi os binary build
    • kubectl๊ฐ™์€ cli๋งŒ๋“ค๊ธฐ
    • azure ad cli
    • embed static file
    • go study
      • pointer
      • module and package
      • string
      • struct
      • goroutine
  • Kubernetes
    • Kubernetes๋Š” ๋ฌด์—‡์ธ๊ฐ€
    • Tools
    • Install with kubespray
    • Kubernetes hardening guidance
    • 11 ways not to get hacked
    • ArgoCD
      • Install
      • CLI
      • Repository
      • Apps
      • AWS ALB ์‚ฌ์šฉ
      • Notification slack
      • Backup / DR
      • Ingress
      • 2021-11-16 Github error
      • Server Config
      • auth0 ์ธ์ฆ ์ถ”๊ฐ€(oauth,OIDC)
    • Extension
      • Longhorn pvc
      • External dns
      • Ingress nginx
      • Cert Manager
      • Kube prometheus
    • Helm
      • Subchart
      • Tip
    • Loki
    • Persistent Volume
    • TIP
      • Job
      • Pod
      • Log
  • KAFKA
    • raft
  • KVM
    • kvm cpu model
  • Linux
    • DNS Bind9
      • Cert-Manager
      • Certbot
      • Dynamic Update
      • Log
    • Export and variable
    • Grep ์‚ฌ์šฉ๋ฒ•
  • Modeling
    • C4 model introduce
    • Mermaid
    • reference
  • Monitoring
    • Readme
    • 0. What is Monitoring
    • 1. install prometheus and grafana
    • 2. grafana provisioning
    • 3. grafana dashboard
    • 4. grafana portable dashboard
    • 5. prometheus ui
    • 6. prometheus oauth2
    • Prometheus
      • Metric type
      • basic
      • rate vs irate
      • k8s-prometheus
    • Grafana
      • Expolorer
    • Node Exporter
      • advance
      • textfile collector
  • Motivation
    • 3 Simple Rule
  • OPENNEBULA
    • Install(ansible)
    • Install
    • Tip
    • Windows vm
  • Reading
    • comfort zone
    • ๋ฐฐ๋ ค
    • elon musk 6 rule for insane productivity
    • Feynman Technique
    • how to interview - elon musk
    • ๊ฒฝ์ฒญ
    • Readme
  • Redis
    • Install
    • Master-slave Architecture
    • Sentinel
    • Redis Cluster
    • Client programming c#
  • SEO
    • Readme
  • Security
    • criminalip.io
      • criminalip.io
  • Stock
    • robinhood-python
  • Terraform
    • moved block
    • output
  • vault
    • Readme
  • VS Code
    • dev container
    • dev container on remote server
  • Old fashione trend
    • curity
    • MAAS
      • Install maas
      • Manage maas
      • Tip
Powered by GitBook
On this page
  • https://github.com/teamsmiley/devops-senima-argocd
  • vm ์ค€๋น„
  • architechture
  • prepare install
  • set custom k8s-cluser.yml
  • review setup-kubespray.sh
  • sh ์„ค๋ช…
  • Run setup haproxy for Kubernetes masters
  • vip ํ™•์ธ
  • keepalived ํ™•์ธ
  • kubernetes ์„ค์น˜
  • ๋ฌธ์ œ๋“ค
  • ์ƒํƒœ ํ™•์ธ
  • ์ ‘์† ์„ค์ • ์ •๋ณด ๊ฐ€์ ธ์˜ค๊ธฐ
  • remove all setup
  • argocd ์„ค์น˜
  • ๋น„๋ฒˆ ์•Œ์•„๋‚ด๊ธฐ
  • ์ ‘์†
  • create repo for add
  • add argocd repo to argocd
  • add core / apps

Was this helpful?

  1. Kubernetes

Install with kubespray

PreviousToolsNextKubernetes hardening guidance

Last updated 1 year ago

Was this helpful?

vm ์ค€๋น„

master 3๋Œ€, node 6๋Œ€๋กœ ์ง„ํ–‰

๊ฐ๊ฐ์˜ ๋…ธ๋“œ์— ๋žฉํƒ‘์—์„œ ์ž๋™ ๋กœ๊ทธ์ธ ์„ค์ •์„ ์ง„ํ–‰ํ•ด ๋‘์–ด์•ผํ•œ๋‹ค. cloud-init์„ ์ด์šฉํ•˜์—ฌ vm์„ค์น˜์‹œ ์ž๋™ํ™”๋ฅผ ์ถ”์ฒœ ๋“œ๋ฆฐ๋‹ค.

architechture

master 1 2 3 ์— keepalived๋ฅผ ์„ค์น˜ํ•˜๊ณ  vip๋ฅผ 10๋ฒˆ์„ ํ• ๋‹นํ•ด์ค€๋‹ค.

master 1 2 3 ์— haproxy๊ฐ€ ์„ค์น˜. ๋ฌธ์ œ๊ฐ€ ๋ฐœ์ƒํ•ด๋„ ์„œ๋น„์Šค์— ๋ฌธ์ œ ์—†๊ฒŒ ํ•œ๋‹ค.

haproxy๊ฐ€ 1 2 3 ๋ฒˆ์— ์„ค์น˜๋œ kube api๋ฅผ ํฌ์ธํŠธ ํ•œ๋‹ค.

node1-6๋Š” vip๋ฅผ kube-api ๊ฐ€ ์—ฐ๊ฒฐ๋œ๋‹ค. ์ด๋Ÿฌ๋ฉด ha ๊ฐ€ ์™„์„ฑ๋œ๋‹ค.

prepare install

๋‚˜๋Š” gitops๋ฅผ ์ข‹์•„ํ•˜๋ฏ€๋กœ ๊นƒ ๋ฆฌํฌ๋ฅผ ํ•˜๋‚˜ ๋งŒ๋“ ๋‹ค.

git clone git@github.com:teamsmiley/custom-kubespray.git
cd custom-kubespray

set custom k8s-cluser.yml

vi k8s-cluster.yml

# Set the drain timeout for pre-upgrade
drain_grace_period: 600
drain_timeout: 600s

kube_version: v1.21.5 # ์›ํ•˜๋Š” ๋ฒ„์ „ ์ถ”๊ฐ€

review setup-kubespray.sh

vi setup-kubespray.sh

KUBESPRAY_VERSION=2.17.0
ENV=xxxx

์›ํ•˜๋Š” ๋ฒ„์ „์œผ๋กœ ์„ค์ •์„ ๋ณ€๊ฒฝํ•œ๋‹ค.

./setup-kubespray.sh

sh ์„ค๋ช…

์ค€๋น„ ์™„๋ฃŒ

kubespray๊ฐ€ ๋‹ค์šด๋˜๊ณ  ์„ค์ •์„ ๋‚ด๊ฐ€ ๋ฏธ๋ฆฌ ์ •ํ•ด๋‘”๋Œ€๋กœ ๋ณ€๊ฒฝํ•œ๋‹ค. ์ด๋Ÿฌ๊ฒŒ ํ•ด์„œ ์ด ํด๋” ์ž์ฒด๋ฅผ ์ปค๋ฐ‹/ํ‘ธ์‹œ๋ฅผ ํ•ด๋‘์–ด์•ผํ•œ๋‹ค. ๊ทธ๋ž˜์•ผ ๋‚˜์ค‘์— ๋‚ด๊ฐ€ ์ด๋ฒ„์ „์„ ์‚ฌ์šฉํ•˜์—ฌ ์„ค์น˜ํ•œ๊ฒƒ์ด ๋‚˜์˜จ๋‹ค. ์‚ฌ์šฉํ•œ ๋ชจ๋“ ๊ฑธ ์Šค๋ƒ…์ƒท์ฐ๋“ฏ์ด ์†Œ์Šค์ฝ”๋“œ์— ๋„ฃ์–ด๋‘ฌ์•ผํ•˜๋Š”๊ฒŒ ๋งž๋‹ค. ๋‚˜์ค‘์— ํ˜น์‹œ ์Šคํฌ๋ฆฝํŠธ๊ฐ€ ๋™์ž‘ํ•˜์ง€ ์•Š์•„ ๋ณ€๊ฒฝํ•˜๋”๋ผ๋„ ๋‹ค์‹œ ๊ณ ์น˜๋”๋ผ๋„ ์Šคํฌ๋ฆฝํŠธ๋กœ ์ž๋™ ์ƒ์„ฑ์ด ๋˜์•ผ ๊ฐœ์ธ์ด ๊ทธ๋ƒฅ ๊ณ ์ณ๋ฒ„๋ฆฌ๋Š”๊ฒƒ๋ณด๋‹ค ๋’ค์— ์ž‘์—…ํ•˜๋Š” ์‚ฌ๋žŒ์ด ๋” ์‰ฝ๊ฒŒ ์ž‘์—…ํ• ์ˆ˜ ์žˆ๋‹ค.

Run setup haproxy for Kubernetes masters

ENV=xxxx
ansible-playbook -i inventory/${ENV}/hosts.yml k8s-setup-haproxy-for-masters.yml -b -v

vip ํ™•์ธ

ping 172.16.4.10 #(vip) ์ด๊ฒŒ ์ž˜ ๋˜๋ฉด ์„ฑ๊ณตํ•œ๊ฑฐ๋‹ค.

keepalived ํ™•์ธ

๋งˆ์Šคํ„ฐ ๋…ธ๋“œ ๊ฐ๊ฐ ๋ฐ๋ชฌ ์ƒํƒœ ํ™•์ธ

ssh c4-master01.c4
sudo systemctl status keepalived
ip addr show ens4
ip addr show eth0

kubernetes ์„ค์น˜

haproxy-for-k8s-masters ๊ด€๋ จ ๋‚ด์šฉ ์ˆ˜์ •

group_vars/all/all.yml

## External LB example config
apiserver_loadbalancer_domain_name: 'api.c4'
loadbalancer_apiserver:
  address: 172.16.4.10
  port: 443

hosts ํŒŒ์ผ๋„ ํ™•์ธ

kube-master:
  hosts:
    c4-master01:
      vrrp_instance_state: MASTER
      vrrp_instance_priority: 101
    c4-master02:
      vrrp_instance_state: BACKUP
      vrrp_instance_priority: 100
    c4-master03:
      vrrp_instance_state: BACKUP
      vrrp_instance_priority: 99
  vars:
    vrrp_interface: ens4
    vrrp_instance_virtual_router_id: 103

์ด๋ ‡๊ฒŒ

๋ฒ„์ „ ์„ ํƒ

kubespray/roles/download/defaults/main.yml์—์„œ ๋ฒ„์ „์„ ํ™•์ธํ• ์ˆ˜ ์žˆ๋‹ค.

v1.20.0๋กœ ์„ ํƒ

cd /data/kube-on-premise/
vi inventory/${ENV}/group_vars/k8s_cluster/k8s-cluster.yml
kube_version: v1.20.0
cd kubespray
ansible-playbook -i inventory/${ENV}/hosts.yml cluster.yml -b -v

๋ฌธ์ œ๋“ค

  1. ์„ค์น˜์‹œ ํŠน์ •๋…ธ๋“œ๋Š” ๋น ์ง€๋Š”๋“ฏ ๋ณด์ž„ ํŠน์ •๋…ธ๋“œ๋งŒ ๋”ฐ๋กœ ์„ค์น˜ํ›„ ์ „์ฒด๋ฅผ ๋‹ค์‹œ ์‹คํ–‰ํ•ด์ฃผ์—ˆ๋‹ค.

# ํ•œ๊ฐœ ๋…ธ๋“œ
ansible-playbook -i inventory/${ENV}/hosts.yml cluster.yml -b -v --limits=c4-node01

# ์ „์ฒด ๋…ธ๋“œ
ansible-playbook -i inventory/${ENV}/hosts.yml cluster.yml -b -v

์ƒํƒœ ํ™•์ธ

kubectl --kubeconfig=inventory/${ENV}/artifacts/admin.conf cluster-info
> Kubernetes master is running at https://172.16.4.10:443

kubectl --kubeconfig=inventory/${ENV}/artifacts/admin.conf get nodes

> NAME          STATUS     ROLES    AGE   VERSION
> c4-master01   Ready      master   64m   v1.18.9
> c4-master02   Ready      master   63m   v1.18.9
> c4-master03   Ready      master   63m   v1.18.9
> c4-node01     Ready      <none>   60m   v1.18.9
> c4-node03     NotReady   <none>   60m   v1.18.9

์ ‘์† ์„ค์ • ์ •๋ณด ๊ฐ€์ ธ์˜ค๊ธฐ

cd /data/kube-on-premise/
git add --all && git commit -am server && git push

ENV=xxxx
scp maas:/data/kube-on-premise/kubespray/deploy/kubespray-2.17.0/inventory/${ENV}/artifacts/admin.conf ~/.kube/${ENV}
export KUBECONFIG=$PWD/inventory/${ENV}/artifacts/admin.conf
cp inventory/${ENV}/artifacts/admin.conf ~/.kube/c4-config

inventory/${ENV}/artifacts/ ์— ์„œ๋ฒ„ ์ ‘์†ํ• ์ˆ˜ ์žˆ๋Š” ์„ค์ •์ด ์ƒ๊ธด๋‹ค.

vi ~/.zshrc

export KUBECONFIG=~/.kube/config:~/.kube/c1-config:~/.kube/c2-config

์ž˜ ์„ค์น˜๋˜์—ˆ๋Š”์ง€ ํ™•์ธํ•œ๋‹ค.

remove all setup

ansible-playbook --flush-cache -i inventory/${ENV}/hosts.yml reset.yml --become -u root

argocd ์„ค์น˜

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

๋น„๋ฒˆ ์•Œ์•„๋‚ด๊ธฐ

k -n argocd get secret argocd-initial-admin-secret \
-o jsonpath="{.data.password}" | base64 -d && echo

NXXhjrWj7lDD54Xb

์ ‘์†

k port-forward svc/argocd-server -n argocd 8080:443

update password

create repo for add

๊ทธ๋ฆผ์ฒ˜๋Ÿผ 2๊ฐœ์˜ ํด๋”๊ฐ€ ์žˆ๋‹ค

  • add-apps

    ์•ฑ์„ ์ถ”๊ฐ€ํ•˜๊ธฐ ์œ„ํ•œ yaml

  • deploy

    app ์ž์ฒด์˜ yaml์„ ๋„ฃ๋Š” ํด๋”

๋‘๊ฐœ์˜ ํด๋”๋Š” ๋‹ค์‹œ ๊ตฌ๋ถ„๋œ๋‹ค.

  • core

    ์ฟ ๋ฒ„๋„คํ‹ฐ์Šค๋ฅผ ์šด์˜ํ•˜๊ธฐ์œ„ํ•ด ๊ด€๋ฆฌํ•ด์•ผํ•˜๋Š” ์•ฑ ํด๋”

  • apps

    ์‚ฌ์šฉํ•˜๋Š” ์•ฑ์„ ๋„ฃ๋Š” ํด๋”

add argocd repo to argocd

argocd login localhost:8080

argocd repo add git@github.com:teamsmiley/argocd-c4.git --ssh-private-key-path ~/.ssh/id_rsa

ui์—์„œ ํ™•์ธ ๊ฐ€๋Šฅ

add core / apps

k apply -f add-apps/core/
k apply -f add-apps/apps/

https://github.com/teamsmiley/custom-kubespray
http://localhost:8080/
https://github.com/teamsmiley/devops-senima-argocd
https://github.com/teamsmiley/devops-senima-argocd