참고 페이지
https://kubernetes.io/ko/docs/setup/production-environment/tools/kubeadm/
kubeadm으로 클러스터 구성하기
운영 수준의 컨테이너 오케스트레이션
kubernetes.io
초기 구성
| OS | CPU | Memory | disk | |
| k8s-control | Rocky-10.0-x86_64-minimal | 4core | 8Gi | 20Gi |
| k8s-compute1 | Rocky-10.0-x86_64-minimal | 4core | 8Gi | 20Gi |
| k8s-compute2 | Rocky-10.0-x86_64-minimal | 4core | 8Gi | 20Gi |
이때 ansible 이 원활히 작동하도록 hostname 설정 후, ssh-copy-id 모두 실시함 (k8s-control 에서 3대 모두 실시)
그리고 /etc/hosts 파일에 모두 서버 정보 3대 모두 입력
github source
https://github.com/cody-manupul/IaC
GitHub - cody-manupul/IaC: create k8s cluster by ansible
create k8s cluster by ansible. Contribute to cody-manupul/IaC development by creating an account on GitHub.
github.com
k8s-control 에서 설치
# dnf install git -y
# git clone https://github.com/tangt64/codelab.git
# dnf install podman
# dnf install openssl -y
# dnf install ansible-core python3-pip -y
[inventory 적절히 수정 후]
# ansible-playbook -i inventory/hosts.ini site.yml
trouble shooting1
### error log
[root@iac-lab32-1 k8s-rocky-crio]# ansible-playbook -i inventory/hosts.ini site.yml
ERROR! couldn't resolve module/action 'selinux'. This often indicates a misspelling, missing collection, or incorrect module path.
The error appears to be in '/root/codelab/bootcamp/IaC-bootcamp/k8s-rocky-crio/roles/prereqs/tasks/main.yml': line 6, column 3, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- name: Disable SELinux
^ here
### 필요 패키지 설치
[root@k8s-control k8s-rocky-crio]# ansible-galaxy collection install ansible.posix
Starting galaxy collection install process
Process install dependency map
Starting collection install process
Downloading https://galaxy.ansible.com/api/v3/plugin/ansible/content/published/collections/artifacts/ansible-posix-2.1.0.tar.gz to /root/.ansible/tmp/ansible-local-4801rhg3bgm2/tmpts3tbx_c/ansible-posix-2.1.0-7y8n60ow
Installing 'ansible.posix:2.1.0' to '/root/.ansible/collections/ansible_collections/ansible/posix'
ansible.posix:2.1.0 was installed successfully
### playbook 수정
[root@k8s-control k8s-rocky-crio]# cat roles/prereqs/tasks/main.yml
- name: Disable SELinux
ansible.posix.selinux:
state: disabled
trouble shooting2
### error log
[root@k8s-control k8s-rocky-crio]# ansible-playbook -i inventory/hosts.ini site.yml
ERROR! couldn't resolve module/action 'modprobe'. This often indicates a misspelling, missing collection, or incorrect module path.
The error appears to be in '/root/IaC-bootcamp/k8s-rocky-crio/roles/prereqs/tasks/main.yml': line 27, column 3, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- name: Enable br_netfilter module
^ here
### 필요 패키지 설치
[root@k8s-control k8s-rocky-crio]# ansible-galaxy collection install community.general
Starting galaxy collection install process
Process install dependency map
Starting collection install process
Downloading https://galaxy.ansible.com/api/v3/plugin/ansible/content/published/collections/artifacts/community-general-11.1.1.tar.gz to /root/.ansible/tmp/ansible-local-5058n67p9697/tmpt0jjote8/community-general-11.1.1-r5vl2ax6
Installing 'community.general:11.1.1' to '/root/.ansible/collections/ansible_collections/community/general'
community.general:11.1.1 was installed successfully
trouble shooting3
### error log
[root@k8s-control k8s-rocky-crio]# ansible-playbook -i inventory/hosts.ini site.yml
ERROR! couldn't resolve module/action 'kubernetes.core.k8s'. This often indicates a misspelling, missing collection, or incorrect module path.
The error appears to be in '/root/IaC-bootcamp/k8s-rocky-crio/roles/install_flannel/tasks/main.yml': line 21, column 3, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- name: Create kube-flannel namespace
^ here
### 필요 패키지 설치
[root@k8s-control k8s-rocky-crio]# ansible-galaxy collection install kubernetes.core
Starting galaxy collection install process
Process install dependency map
Starting collection install process
Downloading https://galaxy.ansible.com/api/v3/plugin/ansible/content/published/collections/artifacts/kubernetes-core-6.0.0.tar.gz to /root/.ansible/tmp/ansible-local-52469d7ar37w/tmptwba9m8w/kubernetes-core-6.0.0-scm8n0hy
Installing 'kubernetes.core:6.0.0' to '/root/.ansible/collections/ansible_collections/kubernetes/core'
kubernetes.core:6.0.0 was installed successfully
trouble shooting4
### error log
fatal: [k8s-control]: FAILED! => {"changed": true, "cmd": "kubeadm init --apiserver-advertise-address=192.168.10.10 --pod-network-cidr=20.244.0.0/16 --cri-socket=unix:///var/run/crio/crio.sock\n", "delta": "0:07:04.420818", "end": "2025-08-01 16:15:27.533766", "msg": "non-zero return code", "rc": 1, "start": "2025-08-01 16:08:23.112948", "stderr": "I0801 16:08:28.212076 9957 version.go:261] remote version is much newer: v1.33.3; falling back to: stable-1.32\nW0801 16:08:33.333152 9957 checks.go:846] detected that the sandbox image \"registry.k8s.io/pause:3.9\" of the container runtime is inconsistent with that used by kubeadm.It is recommended to use \"registry.k8s.io/pause:3.10\" as the CRI sandbox image.\nerror execution phase wait-control-plane: could not initialize a Kubernetes cluster\nTo see the stack trace of this error execute with --v=5 or higher", "stderr_lines": ["I0801 16:08:28.212076 9957 version.go:261] remote version is much newer: v1.33.3; falling back to: stable-1.32", "W0801 16:08:33.333152 9957 checks.go:846] detected that the sandbox image \"registry.k8s.io/pause:3.9\" of the container runtime is inconsistent with that used by kubeadm.It is recommended to use \"registry.k8s.io/pause:3.10\" as the CRI sandbox image.", "error execution phase wait-control-plane: could not initialize a Kubernetes cluster", "To see the stack trace of this error execute with --v=5 or higher"], "stdout": "[init] Using Kubernetes version: v1.32.7\n[preflight] Running pre-flight checks\n[preflight] Pulling images required for setting up a Kubernetes cluster\n[preflight] This might take a minute or two, depending on the speed of your internet connection\n[preflight] You can also perform this action beforehand using 'kubeadm config images pull'\n[certs] Using certificateDir folder \"/etc/kubernetes/pki\"\n[certs] Generating \"ca\" certificate and key\n[certs] Generating \"apiserver\" certificate and key\n[certs] apiserver serving cert is signed for DNS names [k8s-control kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.10.10]\n[certs] Generating \"apiserver-kubelet-client\" certificate and key\n[certs] Generating \"front-proxy-ca\" certificate and key\n[certs] Generating \"front-proxy-client\" certificate and key\n[certs] Generating \"etcd/ca\" certificate and key\n[certs] Generating \"etcd/server\" certificate and key\n[certs] etcd/server serving cert is signed for DNS names [k8s-control localhost] and IPs [192.168.10.10 127.0.0.1 ::1]\n[certs] Generating \"etcd/peer\" certificate and key\n[certs] etcd/peer serving cert is signed for DNS names [k8s-control localhost] and IPs [192.168.10.10 127.0.0.1 ::1]\n[certs] Generating \"etcd/healthcheck-client\" certificate and key\n[certs] Generating \"apiserver-etcd-client\" certificate and key\n[certs] Generating \"sa\" key and public key\n[kubeconfig] Using kubeconfig folder \"/etc/kubernetes\"\n[kubeconfig] Writing \"admin.conf\" kubeconfig file\n[kubeconfig] Writing \"super-admin.conf\" kubeconfig file\n[kubeconfig] Writing \"kubelet.conf\" kubeconfig file\n[kubeconfig] Writing \"controller-manager.conf\" kubeconfig file\n[kubeconfig] Writing \"scheduler.conf\" kubeconfig file\n[etcd] Creating static Pod manifest for local etcd in \"/etc/kubernetes/manifests\"\n[control-plane] Using manifest folder \"/etc/kubernetes/manifests\"\n[control-plane] Creating static Pod manifest for \"kube-apiserver\"\n[control-plane] Creating static Pod manifest for \"kube-controller-manager\"\n[control-plane] Creating static Pod manifest for \"kube-scheduler\"\n[kubelet-start] Writing kubelet environment file with flags to file \"/var/lib/kubelet/kubeadm-flags.env\"\n[kubelet-start] Writing kubelet configuration to file \"/var/lib/kubelet/config.yaml\"\n[kubelet-start] Starting the kubelet\n[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory \"/etc/kubernetes/manifests\"\n[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s\n[kubelet-check] The kubelet is healthy after 501.605233ms\n[api-check] Waiting for a healthy API server. This can take up to 4m0s\n[api-check] The API server is not healthy after 4m0.008349497s\n\nUnfortunately, an error has occurred:\n\tcontext deadline exceeded\n\nThis error is likely caused by:\n\t- The kubelet is not running\n\t- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)\n\nIf you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:\n\t- 'systemctl status kubelet'\n\t- 'journalctl -xeu kubelet'\n\nAdditionally, a control plane component may have crashed or exited when started by the container runtime.\nTo troubleshoot, list all containers using your preferred container runtimes CLI.\nHere is one example how you may list all running Kubernetes containers by using crictl:\n\t- 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock ps -a | grep kube | grep -v pause'\n\tOnce you have found the failing container, you can inspect its logs with:\n\t- 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock logs CONTAINERID'", "stdout_lines": ["[init] Using Kubernetes version: v1.32.7", "[preflight] Running pre-flight checks", "[preflight] Pulling images required for setting up a Kubernetes cluster", "[preflight] This might take a minute or two, depending on the speed of your internet connection", "[preflight] You can also perform this action beforehand using 'kubeadm config images pull'", "[certs] Using certificateDir folder \"/etc/kubernetes/pki\"", "[certs] Generating \"ca\" certificate and key", "[certs] Generating \"apiserver\" certificate and key", "[certs] apiserver serving cert is signed for DNS names [k8s-control kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.10.10]", "[certs] Generating \"apiserver-kubelet-client\" certificate and key", "[certs] Generating \"front-proxy-ca\" certificate and key", "[certs] Generating \"front-proxy-client\" certificate and key", "[certs] Generating \"etcd/ca\" certificate and key", "[certs] Generating \"etcd/server\" certificate and key", "[certs] etcd/server serving cert is signed for DNS names [k8s-control localhost] and IPs [192.168.10.10 127.0.0.1 ::1]", "[certs] Generating \"etcd/peer\" certificate and key", "[certs] etcd/peer serving cert is signed for DNS names [k8s-control localhost] and IPs [192.168.10.10 127.0.0.1 ::1]", "[certs] Generating \"etcd/healthcheck-client\" certificate and key", "[certs] Generating \"apiserver-etcd-client\" certificate and key", "[certs] Generating \"sa\" key and public key", "[kubeconfig] Using kubeconfig folder \"/etc/kubernetes\"", "[kubeconfig] Writing \"admin.conf\" kubeconfig file", "[kubeconfig] Writing \"super-admin.conf\" kubeconfig file", "[kubeconfig] Writing \"kubelet.conf\" kubeconfig file", "[kubeconfig] Writing \"controller-manager.conf\" kubeconfig file", "[kubeconfig] Writing \"scheduler.conf\" kubeconfig file", "[etcd] Creating static Pod manifest for local etcd in \"/etc/kubernetes/manifests\"", "[control-plane] Using manifest folder \"/etc/kubernetes/manifests\"", "[control-plane] Creating static Pod manifest for \"kube-apiserver\"", "[control-plane] Creating static Pod manifest for \"kube-controller-manager\"", "[control-plane] Creating static Pod manifest for \"kube-scheduler\"", "[kubelet-start] Writing kubelet environment file with flags to file \"/var/lib/kubelet/kubeadm-flags.env\"", "[kubelet-start] Writing kubelet configuration to file \"/var/lib/kubelet/config.yaml\"", "[kubelet-start] Starting the kubelet", "[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory \"/etc/kubernetes/manifests\"", "[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s", "[kubelet-check] The kubelet is healthy after 501.605233ms", "[api-check] Waiting for a healthy API server. This can take up to 4m0s", "[api-check] The API server is not healthy after 4m0.008349497s", "", "Unfortunately, an error has occurred:", "\tcontext deadline exceeded", "", "This error is likely caused by:", "\t- The kubelet is not running", "\t- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)", "", "If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:", "\t- 'systemctl status kubelet'", "\t- 'journalctl -xeu kubelet'", "", "Additionally, a control plane component may have crashed or exited when started by the container runtime.", "To troubleshoot, list all containers using your preferred container runtimes CLI.", "Here is one example how you may list all running Kubernetes containers by using crictl:", "\t- 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock ps -a | grep kube | grep -v pause'", "\tOnce you have found the failing container, you can inspect its logs with:", "\t- 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock logs CONTAINERID'"]}
### 구성된 VM 설정에 맞게 playbook 수정
- name: Initialize cluster (single controller)
shell: |
kubeadm init --apiserver-advertise-address=10.0.2.11 --pod-network-cidr=10.50.0.0/16 --cri-socket=unix:///var/run/crio/crio.sock
when: not multi_controller and inventory_hostname == groups['controllers'][0]
register: kubeadm_init
- name: Ensure .kube directory exists on the first controller
file:
path: /root/.kube
state: directory
owner: root
group: root
mode: '0700'
when: inventory_hostname == groups['controllers'][0]
- name: Copy kubeconfig from first controller to localhost
copy:
src: /etc/kubernetes/admin.conf
dest: /root/.kube/config
remote_src: yes
owner: root
group: root
mode: '0700'
when: inventory_hostname == groups['controllers'][0]
delegate_to: localhost
trouble shooting5
### error log
TASK [join_control_planes : Generate control-plane join command] *********************************************************************************************************************************************************************
fatal: [k8s-control -> node1.example.com]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ssh: connect to host node1.example.com port 22: Connection timed out", "unreachable": true}
CNI 설치 실패로 인한 error
### playbook 수정
- name: Install Flannel (Skip TLS)
shell: |
helm install flannel flannel/flannel \
--namespace kube-flannel \
--create-namespace \
--set podCidr="10.50.0.0/16" \
--kube-insecure-skip-tls-verify
### playbook 수정
[root@k8s-control roles]# vi join_control_planes/tasks/main.yml
에서 아래와 같이 조건을 수정
when: inventory_hostname == groups['controllers'][0]
trouble shooting6
### error log
TASK [join_control_planes : Generate control-plane join command] *********************************************************************************************************************************************************************
task path: /root/IaC-bootcamp/k8s-rocky-crio/roles/join_control_planes/tasks/main.yml:10
<node1.example.com> ESTABLISH SSH CONNECTION FOR USER: None
<node1.example.com> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/edfbd3887e"' node1.example.com '/bin/sh -c '"'"'echo ~ && sleep 0'"'"''
<node1.example.com> (255, b'', b'ssh: connect to host node1.example.com port 22: Connection timed out\r\n')
fatal: [k8s-control -> node1.example.com]: UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: ssh: connect to host node1.example.com port 22: Connection timed out",
"unreachable": true
}
### playbook 수정
[root@k8s-control roles]# grep -R node1.example.com
join_control_planes/tasks/main.yml: delegate_to: node1.example.com
join_control_planes/tasks/main.yml: full_join_command: "{{ hostvars['node1.example.com'].full_join_command }}"
join_control_planes/tasks/main.yml: full_join_command: "{{ hostvars['node1.example.com'].full_join_command }}"
prereqs/tasks/main.yml: line: "192.168.10.10 node1.example.com"
에서 node1.example.com 을 모두 k8s-control 로 수정
trouble shooting7
### error log
TASK [join_control_planes : Join other control planes to cluster] ********************************************************************************************************************************************************************
fatal: [k8s-control]: FAILED! => {"changed": true, "cmd": "kubeadm join 10.0.2.11:6443 --token 8r4330.6xp2t8hynbpn5dhi --discovery-token-ca-cert-hash sha256:6c26f452cf59526fb969a11ffb6d70c9eb6cfd2d744d06c77617ef6159953af1 --control-plane --certificate-key f8296a559de9074e2911eef4c43520e08f66ea7d0471e2ab180544c10ec510d1 --cri-socket=unix:///var/run/crio/crio.sock", "delta": "0:00:00.113164", "end": "2025-08-01 17:20:45.048883", "msg": "non-zero return code", "rc": 1, "start": "2025-08-01 17:20:44.935719", "stderr": "error execution phase preflight: [preflight] Some fatal errors occurred:\n\t[ERROR FileAvailable--etc-kubernetes-kubelet.conf]: /etc/kubernetes/kubelet.conf already exists\n\t[ERROR Port-10250]: Port 10250 is in use\n[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`\nTo see the stack trace of this error execute with --v=5 or higher", "stderr_lines": ["error execution phase preflight: [preflight] Some fatal errors occurred:", "\t[ERROR FileAvailable--etc-kubernetes-kubelet.conf]: /etc/kubernetes/kubelet.conf already exists", "\t[ERROR Port-10250]: Port 10250 is in use", "[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`", "To see the stack trace of this error execute with --v=5 or higher"], "stdout": "[preflight] Running pre-flight checks", "stdout_lines": ["[preflight] Running pre-flight checks"]}
### playbook 수정
- name: Wait for kube-apiserver port to open
wait_for:
port: 6443
host: 127.0.0.1
delay: 10
timeout: 120
state: started
when: inventory_hostname == groups['controllers'][0]
- name: Generate control-plane join command
shell: kubeadm token create --print-join-command -v4
register: join_output
delegate_to: k8s-control
run_once: true
- name: Get certificate key
shell: kubeadm init phase upload-certs --upload-certs | tail -1
register: cert_key
delegate_to: k8s-control
run_once: true
- name: Set join command fact
set_fact:
full_join_command: "{{ join_output.stdout }} --control-plane --certificate-key {{ cert_key.stdout }} --cri-socket=unix:///var/run/crio/crio.sock"
join_command_worker: "{{ join_output.stdout }} --cri-socket=unix:///var/run/crio/crio.sock"
when: inventory_hostname == groups['controllers'][0]
- name: Share join command with all hosts
add_host:
name: shared_host
groups: join_group
full_join_command: "{{ full_join_command }}"
join_command_worker: "{{ join_command_worker }}"
when: inventory_hostname == groups['controllers'][0]
- name: Join other control planes to cluster
shell: "{{ hostvars['shared_host'].full_join_command }}"
when: inventory_hostname in groups['controllers'][1:]
- name: Join worker nodes to cluster
shell: "{{ hostvars['shared_host'].join_command_worker }}"
when: inventory_hostname in groups['workers']
설치 완료
CNI pod 가 뜰 때까지 시간이 좀 걸리지만 기다리면 node 와 pod 준비 완료
[root@k8s-control k8s-rocky-crio]# kubectl get node -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k8s-compute1 Ready <none> 103s v1.32.7 10.0.2.12 <none> Rocky Linux 10.0 (Red Quartz) 6.12.0-55.22.1.el10_0.x86_64 cri-o://1.30.14
k8s-compute2 Ready <none> 103s v1.32.7 10.0.2.13 <none> Rocky Linux 10.0 (Red Quartz) 6.12.0-55.22.1.el10_0.x86_64 cri-o://1.30.14
k8s-control Ready control-plane 3m v1.32.7 10.0.2.11 <none> Rocky Linux 10.0 (Red Quartz) 6.12.0-55.22.1.el10_0.x86_64 cri-o://1.30.14[root@k8s-control k8s-rocky-crio]# kubectl get po -A -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kube-flannel kube-flannel-ds-664j7 1/1 Running 0 3m49s 10.0.2.11 k8s-control <none> <none>
kube-flannel kube-flannel-ds-bbw8r 1/1 Running 0 3m35s 10.0.2.13 k8s-compute2 <none> <none>
kube-flannel kube-flannel-ds-tz9lf 1/1 Running 0 3m35s 10.0.2.12 k8s-compute1 <none> <none>
kube-system coredns-668d6bf9bc-7vdjd 1/1 Running 0 4m44s 10.50.0.2 k8s-control <none> <none>
kube-system coredns-668d6bf9bc-flpmw 1/1 Running 0 4m44s 10.50.0.3 k8s-control <none> <none>
kube-system etcd-k8s-control 1/1 Running 8 4m50s 10.0.2.11 k8s-control <none> <none>
kube-system kube-apiserver-k8s-control 1/1 Running 2 4m50s 10.0.2.11 k8s-control <none> <none>
kube-system kube-controller-manager-k8s-control 1/1 Running 8 4m50s 10.0.2.11 k8s-control <none> <none>
kube-system kube-proxy-kfgsr 1/1 Running 0 4m44s 10.0.2.11 k8s-control <none> <none>
kube-system kube-proxy-lpfh8 1/1 Running 0 3m35s 10.0.2.13 k8s-compute2 <none> <none>
kube-system kube-proxy-zbjtg 1/1 Running 0 3m35s 10.0.2.12 k8s-compute1 <none> <none>
kube-system kube-scheduler-k8s-control 1/1 Running 9 4m51s 10.0.2.11 k8s-control <none> <none>
tekton-pipelines-resolvers tekton-pipelines-remote-resolvers-85689d8f65-m2fss 1/1 Running 0 3m39s 10.50.1.3 k8s-compute2 <none> <none>
tekton-pipelines tekton-events-controller-786b59d5cd-rns96 1/1 Running 0 3m39s 10.50.1.2 k8s-compute2 <none> <none>
tekton-pipelines tekton-pipelines-controller-59b6cdbbc-8dlrq 1/1 Running 0 3m39s 10.50.1.5 k8s-compute2 <none> <none>
tekton-pipelines tekton-pipelines-webhook-74b5cdfcc4-s9kqn 1/1 Running 0 3m39s 10.50.1.4 k8s-compute2 <none> <none>
alias 및 자동 완성
$ echo 'source <(kubectl completion bash)' >> ~/.bashrc
$ echo 'alias k=kubectl' >> ~/.bashrc
$ echo 'complete -F __start_kubectl k' >> ~/.bashrc
$ alias ns='kubectl config set-context --current --namespace'
$ source ~/.bashrc
재부팅 후 flannel Trouble Shooting
### error log
[root@k8s-control ~]# k logs kube-flannel-ds-5dxkp
Defaulted container "kube-flannel" out of: kube-flannel, install-cni-plugin (init), install-cni (init)
I0804 08:48:42.456144 1 main.go:213] CLI flags config: {etcdEndpoints:http://127.0.0.1:4001,http://127.0.0.1:2379 etcdPrefix:/coreos.com/network etcdKeyfile: etcdCertfile: etcdCAFile: etcdUsername: etcdPassword: version:false kubeSubnetMgr:true kubeApiUrl: kubeAnnotationPrefix:flannel.alpha.coreos.com kubeConfigFile: iface:[] ifaceRegex:[] ipMasq:true ifaceCanReach: subnetFile:/run/flannel/subnet.env publicIP: publicIPv6: subnetLeaseRenewMargin:60 healthzIP:0.0.0.0 healthzPort:0 iptablesResyncSeconds:5 iptablesForwardRules:true blackholeRoute:false netConfPath:/etc/kube-flannel/net-conf.json setNodeNetworkUnavailable:true}
W0804 08:48:42.456302 1 client_config.go:659] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
I0804 08:48:42.469709 1 kube.go:139] Waiting 10m0s for node controller to sync
I0804 08:48:42.469791 1 kube.go:537] Starting kube subnet manager
I0804 08:48:42.473574 1 kube.go:558] Creating the node lease for IPv4. This is the n.Spec.PodCIDRs: [10.50.2.0/24]
I0804 08:48:42.473659 1 kube.go:558] Creating the node lease for IPv4. This is the n.Spec.PodCIDRs: [10.50.1.0/24]
I0804 08:48:42.473668 1 kube.go:558] Creating the node lease for IPv4. This is the n.Spec.PodCIDRs: [10.50.0.0/24]
I0804 08:48:43.471033 1 kube.go:163] Node controller sync successful
I0804 08:48:43.471095 1 main.go:239] Created subnet manager: Kubernetes Subnet Manager - k8s-compute1
I0804 08:48:43.471102 1 main.go:242] Installing signal handlers
I0804 08:48:43.471616 1 main.go:519] Found network config - Backend type: vxlan
E0804 08:48:43.471673 1 main.go:276] Failed to check br_netfilter: stat /proc/sys/net/bridge/bridge-nf-call-iptables: no such file or directory
### 각 노드에 아래와 같이 조치
[root@k8s-control ~]# cat /etc/sysctl.conf
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
net.bridge.bridge-nf-call-iptables = 1
[root@k8s-control ~]# modprobe br_netfilter
[root@k8s-control ~]# lsmod | grep br_netfilter
br_netfilter 36864 0
bridge 417792 1 br_netfilter
[root@k8s-control ~]# ls /proc/sys/net/bridge/
bridge-nf-call-arptables bridge-nf-call-ip6tables bridge-nf-call-iptables bridge-nf-filter-pppoe-tagged bridge-nf-filter-vlan-tagged bridge-nf-pass-vlan-input-dev
[root@k8s-control ~]# sysctl -p /etc/sysctl.conf
net.bridge.bridge-nf-call-iptables = 1
[root@k8s-control ~]# echo "br_netfilter" > /etc/modules-load.d/k8s.conf
[root@k8s-control ~]# cat /etc/modules-load.d/k8s.conf
br_netfilter
'k8s' 카테고리의 다른 글
| metric server 설치 (0) | 2025.09.02 |
|---|---|
| node join 및 delete (0) | 2025.08.28 |
| PV 테스트 (0) | 2025.07.15 |
| Prometheus 와 Grafana 설치 (1) | 2025.07.11 |
| Pod Garbage Collector (3) | 2025.07.10 |