Cilium 설치

참고 문서

https://docs.cilium.io/en/stable/installation/k8s-install-kubeadm/

 

 

설치

helm repo add cilium https://helm.cilium.io/
helm repo update
helm install cilium cilium/cilium --version 1.18.6 --namespace kube-system

 

 

Trouble Shooting

Warning FailedCreatePodSandBox 1s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_coredns-668d6bf9bc-j7ls2_kube-system_33cd6810-2f61-4486-a41c-9767bb9d758f_0(21332be5b577d6cea546383e56e5bbc244edb7fe7c198db92f7eba993627a0ca): error adding pod kube-system_coredns-668d6bf9bc-j7ls2 to CNI network "cilium": plugin type="cilium-cni" failed (add): unable to create endpoint: Cilium API client timeout exceeded

coreDNS에서 cilium API로의 timeout이 발생

 

[root@k8s-master k8s-rocky-crio]# kubectl -n kube-system logs -l k8s-app=cilium --tail=200
...
time=2026-02-03T08:50:25.569836902Z level=error msg="BPF template object creation failed" module=agent.datapath.loader error="failed to compile template program: Failed to compile bpf_lxc.o: exit status 1" bpfHeaderfileHash=9a9d8c4cc1e4345c86adc3ed4a31e28f28beb8721c21ff5b9b04040c562564a7
time=2026-02-03T08:50:25.569849155Z level=error msg="Error while reloading endpoint BPF program" k8sPodName=kube-system/coredns-668d6bf9bc-j7ls2 ipv6="" ipv4=10.0.0.197 containerID=9c1aa29514 datapathPolicyRevision=0 identity=19835 desiredPolicyRevision=1 ciliumEndpointName=kube-system/coredns-668d6bf9bc-j7ls2 containerInterface="" endpointID=235 subsys=endpoint error="failed to compile template program: Failed to compile bpf_lxc.o: exit status 1"
time=2026-02-03T08:50:25.569965163Z level=info msg="generating BPF for endpoint failed, keeping stale directory" k8sPodName=kube-system/coredns-668d6bf9bc-j7ls2 ipv6="" ipv4=10.0.0.197 containerID=9c1aa29514 datapathPolicyRevision=0 identity=19835 desiredPolicyRevision=0 ciliumEndpointName=kube-system/coredns-668d6bf9bc-j7ls2 containerInterface="" endpointID=235 subsys=endpoint error="failed to compile template program: Failed to compile bpf_lxc.o: exit status 1" file-path=235_next_fail
time=2026-02-03T08:50:25.570139111Z level=warn msg="Regeneration of endpoint failed" k8sPodName=kube-system/coredns-668d6bf9bc-j7ls2 ipv6="" ipv4=10.0.0.197 containerID=9c1aa29514 datapathPolicyRevision=0 identity=19835 desiredPolicyRevision=0 ciliumEndpointName=kube-system/coredns-668d6bf9bc-j7ls2 containerInterface="" endpointID=235 subsys=endpoint reason="retrying regeneration" mapSync=31.63µs prepareBuild=107.743µs bpfCompilation=148.232969ms bpfWaitForELF=148.36586ms waitingForLock=2.876µs waitingForCTClean=1.000495ms policyCalculation=44.123µs selectorPolicyCalculation=0s total=150.444915ms endpointPolicyCalculation=32.091µs proxyPolicyCalculation=53.35µs bpfLoadProg=0s waitingForPolicyRepository=210ns proxyWaitForAck=0s proxyConfiguration=6.182µs bpfCompilation=148.232969ms bpfWaitForELF=148.36586ms bpfLoadProg=0s error="failed to compile template program: Failed to compile bpf_lxc.o: exit status 1"
time=2026-02-03T08:50:25.570242947Z level=error msg="endpoint regeneration failed" k8sPodName=kube-system/coredns-668d6bf9bc-j7ls2 ipv6="" ipv4=10.0.0.197 containerID=9c1aa29514 datapathPolicyRevision=0 identity=19835 desiredPolicyRevision=0 ciliumEndpointName=kube-system/coredns-668d6bf9bc-j7ls2 containerInterface="" endpointID=235 subsys=endpoint error="failed to compile template program: Failed to compile bpf_lxc.o: exit status 1"

cilium 파드에서는 failed to compile 발생하여 cilium에서 문제 발생함을 확인

 

[root@k8s-master k8s-rocky-crio]# ls -ld /usr/src/kernels/$(uname -r)
ls -l /usr/src/kernels/$(uname -r) | head
rpm -q kernel-devel-$(uname -r) kernel-headers-$(uname -r)
ls: cannot access '/usr/src/kernels/5.14.0-570.17.1.el9_6.x86_64': No such file or directory
ls: cannot access '/usr/src/kernels/5.14.0-570.17.1.el9_6.x86_64': No such file or directory
package kernel-devel-5.14.0-570.17.1.el9_6.x86_64 is not installed
package kernel-headers-5.14.0-570.17.1.el9_6.x86_64 is not installed

eBPF 프로그램을 컴파일하고 로드하기 위해 필요한 커널 헤더(Kernel Headers)와 소스 파일이 시스템에 없어서 발샘한 문제

 

 

해결

dnf update -y kernel kernel-core kernel-modules kernel-devel kernel-headers
reboot
uname -r
dnf install -y kernel-devel-$(uname -r) kernel-headers-$(uname -r)

각 노드에서 Kernel Headers와 소스 파일 설치

 

[root@k8s-master ~]# k get po
NAME                                 READY   STATUS    RESTARTS   AGE
cilium-9zj5g                         1/1     Running   2          20h
cilium-envoy-65jn2                   1/1     Running   2          20h
cilium-envoy-7bc2c                   1/1     Running   2          20h
cilium-operator-75cb67ffc7-b7gwl     1/1     Running   3          20h
cilium-operator-75cb67ffc7-cqkw5     1/1     Running   2          20h
cilium-xgwnr                         1/1     Running   2          20h
coredns-668d6bf9bc-h5965             1/1     Running   0          4m17s
coredns-668d6bf9bc-j7ls2             1/1     Running   1          20h

이후 coredns 파드가 정상 기동됨