Warning FailedCreatePodSandBox 1s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_coredns-668d6bf9bc-j7ls2_kube-system_33cd6810-2f61-4486-a41c-9767bb9d758f_0(21332be5b577d6cea546383e56e5bbc244edb7fe7c198db92f7eba993627a0ca): error adding pod kube-system_coredns-668d6bf9bc-j7ls2 to CNI network "cilium": plugin type="cilium-cni" failed (add): unable to create endpoint: Cilium API client timeout exceeded
coreDNS에서 cilium API로의 timeout이 발생
[root@k8s-master k8s-rocky-crio]# kubectl -n kube-system logs -l k8s-app=cilium --tail=200
...
time=2026-02-03T08:50:25.569836902Z level=error msg="BPF template object creation failed" module=agent.datapath.loader error="failed to compile template program: Failed to compile bpf_lxc.o: exit status 1" bpfHeaderfileHash=9a9d8c4cc1e4345c86adc3ed4a31e28f28beb8721c21ff5b9b04040c562564a7
time=2026-02-03T08:50:25.569849155Z level=error msg="Error while reloading endpoint BPF program" k8sPodName=kube-system/coredns-668d6bf9bc-j7ls2 ipv6="" ipv4=10.0.0.197 containerID=9c1aa29514 datapathPolicyRevision=0 identity=19835 desiredPolicyRevision=1 ciliumEndpointName=kube-system/coredns-668d6bf9bc-j7ls2 containerInterface="" endpointID=235 subsys=endpoint error="failed to compile template program: Failed to compile bpf_lxc.o: exit status 1"
time=2026-02-03T08:50:25.569965163Z level=info msg="generating BPF for endpoint failed, keeping stale directory" k8sPodName=kube-system/coredns-668d6bf9bc-j7ls2 ipv6="" ipv4=10.0.0.197 containerID=9c1aa29514 datapathPolicyRevision=0 identity=19835 desiredPolicyRevision=0 ciliumEndpointName=kube-system/coredns-668d6bf9bc-j7ls2 containerInterface="" endpointID=235 subsys=endpoint error="failed to compile template program: Failed to compile bpf_lxc.o: exit status 1" file-path=235_next_fail
time=2026-02-03T08:50:25.570139111Z level=warn msg="Regeneration of endpoint failed" k8sPodName=kube-system/coredns-668d6bf9bc-j7ls2 ipv6="" ipv4=10.0.0.197 containerID=9c1aa29514 datapathPolicyRevision=0 identity=19835 desiredPolicyRevision=0 ciliumEndpointName=kube-system/coredns-668d6bf9bc-j7ls2 containerInterface="" endpointID=235 subsys=endpoint reason="retrying regeneration" mapSync=31.63µs prepareBuild=107.743µs bpfCompilation=148.232969ms bpfWaitForELF=148.36586ms waitingForLock=2.876µs waitingForCTClean=1.000495ms policyCalculation=44.123µs selectorPolicyCalculation=0s total=150.444915ms endpointPolicyCalculation=32.091µs proxyPolicyCalculation=53.35µs bpfLoadProg=0s waitingForPolicyRepository=210ns proxyWaitForAck=0s proxyConfiguration=6.182µs bpfCompilation=148.232969ms bpfWaitForELF=148.36586ms bpfLoadProg=0s error="failed to compile template program: Failed to compile bpf_lxc.o: exit status 1"
time=2026-02-03T08:50:25.570242947Z level=error msg="endpoint regeneration failed" k8sPodName=kube-system/coredns-668d6bf9bc-j7ls2 ipv6="" ipv4=10.0.0.197 containerID=9c1aa29514 datapathPolicyRevision=0 identity=19835 desiredPolicyRevision=0 ciliumEndpointName=kube-system/coredns-668d6bf9bc-j7ls2 containerInterface="" endpointID=235 subsys=endpoint error="failed to compile template program: Failed to compile bpf_lxc.o: exit status 1"
cilium 파드에서는 failed to compile 발생하여 cilium에서 문제 발생함을 확인
[root@k8s-master k8s-rocky-crio]# ls -ld /usr/src/kernels/$(uname -r)
ls -l /usr/src/kernels/$(uname -r) | head
rpm -q kernel-devel-$(uname -r) kernel-headers-$(uname -r)
ls: cannot access '/usr/src/kernels/5.14.0-570.17.1.el9_6.x86_64': No such file or directory
ls: cannot access '/usr/src/kernels/5.14.0-570.17.1.el9_6.x86_64': No such file or directory
package kernel-devel-5.14.0-570.17.1.el9_6.x86_64 is not installed
package kernel-headers-5.14.0-570.17.1.el9_6.x86_64 is not installed
eBPF 프로그램을 컴파일하고 로드하기 위해 필요한 커널 헤더(Kernel Headers)와 소스 파일이 시스템에 없어서 발샘한 문제
설치가 완료되면 아래와 같이 접속 가능 URL 확인이 가능하고 container가 생성되어 있다.
## Skip enabling systemd service for rootless Podman since Docker is used.
### Installation completed successfully
### Access QueryPie at http://10.0.2.17:8000 or https://10.0.2.17:8443 in your browser
### Determine the public IP address of your host machine if needed
[root@k8s-node02 ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4a874571b504 querypie/querypie:11.5.0 "/usr/local/bin/dock…" 4 minutes ago Up 4 minutes 0.0.0.0:6443->6443/tcp, [::]:6443->6443/tcp, 0.0.0.0:7447->7447/tcp, [::]:7447->7447/tcp, 0.0.0.0:9000->9000/tcp, [::]:9000->9000/tcp, 0.0.0.0:40000-40030->40000-40030/tcp, [::]:40000-40030->40000-40030/tcp, 0.0.0.0:8000->80/tcp, [::]:8000->80/tcp, 0.0.0.0:8443->443/tcp, [::]:8443->443/tcp querypie-app-1
38dc5652e610 querypie/redis:7.4.5 "docker-entrypoint.s…" 6 minutes ago Up 6 minutes 0.0.0.0:6379->6379/tcp, [::]:6379->6379/tcp querypie-redis-1
e83acd0f996e querypie/mysql:8.0.42 "docker-entrypoint.s…" 6 minutes ago Up 6 minutes 0.0.0.0:3306->3306/tcp, [::]:3306->3306/tcp, 33060/tcp querypie-mysql-1
[root@k8s-master terraform-access-control]# vi pv.yaml
[root@k8s-master terraform-access-control]# k apply -f pv.yaml
persistentvolume/local-pv created
[root@k8s-master terraform-access-control]# k get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS VOLUMEATTRIBUTESCLASS REASON AGE
local-pv 50Gi RWO Retain Available <unset> 2s
MySQL statefulset 배포
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mysql
namespace: querypie
spec:
selector:
matchLabels:
app: mysql
serviceName: mysql
replicas: 1
template:
metadata:
labels:
app: mysql
spec:
containers:
- name: mysql
image: mysql:8.0
env:
- name: MYSQL_ROOT_PASSWORD
value: querypie
- name: MYSQL_DATABASE
value: querypie
- name: MYSQL_USER
value: querypie
- name: MYSQL_PASSWORD
value: querypie
resources:
requests:
cpu: "0.5"
memory: "2Gi"
ports:
- containerPort: 3306
name: mysql
volumeMounts:
- name: data
mountPath: /var/lib/mysql
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 50Gi
storageClassName: ""
selector:
matchLabels:
app: mysql
---
apiVersion: v1
kind: Service
metadata:
name: mysql
namespace: querypie
labels:
app: mysql
spec:
type: ClusterIP
selector:
app: mysql
ports:
- port: 3306
targetPort: 3306
protocol: TCP
name: mysql
[root@k8s-master terraform-access-control]# vi mysql.yaml
[root@k8s-master terraform-access-control]# k apply -f mysql.yaml
statefulset.apps/mysql created
service/mysql created
[root@k8s-master terraform-access-control]# k get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
mysql ClusterIP 10.98.161.155 <none> 3306/TCP 72s
[root@k8s-master terraform-access-control]# k get po
NAME READY STATUS RESTARTS AGE
mysql-0 1/1 Running 0 74s
DROP DATABASE IF EXISTS querypie;
DROP DATABASE IF EXISTS querypie_log;
DROP DATABASE IF EXISTS querypie_snapshot;
CREATE database querypie CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
CREATE database querypie_log CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
CREATE database querypie_snapshot CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
GRANT ALL privileges ON querypie.* TO querypie@'%';
GRANT ALL privileges ON querypie_log.* TO querypie@'%';
GRANT ALL privileges ON querypie_snapshot.* TO querypie@'%';
FLUSH PRIVILEGES;
[root@k8s-master terraform-access-control]# vi redis.yaml
[root@k8s-master terraform-access-control]# k apply -f redis.yaml
deployment.apps/redis created
service/redis created
[root@k8s-master terraform-access-control]# k get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
mysql ClusterIP 10.98.161.155 <none> 3306/TCP 19m
redis ClusterIP 10.108.214.141 <none> 6379/TCP 50s
[root@k8s-master terraform-access-control]# k get po -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
mysql-0 1/1 Running 0 19m 10.85.2.94 k8s-node02 <none> <none>
redis-69b5bb5694-bzvb2 1/1 Running 0 54s 10.85.2.95 k8s-node02 <none> <none>
# Agent Secret (32자리 임의의 문자열)
AGENT_SECRET=01234567890123456789012345678912
KEK=querypie
# QueryPie Meta DB 접속 정보
DB_CATALOG=querypie
DB_HOST=mysql
DB_USERNAME=querypie
DB_PASSWORD=querypie
DB_PORT=3306
# QueryPie Log DB 접속 정보
LOG_DB_CATALOG=querypie_log
LOG_DB_HOST=mysql
LOG_DB_USERNAME=querypie
LOG_DB_PASSWORD=querypie
LOG_DB_PORT=3306
# QueryPie Snapshot DB 접속 정보
ENG_DB_CATALOG=querypie_snapshot
ENG_DB_HOST=mysql
ENG_DB_USERNAME=querypie
ENG_DB_PASSWORD=querypie
ENG_DB_PORT=3306
# Redis 접속 정보
REDIS_NODES=redis:6379
REDIS_PASSWORD=querypie
[root@k8s-master terraform-access-control]# helm repo add querypie https://chequer-io.github.io/querypie-deployment/helm-chart
"querypie" has been added to your repositories
[root@k8s-master terraform-access-control]# helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "querypie" chart repository
[root@k8s-master terraform-access-control]# helm repo list
NAME URL
querypie https://chequer-io.github.io/querypie-deployment/helm-chart
[root@k8s-master terraform-access-control]# helm search repo querypie
NAME CHART VERSION APP VERSION DESCRIPTION
querypie/querypie 1.4.4 11.5.0 A Helm chart for QueryPie
values.yaml 로 아래 파일 저장
appVersion: &version 11.3.0
global:
image:
# -- Default registry used for all images
registry: harbor.chequer.io
# -- Default image tags used for all images
tag: *version
# -- Default image pull policy for all images
pullPolicy: IfNotPresent
# -- Default labels for all resources deployed by the chart
labels: {}
# -- ServiceAccount for QueryPie and QueryPie tools pods
# It is created for authenticating private image registry and AWS API.
# @default -- {}
serviceAccount:
labels: {}
annotations: {}
# -- The name of the secret used for pulling the QueryPie image from the private registry.
# To create, run `kubectl create secret docker-registry querypie-regcred --docker-server=harbor.chequer.io --docker-username=admin --docker-password=your-password`
imagePullSecrets:
- name: querypie-regcred
querypie:
# -- Labels used for QueryPie pods.
labels: {}
# -- Annotations used for QueryPie pods.
annotations: {}
tolerations: {}
# -- NodeSelector for QueryPie pods
nodeSelector: {}
replicas: 1
# -- PodManagementPolicy for QueryPie pods. OrderedReady is fine for most cases, but Parallel is also good for large-scale deployments.
podManagementPolicy: Parallel
image:
# -- (string) Registry used for the QueryPie image. If not set, the global registry will be used.
registry:
# -- (string) Tag used for the QueryPie image. If not set, the global tag will be used.
tag:
# -- (string) PullPolicy used for the QueryPie image. If not set, the global pullPolicy will be used.
pullPolicy:
# -- Repository used for the QueryPie image. (required)
repository: querypie/querypie
ingress:
# -- (bool) If true, Ingress resource for QueryPie will be created.
enabled: true
labels: {}
# -- Annotations used for the Ingress resource.
# It is useful when you want to enable controller-specific feature for the Ingress resource.
annotations: {}
# -- (string) Class used for the Ingress resource. If not set, the default class will be used.
ingressClassName: ""
# -- Hostname used for the Ingress resource. If not set, every hostname will be accepted.
host: querypie.querypie.io #실제 설정된 url 입력
# -- Extra paths used for the Ingress resource.
# It is mostly used for redirecting HTTP to HTTPS if the ingress controller does not support redirect by default.
proxyService:
# -- Create a service resource for QueryPie Agent connections.
# Basically, it opens 9000/tcp port for the QueryPie Agent, and 40000~/tcp for the Agentless connections.
enabled: true
labels: {}
# -- Annotations used for the Service resource.
# It is useful when you want to enable controller-specific feature for the Service resource.
annotations: {}
type: LoadBalancer
loadBalancerClass: ""
# -- Set the externalTrafficPolicy to Local is recommended for auditing the real client IP address.
externalTrafficPolicy: "Local"
# -- SessionAffinity is not required for the QueryPie.
sessionAffinity: "None"
updateStrategy:
# -- The strategy used for updating the QueryPie pods.
# It is generally recommended to use the RollingUpdate strategy.
# But if required, you can change it to OnDelete for manual operations.
type: RollingUpdate
resources:
requests:
cpu: "2000m"
memory: 16Gi
limits:
# -- More than 2 CPU cores are recommended.
cpu: "2000m"
# -- More than 16Gi memory is recommended.
memory: 16Gi
logrotator:
image:
# -- (string) Registry used for the Logrotate image. If not set, the global registry will be used.
repository: querypie/logrotate
# -- (string) Tag used for the Logrotate image. If not set, the global tag will be used.
tag: latest
# -- External storage for the QueryPie Object Storage.
externalStorage:
# -- Type of the external storage.
# - none: No external storage.
# - persistentVolumeClaim: using a single Persistent Volume Claim for all QueryPie pods.
type: none
# -- Use a persistent volume claim for the external storage. It will be available if the type is persistentVolumeClaim.
persistentVolumeClaim:
# -- Use an existing Persistent Volume Claim for the external storage.
# If you set this to true, this helm chart will not create a new Persistent Volume Claim.
useExisting: false
# -- The name of the existing Persistent Volume Claim to be used.
claimName: ""
# -- Metadata of the Persistent Volume Claim.
metadata:
annotations: {}
labels: {}
# -- Spec of the Persistent Volume Claim.
spec:
storageClassName: ""
resources:
requests:
storage: 100Gi
# -- It is required to have ReadWriteMany access mode on production.
accessModes:
- ReadWriteMany
# -- Extra environment variables used for the QueryPie pods.
# This is useful for experimental features, debugging, workaround, and so on.
# for example:
# API_JVM_HEAPSIZE: '2g' will set the JVM heap size to 2GB.
extraEnvs: {}
tools:
image:
# -- Registry used for the QueryPie tools image. If not set, the global registry will be used.
repository: querypie/querypie-tools
config:
# -- External URL that users use to access the QueryPie via web.
# Specify the scheme, hostname, and port is required.
externalURL: "https://querypie.querypie.io" #실제 설정된 url 입력
secretName: "querypie-secret"
database:
querypie:
# -- The maximum number of connections that QueryPie can use for "metastore" purposes.
connectionPoolSize: 20
dac:
# -- This setting is used to ignore simple queries sent by client tools such as DataGrip.
# Please refer to the commented-out example below.
skipCommandConfigData: "{}"
# skipCommandConfig: |
# {
# "mysql": [
# "^(/\\*.*?\\*/)?\\s*SELECT\\s+@@session\\s*\\.\\s*\\w+\\s*$",
# "^(/\\*.*?\\*/)?\\s*SET\\s+session\\s+transaction\\s+\\w+(\\s+\\w+)*\\s*$",
# "^(/\\*.*?\\*/)?\\s*SET\\s+net_write_timeout\\s*=\\s*\\d+\\s*$",
# "^(/\\*.*?\\*/)?\\s*SELECT\\s+database\\s*\\(\\s*\\)\\s*$",
# "^(/\\*.*?\\*/)?\\s*SET\\s+SQL_SELECT_LIMIT\\s*=\\s*\\w+$",
# "^SHOW\\s+VARIABLES\\s+LIKE\\s+'aurora\\\\_version'\\s*$",
# "^SELECT\\s+version\\s*\\(\\s*\\)\\s*,\\s*@@version_comment\\s*,\\s*database\\s*\\(\\s*\\)\\s*$",
# "^SET\\s+autocommit\\s*=\\s*\\d+$",
# "^(/\\*.*?\\*/)\\s*SELECT\\s+((@@session\\s*\\.\\s*|@@)\\w+(\\s+AS\\s+\\w+)?(\\s*,\\s*)?)+\\s*$"
# ]
# }
# -- The path where the skip command config file will be mounted.
skipCommandConfigFile: /app/arisa/skip_command_config.json
kubectl port-forward -n querypie deploy/poc-querypie-tools 8050:8050 &
curl -X POST http://127.0.0.1:8050/license/upload -F file=@license.crt
[License] Upload: Success # 결과
# port-forward 종료
jobs
kill %1
# README.md
## QueryPie Deployment (PoC)
- Helm 기반 QueryPie 배포 구성
- Okta OIDC 연동 설계 및 Terraform IaC 구현
- Private Harbor Registry ImagePullSecret 구성
### Known Limitation
- QueryPie container images are hosted in a private registry (harbor.chequer.io)
- Image pull requires vendor-provided credentials
- Without registry credentials, Pods remain in ErrImagePull state
