LFS260, k8s Security - LAB7.4 - required modification in tracee s configmap
I installed Tracee 0.24.1 on a kubeadm Kubernetes cluster with two nodes (control plane + worker).
The Tracee DaemonSet pods were starting and generating events correctly, but both stayed in 0/1 Running because the readiness probe was failing.
The error was:
Readiness probe failed: Get "http://<pod-ip>:3366/healthz": dial tcp <pod-ip>:3366: connect: connection refused
The Tracee process itself was clearly working because logs contained normal eBPF events and detections.
The ConfigMap originally contained:
perf-buffer-size: 1024 healthz: true metrics: true pprof: false pyroscope: false listen-addr: :3366
The pod was correctly started with:
["/tracee/tracee"] ["--config","/tracee/config.yaml"]
and /tracee/config.yaml inside the container contained the expected configuration.
However, port 3366 was not listening.
Running:
/tracee/tracee --help
showed that this Tracee version exposes the HTTP server configuration through the --server option:
--server stringArray Examples: http-address=:3366 metrics healthz pprof pyroscope
I therefore changed the ConfigMap from the old top-level server options to:
perf-buffer-size: 1024
server:
- http-address=:3366
- metrics
- healthz
log:
level: info
output:
json:
files:
- stdout
options:
parse-arguments: true
stack-addresses: false
exec-env: false
exec-hash: dev-inode
sort-events: false
I applied the change with:
kubectl -n tracee patch configmap tracee-config \
--type merge \
--patch "$(cat <<'EOF'
data:
config.yaml: |-
perf-buffer-size: 1024
server:
- http-address=:3366
- metrics
- healthz
log:
level: info
output:
json:
files:
- stdout
options:
parse-arguments: true
stack-addresses: false
exec-env: false
exec-hash: dev-inode
sort-events: false
EOF
)"
Then restarted the DaemonSet:
kubectl -n tracee rollout restart daemonset tracee
After the restart, both Tracee pods became healthy:
tracee-hjcj9 1/1 Running tracee-xdmz6 1/1 Running
So the issue was not Tracee itself, the CNI, or the readiness probe definition. The problem was that the old-style configuration did not start the HTTP health server on port 3366, while the newer server: configuration did.
Categories
- All Categories
- 178 LFX Mentorship
- 178 LFX Mentorship: Linux Kernel
- 775 Linux Foundation IT Professional Programs
- 384 Cloud Engineer IT Professional Program
- 175 Advanced Cloud Engineer IT Professional Program
- 75 DevOps IT Professional Program - Discontinued
- 7 DevOps & GitOps IT Professional Program
- 103 Cloud Native Developer IT Professional Program
- 7.7K Training Courses & Learning Paths
- 15 AI & ML Training
- 1 Blockchain & Decentralized Identity Training
- 42 Cloud & Containers Training
- 5 Cybersecurity Training
- 4 DevOps & Site-Reliability Training
- 3 Linux Kernel Development Training
- 2 Networking Training
- 2 Open Source Best Practice Training
- 5 System Administration Training
- 1 System Engineering Training
- 6 Web & Application Development Training
- 798 Hardware
- 202 Drivers
- 68 I/O Devices
- 37 Monitors
- 96 Multimedia
- 173 Networking
- 91 Printers & Scanners
- 92 Storage
- 774 Linux Distributions
- 82 Debian
- 68 Fedora
- 24 Linux Mint
- 13 Mageia
- 24 openSUSE
- 151 Red Hat Enterprise
- 32 Slackware
- 13 SUSE Enterprise
- 356 Ubuntu
- 473 Linux System Administration
- 31 Cloud Computing
- 73 Command Line/Scripting
- Github systems admin projects
- 104 Linux Security
- 79 Network Management
- 102 System Management
- 46 Web Management
- 172 Mobile Computing
- 32 Android
- 125 Development
- 1.2K New to Linux
- 1K Getting Started with Linux
- 407 Off Topic
- 128 Introductions
- 35 Study Material
- 1.1K Programming and Development
- 311 Kernel Development
- 726 Software Development
- 1K Software
- 419 Applications
- 183 Command Line
- 5 Compiling/Installing
- 71 Games
- 320 Installation
- Archived
- 183 Small Talk
- 2 LFD140 Class Forum
- 1.4K LFS258 Class Forum
Upcoming Training
-
August 20, 2018
Kubernetes Administration (LFS458)
-
August 20, 2018
Linux System Administration (LFS301)
-
August 27, 2018
Open Source Virtualization (LFS462)
-
August 27, 2018
Linux Kernel Debugging and Security (LFD440)