CKSKubernetes 1.35

Lessons

Scanning images with Trivy and acting on it

updated 2026-10-22 by seed

The task shape

"Find every Pod in the cluster whose image has CRITICAL vulnerabilities and delete
those Pods." Two halves: the scan, and the API-side action.

Trivy

trivy image nginx:1.21.6
trivy image --severity CRITICAL --ignore-unfixed nginx:1.21.6
trivy image -f json -o /tmp/scan.json registry.example.com/app:1.4.2
trivy fs --scanners vuln,secret,misconfig /src     # filesystem, no image needed

Output sections: OS packages, language packages, and for each — CVE id, severity,
installed version, fixed version. The fixed version column is the actionable
part.

# just the CVE ids at CRITICAL, one per line
trivy image -q --severity CRITICAL -f json nginx:1.21.6 | jq -r '.Results[]?.Vulnerabilities[]?.VulnerabilityID'

Scan what is actually running

# every distinct image in the cluster
kubectl get pods -A -o jsonpath='{range .items[*]}{.spec.containers[*].image}{"\n"}{end}' | sort -u |
while read -r img; do
  n=$(trivy image -q --severity CRITICAL -f json "$img" | jq '[.Results[]?.Vulnerabilities[]?] | length')
  printf '%-60s %s\n' "$img" "$n"
done

Then remove the offenders — and remember what kind of object owns them:

kubectl delete pod bad -n prod                       # bare Pod: gone for good
kubectl delete deployment bad -n prod                # Deployment: Pods go with it

Deleting a Pod owned by a Deployment just recreates it. If the task says "delete
the Pod" and it is managed, you must delete the Deployment (or scale to zero), or
the marker finds the Pod still there.

Grype, the other scanner

grype registry.example.com/app:1.4.2
grype registry.example.com/app:1.4.2 -o json --fail-on critical

Fixing, not just finding

  1. Bump the image to a version where the CVE is fixed (nginx:1.27).
  2. Bump the base if the CVE is in the OS layer (apt-get upgrade in the build, or a newer base tag).
  3. --ignore-unfixed: if there is no fix available, the only real mitigations are removing the package or the workload. Saying that out loud is worth marks.
kubectl set image deployment/app app=registry.example.com/app:1.4.3 -n prod
kubectl rollout status deployment/app -n prod

Traps

  • Scanning the wrong reference — nginx is not nginx:1.21.6. Scan the exact string from the Pod spec.
  • No network to pull the image → scan the local copy if there is one (trivy image --input app.tar), or use --offline-scan with a cached DB.
  • The vulnerability DB is stale: trivy image --download-db-only first if the task hints at updates.
  • A CRITICAL in a dependency the app never loads is still a CRITICAL for the marker.