Lab 6.6 security-review1.yaml intended solution

bulldog98

Version: 2022-11-23
I was able to find out the userid that nginx uses by default, fix the problem with cache, but was not able to figure out (without trying to google) how I get rid of the could not bind port 80 error. Also my tries with capabilities failed, since finding out which would be needed requires research again.

My intuition tells me running the pod as root cannot be the correct solution, what am I missing.

fazlur.khan

Hello @bulldog98

Few take away from the security-review1.yaml

1. In the yaml, we use security context "runAsUser" at the Pod Spec and also at the container level. What we define at the container level takes precedence. In this case it will run as user 3100.

2. But, the pod creation is failing... The reason is the nginx image is built to run as root. It needs write access to /etc/nginx to create the conf file and also /var to create a cache file.

3. We know running as root is not good, so how can we fix it? Well, you can fix it while creating your image, for example using the Dockerfile directive such as "USER" while creating your image.

4. Since we are not creating the image here, what can we do? - We can mount /var/cache and /etc/nginx as volumes with write access and then pod will be created.

Give it a shot, try fixing it. If you need further help, LMK.

icastillejos

FWIW I got it to work using port 8080 (i.e. above 1024) and moving the pid file to one of my mounted volumes. I followed this example to build the nginx conf in a ConfigMap: https://gist.github.com/petitviolet/d36f33d145d0bbf4b54eb187b79d0244

bulldog98

@fazlur.khan ah yes the /var/cache I already found, but my problem was more that the nginx user was not allowed to open port 80

keithwilliam

1. Review the security-review1.yaml file to ensure that it meets security standards and best practices
2. Analyze the security-review1.yaml file to identify any potential security issues.
3. Examine the security-review1.yaml file for any potential security threats or weaknesses
   4.Check the security-review1.yaml file for any potential security flaws or weaknesses.
4. Scan the security-review1.yaml file for any potential security risks or vulnerabilities.

cameronkerrnz

Here's what I'm trying:

• set spec.containers[0].securityContext.runAsUser to be 101, the same as the nginx user in the container image (although I don't like this solution, I'd prefer a solution where root in the container maps to a unique UID on the host)
• using an emptyDir volume for /var/lib/nginx (it wants to create proxy_temp, client_temp, and more)
• set spec.containers[0].securityContext.capabilities.add[0]."NET_BIND_SERVICE"... but that doesn't appear to be effective as the container still gets a permission denied when binding to 0.0.0.0:80

So I went to diagnose what was happening. A altered my Deployment to create the pod with command as ["/usr/bin/sleep", "inf"]... this means it won't start nginx but will just cause the a container to start, which I can then exec into to inspect the environment:

nginx@securityreview:/$ grep ^Cap /proc/1/status
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: 00000000a80425fb
CapAmb: 0000000000000000

Well that's kinda interesting, because the CapPrm ('Permitted', or maximum) capabilities is empty, so something must be removing capabilities.

Compare this with another what we did earlier:

cameron_kerr_nz@a-cp:~/review6$ k exec secondapp -c busy -- grep ^Cap /proc/1/status
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: 00000000aa0435fb
CapAmb: 0000000000000000

cameron_kerr_nz@a-cp:~/review6$ k exec secondapp -c webserver -- grep ^Cap /proc/1/status
CapInh: 0000000000000000
CapPrm: 00000000a80425fb
CapEff: 00000000a80425fb
CapBnd: 00000000a80425fb
CapAmb: 0000000000000000

And here's the configuration from that:

  containers:
  - name: webserver
    image: nginx
  - name: busy
    image: busybox
    command:
    - sleep
    - "3600"
    securityContext:
      runAsUser: 2000
      allowPrivilegeEscalation: false
      capabilities:
        add: ["NET_ADMIN", "SYS_TIME"]

Comparing with the review exercise, it would appear that when securityContext.capabilities.add has been set, we're seeing that CapEff (the 'Effective' set) is not as expected.

It is very useful to realise that this is because of the behaviour of Linux Capabilities when transitioning from UID 0 to UID !0. There's a long-standing bug about this:

• https://github.com/kubernetes/kubernetes/issues/56374
• workaround using sysctls instead to set the range of priviledged ports: https://github.com/kubernetes/kubernetes/issues/56374#issuecomment-928917495
• the other fix for this (lacking ambient capabilities) is to set NET_BIND_SERVICE capability on the executable instead (ie. on nginx). --- I tried this, it wasn't enough
• https://github.com/kubernetes/enhancements/issues/2763 <-- the enhancement that made it as an Alpha in 1.24 (but ended up being a CVE?).

I would say that it should be required to read about this issue, because you're very likely to bump into it; and your experience is likely to be quite unpleasant.

What I ended up doing that did work:

• Use runAsUser: 101 in the container securityContext
• Don't use any capabilities
• Instead, use the sysctls in the Pod's securityContext (not the container's, but the pod's)
• Provide a custom nginx.conf that doesn't specify user and sets the various paths (as per the documentation in the section 'Running Nginx as a non-root user' at https://hub.docker.com/_/nginx)

While you're reading about these issues, I found it useful to compare with what OpenShift does; all pods will run as a non-privileged dynamic UID, but with GID 0. So anything that should be writable by the container should have group-write permission. An interesting way of doing it, although one that many container images aren't written to expect. I suspect there will be container runtimes that can use 'rootless' containers, where the UID running 'within' the container may be 0; but that is mapped to a non-root UID outside of the container. (podman does this already; presumably CRI-O could as well; I haven't looked).