Practicing skills point 33 review6.yaml

borislav.velkov

What is the idea behind finding the correct nginx uid and fixing the pod? If I changed the security context to 101(nginx uid) it still fails.

juampynr

I think that it is about setting the uid to root so it can boot. I did set both user and group to 0 (root) and then the pod would run successfully.

serewicz

Hello,

Indeed, sometimes changing the UID may be an answer, but then the application running inside the container would fail. As a result you may have to change the security context. Security context may be set via an admission controller, not the developer of the application, which may then conflict with the way the container was configured. As a result one or the other would need to be modified if the goal is to have the pod running.

Regards,

kareldebeus

I'm still a bit confused about the answer given above, do they really mean just changing runAsUser to root id=0), that easy.
Question 33 and 34:
33. After finding the errors, log into the container and find the proper id of the nginx use.
34. Edit the pod such that the securityContext is in place and allows the web server to read the proper configuration files.

Why do they say "find the proper id of the nginx use"? Once I get container running by using root, I see that id of nginx is 101

Using runAsUser 101 to start pod/container fails with:
2021/01/16 23:41:41 [warn] 1#1: the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2
nginx: [warn] the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2
2021/01/16 23:41:41 [emerg] 1#1: mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)
nginx: [emerg] mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)

I just want to be sure I understand correctly.
I hope somebody can clarify this for me.

Regards,

JanAdam

Could you elaborate this a little more @serewicz ?
As @kareldebeus pointed out: setting runAsUser =0 allows the Pod to start but defeats the purpose of a securityContext which should normally restrict access. I find that confusing. Is there more behind answering question 33 than we currently see? Or is runAsUser=0 the correct answer?

youwalther65

To be honest I struggled with this question as well. The simplest way is just removing the whole securityContext sections but I thought this solution is to simple ...
So I ended up with:
$ cat review6.yaml
apiVersion: v1
kind: Pod
metadata:
name: securityreview
spec:
securityContext:
runAsUser: 2100
containers:
- name: webguy
image: nginx
securityContext:
runAsUser: 0
allowPrivilegeEscalation: false

but it is basically the same as removing all securityContext. So I guess the task is a little bit misleading. Would be interested to hear other opinions as well!

serewicz

Hello,

If you were required to always have a security context, but someone else had set the wrong security context, how would you fix the problem such that the pod runs but you are meeting the security team requirement that there is a security context set for the pod?

Regards,

dleerob

I'm also confused by this question. It states "After finding the errors, log into the container and find the proper id of the nginx user". I am assuming by "login" you mean attach to it? How are we supposed to attach to it if its not running?
If not, then this question is very badly worded. I have no idea what it's asking me to do.

dleerob

I eventually moved on from this domain review, didn't even bother trying to finish it, the whole thing is so badly worded/explained, no context given on what they want us to achieve. Will go find another resource to learn about Security.

Chatting to other members of my organization doing this course, they also just skipped this calling it "a mess".

serewicz

Thanks for your feedback.

You realize the whole point of the domain review is to give less than all information such that you are testing your own knowledge and ability to function as a system administrator would be called upon to do. correct?

You do realize the exam will not tell you what to type, and that you will have to understand more than following along and figure out what needs to be fixed or configured, correct?

wmaz02

The issue with that task is that we don't know what the constraints and successful implementation are. [I tend to think about those questions in the context of the exam], so I will use the simplest methods first. That would be simply removing security context. If that's not allowed, it should be explicitly stated (this also changes how we can login into container, if I'm restricted from running anything, even temporarily, as root, I would have to overwrite entrypoint).
Now, assuming that we are restricted by [being unable to remove] security context, what exactly is meant by "re-create the pod such that the pod runs without error"? If it means that it must behave exactly as before [when run as root], I don't believe this is possible at all (again, the restrictions being I cannot run anything as root, so not sidecar trickery for example).
It is simply not possible to bind to port 80 non root.
Adding capabilities (NET_BIND_SERVICE) seems obvious choice, but it does not work with non root, see KEP here about ambient capabilities https://github.com/kubernetes/enhancements/issues/2763
The rest is workable (by remounting and redefining required paths and configs), but the port must be moved to >1024. Of course we can add a service to remap it, but again, if this is acceptable, it should be explicitly stated/reworded (eg. by saying that nginx must respond on port 80 when called from other node/externally)

This is a good problem to work out, it's a pity people would skip it because it's pretty unclear what is expected.

kingphil

I found a solution to this that I believe sticks to the spirit of the task (namely, running as the nginx user), but also might be a bit strange. I won't spell it out, but definitely spoilers ahead.

Both runAsUser clauses can be 101. The problem is the basic permissions issue the nginx service has, as it tries to create a cache directory (in a dir owned by root), and later to write a .pid file (in a dir owned by root). I solved these by making the containing dirs mounted to emptyDir volumes. Is that odd, or totally what the examiners would be looking for?

pawelzajac

@kingphil

Thanks for the tip I got it worked out in about 2h
That is a tricky one (I found so far) but workable and indeed secure.

Doesn't change the fact that the task is not clearly formulated (unfortunately) not the only one.

I found one article on the net which give a very good clues (not going to spoil either, but assure everyone that it can be done).

Generally as you have mentioned it is the way nginx manages the /var/cache/nginx and its .pid file in Docker.

jonlar

Am I correct to assume that to configure this properly, you will have to do changes in the application? And changing the "runAsUser" to root is just a workaround to get it running?

pawelzajac

@jonlar it works without making changes to application.
Solved by @kingphil

jonlar

@pawelzajac @kingphil got it working, thanks!
I though the nginx config had to be change to use another port than 80, but that wasn't necessary.

guettli

I don't like this hide-and-seek game which the Linux-Foundation does here with us. We paied for the course and there are several people who struggle with this excercise. Why not publish the correct answer?

It is compleltly unclear what is allowed to get this working. Am I allowed to remove the security context? Am I allowed to change the container image? There are many ways to force a solution.

guettli

This yaml works:

apiVersion: v1
kind: Pod
metadata:
  name: securityreview
spec:
  #  securityContext:
  #  runAsUser: 2100
  containers:
  - name:  webguy
    image: nginx
    securityContext:
      #runAsUser: 3000
      allowPrivilegeEscalation: false

But I have no clue, if this is a valid answer.

pawelzajac

@guettli I understand your frustration.
Not to spoil to much the task and as you are right this is a training I will post here a tutorial which contains the answers, so is not like I give it on plate.
https://bridgecrew.io/blog/creating-a-secure-kubernetes-nginx-deployment-using-checkov/

I think it is generally very good document and worth of sharing.
Good luck.

tranthepq

This is a bit tricky, look at the startup logs, we can found that the user does not have permission to write to /var/cache/nginx/ and /var/run

Not sure if this fix is valid to the exam

apiVersion: v1
kind: Pod
metadata:
  name: securityreview
spec:
  securityContext:
    runAsUser: 2100
  volumes:
    - name: cache
      emptyDir: {}
    - name: run
      emptyDir: {}
  containers:
  - name:  webguy
    image: nginx
    volumeMounts:
      - mountPath: /var/cache/nginx/
        name: cache
      - name: run
        mountPath: /var/run
    securityContext:
      runAsUser: 3000
      allowPrivilegeEscalation: false

behren

So, we still don't have clear, unambiguous instructions for this task and nobody from LF is willing to provide them. This is frustrating, disappointing and does not help anyone willing to find the right solution and nobody will ever find out what this exercise is trying to teach us.

@serewicz You do realize that the exam has to provide clear instructions and a desired result in order for me to solve a task, correct? You do realize that providing zero information to what the desired outcome of the task is makes it basically impossible to solve, correct?

Instead, this task is doing the complete opposite and is misleading users into some obscure tasks like finding the nginx uid which won't help solving this exercise at all.

behren

@tranthepq said:
This is a bit tricky, look at the startup logs, we can found that the user does not have permission to write to /var/cache/nginx/ and /var/run

Not sure if this fix is valid to the exam

apiVersion: v1
kind: Pod
metadata:
  name: securityreview
spec:
  securityContext:
    runAsUser: 2100
  volumes:
    - name: cache
      emptyDir: {}
    - name: run
      emptyDir: {}
  containers:
  - name:  webguy
    image: nginx
    volumeMounts:
      - mountPath: /var/cache/nginx/
        name: cache
      - name: run
        mountPath: /var/run
    securityContext:
      runAsUser: 3000
      allowPrivilegeEscalation: false

Well, this does not work for me as it's not allowed to bind to port 80 as non root or am missing something?

headkaze

Check out this answer on stack overflow

magnezone150

I just went with this for two reasons, least time spent and its CKA not CKS so welp.
However, If I was at work presented with this would be a different story since I have dev and security teams. Also root is the only user on container with /bin/bash access that isn't /bin/false or nologin.
But that's my opinion since you could add a network policy and/or even build your own container.

apiVersion: v1
kind: Pod
metadata:
  name: securityreview
spec:
  containers:
  - name:  webguy
    image: nginx
    securityContext:
      runAsUser: 0
      allowPrivilegeEscalation: false

johannespn

I also struggled with the lack of a clear solution here

wrender

Great course, but I agree. This one is a bit too much of a trick question in my opinion. Just seems to lead to ambiguity. The question "find the proper id of the nginx user" really leads you to believe you are expected to run this as user 100.

Maybe a better example would be to take a different container image that can run as a different user than root? (create a local user for.. use DevDan for example) This would demonstrate that someone can indeed run a container as something other than root which is a common real use case for this out in the wild.

lianwong

If like me, you encounter this error
"[emerg] 1#1: bind() to 0.0.0.0:80 failed (13: Permission denied)"
after you have solved the directory mounting issue (checkov article was quite helpful), this link might help:

https://github.com/kubernetes/ingress-nginx/issues/4061#issuecomment-1118439600

YMMV, as I encountered this error on my VirtualBox Ubuntu cp node, and it took me around 4 hours to find out this solution.

anthoony

Hello,

Just had this issue, and indeed, I don't follow the point to use securityContext to run as root at the end.

Posted on bad topic: https://forum.linuxfoundation.org/discussion/comment/41555/#Comment_41555

For me, about checking .33, it say to check the uid of nginx.

Didn't make sense to check that if we use root at the end imo...