Exercise 6.6: Domain Review - LoadBalancer and Security Context

lyndsey.d.ferguson

Hello,

Is there some sort of Answer reference that I can refer to? I am stuck on Exercise 6.6: Domain Review:

1. I do not understand how to test that the LoadBalancer Service works. I can create the LoadBalancer, but this is a new concept that has not been discussed yet. I've read online that it only works on GCE, and that's all I found. Using curl to test the IP hangs.

2. I have not figured out how to fix security-review1.yaml so that the securityContext stays in place (both?) and it works. I have looked up the ngnix user id and tried using that, tried changing the context so that it is the same fsGroup as root, etc, but I don't know what I'm doing here, online docs, make it seem that I have to run a script beforehand as root in order to make run the nginx start command as that user, etc. It is not clear here: I would appreciate some sort of answer key or help.

lyndsey.d.ferguson

I meant to type GKE, not GCE

chrispokorni

Hi @lyndsey.d.ferguson ,

When working on GCE (not GKE) you may still be able to test the LB type Service, if you create an LB in your VPC and assign the LB IP to the appropriate attribute of the Service. Then test your Kubernetes LB type Service with the public IP of the external LB.

Regards,
-Chris

lyndsey.d.ferguson

Thank you for your response @chrispokorni ; to confirm what I understand, you're saying that I need to create a LoadBalancer in the GCE account, and update it to have the IP address that the k8s LoadBalancer has?

chrispokorni

Create an LB on GCE. Upon creation, it will be assigned a public IP address.
Create an LB type Service in Kubernetes, which will be assigned an internal private virtual Cluster IP.
Then edit the Service to add the external public LB IP in the Service's configuration.

Some examples from the documentation:
https://kubernetes.io/docs/concepts/services-networking/#loadbalancer
https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/

Regards,
-Chris

lyndsey.d.ferguson

Thank you Chris, your help on the Load Balancer has been most helpful! I also continued on the course and see that Load Balancers are explained in the next section, which leads me to believe the question was meant to be part of section 7.

Would you have any guidance on my struggles with security-review1.yaml (see above) ?

chrispokorni

The best resource is Lab exercise 6.1 and while reviewing it understand the effect of each security context, at pod level and container level respectively. For this situation understand the effect of the conflicting security contexts, which one gets to override the other? Is the overriding user allowing the container to run? Why or why not?

Regards,
-Chris

lyndsey.d.ferguson

Thanks for the suggestion Chris. I knew that the container's securityContext needed to be changed. However, after that I still needed to configure how the nginx user ran. I will leave this note as a map to future readers so that they too can get the nginx container to run as non-root.

Pablo.Camino

Hi @lyndsey.d.ferguson , did you manage to resolve this? I am still puzzled by this. I understand I have to change the container to run as user 101, but maybe that is wrong. Can someone assist a little bit on this?

lyndsey.d.ferguson

I can help, but I don't want to give you the answer as the lesson won't stick in your mind. A couple of things:

1. I tried very hard to try and change the running user to match the permissions of the files I thought I needed: false move.
2. Observe the logs of the pod as it is starting up: what do you see? What is failing? Is there a way that you can change things around so that it won't fail? Think back to other lessons (perhaps even prior sections....)

Try that for a couple of hours (in total, this took me 6 to figure out, and when I did, it took 2 minutes to ultimately fix).

Let me know after you've really really really tried.

Pablo.Camino

Thanks fro getting back so quickly. That's fine, I would like to finish the challenge, but I feel like am wondering off. All other lab task were fairly simple and limited to the specific lesson.

I will let the user go for now, though it seems strange. I get that it fails because it can't create a directory. I used a volume so the directory is created before hand. Now I'm running into bind() to 0.0.0.0:80 failed. I've tried adding capabilities to the container but it didn't work.

I am thinking on tweaking nginx to listen on another port. Maybe I can customize the command line somehow. But it seems way off-topic to go and do that. @chrispokorni suggestion made it sound like it was something very related to the user.

I'll leave it for now but a if you can spare a tip it it will be appreaciated.

lyndsey.d.ferguson

You’re following the same path that i did. Look up how to solve the bind port.

This was the hardest question of the entire course. 10x more. It is only slightly related to the user ID. That is only the start.

Pablo.Camino

I got to set a custom config for the nginx and bind to 8080. But now I get nginx: [emerg] open() "/var/run/nginx.pid" failed (13: Permission denied). I think this got way out of scope, it can't be this way. Did you also face this?

I'm starting to think this is something much simple like setting nginx to run as user 0, but that would be way too simple.

lyndsey.d.ferguson

You’re almost there! It is something somewhat simple, but I had to struggle as you are....take a look at what controls how nginx is launched. How can you change that?

lyndsey.d.ferguson

Wait, you just wrote that you are using a custom config. That’s what I did. You just have to update the config to specify a path for that pid file that it can write to (tmp).

Pablo.Camino

Thanks, I was also reading about it here: https://medium.com/@kaur.harsimran301/run-nginx-as-unprivileged-user-in-docker-container-on-kubernetes-6e71564cf78b
I didn't want to look for a full blown example but gave up. I was exactly were you say, I was missing to change the PID. I guess I didn't realize it was something to change and was thinking of it as something that you must access there.

Even though it's running, I don't feel much satisfied with this solution. I mean, it's running as non-root but those security contexts could be anything. This was mostly about configuring an Nginx. At this point I would like to see the official answer, not sure where to request that. I mean, was really this what the instructor was expecting us to do?

I can see I learned quite a bunch on configuring Nginx but little on securityContexts.

lyndsey.d.ferguson

Agreed. We also learned about using volume mounts to replace a file.

lyndsey.d.ferguson

I am surprised there is not a way to get official answers. I would have also liked a test with checks on my answers.

suser

I am stuck here too. Regarding the securityreview pod, I think we should match the user id on container securitycontext with the user id running nginx on container. Or not?

lyndsey.d.ferguson

That's part of it

Pablo.Camino

Hi @suser . The only option I found to use a security context that allows this to run is simply:

    apiVersion: v1
    kind: Pod
    metadata:
      name: securityreview-root
    spec:
      securityContext:
        runAsUser: 2100
      containers:
      - name:  webguy
        image: nginx
        securityContext:
          runAsUser: 000
          allowPrivilegeEscalation: false

This works but seems to defeat the purpose of having a security context as you are just running as root which is the default.
The solution that @lyndsey.d.ferguson provided, or hinted through, gets this running as non-root and is a great learning but the securityContext could be any non-root user.

There is a line on the lab that I couldn't figure out what it points to:

After finding the errors, log into the container and find the proper id of the nginx user.

First you need the container running to log into it.
Then I couldn't find any proper user id. 101, which is specified in the nginx.conf, doesn't solve anything.

tknoops

Would appreciate if someone can post the actual solution to this question.

joem86

It would be good to know what the intention was for this question. I used the information in this post to help get me past it, but it seemed like we needed to research the implementation details of the nginx container to successfully exercise our knowledge of security contexts. Granted, this matches closer to what we encounter in the real-world. I would like to know if this was on-purpose though.

serewicz

Hello,

Indeed, this was an intended part of the domain review. As with the rest of the course the intention is to present common skills as well as common hurdles and issues, to start you on the road to mastery.

Regards,

dbspace1

Can some one post the solution for 6.6 or give the hint of whether a custom config map is needed, because if the solution is simple, I don't want to miss it

dbspace1

Here are the steps that I do to solve the issue. first use something similar https://medium.com/@kaur.harsimran301/run-nginx-as-unprivileged-user-in-docker-container-on-kubernetes-6e71564cf78b, then use initContainer to fix permission issue. Not sure whether that is the best solution

dbspace1

or you can just add all the permission files in the configmap, that seems to be a simple solution

serewicz

Hello,

You may be making the issue more complex than needs be. I think it's great you thought of several ways to solve the problem. Definitely a good skill in open source.
If a Pod were using an permission or user name that did not match what the application requires, you could change the way your application works, or update the security context setting the permission or user name of the pod to match the application.

Regards,

shirleydst

I have wasted a lot of time trying to get item 2 in Domain Review 6.6. working.... perhaps because it should be in the next domain review? Correct?

shirleydst

And I agree that the security is not fixed by using nginx user 10, but rather by changing container-user to 0

gusmenp

I think that one sentence in the exercise description cause some kind of misunderstanding: "... find the proper id of the nginx user."

Find the id of the nginx user does not resolve the problem using the domain of the lesson, this is, the security context and the effect of the conflicting security contexts. Because setting the id of the nginx user (101) in the security context inside the container actually override the value assigned in the PodSecurityContext, which indeed is the domain of the lesson, but this doesn't resolve the problem because the nginx user has not permission to run some process and to modify some files in the filesystem of the official nginx image.

Using the content of the lesson is necessary to override the PodSecurityContext user, setting the runAsUser in the container securityContext to the user needed by the nginx image to run its process correctly, the user in this case is the root (value 0).

Also, I think is valid to modify the container securityContext to runAsUser: 101, set allowPrivilegeEscalation to true or comment allowPrivilegeEscalation and add field privileged with value true. This lets you run the nginx server with the user nginx but equivalent as root (From the kubectl explain command: "Privileged field run container in privileged mode. Processes in privileged containers are essentially equivalent to root on the host. Defaults to false. Note that this field cannot be set when spec.os.name is windows"). I think this is valid as the instructions in the exercise say: "Edit the yaml and re-create the pod such that the pod runs without error" it doesn't specify that the only modification that can be done is in the runAsUser field.

Some proposed to use configmaps to mount a volume with a config file, that can be in case you need to load a custom config for the nginx server, but I don't think this is the case.

Another way to run a nginx server inside a container with the nginx user (id 101) and without using the root is to change the image used to nginxinc/nginx-unprivileged (or create a custom image using the official nginx image as the base image), but I think this wasn't the idea of this exercise.