Welcome to the Linux Foundation Forum!

Exercise 15.2: DevDan doesn't work

Exercise 15.2: Authentication and Authorization

Summary: I can't work with resources associated with the user DevDan, the context DevDan-context, etc.

Details --

Problem symptoms:

$kubectl config use-context kubernetes-admin@kubernetes
Switched to context "kubernetes-admin@kubernetes"

$ kubectl --context=DevDan-context get pods
error: You must be logged in to the server (Unauthorized)

$ sudo kubectl --context=DevDan-context get pods
Error in configuration: context was not found for specified context: DevDan-context

$ kubectl config use-context DevDan-context
Switched to context "DevDan-context"

$ sudo kubectl get pods
The connection to the server localhost:8080 was refused - did you specify the right host or port?

I'm not going to walk through every single step I took - I followed the directions exactly, with the exection of naming the various DevDan auth files "dd.key/.crt/.csr" for brevity.

Here's what I'm working with, with extraneous results removed --

System: Ubuntu 20.04

  • Config details:
$ kubectl version --short
Client Version: v1.24.0
Kustomize Version: v4.5.4
Server Version: v1.24.0

$ kubectl config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://k8scp:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    namespace: development
    user: DevDan
  name: DevDan-context
- context:
    cluster: kubernetes
    namespace: default
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: DevDan
  user:
    client-certificate: /home/nathaniel_lapier/dd.crt
    client-key: /home/nathaniel_lapier/dd.key
- name: kubernetes-admin
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED

$ less .kube/config
contexts:
- context:
    cluster: kubernetes
    namespace: development
    user: DevDan
  name: DevDan-context
- context:
    cluster: kubernetes
    namespace: default
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
current-context: DevDan-context
kind: Config
preferences: {}
users:
- name: DevDan
  user:
    client-certificate: /home/nathaniel_lapier/dd.crt
    client-key: /home/nathaniel_lapier/dd.key
- name: kubernetes-admin
  user:
    client-certificate-data: [...]
    client-key-data: [...]

$ sudo less /etc/kubernetes/kubelet.conf 
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: [...]
    server: https://k8scp:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: system:node:k8stest-raw-1
  name: system:node:k8stest-raw-1@kubernetes
current-context: system:node:k8stest-raw-1@kubernetes
kind: Config
preferences: {}
users:
- name: system:node:k8stest-raw-1
  user:
    client-certificate: /var/lib/kubelet/pki/kubelet-client-current.pem
    client-key: /var/lib/kubelet/pki/kubelet-client-current.pem
  • Remaining exercise-specific settings:
$ kubectl get namespaces
NAME              STATUS   AGE
default           Active   14d
development       Active   17h
[...]
production        Active   17h
small             Active   4d21h

$ sudo less /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
[...]
DevDan:x:1001:1002::/home/DevDan:/bin/bash

nathaniel_lapier@k8stest-raw-1:~$ ls -a
dd.crt    dd.csr    dd.key    .rnd     LFS258    

$ kubectl get roles -A
NAMESPACE     NAME                                         
default       kdash-kubernetes-dashboard                       
default       myingress-ingress-nginx                          
development   developer                                        
kube-public   kubeadm:bootstrap-signer-clusterinfo             
[...]             

$ kubectl get rolebindings -A
NAMESPACE     NAME                                         ROLE 
default      myingress-ingress-nginx   Role/myingress-ingress-nginx                         
development  developer-role-binding                    Role/developer
[...]

Is any part of my confuration not what should be expected? I've walked through a slew of Googled tips around users, contexts, permissions etc. This post shows the results of my second attempt at this exercise - after hitting this dead end on my first attempt, I deleted everything from the exercise and started over, with no effect.

Answers

  • pnts
    pnts Posts: 33
    edited November 2022

    kubectl --context=DevDan-context get pods is correct and should be able to work.

    You do not want sudo when using kubectl.
    It will look for kubeconfig in /root/.kube/config which does probably not exist.

    2 things I would try:

    1. kubectl --context=DevDan-context get pods -v=10 Anything to learn from the verbose output?
    2. Try embedding the client cert and key in the kubeconfig using --embed-certs: kubectl config set-credentials DevDan --client-key dd.key --client-certificate dd.crt --embed-certs
  • nlp
    nlp Posts: 8

    @pnts said:
    kubectl --context=DevDan-context get pods is correct and should be able to work.

    You do not want sudo when using kubectl.
    It will look for kubeconfig in /root/.kube/config which does probably not exist.

    2 things I would try:

    1. kubectl --context=DevDan-context get pods -v=10 Anything to learn from the verbose output?
    2. Try embedding the client cert and key in the kubeconfig using --embed-certs: kubectl config set-credentials DevDan --client-key dd.key --client-certificate dd.crt --embed-certs
    $ kubectl config set-credentials DevDan --client-key dd.key --client-certificate dd.crt --embed-certs
    User "DevDan" set.
    
    $ kubectl --context=DevDan-context get pods -v=10
    I1110 23:14:07.097741 2828059 loader.go:372] Config loaded from file:  /home/nathaniel_lapier/.kube/config
    [tons of kube/cache jsons...]
     curl -v -XGET  -H "Accept: application/json;as=Table;v=v1;g=meta.k8s.io,application/json;as=Table;v=v1beta1;g=meta.k8s.io,application/json" -H "User-Agent: kubectl/v1.24.0 (linux/amd64) kubernetes/4ce5a89" 'https://k8scp:6443/api/v1/namespaces/development/pods?limit=500'
    HTTP Trace: DNS Lookup for k8scp resolved to [{10.2.0.4 }]
     HTTP Trace: Dial to tcp:10.2.0.4:6443 succeed
     GET https://k8scp:6443/api/v1/namespaces/development/pods?limit=500 401 Unauthorized in 13 milliseconds
     HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 9 ms ServerProcessing 2 ms Duration 13 ms
     Response Headers:
         Content-Type: application/json
         Content-Length: 129
         Date: Thu, 10 Nov 2022 23:12:00 GMT
         Audit-Id: 95f6a005-64bf-4b97-a21b-415618577460
         Cache-Control: no-cache, private
     Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Unauthorized","reason":"Unauthorized","code":401}
    I1110 23:12:00.710028 2826896 helpers.go:222] server response object: [{
      "kind": "Status",
      "apiVersion": "v1",
      "metadata": {},
      "status": "Failure",
      "message": "Unauthorized",
      "reason": "Unauthorized",
      "code": 401
    }]
    error: You must be logged in to the server (Unauthorized)
    

    Lots of output, but in the end it's just a 401, even with the certs.

  • nlp
    nlp Posts: 8

    Hah - I'll eat some crow and hopefully help the next person who gets stuck:

    openssl req -new -key DevDan.key \
    -out dd.csr -subj "/CN=DevDan/O=development" #make sure this -subj argument exactly matches your user and namespace
    

Categories

Upcoming Training