Welcome to the Linux Foundation Forum!

Lab 3.1 : Creating insecure registry on k8s

mrmcmuffinz
mrmcmuffinz Posts: 9
edited May 2018 in LFD259 Class Forum

I'm curious why we went with a ClusterIP service since it is only reachable via the cluster itself for the insecure registry instead of a NodeIP? My problem is that I went ahead and copied the k8s configuration file from the master node onto my local macbook pro. This allows me to manage the cluster from my laptop instead of on the master node. Typically in a development environment you are not developing off of the master node or any of the k8s nodes to begin with. 

Comments

  • chrispokorni
    chrispokorni Posts: 2,606

    Hi, 

    You are right, the ClusterIP would be accessible only from inside the cluster, which I believe was the idea behind this exercise. For access from outside the cluster - the host, you would need a NodePort type service.

    -Chris

  • mrmcmuffinz
    mrmcmuffinz Posts: 9

    Honestly, I don't know if we should have even used Kompose to covert the docker-compose deployment to k8s. I think I would have benefited more from understanding how to deploy the registry from scratch than using that method. Perhaps the author was looking for an easy way to do this but me as a student I want to learn how to do it not do it the easy way. I feel like some of these labs were not well thought out or done to hastely. 

  • sunnystormy
    sunnystormy Posts: 1

    I'm in the same boat. These labs seem to be inconsistent when trying to execute them. I ended up using Minikube to get a single-node cluster going (for 2.1) because the Ubuntu shell scripts didn't work properly (used a VM). Right now (on 3.1) I'm getting connection issues with my cluster, so I'm just pushing forward, even though I'm not getting the expected results in the exercises. :(

  • chrispokorni
    chrispokorni Posts: 2,606

    Hi,

    I know how frustrating it is when your own results are different from the ones presented in the lab manual. All the labs have been beta tested and all commands and outputs were reproduced several times. For consistency however, each lab was completed on Google Compute Engines inside VPCs. 

    Can you provide some details about your setup, we may be able to figure out what causes the errors mentioned above. The error outputs may also help. 

    Are your VMs in the cloud, or local VBox/VMware?

    If your infrastructure is good, then the next culprit is yaml indentation. If the whitespacing is not correct, yaml files will cause a lot of headache.

    Regards, 

    -Chris

  • adamrouns
    adamrouns Posts: 1
    edited August 2018

    As was mentioned earlier, the local repository service is tied to clusterIP and not a NodePort. Therefore, page 14 of the lab guide is incorrect when it states to configure the minion to connect to the clusterIP of the master and pull simpleapp. Please provide the correct procedure to connect the minion to the master repository.

  • chrispokorni
    chrispokorni Posts: 2,606
    edited August 2018

    Hi, 

    You are correct about the registry being tied in with the master's ClusterIP. Subsequently, the minion will use the same ClusterIP to connect to the master's registry, just as presented in the lab manual.

    Are you getting any errors at this step? Can you provide an output to help to identify what causes the error, if any?

    A few troubleshooting steps would be to check if the firewalls are disabled on both nodes, and if all traffic is enabled on both nodes. Are the VMs local or cloud?

    Thanks, 

    -Chris

  • SteveMayne
    SteveMayne Posts: 3
    edited October 2018

    I created a NodePort service to access my registry running on a VirtualBox multi-node deployment with calico, but I'm curious as to how this should work on GKE without NodePort. If ClusterIPs are only addressable within the cluster network, where does the author of Lab 3.1 want us to run "curl http://10.110.186.162:5000/v2/" from? exec into a pod and run it from there? Or does GKE do something funky with routes/bridges to allow access from nodes straight into the cluster network ip range?

  • SteveMayne
    SteveMayne Posts: 3
    edited October 2018

    I suspect this issue is pod-network-type dependent. Calico doesn't create a route/bridge from the node network to the cluster network, but I'm told that Flannel (and maybe some others) do.

    I suggest the instructions at the top of Lab 2.1 page 3 saying:

    You should now deploy a pod network to the cluster.
    Run kubectl apply -f [podnetwork].yaml​ with one of the options listed at:
    https://kubernetes.io/docs/concepts/cluster-administration/addons/

    Should be changed to specify a suitable network to complete the rest of the labs.

  • githingeorge
    githingeorge Posts: 4
    edited December 2018

    For local registry lab, i am getting error after kompose step.
    curl http://10.106.87.30:5000/v2/
    {"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}

    With docker-compose up it works fine. But after running it in K8s it doesnt work
    And i would like to know whether i can continue with the course without localregistry lab setup

  • chrispokorni
    chrispokorni Posts: 2,606

    Hi @githingeorge ,
    I have seen similar errors at this step when the nodes were not part of a VPC network on GCE. Are you using cloud VMs? Do you have a VPC network created?
    Regards,
    -Chris

  • githingeorge
    githingeorge Posts: 4

    Yes, I am using Google Cloud VM instances. But this is not because of VPC network. I get the same error if i try it from the same node/instance. I have tried to run curl http://127.0.0.1:5000/v2/ from within the registry container and that also gives me the same error

  • chrispokorni
    chrispokorni Posts: 2,606

    @githingeorge
    Do you have a VPC network and your nodes inside that network?

  • githingeorge
    githingeorge Posts: 4
    edited January 2019

    its the default VPC network and firewall is open for http on all ports on the worker node instance
    Another thing i noticed is for all the lab exercises, the pod placement is on master node, but for me it was always on worker nodes. So initially i and other people taking the lab will face issues regarding executing curl commands from Master node and not working as there pods are on worker nodes. I had to change firewall rules on worker nodes to make my curl commands work from master nodes.

    The lab pdf should mention this in lab 1 itself. I think only in lab 3 it even mentions some thing related to network/firewall

  • chrispokorni
    chrispokorni Posts: 2,606

    Hi @githingeorge ,
    Working out of the default VPC has caused me some issues on this lab, but when I created a custom VPC with a firewall to open all ports, all protocols, from all sources, I was able to complete this lab and move on to the next.
    Lab 2.1 in the Overview section lists all the requirements when working on GCE VMs, before running the installation scripts.
    Regards,
    -Chris

  • githingeorge
    githingeorge Posts: 4

    Ya created new network with everyhinh open and added new instances to it. Now it s working .
    Thanks guys

  • chrispokorni
    chrispokorni Posts: 2,606

    Hi @TITYKOUKI ,
    Your question was addressed earlier in this Discussion:

    <...> there are many options and ways to access the cluster. In this case it was to only done this way to as one of several choices. In later labs you'll make use of NodePort and LoadBalancer, and learn the advantages and disadvantages of each <...>

    Regards,
    -Chris

Categories

Upcoming Training