Help me to understand how containers should spread accross nodes with differents nodeSelector labels

damien.vergnaud

Hello,

According to this lab exercise :

https://trainingportal.linuxfoundation.org/learn/course/kubernetes-fundamentals-lfs258/scheduling/lab-exercises?page=1

Step 6. We create a manifest to deploy 4 containers inside 1 pod with or without a nodeSelector label.

At the step where you shouldn't apply the nodeSelector label

My containers always go to the "worker", even if i increase the number of vip containers created by the pod object.

I'm using minikube + this tutorial https://minikube.sigs.k8s.io/docs/tutorials/multi_node/
In order to have multiple nodes.

I removed the "Master" role of the first node to be in the same situation than the example.
There are no taints.

With NodeSelector, all containers run in one node or the other.

Can someone help me in understanding why i don't get the same result as the exercice state ?
Thank you

oleksazhel

@damien.vergnaud as worker node has less pods than CP scheduler puts pod into worker node. The smallest unit in Kates is pod and not a container. And as a result of this statement containers inside pod cannot be spread accross different nodes (only pods can). That's why no matter how many containers are in your pod it always will land to the node which is under less load.

damien.vergnaud

Hey, thank you for taking the time to answer my questions, @oleksazhel.

So :

• Pod definition is deployed to worker because of less pods.
• All containers gets to worker because pod definition is deployed to worker and pod is the smaller unit kubernetes can deal with.

So why is this sentence in the document of the learning course talking about "Containers".
It clearly confused me, especially the "They should be more evenly spread this time"

Quote :

10. Determine where the new containers have been deployed. They should be more evenly spread this time. Again, the numbers may be different, the change in numbers is what we are looking for. Due to lack of nodeSelector they could go to either node

oleksazhel

@damien.vergnaud because by default CP has more containers then worker node and when you remove nodeSelector status: vip it means that pod will be assigned to less loaded worker node and in turn it means containers should be more evenly spread this time. This is not about our lab pod containers', that is about containers from all pods.

chrispokorni

Hi @damien.vergnaud,

I believe the intent was to describe how pod replicas are spread across nodes when the nodeSelector property is not set in the pod spec. Under default scheduling conditions, without a nodeSelector, the overall expected result is to see all the pods of the cluster evenly distributed across the available nodes - that includes control plane pods and user pods. However, the per-node container count may not reflect the same even distribution, especially if we introduce multi-container pods.

Regards,
-Chris