Welcome to the Linux Foundation Forum!

Lab 3.2: pods evicted, and pending after deploying try1

nbon
nbon Posts: 14
edited January 2021 in LFD259 Class Forum

After I create a new deployment:

the pods never come online, instead, all the pods get evicted, and even the other pods get evicted.

how do I fix this?

Comments

  • nbon
    nbon Posts: 14
    edited January 2021

    this comment was commented in error and should be deleted (dont delete the post, just this comment of the post)

  • chrispokorni
    chrispokorni Posts: 1,962

    Hi @nbon,

    The described behavior indicates that the second command, the docker push, in Step 28 may have been missed or it did not get executed when issued the first time (which also happens occasionally).

    Regards,
    -Chris

  • nbon
    nbon Posts: 14

    @chrispokorni I edited the original post sorry, does your comment still apply?

  • pbbhaskar
    pbbhaskar Posts: 15

    can you run 'kubernetes get events' ?

  • serewicz
    serewicz Posts: 1,000

    Hello,

    If you don't have enough resources it may evict pods.

    Are you running 2CPU/8G nodes?
    What does kubectl get node show?

    Regards,

  • nbon
    nbon Posts: 14
    edited January 2021

    @serewicz said:
    Hello,

    If you don't have enough resources it may evict pods.

    Are you running 2CPU/8G nodes?
    What does kubectl get node show?

    Regards,

    I'm running two VMs on Windows 10 with HyperV (I couldn't use GCP), they both have 4GB of RAM

    I'm not sure how much CPU they have, but my physical laptop has Intel CPU i7-8665U on it.

    It looks like they're definitely out of "ephemeral-storage"
    Here's the Events for the try1 pod:

    Here's the amount of RAM and Disk available on nickkube2 (the second node):

  • serewicz
    serewicz Posts: 1,000

    Hello,

    Undersized VMs will cause brown-out type issues which will lead to evictions and odd behavior. I would say you do not have enough resources and these issues will persist until you add CPU and probably more memory. From the error you may not have enough disk resources for the containers as well.

    Regards,

  • chrispokorni
    chrispokorni Posts: 1,962

    Hi @nbon,

    As you may have realized already, my earlier comment was related to the original content of this Discussion thread, and not the current modified issue.

    I would recommend keeping Discussions with their original titles and content, in order to keep all helpful solutions and feedback from the Linux Foundation staff relevant to the posted issue. Keeping the Discussion and all comments within the same context not only helps you, but also helps learners who may encounter similar issues in the future and search through the forum for possible solutions. With that in mind, when a new issue is encountered, please open a new Discussion thread, unless it has already been addressed in an earlier Discussion.

    Regards,
    -Chris

  • nbon
    nbon Posts: 14
    edited February 2021

    I increased the size of the VMs, but am still seeing pods being evicted when creating the try1 deployment. I changed Assigned RAM to be dynamic.

    @chrispokorni , I will do that going forward.

  • nbon
    nbon Posts: 14
    edited February 2021

    what is "ephemeral storage" and why was simpleapp using 44ki "which exceeds its request of 0"? How do I request more than 0 ephemeral storage?

  • chrispokorni
    chrispokorni Posts: 1,962

    Hi @nbon,

    Kubernetes does not seem to like dynamic virtual resource allocation, so I would recommend all resources of your nodes to be assigned, instead of dynamic.

    Regards,
    -Chris

  • nbon
    nbon Posts: 14
    edited February 2021

    @chrispokorni I changed to fixed 8GB memory for each node, same issue.

  • chrispokorni
    chrispokorni Posts: 1,962

    Ephemeral storage refers to disk space. Is that also dynamic?

  • nbon
    nbon Posts: 14
    edited February 2021

    It's fixed now!
    I learned that HyperV limits disk to 12GB by default, so I increased it to 64 GB, after increasing it on the master node, it started working.
    I also used gparted on the linux VM itself to expand the partition to fill the rest of the disk.
    Deployment try1 works now.

Categories

Upcoming Training