Welcome to the Linux Foundation Forum!

Knowledge Check 4.8, no_grad() vs. inference_mode()

p.hanel
p.hanel Posts: 8
in LFD273 Class Forum

I searched the lab and chapter 4 and only found one mention of the term "no_grad", which was 04 > Evaluation > Context Manager.

"
Up until PyTorch 1.9, there was only one context manager available for this: torch.no_grad(). However, since then, we may also use torch.inference_mode() as the context manager, which offers improved performance over its predecessor.
"

SPOILER:
The question, "It is best practice to wrap the validation inner loop with a torch.no_grad() context manager." I thought was misleading because from the former quote I believed the answer to be "torch.inference_mode()". The correct answer was no_grad. Since there was no explanation in the answer could you please briefly explain why this is the case?

Thank you 😊.

Best Answer

  • dvgodoy
    dvgodoy Posts: 12
    Answer ✓

    Hi @p.hanel ,

    You're spot on once again :smile:
    Both no_grad() and inference_mode() are context managers that are better for validation/inference because they do not keep track of the dynamic computation graph, as no updates are involved in these processes.
    Earlier versions of PyTorch had only no_grad(), and inference_mode() was released later to offer yet a little bit more speed than no_grad() (which can only be perceived when using large models).
    So, yes, the question needs updating so it reads "It is best practice to wrap the validation inner loop with a torch.inference_mode() context manager".
    Thank you for reporting this!
    Best,
    Daniel

Answers

Categories

Upcoming Training