Knowledge Check 4.8, no_grad() vs. inference_mode()

p.hanel

I searched the lab and chapter 4 and only found one mention of the term "no_grad", which was 04 > Evaluation > Context Manager.

"
Up until PyTorch 1.9, there was only one context manager available for this: torch.no_grad(). However, since then, we may also use torch.inference_mode() as the context manager, which offers improved performance over its predecessor.
"

SPOILER:
The question, "It is best practice to wrap the validation inner loop with a torch.no_grad() context manager." I thought was misleading because from the former quote I believed the answer to be "torch.inference_mode()". The correct answer was no_grad. Since there was no explanation in the answer could you please briefly explain why this is the case?

Thank you 😊.

dvgodoy

Hi @p.hanel ,

You're spot on once again
Both no_grad() and inference_mode() are context managers that are better for validation/inference because they do not keep track of the dynamic computation graph, as no updates are involved in these processes.
Earlier versions of PyTorch had only no_grad(), and inference_mode() was released later to offer yet a little bit more speed than no_grad() (which can only be perceived when using large models).
So, yes, the question needs updating so it reads "It is best practice to wrap the validation inner loop with a torch.inference_mode() context manager".
Thank you for reporting this!
Best,
Daniel

dharmon

I appreciate the catch and the clear explanation. The course has been updated with the correct version of the question.

p.hanel

Thank you for the information @dvgodoy & @dharmon 😊.