Focus changing from heat?

I am printing bnw on a diffuser enlarger and I notice the film’s focus changes. After focusing the image, and after a few test prints I decided to double check focus. Brining my eye to the micro focuser I noticed the grain structure to gradually appear over the span of 3-5 seconds.

Am using a devere 504, panoramic 35mm film and holder, a 75mm lens.

I think this is most likely due to the light source’s heat causing the film to expand, sufficient to change the focus.