No. We never took him out. It warmed up the next day. The way that I rigged the burrow worked so well, I think we'll keep it til the really cold season starts.
It's better for me because I can do it myself. Just drag the mattress in front of the entrance and turn on the heater! It gets up to 75F in there while its 50F outside
It's essentially a "shed" for him without having to remove him from his home!