I'm with you.
Took a while to 'train' my daughter but basically if you drag a toddler back inside as soon as they take their hat off they soon get the message.