I keep hearing that from people. But everyone I heard that from refuse to get decent clothes! (base layer, gloves, socks). Even when it's 30 degree! It's actually from over-heating wearing not breathable clothes...I hate stopping for more than like 5 min. Legs get heavy. You get cold again.
I have it the reverse. If it's cold out, my feet get cold and need to go inside to warm up. Other times, I may not be wearing the right jacket. But I never "get cold" from stopping or going indoors!