Whats your personal opinion on keeping halters on whilst a horse in in the pasture?

I have been always told. No. as in, if-you-put-your-horse-in-the-pasture-with-a-halter-on-it-will-probably-suffer-and-DIE kind of super-serious no.
I always lead into the pasture, close the gate, remove halter, then exit. Never ever have I left a halter on, but I have occasionally seen horses out grazing with them on.