I mix it up more than I used to, when I wouldn't run without music.
I'll only listen to tunes in the woods if it's a longer run. I always wear them on easy road runs because otherwise I get too bored. And I couldn't imagine a dreadmill run without it. I also wear them to start hard workouts, but as I fatigue I yank them out. I don't remember when it happened, but my connection got lost towards the end of a hard workout some time ago and rather than trying to fix it I decided to just focus on completing the workout. I have no clue if I am actually faster or not, but I feel like I am accomplishing more without music once I get to that intangible point. When I'm not longer listening to the music - it's just noise. So when I get to that point, which is different day-to-day, I take out the headphones.
Basically, it's rooted in distraction. If the music is distracting then I won't listen to it.