When the Music volume slider is turned all the way down to OFF music still plays. If I increase the Music volume slider then more music plays on top of the already present music. I'm pretty sure the music that plays even with the Music volume slider OFF is meant to be restricted to the main menu, but carries on when a world is loaded.

I've had this happen on older versions of MC too, not sure what versions, but you can look this issue up on Google and other people are complaining about it on version 1.7.5!
Mojang claimed to have fixed this issue in the past, but it still occurs.