If you've ever cringed hearing your voice on a recording, there's a good reason why: you're so exposed to the voice you hear from inside your body that listening to it outside just sounds just weird and bad.
This video from Mental Floss explains that we're made to process the sounds of our own voices differently because we're really processing two channels of sounds at once. There's the sound of our voices outside of our bodies, or air-conducted sound. And there are the vibrations caused by our vocal chords and bones, which make the pitch of our voices sound deeper. This is what's known as bone-conducted sound.
But the reason why you might hate the sound of your voice over the phone is because, well, you hear it so frequently that anything different just sounds inferior. But if you're a musician or work with extensively with your vocals, you'll be exposed to the sound of your air-conducted voice much more frequently, making it much easier to deal with.