Is it a sin that somehow, in my mind, the Virgin Mary has this gentle, sweet, male voice?
It's not a sin, but it is also not an appropriate use of your imagination. The Virgin Mary is a woman. Projecting masculine traits onto her is neither useful nor an accurate representation of God's Mother. I can only recommend to you that you do your best to stop imagining the Blessed Virgin speaking to you in a male voice.