Can only hear out of one headphone for music but both for phone calls

I have had 6 different generations of the iPhone and then same problem persists. No matter what type of headphones I use, I can only hear music out or they left headphone. During calls, I can hear out of both headphones. Anyone had this issue and fixed it? I’ve backed up my data, restored the iPhone to factory settings then restored it and that did not work.