Game sounds take precedence while listening to podcasts.

I often listen to podcasts while playing some mindless games. More recently, the podcasts stop playing entirely about ten seconds after open the game app. Is there a way to "prioritize" which app get access to the speaker? I would like to mute the game sounds while listening to podcasts.