Wireless controller connects but won't interact with anything. Constant issues

I'm using a PS% controller and playing Geforce Now via the Nexus app. The controller has always had issues, but recently it's getting worse and worse.

My controller will connect only if I set it to pairing mode first. Or else it will always connect to my mac in the house. (This controller hasn't been used by anything else) and was bought new. Then if any apps are previously open BEFORE connecting the controller, they won't pick up the input of the controller unless I force close the app and open again.

But it's worse now, the controller connects to the vision pro. I can identify it in the settings, it vibrates.. but other games and the nexus app still wont receive input from it. It's like the vision pro connects.. but doesn't process input from the controller. I usually need to forget the device -> Close all apps -> re-pair -> open Nexus -> pray -> repeat

Any ideas? Anyone else having these issues?