Ringer volume between speaker and airpods

Is there a way to set a different notification “ringer” volume between the speaker and AirPods?

Right now it is either too quiet on speaker, or way too loud on AirPods unless I manually change the volume when start/stop using the AirPods.

This is not the seamless experience I expect of these.