Audio Out through HDMI counts towards maximum displays?

I'm using an M4 Pro, which according to Apple supports up to two external displays. I'm using both of those, one connected with DisplayPort via USB-C, one with LunaDisplay also via USB-C. I also have a relatively high-quality sound system (Teufel Consono & Concept sub), which is capable of HDMI, RCA and USB-C Audio in.

Since I'm already using 2 of my MBP's 3 USB-C slots, I'd like my sound to come out of the MBP's HDMI Out, since I'm not using that for anything else, and also because that's the only way to enable digital audio passthrough in the Music and TV apps. Unfortunately, using the HDMI Out for audio seems to be counted towards the maximum display count supported by the MBP, which I can measure by trying to connect all three at once, which doesn't work, but once I disconnect one, the other two work; no matter which of the three I disconnect, the other two will then suddenly work, and the third won't if I try to connect it again.

Does anyone know what the reason for this is? There's no image information sent through HDMI when using it just for sound, the handshake should communicate this, so macOS should know. Therefore, there would be no reason to count it towards the display limit, since just the audio really shouldn't significantly add towards the processor's load. It might be a technicality with how macOS handles HDMI signals, but then that seems like a fixable oversight; I can't be the only one with this issue (two displays + wanting to use a good external sound system, for which HDMI is a good choice as the connector).

Does anyone know a workaround for this?