Apple Vision Pro turn off virtual keyboard

I have connected an apple bluetooth keyboard to my Vision Pro. But the virtual keyboard keeps popping up and interrupting, rendering the device useless for writing (I'm using Scrivener). I followed the steps I found online, but it keeps reappearing and, again, it makes it impossible to write.

I tried the whole "settings > Accessibility > keyboards.... thing multiple times both on and off.