I understand and see what you are referring to.
This current version of the Myst app does launch with a splash screen and publisher video followed by a book with turning pages at the bottom of the screen. That appears to be a buffering icon and an indication of the game data being loaded. Eventually the game menu should appear, within 10-15 seconds.
If the game freezes at this point, you can try to exit by pressing Command-Q (⌘Q) as with any Mac app.
Then quit all other apps, restart the game and try again.
I understand you have already restarted the Mac to try to get beyond this, so the next step would be to delete the game from your Mac by dragging the app icon from the Applications folder to the Trash and then emptying the Trash. Then return to the App Store and download a fresh copy and try again.
Also, note the Myst system requirement says Sonoma 14.5 is the minimum. Check that your Mac is running the latest Sonoma update now at 14.8.2.
If the game still refuses to launch properly, then a refund may be in order.
This is Apple's guidance on how to request a refund for an App Store purchase:
Request a refund for apps or content that you bought from Apple - Apple Support