As muguy says, all your iCloud Drive files should be locally stored on your Mac internal drive. It's copies of these files that are made available at iCloud.com. If you've chosen to use "Optimize Storage," though, some files may be removed from the Mac if you're running out of space. How much storage space is available on your Mac? Apple recommends at least 10% or your storage remain free.
New in Sequoia is the option to insist that certain files never be removed from your local drive. Right clicking on a file in iCloud Drive brings a pop-up menu
that gives you the option to keep a file downloaded on your Mac, or to remove it until you need it. A new icon represents a file that's kept removed. Try using this option and see if it speeds things up.