How do I remove a photo from one album without deleting it from another album? or the itself?

photos has different flavours of albums. You can remove photos only from standard albums you create yourself. Only the standard albums are offering the "remove from album" option, as described by FoxFifth.

It is not possible to remove photos from smart albums or the predefined albums created automatically for us by Photos. The smart albums and predefined albums (like Imports, recents, Selfies, etc) are populated automatically according to rules specific for the album. You would have to delete photos from the library to remove them from these albums, and that would delete the photos everywhere.

Open the Photos app, navigate to the album, control click on the photo, click on "Remove 1 photo from album." You can also first select multiple photos in that album if you have more than one that you want removed.

It won't delete the photo from other albums or from the Library.