iCloud Photo Library is only available on macOS Standard, macOS Extended, and APFS formatted volumes.

Where is your Photos Library currently stored?

If you want to keep your Photos Library on an external volume, the external volume needs to be prepared as described here: Move your Photos library to save space on your Mac - Apple Support

Essential are the following points:

• The file system format needs to be MacOS Extended (Journaled), or on macOS 10.14 or later APFS, particularly, if it is an SSD drive.
• The external volume needs to have a wired connection, it must not be a network share.
• The external volume must not have been used for Time Machine backups.

On older system versions you could have been seeing this error message erroneously, even with the Photos Library on an internal volume, if your Mac has a fusion drive. Are you running Mojave on a fusion drive? Then you may want to have a look at this link: How to Fix the Error: iCloud Photos is only available on macOS Standard, macOS Extended and APFS formatted volumes

The user williamfromeaston posted this fix - you just have to run the Disk Utility First Aid tool on the system drive after booting into the Recovery partition:

Re: icloud photos - formatting error Oct 9, 2018 2:49 AM in response to Brad2610a

OK, I just got off the phone with Apple Support and it was much easier than it seems.

First, shut down. Then reboot, and then immediately hold the Command+R, until you see the progress bar, then release.

Then, you should get a screen that has 4 choices - choose the bottom one - Disk Utility.

Select your HD on left and run "First Aid" (first button in center menu). It should take a few minutes to run.

Once complete, restart from Apple menu, and sign in as normal. Your iCloud Photos should be working.

This document explains in more detail how to start the Mac in recovery: How to start up your Mac in Recovery Mode

From the following discussion: icloud photos - formatting error