How to update a back up Photos library without re-copying the entire file?

Ideally, you would use a Time Machine backup as it will make incremental backups of your device. For best results, your external drive should be at least 2 times the size of the data you are backing up (for easy reference, twice the size of your Mac hard drive).

Learn how to back up with Time Machine: https://support.apple.com/en-us/HT201250

You can certainly exclude a lot of the other directories on your system so that only the photos folder is backed up.

System Preferences > Time Machine > Options

Navigate to your user folder

Add everything but the photos directory to the list (it's an exclusion list)

That will backup Photos (assuming your library is stored in that folder, which is typical)

You may also get some other incidental system files in the backup some of which you could also exclude (just navigate to higher level folders, your Macintosh HD and exclude the folders outside of users)

You could also use cloning software, like Carbon Copy Cloner. It can update the copies incrementally. But you would have to make the first copy also with the incremental cloning software.