Brother unlocks iPhone with his face

Also note:

About Face ID advanced technology - Apple Support

"The probability that a random person in the population could look at your iPhone X and unlock it using Face ID is approximately 1 in 1,000,000 (versus 1 in 50,000 for Touch ID). As an additional protection, Face ID allows only five unsuccessful match attempts before a passcode is required. The statistical probability is different for twins and siblings that look like you and among children under the age of 13, because their distinct facial features may not have fully developed. If you're concerned about this, we recommend using a passcode to authenticate. You can also use Face ID without enabling Face ID with a mask."

"Also consider that if you two look alike, and this person knows your passcode.. every time they enter your passcode the iPhone will start 'learning' like it's an alternate appearance of you. So you could try setting up Face ID again, changing your passcode, make sure no one else is in the cameras view, don't share your passcode."

from: My family can unlock my iPhone Face ID - Apple Community

SaadKhan12 wrote:

My brother using their face to unlock my iPhone

Does your brother know your passcode? If so, change it and do not share with people you do not trust.

Were you present in the room when your brother unlocks the phone?

Was your brother present in the room when you first set up Face ID?