Cannot add Apple Pay to Wallet on iPhone

maybe

Can’t add Apple Account balance into wall… - Apple Community

If you can't add a card to Apple Wallet to use with Apple Pay - Apple Support

more

your account balance cannot be added to wallet at this time - Google-

suppose it if was me then I would have a talk with the bank that gave you the card and maybe apples support to try to resolve it

Get

Support (apple.com) 

And

or

 Contact

- Official Apple Support

And

or

Contact - How to Contact Us - Apple

Or

Genius

Bar Reservation and Apple Support Options - Apple

Funds in Apple Account are a cash equivalent and cannot be transferred to a bank account (demand deposit account). Funds in your Apple Account can only be used for purchases at Apple.com/store or at Apple Retail Stores.

You can review the Apple Support article I linked to below to learn more about your Apple Account.

What you can buy with your Apple Gift Card or Apple Account balance - Apple Support

Apple Cash may not be the best choice, depending on your needs. You need a bank account and debit card to send and receive in most cases. You might want to look at Cash App, Venmo and PayPal as an alternative. Their cards work in Apple Wallet with Apple Pay and it’s easier to move funds in and out.

Do you still need assistance with this?

Is the $5 on your Apple Account or Apple Cash? They are two different services and accounts.

How to receive and send without a bank or credit card

Using Apple Cash to transfer money

So, you have funds in Apple Cash. What do you want to do with the funds?