Renaming files

I see no built in way to rename a photo and to replace the IMG _001 file name. You can send a photo to the files app and rename it in there but when you send it back to the photos app, the new name is not shown. It still comes up as IMG_001

When you locate the photo or any file in the files app, tap and hold down on the file to bring up the contextual menu where the third option is to rename the file.

What photo icon are you talking about? Please provide details.

You can add a caption by tapping on the icon with the lower case "i" at the bottom of the screen. However, I do not believe it's possible to rename the file from Photos on an iPhone.

Tap on the sharing icon at the bottom left (box with an arrow coming out of it). Select "Save to Files".