I want to reduce the size of files to be transferred to new iMac

The files will need to be reduced in number, or in size, on the old iMac before you begin any network Migration Assistant transfer between the two iMacs. Large libraries such as Movies or Photos may need to be moved to an external drive before the transfer, and then afterward, accessed from that media.

If using the Time Machine drive from the old iMac, the first sentence would not apply as there is no means to reduce file size on that backup before or during the transfer to the new iMac.

You don't state what types of files, as some may already be at their smallest size owing to data compression used during their creation.

You can move outdated file(s)/folder(s) in your home folder (and not part of your Library) to external media. You also do not want to migrate any applications from your old to new iMac. On the new iMac, install compatible applications confirmed for the new iMac operating system version, as old 32-bit applications will not run on macOS Catalina or later.