One possibility here, is a weak or poor Wi-Fi signal between the Router and Mac mini. The Mac mini's are well know for having Wi-Fi and Bluetooth issues, because their wireless antenna is built into the bottom cover. To make matters worse, they can take a big hit sitting on a Hub and/or from poorly shielded cables that are plugged into the Mac mini. Normally this happens with high speed USB and Display cables, but it is possible for any cable that emits an EMF field to cause a problem.
Resolve Wi-Fi and Bluetooth issues caused by wireless interference - Apple Support
For some solutions that might help improve the Wi-Fi connection and overall signal strength.
1) if possible move the Router and Mac mini closer together or use a Wi-Fi extender
2) stand the Mac mini on it's side with the bottom facing towards the Router
3) move all other cables from in-between the Router and Mac mini