As I understand it, UDP packets are buffered because they are not ACKed like TCP packets are. These buffers
can fill up if they can't be passed on quickly enough to the destination port and basically get "backed-up",
causing the problem that you describe.
I have seen this maybe 40-50 times since I started helping people with their RemoteRig setups about eight
years ago. The most problems I saw were with the German Fritz routers, but I have seen that across the board
with Netgear, Linksys, etc.
Anyway, it was almost always solved by just rebooting the router.
73,
Mitch DJ0QN / K7DX