Radek Welcome to the community!
I assume that UPnP is active on your modem. With an active UPnP, the ports on your modem are automatically opened for an internal IP and closed again when not in use. If you don’t want this, disable the function and restart your modem. The opened ports via UPnP will then be released again and no new ones will be reserved, you have to do this manually.

Greetings
Daniele