There are a lot of reasons :
-The first one can be that you are using your Internet connection by WLAN (WiFi). Using the WiFi, you will never have 500/50. It is always better to use the Internet connecting a device by a ethernet cable when the device allows you to do this.
-The second one can be a technical problem in your region. Maybe, there is an internet congestion (too many customers of upc cablecom using the internet at the same time around you).
-And finally, the third one. Your modem can be defective (very rare).
In your case, I bet more on an internet use by wireless because your upload is perfectly fine and not the download. And you talked about 5 and 2.4 ghz connection. You should try to connect (if it’s possible) one of your device to your modem with an ethernet cable. Then, you do a speed test to see if you have 500mbps on download.