So you sound like me. I had the RX470 and it worked great, would sit IDLE at 45*C and I never really played any games. Then I bought ESO and would play it and my computer would freeze. I had completely wiped my HD and reinstalled Windows 10x64. Reloaded drivers, everytime, it would freeze within 10 minutes of entering the game.
Then I noticed that the GPU was heating up, so I popped off the stock cooler and replaced the thermal paste with some good MX4 stuff and while it did drop about 10*C at idle (and now sits at 30*C regularly at idle in 76*F ambient), it would still freeze in game. Then I realized that it's probably heatsoaking a component and that component is causing an issue, so I put a fan on the side of it, sucking air out and viola! I can play ESO on the highest quality settings at 1080p 60fps without an issue and it never froze! So you sound like what I had, it was ridiculous, but that's the issue, these coolers are heatsoaking the board and causing a component to fail. I downloaded the Xtreme Game Engine and set the fan to a constant 80% or more when playing ESO now and it moves enough air to keep temps down and prevent the board from getting too heatsoaked. 100% sounds like a turbine engine but it'll run 45*C. I plan on getting Arctic Xtreme iii-120 next month when I get paid to better handle the issues.