Ping spikes from 80 to 400ms when entering The Eater of Worlds

My game runs okay in all other modes. But when I enter the instance to fight The Eater of Worlds ping goes to 400ms -> impossible to fight. After coming back to the hideout, the ping goes back to 80ms.
I've tried different servers, the issue is the same.