Can't go back to map after completing Loathesome Mire

After completing the Loathesome Mire (clicking on the altar), the portal that spawn to teleport me back to the map is broken. I get the error message : Failed to join instance, disconnected from server. This has happened to multiple times already.