Mighty Silverfist fight - swarm of other mobs

I've seen a lot of discussions about the one tap, and how it's unfair etc. but I have a different issue wit this fight now.

I have been having a problem this league where I am mid-fight with Lord Monkey and get sent endless mobs from the map to also deal with, including rares. Even when i do my best to strictly keep the fight in the ruin area where he is initially.
So if I want to just fight him alone, I have to clear the entire surrounding map every time I die and ain't nobody got time for that!
Last season I had this guy on second try with my sorc, didn't have this happen, now with melee I'm struggling due to the other mobs joining in every time!
Now I appreciate that I am not some amazing meta player so my build is probably not great but still, can't we have some restriction on other mobs (RARES too) agroing on you mid fight??? Especially this fight where it's quite important to position yourself perfectly... Super frustrating...
Anyone else having this issue?

Stick your butt between the ruins in the centre and dunk on him. There you can fight him, 10 of him and 100 mobs too.

same issue yeah. it almost feels as if he is calling his friends of the forest to come screw up my slow methodical approach to combat with this dodge-checking, big-stick-wielding, donkey-kong-rolling banana boy.

"

InvRnd#4436 wrote:

Stick your butt between the ruins in the centre and dunk on him. There you can fight him, 10 of him and 100 mobs too.

You underestimate my noobness. :D But I will try that specific spot again.

"

rhorains#6760 wrote:

same issue yeah. it almost feels as if he is calling his friends of the forest to come screw up my slow methodical approach to combat with this dodge-checking, big-stick-wielding, donkey-kong-rolling banana boy.

Haha yea! Good to know I'm not alone at least! I feel like it wasn't like this last season, not sure if they bumped the mob density on this map or something. Or I just didn't notice last time, not sure...