Continuous Stuttering/Lag Despite Stable FPS and Internet in PoE 2 Early Access

I just want to went a bit: I have been having issues with network lag since the start, sometime 1 map works perfectly and then the next on has ms between 50 an 20000 constantly, without breaks. sometimes when I die and respawn it magically works, mnost of the time it doesn't help.
I have tried everything possible to resolve the issue on my side. It is just so frustrating trying to enjoy my favorite game and being unable to. And yes, the issue is 99% there on weekends :D just when I have time to play