Huge lag when Delirium is on a map

Since the last patch or so, i've been having huge lag spikes despite staying at 130-144fps.
Before starting Delirium it's stable, and the second I go through the portal the shader graph goes ballistic. Network frames also fluctuates a lot, resulting in huge stutter (And server catchup where everything goes fast for a few seconds).

It stays even after delirium is over, I've had a couple breaches and they turned into a slideshow as well.

Frame time didn't move much, and changing graphics settings didn't do anything either.

Without delirium on the map I can blast big packsize breaches with no issue whatsoever.

I'm playing Spark with Cast on Freeze Comet, as a Stormweaver



Specs :

RTX 4080 Super
AMD 7800x3D

32Gb DDR5 Ram
Good SSD

Update : I tried running delirium with my gas grenade mercenary, and didn't notice any stutter.

The weird thing is that it's not about monster density since I can run double breaches with big monster density without any issue on the spark mage.

Did you happen to figure out a fix for this? I have the same exact issues with Delirium.

Perfectly fine until there are mobs on my screen and then my network and shaders start to go crazy.