Massive FPS drop in Maven crucible and Maven encounter

Im seeing massive frame drops when fighting maven and in the maven boss crucibles. My normal gameplay is around 130-200 depending on the content. Drops to 30-60 when doing Maven content.

Im using Vulkan for my renderer and all my graphics settings are basically default. I have VSync turned off.

Specs
Nvidia RTX 4090
13th Gen Intel® Core™ i9-13900K × 24
32 GB Ram
NVMe 2.0 TB hd