POE2: Buff can't be activated for one specific gem

I tried two different Cast on Critical gems:
- Level 21, 20% quality
- level 19, 20% quality

Both gems have 20% quality, no difference in reservation efficiency, it shouldn't matter anyway as I have 281 spirit and no other auras activated.

The core problem:
The Level 21 CoC gem's buff can't be activated, while the level 19 one works. My character flashes and the shadow on the spirit bar, showing the reservation amount appears and disappears with each right click. When I move my mouse the spirit bar doesn't show any change and the buff does not work.

I can't give further instructions for reproducing the bug, as it seems to be tied to that specific skill gem only. The gem has worked before and I don't remember doing anything specific to brick the gem, other than swapping around some gear.

I made a video showing the bug: https://youtu.be/07ACdfnLUaE?si=MsMI9U5x02xGrgM8

Update: I found out how to fix it, swapping my main weapon setup to the second set makes it work again. Using my wand/sceptre in the first weapon set slot makes the aura unusable, despite me having enough spirit.