Hayoxi's Fulmination Annihilation not working properly.

I am playing a stormweaver and am in maps. Whenever I use a spell DURING the period in which elemental weakness is creating cursed ground, the Annihilation property of Hayoxi's does not activate and instead just creates normal cursed ground. IF i wait for the cursed ground to activate, the Annihilation activates even if i cast spells. This results in having to wait for cursed ground if I want the gem to work properly.

I might be tweaking but i cannot figure out why this is like this

Do you have different skills bond to different waepon sets? I have found the same issue as you, but it specifically doesn't activate if I change which weapon set I'm on before the cures has "primed".

I posted a bug report explaining my experience here too:
https://www.pathofexile.com/forum/view-thread/3890098