Acceleration Shrine lowering Action Speed after effect

I've had this happen a couple times:

1. I use an Acceleration shrine and get the buff.
2. Once the effect ends my action speed is permanently way slower than normal, movement speed seemed to be about the same.

Porting out and back in seems to reset everything back to normal.

Edit:

In case it helps, playing Lightning Invoker, active auras are Herald of Thunder, Ghost Shroud and Combat Frenzy.