Losing damage when equipping an amulet?

Using a martial artist I lose damage when equipppping an amulet. Even if the amulet is +1 melee gems.

I lose approximately 1000 damage when equipping an amulet from having literally nothing.

I'm having a similar issue with the Martial Artist.

I lose damage equipping any amulet and my damage goes up without one.

I'm losing about 3k of damage putting one on.

EDIT: I refilled my health at the well and now my damage is back up.
No clue what's going on but it seems to have fixed itself 🤷