Please do something about losing spectres outside combat

Is there any good reason corpse item spectres don't go into spectre pool?

For about the fifth time this league, I had an issue with spectres disappearing, and this time I lost all of them. I simply logged in to find my helmet on my mouse cursor, which of course is where they are socketed. No idea how that happened.

I'm really struggling to face buying them again, as the 15 div that cost is a huge expensive for me, it's at least half of all the currency I've invested in this league. How does it make the game better to have this one mechanic that can erase your progress through trivial little misclicks? I don't even think it makes the game more fun to lose them if they die in combat, but at least that doesn't feel cheap.