drop specter bodies when unequipping gem

I'm completely frustrated, twice in a row I lost all my specters by accidentally equipping another helmet. I'm on the console and I accidentally pressed A while I was in King Mach, and my specters disappeared and didn't drop bodies, I lost 3 div in specter, and this is the second time this has happened, why don't the specters work like the AG that is saved in the gem when unequipped? Or at least it always drops the bodies when unequipped, this is very frustrating.

I lost Perfect Warlord, Perfect Dancing Sword and a Perfect Guardian Turtle in a similar way.

Was... sad.

I wish there was an interface of some sort to manage your corpse collection.