Wildwood spectres lost when changing gear

I am fed up with constantly losing my expensive wildwood spectres when I am merely updating my gear
I am on Xbox

After losing a 1 divine spectre the first time when I changed weapons in my hideout, I have been ultra careful about first going to town to make any adjustments to my gear (adding an eldritch implicit, etc)

However I have just experienced this for the 4th time -- when I return to the hideout my spectre count is reduced from 5 to 4 (as my shield that I tweaked had a +1 specter) and inevitably the missing spectre is one of the wildwood irreplaceable/un-desecratable spectres.

I have read the FAQ and see the GGG comment that when this happens the wildwood spectre will appear as a body when I reenter the hideout. This does not happen and I am out yet another divine.

Very, very frustrating.

Does anyone ever read (much less bother to respond) these??

Just lost another 5do when my finger slipped off a joystick. This is getting to be a bad joke -- I have lost at least 15do to what amounts to trivial mechanical slipups.

It is really that difficult to fix? When I return to my hideout, allow the wildwod specters to reappear when I desecrate. Or allow me to disable weapon swap (although that doesn't fix the problem I had last time where I accidentally unequipped my amulet losing the extra spectre and about 3do)

Whenever I DO try to open/close my backpack it is super stressful as I am so scared I will slip and lose a spectre. Really takes a lot of fun out of the game!!!