Minions despawning

I have 4 persistent minions and I'm running mind over matter, When in battle and mana is getting low the game will unsummon one of my 4 minions and unreserve the spirit, i then minus the minion in the skill book and press + again and it comes out, this is happening quite frequently, as in multiple times per map

I have this problem too from time to time, but not always.