Cant Progress in Simi if you go to town..... for trade etc

This seems to occur more often than not when i'm doing a Simi and someone wants to buy something I have to go to town and trade. However when I come back to the Simi and try to progress it will not let me activate the encounter...

Idk if its because i'm using Rite of Passage and the primate permanent minions have anything to do with it or because I used the provided portal to go to my hideout instead of creating my own. Regardless it is not allowing me to continue with the simi any longer. I am unable to do anything other than start a new one.


This has occurred at least 5 times now. I was hoping this issue would be fixed.