Vaal temple lost

I entered a vaal temple, the first one I've entered in act 4, I set up my tiles and entered the temple and had to go do something for a while so I exited to the character screen. Logged back in at the temple console, lost a bunch of tiles and was unable to enter the temple to run the map. Feels bad.