Temple no longer thought I owned it after inviting a friend

I was in a temple, partially completed.
Invited a friend in, after a bit I opened a portal so I could stash some loop.
However, I couldn't re-enter the temple from where I warped out to.
I had to portal directly to my friend and then enter the temple there.

We completed the temple that had like 2 rooms left.
I opened a new temple, we went in, however I couldn't interact with the tables etc... the way I should be able to, it was like I wasn't the instance owner.

We got stuck in the room where you have to interact with that big multi-tentacle machine, but I couldn't interact with it and neither could my friend.

So, had to leave the temple in like room 3 of 42. Kind of annoying.