Can't use portals from map to hideout sometimes

Sometimes openend portals in a map can't be interacted with and don't show a name above them. After relogging it's possible to enter the map again but the portals still aren't working. It seems like it's a instance specific issue as it isn't happening every map but if it happens the issue persists until a new map/instance is loaded.
Video showcase is linked. I'm playing on Xbox Series X.
https://www.xbox.com/play/media/wNqZu6DACS