Navigation issues with Skill screen on controller

This is happening on an Xbox Series X with a controller and the Panel Mode set to Left and Right.

When trying to use a Lesser Jeweller's Orb, it usually does not allow you to navigate down using the left nor right stick. Edit: I previously thought Greater and Perfect Jeweller's Orb don't have this issue as they hadn't when testing, however I just had a Greater do the same behavior.

(It seems a temporary fix for the Lesser Orbs may be to hold the back button to go to full screen (where navigation seems to work), as after testing that I was able to navigate using LS/RS on panel mode. You can also unsocket the gem and use the orb on it while it's in the inventory as a work around.)

Whatever causes this also seems to apply to socketing support gems on Left and Right panel mode. I have a Skeletal Archer gem at the bottom of the list, I went to socket Rapid Attack I and it would not navigate down past my innate weapon/shield skills which are towards the top of the list. (This too can require full screen to properly get things socketed.)

Edit: Tonight I am trying to set up a Duelist Wand's Spellslinger with a Comet skill gem, it is unable to be socketed using panel mode and I had to use full screen.