Unreachable Gem Socket on item

I'm playing on Linux with Proton 9.0-4 selected in Steam and use an XBox Controller.

For reference:


When I try to socket the red gem I can switch to every red socket on my gear except for the empty one. It also does not work if there is a gem socketed in that slot.
The socket simply cannot be reached with the controller.

If I don't have a gem selected, navigate to the body armour and press B to select the sockets on the item, I can reach that empty socket though.

It seems the problem is caused by having a gem selected for placing in a socket.