Scion ascendency refund

I was specced into guardian and pathfinder, went to respec and i cannot take out the two points traveling to either ascendency. I can take out the large point that gives me guardian or pathfinder, but not the two previous points. I have enough respec points, how do i fix this?

Getting 4th ascendency fixed my ability to remove those points. all good now