With Controller how do I prioritize objective before monsters

I don't know how to prioritize objectives. It hasn't been much of a burden till now in act 2 during the trials where I have a time limit have to reach each objective, with a controller I'm forced to kill the monsters before I can click the what I need to. Mind you I've only tried once and failed miserably but since this is a game mode within the game I would like to know is there a way I can prioritize the objective over the monsters

You need to keep the "hand" button unassigned (or better you only have the hand assigned but not other skills).

thx for the help :)