Specific Golem Mastery

Is it possible to combine a skill/attack gem relative to the color gem a golem is created (or not?) in which you could control what skill the golem uses? Not necessarily access to all skills, but unarmed melee or even possibly caster (since fire golems do spew forth)?

If not golems, then animate guardian and control its attack method