Humming totem in ritual immune to damage

Greetings,
Recently I tried farming ritual with my impending doom character. Sometimes after I clear the ritual there is only humming totem left, which seems to not be able to take any damage from my spells, feels like only my minions slowly wittle it down.