"Output Dialogue to Chat" option broken

I disabled the option to output dialogue to chat.

I still see all NPC dialogue in chat. I also tried turning it on to check if the option is reversed, but the text is still displayed. Also restarted the game.

Game language is English. Not sure if that helps.