Is followers only chat bad?

So I've been streaming for a few months now and I turned on followers only chat (no time restriction) to try to stop all the bots from dropping links in my chat all the time.

I haven't had any issues up until a few days ago I had someone follow just to chat and complain about the followers only chat tell me to f myself then unfollowed and left.

I'm not too worried about an unfollow because if you're gonna act like that I don't want you in my chat anyway but now I'm just curious if it's bad to have it on.