Twitch is about to roll out a new tool called Suspicious User Detection. This aims to do exactly what it says on the box: try to detect users who are being suspicious. More specifically, those trying to evade existing bans.
In an announcement blog, Twitch says:
When you ban someone from your channel, they should be banned from your community for good. Unfortunately, bad actors often choose to create new accounts, jump back into Chat, and continue their abusive behavior. Suspicious User Detection, powered by machine learning, is here to help you identify those users based on a number of account signals. By detecting and analyzing these signals, this tool will flag suspicious accounts as either “likely” or “possible” channel-ban evaders, so you can take action as needed.
What happens after a user has been flagged depends on the flag. Those suspected of being “Likely” won’t have any of their messages displayed in general chat, but they will still be visible to mods and creators, who can then take further action against the user.
“Possible” cases, meanwhile, will be able to post messages in chat and see them appear, but on the creator/mod end those messages will be flagged “so they can monitor the user and restrict them from chatting if needed.”
Suspicious User Detection is going to be turned on by default when it goes live, but streamers will be able to then go into settings and tinker with it. It will let them do stuff like increase the posting restrictions on “Possible” cases, or manually add users that the machine learning hasn’t detected but they’re suspicious about anyway.
Which all sounds wonderful, but then it’s also putting some pretty powerful tools in the hand of a machine, something that Twitch have to caution for, adding:
One thing to prepare for, particularly around launch, is that no machine learning will ever be 100% accurate, which means there is a possibility of false positives and false negatives. That’s why Suspicious User Detection doesn’t automatically ban all possible or likely evaders. You’re the expert when it comes to your community, and you should make the final call on who can participate. The tool will learn from the actions you take and the accuracy of its predictions should improve over time as a result.
You can see the system in action in this video: