Spam – the big issue. Part 2 – How to control and stop it.
The site anitspam system I designed is crude, harsh and rough, but very effective. It was mostly “manual” at first progressing to automatic with the site’s grow. It does affect “innocent bystanders” (not too much) but as a result we are almost spam free.
The first thing I did was writing a comment guideline – what is exactly considered spam and abusive. This is important. If users don’t know what you think is wrong in their behavior how are they supposed to correct it?
Antispam system
We made some changes who and when can comment – if you want to comment you must have an account, but creating an account requires captcha. Once you do have an account you encounter:
1 level Manual check-ups
A spammer – user or bot – will spam your most “upper” content – this is where the spam will get the most attention. There is no point in spamming videos no one is watching. That’s why the first line of defense, the first thing I started with, was checking the comments on the top 1000 videos for the last 24 hours.
This might sound unwise – “why do it manually, it probably wil take a lot less time if you create a very smart algorithm that checks it, or you might watch over user blocks, or you might bla bla bla” Bullshit.
You can do all this when your startup really grows big and you know your users, you know what they spam with, what they aim for and how they do it. Delving in creating state of the art algorithm, wasting server power, just to prevent those “do this, or your mother will die” popping here and there – that would’ve been really stupid.
Just spin the scroll button untill you hit at least 100K unique users daily.
2 level Analyze your Manual check-ups
Doing this manual checks we saw that what users spam the most with were links – pointing to other clips, or pointing to sites. Normally, almost noone would put a link in a comment. So:
- No links in comments – active or not – It is not possible to write any link properly, unless internal for the site. We left this option open. The comment system won’t accept anything starting with http:// or containing the most common domains. Important note – this is not even mildly as terrifying as it sounds. Because in our case, our normal comments had no links anyway – we made our homework over this. They just didn’t – it was a fact.
- Banned keywords and phrases – Everything specific you try to spam with like “lockers” is banned. Since the word is so specific usually it does not affect normal comments.
3 level Less then X comments per hour
If you try to post more then X comments per hour it won’t allow you. You’ll have to wait. A video is typically 3-4 minutes long. You won’t, typically, comment each video you watch, and you most certainly won’t do it every 3 seconds. That prevents spambots which managed to pass the captcha.
4 level Flagging by the users
I worked extensively with users for about 2 years, promoting the “flag” links, giving awards to the most active users, discussing in groups the issue and strongly emphasizing on “spamming is a bad thing to do” every single time.
Restrictions
Flaged spammers are checked and
- If the user is bot is deleted – The bastards are getting more and more creative these days. They post vague comments simulating real humans. Strong captcha usually solves it, also the “less then X comments per hour” restriction. We now have internal rating for each profile – we bugs you with the captcha thing as little as possible – once you achieve a required minimum it’s off.
- If the account is created just for spam is deleted (or banned if we want to keep track will he/she try again)
- If it’s a normal user, spam is deleted and the user receives notification. (or all comments are deleted – depends on the percentage of the spam) Every user has the right to 2 notifications. It’s a simple pm – “Please, don’t spam :) You can read what is considered as spam here *link* Regards, … team”
- The next step is usually ban. If the user is persistent it might cause IP ban, but that’s rare. If the user is really persistent I “corner” him/her, I send personal messages, I try to talk him/her out of doing it. It’s amazing but ex-spammers can be converted into dedicated antispam users if you manage to hit the right buttons.
So 4 years later …
I am proud to say that starting from mass spaming, we are nearly to zero. We have a little bit every day but it has a short life span. We have around 200 “comment readers” type of users that are devoted on hitting that “flag” link (I love ‘em with all my heart)
The above measures are applied over a site which is big, but not enormous. I suppose YouTube probably use a lot of algorithms and analysis to mark a comment as potential spam (hence they are not deleting any comments, just hiding them) I will take a wild gues that
- X persons must flag a comment
- Each time you flag appropriately your rating is going up giving your next vote more “weight”
- Each time you flag wrong your rating goes down, until eventually you lose the right to flag or you are banned
- They don’t ban keywords but key symbol sequences
- Max X comments per time unit
As a community manager what I try to do is to express my (and the team’s) gratitude when people are taking their precious time to flag
- I visit their groups and I give them props in front of their friends – which is highly flattering
- We give profile badges to really active users
- We send “thank you” messages to active users
- I stress how important is flaging and how bad is spamming constantly
