Spam challenge: the winners!

Spam challenge: the winners!

The votes are in, the polls are closed and the results are ready…

Regular readers will remember that about a month ago, we decided to conduct a spam experiment to see what email providers perform best when dealing with junk mail. Bets were made, stances taken and friendships placed on the line, but NOBODY could have predicted the results. Read on, and prepare to be surprised!

Just to remind people, we took some deliberate risks to make sure the email accounts were given a run for their money. I entered the email addresses into a dodgy-looking Internet competition AND left the poor defenseless little fellas on the blog to be stolen by any passing spambots. We were looking for a challenge, and a challenge we got.


Many people, including myself, had high hopes for Gmail. When I took a peek, all seemed quiet – 3 emails in the inbox and a whopping 49 in spam. Unfortunately, and unexpectedly, the 3 in the inbox were out-and-out spam, while the vast majority of the ones in spam were actually genuinely from my competition!

hotmail01.pngHotmail received the lowest number of emails overall. We found 11 in the inbox, and 7 in the junk folder. Almost all of the emails in the inbox were correctly classified, but most the ones in spam were actually genuine.


AOL took an interesting approach to the situation. There were only 2 emails in the spam folder, and both were correctly identified as junk. Over in the inbox, all but 4 of the 26 were genuinely related to the competition. Not bad going…


Poor Hushmail was very disappointing: all of the 19 messages in the inbox were spam, while the 12 in the ‘pending’ spam folder were actually a mixture of emails relating to the competition and junk. And this from the service provider that claims to have extra-special spam capabilities!


Lastly we took a look at Yahoo, one of the oldest webmail services on the block. Although it had the highest number of emails of all the accounts – 78 in total – it did well. All of the 37 emails in the inbox were related to the competition, and all of the 48 in the spam folder were total garbage.

The results were not exactly what we were expecting, but very interesting all the same. To be fair, the Internet competition that we entered generated a huge amount of correspondence and I have to admit that the emails definitely looked like spam. The key point to remember, however, is that although they look dodgy, I did solicit the communication. One of the biggest surprises was the variation in the total number of emails received by each account. Hotmail was best at keeping the numbers down, with a paltry 18, although as I counted a total of over 20 genuine emails in other accounts, it does make me wonder where the others disappeared to.

The main point of the test, however, was to see which provider did best in picking out spam and leaving the genuine – if spammy-looking – alone. To judge this, we looked at both the number of spam mails received AND the number of false positives, and when we did, the winner was clear. Yahoo, would you please step forward!

What can you learn from our little test? Well, bear in mind that the situation we created is somewhat unnatural, and that when picking an email account, there is more to consider than spam capabilities; security, usability and features are all hugely important. It is also worth asking around for the opinions of actual real-life users – even though Gmail didn’t perform great in the test, it always performs perfectly in my real accounts. That said, the test highlighted some definite winners and losers, and even though I have no intention of changing from Gmail, I certainly know which email providers I won’t be recommending to my friends!

Loading comments