What does a good anti-spam solution look like?

Spam has become the bane of the average mail administrator’s existence. Business has become reliant on email as a communication tool and spammers are all too happy to take advantage of that fact. This unwanted deluge hawking everything from cheap drugs to hot stock tips has clogged mailboxes around the world. All this unwanted junk mail is a major annoyance, but the real business impact of spam is lost productivity.

Thinking about the real cost of spam, all one has to do is look at how an average employee is affected. Assume our hypothetical employee gets 100 messages a day, and that ~ 70% of that is spam. That is 70 messages a day.  The average user can sort out the junk in 2 minutes or so. That is 10 minutes a week and between 8 and 9 hours a year. In this scenario, spam costs our employee around one day a year worth of productivity. In large organizations, those costs can quickly add-up to tens or hundreds of thousands of dollars annually. And this is assuming that our employee does nothing but scans the subject and deletes the spam message. Spam often contains malware, which can make the productivity loss much greater. It only takes one person opening an infected message to completely disrupt a network.  Of course the flip-side of this is the false positive. Losing a legitimate email can be anywhere from a minor annoyance to a major disaster. 

So what can be done to deal with the flood of garbage, without losing vital information? There have been many solutions proposed and implemented to stop spam. Most of the early efforts focused on the content of the message. Many of these filters are still used, but they suffer from the fact that spammers have become very sophisticated in their efforts to avoid detection. These methods also tend to either have low catch rates or high false positive rates.  Another way to handle spam is to setup black-lists which catalog the IP addresses of known spammers. Unfortunately, spammers are usually moving targets. As a result, this technique is often of limited use, and worse, overzealous list admins and infrequent updates can lead to very high false positive rates. Having their mail server get black-listed due to differing interpretations of RFC or some other similar issue has soured more than one mail admin on the black-list idea. Another idea is to create signatures to identify spam. This can work, but getting these signatures to the front line mail servers can be a daunting challenge.  Another complication is the fact that often spam outbreaks are short-lived.  By the time you develop the signature and get to the scanning software, the outbreak is mostly over. You may catch the last few sent, but not the bulk of them.

An effective anti-spam solution has to be able to correctly identify spam without catching legitimate mail. This solution has to be able to handle the scanning of the email volume your organization receives. There are solutions out there which do a good job of filtering mail, but are either too slow or too resource intensive to be useful in all cases. Using multiple filter types is also a common technique, but again the resource cost has to be considered. Another factor is the equation is the amount of effort required to keep your solution current. Any solution which requires a significant amount of time to maintain is often costing more than the solution is worth. One more often over-looked factor is the impact of the end-user. If the end user does not know the end result of your solution, even the best solution will result is confusion and frustration for the average user. To illustrate this, consider what happens once a message it caught as spam. Is it deleted? Is the subject modified? Do you insert an X-header? Are there message rule in place? The end user has to understand how to identify spam caught by your solution. If they are expected to use rules or some other method to sort things out on their end, they need to be education how to do this. They also need to know how to handle false positives.

So what does a good anti-spam solution look like? First and foremost, it has to be cost effective. Second, it has to deal with the issue without interfering with vital business communication. To do this you need a high catch rate with low false positive. Resource cost in terms of personnel and equipment also have to be factored in. And lastly end-user impact and education have considered. A good solution has to adequately address each of these factors.