A Good Antispam and Antivirus Setup

From Niversoft
Jump to: navigation, search

In this text, to simplify, the spam word means UCE as well as phishing, rogue, malware or junk links, scams messages.

What to do with spam and viruses? In previous versions of Communigate Pro (up to 5.0), there wasn't a lot of choices, you had to accept them unless they came from a RBL-listed source. Once accepted, you could either quarantine, discard or bounce them.

Synchronous refusal

Since CGP 5.1, the synchronous ENQUEUER module allows to reject the messages before accepting them. Reject them during the SMTP connection. Instead of bouncing, where you weren't sure the Return-Path wasn't forged, rejecting just gives a refusal message to the sending party. You are 100% sure the refusal goes to the correct party. Even if you don't control what that party does with the refusal, you didn't involve an unknowing third party in your war against this particular unwanted message.

How to enable this synchronous mode? Easy: In CGP Webadmin, Settings/Mail/Queue. Untick the "Enqueue Asynchronously" checkbox. But wait! There is at least some things you should check first.

  • Synchronous Enqueuer means that all server-rules processing will happen before the message is accepted, while the SMTP channel is still open. You should verify the number of SMTP channels is high enough to accommodate. If you have a lot of traffic and your server is not able to handle the rules processing quickly enough in peak periods, the SMTP connection may timeout or the sending party may become impatient. The SMTP RFC specifies a reasonable Data Termination timeout of 10 minutes, but 5 minutes is a good target.
  • Your actual rules may be crafted in terms of async processing. Changing to synchronous mode is usually not a problem, but you may want to read more about the implication to optimized your rules.

False positives

Ok, if you reached this paragraph, we assume your server is now in synchronous processing mode.

All spam filters, including PolluStop includes a small percentage of false positives, and in our opinion no spam filter should be trusted enough to allow them to blindly discard spam. Rejection, as described above, is a good solution, as legitimate senders will know they didn't reach you and will be able to try again or by another method. However, for various reasons, we chose to use the quarantine method in our setup - we store messages marked as spam by PolluStop in a per-user Junk mail mailbox, and leave to the user the responsibility of clearing them.

Viruses and very obvious spam

But there's very obvious messages that shouldn't ever have to reach that state, these are messages that can easily be caught by signature-based Spam filters. A company named SaneSecurity releases spam/malware/etc signature files for the popular open-source antivirus engine ClamAV. As ClamAV is a signature-based AV usually employed to detect viruses, it was quite easy to create signatures that would also detect static spam. Interestingly, ClamAV is very good at filtering these exact forms of spam PolluStop usually has problems with.

ClamAV engine and CGP

CGP-ClamAV is an antivirus filter using the ClamAV engine and optimized for CGP. Previous solutions linking CGP to the ClamAV engine usually did it through clamd. This posed problems, as the clamd engine analyzes the content of files and tries to determine its type to extract parts of compressed files, scan relevant parts of binaries, etc. The CGP envelope information at the beginning of the mail file prevents clamd from recognizing it as a MIME file, so clamd analyzes it in shallow mode only. If a virus (such as the EICAR signature) is present at top-level, clamd will detect it. But if the virus is contained in, for instance, a zip attachment, and the zip itself doesn't have a virus signature, clamd will fail to detect it.

CGP-ClamAV bypasses the detection mechanism. By integrating the ClamAV engine statically, with a very simple modification, the message is sent directly to the MIME structure parser. This is more efficient - no need to do IPC with clamd, no need to analyze the file to decide how to scan it, as we already know what it is.

Third-party databases

Recently, we provided with CGP-ClamAV a script to download the SaneSecurity database. As already said, this has proven very useful. The false positives rate of most of the SaneSecurity databases is low enough to be ignored. So we configured CGP-ClamAV in sync mode to reject viruses and obvious spam with a clear error message during the SMTP connection. In the event of a false positive, the legitimate sender would receive a message such as the message you sent us has unfortunately been marked as spam by our filters. Please try again with a different or shorter message.

Summary

In short, our anti-spam / anti-virus setup is as follow

  • Message is scanned by CGP-ClamAV with the SaneSecurity db
  • Messages marked as virus/spam are rejected during the SMTP transaction by a rule - we also store these messages in a postmaster quarantine mailbox for further training purposes
  • Message passes through a series of simple rules to reject non-spam, non-virus but unacceptable mail for us, such as messages without a From: header, or with very specific headers. Such messages are rejected with the same error as above
  • Messages then passes through PolluStop, configured with a few regular expressions to complement its filtering
  • Messages marked as spam by PolluStop are sorted in per-user Junk mailboxes for manual processing

With that setup, high-activity users receives between 20 and 50 junk messages per day in their junk mailbox. Before that, it was more like 150 messages per day.

Of course we do not claim this is the best setup. But it is still a very good one, better than most of those we've seen in many server, including ours a few months before writing these lines.