Greylisting (email)

Greylisting is a method of defending e-mail users against spam. A mail transfer agent (MTA) using greylisting will "temporarily reject" any email from a sender it does not recognize. If the mail is legitimate, the originating server will try again after a delay, and if sufficient time has elapsed, the email will be accepted.

Mechanism

A server employing greylisting temporarily rejects email from unknown or suspicious sources by sending 4xx reply codes ("please call back later"), as defined in the Simple Mail Transfer Protocol (SMTP). Fully capable SMTP implementations are expected to maintain queues for retrying message transmissions in such cases, and so while legitimate mail may be delayed, it should still get through.

Temporary rejection can be issued at different stages of the SMTP dialogue, allowing for an implementation to store more or less data about the incoming message. The trade-off is more work and bandwidth for more exact matching of retries with original messages. Rejecting a message after its content has been received allows the server to store a choice of headers and/or a hash of the message body.

In addition to whitelisting good senders, a greylister can provide for exceptions. Greylisting can generally be overridden by a fully validated TLS connection with a matching certificate. Because large senders often have a pool of machines that can send (and resend) email, IP addresses that have the most-significant 24 bits (/24) the same are treated as equivalent, or in some cases SPF records are used to determine the sending pool. Similarly, some e-mail systems use unique per-message return-paths, for example variable envelope return path (VERP) for mailing lists, Sender Rewriting Scheme for forwarded e-mail, Bounce Address Tag Validation for backscatter protection, etc. If an exact match on the sender address is required, every e-mail from such systems will be delayed. Some greylisting systems try to avoid this delay by eliminating the variable parts of the VERP by using only the sender domain and the beginning of the local-part of the sender address.

Greylisting is effective against mass email tools used by spammers that do not queue and reattempt mail delivery as a regular mail transport agent normally does. Delaying delivery also gives real-time blackhole lists and similar lists the time to identify and flag the spam source. Thus, these subsequent attempts are more likely to be detected as spam by other mechanisms than they were before the greylisting delay.

Advantages

The main advantage from the user's point of view is that greylisting requires no additional user configuration. If the server utilizing greylisting is configured appropriately, the end user will only notice a delay on the first message from a given sender, so long as the sending email server is identified as belonging to the same whitelisted group as earlier messages. If mail from the same sender is repeatedly greylisted it may be worth contacting the mail system administrator with detailed headers of delayed mail.

From a mail administrator's point of view the benefit is twofold. Greylisting takes minimal configuration to get up and running with occasional modifications of any local whitelists. The second benefit is that rejecting email with a temporary 451 error (actual error code is implementation dependent) is very cheap in system resources. Most spam filtering tools are very intensive users of CPU and memory. By stopping spam before it hits filtering processes, far fewer system resources are used.

Disadvantages

Delayed delivery issues

The biggest disadvantage of greylisting is that for unrecognized servers, it destroys the near-instantaneous nature of email that users expect. Mail from unrecognized servers is typically delayed by about 15 minutes, and could be delayed up to a few days for poorly configured sending systems. Explaining this to users who have become accustomed to immediate email delivery will probably not convince them that a mail server that uses greylisting is behaving correctly.

This can be a particular problem with websites that require an account to be created and the email address confirmed before they can be used – or when a user of a greylisting mailserver attempts to reset their credentials on a website that uses email confirmation of password resets. If the sending MTA of the site is poorly configured, greylisting may delay the initial email link. In extreme cases, the delivery delay imposed by the greylister can exceed the expiry time of the password reset token delivered in email. In these cases, manual intervention may be required to whitelist the website's mailserver such that the email containing the reset token can be used before it expires.

When a mail server is greylisted, the duration of time between the initial delay and the retransmission is variable; the greylisting server has no control or visibility of the delay. SMTP says the retry interval should be at least 30 minutes, while the give-up time needs to be at least 4–5 days; but actual values vary widely between different mail server software.

Modern greylisting applications (such as Postgrey for Unix-like operating systems) automatically whitelist senders that prove themselves capable of recovering from temporary errors, regardless of the reputed spamminess of the sender.

Implementation also generally include the ability to manually whitelist some mailservers.

One 2007 analysis of greylisting considers it totally undesirable due to the delay to mail, and unreliable as, if greylisting becomes widespread, junkmailers can adapt their systems to get around it. The conclusion is that the purpose of greylisting is to reduce the amount of spam that the server's spam-filtering software needs to analyze, resource-intensively, and save money on servers, not to reduce the spam reaching users. The conclusion: "[Greylisting] is very, very annoying. Much more annoying than spam."

Other problems

The current SMTP specification (RFC 5321) clearly states that "the SMTP client retains responsibility for delivery of that message" (section 4.2.5) and "mail that cannot be transmitted immediately MUST be queued and periodically retried by the sender." (section 4.5.4.1). Most MTAs will therefore queue and retry messages, but a small number do not. These are typically handled by whitelisting or exception lists.

Also, legitimate mail might not get delivered if the retry comes from a different IP address than the original attempt. When the source of an email is a server farm or goes out through some other kind of relay service, it is likely that a server other than the original one will make the next attempt. For network fault tolerance, their IPs can belong to completely unrelated address blocks, thereby defying the simple technique of identifying the most significant part of the address. Since the IP addresses will be different, the recipient's server will fail to recognize that a series of attempts are related, and refuse each of them in turn. This can continue until the message ages out of the queue if the number of servers is large enough. This problem can partially be bypassed by proactively identifying as exceptions such server farms. Likewise, exception have to be configured for multihomed hosts and hosts using DHCP. In the extreme case, a sender could (legitimately) use a different IPv6 address for each outbound SMTP connection.

A sender server subjected to greylisting might also reattempt delivery to another receiving mailserver if the receiving domain has more than one MX record. This may cause problems if all such hosts do not implement the same greylisting policy and share the same database.

See also

• Nolisting
• Bandwidth throttling
• Tarpit (networking)

External links

• A greylisting whitepaper by Evan Harris
• A greylisting implementation for netqmail
• Microsoft Exchange Greylisting Problems - Newsgroup Article
• RFC 6647 of the Internet Engineering Task Force, June 2012: Standardizes the current state of the art