Configuring SPF to make Google happy

Lately, Google mail servers are getting more extravagant in what mail they accept. Setting up a Sender Policy Framework (SPF) DNS record is a relatively low effort way to get mail though users of the Google mail service. (Those users aren't necessarily identifiable by the Gmail domain as it's also possible to point the MX of another domain to Google mail servers and use the Google mail services in that way.)

Background

For years, there are some best practices for mail servers to follow in order to be accepted by most other mail servers as legitimate mail server. For example, the identification uses in a HELO/EHELO command should be a valid domain name that has an address record which matches the actual IP address used in the connection. Also, this IP address should have a reverse DNS record that point back to that hostname.

Checking just this is a quite simple and effective measure against a lot of spam, because most spammers don't get this right.

Another way to proof that your mail server is a good citizen is to not get blacklisted. Even if you don't use blacklists, many other mail servers do. The criteria used by the different blacklists vary much, but usually it captures some kind of bad behaviour or just behaviour that isn't expected of a real mail server. Getting caught by a honey trap sending spam, violating some mail related RFCs (e.g. not having an abuse address) or being in a dial-up IP address range are examples for this.

Google

After occasionally sending mails to Gmail destinations without any issue, I recently received the following bounce message from a Google mail server:

<support@example.com>: host aspmx.l.google.com[2a00:1450:400c:c09::1b] said:
    550-5.7.1 This message does not have authentication information or fails to pass
    550-5.7.1 authentication checks. To best protect our users from spam, the
    550-5.7.1 message has been blocked. Please visit
    550-5.7.1 https://support.google.com/mail/answer/81126#authentication for more
    550 5.7.1 information. a123456789b123xy.11 - gsmtp (in reply to
              end of DATA command)

The original recipient (a small enterprise) doesn't even have a Gmail address, but it uses the Google mail services with their own domain.

Sure enough the referenced Google page has some additional information:

To ensure that Gmail can identify you:

  • Use a consistent IP address to send bulk mail.
  • Keep valid reverse DNS records for the IP address(es) from which you send mail, pointing to your domain.
  • Use the same address in the 'From:' header on every bulk mail you send.

We also recommend the following:

  • Sign messages with DKIM. We do not authenticate messages signed with keys using fewer than 1024 bits.
  • Publish an SPF record.
  • Publish a DMARC policy.

Strange, first of all, the outgoing mail server doesn't send bulk mail and its reverse DNS record is fine. But there's more:

Additional guidelines for IPv6

The sending IP must have a PTR record (i.e., a reverse DNS of the sending IP) and it should match the IP obtained via the forward DNS resolution of the hostname specified in the PTR record. Otherwise, mail will be marked as spam or possibly rejected. The sending domain should pass either SPF check or DKIM check. Otherwise, mail might be marked as spam.

Indeed, the outgoing mail server is a proper IPv4/IPv6 dual stack host and the connection to the Google mail server was established over IPv6 (cf. the bounce message).

Since the reverse DNS record for the used IPv6 address is fine, as well, the Google mail server really insists on a SPF or DKIM configuration.

SPF

The Sender Policy Framework (SPF) standard specifies a payload for the TXT DNS record that a receiving mail server can consult to decide whether the sender is the one designated by the DNS hostmaster for sending mails for that domain or not. This protects against spammers forging From addresses.

In other words, it's analogous to the role the MX record has for a sending mail server.

SPF is definitely easier to set up than Domain Keys Identified Mail (DKIM) and it doesn't have any plausible deniability implications for the message content of all sent messages.

For small email servers, the simplest and probably most common setup is to use the same IP addresses and hostnames for incoming and outgoing SMTP communication. This can be specified with a simple SPF expression:

v=spf1 a mx -all

Meaning: it's a SPF version 1 record, valid hosts for sending mail for this domain are either ones that use an address listed in an address record or ones that use a hostname listed in a MX record, and every other host isn't expected to send any mail for this domain.

Thus, such an expression is specific enough to identify the legitimate mail servers but also generic enough such that that it's sufficient for small mail servers without any MX records, small mail server ensembles with multiple address records and/or multiple MX records.

SPF doesn't differentiate between IPv4 and IPv6, thus, the a should match both address types.

Setting up such a TXT record should be simple enough with any nameserver or nameserver hoster. The entry can be verified like this:

$ dig +noall +answer georg.so txt
georg.so.       3600    IN  TXT "v=spf1 a mx -all"

And indeed, this makes the Google mail servers happy such that mails delivered via IPv6 addresses aren't bounced anymore.

Closing

Since it's so common to have outgoing SMTP connections from the same address as incoming ones, it would be a quite sensible default for the Google mail servers to assume the SPF expression v=spf1 a mx -all as default in case neither SPF nor DKIM is configured.