SPF record is used to accurately calculate the reputations of envelope-sender domains and mailservers by mailbox-providers. It works as follows: If many users flag emails as spam from example.com, then SPF allows the mailbox-provider to more accurately say that, “example.com is a spammer”, because the SPF record of example.com is used to identify it as a spam. Conversely, if any user flag spoofed email as spam from example.com, then SPF allows the mailbox-providers to maintain the good reputation for example.com and makes the email flow normal. So, we should always use SPF to make spam filtering more accurate.
As a PHP developer, everyone familiars with the PHP mail (http://php.net/manual/en/function.mail.php) function, which is used for sending emails. But, one can’t send email using SMTP authentication using this mail function. This makes most outgoing emails caught by spam-filters because of the SPF (Sender Policy Framework) check.
An important thing about SPF is that, it only performs checks on the envelope sender (Return-Path header), not on the user-visible from header. You can read about this at http://www.openspf.org/FAQ/Envelope_from_scope.
In most of the application, we use PHP’s mail function. If we check the outgoing mail header sent using the mail function, we can see that the envelope sender (Return-Path header) is a common address of the server or localhost even if you provide the From header of your domain to the mail.
So, when SPF checks performed by mailserver, it checks only the envelope sender which is a common address of the server or localhost and ignoring the from address, which is a genuine email address of the domain. So, our SPF record will be ignored. And, the server may not have an SPF record. In such case mail server detect a softfail. If you notice in Gmail, it identifies such softfail with phrase “best guess record”. Gmail uses a heuristic system to create best-guess SPF records for domains that don’t use SPF. For more details, go to http://www.ceas.cc/2006/19.pdf
As Gmail is using a best-guess records for server/domains that don’t have an SPF record, this guess is sometimes wrong, which cause a softfail. If your mail is being routed through a new mailserver, which may not add to the best-guess SPF records by Gmail, then we can see the softfail disappear after a while.
There is some basic solution to avoid softfails due to SPF record.
One solution would be to use SMTP mailer instead of PHP’s simple mail function. There are many SMTP mailer classes available free, like PHPMailer. SMTP mailer is directly talking to the mail server, so it can set the envelope-sender to be same as the From address. You can also specify the envelope-sender while using SMTP mailer.
Alternatively, we can also use Google’s SMTP (smtp.google.com) to send outgoing mails using Google’s username and password. In this case, we have to simplify our SPF record to identify Google by adding “include:_spf.google.com ~all”.
But, if we have already used a PHP’s mail function throughout the application, and switching to SMTP mailer requires so much time and effort, we can still set an envelope-sender using PHP’s mail function. To set the envelope-sender, use sendmail’s -f/-F command as a fifth additional-options parameter of the PHP’s mail function.
PHP Code:
mail("user@example.com", "test subject", "test message", $headers, "-F 'Example Envelope-sender' -f returnpath@example.com");
We can use any of domain email addresses or from address instead of returnpath@example.com in the above code snippet.
There may be other solutions to achieve this. The key is to set the envelope-sender (Return-Path) to the email address of the domain.