SPF says rollernet doesn't send mail out through mail, mail2

tgarcia3 · Post by **tgarcia3** » Thu Jun 30, 2005 11:00 am

When trying to send mail from a yahoo.com to a hivemind.org address, and the mail has been held on mail.rollernet.us before being forwarded to primary MX (mxin.mxes.net - tuffmail imap service), I get a warning in the header linking to:

http://spf.pobox.com/why.html?sender=ro ... 1.mxes.net

"mxout-01.mxes.net saw a message coming from the IP address 208.11.75.2 which is mail.rollernet.us; the sender claimed to be rollernet.us. However, rollernet.us has announced using SPF that it does not send mail out through 208.11.75.2."

I thought that SPF was only about the main sender address and not about checking intermediate mail servers, in which case this warning would be incorrect, but maybe I am wrong and it is right...? indeed the rollernet.us spf record is "v=spf1 ip4:67.118.43.88/29 ip4:66.224.163.66/32 -all" which makes no mention of these mail servers.

Thanks for any advice/thoughts.

Post by **RollerNetSupport** » Thu Jun 30, 2005 12:08 pm

SPF is correct; we don't send out any mail with an @rollernet.us address from either of those servers. All of the administrative mail comes from mail.mattinen.org (on 67.118.43.88/29), and 66.224.163.66/32 sends messages to our pagers and other reports through email.

If something is appending or changing the MAIL FROM address to something@rollernet.us, this is an incorrect behavior. Or the SPF filter doing the checking is doing something abnormal (like stripping off the hostname of the domain and comparing the SPF records that way rather than using the MAIL FROM address as is). Otherwise I'd have to see the full headers.

tgarcia3 · Post by **tgarcia3** » Thu Jun 30, 2005 12:27 pm

Full headers (w's, x's, y's substituted for actual usernames; remove zzzzz's entirely) given below. The headers seem to show sender @yahoo.com throughout, so I guess it's really just making the (wrong) check as if from @rollernet.us?

[edit] I have just received a reply from Tuffmail technical support telling me that this is a standard fall-back thing when the envelope sender (yahoo.com) has no SPF record (which it doesn't). This surprises me, as I would have thought rollernet.us SPF records have nothing to do with what mail servers either yahoo.com or hivemind.org use, but maybe I have not understood something yet.... time for further reading.

Code: Select all

X-Sieve: CMU Sieve 2.3
Return-Path: <xxxxx@yahoo.com>
Received: from mxout-01.mxes.net (mxout-01.mxes.net [205.237.194.32])
	by ms1.mxes.net (Postfix) with ESMTP id 8E4C596CDC
	for <hivemindzzzzz_tmbox.com@ms1.mxes.net>; Thu, 30 Jun 2005 13:03:03 -0400 (EDT)
Received: from mxout-01.mxes.net (mxout-01.mxes.net [205.237.194.32])
	by mxout-01.mxes.net (Postfix) with ESMTP id 7018333F873
	for <yyyyy@hivemindzzzzz.org>; Thu, 30 Jun 2005 13:03:03 -0400 (EDT)
Received: from mail.rollernet.us (mail.rollernet.us [208.11.75.2])
	by mxin.mxes.net (Postfix) with ESMTP id 6E19633F787
	for <yyyyy@hivemindzzzzz.org>; Thu, 30 Jun 2005 13:03:02 -0400 (EDT)
X-RollerNet-Abuse: Roller Network SMTP Services. See http://rollernet.us/abuse.php
Received: from web34403.mail.mud.yahoo.com (web34403.mail.mud.yahoo.com [66.163.178.152])
	by mail.rollernet.us (Postfix) with SMTP id 9BDCC529DCD
	for <yyyyy@hivemindzzzzz.org>; Thu, 30 Jun 2005 09:14:50 -0700 (PDT)
Received: (qmail 85523 invoked by uid 60001); 30 Jun 2005 16:14:50 -0000
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=Message-ID:Received:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; [snip]
Message-ID: <20050630161450.85521.qmail@web34403.mail.mud.yahoo.com>
Received: from [81.5.183.197] by web34403.mail.mud.yahoo.com via HTTP; Thu, 30 Jun 2005 09:14:50 PDT
Date: Thu, 30 Jun 2005 09:14:50 -0700 (PDT)
From: wwwww <xxxxx@yahoo.com>
Subject: with grey off
To: yyyyy@hivemindzzzzz.org
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit
X-Virus-Scanned: ClamAV
X-Spam-Score: -2.6
X-Spam-Check: Enabled,6.0,13.0,1,1,42,[SPAM]
X-Spam-Status: No, score=-2.6 threshold=6.0,13.0 
X-Spam-Report:  Content analysis details:
  0.0 SPF_HELO_FAIL          SPF: HELO does not match SPF record (fail)
  [SPF failed: Please see http://spf.pobox.com/why.html?sender=rollernet.us&ip=208.11.75.2&receiver=mxout-01.mxes.net]
  -2.6 BAYES_00               BODY: Bayesian spam probability is 0 to 1%
  [score: 0.0019]
X-Envelope-To: <yyyyy@hivemindzzzzz.org>

Post by **RollerNetSupport** » Thu Jun 30, 2005 4:37 pm

[edit] I have just received a reply from Tuffmail technical support telling me that this is a standard fall-back thing when the envelope sender (yahoo.com) has no SPF record (which it doesn't). This surprises me, as I would have thought rollernet.us SPF records have nothing to do with what mail servers either yahoo.com or hivemind.org use, but maybe I have not understood something yet.... time for further reading.

No, you aren't misunderstanding anything. Our SPF records have nothing to do with any mail except that purported to be addressed from anyone@rollernet.us. The fallback is not something that is normally done; this is the first time I've heard of it. If the domain doesn't have an SPF record, then the specification says it should return a "None" result, not keep checking for other things to reject. As far as I can tell, the SPF spec says it must check "MAIL FROM" and may check "HELO", but that's all. (http://www.ietf.org/internet-drafts/dra ... sic-02.txt) That's how the rollernet SPF filter works; mainly because I'm the one who wrote it. I used the Mail::SPF::Query Perl module as a reference point and for testing.

The old draft (http://spf.pobox.com/spf-draft-200406.txt) doesn't say anything about expanding SPF past HELO and MAIL FROM either, so I'd say that they have a badly written SPF implementation. For me to allow rollernet.us to pass their SPF filter as they have written it, I would have to either allow all IP addresses to claim to be me, or remove my SPF record. Either of those actions would defeat the purpose of using SPF.

tgarcia3 · Post by **tgarcia3** » Thu Jun 30, 2005 11:28 pm

In this case what I think I mean (I am learning) by the standard fallback thing is checking the helo/ehlo, as you mention... in other words, because mxin.mxes.net sees "helo mail.rollernet.us", it checks rollernet.us spf records. Strictly, the draft says:

At least the "MAIL FROM" identity MUST be checked, but it is RECOMMENDED that the "HELO" identity also be checked beforehand.

So in fact the recommendation is to examine helo first, but not a MUST; even in the event yahoo.com had spf records, it might happen. Tuffmail/mxes.net seems to be following spec.

I'm not sure by what mechanism the rollernet.us spf might then say "if the MAIL FROM is one of these addresses, accept it" - maybe a macro definition? Either way, it seems unworkable to have a huge list of allowed domains somehow in the rollernet spf record, so perhaps in the forwarding case one of the following is expected:

1. Forwarder SPF record includes e.g. some sort of "exists" lookup using a macro expansion to determine whether the mail was validly sent through the server; this assuming (maybe?) the "s"(ender) macro expands always to MAIL FROM, and not to HELO when the HELO is being checked;
2. As you say, allow all your IPs to claim to be you, which would be partly defeating spf.

Post by **RollerNetSupport** » Fri Jul 01, 2005 4:44 am

Agh... that went completely over my head even after I said it. You are right, of course, that a HELO fallback would cause this behavior.

For now, I'm going to add "mx:all" to the rollernet.us SPF records. That should, in theory, allow the SPF lookup to pass in this situation.

tgarcia3 · Post by **tgarcia3** » Fri Jul 01, 2005 12:10 pm

Ok, it's making sense now: spf has a dual role and now I know about the (weirder) HELO one. Thanks for your input. BTW I think it's "mx" only rather than "mx:all" if the intention is to accept all mx for rollernet.us, the keyword "all" being solely for the ~all/-all catch-all.

I was also chatting to someone at Tuffmail and it does seem this is just a general problem with forwarding and spf, and that adding a record such as you have is (alas) the solution for the HELO thing.

A further possibility... you could create a TXT record spf.rollernet.us - e.g. as spf.mxes.net - which lists all the servers you forward mail through. Then your users can be encouraged to simply add include:spf.rollernet.us to their own spf record. Mind you, I believe this is just functionally equivalent to adding "a:mail.rollernet.us a:mail2.rollernet.us"; and this isn't a substitute for the previous paragraph's solution, as the HELO response domain might still be checked. Fun!

Post by **RollerNetSupport** » Fri Jul 01, 2005 3:20 pm

Okay, changed it to simply "mx" now. I found this explanation: "All the A records for all the MX records for domain are tested in order of MX priority. If the client IP is found among them, this mechanism matches." (http://spf.pobox.com/mechanisms.html) So if they check the HELO for the forward you originally posted about, they would look up the TXT record for rollernet.us, the MX records for your domain, and see if the connecting IP matched one of the MX records. Assuming all the DNS lookups that go with that worked, it should result in a pass.

I'll probably implement your second suggestion once we look in to providing SMTP smart host service, i.e., we originate the message. Right now it only works as an intermediary or secondary. The advantage to having an end user say "include:spf.rollernet.us" is that I can alter the spf.rollernet.us TXT record at will and the changes are automatically included, rather than everyone else being forced to update their TXT records. I think I can still throw the "mx" keyword in there to pass the HELO check.

Thanks for pointing all this out!