[RndTbl] SpamAssassin false positives on DATE_IN_FUTURE_96_Q?

Tue Feb 23 13:15:26 CST 2016

I know this is a really old thread, but it popped up in a search as I've 
been trying to solve the same problem you ran into over 4 years ago. 
Like you, I used -I in the EXTRA_FLAGS in /etc/sysconfig/spamass-milter, 
and also like you I've still been getting a lot of false positives on 
other mail. Some of these DATE_IN_FUTURE tests have really large scores, 
so getting them wrong really messes things up.

I think I found the answer in this bug report, though: 
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=775183

It seems the problem is adding the b macro to confMILTER_MACROS_ENVRCPT, 
or even to confMILTER_MACROS_ENVFROM, doesn't work reliably. Sometimes 
sendmail gives the milter the wrong date - the date that sendmail 
started, rather than the date of the e-mail. That's why the synthesized 
Received header is in the past, making it appear that the Date header is 
in the future. The fix seems to be to have the b macro in 
confMILTER_MACROS_CONNECT alone, and neither of the others. I've done 
that in my configuration, and so far so good, but it might be too early 
to tell for sure. It doesn't look like this is only an issue with RHEL5 
& clones, as it seems to affect Debian users using a couple different 
versions of sendmail as well. I don't know if the latest version of 
sendmail fixes this or not, but I don't see this config tweak as a 
problem as long as the CONNECT macro handles b consistently. If this 
does indeed solve the problem, it beats setting all the DATE_IN_FUTURE 
scores to 0 in /var/lib/spamass-milter/.spamassassin/user_prefs as I had 
done earlier today in desperation (and since backed out).

On 11/18/2011 03:41 PM, Gilbert E. Detillieux wrote:
> On 2011-11-18 10:29, Peter O'Gorman wrote:
>> Google turned up this:
>> http://lists.nongnu.org/archive/html/spamass-milt-list/2010-05/msg00001.html 
>>
>>
>> Looks like the problem is spamass-milter's synthesized Received header,
>> rather than the spamassassin rule.
>
> Well spotted, Peter.  I guess I hadn't come up on the right sequence 
> of keywords to bring that Google search result into the first page.
>
> Yes, that does seem like the bug I've encountered.  I noticed that a 
> lot of my false-positives (though not all) were on messages directly 
> sent by authenticated SMTP clients, i.e. where there was no 
> pre-existing Received header.  (I didn't realize that spamass-milter 
> was faking one.)
>
> My fix for authenticated clients was much simpler and cleaner than 
> what was proposed on that page, however:  spamass-milter has a "-I" 
> option that will skip calling spamc/spamd altogether for authenticated 
> SMTP connections!
>
> However, the false-positives still remain for non-authenticated 
> clients.  At least I have a better idea of where the problem lies, and 
> what to look for in terms of a solution.
>
> Thanks,
> Gilbert
>
>> On 11/15/2011 10:45 AM, Gilbert E. Detillieux wrote:
>>> On 2011-11-14 17:46, Kevin McGregor wrote:
>>>> So you've changed the date manually to be exactly the same, and the 
>>>> rule
>>>> doesn't trigger?
>>>
>>> Well... Here's the weird thing: if I pass the exact same message 
>>> through
>>> spamc manually, I don't get the false positive on that rule. So, I 
>>> tried
>>> mailing that message back to myself from a non-local mailer (so that it
>>> goes through spamass-milter again), but this generates extra "Received"
>>> headers that change the behaviour. (I now get a trigger on the
>>> DATE_IN_PAST_24_48 rule, since the message is now that old.)
>>>
>>> So, I can't test under exactly the same conditions. Given that running
>>> the message through spamc manually didn't trigger the rule, I'm tempted
>>> to think it might be something in the spamass-milter configuration,
>>> which is causing some information to not be transferred to spamc, or to
>>> be transferred incorrectly. Not sure at this point.
>>>
>>> Gilbert
>>>
>>>> On Mon, Nov 14, 2011 at 4:56 PM, Gilbert E. Detillieux
>>>> <gedetil at cs.umanitoba.ca <mailto:gedetil at cs.umanitoba.ca>> wrote:
>>>>
>>>> I mentioned this problem at the last round-table session, but didn't
>>>> get a solution, so I thought I'd post it here, just in case anyone
>>>> has any suggestions to offer.
>>>>
>>>> I'm still seeing a whole bunch of false positives in SpamAssassin,
>>>> since an update was installed in mid-September on a CentOS 5.7
>>>> system, for a rule called DATE_IN_FUTURE_96_Q, which is only
>>>> supposed to be triggered when the "Date:" header has a date that is
>>>> 4 days to 4 month ahead of the date in the "Received" header that
>>>> has the _smallest_ difference in date.
>>>>
>>>> Here are the headers from the latest e-mail I've received with this
>>>> false-positive. (I've stripped out irrelevant headers, for the sake
>>>> of clarity and simplicity.)
>>>>
>>>> >From topfivestories at messagent.__itworldcanada.com
>>>> <mailto:topfivestories at messagent.itworldcanada.com> Mon Nov 14
>>>> 07:50:13 2011
>>>> Received: from mail.messagent.itworldcanada.__com
>>>> <http://mail.messagent.itworldcanada.com>
>>>> (mail.messagent.itworldcanada.__com
>>>> <http://mail.messagent.itworldcanada.com> [207.112.10.80])
>>>> by palladium.cs.umanitoba.ca
>>>> <http://palladium.cs.umanitoba.ca> (8.13.8/8.13.8) with SMTP id
>>>> pAEDoAxV028594
>>>> for <gedetil at cs.umanitoba.ca
>>>> <mailto:gedetil at cs.umanitoba.ca>>; Mon, 14 Nov 2011 07:50:12 -0600
>>>> Date: Mon, 14 Nov 2011 08:50:13 -0500
>>>> X-Spam-Status: No, score=-0.3 required=5.0
>>>> tests=BAYES_00,DATE_IN_FUTURE___96_Q,
>>>> HTML_MESSAGE,RP_MATCHES_RCVD autolearn=no version=3.3.1
>>>> X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
>>>> palladium.cs.umanitoba.ca <http://palladium.cs.umanitoba.ca>
>>>>
>>>> Note that I'm calling spamd via the spamass-milter on a system
>>>> running sendmail. Note also, that in the above example, the only
>>>> "Received" header was the one generated by my own server. (I've had
>>>> other false positives, however, with multiple "Received" headers,
>>>> all of which were within seconds of the time in the "Date" header.)
>>>>
>>>> Any ideas?
>

-- 
Gilles R. Detillieux              E-mail: <grdetil at scrc.umanitoba.ca>
Spinal Cord Research Centre       WWW:    http://www.scrc.umanitoba.ca/
Dept. of Physiology and Pathophysiology, Faculty of Health Sciences,
Univ. of Manitoba  Winnipeg, MB  R3E 0J9  (Canada)