[RndTbl] SpamAssassin false positives on DATE_IN_FUTURE_96_Q?

Tue Feb 23 14:47:34 CST 2016

It occurred to me that even if this change DIDN'T work, I wouldn't start 
seeing false positives again until 3 hours after restarting sendmail, 
because the first DATE_IN_FUTURE rule is 03_06 which kicks in when 
there's a discrepancy of at least 3 hours (i.e. one more hour from now, 
as I restarted sendmail 2 hours ago). The 96_Q won't kick in until 
Saturday. So I'll have to monitor things over the course of the next 
week or so.

Before changing the config & restarting sendmail, I was getting 1-3 of 
these 96_Q false positives a day in my own e-mail, with sendmail having 
started 34 days ago at the last reboot. Those would be the most common 
false-positives because of the large range involved (96-2920 hrs or 4 
days-1 quarter), and they stand out because of the high score (3.199). 
Only 03_06 has a (slightly) higher score (3.399).

On 02/23/2016 02:19 PM, Gilbert Detillieux wrote:
> Thanks for the tip.  I haven't noticed the DATE_IN_FUTURE_96_Q being a 
> problem lately, but I decided to change my milter macros just in case.
>
> Gilbert
>
> On 23/02/2016 2:15 PM, Gilles Detillieux wrote:
>> I know this is a really old thread, but it popped up in a search as I've
>> been trying to solve the same problem you ran into over 4 years ago.
>> Like you, I used -I in the EXTRA_FLAGS in /etc/sysconfig/spamass-milter,
>> and also like you I've still been getting a lot of false positives on
>> other mail. Some of these DATE_IN_FUTURE tests have really large scores,
>> so getting them wrong really messes things up.
>>
>> I think I found the answer in this bug report, though:
>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=775183
>>
>> It seems the problem is adding the b macro to confMILTER_MACROS_ENVRCPT,
>> or even to confMILTER_MACROS_ENVFROM, doesn't work reliably. Sometimes
>> sendmail gives the milter the wrong date - the date that sendmail
>> started, rather than the date of the e-mail. That's why the synthesized
>> Received header is in the past, making it appear that the Date header is
>> in the future. The fix seems to be to have the b macro in
>> confMILTER_MACROS_CONNECT alone, and neither of the others. I've done
>> that in my configuration, and so far so good, but it might be too early
>> to tell for sure. It doesn't look like this is only an issue with RHEL5
>> & clones, as it seems to affect Debian users using a couple different
>> versions of sendmail as well. I don't know if the latest version of
>> sendmail fixes this or not, but I don't see this config tweak as a
>> problem as long as the CONNECT macro handles b consistently. If this
>> does indeed solve the problem, it beats setting all the DATE_IN_FUTURE
>> scores to 0 in /var/lib/spamass-milter/.spamassassin/user_prefs as I had
>> done earlier today in desperation (and since backed out).
>>
>> On 11/18/2011 03:41 PM, Gilbert E. Detillieux wrote:
>>> On 2011-11-18 10:29, Peter O'Gorman wrote:
>>>> Google turned up this:
>>>> http://lists.nongnu.org/archive/html/spamass-milt-list/2010-05/msg00001.html 
>>>>
>>>>
>>>>
>>>> Looks like the problem is spamass-milter's synthesized Received 
>>>> header,
>>>> rather than the spamassassin rule.
>>>
>>> Well spotted, Peter.  I guess I hadn't come up on the right sequence
>>> of keywords to bring that Google search result into the first page.
>>>
>>> Yes, that does seem like the bug I've encountered.  I noticed that a
>>> lot of my false-positives (though not all) were on messages directly
>>> sent by authenticated SMTP clients, i.e. where there was no
>>> pre-existing Received header.  (I didn't realize that spamass-milter
>>> was faking one.)
>>>
>>> My fix for authenticated clients was much simpler and cleaner than
>>> what was proposed on that page, however:  spamass-milter has a "-I"
>>> option that will skip calling spamc/spamd altogether for authenticated
>>> SMTP connections!
>>>
>>> However, the false-positives still remain for non-authenticated
>>> clients.  At least I have a better idea of where the problem lies, and
>>> what to look for in terms of a solution.
>>>
>>> Thanks,
>>> Gilbert
>>>
>>>> On 11/15/2011 10:45 AM, Gilbert E. Detillieux wrote:
>>>>> On 2011-11-14 17:46, Kevin McGregor wrote:
>>>>>> So you've changed the date manually to be exactly the same, and the
>>>>>> rule
>>>>>> doesn't trigger?
>>>>>
>>>>> Well... Here's the weird thing: if I pass the exact same message
>>>>> through
>>>>> spamc manually, I don't get the false positive on that rule. So, I
>>>>> tried
>>>>> mailing that message back to myself from a non-local mailer (so 
>>>>> that it
>>>>> goes through spamass-milter again), but this generates extra 
>>>>> "Received"
>>>>> headers that change the behaviour. (I now get a trigger on the
>>>>> DATE_IN_PAST_24_48 rule, since the message is now that old.)
>>>>>
>>>>> So, I can't test under exactly the same conditions. Given that 
>>>>> running
>>>>> the message through spamc manually didn't trigger the rule, I'm 
>>>>> tempted
>>>>> to think it might be something in the spamass-milter configuration,
>>>>> which is causing some information to not be transferred to spamc, 
>>>>> or to
>>>>> be transferred incorrectly. Not sure at this point.
>>>>>
>>>>> Gilbert
>>>>>
>>>>>> On Mon, Nov 14, 2011 at 4:56 PM, Gilbert E. Detillieux
>>>>>> <gedetil at cs.umanitoba.ca <mailto:gedetil at cs.umanitoba.ca>> wrote:
>>>>>>
>>>>>> I mentioned this problem at the last round-table session, but didn't
>>>>>> get a solution, so I thought I'd post it here, just in case anyone
>>>>>> has any suggestions to offer.
>>>>>>
>>>>>> I'm still seeing a whole bunch of false positives in SpamAssassin,
>>>>>> since an update was installed in mid-September on a CentOS 5.7
>>>>>> system, for a rule called DATE_IN_FUTURE_96_Q, which is only
>>>>>> supposed to be triggered when the "Date:" header has a date that is
>>>>>> 4 days to 4 month ahead of the date in the "Received" header that
>>>>>> has the _smallest_ difference in date.
>>>>>>
>>>>>> Here are the headers from the latest e-mail I've received with this
>>>>>> false-positive. (I've stripped out irrelevant headers, for the sake
>>>>>> of clarity and simplicity.)
>>>>>>
>>>>>> >From topfivestories at messagent.__itworldcanada.com
>>>>>> <mailto:topfivestories at messagent.itworldcanada.com> Mon Nov 14
>>>>>> 07:50:13 2011
>>>>>> Received: from mail.messagent.itworldcanada.__com
>>>>>> <http://mail.messagent.itworldcanada.com>
>>>>>> (mail.messagent.itworldcanada.__com
>>>>>> <http://mail.messagent.itworldcanada.com> [207.112.10.80])
>>>>>> by palladium.cs.umanitoba.ca
>>>>>> <http://palladium.cs.umanitoba.ca> (8.13.8/8.13.8) with SMTP id
>>>>>> pAEDoAxV028594
>>>>>> for <gedetil at cs.umanitoba.ca
>>>>>> <mailto:gedetil at cs.umanitoba.ca>>; Mon, 14 Nov 2011 07:50:12 -0600
>>>>>> Date: Mon, 14 Nov 2011 08:50:13 -0500
>>>>>> X-Spam-Status: No, score=-0.3 required=5.0
>>>>>> tests=BAYES_00,DATE_IN_FUTURE___96_Q,
>>>>>> HTML_MESSAGE,RP_MATCHES_RCVD autolearn=no version=3.3.1
>>>>>> X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
>>>>>> palladium.cs.umanitoba.ca <http://palladium.cs.umanitoba.ca>
>>>>>>
>>>>>> Note that I'm calling spamd via the spamass-milter on a system
>>>>>> running sendmail. Note also, that in the above example, the only
>>>>>> "Received" header was the one generated by my own server. (I've had
>>>>>> other false positives, however, with multiple "Received" headers,
>>>>>> all of which were within seconds of the time in the "Date" header.)
>>>>>>
>>>>>> Any ideas?
>>>
>>
>

-- 
Gilles R. Detillieux              E-mail: <grdetil at scrc.umanitoba.ca>
Spinal Cord Research Centre       WWW:    http://www.scrc.umanitoba.ca/
Dept. of Physiology and Pathophysiology, Faculty of Health Sciences,
Univ. of Manitoba  Winnipeg, MB  R3E 0J9  (Canada)