X-Spam-Score parsing should be better
Reported by Bill Cole | January 31st, 2012 @ 10:39 PM
It is a somewhat common practice to format the X-Spam-Score header as suggested by the MIMEDefang "milter" (widely used with Sendmail and Postfix) like this:
X-Spam-Score: $hits ($score) $names
Where $hits is a numeric score (e.g. from SpamAssassin), $score is a string of asterisks as long as the integer part of $hits, and $names is a string containing a comma-delimited list of the specific rules matches.
MM tries to parse that header, but it only works when $hits is a negative number. Any message with a positive value for $hits is shown in the message list "Spam Score" column as 0.000. (Parsing works correctly with mail filters that only put a number in X-Spam-Score.)
Comments and changes to this ticket
-
benny February 1st, 2012 @ 09:16 AM
- State changed from new to accepted
Actually, MailMate does not parse
X-Spam-Score
. It parsesX-Spam-Status
which in one of my messages look like this (added by SpamSieve I believe):X-Spam-Status: No, score=-2.532 tagged_above=-100 required=6.31 tests=[BAYES_00=-2.599, RCVD_BY_IP=0.067]
That message also has:
X-Spam-Score: -2.532
So, it may work to base the column on that header instead, but I'm not sure that is true in general. To be sure, I could base it on
X-Spam-Score
and then fall back toX-Spam-Status
.Could you provide me with some examples of your
X-Spam-Score
headers, so I can be sure it'll work for you?(Not sure what is going on with the the positive values. I have both positive and negative values in my
X-Spam-Status
headers and they seem to work fine in the Spam Score column.) -
Bill Cole February 2nd, 2012 @ 04:33 AM
Amazing! It appears I've had a bug in some filter code that has gone completely unnoticed for years. Apparently it is adding a X-Spam-Status header similar to the SpamSieve one (less [] around he test list) only to mail scoring below -1. How very odd! I have some work to do...
Below are 20 examples of X-Spam-Score headers generated by the MIMEDefang default pattern. I've trimmed them to 79 characters to prevent wrapping: the trimmed-off parts were just more of the comma-delimited rule name list. Note that I can't provide samples for scores over 4.3 because I reject those messages in SMTP, but a system that accepts and delivers obvious spam might add headers with scores that are very high. I've seen spam scores over 100, but not often. It is also possible to see negative scores of that magnitude, usually because of manual whitelisting. It should be simple enough to just extract the first token of the header value and confirm that it looks like a number.
X-Spam-Score: -8.7 () BAYES_50,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,RCVD_IN_RP_RNBL X-Spam-Score: 3.826 (***) BAYES_99,DATE_IN_PAST_03_06,HTML_FONT_SIZE_HUGE,HTML_ X-Spam-Score: -0.009 () BAYES_50,HTML_MESSAGE,SCC_DEBUG,T_RP_MATCHES_RCVD X-Spam-Score: 2.7 (**) BAYES_50,HTML_MESSAGE,SCC_DEBUG,TVD_FROM_1 X-Spam-Score: 1.061 (*) BAYES_50,NO_DNS_FOR_FROM,RCVD_IN_DNSWL_NONE,RCVD_IN_SCC X-Spam-Score: -1.06 () AWL,BAYES_00,HTML_MESSAGE,SCC_DEBUG,T_RP_MATCHES_RCVD X-Spam-Score: 0.011 () BAYES_50,DKIM_SIGNED,DKIM_VALID,RCVD_IN_DNSWL_NONE,SCC_D X-Spam-Score: 2.141 (**) BAYES_50,RCVD_IN_DNSWL_MED,SCC_DEBUG,SINGLE_HEADER_1K, X-Spam-Score: 0.937 () BAYES_50,HTML_IMAGE_RATIO_02,HTML_MESSAGE,SCC_DEBUG,SCC_ X-Spam-Score: 1.3 (*) BAYES_40,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SCC_DEBUG,SCC_RCV X-Spam-Score: 2.772 (**) BAYES_50,DKIM_ADSP_CUSTOM_MED,FORGED_YAHOO_RCVD,FREEMA X-Spam-Score: 2.449 (**) BAYES_60,HTML_MESSAGE,RCVD_IN_BRBL_LASTEXT,SCC_DEBUG X-Spam-Score: -2.001 () BAYES_60,HTML_MESSAGE,RCVD_IN_DNSWL_HI,SCC_DEBUG,SPF_PA X-Spam-Score: 1.735 (*) BAYES_50,FREEMAIL_FROM,HTML_MESSAGE,MIME_QP_LONG_LINE,R X-Spam-Score: -0.712 () BAYES_20,SCC_DEBUG,SPF_PASS,T_RP_MATCHES_RCVD X-Spam-Score: 0.002 () BAYES_50,HTML_MESSAGE,LOTS_OF_MONEY,SCC_DEBUG,SCC_SHORTN X-Spam-Score: 3.758 (***) BAYES_50,RCVD_IN_BRBL_LASTEXT,RDNS_NONE,SCC_DEBUG,SPF X-Spam-Score: 0.089 () BAYES_50,SCC_DEBUG,SCC_RCVD_FORMAT_NOT_SENDMAIL,SPF_PASS X-Spam-Score: 2.252 (**) BAYES_50,DKIM_ADSP_CUSTOM_MED,FREEMAIL_ENVFROM_END_DIG X-Spam-Score: 2.252 (**) BAYES_50,DKIM_ADSP_CUSTOM_MED,FREEMAIL_ENVFROM_END_DIG
-
benny February 2nd, 2012 @ 09:13 AM
- State changed from accepted to fixcommitted
Based on your examples I am now also parsing
X-Spam-Score
and I improved the parser forX-Spam-Status
. Only problem is that I only know that it'll work for my messages and your examples. These headers really need to be standardized :-)The Spam Score column is based on
X-Spam-Status
if a score is found within that and otherwise it uses theX-Spam-Score
header. (Unfortunately sorting only works forX-Spam-Status
.)And now the fun part. I also added parsing of the test names (for both headers). So now it is possible to use the Statistics layout and, e.g., see what it comes up with if one selects “X-Spam-Status ▸ Tests ▸ Test”. Here is my top 10 for some mailing list messages not marked as spam:
% Name (count) 40.6: BAYES_00 (5029) 22.4: AWL (2779) 16.6: RCVD_BY_IP (2062) 8.3: HTML_MESSAGE (1032) 1.8: HTML_30_40 (219) 1.8: HTML_40_50 (218) 1.2: HTML_50_60 (145) 1.0: RATWARE_GECKO_BUILD (122) 0.9: HTML_20_30 (113) 0.7: HTML_10_20 (86)
Maybe not very useful :-)
-
Michael Tsai March 29th, 2016 @ 05:55 PM
Actually, MailMate does not parse X-Spam-Score. It parses X-Spam-Status which in one of my messages look like this (added by SpamSieve I believe)
I just came across this old thread and wanted to clarify that SpamSieve does not add or change the X-Spam-Status (or any other header).
-
benny March 29th, 2016 @ 06:35 PM
@Michael:I have no idea why I would have written that since it makes no sense :-) Most likely I meant to write SpamAssassin.
Please Sign in or create a free account to add a new ticket.
With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.