I’ve set up Axigen to use rspamd using the “Add Filter” and “Execute Filter” acceptance/routing rules. However rspamd consistently flags emails (even the activation email for this very community) with HFILTER_HOSTNAME_UNKNOWN, leading to the email being marked as spam.
I looked into rspamd and this occurs when rspamd cannot find a hostname for its internal “task”. This “task” apparently is meant to get it from the MTA.
When I take a raw email and put it into rspamd’s “Scan/Learn” tool, it shows the same flag but if I set a hostname, the flag goes away.
Is it possible that Axigen isn’t passing the hostname? Is it possible to check somehow (e.g. enabling some debug setting)?
Hello,
This is a known issue (tracked on our side via ticket AXIGEN#2021063099000146).
The problem is that rspamd (as a milter server) is expecting <Axigen (as a milter client) to present the remote hostname at CONNECT event.
As you probably know the on connect event is triggered at the moment when the TCP connection is established. At this moment only the remote IP and remote port are available and Axigen (as a milter client) is sending them to the milter server.
Now, in order to get the hostname (like the reverse DNS of the remote IP address) Axigen (like any other milter client) has to execute a DNS query as there is no other means to get that information.
In other words, rspamd is delegating the task of executing the reverse DNS query to Axigen but we are deferring this task (by sending hostname as a <null> string) because our SMTP implementation will trigger any DNS related queries at later SMTP events (like at HELO / EHLO event, when we start checking the information presented by remote party).
We are counting on your support to to put this into the attention of rspamd support team (if possible) as maybe something has changed in the last 2 years and our assessment doen in 2021 is not anymore up-to-date.
In the mean time the obvious possible workarounds are:
1/ Disable the option related to that symbol:
in: /etc/rspamd/local.d/hfilter.conf set hostname_enabled = false;
2/ Change the score for HFILTER_HOSTNAME_UNKNOWN from 2.5 to 0 (or a lower value)
in: /etc/rspamd/conf/scores.d/hfilter_group.conf
HTH,
Ioan
1 Like
Thanks for the quick response!
In the case where the MTA doesn’t do the PTR lookup, the spec seems to indicate that you can write the incoming IP address in square brackets instead:
the host name of the message sender, as determined by a reverse lookup on the host address. If the reverse lookup fails or if none of the IP addresses of the resolved host name matches the original IP address, hostname will contain the message sender’s IP address enclosed in square brackets (e.g. [a.b.c.d]). If the SMTP connection is made via stdin the value is localhost.
rspamd supports this and is happy to do the PTR lookup itself.
Do you think implementing this change on your side is a possibility?
Hello,
Thank you for your suggestion but we have already tried to send the hostname as [1.2.3.4] (where 1.2.3.4 is the detected remote IP address at the CONNECT event) but, at least in 2021, we saw that the reported rule (HFILTER_HOSTNAME_UNKNOWN) was still triggered.
More, we have checked rspamd milter related code (from here) and we see that even they are aware that the hostname could be null.
In case you have a working implementation where a milter client is providing the hostname as [1.2.3.4] and rspamd is not triggering the HFILTER_HOSTNAME_UNKNOWN rule please let us know as we need some evidence (like network trace for milter sessions obtained with tcpdump and corresponding rspamd logs) before considering to re-open the investigation on our side.
HTH,
Ioan
Thank you for your suggestion but we have already tried to send the hostname as [1.2.3.4] (where 1.2.3.4 is the detected remote IP address at the CONNECT event) but, at least in 2021, we saw that the reported rule (HFILTER_HOSTNAME_UNKNOWN) was still triggered.
In case you have a working implementation where a milter client is providing the hostname as [1.2.3.4] and rspamd is not triggering the HFILTER_HOSTNAME_UNKNOWN rule please let us know as we need some evidence (like network trace for milter sessions obtained with tcpdump and corresponding rspamd logs) before considering to re-open the investigation on our side.
Good you asked for a test because it turns out I was wrong because I misread the code and comments. When square brackets are present, we still get nil, so we still get HFILTER_HOSTNAME_UNKNOWN.
I tested various permutations with opendkim’s miltertest and got you some packet captures:
rspamd_pcaps.zip (6.6 KB). I also checked the rspamd logs but for the brackets and empty tests there’s nothing useful. I assume there’s some caching involved.
Results:
- Empty hostname: fail (no log output)
- IP in brackets hostname: fail (no log output)
- IP hostname: success (log shows no
HFILTER_HOSTNAME_UNKNOWN)
It looks like rspamd does the lookup itself in the once_received module but unfortunately it looks like hfilter won’t use the result.
I sent a PR to rspamd that seems to resolve this by doing a PTR lookup from inside hfilter when one is necessary.
The developer of rspamd responded to my pull request (I mentioned it previously in this thread but apparently it’s been flagged as spam!?). His answer was:
“I have checked the original ticket and I think that this change is not needed at all. Rspamd indeed, tries to set the hostname from the milter connect stage. However, if there is no information about hostname at the connect stage Rspamd tries to get that from the {client_name} macro that must be announced by MTA. So I think that the problem is clearly on the MTA side: it specifies the IP address as a hostname, that leads to various issues. Hence, I think it is better to fix the MTA behaviour than to add this plugin as a patchwork.”
Thoughts? Do you disagree with his assessment?
Hello,
Thank you for following this issue - we are sorry to see you struggling to resolve somehow this milter interoperability issue hopefully we’ll reach soon to a conclusion.
Now, coming back to Rspamd response (that Axigen should send the hostname via {client_name} macro) our contra-argument is that we are sending the HELO / EHLO string received from remote end (when reaching HELO stage ) so the milter server should have all information on their side.
As mentioned before, we consider that the milter client should not be enforced to make any extra DNS checks and present that information to the milter server.
I’ve also checked your PCAP files and there is no {client_name} macros inside.
HTH,
Ioan
Hello @Theology9356
In case you are still interested to look into this issue we need some data from your side:
1/ check your rspamd.log for HFILTER_HOSTNAME_UNKNOWN lines and extract from them the id: value - it should correspond to the email Message-ID header.
For example you may use a command like:
$ grep HFILTER_HOSTNAME_UNKNOWN /var/log/rspamd/rspamd.log | sed 's|^.* id: \(<[^>]*>\),.*$|\1|' > /tmp/ids.txt
2/ now, having one or more such ids please make a grep for them into your Axigen logs
$ grep -Ff /tmp/ids.txt /var/opt/axigen/log/* > /tmp/info.txt
At this stage we need to receive from your side* the info.txt file as generated from above ( of course if it is not empty 
)
HTH,
Ioan
- you may send a private message to me if you think that the generated output should not be shared openly
Unfortunately it didn’t look like this was going to progress so I moved on to a different email system and no longer have my rspamd and Axigen logs.
No problems - many thanks for sharing all the data and the time you have invested into this issue.
BR,
Ioan