I have a domain’s “catch-all” setting configured to redirect emails to a special account that has several message filters (rules) that intelligently route to various destination accounts based on headers detected by the rules.
Since updating to X5, I have noticed that when Axigen redirects a message to the account configured as a catch-all, the message filters no longer execute. They executed as expected in X3, the previous version I used. They also execute if I email the account used as a catch-all directly.
Is there a setting change that I have missed that would allow me to enable filter execution after catch-all redirects? Any other advice on how to re-enable this functionality?
In doing some troubleshooting, I have observed that message filters do execute as expected on the catch-all account after a redirect, but only if I have reduced the size of the message filters list. The full list worked fine in X3, but in X5, it seems the filter list must be fairly small in order to properly execute filters when handling a catch-all redirect.
Note that simply disabling some of the filters did not resolve the issue. I had to delete some filters in order to reduce the size of the waDomainFilter.sieve file.
In order to understand the reported behavior please increase the log level for PROCESSING to Protocol Level, add more admin rules so you could replicate the issue (for example during a maintenance window) and share the related PROCESSING log lines.
Thanks for the advice on the log level. I think this confirms my hunch that the length of the file is causing a problem:
2023-12-08 19:05:59 +0000 04 mail PROCESSING:00000000: Error parsing sieve filter: [Syntax Error]: Error on line 347: memory exhausted 2023-12-08 19:05:59 +0000 04 mail PROCESSING:000A5481: Unable to load sieve filter from file @00-00-8D40-5DE2D2CE-691438D5:00000002:13
This is with a waDomainFilter.sieve file that is 17,205 bytes, which I believe to be approximately at the threshold. (I removed a bunch of filters and then re-added them until it failed.) Also note that within the file, one of the filters is quite long, perhaps ~10 kilobytes on its own, in case the size of individual filters is notable for some reason.
A quick follow-up: I did some additional testing with the long rule I mentioned above, since I observed that the failure line from the quote above, line 347, was within that long rule. The rule in question works at its current size of 10,698 bytes and 243 lines in the if anyof block. But adding another line to the if anyof block will cause the memory exhausted message quoted above.
(Just for context, this rule matches a large number of “From” addresses and routes them to a specific recipient.)
$ /opt/axigen/bin/axigen --version Axigen server version: 10.5.12 (Linux/x86_64) $ uname -a Linux [redacted] 4.19.0-18-cloud-amd64 #1 SMP Debian 4.19.208-1 (2021-09-29) x86_64 GNU/Linux
The filters/rules are in the domain-wide location: Edit Domain > Message Filters > Incoming Message Rules. I am checking the size and contents of the domain’s waDomainFilter.sieve file via Axigen’s ftp backup service. I suppose this means these filters apply to all users on the domain, but that has not been a problem. This configuration worked well in past Axigen versions, and appears to work well in X5 as well, barring the memory exhaustion issue.
I could attempt moving the filters to the catch-all account specifically, and in fact had started doing that before posting this thread while trying to diagnose the issue myself, but I did not test a full migration. I had moved a single filter over to the catch-all account and observed that it wasn’t running, but this was with the full battery of domain filters still present, which I assume means the server was hitting a memory limit, causing all filter execution to stop. Let me know if it will help your team’s diagnosis to test with the full battery of rules moved over to the catch-all user specifically.
I have observed the filters executing correctly (that is, apparently without the same memory limitation) when I email the catch-all account directly, and failing when I expect them to execute after a catch-all redirect action. However, the underlying issue may or may not be related to the catch-all function. It is merely the scenario where I observe the memory exhaustion issue. Perhaps in the catch-all redirect scenario, there is more memory pressure from elsewhere? But it’s also possibly my tests weren’t correct?
We were able to reproduce your report by using more than 240 “mail from contains” conditions bundled into a if-any section.
Checking internally we got the explanation that we are reaching a limit for the number of so called parsing tokens and not for the entire file.
Thus, starting with a rule that had initially 1000 conditions we were able to split them in 5 rules and all were fine, meaning no more ,og errors like: PROCESSING:00000000: Error parsing sieve filter: [Syntax Error]: Error on line 388: memory exhausted
This is why our suggestion is to review your rules (around the line where the log line is complaining about memory exhaustion) and adapt them so each condition sections are not big enough to trigger the error.
HTH,
Ioan
PS: And no, the way it was designed, the incoming rule feature was not intended to support “hundreds” of conditions; for this to happen, a redesign in the parser section of the code is needed, which is improbable to happen in the next period of time.