ISP Scenarios
Characteristics
In the case of Internet Service Providers (ISP), the average message size is usually small and users connect trough the POP3 protocol. The extremely high number of users significantly increases the traffic load a messaging solution has to handle; moreover the contractual relationship between the service provider and email users imposes a high standard of service continuity and availability to the supplier.
Testing results
We observed the same indicators as in the previous scenario in order to take notice of the total effect of server overload that is perceivable by the end user.
Unlike the previous scenario, all three tested solutions maintain a very good balance between the accepted and delivered number of messages, not causing queue overload issues. We see that, Sendmail as well as Postfix, although their performance increases proportionally with the number of parallel connections, reach a maximum performance level between 4 and 8 parallel connections and even show a slight decrease (probably caused by the server overload).
Postfix’ performance is two times better than Sendmail's. However, AXIGEN’s performance is far superior reaching in this scenario a level 7 times better than Postfix and 13 times better than Sendmail. If in the first scenario the processing speed is a critical factor only in peak traffic moments, in the case of Internet Service Providers mail server performance is a critical factor. Solution performance translates into efficient usage of hardware resources; thus, double the performance means half the number of necessary servers.
We further analyzed the average response time for the three solutions in the same scenario.
The server response time can employed to efficiently measure the overload for the server the solution runs on. Our results show that the response time increases when the number of parallel connections is higher in the case of all three solutions; AXIGEN reports a rather small response time increase, while Sendmail and Postfix show dramatic growth, especially when switching from 4 to 8 parallel connections. This phenomenon also affects the overall solutions performance (the number of processed messages per second), as previously shown in the above graphic analysis.
Testing Platforms
Conclusions
When choosing a messaging solution, it’s critically important to take into account the average estimated traffic, the maximum load and the expected response time. Even though open source solutions (Postfix and Sendmail) ensure a reasonable performance for a low traffic level, an integrated (commercial) solution features a much higher performance when handling extreme requests, while optimizing hardware resources usage and diminishing peak traffic downtime.
Originally published in PC Magazine Romania (article in Romanian).