Proprietary Storage

The Axigen Solution: Overview & Architecture

The Axigen storage is a specific file structure with index based access allowing fast mail delivery, retrieve and query.

The Axigen mail storage checks the consistency of the messages placed in the storage and empties the queue only if the mail message is correctly stored.

All domain and user configuration along with user messages are stored in Axigen specific storage.

Each Axigen storage is defined by three elements:

  • Storage directory: the directory where all storage file will be created;
  • Max. file size: maximum size of a data file (Storage Container). The default value is 1 GB;
  • Max. files: maximum number of files. The default value is 128 files.

Therefore the maximum capacity of each storage is Max. file size * Max. files and the default capacity is 128 GB.

Inside the storage directory, a list of files, named with 2 hexa digits followed by the .hsf extension - e.g. 2A.hsf - are created. There is also a file named hsf.dat which contains an unique id of the storage and the relation with other storages of the same domain. This information is useful in case some of the storage directories are moved to other locations.

Another feature of Axigen storage is that it supports transactions, so that some critical operations of domain configuration changes are made safely.

Filling the containers

When a storage container approaches its maximum size, (defined by the "Max. file size" parameter), another storage container will be created and the new messages will be stored herein. If the number of storage containers reaches the maximum value (defined by the "Max. files" parameter) and all of them have reached the maximum size, the storage is considered full and no more messages will be inserted.

The data in the storage containers is written in blocks of 4KB, therefore usually the files size is a multiple of 4KB. These memory blocks are called nodes. Smaller blocks of memory are also available, for message parts smaller than 4KB. These smaller blocks are called formatted nodes.

Each storage file can contain a maximum of 16 millions messages, and the maximum theoretical file size is 64GB (some limitations might apply, depending on your system configuration; currently Axigen limits this maximum size to 2GB). There can be maximum 128 files in one storage, and one domain can have over 4 billion message storages defined.

The actual maximum capacity in terms of total message count and size depends on the specific messages in the storage.

For each domain, at least three storages are used:

  • one storage for domain configuration, where all domain specific configuration, the public folder and the list of domain objects (users, maillist, forwarders, etc) are stored;
  • one storage for domain objects configuration, where all domain objects configurations and folders are stored;
  • one or more storages for messages, where all mails and other data associated with mails are stored; it is recommended to define each message storage on a different physical disk, since Axigen will use these storages in parallel.

Space saving filling procedure

The storage files with more free space have a priority when it comes to selecting the files in which a new message is added. The usage of the free space is also enhanced by message deletion.

Each message in a storage file is identified by a pointerID (type UINT). The information related to these pointers-to-messages is stored in the same storage file.

  • Indexed data structure

This ensures an optimal balance between the space used for storing indexes and the time to access information by introducing multiple levels of indexes depending on how often information is accessed.

  • Transactional access

Axigen provides structural integrity for the stored information by performing transactional data access.

  • Expandable storage

You can add subsequent storage units to one domain, thus increasing the available space.

  • Single storage of emails with multiple recipients

Only one copy of a message received by multiple recipients is stored (particularly useful for distribution lists and groups).

  • Repairing corrupted accounts

If an account’s data becomes corrupted (due to, for instance, disk failure), Axigen allows you to attempt account data recovery.

  • Storage overload prevention

Overload prevention is ensured by limiting allowed message sizes according to specific policies based on multiple message or connection parameters.