The Unix world has two ways of storing mail messages, the traditional mbox format and the newer maildir format. Postfix and Dovecot supports the two mail storage format so you can use any format, but I highly recommend you use the maildir format.
The Mbox Format
This is the traditional way of storing mail messages in the Unix world. In this format, a regular text file which serves as the mail user’s mailbox file is created.
Fig. 1: Mbox storage format
How Mbox works
Receiving and storing a mail
- Lock the mailbox.
- Append the header (usually “From [sender’s email address] [date and time received]”) and the mail into the mailbox file.
- Unlock the mailbox.
Retrieving a mail
- Lock the mailbox.
- Locate and read the mail.
- Update the mail status flag.
- Unlock the mailbox.
Deleting a mail
- Lock the mailbox.
- Move the contents of the mailbox, beginning from the position right after the mail to be deleted until the end of the mailbox, into the position of the mail to be deleted.
- Reduce the size of the mailbox file by the size of the deleted mail.
- Unlock the mailbox.
Searching a mail
- Lock the mailbox.
- Search the mailbox.
- Unlock the mailbox.
Advantages
- Format is universally supported.
- Appending a new mail into the mailbox file is fast.
- Searching text inside a single mailbox file is fast.
Disadvantages
- Has file locking problems.
- Has problems when used with network file systems.
- Format is prone to corruption.
The Maildir Format
This is a new way of storing mail messages. In this format, a directory usually named Maildir is created for each mail user. Under this directory are three more directories named new, cur and tmp.
Fig. 2: Maildir storage format
How Maildir works
Receiving and storing a mail
- Create a unique file in the tmp directory.
- Write the mail into the newly created file.
- Move the completely written mail into the new directory.
Retrieving a mail
- Locate and read the mail.
- Move the mail from new into the cur directory and append the mail status flag into the filename.
Deleting a mail
- Delete the file containing the mail.
Searching a mail
- Search each and every mail file.
Advantages
- Locating, retrieving and deleting a specific mail is fast.
- Minimal to no file locking needed.
- Can be used on network file system.
- Immune to mailbox corruption (assuming the hardware will not fail).
Disadvantages
- Some filesystems may not efficiently handle a large number of small files.
- Searching text, which requires all mail files to be opened is slow.
Visit the forum to ask for help or to give a comment.
***
Posted on 3/20/2007 and last updated on 11/7/2009
Filed under Technical Articles
March 29th, 2008 at 8:26 am
I’m planning to migrate from a mail server running sendmail using the mailbox format, to a server configured using the guide in this site with users from ldap running postfix. Is there a way to migrate from the old mailbox format to the new maildir format without loosing data?
I found some scripts on the net. Anyone has experience using these things?
March 29th, 2008 at 2:02 pm
I do. I used mb2md.pl
I initially configured my mail server to use mbox. The users encountered locking problems when they began to check their mails so I immediately converted the mailboxes to maildir using the script above.
March 30th, 2008 at 3:03 am
Thanks for the info. Will test that once I finalize the setup i’m putting up using the guide for CentOS 5 on this site.
June 2nd, 2008 at 4:08 pm
1) mbox’s “Rewrite the whole mailbox file, leaving out the mail to be deleted” isn’t exactly right. The data is moved within the file, so deleting mails from the beginning of the mbox causes more data to be written than mails deleted from end of file. So deleting the last mail(s) wouldn’t cause any data to be written at all, just a ftruncate() call.
2) Maildir is missing one disadvantage (mbox’s advantage): Maildir is slower when all files have to be opened (e.g. searching a message). It’s faster to read through one big file than open and read lots of small files.
June 2nd, 2008 at 4:50 pm
Hi Timo,
I’m very honored to hear from the author of Dovecot. Thank you very much for your comment, I’ve updated the text above.
1. Corrected the mbox delete algorithm.
2. Added the algorithm for the mbox and maildir search.
3. Discussed the advantages and disadvantages of searching when using mbox and maildir.
December 27th, 2008 at 11:08 am
Thanks for the indepth explination of the two formats. I’m using Maildir for my email server, however (I’m using postfix) the default setup uses mbox. I wish I had have found your site a little earlier!!! 😛 and the mb2md.pl (well I have it now!!!)