Understanding DomainPOP duplicates

DomainPOP relies on retrieving mail from your ISP and delivering it to local users based on parsing the available headers and essentially guessing at the best recipient or recipients.  This is very different then SMTP which has a well defined RCPT TO command which defines exactly who the message is addressed to.

To create an analogy, imagine I write a letter addressed to Bob, and courtesy copied to Frank and Henry.

Inside the envelopes, the letters are identical, they’re all addressed to Bob, and courtesy copied to both Frank and Henry.  However, on the envelope I only write the one recipient who is to receive that copy of the message.  When the messages are delivered to your ISP via SMTP, there is both an envelope and the a body (the “letter” in this analogy)

In the case of DomainPOP, it’s like someone opens all the envelopes and throws them out, all you receive is the letter.  Without the envelope, you don’t know exactly who the message should be delivered to.  You can guess based on the available data, the TO and CC headers, but this won’t help you with BCC’d recipients (blind carbon copy, by definition, doesn’t have the recipient’s address in the TO or CC Fields)

By default MDaemon takes the safest route and delivers one copy to everyone mentioned in the various headers MDaemon reviews, taking the approach that it is better to deliver multiple copies then none at all.

Depending on your upstream ISP’s capabilities it may be possible to avoid these duplicates entirely.  There are two options, start with #1 and if that fails, go to #2.

1) A unique header.

Some ISPs are kind enough to insert the RCPT TO command of the SMTP session into the message.  MDaemon, for example, does this in both the X-MDRcpt-To and X-Rcpt-To headers, as well as a Received header (in most cases).

To test if this is the case, you’ll need an off-site email address, one that has nothing to do with your domain.  Gmail/Hotmail are perfect.  If you don’t have access to one, let me know and I’ll send the test message from here.  For the purposes of this article, I’ll use [email protected] as the address which is receiving duplicates and [email protected] as the off-site address.

In MDaemon, go to Setup –> DomainPOP –> [Security] tab, enable the “Place an extra copy” option, and set the directory to a known path.  C:\MDaemon\DomainPOP would be great as this directory doesn’t exist and is easy to find.

Go to your off-site (Gmail) mailbox, and send a message addressed to itself (it should be FROM [email protected] and TO [email protected]), and then BCC yourself ([email protected] in this example)

Once it is received by your server, go to the C:\MDaemon\DomainPOP directory and find the message, open it in Notepad.  Look for the [email protected] address, and if you find it, check out the header that contains the address.

If it’s a X-Delivered-To or X-Rcpt-To or something like that, then it means your ISP likely creates a header showing the actual recipients of the message.

Once you find a header you want to test with, go to the MDaemon –> DomainPOP dialog again, this time to the [Parsing] tab, remove all the headers in the list and add the header you found (unless it happens to be a Received header, if so, there is a checkbox for the Received header above as this header needs some special handling)

Now, you’ll want to test this before proceeding, you need to make sure that your ISP added the headers and not the sender.  If possible, test from Gmail, Hotmail and Yahoo, or have a couple friends or coworkers try emailing you and make sure it works.

2) DomainPOP’s de-dupe feature

You should only use this if #1 doesn’t reveal an appropriate header to use.

To use this feature, in the DomainPOP dialog’s parsing tab, enable the Dedupe feature.  I’d suggest using the Message-ID header, this is the safest, but some mail clients don’t generate a Message-ID so you’ll still see some duplicates.

Another possible header is the Date header, virtually all mail has a Date header, but if two senders happened to send mail at exactly the same second, from the same timezone, it’s possible that one of the messages could get lost, so there is a bit of risk involved with this choice.

Understanding DomainPOP duplicates by Dave Warren (everything-mdaemon.com) is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.