Understanding DomainPOP duplicates

DomainPOP relies on retrieving mail from your ISP and delivering it to local users based on parsing the available headers and essentially guessing at the best recipient or recipients.  This is very different then SMTP which has a well defined RCPT TO command which defines exactly who the message is addressed to.

To create an analogy, imagine I write a letter addressed to Bob, and courtesy copied to Frank and Henry.

Inside the envelopes, the letters are identical, they’re all addressed to Bob, and courtesy copied to both Frank and Henry.  However, on the envelope I only write the one recipient who is to receive that copy of the message.  When the messages are delivered to your ISP via SMTP, there is both an envelope and the a body (the “letter” in this analogy)

In the case of DomainPOP, it’s like someone opens all the envelopes and throws them out, all you receive is the letter.  Without the envelope, you don’t know exactly who the message should be delivered to.  You can guess based on the available data, the TO and CC headers, but this won’t help you with BCC’d recipients (blind carbon copy, by definition, doesn’t have the recipient’s address in the TO or CC Fields)

By default MDaemon takes the safest route and delivers one copy to everyone mentioned in the various headers MDaemon reviews, taking the approach that it is better to deliver multiple copies then none at all.

Depending on your upstream ISP’s capabilities it may be possible to avoid these duplicates entirely.  There are two options, start with #1 and if that fails, go to #2.

1) A unique header.

Some ISPs are kind enough to insert the RCPT TO command of the SMTP session into the message.  MDaemon, for example, does this in both the X-MDRcpt-To and X-Rcpt-To headers, as well as a Received header (in most cases).

To test if this is the case, you’ll need an off-site email address, one that has nothing to do with your domain.  Gmail/Hotmail are perfect.  If you don’t have access to one, let me know and I’ll send the test message from here.  For the purposes of this article, I’ll use dave.warren@altn.com as the address which is receiving duplicates and dave.warren@gmail.com as the off-site address.

In MDaemon, go to Setup –> DomainPOP –> [Security] tab, enable the “Place an extra copy” option, and set the directory to a known path.  C:\MDaemon\DomainPOP would be great as this directory doesn’t exist and is easy to find.

Go to your off-site (Gmail) mailbox, and send a message addressed to itself (it should be FROM dave.warren@gmail.com and TO dave.warren@gmail.com), and then BCC yourself (dave.warren@altn.com in this example)

Once it is received by your server, go to the C:\MDaemon\DomainPOP directory and find the message, open it in Notepad.  Look for the dave.warren@altn.com address, and if you find it, check out the header that contains the address.

If it’s a X-Delivered-To or X-Rcpt-To or something like that, then it means your ISP likely creates a header showing the actual recipients of the message.

Once you find a header you want to test with, go to the MDaemon –> DomainPOP dialog again, this time to the [Parsing] tab, remove all the headers in the list and add the header you found (unless it happens to be a Received header, if so, there is a checkbox for the Received header above as this header needs some special handling)

Now, you’ll want to test this before proceeding, you need to make sure that your ISP added the headers and not the sender.  If possible, test from Gmail, Hotmail and Yahoo, or have a couple friends or coworkers try emailing you and make sure it works.

2) DomainPOP’s de-dupe feature

You should only use this if #1 doesn’t reveal an appropriate header to use.

To use this feature, in the DomainPOP dialog’s parsing tab, enable the Dedupe feature.  I’d suggest using the Message-ID header, this is the safest, but some mail clients don’t generate a Message-ID so you’ll still see some duplicates.

Another possible header is the Date header, virtually all mail has a Date header, but if two senders happened to send mail at exactly the same second, from the same timezone, it’s possible that one of the messages could get lost, so there is a bit of risk involved with this choice.

CC BY-NC-ND 4.0 Understanding DomainPOP duplicates by Dave Warren (everything-mdaemon.com) is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

14 Replies to “Understanding DomainPOP duplicates”

  1. Please, I need answers to the following on MDaemon:

    1. How to use domain POP to send mails to user’s mailbox.
    2. How to use MDaemon to send out mails to the internet
    3. How to use MDaemon to browse.

    Please, I need the outlines and steps involved only.

    I shall be indeed grateful if my questions are favourably entertained.

    Thanks.

    Kenny

  2. sir,
    i need to configure gmail account under mdaemon e-mail server. could find the way out. pls suggest how & where to put port number for incoming & outgoing.

    thanking you
    manoj dhawan

  3. Hi Dave! Thanks for all the MDaemon tips. I’ve been running MDaemon ever since Arvil used to answer his own emails. I’ve version 10 at the office and am trying to use it as a secondary gateway for our office mail. We have just switched our mail over to Gmail apps, and I was first delighted to see that multipop would work with Gmail, then crushed to see that SSL was a “pro” feature.

    I might be able to do a workaround if I could get domainpop to work with Gmail but don’t see any possibility of that in the Standard version.

    Do you see any hope?

    PS: I’m about to drop MD over this (and the pernicious notion of putting DomainKeys/DKIM in the Pro edition!).

  4. Unfortunately most new features will likely be Pro features, that includes SSL as well as DKIM.

    Even if you’re using Google Apps you don’t have to use Google for mail, you can deliver directly to your server by specifying your MDaemon server in your MX records, bypassing Google’s servers entirely.

    Alternatively, you can work around SSL by adding stunnel to the mix, but you won’t be able to work around DK/DKIM as easily.

  5. Hi All
    We’re using MDaemon 9.06. ISP= iinet.net.au We can receive mail from the likes of @yahoo or @hotmail, however, when it comes to @gmail we just don’t receive it. It appears to dissapear off the radar. More and more of our clients are using gmail and we’re just not receiving any of them ?
    It’d be great if someone could point me in the right direction to start investigating ?
    Regards
    Tom

  6. I am having difficulties in logging in, every time i log in, it always say “invalid IP” what should i do?

  7. Saludos, tengo interés en montar un servidor en casa con MDaemon pero quiero saber primero si se puede implementar usando Google Apps con la posibilidad de correo.
    Gracias de antemano.

    Carlos

  8. sir kindly sagest me i have mdeamon 8.1.3 i configar multy pop account and some local account i want to send emai local account to multy pop account oute site like gmail or yahoo or any account how can i setting mdeamon local account email go gamil or yahoo or any other account

    for example

    faisal@local.mail to gmail.com or yahoo.com any other

  9. Users cant receive/send foreign mail to/from mdaemon mail server. please help me with the right configuration so that users can receive mails from yahoo, gmail, Hotmail…. and other foreign domains

  10. MultiPOP and DomainPOP aren’t really comparable, they solve different problems in different ways. DomainPOP has the advantage of not needing any per-user configuration, but you need a deeper understanding of your ISP’s headers, and you rely on your ISP not changing their headers. MultiPOP, on the other hand, needs per-user setup, but once it’s done, you’ll never have misrouted mail ever.

    MultiPOP is arguably more resource intensive, but it’s a fairly minor issue for most deployments.

    Personally, I believe that if you’re going to run a mail server, SMTP is a better choice than anything in the POP3 family.

Leave a Reply

Your email address will not be published. Required fields are marked *

*