Freron MailMate

Comments and changes to this ticket

• 

  benny October 23rd, 2018 @ 08:50 AM

  • State changed from “new” to “accepted”

  No, such a preference does not currently exist. I do get an increasing number of requests for such a feature because apparently Outlook accounts have recently changed default behavior. Gmail actually has the same behavior, but this is different because Gmail does not allow duplicates in general and therefore it's a bit of a race condition (unpredictable) which copy survives. I believe FastMail also has this behavior.

  Let me first explain the background. Traditionally, IMAP and SMTP are two completely different things. One is used for receiving and storing emails and the other is used for sending emails. There's no (standard) way to connect to an IMAP server and tell it to send message X in mailbox Y. Instead, a separate connection is done with an SMTP server and the entire message is sent to this server. After sending it, the email client uploads the email to the IMAP server in order for the sender to have a copy of the email.

  Today, most often the IMAP and the SMTP server is the same server and therefore it makes sense for SMTP to automatically save a copy in the IMAP account — but it's not standard behavior. More importantly, there is (to my knowledge) no simple way for the email client to know that this happens.

  Now, why has the feature you request not been implemented? It's mainly because it's non-trivial to implement. There's a lot to consider:

  • The two copies are not necessarily identical. For example, Exchange adds a lot of headers. It's also not quite clear what happens when a Bcc header is involved. So, should the server copy or the client copy be the one that survives? Should it be optional?
  • A simple implementation which allows the user to discard the local copy automatically is a risky approach. What if the server reverts its behavior? Then nothing is saved until the user realizes that the setting should be disabled!
  • Also, if the local copy is simply deleted then the sent message is unavailable until the message is re-downloaded from the IMAP server.
  • What if the local message has been tagged/flagged? If the local copy is deleted then this information is lost.
  • If MailMate somehow (e.g., based on Message-ID) never re-downloads the message, but simply “maps” it to its local copy then MailMate is telling the user something that isn't quite true. The two messages are likely not really identical. Gmail, for example, some times rewrites the From header before sending. What is “enough” for MailMate to claim it's the same message?
  • If MailMate does “map” like this then there is a race condition: How long should MailMate wait for the message to appear in Sent Messages before giving up and uploading its own local copy?
  • Most users shouldn't need to know what happens. A solution should work out-of-the-box if possible.

  Based on the above I think a solution could/should be a single Composer setting like this (enabled by default):

  [v] Detect and handle duplicate sent messages
  

  When enabled:

  • MailMate compares new emails in Sent Messages with recently sent messages. If a match is found then it maps the local copy to the server copy.
  • Preferably, this happens without downloading the entire server copy.
  • Any flags/tags on the local copy must be applied to the server copy.
  • Preferably, it's not a match if important headers are not identical (From, Subject, ...).
  • Preferably, the potential race condition is solved gracefully. For example, the waiting time could be increased/decreased based on whether or not duplicates have been detected in the past.

  I cannot provide a time frame on this issue, but it helps every time I have to reconsider it :)

  Currently, a workaround is to tell MailMate to use a different mailbox for its sent messages. Create it using “Mailbox ▸ New IMAP Mailbox” and use “Mailbox ▸ Mailbox Type” to assign it the “Sent Messages” type. This way you can at least separate the messages and more easily delete one of the sets if needed.

• You flagged this item as spam.
  

  Stefan Seiz October 26th, 2018 @ 06:41 AM

  Hi Benny,

  awesome. Great suggestion.

• You flagged this item as spam.
  

  Will Wright October 30th, 2018 @ 12:29 PM

  I use MailMate with IMAP/SMTP against an Office 365 account, and I noticed recently that my sent messages was full of duplicates :-(

  Apparently SMTP is a protocol used by legacy email clients (their words, not mine!). Here's the blog post which describes the change.

  There's an Office 365 feedback page where you can vote to be able to turn off saving a copy of sent messages which you might like to do if you're affected by this. (Also worth keeping an eye on in case they do add an option to disable this.)

  Here's the link to the original Office 365 feedback page which asked for this feature in the first place.

  I love the workaround - thank you Benny - and the proposed longer term solution.

  I personally can see use cases for the server-side saving of sent messages, but would much prefer it to be optional!

• You flagged this item as spam.
  

  Will Wright October 30th, 2018 @ 12:48 PM

  I also use MailMate with FastMail, which has a Save a copy when sending through 3rd party clients option. If you use MailMate and have that option enabled, then you'll get duplicates in your sent messages.

  Here's a link to the FastMail documentation which explains how to change the setting. There's also a blog post but the interface has changed a bit since that was written.

• You flagged this item as spam.
  

  Matthew September 9th, 2019 @ 07:37 PM

  I've got an Office 365 account and I'm also experiencing this.

  Just to confirm that I understand what's going on...

  • The messages are not actually being sent twice
  • They're appearing twice in the 'Sent' folder
  • The reason is that Microsoft is now automatically saving a copy of sent messages, which they did not used to do
  • It's not an issue with Gmail, because they don't allow duplicate messages

  Is that correct? And the workaround is to dump sent messages into a different mailbox? Or just live with 2 copies in the sent folder?

  Thanks!

• You flagged this item as spam.
  

  Williper Larnloge October 31st, 2019 @ 08:29 PM

  I'm having the same issue with 365.

• You flagged this item as spam.
  

  Klay November 27th, 2019 @ 01:02 AM

  Heart broken......this issue has hit me with Office 365 service and the only realistic option is to change my email client....

• You flagged this item as spam.
  

  Will Wright November 27th, 2019 @ 02:20 PM

  I've been using the workaround Benny suggested for over a year (tell MailMate to use a different mailbox/folder for sent messages) and it works fine for me.

  @Matthew - yes, everything you said (except the Gmail not allowing duplicates part, because I'm not familiar with that) matches my experience of what happens.

  @Klay - maybe worth trying the workaround

  Microsoft should really provide a way of disabling this feature.

• You flagged this item as spam.
  

  Klay November 28th, 2019 @ 01:35 PM

  @Will - now this is where I eat humble pie and admit that I didn't understand the workaround nor did I try it. Having now done so, I can live with it and RELIEVED you posted that response! That workaround is acceptable and more importantly, I can continue to use my most favourite email client. You have no idea how much that made my day.

  Thank you sir.

  @Benny - thank you for the workaround.

• You flagged this item as spam.
  

  Antelope9 December 19th, 2019 @ 07:05 PM

  I experience this also using O365 and accessing via IMAP and smtp. I've done the workaround so I can at least delete these dupes.

• You flagged this item as spam.
  

  Colin June 28th, 2020 @ 01:56 PM

  I have created a 'sent items' folder and set it as mailbox type sent message as above. Now I have changed to a different smtp server and no longer have duplicates. Now I have the usual sent messages folder and my created sent items both contain all my sent items. I want to remove the folder 'sent items' without losing my sent messages. How do I do this? It seems impossible now to change its type.

• You flagged this item as spam.
  

  Shri August 13th, 2020 @ 05:12 PM

  Thanks Benny for the informative post on the background and pointing out the complexities in addressing a seemingly simple issue. The proposed workaround is working well for me. Thanks!

• You flagged this item as spam.
  

  Warren Beck February 20th, 2021 @ 09:55 PM

  Just a report that I am also observing the duplicate sent messages behavior reported above when sending messages from my Office365 IMAP account; this is in macOS 11.2.1 "Big Sur" and running MailMate Version 1.14 (5757).

  Benny's workaround to make an extra sent messages folder works for me; one copy goes into the default "Sent Items" folder and one into the new, "Extra Sent" folder.

  This dual sent messages business does not happen with my iCloud IMAP account.

• You flagged this item as spam.
  

  Will Wright March 11th, 2021 @ 09:02 PM

  I think Microsoft have added a way to disable the server-side saving of sent items for mail sent via SMTP (or at least SMTP AUTH).

  They had documented it on their feedback page which I linked to in an earlier post, but unfortunately they have since shutdown their feedback site, so the page is no longer available :-(

  I managed to pull the following from Google's cache:

  We have completed adding the setting to turn off saving messages in the Sent Items folder for the SMTP AUTH client submission protocol. You can edit the setting for a given mailbox by using PowerShell and the following command:

  Set-Mailbox -Identity bob@contoso.com -MessageCopyForSMTPClientSubmissionEnabled $false
  

  There's a bit more to it than just firing up a PowerShell prompt - you'll need to connect to Office 365 (Exchange) via PowerShell first. The first steps in this blog post cover how to get started.

  I haven't tried this setting yet, but it looks promising.

• You flagged this item as spam.
  

  vg March 11th, 2021 @ 10:14 PM

  Confirming that this resolves the issue, the setting takes about 15 minutes to go into effect.

  Thanks for finding that Will!

• You flagged this item as spam.
  

  Warren Beck March 11th, 2021 @ 10:35 PM

  Will Wright's solution does not help users that are using Microsoft's hosted Office 365 email services. There are no settings accessible to the user to change server handling of sent messages.

  I would like to vote for addition to MailMate on an account basis the option not to save sent messages. For Office 365 users, this is the way to solve this problem.

• You flagged this item as spam.
  

  Will Wright March 11th, 2021 @ 11:11 PM

  @vg Thanks for confirming, good to know it works!

  @Warren this certainly is for hosted Office 365 (in fact what little Microsoft documentation there is says this is cloud-only, presumably meaning it doesn't work for on-premises Exchange.) The PowerShell is effectively logging into the backend of your hosted Office 365 instance, and running the Set-Mailbox command on the "Exchange Online" server that is handling your O365 mail. There are lots of Office 365 settings that are not accessible via the web admin, just through PowerShell.

  I should have mentioned - because the PowerShell is effectively doing admin on your O365 instance, you need to authenticate with an account which has (probably) the Exchange admin role.

• You flagged this item as spam.
  

  Warren Beck March 12th, 2021 @ 03:25 AM

  @Will Understood, but for a user of an institution's Office 365 account, change of settings requiring admin privileges is not usually going to be available.

  Benny's solution actually works, but what I had to do was to move all of my Office365 sent messages into the new Sent Items folder so that they would show in the Sent Messages folder in MailMate. This is fine as long as one just uses MailMate to access the account, but if the user chooses to use the Office365 web client, then the sent messages will not be in the UI's Sent Messages folder; they will be in the perhaps hidden new Sent Items folder. This is not ideal, so I still favor some kind of MailMate preference about this.

• 

  benny March 12th, 2021 @ 08:00 AM

  @Will: Thanks for sharing the Exchange setting (for those able to use it).

  @Warren: A setting in MailMate would be the ideal/easy solution for most users, but as I have described above, it's far from straightforward to implement in a safe/sensible/robust way.

• You flagged this item as spam.
  

  Tim November 11th, 2021 @ 04:31 AM

  Despite the solutions available, I am finding this issue difficult to address — this is the only issue holding me back from fully adopting MailMate.

  Has there been further consideration to implement a setting in MailMate, @benny?

• 

  benny November 14th, 2021 @ 11:14 AM

  @Tim: Sorry, this is still a todo item (and not one of the fun ones workarounds -- bad servers/standardization). As described above, it's also pretty hard to come up with a good workaround. I'm not rejecting it -- just don't expect it soon.

• You flagged this item as spam.
  

  John Kree March 12th, 2022 @ 04:55 PM

  I would love to see a setting in Mailmate for this issue.
  As much as I understand even Outlook for Windows and Outlook for Mac use a setting for this matter.
  https://support.microsoft.com/en-us/office/imap-sent-messages-are-d...

  And Mail.app on Mac has the option to just choose "save sent emails on server".

• You flagged this item as spam.
  

  Angelo Machils October 4th, 2022 @ 07:51 AM

  • State changed from “accepted” to “new”

  Hello,

  Just a little nudge. It would be really great to have such an option in MailMate! I'm also running into this now with an Office365 account.

  With regards,
  Angelo Machils

• You flagged this item as spam.
  

  Tariq Magdon-Ismail October 30th, 2022 @ 05:11 AM

  +1

• You flagged this item as spam.
  

  Christian Bailey July 3rd, 2023 @ 07:06 PM

  I now have the opposite problem. MailMate is not uploading a sent message to the corresponding IMAP server.

  The problem arises because I'm using IMAP to access a Microsoft 365 tenant which has SMTP disabled. So I'm using Google as the SMTP only (it is also included in the SPF record).

  However, after sending mail successfully, MM does not upload a copy to the IMAP Sent Items folder. I have ensured that right-click Mailbox Type is set to "Sent Messages".

  Is there a way to force this behavior?

• You flagged this item as spam.
  

  Christian Bailey July 4th, 2023 @ 06:12 PM

  I discovered this only happens because I had left the Google account as a separate Source. Even though I was sending using the new Source (Microsoft IMAP, Google SMTP), MM was apparently not saving in new Source Sent Items because it was aware of the Google account. Once I deleted the Google account, sent messages are saved as expected. Maybe an artefact of the anti-duplicate logic you built @Benny. In any case, it's working for my use case now.

• You flagged this item as spam.
  

  Eivind December 29th, 2023 @ 05:33 PM

  Hi Benny,
  Bumping this thread, would be super nice to be able to use MailMate smoothly with Office365 again. I finally have a setup that works, but it is incredibly awkward: First set up DavMail for two-factor authentication. It has recently started crashing upon startup, so before opening MailMate, open DavMail. In settings, change Exchange protocol to O365Modern. Go to http://localhost:1080 and authenticate with two-factor authentication. Close website, change Exchange protocol to O365Interactive in Davmail, then open MailMate. This routine needs to be repeated every morning, and that's just the start of it.

  In MailMate, I followed Bennys advice to create a 'sent-workaround' mailbox of type "Sent Messages". Now after every sent email in MailMate, press esc in the error-dialogue*, then right-click the name of the mailbox under Sent Messages and press "Take 'mailbox-name' offline". Do the same for deleted messages. Delete the most recent mail from sent-workaround, then delete the email again from the mailbox for deleted messages. Go online again for both. The sent email is then present in the sent folder, but only one copy, as it should. Puh! I really dislike Office365, but this workaround is almost as bad.

  I really love the MailMate software, but it would be super nice with a better fix for this. Any tips are highly welcome, there should be a better approach, right? I don't have administrator privileges for my Office365 email, so no approach that requires cliend_id and client_secret is possible. Thank you!

  *(Log says something like: APPEND UIDVALIDITY (1157) did not match known value (1))

  Best Regards
  Eivind

• You flagged this item as spam.
  

  Christian Bailey March 5th, 2024 @ 04:28 PM

  Some additional detail which may help someone else find a solution. I'm only seeing this duplicate sent items behavior on one M365 tenant (of which I'm not an admin), and not on another one (of which I am an admin). So presumably there is some server-side setting that might be changed? Can anyone point me where to look?

• You flagged this item as spam.
  

  Christian Bailey April 18th, 2024 @ 06:36 PM

  I'm still seeing this bug in r6030. As a test, I added another account from our M365 tenant and that one also causes the bug in Mailmate. Two identical copies of every sent email in Sent Items. So Mailmate is not detecting the duplicate. Benny, do you need any help troubleshooting?

  

• You flagged this item as spam.
  

  Deborah Mundorff June 3rd, 2024 @ 04:51 PM

  Checking in on the duplicate emails in sent folder... any updates or ETA?

• You flagged this item as spam.
  

  Christian Bailey June 12th, 2024 @ 05:35 PM

  I just updated to MailMate_r6038.tbz. This bug still exists on the same two out of three M365 accounts.

• You flagged this item as spam.
  

  Christian Bailey June 28th, 2024 @ 07:43 PM

  I just updated to r6046. This bug still exists on the same two out of three M365 accounts. Every few days I need to manually delete the duplicates. Is there any chance this will get looked at?

• 

  benny June 30th, 2024 @ 09:00 PM

  @Christian: I'm not an expert on what is possible to configure with Office365/Exchange. Maybe this earlier reply from another user is helpful.

  @Eivind: You do not have to use Davmail. MailMate should work with OAuth2 if you use the latest test release: r6037. It should also work for outlook.com/hotmail.com/live.com addresses.

  In general, I still refer to my first reply in this thread in which I also describe a possible workaround (see the last paragraph). I'm not opposing some smart workaround handling this non-standard server behavior, but I'd like to re-state that the current MailMate behavior is not a bug.

• You flagged this item as spam.
  

  Christian Bailey July 23rd, 2024 @ 03:33 PM

  @benny: thank you replying. Sadly the workaround Will Wright posted of changing a Microsoft365 flag through powershell doesn't work for me because I'm not an admin on one of the tenants. Understand it's not a bug. Hopefully this will be a client side feature in the future.