Freron MailMate

Comments and changes to this ticket

• You flagged this item as spam.
  

  Tikitu de Jager March 10th, 2014 @ 04:26 PM

  Uninstall/reinstall didn't help.

  Timing information: according to the spinners on the folder bar, the crash hits shortly after synching with my mail provider (Dreamhost) has completed.

• You flagged this item as spam.
  

  Tikitu de Jager March 10th, 2014 @ 04:48 PM

  Terminal output attached. The eml file it's failing on doesn't exist, but there is a (spam) .emlh file with the same number. Not sure if I should be worried about those database warnings or not?

  Some more info about the file that shows up in the terminal output:

  ➜  ~  ls /Users/tikitu/Library/Application\ Support/MailMate/Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22081.eml
  ls: /Users/tikitu/Library/Application Support/MailMate/Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22081.eml: No such file or directory
  ➜  ~  ls /Users/tikitu/Library/Application\ Support/MailMate/Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22081.*
  /Users/tikitu/Library/Application Support/MailMate/Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22081.eml_backup
  /Users/tikitu/Library/Application Support/MailMate/Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22081.emlh
  ➜  ~  ls -l /Users/tikitu/Library/Application\ Support/MailMate/Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22081.*
  -rw-------  1 tikitu  staff     0 10 Mar 18:44 /Users/tikitu/Library/Application Support/MailMate/Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22081.eml_backup
  -rw-------@ 1 tikitu  staff  1940 10 Mar 13:46 /Users/tikitu/Library/Application Support/MailMate/Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22081.emlh
  ➜  ~  ls -ld /Users/tikitu/Library/Application\ Support/MailMate/Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/
  drwx------  102 tikitu  staff  3468 10 Mar 18:44 /Users/tikitu/Library/Application Support/MailMate/Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/
  ➜  ~
  

  • mailmate-terminal.txt 5.4 KB
• 

  benny March 10th, 2014 @ 09:08 PM

  Thanks for the report. It seems it might be related to ticket #453 which was unfortunately never resolved. I'm very interested in more details. First of all, if you take the account offline (or the machine offline), can you then launch MailMate without it crashing?

• You flagged this item as spam.
  

  Tikitu de Jager March 11th, 2014 @ 06:26 AM

  Yes. Attached is a terminal trace -- I've marked it up with when I went online again (it crashed shortly afterwards).

  From the other thread I've understood that the "truncating" messages are just it recovering from a crash: a bit surprising, in that case, to see exactly when they happen on the second invocation (it seems to be part of the crash event, as well as recovery on next invocation).

  Any more details I can get you, just let me know.

• You flagged this item as spam.
  

  Tikitu de Jager March 11th, 2014 @ 06:27 AM

  Sorry, attached the wrong file. This time...

• You flagged this item as spam.
  

  Tikitu de Jager March 11th, 2014 @ 06:27 AM

  Third time lucky?

  • mailmate-crash.txt 8.3 KB
• 

  benny March 11th, 2014 @ 08:35 AM

  Thanks for the output. I'll change the wording to make the truncation sound less serious. MailMate loads various index files lazily and cleans them up when needed (after a recent crash). Essentially, it's rolling each file back to the latest known safe state before using it.

  This is the interesting/unexpected part of the output:

  open: Permission denied  
  Unable to set attributes for new/updated file: '/Users/tikitu/Library/Application Support/MailMate/Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22081.eml'
  

  Does this file exist?

  If you launch MailMate while the account is offline (letting MailMate clean up after the crash), does the file then (still?) exist?

  You can perhaps reduce the problem to the INBOX by taking this mailbox offline while keeping all the other mailboxes online. (That's not very useful, but it's just to narrow down the problem.)

  Thanks in advance.

• You flagged this item as spam.
  

  Tikitu de Jager March 11th, 2014 @ 09:43 AM

  Ah, I think I get it: the odd timing of the "truncating" messages (during a crash, not after) might just be the lazy loading then (recovering to the previous pre-crash state).

  If I go offline and open MailMate (no crash), the suspect file 22081.eml does not exist; there is a corresponding header file, 22081.emlh, though.

  After going online and MailMate crashing 22081.eml still doesn't exist, but there is a file 22081.eml_backup created: 0 bytes, and with permissions

  -rw-------
  

  (I noticed that most of the other files in that dir have the "additional attributes" @-symbol in the permissions, which this one doesn't; also that there are a number of other 0-byte .eml_backup files in the dir, also without the "additional attributes" @. All these files are cleaned up by a successful --offline-- MM launch.)

  Taking the INBOX offline confirmed that this is the source of the problem: no crash when the machine went online, then after taking the INBOX back online, a crash a few seconds after the synchronising spinner disappeared.

  In case it's relevant: I use Dreamhost mail hosting, but I used to have a complicated cross-hookup with gmail, so my IMAP folder layout is probably quite non-standard (I've had problems with it in the past, with a web-based IMAP tool). If you think that might be important I can get more details for you (handy if that's accessible somehow through MM, as I don't know offhand the simplest way to extract this).

• 

  benny March 11th, 2014 @ 09:47 AM

  @tikitu: I think the problem is a MailMate bug related to that .emlh file, but I haven't figured it out yet. I have an idea though. I'll look into it.

• You flagged this item as spam.
  

  Tikitu de Jager March 11th, 2014 @ 11:23 AM

  Thanks. If I try deleting that file and restarting, am I likely to make the situation worse? (I can wait to do so until you've poked at it a bit, if you like.) I attach the file itself, just in case there's something in its content that might be relevant.

  • 22081.emlh 1.9 KB
• 

  benny March 11th, 2014 @ 01:24 PM

  Thanks for the file, but I don't (yet) think it is related to its contents.

  It might work to delete the file (MailMate might ask you to rebuild the database), but then I cannot get any more information :-)

  You can, however, move the following folder to start over and then you can swap it in if I have more requests (first quit MailMate):

  ~/Library/Application Support/MailMate
  

  For now, I would like you to enable the following:

  defaults write com.freron.MailMate MmDebugAttributesCrash -bool YES
  

  Hold down ⌥ when clicking “Check Now” in the Software Update preferences pane (r4063) and then run MailMate from the Terminal. There should be a bit more output before the crash which might be helpful for me.

  Thanks in advance!

• You flagged this item as spam.
  

  Tikitu de Jager March 11th, 2014 @ 02:07 PM

  set_attr (com.freron.MailMate.#uid = 17110): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22118.emlh
  set_attr (com.freron.MailMate.#uid = 17111): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22119.emlh
  set_attr (com.freron.MailMate.#uid = 17112): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22120.emlh
  set_attr (com.freron.MailMate.#uid = 17113): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22121.emlh
  set_attr (com.freron.MailMate.#uid = 17114): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22122.emlh
  set_attr (com.freron.MailMate.#uid = 17115): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22123.emlh
  set_attr (com.freron.MailMate.#uid = 17105): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22109.eml
  Warning: File is larger than expected (165312 > 165072). This is normal behavior after a recent crash (/Users/tikitu/Library/Application Support/MailMate/Database.noindex/Headers/#root-body-part.offsets)
  Warning: File is larger than expected (28475 > 28415). This is normal behavior after a recent crash (/Users/tikitu/Library/Application Support/MailMate/Database.noindex/Headers/#root-body-part.cache)
  Warning: File is larger than expected (165264 > 165024). This is normal behavior after a recent crash (/Users/tikitu/Library/Application Support/MailMate/Database.noindex/Headers/#subpart-number.offsets)
  set_attr (com.freron.MailMate.#uid = 17106): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22110.eml
  set_attr (com.freron.MailMate.#uid = 17107): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22111.eml
  set_attr (com.freron.MailMate.#uid = 17108): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22114.eml
  set_attr (com.freron.MailMate.#uid = 17109): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22115.eml
  set_attr (com.freron.MailMate.#uid = 17110): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22118.eml
  set_attr (com.freron.MailMate.#uid = 17111): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22119.eml
  Problems while parsing reported for one or more messages. Details can be found in
  /tmp/mailmate_logs/mailmate_parser_problems.log set_attr (com.freron.MailMate.#uid = 17112): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22120.eml
  set_attr (com.freron.MailMate.#uid = 17113): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22121.eml
  set_attr (com.freron.MailMate.#uid = 17114): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22122.eml
  set_attr (com.freron.MailMate.#uid = 17115): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22123.eml
  Warning: File is larger than expected (12072 > 12048). This is normal behavior after a recent crash (/Users/tikitu/Library/Application Support/MailMate/Database.noindex/Headers/#mime-preamble.offsets)
  set_attr (com.freron.MailMate.#uid = 17085): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22075.eml
  set_attr (com.freron.MailMate.#uid = 17086): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22076.eml
  Warning: File is larger than expected (11904 > 11880). This is normal behavior after a recent crash (/Users/tikitu/Library/Application Support/MailMate/Database.noindex/Headers/content-id.offsets)
  set_attr (com.freron.MailMate.#uid = 17087): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22077.eml
  set_attr (com.freron.MailMate.#uid = 17088): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22078.eml
  set_attr (com.freron.MailMate.#uid = 17089): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22079.eml
  set_attr (com.freron.MailMate.#uid = 17090): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22080.eml
  set_attr (com.freron.MailMate.#uid = 17091): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22081.eml
  set_attr (com.freron.MailMate.#uid = 17103): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22107.eml
  set_attr (com.freron.MailMate.#uid = 17104): Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22108.eml
  open: Permission denied
  Unable to set attributes for new/updated file: '/Users/tikitu/Library/Application Support/MailMate/Messages/IMAP/tikitu%40logophile.org@mail.logophile.org/INBOX.mailbox/Messages/22081.eml'

  • mailmate_parser_problems.log 464 Bytes
• 

  benny March 11th, 2014 @ 02:27 PM

  A quick question: Can you locate the message (based on the headers in 22081.emlh) on the server? That is, does it exist server-side?

• You flagged this item as spam.
  

  Tikitu de Jager March 11th, 2014 @ 02:38 PM

  Yep. It has a 9k zipped attachment which presumably is malware of some description, and a short text body.

• 

  benny March 11th, 2014 @ 02:41 PM

  Thanks. I don't have more time right now, but when I do I'll try to add even more debug output to get more information. Thanks for the feedback.

• You flagged this item as spam.
  

  Tikitu de Jager March 11th, 2014 @ 03:18 PM

  Thank you. I'll switch that Application Support folder, but I'll keep the old one around in case you'd like to do any more debugging.

• You flagged this item as spam.
  

  Tikitu de Jager March 12th, 2014 @ 05:17 AM

  Another data point for you: moving the Application Support folder and starting again did not solve the issue: MailMate crashes on the exact same message (different filename, of course, but it's the same header file). I suppose probably I can solve the issue by deleting it on the server, but I'll wait to do that until I hear from you that there's no more debugging you want to do.

• 

  benny March 12th, 2014 @ 08:24 AM

  That's very interesting. This means that it probably is a particular message which can trigger this issue. This also means that I should be able to reproduce the issue with this message -- except that the problem might also be related to the IMAP server used. I'm thinking that the problem might be related to MailMate somehow ending up with a completely empty message...

  You could try sending the message to me (http://freron.com/contact). (It would be even better if you could reproduce the problem by copying the message to a test IMAP account I could use, e.g., an otherwise empty Gmail account — but I know this is a lot to ask.)

• You flagged this item as spam.
  

  Tikitu de Jager March 12th, 2014 @ 09:38 AM

  Happy to copy the message to an IMAP account -- I've just created a gmail account we can use, and moved the message to it (login details coming in mail).

  After moving it I realised that was probably a dumb idea: if it's deleted from my Dreamhost IMAP server, we might lose the symptoms entirely. Well, it is deleted from Dreamhost (at least, I don't see it in their webmail client any more), but MailMate still crashes complaining about the same file; I'm presuming that's because of the header file it already downloaded (which presumably no longer corresponds to anything on the server, although I don't know IMAP details: maybe there's a "deleted" flag and it's still there until some cleanup happens?).

• 

  benny March 12th, 2014 @ 09:52 AM

  That's why I wrote copying :-) I should have emphasized it a bit more.

  Yes, the “deleted flag” theory is valid. But it'll also be a useful data point if MailMate crashes with just the local .emlh file. I'm kind of looking at this problem knowing that it should be obvious (to me), but it keeps escaping me exactly what goes wrong.

• 

  benny March 12th, 2014 @ 10:22 AM

  Long shot: Do you have any antivirus software installed?

• You flagged this item as spam.
  

  Tikitu de Jager March 12th, 2014 @ 10:53 AM

  Oh ha ha bloody ha. I take my hat off to your debugging instincts.

  Yes, I do, and yes, it's "quarantining" that file; the timing makes it very likely indeed that that's the source of the problem.

  Well done!
  Tikitu

• 

  benny March 12th, 2014 @ 11:01 AM

  • State changed from “new” to “resolved”

  Well, I recently had another problem caused by antivirus software. Ticket 417.

  How can antivirus behavior like that not break all kinds of things... I guess I should somehow fail gracefully, but I'm not sure how to best do that.

  Do you know exactly what happens when a file is quarantined? (Antivirus software documentation?)

• You flagged this item as spam.
  

  Tikitu de Jager March 12th, 2014 @ 11:18 AM

  The antivirus in question is Avira. Its docs say "The file is packaged into a special format (*.qua) and moved to the Quarantine directory on your hard disk, so that direct access is no longer possible. Files can be restored, rescanned, deleted or sent to Avira, if necessary." I didn't track down the quarantine dir, but it's consistent with what we've seen that it simply moves the file out of the IMAP dir, leaving nothing in its place. Since that happens after MailMate has downloaded the file and (presumably) updated all its indexes, graceful recovery is gonna be a pain, yes. I'm sorry.

• You flagged this item as spam.
  

  Tikitu de Jager March 24th, 2014 @ 03:53 PM

  Is there any hope of a fix for this issue? I've hit it again, in a more virulent version: I had just enough time to archive the offending message before MailMate crashed, and now it crashes immediately on startup without giving me time to take the mailbox offline. (You'll have seen a few crash reports from me I guess.)

• 

  benny March 25th, 2014 @ 10:14 AM

  You can take the computer offline to make sure MailMate can launch, but I'm not sure how to best fix the problem after that. The best would be putting the file back in place.

  Doesn't the antivirus software have a setting that allows you to force it to ignore MailMate messages? Viruses would be in attachments and those are saved in ~/Library/Application Support/MailMate/Attachments and I don't think MailMate is going to crash if a file is quarantined there (although I'm not 100% sure).

  I understand the idea of this feature of the antivirus software, but I also think it's a really bad idea. It could at least let MailMate hang until the user had decided what to do with the file. Removing files from the private folder of an application is bound to result in problems.

  I'll think about how MailMate could handle this more gracefully, but it's tricky. It would be natural for MailMate to refetch a disappeared message, but that'll probably just result in an infinite loop (another reason I think it's a questionable antivirus feature).

• You flagged this item as spam.
  

  Tikitu de Jager March 25th, 2014 @ 03:17 PM

  Actually, taking the computer offline doesn't prevent the crash. I'll see if I can get Avira to reinstate the file, and stay away from MailMate's attachments folder.

• 

  benny March 25th, 2014 @ 08:42 PM

  Just to emphasise: It needs to stay away from the “Messages” folder (not Attachments).