#2736 ✓fixcommitted
Eric

Freeze on launch (beachball); unable to rebuild database

Reported by Eric | January 20th, 2021 @ 02:08 PM

Hi,

I have a fairly complex MailMate config, with 23 email accounts, a number of rules, hundreds of thousands of email (about 200,000 just in unread but important transaction log emails that I need to be able to search), etc.

After the last Mailmate update, I started having problems with MailMate revving up and using up a lot of RAM (4-5Gb) and processor power. I reinstalled my system and MailMate in case there was something fishy. Shortly thereafter, I had an unrelated system freeze.

Since then, my Mailmate has been systematically freezing on launch (eternal beachball, 100% processor usage, 1.5GB memory usage) and wjile I see my mail browsing window, it is non-responsive.

I went throught the usual suspects, reinstalled MacOS, ran Onyx and rebuilt caches and all the generic elements I could think of, but am unable to rebuild the MailMate database.

Specifically, 'rm ~/Library/Application\ Support/MailMate/*.tmp' results in: 'No such file or directory'.

I can't find anything significant in Console (there is activity from MailMate) but then again, I may not know what I'm looking for.

As you can imagine, with 23 accounts and multiple rules, I don't want to reconfigure Mailmate from scratch.

My basic specs:
MacBook Pro (15-inch, 2018), 2,2 GHz i7, 32Gb RAM, Radeon Pro 560X 4 Go, MacOS 10.14.6, MailMate 1.13.2

Any clues or hints?

Thanks!

Comments and changes to this ticket

  • benny

    benny January 21st, 2021 @ 07:49 PM

    I'll do what I can to help.

    Try launching MailMate from a Terminal window as follows and let me know if you get any output:

    /Applications/MailMate.app/Contents/MacOS/MailMate
    

    Which revision of MailMate are you using?

  • Eric

    Eric January 21st, 2021 @ 09:37 PM

    Hi Benny,

    Thanks for the quick response. Also, I feel a bit remiss in listing all my problems above without also saying how much I'm thankful for MailMate.
    It's been an incredible help in my daily work, especially the advanced search functions.

    • My memory usage seems high but is it expected? I don't really have comparison points.
    • How can I tell the revision from outside the App? I can't launch so I can't look from within the active app. I usually keep Mailmate on currently supported app.
    • Below is the output in Terminal (what slowly appears in the first 10 minutes or so at least) :
    Problems while parsing reported for one or more messages. Details can be found in /tmp/mailmate_logs/mailmate_parser_problems.log
    Refusing to save data for header containing a '/': x-aaid/cid
    IMAP-related error (1014): Failed to fetch data for 6 messages.
    Warning: No flags headers exists for message 2263090 (assuming empty flags)
    Refusing to save data for header containing a '/': x-aaid/cid
    

    If you need it, I'm going to have to clean up < mailmate_parser_problems.log > a bit before sending because it does contain private information.

    Cheers.

  • benny

    benny January 22nd, 2021 @ 02:28 PM

    No reason to feel thankful for something that doesn't work :)

    • The memory usage does seem excessive.
    • This will work for the revision (but see further below):

      defaults read /Applications/MailMate.app/Contents/Info.plist CFBundleVersion
      
    • The output does not seem related. I don't need the parser problems file.

    Try this cutting edge release of MailMate: http://updates.mailmate-app.com/x86_64/archives/MailMate_r5761.tbz

    If it behaves the same way then try running the the following command while MailMate is hanging:

    sample MailMate 10 > ~/Desktop/mailmate_sample.txt
    

    You cannot use MailMate to send the file, but maybe you have another email client configured. Send it to, e.g., mm-feedback at my domain @freron.com.

  • Eric

    Eric January 22nd, 2021 @ 04:50 PM

    Hi,

    Well then, let's just say I'm filled with optimism that this will work. ;-)

    Current revision is: 5673

    Mailmate cutting edge version 5761 launches, but will not let me authenticate email accounts in-browser anymore yet, still have to reboot and test various options.

  • benny

    benny January 22nd, 2021 @ 05:20 PM

    What happens when you try to authenticate?

    Maybe try to delete the token entries saved in Keychain Access to make sure authentication starts from scratch (just do it for 1 account). These entries are named com.freron.MailMate.*.oauth-*-token.

  • benny

    benny January 22nd, 2021 @ 05:20 PM

    (The code related to authentication has recently changed which doesn't make it unlikely that something isn't working as well as it could.)

  • Eric

    Eric January 22nd, 2021 @ 06:49 PM

    I only have Google token entries in Keychain (in the com.freron.MailMate.* format you specify), so nothing to remove.

    Only the Office365 accounts are now offline and all require re-authentication.

    As to effect, well, nothing happens. I get the normal dialog prompt to "Open Browser Window", click on the button and... nothing happens.
    I tried rebooting, rebooting in Safe Mode, manually opening browser, changing default browser, logging in manually to Office365 in the browser, etc. with no success.

    Other ideas?

    Thanks.

  • benny

    benny January 22nd, 2021 @ 08:40 PM

    Sounds like you had password based access to these accounts (before updating MailMate). You can go into the IMAP account editor and disable OAuth2 to go back to that.

    But I'd also like to learn more about the non-reacting behavior of the Open Browser button. Try launching MailMate from a Terminal window after enabling a hidden preference:

    defaults write com.freron.MailMate MmDebugXOAUTH2 -bool YES
    /Applications/MailMate.app/Contents/MacOS/MailMate
    

    Try the button and send me the output.

    (I'll also verify again that it works for me in case I recently broke something.)

  • benny

    benny January 22nd, 2021 @ 08:43 PM

    Oh, it does fail for me, but it seems something changed at Microsoft breaking some recent changes to MailMate. It might not be an easy fix. I'll need time to look into this. Sorry about the inconvenience. (Maybe the password approach will work for you.)

    You can also try, for example, r5726 to see if that will authenticate and then return to the latest test release if/when it works: https://updates.mailmate-app.com/archives/

  • benny

    benny January 22nd, 2021 @ 08:52 PM

    Ok, the good news is that I know what the problem is. I already worked around it once. It's a Microsoft issue where they generate invalid JSON for automatic configuration of the OAuth2 process. The framework I'm using for OAuth2 doesn't like that. Somehow my workaround for this isn't in effect in my latest builds.

    I can fix it, but I don't have time right now. I'll let you know when an update is ready. (I think it worked in r5757, but that might be wrong.)

  • Eric

    Eric January 22nd, 2021 @ 09:53 PM

    Thanks Benny,

    Here's the output with the ineffective button (repeated in similar fashion for every account, please tell me if you need more).

    Updating access token for me@myOffice365domain.com (force refresh: 0).
    retrieve_token: me@myOffice365domain.com, com.freron.MailMate.Office365.oauth-refresh-token
    SecKeychainFindGenericPassword failed with error: -25300
     result: 
    Get new refresh token (asynchronously).
    2021-01-22 16:45:55.039 MailMate[7831:643382] Open browser
    2021-01-22 16:45:55.039 MailMate[7831:643382]  kIssuer: https://login.microsoftonline.com/common/v2.0
    2021-01-22 16:45:55.039 MailMate[7831:643382]  kClientID: 2316c52f-c664-4efb-93c2-830b1ba5f128
    2021-01-22 16:45:55.039 MailMate[7831:643382]  kClientSecret: (null)
    2021-01-22 16:45:55.040 MailMate[7831:643382] Listening on http://127.0.0.1:57838/
    2021-01-22 16:45:55.040 MailMate[7831:643382] Fetching configuration for issuer: https://login.microsoftonline.com/common/v2.0
    2021-01-22 16:45:55.040 MailMate[7831:643382] Waiting for result
    2021-01-22 16:45:55.115 MailMate[7831:643382] Configuration not fetched (JSON error parsing document at 'https://login.microsoftonline.com/common/v2.0/.well-known/openid-configuration': Invalid URL: issuer)
    

    Installing 5726 does allow to authenticate to most offline accounts (in a captive window style login), but I have to restart MailMate between each login as it seems to try and apply only one authentication for all remaining offline accounts if I don't do that.

    I have one last Office365 account which will not authenticate. It's the only one that requires 2FA authentication (usually through the Microsoft mobile App). In the captive windows it accepts my login, then accepts my acknowledgment in the mobile app, but finally fails to authenticate because it asks for my administrator access (which I do not have). So I cannot get this account online.

    Almost there!

  • Eric

    Eric January 22nd, 2021 @ 10:10 PM

    I can also confirm that I restarted r5761 and it inherits all the (valid) authentications from r5726.

  • Eric

    Eric January 23rd, 2021 @ 12:33 AM

    I will add a perhaps unrelated comment which I can put elsewhere if relevant:

    Something in the steps you asked me to go through today had a positive effect: on top of having a smaller memory footprint (1-2.5 Gb), searches are now also significantly faster.

    A new negative effect however is that refreshing windows when I select different folders is now slower.

  • benny

    benny January 23rd, 2021 @ 09:43 AM

    • State changed from “new” to “fixcommitted”

    The next update will fix the authentication issue. (I'll mark the ticket as fixcommitted even though it kind of deals with multiple issues :) ).

    You mean switching mailboxes is now slower?

    Or maybe that message display is slower (e.g, for large emails?)?

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

Mac OS X email client.

Shared Ticket Bins

People watching this ticket

Pages