#3158 new
Stefan Doehla

better usage of multi-core CPUs - bring MM beyond 100% CPU usage ;)

Reported by Stefan Doehla | November 28th, 2022 @ 09:02 AM

Hi Benny,

sometimes tasks in MM can be quite slow because (at least afaiks from the outside) the application is mostly single-threaded (in terms of process threads not mail threads), i.e. most work is done as part of the com.apple.main-thread (as also visible by the max 101% CPU usage).

A recent example is that I borked my filter conditions for "All Messages" (by adding a rule by accident to match john.appleseed). Removing that rule stalls MM (already for two hours somewhere in here:

    2141 Thread_5652   DispatchQueue_1: com.apple.main-thread  (serial)
    + 2141 start  (in dyld) + 520  [0x101e3108c]
    +   2141 main  (in MailMate) + 824  [0x100eed928]
    +     2141 NSApplicationMain  (in AppKit) + 1132  [0x19b6886fc]
    +       2141 -[NSApplication run]  (in AppKit) + 636  [0x19b6b7008]
    +         2141 -[NSApplication _handleEvent:]  (in AppKit) + 76  [0x19baef0f4]
    +           2141 -[NSApplication(NSEvent) sendEvent:]  (in AppKit) + 2780  [0x19b8354b4]
    +             2141 -[OakDeleteKeyWindow sendEvent:]  (in MailMate) + 68  [0x100fc570c]
    +               2141 -[NSWindow(NSEventRouting) sendEvent:]  (in AppKit) + 348  [0x19b83650c]
    +                 2141 -[NSWindow(NSEventRouting) _reallySendEvent:isDelayedEvent:]  (in AppKit) + 2444  [0x19b837100]
    +                   2141 -[NSWindow(NSEventRouting) _handleMouseDownEvent:isDelayedEvent:]  (in AppKit) + 4528  [0x19b8c3d80]
    +                     2141 -[NSControl mouseDown:]  (in AppKit) + 632  [0x19b8c5910]
    +                       2141 -[NSButtonCell trackMouse:inRect:ofView:untilMouseUp:]  (in AppKit) + 740  [0x19b8c66a0]
    +                         2141 -[NSCell trackMouse:inRect:ofView:untilMouseUp:]  (in AppKit) + 160  [0x19b8c682c]
    +                           2141 NSControlTrackMouse  (in AppKit) + 1752  [0x19b8c6f30]
    +                             2141 -[NSButtonCell _sendActionFrom:]  (in AppKit) + 104  [0x19b8ca0e4]
    +                               2141 -[NSCell _sendActionFrom:]  (in AppKit) + 204  [0x19b8ca1c0]
    +                                 2141 __26-[NSCell _sendActionFrom:]_block_invoke  (in AppKit) + 152  [0x19b8ca2cc]
    +                                   2141 -[NSControl sendAction:to:]  (in AppKit) + 96  [0x19b8ca3c4]
    +                                     2141 -[NSApplication(NSResponder) sendAction:to:from:]  (in AppKit) + 460  [0x19b8ca5c8]
    +                                       2141 -[MmGeneralRuleEditor toggledCompoundRow:]  (in MailMate) + 216  [0x100e57804]
    +                                         2141 -[MmFilterRuleEditor didChangeFilter]  (in MailMate) + 264  [0x100e494c8]
    +                                           2141 -[MmConditionsViewController didChangeFilterInEditor:]  (in MailMate) + 220  [0x100e43a28]
    +                                             2141 -[MmMailbox updateSet:filter:]  (in MailMate) + 488  [0x100f010d0]
    +                                               2141 -[MmMailbox update]  (in MailMate) + 20  [0x100f00cf8]
    +                                                 2141 -[MmMailbox updateAlias]  (in MailMate) + 152  [0x100f00be4]
    +                                                   2141 query::manager_implementation_t::set_alias(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)  (in MailMate) + 272  [0x1010bdce4]
    +                                                     2141 query::alias_query_t::set_query(query::query_t*)  (in MailMate) + 800  [0x1010adf70]
    +                                                       2141 query::alias_query_t::dispatch_changed_msg_ids(query::query_t*)  (in MailMate) + 48  [0x1010adbec]
    +                                                         2141 query::query_t::did_change_msg_ids(query::query_t*)  (in MailMate) + 476  [0x1010a2f40]
    +                                                           2141 query::field_query_t::dispatch_changed_msg_ids(query::query_t*)  (in MailMate) + 48  [0x1010a7f90]
    +                                                             2141 query::query_t::did_change_msg_ids(query::query_t*)  (in MailMate) + 456  [0x1010a2f2c]
    +                                                               2141 query::comparison_query_t::compute_changed_msg_ids(query::query_t*)  (in MailMate) + 352  [0x1010a9f74]
    +                                                                 2141 query::comparison_query_t::compare_for_msg_id(int)  (in MailMate) + 116  [0x1010a9700]
    +                                                                   2141 header_oracle_t::apply_no_return(int, std::__1::function<void (header_string_t const&)> const&, header::apply_options_t)  (in MailMate) + 60  [0x10103330c]
    +                                                                     2141 header_oracle_t::apply(int, std::__1::function<bool (header_string_t const&)> const&, header::apply_options_t)  (in MailMate) + 208  [0x101032a3c]
    +                                                                       2136 header_oracle_t::apply(boost::container::flat_set<int, std::__1::less<int>, void> const&, std::__1::function<bool (body_header_string_t const&)> const&, header::apply_options_t)  (in MailMate) + 744  [0x101032e4c]
    +                                                                       ! 1926 specifier_header_oracle_t::prepare_values_for_msg_ids(boost::container::flat_set<int, std::__1::less<int>, void> const&)  (in MailMate) + 1040  [0x101039328]
    +                                                                       ! : 1926 specifier_header_oracle_t::prepare_values_for_msg_ids(boost::container::flat_set<int, std::__1::less<int>, void> const&)  (in MailMate) + 1104,1060,...  [0x101039368,0x10103933c,...]
    +                                                                       ! 198 specifier_header_oracle_t::prepare_values_for_msg_ids(boost::container::flat_set<int, std::__1::less<int>, void> const&)  (in MailMate) + 1296  [0x101039428]
    +                                                                       ! : 188 free_large  (in libsystem_malloc.dylib) + 584  [0x1988ae0c8]
    +                                                                       ! : | 188 madvise  (in libsystem_kernel.dylib) + 8  [0x198a5970c]
    +                                                                       ! : 9 free_large  (in libsystem_malloc.dylib) + 156  [0x1988adf1c]
    +                                                                       ! : | 9 madvise  (in libsystem_kernel.dylib) + 8  [0x198a5970c]
    +                                                                       ! : 1 operator delete(void*)  (in libc++abi.dylib) + 0  [0x198a50944]
    +                                                                       ! 11 specifier_header_oracle_t::prepare_values_for_msg_ids(boost::container::flat_set<int, std::__1::less<int>, void> const&)  (in MailMate) + 1132  [0x101039384]
    +                                                                       ! : 11 _platform_memmove  (in libsystem_platform.dylib) + 88,120  [0x198aab128,0x198aab148]
    +                                                                       ! 1 specifier_header_oracle_t::prepare_values_for_msg_ids(boost::container::flat_set<int, std::__1::less<int>, void> const&)  (in MailMate) + 784  [0x101039228]
    +                                                                       !   1 specifier_header_oracle_t::prepare_values_for_msg_ids(boost::container::flat_set<int, std::__1::less<int>, void> const&)  (in MailMate) + 784  [0x101039228]
    +                                                                       !     1 void std::__1::__sort<header_oracle_t::less_body_part_id_t&, body_header_string_t*>(body_header_string_t*, body_header_string_t*, header_oracle_t::less_body_part_id_t&)  (in MailMate) + 1132  [0x10103d388]
    +                                                                       3 header_oracle_t::apply(boost::container::flat_set<int, std::__1::less<int>, void> const&, std::__1::function<bool (body_header_string_t const&)> const&, header::apply_options_t)  (in MailMate) + 452  [0x101032d28]
    +                                                                       ! 2 body_part_relations_t::all_subpart_ids(int)  (in MailMate) + 424  [0x10102f5cc]
    +                                                                       ! : 2 boost::container::vec_iterator<boost::container::dtl::pair<int, boost::container::flat_set<int, std::__1::less<int>, void> >*, false> boost::container::dtl::flat_tree<boost::container::dtl::pair<int, boost::container::flat_set<int, std::__1::less<int>, void> >, boost::container::dtl::select1st<int>, std::__1::less<int>, boost::container::new_allocator<boost::container::dtl::pair<int, boost::container::flat_set<int, std::__1::less<int>, void> > > >::emplace_hint_unique<std::__1::pair<int, boost::container::flat_set<int, std::__1::less<int>, void> >&>(boost::container::vec_iterator<boost::container::dtl::pair<int, boost::container::flat_set<int, std::__1::less<int>, void> >*, true>, std::__1::pair<int, boost::container::flat_set<int, std::__1::less<int>, void> >&)  (in MailMate) + 56  [0x10103e098]
    +                                                                       ! :   2 boost::container::vector<int, boost::container::new_allocator<int>, void>::vector(boost::container::vector<int, boost::container::new_allocator<int>, void> const&)  (in MailMate) + 0  [0x100ea9014]
    +                                                                       ! 1 body_part_relations_t::all_subpart_ids(int)  (in MailMate) + 316  [0x10102f560]
    +                                                                       !   1 body_part_relations_t::all_subpart_ids(int, std::__1::vector<int, std::__1::allocator<int> >&)  (in MailMate) + 240  [0x10102f268]
    +                                                                       !     1 std::__1::to_string(int)  (in libc++.1.dylib) + 152  [0x1989ed9e4]
    +                                                                       !       1 std::__1::__itoa::__u32toa(unsigned int, char*)  (in libc++.1.dylib) + 0  [0x1989e2c80]
    +                                                                       1 header_oracle_t::apply(boost::container::flat_set<int, std::__1::less<int>, void> const&, std::__1::function<bool (body_header_string_t const&)> const&, header::apply_options_t)  (in MailMate) + 804  [0x101032e88]
    +                                                                       ! 1 header_oracle_t::apply(boost::container::flat_set<int, std::__1::less<int>, void> const&, std::__1::function<bool (body_header_string_t const&)> const&, header::apply_options_t)  (in MailMate) + 804  [0x101032e88]
    +                                                                       !   1 body_part_relations_t::all_subpart_ids(int)  (in MailMate) + 88  [0x10102f47c]
    +                                                                       1 header_oracle_t::apply(boost::container::flat_set<int, std::__1::less<int>, void> const&, std::__1::function<bool (body_header_string_t const&)> const&, header::apply_options_t)  (in MailMate) + 836  [0x101032ea8]
    +                                                                         1 header_oracle_t::apply(boost::container::flat_set<int, std::__1::less<int>, void> const&, std::__1::function<bool (body_header_string_t const&)> const&, header::apply_options_t)  (in MailMate) + 836  [0x101032ea8]
    +                                                                           1 header_oracle_t::private_header_map_equal_range(int)  (in MailMate) + 172  [0x101033170]

). This I think is a homemade problem by me (and I give it two more hours before force-quitting), but it illustrates the issue.

I therefore ask whether some of the tasks could be splitted off and instead "workers" being used so that the UI is not affected and multiple things could be done in parallel. Afair browsers also started with that concept a decade ago to put every tab in a separate thread to detach it from the rest. Things that are pending a result could simply be greyed out or a spinning wheel be added so that a user knows that this part is not yet ready. In general multiple good ways exist to make users aware of things happening in the background while they can continue their work. Also one doesn't have to be shy of creating many many threads, that's not an issue on recent architectures. It just requires proper thread control.

Long story short: I think it would be a big benefit if MM would split more work into worker threads to take advantage of recent Apple CPUs with >8 cores and thereby increase overall speed and responsiveness (and also get beyond 100% CPU usage).

Comments and changes to this ticket

  • Stefan Doehla

    Stefan Doehla November 28th, 2022 @ 12:00 PM

    Just an update to the example: after ~3h (on an M1 Macbook Pro) MM became responsive again for ~1.2M messages covered by the default "All Messages" smart folder.

  • benny

    benny December 2nd, 2022 @ 04:21 PM

    In this particular case, threading wouldn't help much since 3 hours divided by small n is still a very long time. It's mainly a performance issue for large message stores. (It's not really an option to just put the computation in a thread, because the result is needed to continue working with MailMate. That doesn't mean that there couldn't be some way to stop/revert the computation.)

    Now, given that it probably doesn't take 3 hours to launch MailMate, it seems that there might be a performance bug here. Maybe you could send me your Mailboxes.plist file via “Help > Send Feedback” and let me know what kind of condition you enabled/disabled.

    I've also got some notes on making the mailbox editor less dynamic. That is, have some kind of Cancel/Ok buttons to let the user finish changes before committing.

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

Mac OS X email client.

Shared Ticket Bins

People watching this ticket

Pages