Freron MailMate

Hi Benny,

sometimes tasks in MM can be quite slow because (at least afaiks from the outside) the application is mostly single-threaded (in terms of process threads not mail threads), i.e. most work is done as part of the com.apple.main-thread (as also visible by the max 101% CPU usage).

A recent example is that I borked my filter conditions for "All Messages" (by adding a rule by accident to match john.appleseed). Removing that rule stalls MM (already for two hours somewhere in here:

    2141 Thread_5652   DispatchQueue_1: com.apple.main-thread  (serial)
    + 2141 start  (in dyld) + 520  [0x101e3108c]
    +   2141 main  (in MailMate) + 824  [0x100eed928]
    +     2141 NSApplicationMain  (in AppKit) + 1132  [0x19b6886fc]
    +       2141 -[NSApplication run]  (in AppKit) + 636  [0x19b6b7008]
    +         2141 -[NSApplication _handleEvent:]  (in AppKit) + 76  [0x19baef0f4]
    +           2141 -[NSApplication(NSEvent) sendEvent:]  (in AppKit) + 2780  [0x19b8354b4]
    +             2141 -[OakDeleteKeyWindow sendEvent:]  (in MailMate) + 68  [0x100fc570c]
    +               2141 -[NSWindow(NSEventRouting) sendEvent:]  (in AppKit) + 348  [0x19b83650c]
    +                 2141 -[NSWindow(NSEventRouting) _reallySendEvent:isDelayedEvent:]  (in AppKit) + 2444  [0x19b837100]
    +                   2141 -[NSWindow(NSEventRouting) _handleMouseDownEvent:isDelayedEvent:]  (in AppKit) + 4528  [0x19b8c3d80]
    +                     2141 -[NSControl mouseDown:]  (in AppKit) + 632  [0x19b8c5910]
    +                       2141 -[NSButtonCell trackMouse:inRect:ofView:untilMouseUp:]  (in AppKit) + 740  [0x19b8c66a0]
    +                         2141 -[NSCell trackMouse:inRect:ofView:untilMouseUp:]  (in AppKit) + 160  [0x19b8c682c]
    +                           2141 NSControlTrackMouse  (in AppKit) + 1752  [0x19b8c6f30]
    +                             2141 -[NSButtonCell _sendActionFrom:]  (in AppKit) + 104  [0x19b8ca0e4]
    +                               2141 -[NSCell _sendActionFrom:]  (in AppKit) + 204  [0x19b8ca1c0]
    +                                 2141 __26-[NSCell _sendActionFrom:]_block_invoke  (in AppKit) + 152  [0x19b8ca2cc]
    +                                   2141 -[NSControl sendAction:to:]  (in AppKit) + 96  [0x19b8ca3c4]
    +                                     2141 -[NSApplication(NSResponder) sendAction:to:from:]  (in AppKit) + 460  [0x19b8ca5c8]
    +                                       2141 -[MmGeneralRuleEditor toggledCompoundRow:]  (in MailMate) + 216  [0x100e57804]
    +                                         2141 -[MmFilterRuleEditor didChangeFilter]  (in MailMate) + 264  [0x100e494c8]
    +                                           2141 -[MmConditionsViewController didChangeFilterInEditor:]  (in MailMate) + 220  [0x100e43a28]
    +                                             2141 -[MmMailbox updateSet:filter:]  (in MailMate) + 488  [0x100f010d0]
    +                                               2141 -[MmMailbox update]  (in MailMate) + 20  [0x100f00cf8]
    +                                                 2141 -[MmMailbox updateAlias]  (in MailMate) + 152  [0x100f00be4]
    +                                                   2141 query::manager_implementation_t::set_alias(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)  (in MailMate) + 272  [0x1010bdce4]
    +                                                     2141 query::alias_query_t::set_query(query::query_t*)  (in MailMate) + 800  [0x1010adf70]
    +                                                       2141 query::alias_query_t::dispatch_changed_msg_ids(query::query_t*)  (in MailMate) + 48  [0x1010adbec]
    +                                                         2141 query::query_t::did_change_msg_ids(query::query_t*)  (in MailMate) + 476  [0x1010a2f40]
    +                                                           2141 query::field_query_t::dispatch_changed_msg_ids(query::query_t*)  (in MailMate) + 48  [0x1010a7f90]
    +                                                             2141 query::query_t::did_change_msg_ids(query::query_t*)  (in MailMate) + 456  [0x1010a2f2c]
    +                                                               2141 query::comparison_query_t::compute_changed_msg_ids(query::query_t*)  (in MailMate) + 352  [0x1010a9f74]
    +                                                                 2141 query::comparison_query_t::compare_for_msg_id(int)  (in MailMate) + 116  [0x1010a9700]
    +                                                                   2141 header_oracle_t::apply_no_return(int, std::__1::function<void (header_string_t const&)> const&, header::apply_options_t)  (in MailMate) + 60  [0x10103330c]
    +                                                                     2141 header_oracle_t::apply(int, std::__1::function<bool (header_string_t const&)> const&, header::apply_options_t)  (in MailMate) + 208  [0x101032a3c]
    +                                                                       2136 header_oracle_t::apply(boost::container::flat_set<int, std::__1::less<int>, void> const&, std::__1::function<bool (body_header_string_t const&)> const&, header::apply_options_t)  (in MailMate) + 744  [0x101032e4c]
    +                                                                       ! 1926 specifier_header_oracle_t::prepare_values_for_msg_ids(boost::container::flat_set<int, std::__1::less<int>, void> const&)  (in MailMate) + 1040  [0x101039328]
    +                                                                       ! : 1926 specifier_header_oracle_t::prepare_values_for_msg_ids(boost::container::flat_set<int, std::__1::less<int>, void> const&)  (in MailMate) + 1104,1060,...  [0x101039368,0x10103933c,...]
    +                                                                       ! 198 specifier_header_oracle_t::prepare_values_for_msg_ids(boost::container::flat_set<int, std::__1::less<int>, void> const&)  (in MailMate) + 1296  [0x101039428]
    +                                                                       ! : 188 free_large  (in libsystem_malloc.dylib) + 584  [0x1988ae0c8]
    +                                                                       ! : | 188 madvise  (in libsystem_kernel.dylib) + 8  [0x198a5970c]
    +                                                                       ! : 9 free_large  (in libsystem_malloc.dylib) + 156  [0x1988adf1c]
    +                                                                       ! : | 9 madvise  (in libsystem_kernel.dylib) + 8  [0x198a5970c]
    +                                                                       ! : 1 operator delete(void*)  (in libc++abi.dylib) + 0  [0x198a50944]
    +                                                                       ! 11 specifier_header_oracle_t::prepare_values_for_msg_ids(boost::container::flat_set<int, std::__1::less<int>, void> const&)  (in MailMate) + 1132  [0x101039384]
    +                                                                       ! : 11 _platform_memmove  (in libsystem_platform.dylib) + 88,120  [0x198aab128,0x198aab148]
    +                                                                       ! 1 specifier_header_oracle_t::prepare_values_for_msg_ids(boost::container::flat_set<int, std::__1::less<int>, void> const&)  (in MailMate) + 784  [0x101039228]
    +                                                                       !   1 specifier_header_oracle_t::prepare_values_for_msg_ids(boost::container::flat_set<int, std::__1::less<int>, void> const&)  (in MailMate) + 784  [0x101039228]
    +                                                                       !     1 void std::__1::__sort<header_oracle_t::less_body_part_id_t&, body_header_string_t*>(body_header_string_t*, body_header_string_t*, header_oracle_t::less_body_part_id_t&)  (in MailMate) + 1132  [0x10103d388]
    +                                                                       3 header_oracle_t::apply(boost::container::flat_set<int, std::__1::less<int>, void> const&, std::__1::function<bool (body_header_string_t const&)> const&, header::apply_options_t)  (in MailMate) + 452  [0x101032d28]
    +                                                                       ! 2 body_part_relations_t::all_subpart_ids(int)  (in MailMate) + 424  [0x10102f5cc]
    +                                                                       ! : 2 boost::container::vec_iterator<boost::container::dtl::pair<int, boost::container::flat_set<int, std::__1::less<int>, void> >*, false> boost::container::dtl::flat_tree<boost::container::dtl::pair<int, boost::container::flat_set<int, std::__1::less<int>, void> >, boost::container::dtl::select1st<int>, std::__1::less<int>, boost::container::new_allocator<boost::container::dtl::pair<int, boost::container::flat_set<int, std::__1::less<int>, void> > > >::emplace_hint_unique<std::__1::pair<int, boost::container::flat_set<int, std::__1::less<int>, void> >&>(boost::container::vec_iterator<boost::container::dtl::pair<int, boost::container::flat_set<int, std::__1::less<int>, void> >*, true>, std::__1::pair<int, boost::container::flat_set<int, std::__1::less<int>, void> >&)  (in MailMate) + 56  [0x10103e098]
    +                                                                       ! :   2 boost::container::vector<int, boost::container::new_allocator<int>, void>::vector(boost::container::vector<int, boost::container::new_allocator<int>, void> const&)  (in MailMate) + 0  [0x100ea9014]
    +                                                                       ! 1 body_part_relations_t::all_subpart_ids(int)  (in MailMate) + 316  [0x10102f560]
    +                                                                       !   1 body_part_relations_t::all_subpart_ids(int, std::__1::vector<int, std::__1::allocator<int> >&)  (in MailMate) + 240  [0x10102f268]
    +                                                                       !     1 std::__1::to_string(int)  (in libc++.1.dylib) + 152  [0x1989ed9e4]
    +                                                                       !       1 std::__1::__itoa::__u32toa(unsigned int, char*)  (in libc++.1.dylib) + 0  [0x1989e2c80]
    +                                                                       1 header_oracle_t::apply(boost::container::flat_set<int, std::__1::less<int>, void> const&, std::__1::function<bool (body_header_string_t const&)> const&, header::apply_options_t)  (in MailMate) + 804  [0x101032e88]
    +                                                                       ! 1 header_oracle_t::apply(boost::container::flat_set<int, std::__1::less<int>, void> const&, std::__1::function<bool (body_header_string_t const&)> const&, header::apply_options_t)  (in MailMate) + 804  [0x101032e88]
    +                                                                       !   1 body_part_relations_t::all_subpart_ids(int)  (in MailMate) + 88  [0x10102f47c]
    +                                                                       1 header_oracle_t::apply(boost::container::flat_set<int, std::__1::less<int>, void> const&, std::__1::function<bool (body_header_string_t const&)> const&, header::apply_options_t)  (in MailMate) + 836  [0x101032ea8]
    +                                                                         1 header_oracle_t::apply(boost::container::flat_set<int, std::__1::less<int>, void> const&, std::__1::function<bool (body_header_string_t const&)> const&, header::apply_options_t)  (in MailMate) + 836  [0x101032ea8]
    +                                                                           1 header_oracle_t::private_header_map_equal_range(int)  (in MailMate) + 172  [0x101033170]

). This I think is a homemade problem by me (and I give it two more hours before force-quitting), but it illustrates the issue.

I therefore ask whether some of the tasks could be splitted off and instead "workers" being used so that the UI is not affected and multiple things could be done in parallel. Afair browsers also started with that concept a decade ago to put every tab in a separate thread to detach it from the rest. Things that are pending a result could simply be greyed out or a spinning wheel be added so that a user knows that this part is not yet ready. In general multiple good ways exist to make users aware of things happening in the background while they can continue their work. Also one doesn't have to be shy of creating many many threads, that's not an issue on recent architectures. It just requires proper thread control.

Long story short: I think it would be a big benefit if MM would split more work into worker threads to take advantage of recent Apple CPUs with >8 cores and thereby increase overall speed and responsiveness (and also get beyond 100% CPU usage).