Freron MailMate

I decided to use MailMate as a mailstore for many years of old emails. I used Mailchemy to combine various scattered mbox files and eliminate duplicate emails. The result is a huge repository of emails (950,000 emails in a 10GB file). Mailchemy provides an IMAP server implementation in the event that the client cannot import files directly.

I readily admit that I'm doing some guessing here - I've been wrong before!

All that being said, the process works very well when MailMate connects to the server (via localhost) but appears to be impatient when interrogating the server! In the initial discovery process some iterations are called for but are not successful. My belief is that MailMate is timing out because it doesn't get the IMAP OK soon enough. Though it is getting a stream of incoming data, that OK (which is behind all that data) is what it is really waiting for. Here's some log:

12:11:10 S: * 16715 FETCH (UID 16715 INTERNALDATE "10-Nov-2010 17:26:42 -0500")
12:11:10 S: * 16716 FETCH (UID 16716 INTERNALDATE "28-Sep-2010 10:48:53 -0400")
12:11:10 S: * 16717 FETCH (UID 16717 INTERNALDATE "27-Sep-2010 10:50:04 -0400")
12:11:10 S: * 16718 FETCH (UID 16718 INTERNALDATE "21-Oct-2010 08:14:16 -0400")
12:14:05 S: * 16719 FETCH (UID 16719 INTERNALDATE 
12:14:05 Error code: 9
12:14:05 New timeout values (157/8): 262/81
12:14:05 Failed action (1003). Reset observed read/write timeouts: 131/8

12:14:05 Handling reply
12:14:05 Error: Failed multiple retries (4). Final error code was 9.
12:14:05 Terminating non-running connection...
12:16:05 Running action
12:16:05 Handling request
12:16:05 Trying to disconnect nicely (30)...
12:16:05 C: R7 LOGOUT
12:16:35 S: 
12:16:35 Clearing connection to localhost
12:16:35 Ready to run action (type: 3, retry count: 0)
12:16:35 Disconnecting
12:16:35 Clearing connection to localhost
12:16:35 Completed action (3). Observed read/write timeouts: 131/8

It looks like MailMate increases the wait time after timeouts but maybe not enough for this perverse use case.

While I was typing this, the above process transferred a few thousand messages (that's happening on another Mac, with no screen, dedicated to this task which may take days) so maybe I'm the entity that should be patient! Anyway, I'm going to leave this ticket here in case some comments on it, or, suggests a better method.

PS: IMAP doesn't need to gather that initial info about the what the server is holding as one massive transaction .. it could ask for smaller chunks (10k?) when faced with a deluge like mine.