Add STARTTLS modes, source-mailbox flag, IMAP IDLE for destination, progress messages, and more error handling

[doransmestad]

First off, @Schluggi this is awesome. I was looking for a way to migrate email from one IMAP server to another IMAP server, and this is exactly what I was looking for! Awesome stuff.

I ran into a couple of problems while using it, mostly because of either large email mailboxes, weird email messages that seemed to be missing important headers, and oddities from the IMAP servers themselves (like a disconnect on a bad append attempt 🤷‍♂️)

So I added some features/fixes! Please see the included code here for:

1. The addition of a --source-mailbox flag to target specific mailboxes to sync from the source to the destination. This allowed me "slow roll" a particularly large sync, as well as debug certain folder issues quickly. It can be specified multiple times to sync any number of source mailboxes.

2. Addition of STARTTLS support. I had to add this in order to sync to a server that only exposed port 143, and also ensure I still used an encrypted connection. If the flag --source-use-starttls and/or --destination-use-starttls is specified a connection will be established then STARTTLS issued prior to any login attempt. If STARTTLS fails, an exit will occur.

3. Addition of IMAP IDLE to the destination connection if --destination-use-idle is added to the command line. This will invoke IMAP IDLE after the initial connection and login check to keep the connection alive until the script is ready to copy emails over.

4. Unfortunately, IMAP IDLE wasn't enough for my case (connection timeout when transferring to Gmail due to a massive source mailbox), so I added a re-connect to the destination after the looping over the source mailbox was completed. It feels a bit hacky, and could be cleaned up a bit by wrapping it in an if-statement I suppose, but it worked perfectly.

5. I also added in a couple more print messages to output status more regularly + tweaked error reporting a bit. Transferring a mailbox of 250k+ emails took a long time so it was helpful to see the folder + email count it was at during the source loop. I also had a couple errors that were tricky to debug and the added logging is for that.

6. I discovered that Gmail aborts the whole connection if an email transfer fails, so I added a re-connect in the error handling code as well. This could be wrapped in an if statement as well (check for connection loss first) I suppose, but it worked for me.

7. Finally - I updated the email copy success check to also looking for a message of "(success)" as that appears to be what Gmail returns instead of the "append completed" that the code originally only looked for.

All in all, a HUGE thank you for writing and open sourcing this, it was really helpful! One drawback I should acknowledge in my PR here is that my edits have made the screen output a bit more verbose.. This was very helpful for me, and hopefully it's helpful for others too, but maybe a log-level flag in the future? 🤔

Let me know what you think of the changes here?

Thanks again.

[Schluggi]

Thank you for your feedback and this pull request.
Your changes look great. I'll merge after some tests 👍

[Schluggi]

Ready to merge

[Schluggi]

Just merged! Huge thanks to all this work and your ideas! Logging will be added in the future (#14).

Have a nice weekend 👍