Parsing the mbox file manually instead of using Python's built-in
parser allows us to process large files without loading them into
memory all at once.

This fixes a regression introduced by the previous commit where messages no longer fetch the date from the mbox 'From ' line. For messages without a Date header this means we lose information about the delivery date.

Some messages (like gchat logs) don't have message ids and therefore don't save properly. This commit uses the gmail X-GM-THRID if the Message-Id is missing.

The function email.utils.parsedate_tz expects a str, but we were passing bytes. Casting to str fixes an exception in messages where the Date header is missing and the delivery time must be inferred from the mbox header.

Some messages (like chats) don't have a Message-Id mime header, so the message is saved without a primary key.

A previous commit used the thread id in this situation, but the same thread id can be used for multiple messages. This id, which is the message id used by the gmail api, should be unique across all messages.

The docs note: "The policy keyword should always be specified; The default will change to email.policy.default in a future version of Python."

This shouldn't happen in RFC-abiding messages, but raw unicode or other non-ascii content will cause the header parser to return a Header object rather than a str. Improve handling of this case and add a simple unit test.

If the string is invalid, the undecoded string is returned instead.

In some instances tables would be created with the wrong column types if the initial records had unexpected types. This fixes the issue by explicitly creating the table and specifying types.

Using this newer email parsing code enables parsing of attachments and easier parsing of html emails in the future.

This may be more robust than the tree-walking method we were using earlier, and will enable parsing of html email contents in a future commit.

(Only if no text/plain alternative exists)