Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Filenames from libpff #145

Closed
wants to merge 42 commits into from
Closed

Filenames from libpff #145

wants to merge 42 commits into from

Conversation

gwiedeman
Copy link
Collaborator

Type of Contribution

  • Bugfix (non-breaking change which fixes an issue)
  • New component
  • Refactoring (no functional changes)
  • Documentation-only

What does this implement/fix? Explain your changes.

Previously we could only read attachment filenames with the get_filenames() method which has yet to be merged into libpff.

Now this reads filenames from hex entries which works with libpff.

Link to issue?

n/a

  • Issue closed
  • Remain open

Pull Request Checklist

Please check if your PR fulfills the following requirements:

  • Make sure you are requesting to the develop branch. Don't PR to main!
  • This contribution has sufficient documentation
  • Tests for the changes have been added
  • All tests pass

How has this been tested?

Operating System: win10
Python Version: 3.7.3

Licensing

  • I agree that the Mailbag Project and the University at Albany, SUNY can release this code under the MIT license.

baibhav and others added 30 commits April 5, 2022 05:11
Ignore project setting files
4621ca1
Attachment model added to store list of attachments
b2447ac
MimeType of attachments added to model for eml,mbox,msg, and pst formats
c1e560d
Walk extract attachment replaces iter_attachment for mbox and eml
3375ab1
@haritgarg
helper method for html formatting and pdf derivative
8f3e9b4
@gwiedeman
merged with develop
4ca0c7d
@gwiedeman
make parsing EML and MBOX parts into a helper, also made error handli…
92a2fe1 
…ng into a helper. Minor attachement fixes as well
@gwiedeman
also caught this old error code which would have failed
ed6f2d6
@gwiedeman
cleaned up encoding and variable names and removed some excess code
f26cb50
@gwiedeman
changed sample data
698db66
@gwiedeman
msg and eml tests now running, object compare still not working
66c4d12
@gwiedeman
added dump and read methods to model and script to dump messages
b739fb9
@gwiedeman
worked on tests, they pass now but don't compare the header and messa…
cfa08d9 
…ge objects
@gwiedeman
now tests headers too
bbcb486
@gwiedeman
removed test load file
e2aa7b4
@gwiedeman
refactored pickle load to hopefully prevent truncating
d9b3c37
@gwiedeman
dumping the data during tests to see if that fixes the issue
94620a0
@haritgarg
--
1b16a10
@haritgarg
fixed encoding issue, added meta tag with beautiful soup,
ba98fc1 
adding table to html when pdf derivative is parsed
@gwiedeman
refactored a big and adjusted headers table to be more usable
65cee19
@gwiedeman
merged with develop
7c70ec5
@gwiedeman
dump now writes LF endings and .gitattributes changed to hopefully no…
d8b72be 
…t corrupt same data files
@gwiedeman
hopefully this will leave binary dump files as-is
8c83e57
@gwiedeman
dump files now have appropriate extentsions and git should stop messi…
9e76e90 
…ng with line endings in sample data
@gwiedeman
tests now match correct message with dump file and correct attachments
c7985bf
@gwiedeman
changed how the build installs dependancies which should make tests work
4411a76
@gwiedeman
dump files should be the same across os platforms now
f70692f
@gwiedeman
merged with develop
1ba91ee
@gwiedeman
fixed bad merge and handled some bugs
20fae34
@gwiedeman
merged with develop/pdf derivatives
1dba500
@gwiedeman gwiedeman requested a review from baibhavr April 22, 2022 21:13
@gwiedeman gwiedeman added the Input Parsing input data, such as MBOX, IMAP, PST, EML, etc. label Apr 22, 2022
@gwiedeman gwiedeman added this to the early release milestone Apr 22, 2022
baibhavr
baibhavr reviewed Apr 28, 2022
View reviewed changes
Copy link
Collaborator

@baibhavr baibhavr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than the comment mentioned, I did not come across other issues.

Dockerfile
# Install Python dependancies
RUN pip install -r requirements.txt
# Install Optional Python dependancies
RUN pip install libpff-python-ratom==20200808
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we still keeping the ratom libpff? I saw ratom in pipfiles too.
I didn't come across other issues.

@gwiedeman gwiedeman mentioned this pull request Apr 28, 2022
Merged
11 tasks
@gwiedeman
Copy link
Collaborator Author

Closing as #150 addresses this and more.

@gwiedeman gwiedeman closed this Apr 28, 2022
@gwiedeman gwiedeman deleted the filenames_from_libpff branch May 5, 2022 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@baibhavr baibhavr baibhavr left review comments

Assignees
No one assigned
Labels
Input Parsing input data, such as MBOX, IMAP, PST, EML, etc.
Projects
Mailbag tool development
Status: Done, merged to develop
Milestone
early release
Development

Successfully merging this pull request may close these issues.
3 participants
@gwiedeman @baibhavr @haritgarg