BREAKING CHANGE: remove legacy detectron2 model; remove layoutparser extras
#350
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
MthwRobinson
changed the title
detectron2 model; remove layoutparser extras
MthwRobinson
commented
May 22, 2024
| portalocker==2.8.2 | ||
| # via iopath | ||
| protobuf==5.26.1 | ||
| # via | ||
| # onnx | ||
| # onnxruntime | ||
| pycocotools==2.0.7 |
There was a problem hiding this comment.
Here's the dependency we needed to remove
qued
reviewed
May 22, 2024
| @@ -1,10 +1,12 @@ | |||
| -c constraints.in | |||
| layoutparser[layoutmodels,tesseract] | |||
| layoutparser | |||
There was a problem hiding this comment.
Are we not able to lose layoutparser completely?
There was a problem hiding this comment.
We still have this TextBox element we rely on from layoutparser, but I wonder if we can just copy that over
There was a problem hiding this comment.
That class has dependencies from a couple different modules in layoutparser, doesn't look as simpler as copying over a small class. Opened #351 for follow on work to see if we can get rid of TextBlock.
|
@qued - Tests are passing now, should be good for final review! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
First step in resolving Unstructured-IO/unstructured#3051. Per this comment, we were having troubling running
unstructuredin the Python 3.12wolfi-basecontain due to issues related topycocotools, which is only used for the legacydetectron2model fromlayoutparser. Since we've replaced this withdetectron2onnx, this PR removes thelayoutparserextra dependencies that caused issues with Python 3.12.The
layoutparserbase dependency is still required because we use layout objects from that library. It's likely we could remove these in a future iteration.Temporarily disabled the ingest tests, because they seem to have been broken for the past six months. Last commit that they passed for was this one. Opened #352 to reenable them.
Testing
If CI passes we should be good to go.