-
Notifications
You must be signed in to change notification settings - Fork 20
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move to Python 3.11 baseline; increase blocklist import verbosity #63
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I offer one comment as food for thought, but am happy to provisionally approve this for your merge, once considered @eloquence
@@ -31,6 +31,8 @@ readability-lxml = "^0.8.1" | |||
requests = "^2.31.0" | |||
tweepy = "^4.14.0" | |||
"Mastodon.py" = "^1.8.1" | |||
lxml = {extras = ["html-clean"], version = "^5.3.0"} | |||
beautifulsoup4 = "^4.12.3" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm ambivalent about this. If it's not being used by the core script, it feels like this should be codified in the specific implementation that's using it instead. Like, ideally, our internal ttnconfig repo that includes the blocklist should have a separate requirements spec that lists both ttn and bs4. (If someone else wanted to use PyQuery
for example, they'd still have to install that themselves.)
(Looks like I actually removed this requirement in the past for similar reasons heh 64bcb02)
On the other hand, this is a small project that's only used by a handful of people besides us, so I'd accept if it feels like too much bikeshedding to debate this. Probably not a big deal to include.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you think about dropping an abstract base class in this repo that does import bs4? Basically, more of an opinionated plugin-style design pattern intended to minimize the complexity folks may have to wrangle to add something like a basic blocklist.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, sounds reasonable enough!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, I'll poke at this still in this PR, dragging back to WIP until that's done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is now done; I've avoided importing BeautifulSoup in the ABC since it serves no purpose there, but I've annotated pyproject.toml
to make it clearer why it's there.
d4b082d
to
cdb8a3b
Compare
cdb8a3b
to
4320211
Compare
Back to you Harris; if this looks good, we'll need to land freedomofpress/foiafeed-ttnconfig#3 alongside it. |
We're still not using bs4 in this repo, which is what I thought you meant 😅 But I also don't need to overlitigate this one common dependency. It's good, let's merge! |
I did mean that originally, but importing it in an abstract base class and then not using it there will just make the linter unhappy, so I went with the "comment the dep" approach instead. |
html-clean
extrabs4
dependency even though it's used in our blocklist; this won't work in production. Added it to dependencies explicitlyblocklist.py
import as a result of the above was a bit annoying to debug; made the logic a bit more verbose so we can catch such issues more quickly in future.gitignore
to catch all pycache directoriesResolves #62