Implement a rate limit on scaping #608
AndyTheFactory
started this conversation in
Areas in need for your contribution
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
When building sources and/or downloading articles using multiple threads, this could lead to hitting the news server rate limit protections.
It would be nice if there could be a configuration attribute that would limit the max nr of requests per second to the news site.
Beta Was this translation helpful? Give feedback.
All reactions