Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider switching over the Regex calculations (or any other CPU intensive tasks) to worker threads #5

Open
JaneJeon opened this issue Sep 15, 2021 · 0 comments
Labels
help wanted Extra attention is needed

Comments

@JaneJeon
Copy link
Collaborator

JaneJeon commented Sep 15, 2021

https://nodejs.org/api/worker_threads.html

But first, we need to bench & profile our existing code to make sure we're not prematurely optimizing, and to figure out what other parts block the loop with intensive computation.

For now, the two obvious culprits look to be the numerous regex matches on strip-trackers, and the parsing & matching of HTML in cheerio in canonicization. Both are currently heavily cached.

@JaneJeon JaneJeon added the help wanted Extra attention is needed label Sep 15, 2021
@JaneJeon JaneJeon assigned JaneJeon and unassigned JaneJeon Sep 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant