Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hope that users can be given the option to more precisely control crawling behavior #21

Open
1 of 2 tasks
nonhana opened this issue Jul 17, 2024 · 2 comments
Open
1 of 2 tasks
Assignees
Labels
enhancement New feature or request question Further information is requested

Comments

@nonhana
Copy link
Contributor

nonhana commented Jul 17, 2024

  • It is hoped that when crawling works, information such as the title of the work, the name of the author, and the avatar can be obtained at the same time.
  • Some users using an external network environment may be given the option to control whether to enable the proxy.
@CWHer CWHer self-assigned this Jul 18, 2024
@CWHer CWHer added enhancement New feature or request question Further information is requested labels Jul 18, 2024
@CWHer
Copy link
Owner

CWHer commented Jul 19, 2024

Thanks for opening this issue and sorry for the delayed reply.

You are welcome to drop a PR if you would like to help enhance it. Besides, it would be better to briefly introduce your PR design here before hands-on it.

@CWHer
Copy link
Owner

CWHer commented Jul 19, 2024

As for these two issues, existing APIs can partially resolve them.

  1. By enabling download_config.with_tag, tags of artworks, including avatar description, will be collected. However, it seems the title and author information are missing. I believe expanding with_tag option into with_metadata option could be a elegant solution.

  2. Proxy can be disabled by setting network_config.proxy["https"] = "", but this still lacks some flexibility for more complicated scenarios.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants