You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Scraping / Inserting data via Selenium was originally done to make the usage of this synchronisation script easier. Especially since some sites didn't offer an API, it was not free to use, or it was lacking data (e.g. for private profiles).
Since some sites have introduced some quite effective anti-bot measures (e.g. captchas on IMDb), this reduces the usefulness of this script/application.
With this issue, re-implement the most-important APIs to replace the Selenium-based scraping&inserting. The sites where a simple CSV upload/download is performed are out-of scope for the first phase of this.
The user might then have to generate private API credentials (e.g. client ID + secret) - this would need to be documented as a step-by-step how-to guide.
For each of the sites, this should be a feature toggle to let the user chose which way to use.
The text was updated successfully, but these errors were encountered:
Scraping / Inserting data via Selenium was originally done to make the usage of this synchronisation script easier. Especially since some sites didn't offer an API, it was not free to use, or it was lacking data (e.g. for private profiles).
Since some sites have introduced some quite effective anti-bot measures (e.g. captchas on IMDb), this reduces the usefulness of this script/application.
With this issue, re-implement the most-important APIs to replace the Selenium-based scraping&inserting. The sites where a simple CSV upload/download is performed are out-of scope for the first phase of this.
The user might then have to generate private API credentials (e.g. client ID + secret) - this would need to be documented as a step-by-step how-to guide.
For each of the sites, this should be a feature toggle to let the user chose which way to use.
The text was updated successfully, but these errors were encountered: