google-scholar-scraper

Scraping documentation on Google Scholar. The key feature is to download the pdf files directly to simplify the search of documentation.

This was designed for personal purpose.

Using :

• python 3
• selenium (firefox) installation link
• langid python library

Pull this git repo and then:

git submodule update --init --recursive

Note: be sure to use the version 74 of firefox or change the geckodriver by visiting the selenium website.

Usage

./search.py <QUERY> <LIMIT>

<QUERY> is required.

<LIMIT> is optional.

Contribution

You can ask or add features. Feel free to report errors here in the issue https://github.com/lucgerrits/google-scholar-scraper/issues.

TODO

• add argument parser
• Security
• Some improvements