Skip to content

Latest commit

 

History

History
49 lines (37 loc) · 1.86 KB

README.md

File metadata and controls

49 lines (37 loc) · 1.86 KB
Raw

Instagram.com Scraper

This scraper is using scrapfly.io and Python to scrape user and user post datasets from Instagram.com

Full tutorial https://scrapfly.io/blog/how-to-scrape-instagram/

The scraping code is located in the instagram.py file. It's fully documented and simplified for educational purposes and the example scraper run code can be found in run.py file.

This scraper scrapes:

  • Instagram.com user information
  • Instagram.com user posts

For output examples see the ./results directory.

Note: This scraper is only scraping public Instagram data that doesn't require a login to access.

Fair Use Disclaimer

Note that this code is provided free of charge as is, and Scrapfly does not provide free web scraping support or consultation. For any bugs, see the issue tracker.

Setup and Use

This Instagram.com scraper uses Python 3.10 with scrapfly-sdk package which is used to scrape and parse Instagram's data.

  1. Ensure you have Python 3.10 and poetry Python package manager on your system.
  2. Retrieve your Scrapfly API key from https://scrapfly.io/dashboard and set SCRAPFLY_KEY environment variable: 
    $ export SCRAPFLY_KEY="YOUR SCRAPFLY KEY"
  3. Clone and install Python environment: 
    $ git clone https://github.com/scrapfly/scrapfly-scrapers.git
$ cd scrapfly-scrapers/crunchbase-scraper
$ poetry install
  4. Run example scrape: 
    $ poetry run python run.py
  5. Run tests: 
    $ poetry install --with dev
$ poetry run pytest test.py
# or specific scraping areas
$ poetry run pytest test.py -k test_user_scraping
$ poetry run pytest test.py -k test_post_scraping
$ poetry run pytest test.py -k test_user_post_scraping