Scrap The Web Collection; blog posts.
This repo contains Scrapy sample code to scrap the following kind of websites:
- Do you want to learn Scrapy? ScrapScrapy is gonna be your first scrapy project in that case.
- If you want to scrap a simple website without any javascript or AJAX calls,then you should have a look at this project. This uses CrawlSpider.
- If you want to use selenium with scrapy, then look at this project.
- You can refer this project, if you want to save to Django DB as you scrap up the things.