Skip to content

L-Steinmacher/basic_web_scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Basic Web Scraper

A basic web scraper for DiverseIo

Set up Project

  1. Within the root of the repo, run python -m venv "env name" to create your own virtual environment
  2. Activate the VE using source "env name"/bin/activate , then run pip install -r requirements.txt . This will give you all of the dependencies listed in the requirements file.

Running the Script

  1. This script must be run with ./scraper.py and requires a url after the command to scrape.
  2. Script has multiple flags for different output. Check out the help by running ./scraper.py -h for a list of flags.

About

A basic web scraper for DiverseIo

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages