Skip to content

Latest commit

 

History

History
64 lines (45 loc) · 2.1 KB

README.md

File metadata and controls

64 lines (45 loc) · 2.1 KB
Raw

ksl_automated_scraper

Automatically send an email or Discord message including KSL Classifieds search results for a specified query, matching a price range.

Already sent links will be kept track of via an SQLite database. If no new results are found, no email/message/notification is sent.

This only works on the Classifieds section. For https://cars.ksl.com, see: ksl_cars_scraper

Dependencies:

  • BeautifulSoup4
  • Requests
  • aiosqlite

Optional

  • PyCord (discord2)

Usage

git clone https://github.com/Bay40k/ksl_automated_scraper
cd ksl_automated_scraper
pip install -r requirements.txt

After installing, make sure to rename config_EXAMPLE.py to config.py after configuring your settings.

Then, to run the scrape and send the results to your configured email:

python automated_scrape_email.py

Or, if you have configured Discord settings:

python automated_scrape_discord.py

Example Output

censored_example2

Discord

  1. To use Discord, you must first create a bot account and generate an OAuth URL: https://docs.pycord.dev/en/master/discord.html

  2. Make sure it has "Bot" and "Send messages" permissions, then add it to a server you're in using the OAuth URL you generated above.

  3. Then, copy your bot's token and your account's User ID to config.py

  4. Simply run/schedule python automated_scrape_discord.py.

Scheduling

Cron job which runs scrape every 12 hours

0 */12 * * * python ~/ksl_automated_scraper/automated_scrape_email.py

Or, for Discord (if configured):

0 */12 * * * python ~/ksl_automated_scraper/automated_scrape_discord.py

Or create a scheduled task in Windows: https://www.askpython.com/python/examples/execute-python-windows-task-scheduler

Embedding

View embedding_examples.ipynb for examples of how to embed this script.