Initial build of IQM2 scraper

[krammy19]

We need to design a web scraper for the IQM2 agenda hosting platform. Examples of cities that use this software including Arcata, Barstow, and Brentwood

Input: a path endpoint or a batch .csv file of endpoints to scrape. This url should point to the specific page on a city website where agendas are listed out for public review.

Output: a .csv table with the following column data for each agenda listed on this page:

{Column} | {Description}
Index | Autoincrement index
City | City or agency name
Meeting Name | Title of government body
Date / Time | Date / time of meeting
Agenda | URL of agenda pdf
Meeting video | URL of meeting video (if available)
Published minutes | URL of minutes pdf (if available)

For reference and examples, please see this scraping walkthrough for Legistar.

As you build this scraper, keep in mind that we will need to eventually add additional features, including:

• Accessing past agendas not included on the city’s main page, e.g. past years.

• Filtering agenda scraping by Date range

• Filtering agenda scraping by Meeting Name

• Downloading all agendas from scraped urls into a specified directory

• Scraping staff report urls from scraped agendas.