MUSA-620-Week-6: Web Scraping Part 1

Links:

• HTML vs DOM, Example 1
• HTML vs DOM, Example 2
• rvest package for parsing HTML

Getting to Know Your Browser's Web Inspector

Elements tab

Use the elements tab to explore a webpage's DOM.

Examples:

• Simple example
• Grab text from sites that have copying disabled: NY Times
• Find and download a hidden video file: Imgur

Network tab

Use the network tab to find the data used in these webpages.

FiveThirtyEight: Congressional Districts

• Congressional district map

WSJ: California Drought

• CA outline
• Drought isocrones
• Major cities
• Population affected

Data: Wall Street Journal / code: web-inspector-data.r

Washington Post: US Sunlight Map

• See if you can you can find the data to recreate their sunlight map yourself. The steps for doing this have now been added to web-inspector-data.r

Data: Washington Post / code: web-inspector-data.r

The Economist: Property Values by US County

• Sometimes data will be hidden in .js files: county-level real estate values
• Used as the data source for the map below

Total residential property value of each US county