Scrape patco elevator status page and create new endpoint (like septa's) and apply results to map (storing outage history etc)

[jamestyack]

No description provided.

[josomo]

I'll try this. I need to scrape (http://www.ridepatco.org/schedules/alerts.asp), store the statuses in the mongo db, and update the 17 stations on the red line on the mapbox map, right?

[jamestyack]

Awesome that you want to help!

I don't think you need to store the result of the scrape in Mongo (for now), but that could be an option if we want to cache once we get it working...

Here's what I was thinking...

• Take a look at the response we're getting back from SEPTA... would be good to create a function that returns JSON in a similar format except it's the result of scraping the PATCO page (instead of reaching out to the SEPTA API) using some Ruby gem to do the scraping .. .this is the septa response http://www3.septa.org/hackathon/elevator/ (I've stored actual examples at https://github.com/UnlockedLab/unlockphilly/tree/master/data/elevator_outage_json_examples because there isn't always an outage.. .like right now! )
• Unlike the SEPTA response, which doesn't contain IDs, it would be good to use the station IDs in the response for easy matching (you can see the IDs in the Mongo DB data I stored at https://github.com/UnlockedLab/unlockphilly/blob/master/datasets/combined_lines/bus_el_patco_stops_with_lines.csv
• Add the new function ( something like getPatcoElevatorStatusJson() ) to https://github.com/UnlockedLab/unlockphilly/blob/master/restWebService.rb and then share... we can then do the next stage of updating the map based on the result of the function - go ahead and tackle this however you like and reach out to me any time with questions.

Will you be at CodeForPhilly next week? We can catch up, and I can help with any setup you need.

[josomo]

Ok, I'll see what I can do. Not sure if I'll be at the next meeting but I'm around if you wanna chat on IRC or email or whatever.

Should I add escalator outage information also, or just elevator status?

[jamestyack]

Interesting article ... http://articles.philly.com/2014-01-15/news/46188785_1_tim-ireland-patco-officials-patco-passengers

[jamestyack]

Next steps:-

• Refactor/use the new function from the get '/septa/stations/line/:line' function so that as well as calling SEPTA, it also calls the PATCO JSON and merges results together into the response (and also stores the outage data for PATCO to the outageTrackerColl and outagesByDayColl
• Update the API endpoint I temporarily added (/septa/elevator/outages), so that it uses the new function, but has same response format as /septa/elevator/outages (this endpoint is used by ajax call to populate the info box visible when the map first loads)

(if you prefer a different approach, that's cool too.... the one I'm proposing should mean that the stations are updated on the UI.. .the only change on the UI side I think will be to change the way the info box gets populated to add a call to the PATCO endpoint and populate a new PATCO section/links)

[josomo]

Ok, I'll give it a shot. I need to tweak the update time too (it returns the time of the scrape, not the time the statuses were updated).

Wont be able to make the meeting this week.

[jamestyack]

Sorry to miss you; hack4access this weekend.

[josomo]

Still working on this... I fixed the "time updated" problem but I'm still working on the other changes.

[josomo]

Hey James, I see that you're planning to attend the Philly.rb meetup. I will be there maybe we can catch up.

[jamestyack]

Cool, I'll see you later

[jamestyack]

Sorry I missed you; couldn't make it.

[josomo]

No prob. I had a good conversation about mongodb at the meetup so I'm going to take another shot at the issue this weekend.

On Jun 11, 2014, at 8:59 PM, jamestyack [email protected] wrote:

Sorry I missed you; couldn't make it.

—
Reply to this email directly or view it on GitHub.

[jamestyack]

Excellent thanks! BTW, I'm doing some cleanup on the script that loads the stations to Mongo on my local machine and will push into the repo.

[josomo]

Hey James. My email is [email protected]. Maybe it's better to communicate that way.