GeoInterpolations

A couple of weeks ago Ed Vielmetti, organizer of the a2b3 weekly lunch meetings, posted the following:

Subject: DDA meeting, downtown, today; I have 4 minutes on the agenda
To: a2b3@yahoogroups.com

I have 4 minutes on their agenda. Where do you start? Send me suggestions.

1. Sidewalks, and the shoveling of.
2. Buses, in general.
3. Buses, particularly real time bus information a la ridetrak.
4. Real time bus info at stops.
5. Parking structures, and the surplus of; occupancy rates etc.

Only 4 minutes though, and I need to be prompt about it all.

–
Edward Vielmetti
Ann Arbor, MI

After a few replies about the need for real-time bus information (which the DDA doesn’t have), the thread quickly moved to parking structure data (which DDA has available in real-time on their website). There had been some creative use of the data in the past, including a map showing the proportion of free spaces in each structure, an analysis of a month’s worth of data, and an RSS feed of the data.

At the the time of Ed’s meeting, the DDA was exposing the data on a simple html page that contained a single table with the current spaces available for each structure. Within a day, Fred Posner had put together an telephony application that allows people to call a phone number to check the available spaces. Very cool.

I work in the area of geostatistics (among other things) and remote sensing and my initial interest in the dataset was in the temporal and spatial aspects of the data. But since the DDA website only lists the current conditions, the data would have to be scraped from the DDA website and stored for future use. So I set out to create a data system that would harvest and store the data in a format that would support later analyses on the dataset.

I settled on using the python web framework Django (which I regularly use), hosted on Google App Engine (which I was interested in trying out). The Beautiful Soup library was chosen for scraping the parking status data from the DDA web page. Ed pointed me in the direction of IUI which allows iPhone/iPod like navigation in a web page.

Here is how the project progressed:

• 1/13/09 ~3:00pm - the application was up and running, harvesting data from http://www.a2dda.org/parking.php .
• 1/15/09 ~3:00pm - the web page http://www.a2dda.org/parking.php was taken down
• 1/16/09 ~7:00am - I modified the application to scrape data from a second page http://www.a2dda.org/parking__transportation/available_parking_spots/
• 1/16/09 ~5:30pm - the harvesting script was ‘banned’ from accessing the second web page with the parking data

Apparently someone is worried that the parking status data is actually getting used…

I intend to talk to the DDA next week to find out why access was blocked, and hopefully reinstate access.

So at this point, there are a 3 days of time series data for nine garages in the database, and the website has few basic pages that display the most recent data as HTML or a time history as a CSV file. It’s a starting point, upon which additional output formats can be added. Just need to get the data flowing again…

The source code for the project is available at: http://bitbucket.org/tylere/a2-park

Trackback URI | Comments RSS