tag:blogger.com,1999:blog-59280265358745407052024-03-27T23:14:33.631-07:00Metro Chicago Open Data ExamplesUnknownnoreply@blogger.comBlogger31125tag:blogger.com,1999:blog-5928026535874540705.post-44314876851561862472013-01-24T13:34:00.001-08:002013-01-24T13:34:23.737-08:00SODA V2 - There's a new APIIn October, Socrata released a new version of the API, SODA V2. It's a major change and a great improvement over V1. As a result, all of the examples and discussion of the API in the previous posts are now out of date. They should continue to work for some time but eventually the old API will be discontinued and the examples will no longer work.<br />
<br />
I don't know when, or if, I'll have time to create a new set of examples with the new API.<br />
<br />
You can read about the API and how to get started with it at the Socrata's, <a href="http://dev.socrata.com/consumers/getting-started">Getting Started</a>, page.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-57333617377509141012011-09-26T08:59:00.000-07:002011-09-30T18:01:59.086-07:00Other open data sourcesIn addition to the data portals for the city of Chicago, Cook County, and state of Illinois, metro Chicago has a couple other places with open data:<br />
<br />
<ul>
<li><a href="http://www.cmap.illinois.gov/">CMAP</a> - Chicago Metropolitan Agency for Planning has an API to access thousands of data sets. Start at their <a href="http://data.cmap.illinois.gov/ApiHome.html">Data API</a> page to read about it.</li>
<li>Chicago Tribune - has worked with others to make U.S. Census data available. You can find datasets for Chicago census data at the <a href="http://census.ire.org/">Investigative Reporters and Editors</a> site.</li>
</ul>
Unknownnoreply@blogger.com6tag:blogger.com,1999:blog-5928026535874540705.post-66596036778630686222011-09-26T08:35:00.000-07:002011-09-26T08:36:34.607-07:00Code and reference card are on githubAll the code found used in the examples is now on github under <a href="https://github.com/snewell4/Metro-Chicago-Open-Data-Examples"><span style="font-family: "Courier New",Courier,monospace;">snewell4/Metro Chicago Open Data Examples</span></a>. In addition, a PDF of the views API reference card from Example #10 is included.<br />
<br />
The github README explains it:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">9/26/11</span><br />
<span style="font-family: "Courier New",Courier,monospace;">These files accompany the Metro Chicago Open Data Examples blog. They are</span><br />
<span style="font-family: "Courier New",Courier,monospace;">simple examples of using the Socrata Open Data API (SODA) with the Socrata data</span><br />
<span style="font-family: "Courier New",Courier,monospace;">portals in use at City of Chicago and Cook County.</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">The blog has a series of examples each intended to explain the API and</span><br />
<span style="font-family: "Courier New",Courier,monospace;">different types of uses. The blog entries are titled "Example #n". The</span><br />
<span style="font-family: "Courier New",Courier,monospace;">filenames use the same naming convention. You can read about the purpose</span><br />
<span style="font-family: "Courier New",Courier,monospace;">and the explanation of the code in the blog.</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">There are two exceptions to the naming convention:</span><br />
<span style="font-family: "Courier New",Courier,monospace;">1) The blog entry for Example #20 describes three command line utilities.</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> Those utilities are named getcolumns.py, getquery.py, and getview.py. </span><br />
<span style="font-family: "Courier New",Courier,monospace;">2) Example #10 includes a one-page overview of the view API. A copy of that</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> overview has the filename, view-ref-card.pdf</span>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-73405974599062973642011-09-25T17:59:00.000-07:002011-09-26T05:30:40.965-07:00Example #23 - Creating a simple web appLet's take the code from Example 19 and turn it into a web app. This will be a trivial example that can be the starting point for building a full web application.<br />
<br />
Example 19 used the command line to run a Python program. One of the program's arguments was a street address. The program ran and displayed a map in your browser of nearby rat sightings. In this example, you can enter the street address into a form on a web page and the same sort of map will be displayed.<br />
<br />
Here's what it looks like. You type in the street address and click "Search".<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8lm8jrRO-AcemmqJsCqjjbzswiKxpHsi6uJD6yu-P3KqibpBXIWKI6elzLC7wGAR4YusF6fWz0pM_B2WILvAsz783C6aGK0-foNSk9AF5V3rHqMVzwtU1PbPsW8WY3B2zJ_pYKLuKjTvi/s1600/Browser1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8lm8jrRO-AcemmqJsCqjjbzswiKxpHsi6uJD6yu-P3KqibpBXIWKI6elzLC7wGAR4YusF6fWz0pM_B2WILvAsz783C6aGK0-foNSk9AF5V3rHqMVzwtU1PbPsW8WY3B2zJ_pYKLuKjTvi/s640/Browser1.jpg" width="560" /></a></div>
<br />
<br />
The map is then displayed with the address you entered shown as the blue 'C' marker that denotes the center of the map and the rat sightings (ie., the 311 requests for rodent baitings) shown as red 'R' markers.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfOVtOx45nNvv1Jzt8dV9D0MByhB5_Zwy-bdRS7s25p2ntCQIEJRTP75pvPwbeYq7VHID2pxwgAIQIBr3xUJVVtwodiV7lTu8uVAKePYJ__M1UhD06dAMFSOFEYDM_UEHKWJOhxuAeeYjS/s1600/Browser2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfOVtOx45nNvv1Jzt8dV9D0MByhB5_Zwy-bdRS7s25p2ntCQIEJRTP75pvPwbeYq7VHID2pxwgAIQIBr3xUJVVtwodiV7lTu8uVAKePYJ__M1UhD06dAMFSOFEYDM_UEHKWJOhxuAeeYjS/s640/Browser2.jpg" width="560" /></a></div>
<br />
<br />
<span style="font-size: x-large;">Django </span><br />
I created this using the Django framework. Django is a platform that supports Python web applications. You can read about it at the <a href="https://www.djangoproject.com/">Django project site</a>. <br />
<br />
The example runs on my laptop using the free <a href="http://bitnami.org/stack/djangostack">Windows DjangoStack</a> for Windows from BitNami. This provides the web server, database, and django environment to build apps and test them on your own system.<br />
<br />
For production use, you could take the code after it has been tested out on your system and move it to a web hosting provider that runs Django. I have not done that. I just wanted to get something simple running on Django.<br />
<br />
There is a lot of information on the web about using Django. The two resources that were most helpful to me were:<br />
<ol>
<li>Django Project's, <a href="https://docs.djangoproject.com/en/1.3/intro/tutorial01/">Tutorial</a> for writing your first Django app</li>
<li>Django Project's, <a href="http://www.djangobook.com/en/2.0/">The Django Book</a>, esp. Chapter 7 on forms</li>
</ol>
<br />
<span style="font-size: x-large;">Installation</span><br />
Follow BitNami's installation instructions.<br />
<br />
I installed MySQL but database choice really doesn't matter. The application does not use any tables. <br />
<br />
I called my project, "Rats." I then called the only application for the project, "rats." I didn't understand the naming convention. It might have made more sense to call the project something like, "opendata," and the application, "rats."<br />
<br />
After installing, the directory for the Rats project was:<br />
<div style="font-family: "Courier New",Courier,monospace;">
C:\Documents and Settings\Administrator\BitNami DjangoStack projects\Rats</div>
<br />
The contents of the Rats project directory:<br />
<span style="font-family: "Courier New",Courier,monospace;">09/23/2011 04:08 PM 517 manage.py</span><br />
<span style="font-family: "Courier New",Courier,monospace;">09/25/2011 06:19 PM <DIR> rats</span><br />
<span style="font-family: "Courier New",Courier,monospace;">09/25/2011 04:51 PM 5,586 settings.py</span><br />
<span style="font-family: "Courier New",Courier,monospace;">09/25/2011 04:51 PM 3,086 settings.pyc</span><br />
<span style="font-family: "Courier New",Courier,monospace;"></span><span style="font-family: "Courier New",Courier,monospace;">09/25/2011 06:18 PM 199 urls.py</span><br />
<span style="font-family: "Courier New",Courier,monospace;">09/25/2011 06:19 PM 479 urls.pyc</span><br />
<span style="font-family: "Courier New",Courier,monospace;"></span><span style="font-family: "Courier New",Courier,monospace;">09/23/2011 04:08 PM 0 __init__.py</span><br />
<span style="font-family: "Courier New",Courier,monospace;">09/23/2011 05:32 PM 172 __init__.pyc</span><br />
<br />
<br />
The directory for the rats application:<br />
<div style="font-family: "Courier New",Courier,monospace;">
C:\Documents and Settings\Administrator\BitNami DjangoStack projects\Rats\rats</div>
<br />
The contents of the rats application directory:<br />
<span style="font-family: "Courier New",Courier,monospace;">09/23/2011 08:31 PM 60 models.py</span><br />
<span style="font-family: "Courier New",Courier,monospace;">09/23/2011 08:34 PM 231 models.pyc</span><br />
<span style="font-family: "Courier New",Courier,monospace;">09/25/2011 05:47 PM 7,066 ratslib.py</span><br />
<span style="font-family: "Courier New",Courier,monospace;">09/25/2011 05:48 PM 3,815 ratslib.pyc</span><br />
<span style="font-family: "Courier New",Courier,monospace;"></span><span style="font-family: "Courier New",Courier,monospace;">09/25/2011 05:19 PM <DIR> templates</span><br />
<span style="font-family: "Courier New",Courier,monospace;">09/23/2011 08:31 PM 399 tests.py</span><br />
<span style="font-family: "Courier New",Courier,monospace;">09/25/2011 06:17 PM 565 views.py</span><br />
<span style="font-family: "Courier New",Courier,monospace;">09/25/2011 06:17 PM 1,143 views.pyc</span><br />
<span style="font-family: "Courier New",Courier,monospace;"></span><span style="font-family: "Courier New",Courier,monospace;">09/23/2011 08:31 PM 0 __init__.py</span><br />
<span style="font-family: "Courier New",Courier,monospace;">09/23/2011 08:34 PM 177 __init__.pyc</span><br />
<br />
The template under the rats directory has one html file:<br />
<span style="font-family: "Courier New",Courier,monospace;">09/25/2011 05:19 PM 274 search_form.html</span><br />
<br />
<br />
<span style="font-size: x-large;">Files</span><br />
Following are the contents of the files for this application. Details about what they do and how
they fit together can be found in the two references listed above. I'll
provide a brief overview after listing the files.<br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
<b><span style="font-size: large;">settings.py</span></b></div>
I made a few changes to <span style="font-family: "Courier New",Courier,monospace;">../Rats/settings.py</span><br />
<ol>
<li>Set the location for my urlconf file:<br /><span style="font-family: "Courier New",Courier,monospace;">ROOT_URLCONF = 'Rats.urls' </span></li>
<li>Added my app to the list of installed apps:<br /><span style="font-family: "Courier New",Courier,monospace;">INSTALLED_APPS = (</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> 'django.contrib.auth',</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> 'django.contrib.contenttypes',</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> 'django.contrib.sessions',</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> 'django.contrib.sites',</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> 'django.contrib.messages',</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> 'django.contrib.staticfiles',</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> 'rats', </span> </li>
<li>Added my template directory for my html file(s):<br /><span style="font-family: "Courier New",Courier,monospace;">TEMPLATE_DIRS = (</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"></span><span style="font-family: "Courier New",Courier,monospace;">DjangoStack projects\Rats\rats\templates',</span><br />)</li>
</ol>
<div style="font-family: "Courier New",Courier,monospace;">
<b><span style="font-size: large;">urls.py</span></b></div>
<div style="font-family: "Courier New",Courier,monospace;">
from django.conf.urls.defaults import patterns, include, url<br />
<br />
urlpatterns = patterns('',<br />
(r'^search/$', 'rats.views.search'),<br />
(r'^map/$', 'rats.views.map')</div>
<br />
<br />
<b><span style="font-size: large;"><span style="font-family: "Courier New",Courier,monospace;">views.py</span></span></b><br />
<div style="font-family: "Courier New",Courier,monospace;">
from django.http import HttpResponse, HttpResponseRedirect<br />
from django.shortcuts import render_to_response<br />
from ratslib import geocode, getRatLocs, createMapUrl<br />
<br />
def search(request):<br />
return render_to_response('search_form.html')<br />
<br />
def map(request):<br />
if 'q' in request.GET:<br />
err1, lat, lng = geocode( request.GET['q'], 'Chicago', 'IL')<br />
err2, locs = getRatLocs(lat, lng, '100')<br />
url = createMapUrl(lat, lng, locs, 20)<br />
return HttpResponseRedirect(url)<br />
else:<br />
return HttpResponse('You submitted an empty form.'</div>
<br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
<b><span style="font-size: large;">search_form.html</span></b></div>
<span style="font-family: "Courier New",Courier,monospace;"><html></span><br />
<span style="font-family: "Courier New",Courier,monospace;"> <head></span><br />
<span style="font-family: "Courier New",Courier,monospace;"> <title>Search for Rats</title></span><br />
<span style="font-family: "Courier New",Courier,monospace;"> </head></span><br />
<span style="font-family: "Courier New",Courier,monospace;"> <body></span><br />
<span style="font-family: "Courier New",Courier,monospace;"> <form action="/map/" method="get"></span><br />
<span style="font-family: "Courier New",Courier,monospace;"> <input type="text" name="q"></span><br />
<span style="font-family: "Courier New",Courier,monospace;"> <input type="submit" value="Search"></span><br />
<span style="font-family: "Courier New",Courier,monospace;"> </form></span><br />
<span style="font-family: "Courier New",Courier,monospace;"> </body></span><br />
<span style="font-family: "Courier New",Courier,monospace;"></html></span><br />
<br />
<br />
<span style="font-size: large;">Other files</span><br />
<span style="font-family: "Courier New",Courier,monospace;">models.py</span> - No changes. There are no models (database tables) needed for this application.<br />
<span style="font-family: "Courier New",Courier,monospace;">tests.py</span> - No changes<br />
<span style="font-family: "Courier New",Courier,monospace;">manage.py</span> - No changes. <br />
<span style="font-family: "Courier New",Courier,monospace;">ratslib.py</span> - this contains functions used in <span style="font-family: "Courier New",Courier,monospace;">views.py</span>. It is a module holding the <span style="font-family: "Courier New",Courier,monospace;">geocode()</span>, <span style="font-family: "Courier New",Courier,monospace;">getRatLocs()</span>, and <span style="font-family: "Courier New",Courier,monospace;">createMapUrl()</span> functions from Example 19. See that example to understand the code. Because it's lengthy and not really germane to how to use Django, you'll find the contents of this file at the bottom of this entry.<br />
<br />
<br />
<span style="font-size: x-large;"></span><br />
<span style="font-size: x-large;">Overview</span><br />
So, how's this all work?<br />
<ol>
<li>When you start up Django from the <span style="font-family: "Courier New",Courier,monospace;">Rats</span> directory with <span style="font-family: "Courier New",Courier,monospace;">python manage.py runserver</span>, it gets the config settings from <span style="font-family: "Courier New",Courier,monospace;">settings.py</span>. Now, it knows there's a rats app (in the subdirectory, <span style="font-family: "Courier New",Courier,monospace;">rats</span>), where the <span style="font-family: "Courier New",Courier,monospace;">urlconf </span>is located, and where to find the html template files.</li>
<li>Go to a browser and use the URL <span style="font-family: "Courier New",Courier,monospace;">localhost:8000/search</span>. Django finds out from <span style="font-family: "Courier New",Courier,monospace;">urls.py</span> to call the function <span style="font-family: "Courier New",Courier,monospace;">search()</span> in <span style="font-family: "Courier New",Courier,monospace;">views.py</span>.</li>
<li>The <span style="font-family: "Courier New",Courier,monospace;">search(request)</span> function in <span style="font-family: "Courier New",Courier,monospace;">views.py</span> runs. Django goes to the template directory and opens <span style="font-family: "Courier New",Courier,monospace;">search_form.html</span> and sends it to the browser. This file has the HTML for the simplest query form you could have: one text entry box and a submit button. When <span style="font-family: "Courier New",Courier,monospace;">search()</span> finishes, the browser looks like the first screenshot above. The user will enter a street address and click the Submit button. The HTML generates a URL: <span style="font-family: "Courier New",Courier,monospace;">localhost:8000/map</span>, as a <span style="font-family: "Courier New",Courier,monospace;">GET request</span>. The street address, entered by the user, will be tacked on to the end of the URL in a name/value pair with the name of <span style="font-family: "Courier New",Courier,monospace;">q</span> and the value being the street address.</li>
<li>The Django server now gets the URL, <span style="font-family: "Courier New",Courier,monospace;">localhost:8000/map/q?<street address></span>. Django looks in <span style="font-family: "Courier New",Courier,monospace;">urls.py</span> and finds it should call <span style="font-family: "Courier New",Courier,monospace;">map(request)</span> in <span style="font-family: "Courier New",Courier,monospace;">views.py</span>. </li>
<li>The <span style="font-family: "Courier New",Courier,monospace;">map(request)</span> function in <span style="font-family: "Courier New",Courier,monospace;">views.py</span> runs. It checks that there is a '<span style="font-family: "Courier New",Courier,monospace;">q</span>' in the URL string. If not, the user clicked Submit without entering a street address so give an error message. Now, use the code in the <span style="font-family: "Courier New",Courier,monospace;">ratslib</span> to do the hard work. First, pass in the address to <span style="font-family: "Courier New",Courier,monospace;">geocode()</span>. The address is the value from the <span style="font-family: "Courier New",Courier,monospace;">GET</span> request associated with the name of <span style="font-family: "Courier New",Courier,monospace;">q</span>. After geocoding the street address, use the lat/lng to query for the rat locations, and build the URL for the static map. Finally, call <span style="font-family: "Courier New",Courier,monospace;">HttpResponseRedirect(url)</span>. By redirecting to that URL, Google will generate the map and paint it in the browser.</li>
</ol>
<span style="font-size: x-large;">Comments</span><br />
This example is just the basics for web application doing a query using SODA. You can take this in a lot of different directions:<br />
<ul>
<li>By changing <span style="font-family: "Courier New",Courier,monospace;">getRatLocs()</span>, you could query other views that have locations. For example, you could search for graffiti, crime, or fallen tree limbs. You'll need to set the right values for <span style="font-family: "Courier New",Courier,monospace;">hostName</span>, <span style="font-family: "Courier New",Courier,monospace;">originalViewId</span>, <span style="font-family: "Courier New",Courier,monospace;">columnId</span>, <span style="font-family: "Courier New",Courier,monospace;">colNumLat</span>, and <span style="font-family: "Courier New",Courier,monospace;">colNumLng</span>.</li>
<li>Use a Google dynamic map instead of a static map. The dynamic map will let the user zoom in and out.</li>
<li>Use paging to bring back, say 20, rats at a time and place on the map.</li>
<li>Add an entry field or radio buttons on the form for the radius of the circle. The user could enter the street address and enter or select the size of the circle to search.</li>
</ul>
The example left out considerations for security, performance, user experience, and best django coding techniques. There's work needed in all these areas before making it a public app.<br />
<br />
<b><span style="font-size: large;"><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">ratslib.py</span></span></b><br />
<span style="font-family: "Courier New",Courier,monospace;">import httplib</span><br />
<span style="font-family: "Courier New",Courier,monospace;">import json</span><br />
<span style="font-family: "Courier New",Courier,monospace;">import urllib</span><br />
<span style="font-family: "Courier New",Courier,monospace;">from urllib2 import Request, urlopen, URLError, HTTPError</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">#</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># geocode(street, city, state) returns the latitude and longitude of a</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># street address using Google's map API. Note, there is a limit on</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># the number of geocode requests you can make each day to Google.</span><br />
<span style="font-family: "Courier New",Courier,monospace;">#</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># Args:</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># street - string - street address</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># city - string - city name</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># state - string - state name</span><br />
<span style="font-family: "Courier New",Courier,monospace;">#</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># Returns:</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># err - string - error information. If empty, geocode was successful</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># lat - string - latitude of the street address</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># lng - string - longitude of the street address</span><br />
<span style="font-family: "Courier New",Courier,monospace;">#</span><br />
<span style="font-family: "Courier New",Courier,monospace;">def geocode(street, city, state):</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> err = ""</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> lat = ""</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> lng = ""</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> url = "http://maps.googleapis.com/maps/api/geocode/json?address=" + \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> urllib.quote_plus(street) + ',' + \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> urllib.quote_plus(city) + ',' + \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> urllib.quote_plus(state) + \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "&sensor=false"</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> req = Request(url)</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> try:</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> u = urlopen(req)</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> except URLError, e:</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> if hasattr(e, 'reason'):</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> err = e.reason</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> elif hasattr(e, 'code'):</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> err = e.code</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> else:</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> response = json.load(u)</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> # Check that Google sent back valid data. If so, get lat and long.</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> if response['status'] == 'OK':</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> lat = response['results'][0]['geometry']['location']['lat']</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> lng = response['results'][0]['geometry']['location']['lng']</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> # otherwise, Google returned an error</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> else:</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> err = 'Google error code: %s\n' % response['status']</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> return err, str(lat), str(lng)</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">#</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># getRatLocs(lat, lng, rad) returns a list of the locations of rats</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># from the Chicago 311 service requests for rodent baiting. The</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># locations will be within the circle that has a center point at</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># lat/lon and a radius of rad meters.</span><br />
<span style="font-family: "Courier New",Courier,monospace;">#</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># Args:</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># lat - string - latitude of the center point of the circle</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># lng - string - longitude of the center point of the circle</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># rad - string - radius, in meters, of the circle</span><br />
<span style="font-family: "Courier New",Courier,monospace;">#</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># Returns:</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># err - string - error information. If empty, geocode was successful</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># locs - list - each item being a sublist containing two elements: </span><br />
<span style="font-family: "Courier New",Courier,monospace;"># latitude and longitude that represents the location of a rat</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># baiting request.</span><br />
<span style="font-family: "Courier New",Courier,monospace;">#</span><br />
<span style="font-family: "Courier New",Courier,monospace;">def getRatLocs(lat, lng, rad):</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> # parameters used in the SODA POST request to do the search</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> hostName = "data.cityofchicago.org"</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> service = "/views/INLINE/rows"</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> formatType = "json"</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> parameters = "method=index"</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> headers = { "Content-type:" : "application/json" }</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> # SODA inline query. Lat, Lng, Radius are set to 0 here.</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> query = {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "originalViewId": "97t6-zrhs",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "name": "Nearby rats",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "query": {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "filterCondition": {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "type": "operator",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "value": "within_circle",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "children": </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> [</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "type": "column",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "columnId": 2849547</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> },</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "type": "literal",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "value": 0</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> },</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> { </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "type": "literal",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "value": 0</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> },</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "type": "literal",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "value": 0</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> ]</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> # constants used to index into inline filter children[]</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> queryLat = 1</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> queryLng = 2</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> queryRad = 3</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> # constants for table column numbers in query return data</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> colNumLat = 22</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> colNumLng = 23</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> # constants used to index into locs</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> locLat = 0</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> locLng = 1</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> # initialize return variables to be empty</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> err = ''</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> locs = list()</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> # put lat, lng, radius into the inline query</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> query["query"]["filterCondition"]["children"][queryLat]["value"] = lat</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> query["query"]["filterCondition"]["children"][queryLng]["value"] = lng</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> query["query"]["filterCondition"]["children"][queryRad]["value"] = rad</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> # setup and send the inline query</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> jsonQuery = json.dumps(query)</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> request = service + '.' + formatType + '?' + parameters </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> conn = httplib.HTTPConnection(hostName)</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> conn.request("POST", request, jsonQuery, headers)</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> response = conn.getresponse()</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> # check for good response, pull data out of response and setup locs</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> if response.reason != 'OK':</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> err = "%s %s" % (response.status, response.reason)</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> else:</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> rawResponse = response.read()</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> jsonResponse = json.loads(rawResponse)</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> for rowData in jsonResponse['data']:</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> locs.append([rowData[colNumLat], rowData[colNumLng]])</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> return err, locs</span><br />
<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">#</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># createMapUrl(centerLat, centerLng, locs) returns the URL for a Google</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># static map that has a marker for the center of the map and markers</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># for the locations in locs.</span><br />
<span style="font-family: "Courier New",Courier,monospace;">#</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># Args:</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># centerLat - string - latitude of the center point of the map</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># centerLng - string - longitude of the center point of the map</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># locs - list - where each item is a location to be marked on the map.</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># Each item is a sublist with two strings: latitude and longitude</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># n - integer - the first n locations in locs will be marked on the</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># map. There is an upper limit on the size of the URL sent to</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># the Google Map API. A long list of locations in the URL will</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># not be accepted. n keeps the URL within the limit.</span><br />
<span style="font-family: "Courier New",Courier,monospace;">#</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># Returns:</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># URL - string - URL to send to the Google Map API to create a static</span><br />
<span style="font-family: "Courier New",Courier,monospace;"># map in the default browser.</span><br />
<span style="font-family: "Courier New",Courier,monospace;">#</span><br />
<span style="font-family: "Courier New",Courier,monospace;">def createMapUrl(centerLat, centerLng, locs, n):</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> # define constants for indexing into the locs sublist</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> locLat = 0</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> locLng = 1</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> # define parameters for the map</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> urlMapAPI = "http://maps.googleapis.com/maps/api/staticmap"</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> mapZoomLevel = "14"</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> mapHeight = "512"</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> mapWidth = "512"</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> mapType = "roadmap"</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> centerMarkerColor = "blue"</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> centerMarkerLabel = "C"</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> locMarkerColor = "red"</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> locMarkerLabel = "R"</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> url = \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> urlMapAPI + '?' + \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> 'center=' + centerLat + ',' + centerLng + \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> '&' + \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> 'size=' + mapHeight + 'x' + mapWidth + \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> '&' + \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> 'maptype=' + mapType + \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> '&' + \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> 'sensor=false' + \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> '&' + \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> 'markers=color:' + centerMarkerColor + '%7C' + \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> 'label:' + centerMarkerLabel + '%7C' + centerLat + ',' + centerLng </span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> i = 0</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> for loc in locs:</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> if i < n:</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> url += '&' + \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> 'markers=color:' + locMarkerColor + '%7C' + \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> 'label:' + locMarkerLabel + '%7C' + \</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> loc[locLat] + ',' + loc[locLng]</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> i +=1</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> return url</span><br />
<br />
<br />Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-5928026535874540705.post-54398625403924771642011-09-07T18:37:00.000-07:002011-09-07T18:54:18.784-07:00Example #22 - Receiving data via pagingFor a couple of the API requests that return large amounts of data, you can receive the data in pages. Instead of waiting for and then receiving one very long stream of output, you can ask for and receive consecutive chunks - pages.<br />
<br />
Here are two examples: receiving the metadata for all views at a site and receiving all rows in a view.<br />
<br />
<span style="font-size: x-large;">Paging the return of all views</span><br />
This uses the /api/views API service with two parameters. <span style="font-family: "Courier New",Courier,monospace;">page</span> is the number of the page to retrieve. <span style="font-family: "Courier New",Courier,monospace;">limit</span> is the number of views to return; the size of the page. The code prints out the names of all the views currently available on the Cook County site. Note the first page number is 1, not 0.<br />
<br />
<span style="font-size: large;">Code</span><br />
<div style="font-family: "Courier New",Courier,monospace;">
from urllib2 import urlopen</div>
<div style="font-family: "Courier New",Courier,monospace;">
from json import load</div>
<div style="font-family: "Courier New",Courier,monospace;">
<br /></div>
<div style="font-family: "Courier New",Courier,monospace;">
hostname = "datacatalog.cookcountyil.gov"</div>
<div style="font-family: "Courier New",Courier,monospace;">
<br /></div>
<div style="font-family: "Courier New",Courier,monospace;">
pageSize = '20'</div>
<div style="font-family: "Courier New",Courier,monospace;">
pageNum = 1 </div>
<div style="font-family: "Courier New",Courier,monospace;">
while True:</div>
<div style="font-family: "Courier New",Courier,monospace;">
url = "http://%s/api/views.json?page=%s&limit=%s" % (hostname, pageNum, pageSize)</div>
<div style="font-family: "Courier New",Courier,monospace;">
response = urlopen(url)</div>
<div style="font-family: "Courier New",Courier,monospace;">
views = load(response)</div>
<div style="font-family: "Courier New",Courier,monospace;">
numViewsRcvd = len(views)</div>
<div style="font-family: "Courier New",Courier,monospace;">
if numViewsRcvd == 0:</div>
<div style="font-family: "Courier New",Courier,monospace;">
break</div>
<div style="font-family: "Courier New",Courier,monospace;">
print '\n====== PAGE %s - %s VIEWS DOWNLOADED: ======' % (pageNum, numViewsRcvd)</div>
<div style="font-family: "Courier New",Courier,monospace;">
for view in views:</div>
<div style="font-family: "Courier New",Courier,monospace;">
print view['name']</div>
<div style="font-family: "Courier New",Courier,monospace;">
pageNum += 1</div>
<br />
<span style="font-size: large;">Output</span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxVv8TMSNUSwLl2eFs1RUtFtAJxE6wnKZvt6MaaSeprplycb4wJeZywbvMtF9m5MOQPdZXGnul8Mdk37j70F6Qb6q5AL9EsqFG9uZIAj65W5rM_yJuZqxRn3mBQJxRbH5tUOTlLCKwBW-J/s1600/Example+22a+output.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxVv8TMSNUSwLl2eFs1RUtFtAJxE6wnKZvt6MaaSeprplycb4wJeZywbvMtF9m5MOQPdZXGnul8Mdk37j70F6Qb6q5AL9EsqFG9uZIAj65W5rM_yJuZqxRn3mBQJxRbH5tUOTlLCKwBW-J/s640/Example+22a+output.jpg" width="632" /></a></div>
<br />
<span style="font-size: x-large;">Paging the return of all rows in a table</span><br />
This example uses the Chicago Ward Office table. The code prints the contents for the Ward Number column for each of the rows. Note that the first row is 0.<br />
<br />
<span style="font-size: large;">Code</span><br />
<div style="font-family: "Courier New",Courier,monospace;">
from urllib2 import urlopen<br />from json import load<br /><br />hostname = "data.cityofchicago.org"<br />viewID = "htai-wnw4"<br />startRow = 0<br />numRows = 20<br /><br />while True:<br /> url = "http://%s/api/views/%s/rows.json?method=getRows&start=%s&length=%s" % (hostname, viewID, str(startRow), str(numRows))<br /> response = urlopen(url)<br /> rows = load(response)<br /> numRowsRcvd = len(rows)<br /> if numRowsRcvd == 0:<br /> break<br /> print '\n====== %s ROWS DOWNLOADED: ======' % (numRowsRcvd)<br /> for row in rows:<br /> print row['2609304'] # rowID for ward number<br /> startRow += numRow</div>
<br />
<span style="font-size: large;">Output</span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEijE-0HBmZ6HLSgV1oOqodYz01i_78Y2V7dIL0TOZgmfc3QPZ4FH661Pn3bBD-8XU2wVTNuW886cYLkIn5a8mWaqbszf-zf-yX-iM2TSzC2yBM5b203EsSV9KIVV8lZqKpIX-FQx9-4jhCn/s1600/Example+22b+output.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEijE-0HBmZ6HLSgV1oOqodYz01i_78Y2V7dIL0TOZgmfc3QPZ4FH661Pn3bBD-8XU2wVTNuW886cYLkIn5a8mWaqbszf-zf-yX-iM2TSzC2yBM5b203EsSV9KIVV8lZqKpIX-FQx9-4jhCn/s640/Example+22b+output.jpg" width="376" /></a></div>
<br />
<span style="font-size: x-large;">Summary</span><br />
Receiving your data in pages might make it easier for your app. You may want to show, say, 20 results at a time. Or, you may want to show the first 20 while you are getting the rest of the results.<br />
<br />
For getting all the views in a site, you need to use paging if you there are more than 200. The default is to receive 50 views. You can use a parameter, XXX, to get 200 in one call. Beyond that, you have to page to get all the views.Unknownnoreply@blogger.com3tag:blogger.com,1999:blog-5928026535874540705.post-81625440547149855802011-09-06T19:31:00.000-07:002011-09-06T19:31:47.952-07:00Example #21 - Operators for inline filtersI couldn't find a complete list on the SODA site of the operators available for use in an inline filter. Socrata kindly provide one.<br />
<br />
<b>All types:</b><br /><span style="font-family: "Courier New",Courier,monospace;">EQUALS</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">NOT_EQUALS</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">IS_BLANK</span> <br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">IS_NOT_BLANK</span><br />
<br />
<b>Textual types:</b><br /><span style="font-family: "Courier New",Courier,monospace;">STARTS_WITH</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">CONTAINS</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">NOT_CONTAINS</span><br />
<br />
<b>Numeric and date types:</b><br /><span style="font-family: "Courier New",Courier,monospace;">LESS_THAN</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">LESS_THAN_OR_EQUALS</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">GREATER_THAN</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">GREATER_THAN_OR_EQUALS</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">BETWEEN</span><br />
<br />
<b>Location:</b><br /><span style="font-family: "Courier New",Courier,monospace;">WITHIN_CIRCLE</span><br />
<br />All operators are binary except for is/isnot blank, which are unary, and within circle, which is tri-ary. <br /><br />(Thanks Clint!)Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-83505148734199754152011-09-04T19:34:00.000-07:002011-09-08T18:18:02.443-07:00Example #20 - Three utilities for building inline filters<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
Along the way, I put together three short scripts that help in finding the details to build the inline filters.<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">getcolumns.py</span> - prints the <span style="font-family: "Courier New",Courier,monospace;">columnName</span>s and <span style="font-family: "Courier New",Courier,monospace;">columnID</span>s from a view<br />
<span style="font-family: "Courier New",Courier,monospace;">getquery.py</span> - prints the query value for a filtered view<br />
<span style="font-family: "Courier New",Courier,monospace;">getview.py</span> - prints the metadata for a view<br />
<br />
They've been helpful for quickly getting the behind-the-scenes information from one of the portals.<br />
<br />
They share the same command syntax:<br />
<div style="font-family: "Courier New",Courier,monospace;">
python <program> [-cook | -chicago | - il | <hostname>] <viewID></div>
<br />
Where:<br />
<span style="font-family: "Courier New",Courier,monospace;">-cook</span> opens the Cook County Data Portal<br />
<span style="font-family: "Courier New",Courier,monospace;">-chicago</span> opens the City of Chicago Data Portal<br />
<span style="font-family: "Courier New",Courier,monospace;">-ill </span>opens the State of Illinois Data Portal<br />
<span style="font-family: "Courier New",Courier,monospace;"><hostname></span> - allows the user to specify another Socrata site<br />
<span style="font-family: "Courier New",Courier,monospace;"><viewID></span> - is the 9 character ID of the view to be used. The format is <span style="font-family: "Courier New",Courier,monospace;"><i>cccc</i>-<i>cccc</i></span><br />
<br />
<span style="font-size: x-large;">Examples</span><br />
Here is one example of each utility.<br />
<br />
<span style="font-size: large;">getcolumns</span><br />
<span style="font-family: "Courier New",Courier,monospace;">python getcolumns.py -cook e9qr-rmq4</span><br />
Prints the IDs and names for all columns in the Cook County view for the DEC2010 Check Register. This helps in getting the <span style="font-family: "Courier New",Courier,monospace;">columnID</span> to put in your own filter.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizzQF6qUPrhMNvz2YOvlyLcUbWnrD3wgdNjOw1lxLd4Y2emDhIB0aY3fs5iJt2rbDBOHSkcZAFVn-cPO2OoFj_17bs1K-Abby-DFmA-q_BVJGCJ2gDpAw5XihEE-SLhlgjsT2WeTkJtqH1/s1600/getcolumns+output.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="227" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizzQF6qUPrhMNvz2YOvlyLcUbWnrD3wgdNjOw1lxLd4Y2emDhIB0aY3fs5iJt2rbDBOHSkcZAFVn-cPO2OoFj_17bs1K-Abby-DFmA-q_BVJGCJ2gDpAw5XihEE-SLhlgjsT2WeTkJtqH1/s640/getcolumns+output.jpg" width="640" /></a></div>
<br />
<span style="font-size: large;">getquery</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">python getquery.py -cook 2wek-2jap</span><br />
Prints the filter query Socrata generated for Example #15 where I used the web UI to search the DEC2010 Check Register for payments between $1m and $10m for construction services (which is a product code of 912). You can create a filter using your browser, use this to see it, and use it to guide you in writing one in your code.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7einobSZnWiWqCNzqAd8hhUrtwLvdr_iJrpGJZw5qS2MK-sULeLSlNfPPa3xbmjJPPdCvJcYZ9iXFPtURCokAy6y4JL4vwgn9KKPDs922Xlf54GCT75kIMX10j1axKVwmHCj12d-wMuve/s1600/getquery+output.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7einobSZnWiWqCNzqAd8hhUrtwLvdr_iJrpGJZw5qS2MK-sULeLSlNfPPa3xbmjJPPdCvJcYZ9iXFPtURCokAy6y4JL4vwgn9KKPDs922Xlf54GCT75kIMX10j1axKVwmHCj12d-wMuve/s640/getquery+output.jpg" width="640" /></a></div>
<br />
<span style="font-size: large;">getview</span><br />
<span style="font-family: "Courier New",Courier,monospace;">python getview.py opendata.socrata.com n5m4-mism</span><br />
Prints the metadata for the <a href="http://opendata.socrata.com/Government/The-White-House-Nominations-Appointments/n5m4-mism">White House Nominations and Appointments dataset</a>. Direct the output to a file and use a text editor to browse through it to peruse a view. The screenshot shows the head of the output. There's much more that is printed.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgSftIvKDqE4KfmnjvI897O4avpI3tLWOWHYt0Z8WdjbVfEAJmutTZiCIEQU6X6ZKTByh9lau4Jq1U3NP-tP0Vd2Is5-u5AQdFUDfb5pp2vVgkhcNVCE3t0vzlVQY4UUJY32Tl3sqpz-xsr/s1600/getview+output.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="534" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgSftIvKDqE4KfmnjvI897O4avpI3tLWOWHYt0Z8WdjbVfEAJmutTZiCIEQU6X6ZKTByh9lau4Jq1U3NP-tP0Vd2Is5-u5AQdFUDfb5pp2vVgkhcNVCE3t0vzlVQY4UUJY32Tl3sqpz-xsr/s640/getview+output.jpg" width="640" /></a></div>
<br />
<br />
<span style="font-size: x-large;">Code</span><br />
<span style="font-size: large;">getcolumns</span><br />
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: small;">#<br /># getcolumns - Prints the column IDs along with the column names for a view.<br />#<br /># args: <hostname> <viewID><br /># If <hostname> is "-cook", use the Cook County Data Portal host name<br /># If <hostname> is "-chicago", use the City of Chicago Data Portal host name<br /># If <hostname> is "-ill", use the State of Illinois Data Portal host name<br /># Other <hosthame> is the host name for the Socrata site. Do not include "http://"<br /># <viewID> is the 9 character Socrata view (aka dataset) ID. Format is: "cccc-cccc"<br />#<br /># output: Lists the view's columnID and column name for all columns; one per line.<br /># The column ID is the value for the 'id' key. <br />#<br />import sys<br />from urllib2 import urlopen, URLError, HTTPError<br />from json import load<br />import pprint<br /><br /><br />hostNameChicago = "data.cityofchicago.org"<br />hostNameCook = "datacatalog.cookcountyil.gov"<br />hostNameIll = "data.illinois.gov"<br /><br />if len(sys.argv) !=3:<br /> sys.stderr.write("Usage: python %s [-chicago | -cook | -ill | <hostname>] <viewID>\n" % sys.argv[0])<br /> raise SystemExit(1)<br /><br />if sys.argv[1] == "-chicago":<br /> hostName = hostNameChicago<br />elif sys.argv[1] == "-cook":<br /> hostName = hostNameCook<br />elif sys.argv[1] == "-ill":<br /> hostName = hostNameIll<br />else:<br /> hostName = sys.argv[1]<br /><br />viewID = sys.argv[2]<br />url = "http://%s/api/views/%s/columns.json" % (hostName, viewID)<br /><br />try:<br /> u = urlopen(url)<br />except HTTPError, e:<br /> print "The server at %s could not handle the request." % url<br /> print "Error code: ", e.code<br />except URLError, e:<br /> print "We failed to reach the server at %s." % url<br /> print "Reason: ", e.reason<br />else:<br /> response = load(u)<br /><br /> i = 0<br /> while i < len(response):<br /> print response[i]['id'], ' : ', response[i]['name']<br /> i += 1</span></div>
<span style="font-size: x-small;"></span><br />
<span style="font-size: large;"><br /></span><br />
<span style="font-size: large;">getquery</span><br />
<span style="font-size: small;"><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># getquery - Prints the query resource in a filtered view</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># This is used to quickly create the query document used in an INLINE filter API request.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># args: <hostname> <viewID></span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># If <hostname> is "-cook", use the Cook County Data Portal host name</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># If <hostname> is "-chicago", use the City of Chicago Data Portal host name</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># If <hostname> is "-ill", use the State of Illinois Data Portal host name</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># Other <hosthame> is the host name for the Socrata site. Do not include "http://"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># <viewID> is the 9 character Socrata view (aka dataset) ID. Format is: "cccc-cccc"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># output: The 'query' portion of the view metadata. Unicode indicators are stripped off the strings.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># The output is intended to be ready to be used in a program that is going to call the Socrata</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># views service API with an INLINE filter request.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">import sys</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">from urllib2 import urlopen, URLError, HTTPError</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">from json import load</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">import pprint</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">hostNameChicago = "data.cityofchicago.org"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">hostNameCook = "datacatalog.cookcountyil.gov"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">hostNameIll = "data.illinois.gov"</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">if len(sys.argv) !=3:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> sys.stderr.write("Usage: python %s [-chicago | -cook | -ill | <hostname>] <viewID>\n" % sys.argv[0])</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> raise SystemExit(1)</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">if sys.argv[1] == "-chicago":</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> hostName = hostNameChicago</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">elif sys.argv[1] == "-cook":</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> hostName = hostNameCook</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">elif sys.argv[1] == "-ill":</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> hostName = hostNameIll</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">else:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> hostName = sys.argv[1]</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">viewID = sys.argv[2]</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">url = "http://%s/api/views/%s/rows.json" % (hostName, viewID)</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">try:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> u = urlopen(url)</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">except HTTPError, e:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> print "The server at %s could not handle the request." % url</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> print "Error code: ", e.code</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">except URLError, e:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> print "We failed to reach the server at %s." % url</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> print "Reason: ", e.reason</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">else:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> response = load(u)</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> pp = pprint.PrettyPrinter(indent=3)</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> pp.pprint(response['meta']['view']['query'])</span></span><br />
<br />
<span style="font-size: large;">getview</span><br />
<span style="font-size: small;"><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># getview - Prints the metadata for a view</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># args: <hostname> <viewID></span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># If <hostname> is "-cook", use the Cook County Data Portal host name</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># If <hostname> is "-chicago", use the City of Chicago Data Portal host name</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># If <hostname> is "-ill", use the State of Illinois Data Portal host name</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># Other <hosthame> is the host name for the Socrata site. Do not include "http://"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># <viewID> is the 9 character Socrata view (aka dataset) ID. Format is: "cccc-cccc"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># output: An indented print the response from /api/views/<viewID>. </span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">import sys</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">from urllib2 import urlopen, URLError, HTTPError</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">from json import load</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">import pprint</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">hostNameChicago = "data.cityofchicago.org"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">hostNameCook = "datacatalog.cookcountyil.gov"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">hostNameIll = "data.illinois.gov"</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">if len(sys.argv) != 3:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> sys.stderr.write("Usage: python %s [-chicago | -cook | -ill | <hostname>] <viewID>\n" % sys.argv[0])</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> raise SystemExit(1)</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">if sys.argv[1] == "-chicago":</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> hostName = hostNameChicago</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">elif sys.argv[1] == "-cook":</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> hostName = hostNameCook</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">elif sys.argv[1] == "-ill":</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> hostName = hostNameIll</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">else:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> hostName = sys.argv[1]</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">viewID = sys.argv[2]</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">url = "http://%s/api/views/%s.json" % (hostName, viewID)</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">try:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> u = urlopen(url)</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">except HTTPError, e:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> print "The server at %s could not handle the request." % url</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> print "Error code: ", e.code</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">except URLError, e:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> print "We failed to reach the server at %s." % url</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> print "Reason: ", e.reason</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">else:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> response = load(u)</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> </span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> pp = pprint.PrettyPrinter(indent=4)</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> pp.pprint(response)</span></span>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-82065563461685071752011-09-04T15:51:00.000-07:002011-09-25T15:47:20.883-07:00Example #19 - Rats! - a crude appWe can combine the functionality of previous examples to construct a simple app.<br />
<br />
Let's build <span style="font-family: "Courier New",Courier,monospace;">rats.py</span>. Instead of potholes, we'll search the Chicago Data Portal dataset, <a href="http://data.cityofchicago.org/Service-Requests/311-Service-Requests-Rodent-Baiting-2011-/97t6-zrhs">311 Service Requests - Rodent Baiting (2011)</a> for locations of rodent complaints and display a map with markers showing nearby rat sightings.<br />
<br />
Example #18's <span style="font-family: "Courier New",Courier,monospace;">getgeo.py</span> has the basics for taking a street address as an argument and determining the latitude and longitude. This is the center of our circle and the user can take a default radius or specify the radius of the circle.<br />
<br />
Example #17's simple geo query code has the basics to search the Rodent Baiting dataset to find rat complaints inside that circle. The search will return the latitude and longitude of those nearby rat complaints.<br />
<br />
Example #18's code for creating a Google map has the basics for constructing a URL with all the rat complaint locations and sending it to Google to display a map on the user's browser.<br />
<br />
The code is lengthy. It's posted at the end. Soon, it'll be on github.<br />
<br />
Here are two examples of running it and the output. It runs from the command prompt. Displays a short output message in that window and opens up a browser tab with a static Google map.<br />
<br />
<span style="font-size: large;">Example with default radius </span><br />
Check for rats at 2400 W. Fullerton Ave in the default circle radius of 100 meters.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikbfHNaFRjm7YeV6ZxeN5ZRX3bqqzu1aGuEEtueczJZeOY5mE7q6mZ3808kFbh2NusoOG_4ko7WBNQCaEDCPI97Rng6Gy4Y5oPcl16S8ufO4QNVpgSnQY02p4x9MeELYTFb4qPPRgNQAcs/s1600/rats+output+1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="150" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikbfHNaFRjm7YeV6ZxeN5ZRX3bqqzu1aGuEEtueczJZeOY5mE7q6mZ3808kFbh2NusoOG_4ko7WBNQCaEDCPI97Rng6Gy4Y5oPcl16S8ufO4QNVpgSnQY02p4x9MeELYTFb4qPPRgNQAcs/s640/rats+output+1.jpg" width="640" /></a></div>
<br />
There are eight rat complaints in this circle. Here's the map of them.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_Gic6VReOIbjWSZ1vUuX6eUG_tX8BTo8jLL7FkmLd3Jy_62gRN2XEFqMS6l8mFFVcTeBzXWzvBAvq6aTe6ymKX5FEpSiarVLLKy7E_oIjAy-2M-KQzpOaRBGCu4b6jhtDfJbo6LebSJ1v/s1600/rats+map+1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_Gic6VReOIbjWSZ1vUuX6eUG_tX8BTo8jLL7FkmLd3Jy_62gRN2XEFqMS6l8mFFVcTeBzXWzvBAvq6aTe6ymKX5FEpSiarVLLKy7E_oIjAy-2M-KQzpOaRBGCu4b6jhtDfJbo6LebSJ1v/s640/rats+map+1.jpg" width="594" /></a></div>
<br />
<span style="font-size: large;">Example with bigger radius</span><br />
Ask again with a bigger circle. Try 300 meters.<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikwHt7dE4cSKMaTzyQhEnC77oIruYjPpHBFr_uUg8PfQmOj0NdQIrZJw1cYZVEb_Cz5rli39FlbzWM8-mBtf0meM4vs9DpiOiBnMrGFUIZrWBQiV63YKXrXN9n95c5ggeK96MXBBRRjRW3/s1600/rats+output+2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="166" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikwHt7dE4cSKMaTzyQhEnC77oIruYjPpHBFr_uUg8PfQmOj0NdQIrZJw1cYZVEb_Cz5rli39FlbzWM8-mBtf0meM4vs9DpiOiBnMrGFUIZrWBQiV63YKXrXN9n95c5ggeK96MXBBRRjRW3/s640/rats+output+2.jpg" width="640" /></a><br />
<br />
There are 85 complaints in this bigger circle. Only the first 20 are displayed on the map. Trying to display all of these as markers makes the URL too long for Google to process and generates an error.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkY4CvsiXrmUXaRztl9AyoG7B2dRaQedsloCTHRU6gpW7vRwAmkbq1YftwhLyHVZvGoHzZ-j5D8xqEOLhzQyKtt_wxX1j6DpIisHq4HDFa-eCw6y00K6JPGNumvrIz1OsJTgYyU4dKHuxw/s1600/rats+map+2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkY4CvsiXrmUXaRztl9AyoG7B2dRaQedsloCTHRU6gpW7vRwAmkbq1YftwhLyHVZvGoHzZ-j5D8xqEOLhzQyKtt_wxX1j6DpIisHq4HDFa-eCw6y00K6JPGNumvrIz1OsJTgYyU4dKHuxw/s640/rats+map+2.jpg" width="568" /> </a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<span style="font-size: x-large;">Comments</span><br />
<span style="font-family: "Courier New",Courier,monospace;">rats.py</span> is a prototype. It gives you the basics and the idea of one way to use the data. There are many, many improvements that could be made:<br />
<ul>
<li>turn it into a web app where the user enters the address from a web browser and sees the results in the same window.</li>
<li>make it a mobile app where the user's current location is used for the center of the circle.</li>
<li>use dynamic maps to allow the user to pan and zoom the map</li>
<li>page the results from the query back in chunks of 20 rows so the user can see all the rats in the circle</li>
<li>add flyovers to the markers so the user can click on the marker to get details in a balloon of text</li>
<li>remove duplicate rat complaints</li>
<li>use different colored markers for open and closed complaints. </li>
<li>use a custom map marker that looks like little rat</li>
<li>and so on.</li>
</ul>
<br />
<ul>
</ul>
<span style="font-size: x-large;">Code</span><br />
<span style="font-size: small;"><span style="font-family: inherit;"><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># rats - Takes a Chicago street address as input and opens a map in your</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># default browser with nearby locations of rat complaints. The</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># address is the center of a circle that is <radius> meters. Rodent</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># baiting requests in that circle will appear on the map. The total</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># number of rats found is printed in the command prompt window.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># The maximum number of markers on the map is maxNumRatsInMap. If the</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># URL has too many markers, it becomes too long for Google and</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># generates and error. Playing with this max keeps the URL short</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># enough to process. If more rats than the max are found, the first</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># maxNumRatsInMap are displayed on map. A message is printed to the</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># command prompt window informing user the number of rats in the map.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># Uses Google Maps API for geocoding and map display.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># Uses Chicago Data Portal 311 Service Requests for Rodent Baitig in</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># 2011.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># Does not check for duplicate complaints or open/closed complaints.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># usage: python rats.py <street> [radius]</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># example: python rats.py "2400 West Fullerton"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># python rats.py "2400 w fullerton" 200</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># args: <street> - street address in quotes. Assumes city is Chicago and</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># state is IL</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># <radius> - find rat complaints in circle centered on <street> that is</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># <radius> meters in radius. Default is 100. Must be greater than 0.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># </span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">import httplib</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">import sys</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">import pprint</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">import json</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">import urllib</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">from urllib2 import Request, urlopen, URLError, HTTPError</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">import os</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># geocode(street, city, state) returns the latitude and longitude of a</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># street address using Google's map API. Note, there is a limit on</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># the number of geocode requests you can make each day to Google.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># Args:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># street - string - street address</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># city - string - city name</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># state - string - state name</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># Returns:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># err - string - error information. If empty, geocode was successful</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># lat - string - latitude of the street address</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># lng - string - longitude of the street address</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">def geocode(street, city, state):</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> err = ""</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> lat = ""</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> lng = ""</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> url = "http://maps.googleapis.com/maps/api/geocode/json?address=" + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> urllib.quote_plus(street) + ',' + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> urllib.quote_plus(city) + ',' + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> urllib.quote_plus(state) + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> "&sensor=false"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> req = Request(url)</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> try:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> u = urlopen(req)</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> except URLError, e:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> if hasattr(e, 'reason'):</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> err = e.reason</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> elif hasattr(e, 'code'):</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> err = e.code</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> else:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> response = json.load(u)</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> # Check that Google sent back valid data. If so, get lat and long.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> if response['status'] == 'OK':</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> lat = response['results'][0]['geometry']['location']['lat']</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> lng = response['results'][0]['geometry']['location']['lng']</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> # otherwise, Google returned an error</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> else:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> err = 'Google error code: %s\n' % response['status']</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> return err, str(lat), str(lng)</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># getRatLocs(lat, lng, rad) returns a list of the locations of rats</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># from the Chicago 311 service requests for rodent baiting. The</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># locations will be within the circle that has a center point at</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># lat/lon and a radius of rad meters.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># Args:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># lat - latitude of the center point of the circle</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># lng - longitude of the center point of the circle</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># rad - radius, in meters, of the circle</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># Returns:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># err - string with error information. If empty, geocode was successful</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># locs - a list with each item being a sublist containing two</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># elements: latitude and longitude that represents the location of</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># a rat baiting request.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">def getRatLocs(lat, lng, rad):</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> # parameters used in the SODA POST request to do the search</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> hostName = "data.cityofchicago.org"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> service = "/views/INLINE/rows"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> formatType = "json"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> parameters = "method=index"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> headers = { "Content-type:" : "application/json" }</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> # SODA inline query. Lat, Lng, Radius are set to 0 here.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> query = {</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> "originalViewId": "97t6-zrhs",</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> "name": "Nearby rats",</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> "query": {</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> "filterCondition": {</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> "type": "operator",</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> "value": "within_circle",</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> "children": </span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> [</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> {</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> "type": "column",</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> "columnId": 2849547</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> },</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> {</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> "type": "literal",</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> "value": 0</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> },</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> { </span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> "type": "literal",</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> "value": 0</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> },</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> {</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> "type": "literal",</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> "value": 0</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> }</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> ]</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> }</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> }</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> }</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> # constants used to index into inline filter children[]</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> queryLat = 1</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> queryLng = 2</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> queryRad = 3</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> # constants for table column numbers in query return data</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> colNumLat = 22</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> colNumLng = 23</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> # constants used to index into locs</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> locLat = 0</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> locLng = 1</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> # initialize return variables to be empty</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> err = ''</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> locs = list()</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> # put lat, lng, radius into the inline query</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> query["query"]["filterCondition"]["children"][queryLat]["value"] = lat</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> query["query"]["filterCondition"]["children"][queryLng]["value"] = lng</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> query["query"]["filterCondition"]["children"][queryRad]["value"] = rad</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> # setup and send the inline query</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> jsonQuery = json.dumps(query)</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> request = service + '.' + formatType + '?' + parameters </span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> conn = httplib.HTTPConnection(hostName)</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> conn.request("POST", request, jsonQuery, headers)</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> response = conn.getresponse()</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> # check for good response, pull data out of response and setup locs</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> if response.reason != 'OK':</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> err = "%s %s" % (response.status, response.reason)</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> else:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> rawResponse = response.read()</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> jsonResponse = json.loads(rawResponse)</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> for rowData in jsonResponse['data']:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> locs.append([rowData[colNumLat], rowData[colNumLng]])</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> return err, locs</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># createMapUrl(centerLat, centerLng, locs) returns the URL for a Google</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># static map that has a marker for the center of the map and markers</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># for the locations in locs.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># Args:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># centerLat - string - latitude of the center point of the map</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># centerLng - string - longitude of the center point of the map</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># locs - list - where each item is a location to be marked on the map.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># Each item is a sublist with two strings: latitude and longitude</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># n - integer - the first n locations in locs will be marked on the</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># map. There is an upper limit on the size of the URL sent to</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># the Google Map API. A long list of locations in the URL will</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># not be accepted. n keeps the URL within the limit.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># Returns:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># URL - string - URL to send to the Google Map API to create a static</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># map in the default browser.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">def createMapUrl(centerLat, centerLng, locs, n):</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> # define constants for indexing into the locs sublist</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> locLat = 0</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> locLng = 1</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> # define parameters for the map</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> urlMapAPI = "http://maps.googleapis.com/maps/api/staticmap"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> mapZoomLevel = "14"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> mapHeight = "512"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> mapWidth = "512"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> mapType = "roadmap"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> centerMarkerColor = "blue"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> centerMarkerLabel = "C"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> locMarkerColor = "red"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> locMarkerLabel = "R"</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> url = \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> urlMapAPI + '?' + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> 'center=' + centerLat + ',' + centerLng + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> '&' + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> 'size=' + mapHeight + 'x' + mapWidth + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> '&' + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> 'maptype=' + mapType + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> '&' + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> 'sensor=false' + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> '&' + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> 'markers=color:' + centerMarkerColor + '%7C' + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> 'label:' + centerMarkerLabel + '%7C' + centerLat + ',' + centerLng </span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> i = 0</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> for loc in locs:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> if i < n:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> url += '&' + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> 'markers=color:' + locMarkerColor + '%7C' + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> 'label:' + locMarkerLabel + '%7C' + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> loc[locLat] + ',' + loc[locLng]</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> i +=1</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> return url</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># constants used to index into addr[]</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">indexStreet = 0</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">indexCity = 1</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">indexState = 2</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># defaults</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">circleRadius = 100 </span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">maxNumRatsInMap = 20 </span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">if len(sys.argv) < 2 or len(sys.argv) > 3:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> sys.stderr.write("Usage: python %s \"street\" [radius]\n" % sys.argv[0])</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> raise SystemExit(1)</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">addr = sys.argv[1]</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">if len(sys.argv) == 3:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> try:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> circleRadius = int(sys.argv[2])</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> except:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> sys.stderr.write("Usage: python %s \"street\" [radius]\n" % sys.argv[0])</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> raise SystemExit(1)</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">if circleRadius <= 0:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> sys.stderr.write("Usage: python %s \"street\" [radius]\n" % sys.argv[0])</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> raise SystemExit(1)</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">err, circleCenterLat, circleCenterLng = geocode(addr, 'Chicago', 'IL')</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">if (err != ""):</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> print "Geocoding error = %s" % err</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> sys.exit(0)</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">err, ratLocations = getRatLocs(circleCenterLat, circleCenterLng, circleRadius)</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">if (err != ""):</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> print "Query error = %s" % err</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> sys.exit(0)</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">url = createMapUrl(circleCenterLat, circleCenterLng, ratLocations, maxNumRatsInMap)</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">numRats = len(ratLocations)</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">print "%d rats!" % numRats</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">if numRats > maxNumRatsInMap :</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> print "The map only displays the first %d rats.\n" % maxNumRatsInMap</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">os.startfile(url)</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">sys.exit(0)</span></span></span><br />
<br /> <br /><span style="font-size: large;">Note</span><br />If you get <span style="font-family: "Courier New",Courier,monospace;">err = 400 Bad Request</span> from <span style="font-family: "Courier New",Courier,monospace;">getRatLocs()</span>, check the <span style="font-family: "Courier New",Courier,monospace;">columnID</span> of the <span style="font-family: "Courier New",Courier,monospace;">Location</span> column. The city redefined the view or something like that in the portal, the <span style="font-family: "Courier New",Courier,monospace;">columnID</span> changed on me, and this example started failing. Run <span style="font-family: "Courier New",Courier,monospace;">getcolumns.py</span> on the view and compare the <span style="font-family: "Courier New",Courier,monospace;">columnID</span> it returns with the <span style="font-family: "Courier New",Courier,monospace;">columnID</span> in this code. If they differ, update the code with the ID returned by <span style="font-family: "Courier New",Courier,monospace;">getcolumns.py</span>. (You can also use your browser and look at the table in the portal and find the ID for the columns under Export tab -> API.)<br /><ul>
</ul>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-62825403817015884172011-09-04T10:56:00.000-07:002011-09-06T19:00:34.718-07:00Example #18 - Google Maps APIGoogle provides a number of APIs to use their map services. The web services API, documented on <a href="http://code.google.com/apis/maps/documentation/webservices/index.html">this page</a>, includes geocoding. Geocoding converts a street address to a latitude and a longitude. That lat/long pair can be used to do SODA geoqueries or place markers on a map.<br />
<br />
<span style="font-size: x-large;">getgeo - geocode using Google Map API</span><br />
getgeo takes an address as input and outputs the latitude and longitude. Optionally, with the verbose argument, you can also output all the info that Google returns. Take a look at that. It might be useful to your app. You get back the standardized name for the street, city, state, and country plus zip code, neighborhood name.<br />
<br />
<span style="font-size: large;">Code</span><br />
<span style="font-size: small;"><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># getgeo - Returns the geocode for a street address. Uses Google Maps API.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># Outputs either latitude/longitude or all geocoding information</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># example: python getgeo.py "2400 West Fullerton,Chicago,IL"</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># python getgeo.py "2400 west fullerton,chicago,il" -v</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">#</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># args: <street address> - must be street, city, state and comma-separated</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># and in quotes.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># -v - verbose. Will prettyprint all the information Google API returns</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># </span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">from urllib import urlencode, quote_plus</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">from urllib2 import urlopen, HTTPError, URLError</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">from json import load</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">import sys</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">import pprint</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># constants used to index into addr[]</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">street = 0</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">city = 1</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">state = 2</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">if len(sys.argv) < 2:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> sys.stderr.write("Usage: python %s \"<street,city,state>\" [-v]\n" % sys.argv[0])</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> raise SystemExit(1)</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"># addr will have [<street>, <city>, <state>]</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">addr = sys.argv[1].split(',')</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">if len(addr) != 3:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> sys.stderr.write("Usage: python %s \"<street,city,state>\" [-v]\n" % sys.argv[0])</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> raise SystemExit(1)</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">verbose = False</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">if len(sys.argv) == 3:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> if sys.argv[2] == "-v":</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> verbose = True</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> else:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> sys.stderr.write("%s is an unknown argument.\n" % sys.argv[2])</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> raise SystemExit(1)</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">url = "http://maps.googleapis.com/maps/api/geocode/json?address=" + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> quote_plus(addr[street]) + ',' + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> quote_plus(addr[city]) + ',' + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> quote_plus(addr[state]) + \</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> "&sensor=false"</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">try:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> u = urlopen(url)</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">except HTTPError, e:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> print 'The server could not handle the request and returned the HTTP error code: ', e.code</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> print 'The server returned the following error information:'</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> print e.read()</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">except URLError, e:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> print 'We failed to reach a server and the URL error reason is: ', e.reason</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">else:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> response = load(u)</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> # Check that Google sent back valid data. If so, print lat and long.</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> if response['status'] == 'OK':</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> print "%s %s" % (response['results'][0]['geometry']['location']['lat'],</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> response['results'][0]['geometry']['location']['lng'])</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> # otherwise, print the error Google returned and use verbose to show all other info</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> else:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> sys.stderr.write("Google returned an error code: %s\n" % response['status'])</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> verbose = True # turn on verbose to help with debugging</span><br style="font-family: "Courier New",Courier,monospace;" /><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> if verbose:</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> pp = pprint.PrettyPrinter(indent=4)</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> pp.pprint(response)</span></span><br />
<br />
<span style="font-size: large;">Output </span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguALZM-h41cz1splcEk5WG8I9Yc5GZOpkKKNr1v5xxE1VWLi2WiGlmtwQv8MuGoqpxim1vb-avczkfxa_jahzKL8nE44maXZJprc2qXZVaW7ijQWyB0AVHvIWPjnps523wUJfxY3eweak4/s1600/geocode+output.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="464" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguALZM-h41cz1splcEk5WG8I9Yc5GZOpkKKNr1v5xxE1VWLi2WiGlmtwQv8MuGoqpxim1vb-avczkfxa_jahzKL8nE44maXZJprc2qXZVaW7ijQWyB0AVHvIWPjnps523wUJfxY3eweak4/s640/geocode+output.jpg" width="640" /></a></div>
<br />
<span style="font-size: x-large;">Create a static Google map with markers</span><br />
You can use the lat/longs from the datasets or a filtered result to build your own map. There is a Google API documented on <a href="http://code.google.com/apis/maps/documentation/staticmaps/">this page</a>, to specify the characteristics of a map, add markers, and display it.<br />
<br />
The following example gives you an idea of what you could do with Example #17's pothole results. This code builds a map with two markers. The first one, I called it the center marker, has the lat/long of 2400 W. Fullerton. It's the center of the circle of potholes. The second one, represents the pothole at 2400 W. Medill. I arbitrarily made the center marker blue with a label of C and the pothole marker green with a label of P.<br />
<br />
<span style="font-size: large;">Code </span><br />
The program sets up all the map variables, constructs the URL, and asks the operating system to start up your default browser with the URL.<br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: small;">import os, urllib<br /><br />urlMapAPI = "http://maps.googleapis.com/maps/api/staticmap"<br />mapCenterLat = "41.9249"<br />mapCenterLong = "-87.6876"<br />mapZoomLevel = "14"<br />mapHeight = "512"<br />mapWidth = "512"<br />mapType = "roadmap"<br />centerMarkerColor = "blue"<br />centerMarkerLabel = "C"<br />mapMarkerColor = "green"<br />mapMarkerLabel = "P"<br />mapMarkerLat = "41.9240"<br />mapMarkerLong = "-87.6876"<br /><br /><br />url = \<br /> urlMapAPI + '?' + \<br /> 'center=' + mapCenterLat + ',' + mapCenterLong + \<br /> '&' + \<br /> 'size=' + mapHeight + 'x' + mapWidth + \<br /> '&' + \<br /> 'maptype=' + mapType + \<br /> '&' + \<br /> 'sensor=false' + \<br /> '&' + \<br /> 'markers=color:' + centerMarkerColor + '%7C' + 'label:' + centerMarkerLabel + '%7C' + mapCenterLat + ',' + mapCenterLong + \<br /> '&' + \<br /> 'markers=color:' + mapMarkerColor + '%7C' + 'label:' + mapMarkerLabel + '%7C' + mapMarkerLat + ',' + mapMarkerLong <br /><br />os.startfile(url)</span></div>
<br />
<span style="font-size: large;">Output</span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHD0ad6EAB7a1YQcGbyFZmzMPqdgxSK9ptcyq8d1_uk4vWsqvWOS808QiMsFSE3rwcg9nXxxFtmx6CxU9-hUyYyD7h_UCn42Snm11eCS9p3xil6BpXccPnURcc_EJjaQed4q3tnMz6JN4-/s1600/example18+output.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHD0ad6EAB7a1YQcGbyFZmzMPqdgxSK9ptcyq8d1_uk4vWsqvWOS808QiMsFSE3rwcg9nXxxFtmx6CxU9-hUyYyD7h_UCn42Snm11eCS9p3xil6BpXccPnURcc_EJjaQed4q3tnMz6JN4-/s640/example18+output.jpg" width="610" /></a></div>
<br />
This gives you an idea of how you might build a map URL dynamically with a number of markers and then have it displayed. There are many other parameters to specify the map. See the Google site for possibilities. <br />
<br />
<span style="font-size: x-large;">Summary</span><br />
This was a side trip away from the Socrata platform and API. Google has useful APIs that you can use for an app that involves locations. The utility program, <span style="font-family: "Courier New",Courier,monospace;">geocode.py</span>, can translate an address into latitude and longitude. You can use it for experimenting and testing or pull out pieces to add to your app.<br />
<br />
The second example was the minimal code needed to generate a static Google map. This starts to open up the possibilities you have for displaying locations.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-57518475693523892512011-09-03T10:17:00.000-07:002011-09-06T19:00:10.569-07:00Example #17 - Geo query<br />
If the dataset has a location column, you can use an inline filter to find rows within a geographic circle. You submit a longitude and latitude of a center point along with a radius, in meters. You'll get back rows where the location position is within that circle.<br />
<br />
Let's look for nearby potholes. Use the Chicago Data Portal's <i><a href="https://data.cityofchicago.org/Service-Requests/311-Service-Requests-Pot-Holes-Reported-2011-/7as2-ds3y">311 Service Requests - Potholes Reported (2011)</a> </i>dataset. It has a row for every pothole complaint that includes a location column.<br />
<br />
The example code was based on the SODA query shown at the bottom of <a href="http://dev.socrata.com/querying-datasets">this page</a>.<br />
<br />
The first example will find potholes within 100m of 2400 W. Fullerton Ave. That address currently appears in the fourth row in the table. We can see the latitude and longitude in the table and hard-code them in this code <br />
<br />
<span style="font-size: x-large;">Create the inline filter</span><br />
When I created the code, I cut and pasted the filter from the SODA page. I then made eight changes to the query:<br />
<ol>
<li>Changed the value for <span style="font-family: 'Courier New',Courier,monospace;">"originalViewId" </span>to <span style="font-family: 'Courier New',Courier,monospace;">7as2-ds3y</span>. This is the ID for the pothole dataset.</li>
<li>Changed the value for <span style="font-family: 'Courier New',Courier,monospace;">"name"</span> to "Nearby potholes"</li>
<li>Removed the "metadata" key/value. It seems unneeded. There's no documentation so this is a guess that seems to work out.</li>
<li>Removed the "OR" operator. Ditto.</li>
<li>Changed the value for <span style="font-family: 'Courier New',Courier,monospace;">"column"</span> to <span style="font-family: 'Courier New',Courier,monospace;">2793590</span>. This is the ID for the location column in the view.</li>
<li>Changed the value for the first <span style="font-family: 'Courier New',Courier,monospace;">"literal"</span> to <span style="font-family: 'Courier New',Courier,monospace;">41.9249</span>. This is latitude.</li>
<li>Changed the value for the second <span style="font-family: 'Courier New',Courier,monospace;">"literal"</span> to <span style="font-family: 'Courier New',Courier,monospace;">-87.6876</span>. This is longitude. I took the lat and long from the dataset for 2400 W. Fullerton Ave. It's currently the fourth row in the table. We should get at least this pothole back in our results.</li>
<li>Changed the value for the literal "1000" to "100". This is the circle radius in meters. The bigger circle returns many potholes - more than we need for this example. </li>
</ol>
<span style="font-size: x-large;">Example of simple geo query </span><br />
The rest of the code was copied from Example #16b. It will search through the dataset, find the rows with potholes within 100m of latitude 41.9249/longitude -87.6876, and print the column names along with the values for that row.<br />
<br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">import json</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">import httplib</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">import pprint</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">from sys import exit</span><br />
<span style="font-size: small;"><br /></span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">hostName = "data.cityofchicago.org"</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">service = "/views/INLINE/rows"</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">formatType = "json"</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">parameters = "method=index"</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">headers = { "Content-type:" : "application/json" }</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">query = {</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> "originalViewId": "7as2-ds3y",</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> "name": "Nearby potholes",</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> "query": {</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> "filterCondition": {</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> "type": "operator",</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> "value": "within_circle",</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> "children": </span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> [</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> {</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> "type": "column",</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> "columnId": 2793590</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> },</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> {</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> "type": "literal",</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> "value": 41.9249</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> },</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> {</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> "type": "literal",</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> "value": -87.6876</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> },</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> {</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> "type": "literal",</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> "value": 100</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> }</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> ]</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> }</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> }</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> } </span><br />
<span style="font-size: small;"><br /></span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">jsonQuery = json.dumps(query)</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">request = service + '.' + formatType + '?' + parameters </span><br />
<span style="font-size: small;"><br /></span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">conn = httplib.HTTPConnection(hostName)</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">conn.request("POST", request, jsonQuery, headers)</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">response = conn.getresponse()</span><br />
<span style="font-size: small;"><br /></span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">if response.reason != 'OK':</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> print "There was an error detected."</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> print "Response status = %s.\n" % response.status</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> print "Response reason = %s.\n" % response.reason</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> raise SystemExit(1)</span><br />
<span style="font-size: small;"><br /></span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">rawResponse = response.read()</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">jsonResponse = json.loads(rawResponse)</span><br />
<span style="font-size: small;"><br /></span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">rowNum = 1</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;">for rowData in jsonResponse['data']:</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> print "\n===== Row %d ====" % (rowNum)</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> colNum = 0</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> for columnMeta in jsonResponse['meta']['view']['columns']:</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> if columnMeta['id'] > 0:</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> print columnMeta['name'], ' = ', rowData[colNum]</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> colNum += 1</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: small;"> rowNum += 1</span><br />
<br />
<span style="font-size: large;">Output</span><br />
<span style="font-family: inherit;">74 potholes</span> are listed. Below is a screenshot of the first four<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiIBAj7Ja8wunGJz-p9Cb1b3wRe69CBRd674brkHGQHAH7YgRPQQYBHuyUCU0m2G4l9O9IFKgW6d4KnrEg7BMOTxnApMUthwFgTIwgnpPpTq00f83BqaedHN8yZROJHElnITFd4YFgfeHLX/s1600/Example+17a+output.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiIBAj7Ja8wunGJz-p9Cb1b3wRe69CBRd674brkHGQHAH7YgRPQQYBHuyUCU0m2G4l9O9IFKgW6d4KnrEg7BMOTxnApMUthwFgTIwgnpPpTq00f83BqaedHN8yZROJHElnITFd4YFgfeHLX/s640/Example+17a+output.jpg" width="596" /></a></div>
<br />
When I look through the output, I notice a number of rows have <span style="font-family: "Courier New",Courier,monospace;">STATUS = Completed - Dup</span>. These are duplicate entries for the same pothole. Since I don't want to over-report the number of potholes, a new filter is needed. Read on.<br />
<br />
<span style="font-size: x-large;">Example of more complex geo query</span><br />
This filter will do the same query as above plus filter out any rows with a status that has <span style="font-family: "Courier New",Courier,monospace;">'Dup'</span> in it. A shorthand of the filter:<br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: small;">(AND(within_circle(location, latitude, longitude, radius)),(NOT_CONTAINS(status, 'Dup')))</span></div>
<br />
<span style="font-size: large;">Inline filter</span><br />
I only changed the <span style="font-family: "Courier New",Courier,monospace;">query</span> value. The rest of the code is the same. Here's that updated <span style="font-family: "Courier New",Courier,monospace;">query</span>:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> "query": {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "filterCondition": {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "type" : "operator",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "value" : "AND",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "children" : [ {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "type": "operator",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "value": "within_circle",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "children": [ {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "type": "column",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "columnId": 2793590</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> },{</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "type": "literal",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "value": 41.9249</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> },{</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "type": "literal",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "value": -87.6876</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> },{</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "type": "literal",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "value": 100</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> } ]</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> },{</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "type" : "operator",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "value" : "NOT_CONTAINS",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "children" : [ {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "columnId" : 2776630,</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "type" : "column"</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> },{</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "type" : "literal",</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> "value" : "Dup"</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> } ]</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> } ]</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> }</span><span style="font-family: "Courier New",Courier,monospace;"> </span><br />
<br />
<span style="font-size: large;">Output </span><br />
This time only 15 rows are printed. The rows noted as <i>dup</i> are no longer included. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNMvPyWw3rwJKXXVU8PXdsa63QpABa8JAFVC7HrMQnOAdnBXKo2eakmXq7uyfnaFa5rhzYcLQJy1KIcpB-gfaTEXxXw3_rrotntk3XNcSc4WOK6UIonyOIxRkO-lSv9mOq9o7KR4xvUHUy/s1600/Example+17b+output.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNMvPyWw3rwJKXXVU8PXdsa63QpABa8JAFVC7HrMQnOAdnBXKo2eakmXq7uyfnaFa5rhzYcLQJy1KIcpB-gfaTEXxXw3_rrotntk3XNcSc4WOK6UIonyOIxRkO-lSv9mOq9o7KR4xvUHUy/s640/Example+17b+output.jpg" width="552" /></a></div>
<br />
If you study the output, there is a column for Number of Potholes Filled on Block. This has values ranging from 0 to 40. The filter could be expanded to skip rows with zero potholes filled. Your app might do something special with streets where many potholes were filled. This example doesn't need to go that far but you can start thinking of the possibilities.<br />
<br />
I'm also noticing other oddities in the data. Potholes filled on 2400 N Western have the same lat and long as potholes filled on 2400 W. Fullerton. So, beware of the data.<br />
<br />
<span style="font-size: x-large;">Summary</span><br />
You can do geo-queries on data sets with a location field. Pick an address, geocode it to get the lat/long, and find all rows within a radius from that address. The first program in this example showed how that is done. The second program added a test on another column in the dataset to refine the search results. Take a look at the Example #18 for more on geocoding.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-52753295141034088722011-09-01T16:38:00.000-07:002011-09-08T18:23:41.968-07:00Example #16 - Complex query using inline filterExample #15 showed how to use a filtered view to do a query. There is a second approach: the query is in your code. There is no filtered view stored at the data portal. Your code creates a temporary filter and use it to get your results. This is called an inline filter.<br />
<br />
We'll start with an example to find all payments for consulting services made by Cook County in December 2010. Our filter will search the <i>Product Desritpion NIGP Code </i>(sic) column for all rows equal to 918, the code for consulting services.<br />
<br />
<span class="Apple-style-span" style="font-size: x-large;">Get the IDs</span><br />
You'll need the ID of the data set you wish to search. This will be the <span style="font-family: 'Courier New',Courier,monospace;">originalViewID</span>. You'll also need the <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">columnID</span> for the column(s) in your query. There are a couple ways to get that info. <br />
<br />
<span class="Apple-style-span" style="font-size: large;">Use your browser</span><br />
You can quickly find these values using your web browser. Open the <i><a href="http://datacatalog.cookcountyil.gov/dataset/Chkregist-DEC2010/e9qr-rmq4">Chkregist DEC2010</a></i> dataset in your browser, click on the <b>Export</b> button, and click on the <b>API </b>tab. You'll find the <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">originalViewID</span> in the <b>API Access Endpoint URL</b> near the end of the string. You can find the <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">columnID</span>s a little farther down in the tab. Here's a screenshot.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0fE4sgKoUm-AHroarMgW21vmnHoDDmhDKtSWBrA93z9JOYuGG3OrklJ1a_jeQHTZ2tGR6umEebV2AnPUoERnF2MBWA8m6gA4rFTgd02w3sMAu_cI_iLIrGcPRLJ6q32O8fROq8sWLXtfw/s1600/Browser+screenshot.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="568" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0fE4sgKoUm-AHroarMgW21vmnHoDDmhDKtSWBrA93z9JOYuGG3OrklJ1a_jeQHTZ2tGR6umEebV2AnPUoERnF2MBWA8m6gA4rFTgd02w3sMAu_cI_iLIrGcPRLJ6q32O8fROq8sWLXtfw/s640/Browser+screenshot.jpg" width="640" /></a></div>
<br />
<span class="Apple-style-span" style="font-size: large;">Use code</span><br />
Another way to get the ID for the column is to write code to dig them out of the view's metadata. Here's a small program to print out the column names and column IDs for the <i>Chkregist DEC2010</i> dataset on the Cook County Data Portal.<br />
<br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">from urllib2 import urlopen</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">from json import load</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"><br /></span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">url = 'http://datacatalog.cookcountyil.gov/api/views/e9qr-rmq4/rows.json'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">u = urlopen(url)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">response = load(u)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"><br /></span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">for columnMeta in response['meta']['view']['columns']:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"><span class="Apple-tab-span" style="white-space: pre;"> </span> print columnMeta['id'], ' : ', columnMeta['name']</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"><br /></span><br />
Here's the output. For each row in the table, you'll see the column ID and the column name. The first eight columns in the view are metadata. They have an ID of -1.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigb7FK8bqbSwRg-7Cd2I0srRoHgy2xuqFlZoft_AP66xif3gSB4m-Xk9mDAlbrIXd7gvfcoEjehmzW-CxwgtbCczuBZYnxFDwZNcWjpmu4-NMRij4u5oZOoZjQiVbn2xfTOR6w6jHVJe8E/s1600/Example16a+output.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="348" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigb7FK8bqbSwRg-7Cd2I0srRoHgy2xuqFlZoft_AP66xif3gSB4m-Xk9mDAlbrIXd7gvfcoEjehmzW-CxwgtbCczuBZYnxFDwZNcWjpmu4-NMRij4u5oZOoZjQiVbn2xfTOR6w6jHVJe8E/s640/Example16a+output.jpg" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
From the browser, the <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">originalViewID</span> is <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">e9qr-rmq4</span>. From the browser or the code, the Product Code <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">columnID</span> is <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">2700017</span>.<br />
<br />
<span class="Apple-style-span" style="font-size: x-large;">Creating the query</span><br />
I used the Socrata example about halfway down on <a href="http://dev.socrata.com/querying-datasets">this page</a> as a guide to create the query. There's very little documentation that I could find on how to write an inline filter. This example seems to be the simplest possible. It using a prefix operator with two operands to request rows that have the value 918 in the column holding the Product Codes. Here's the Python snippet:<br />
<br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">query = {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "originalViewId" : "e9qr-rmq4",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "name" : "Inline Filter",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "query" : {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "filterCondition" : {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "type" : "operator",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "value" : "EQUALS",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "children" : [</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> { </span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "columnId" : 2700017,</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "type" : "column"</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> },</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "type" : "literal",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "value" : "918"</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> }</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> ]</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> }</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> }</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> }</span><br />
<br />
<span class="Apple-style-span" style="font-size: x-large;">Example program with an inline filter</span><br />
Here's a program to run the inline filter. For each row with consulting services, it prints out a separator with the count along with the column name and their values. This was lifted from a previous example. Currently,<br />
the filter produces 47 rows. Translation: the county wrote 47 checks in December 2010 for consulting services.<br />
<br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">import httplib</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">import json</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">import pprint</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"><br /></span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">hostName = "datacatalog.cookcountyil.gov"</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">service = "/views/INLINE/rows"</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">formatType = "json"</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">parameters = "method=index"</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">headers = { "Content-type:" : "application/json" }</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">query = {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "originalViewId" : "e9qr-rmq4",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "name" : "Inline Filter",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "query" : {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "filterCondition" : {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "type" : "operator",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "value" : "EQUALS",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "children" : [</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> { </span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "columnId" : 2700017,</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "type" : "column"</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> },</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "type" : "literal",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "value" : "918"</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> }</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> ]</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> }</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> }</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> }</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"><br /></span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">jsonQuery = json.dumps(query)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">request = service + '.' + formatType + '?' + parameters </span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"><br /></span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">conn = httplib.HTTPConnection(hostName)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">conn.request("POST", request, jsonQuery, headers)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">response = conn.getresponse()</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"><br /></span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">if response.reason != 'OK':</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>print "There was an error detected."</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>print "Response status = %s.\n" % response.status</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>print "Response reason = %s.\n" % response.reason</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>raise SystemExit(1)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"><br /></span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">rawResponse = response.read()</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">jsonResponse = json.loads(rawResponse)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"><br /></span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">rowNum = 1</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">for rowData in jsonResponse['data']:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print "\n======== %d =======" % (rowNum)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> colNum = 0</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> for columnMeta in jsonResponse['meta']['view']['columns']:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> if columnMeta['id'] > 0:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print columnMeta['name'], ' = ', rowData[colNum]</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> colNum += 1</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> rowNum += 1</span><br />
<br />
<span class="Apple-style-span" style="font-size: large;">Coding the filter</span><br />
Asking the API to perform an inline filter is quite a bit different from the past API requests. You must:<br />
<ul>
<li>Use an HTTP <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">POST</span>. All previous examples were HTTP <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">GET</span>.</li>
<li>Send an HTTP header with a specific key/value pair.</li>
<li>Send the query as a <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">JSON</span> or <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">XML</span> document </li>
</ul>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">urllib2.urlopen()</span> doesn't give us the flexibility we need. There's another library, <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">httplib</span>, that has a class, <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">HTTPConnection()</span>, which works nicely. Details can be found in the <a href="http://docs.python.org/library/httplib.html?highlight=httplib#httplib.HTTPConnection">Python documentation</a>.<br />
<br />
<span class="Apple-style-span" style="font-size: large;">Comments on the code</span><br />
<ul>
<li><span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">json.dumps()</span> does the work of turning query into its equivalent <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">JSON</span> format which the API will accept.</li>
<li><span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">request</span> is just the concatenation of the pieces that make up the API request. I kept those components as individual variables to make it easier to change or have a fancier program modify.</li>
<li><span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">HTTPConnection()</span> sets up the connection to the portal. This has to be in place before we make a <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">request()</span>.</li>
<li><span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">conn.request()</span> sends the API request. We tell it to use a <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">POST</span>. Send the API request string we've built. Send the JSON-formatted query. Send the right headers. Use the key/value pair shown.</li>
<li><span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">conn.getresponse()</span> provides a handle to the response from the API service. We use that later to <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">read()</span> the response - get all the data sent back from the API.</li>
<li><span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">response.reason</span> is returned by the API server and will indicate if there were problems.</li>
<li><span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">loads()</span> turns the what the API sent back into JSON for easier processing.</li>
<li>the last section prints out the rows that matched the filter. This code is almost identical to a script used in a previous example.</li>
</ul>
<span class="Apple-style-span" style="font-size: large;">Output</span><br />
Currently, 47 rows match the filter and are printed. Here's the output of the first four:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_G7-0lISMvqkgukaPNtwFHQlx_ioJLhEBkaAc198kJoEN60XswSHxd-GIZrvG4AeE7nWklYXMeAOiR5oOUHERedFj87RUw5Sc0ewLmTb7fp83IDnz07QAdpyMe2yyOVrroD02_RV7pHcE/s1600/Example16b+output.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="530" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_G7-0lISMvqkgukaPNtwFHQlx_ioJLhEBkaAc198kJoEN60XswSHxd-GIZrvG4AeE7nWklYXMeAOiR5oOUHERedFj87RUw5Sc0ewLmTb7fp83IDnz07QAdpyMe2yyOVrroD02_RV7pHcE/s640/Example16b+output.jpg" width="640" /></a></div>
<br />
<span class="Apple-style-span" style="font-size: x-large;">Example program with a more complex filter</span><br />
Here's the same program with a more complex query. This time we'll add the condition that the payment needs to be greater than $10,000. So, our filter is going to look for rows where Product Code = 918 and Payment > 10000.<br />
<br />
Here's the filter. The rest of the code is the same.<br />
<br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">query = {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "originalViewId" : "e9qr-rmq4",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "name" : "Inline Filter",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "query" : {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "filterCondition" : {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "type" : "operator",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "value" : "AND",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"><span class="Apple-tab-span" style="white-space: pre;"> </span> "children": [ {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "type" : "operator",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "value" : "EQUALS",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "children" : [ { </span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "columnId" : 2700017,</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "type" : "column"</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> }, {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "type" : "literal",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "value" : "918"</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> } ]</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> }, {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "type" : "operator",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "value" : "GREATER_THAN",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "children" : [ { </span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "columnId" : 2700027,</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "type" : "column"</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> }, {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "type" : "literal",</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> "value" : "10000"</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> } ]</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> } ]</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> }</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> }</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> }</span><br />
<br />
<span class="Apple-style-span" style="font-size: large;">Output</span><br />
Here are the first four rows that match. There are total of 18 rows that match this filter.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgii5lZqkmp8qK1VVtZGNm1JHgrqxVdrgXtAMcdgevCwsv1BYAo6Ovw-UomrxjX-SLCJNg2Jqg-lr7W0zsLcKbS7M2Tx3BHpfIqwwRI8ibvKchplcAm7oWiA3ulJNHYu9OFHUPH6h_szV4B/s1600/Example16c+output.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgii5lZqkmp8qK1VVtZGNm1JHgrqxVdrgXtAMcdgevCwsv1BYAo6Ovw-UomrxjX-SLCJNg2Jqg-lr7W0zsLcKbS7M2Tx3BHpfIqwwRI8ibvKchplcAm7oWiA3ulJNHYu9OFHUPH6h_szV4B/s640/Example16c+output.jpg" width="624" /></a></div>
<br />
<span class="Apple-style-span" style="font-size: x-large;">Alter the filter with your code</span><br />
The above examples all used a fixed inline filter. You hard-coded the query and ran it. Most apps will need to let the user set the query parameters. Let's peek at how your app could adjust the query based on what the user enters.<br />
<br />
You could have an app that asks the user for the payment threshold for consulting services. Let's say they enter, $60,000. You could then alter the filter to look for payments greater than that threshold and then send the filter in a request to the API.<br />
<br />
These two lines demonstrate how you could make this work. Place these two lines after <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">query</span> is defined and before <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">jsonQuery</span> is set.<br />
<br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">paymentThreshold = "60000"</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">query['query']['filterCondition']['children'][1]['children'][1]['value'] = paymentThreshold</span><br />
<br />
<br />
I've set the value for the <span style="font-family: "Courier New",Courier,monospace;">paymentThreshold</span>. In an app, you would get that from the user. The important line is the second one. It shows how you can set a new value to use in the test for the payment amount.<br />
<br />
<span class="Apple-style-span" style="font-size: large;">Output</span><br />
When I run that code, here's the output. There are only five rows with payments over $60,000 for consulting services.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJm5KwUa-1yPCTEqRdhfM3Ls0lSjLh4uFKZOSHYQ0nJcbOp31jO4eg37wgrLIgCrOMOmqOSpqAWYqGzj1XwiJtCJ5gEHR3CkmhrNr0_iWpEjcaRAEXHa4l3fHrTgEFfgxEN4_10aukHQ5t/s1600/Example16d+output.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJm5KwUa-1yPCTEqRdhfM3Ls0lSjLh4uFKZOSHYQ0nJcbOp31jO4eg37wgrLIgCrOMOmqOSpqAWYqGzj1XwiJtCJ5gEHR3CkmhrNr0_iWpEjcaRAEXHa4l3fHrTgEFfgxEN4_10aukHQ5t/s640/Example16d+output.jpg" width="424" /></a></div>
<br />
<span class="Apple-style-span" style="font-size: large;">Compare to web UI filter</span><br />
Something looks fishy here. The first three rows are identical. The last two are identical. I dug into the view and those five rows are all in the table. I also set up the same filter using the web UI to doublecheck. Yep, those rows all appear in the data. It's not the code going awry.<br />
<br />
Here's a screenshot with the results of the same filter using the web browser. The same five rows are displayed.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh54HZthMSAjZ0nIG539L1GJBNn1JoqwQbQVxMUHgrhYGP8Y7v4YfxGp0DTFmoVBAo_T_ejZJxgq_iwybL_pDgtvvKadA0OfYMvukSgGTYBPHmhwu58Jk1N6sJTWdk4bZjojc2ngG5wujCI/s1600/Browser+screenshot+for+Example+16d.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="310" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh54HZthMSAjZ0nIG539L1GJBNn1JoqwQbQVxMUHgrhYGP8Y7v4YfxGp0DTFmoVBAo_T_ejZJxgq_iwybL_pDgtvvKadA0OfYMvukSgGTYBPHmhwu58Jk1N6sJTWdk4bZjojc2ngG5wujCI/s640/Browser+screenshot+for+Example+16d.jpg" width="640" /></a></div>
<br />
<br />
<span class="Apple-style-span" style="font-size: x-large;">Summary</span><br />
If your app will use the API to query data sets and you want to dynamically set the parameters of the filter, you will need to use an inline filter. There were two examples of filters provided. The first had a simple condition - find all rows with a specific product code. The second example was a little more complicated - find all rows with a specific product code and had a payment amount above a given threshold. Finally, a third example showed the line of code that could be used to adjust the threshold for the payment amount on the fly.<br />
<br />
There's little documentation on inline filters. The SODA site recommends creating a filter using the web UI and then looking at the query that was generated. You can write a small program to get the filtered view you create. Then, you look through it to find and study the query. I did this for several to get a better understanding of the operators and structure of a filter. It's not easy. Making it worse, the UI-generated queries throw in seemingly superfluous operators and operands. Without documentation, it's trial and error to get a filter working.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-28036893559764997362011-08-31T18:57:00.000-07:002011-09-06T18:34:29.737-07:00Example #15 - Complex query using a filtered viewSo far, I've shown examples of querying for a word or phrase in name field or the description field. Most apps will need more complicated queries. <br />
<br />
Socrata has a <a href="http://dev.socrata.com/querying-datasets">page</a> dedicated to this topic. It's the basis for this example and the next example.<br />
<br />
There's a new term involved - <b>filtered</b>. In the Socrata web browser interface where you are using a data portal you can create a filter by specifying which rows to include based on their values, summing values in rows, and sorting rows. It's the way to do the operations to build or find the rows that meet your needs.<br />
<br />
There's a simple way to do a complex query. Socrata calls it piggybacking on an existing filtered view. Use your web browser, go to the portal, build your query, save it as a view, and in your code you specify the <span style="font-family: 'Courier New',Courier,monospace;">viewID</span> for that query. The result: your code calls the stored filter. That filter is just another view. The advantage: it is quick and you already know how to write the code to request a view. There are a couple of disadvantages: 1) It's fixed. Your app can't modify the query. 2) Your code is in two places. Your query is stored at the portal and your app calls that query. You might be maintaining things in two places instead of <br />
one.<br />
<br />
<span style="font-size: x-large;">Create your filter</span><br />
We'll use the <i>Chkregist DEC2010</i> data set on the Cook County Data Portal. Here's a screenshot of it. You can find the <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">viewID</span> in the URL in the URL bar at the top of the web browser. It is <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">e9qr-rmq4</span>.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWwi43JjT_cZPIWsXZRLsPHzVS9T-HiLr_CcDBWra7B9Cc6NvawB6YIDyg-geHDOw3cq6D-YgT3JziRnPdw7gBbuEu33tijJTBXlRuMAnjvv5kFfdTL0NlWhrlpU28VIJEIMeZcgcT4Dzn/s1600/Chkreg+dataset.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="378" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWwi43JjT_cZPIWsXZRLsPHzVS9T-HiLr_CcDBWra7B9Cc6NvawB6YIDyg-geHDOw3cq6D-YgT3JziRnPdw7gBbuEu33tijJTBXlRuMAnjvv5kFfdTL0NlWhrlpU28VIJEIMeZcgcT4Dzn/s640/Chkreg+dataset.jpg" width="640" /></a></div>
<br />
<br />
Let's find the checks for construction services between $1m and $10m. This screenshot shows the filter parameters on the right. I'm asking for all rows that have a product description of 912 (this means construction services) and an amount between $1m and $10m. After creating it, I saved the filtered view as <i>Construction over 1m</i>. You can see the <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">viewID</span> Socrata assigned to my filtered view in the URL in the URL bar at the top of the web browser. It is <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">2wek-2jap</span>.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjh0cEE_psnMvVFbM4iqMAkMOnvQOtJQMH2gxv814ctT9XvcHvUpT6T0bnryYPDjUQqeSJw3HYjks7cjIu_H7eqpp0Zr7hdivXiT7GaeJbc_zMTgXWSzeQEjqVJWMoEVFxKJcUhtbsNoV-7/s1600/Screen+shot+1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="378" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjh0cEE_psnMvVFbM4iqMAkMOnvQOtJQMH2gxv814ctT9XvcHvUpT6T0bnryYPDjUQqeSJw3HYjks7cjIu_H7eqpp0Zr7hdivXiT7GaeJbc_zMTgXWSzeQEjqVJWMoEVFxKJcUhtbsNoV-7/s640/Screen+shot+1.jpg" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<span style="font-size: x-large;">Use API to retrieve filtered results </span><br />
Now that the filtered view is in place you can access it from a program. Here's the code to get all the rows from the filtered view, load the JSON into an array, and print the array. This should look familiar.<br />
<br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;">from urllib2 import urlopen<br />
from json import load<br />
import pprint<br />
<br />
url = 'http://datacatalog.cookcountyil.gov/api/views/2wek-2jap/rows.json'<br />
u = urlopen(url)<br />
response = load(u)<br />
<br />
pp = pprint.PrettyPrinter(indent=4)<br />
pp.pprint(response)</span><br />
<br />
You get back the data in the rows along with the metadata for columns. I ran it by typing <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">python example15a.py > out.txt</span> and then looked at <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">out.txt</span> in a text editor.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwKPdAGAokV-rtR_W5Ko7qk-yxvgxLx3Wx7nTL6Kzzhyphenhyphenf9UDaw9ZA1fZcnaN9khtnlOUV37AWkIAKs5j7ssOAYHSuSknTa4fBGFscOP0JhwBiHRYgTmZKxv4IpnZAtfXD9qSB-2G25q2kv/s1600/example15a+output.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="588" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwKPdAGAokV-rtR_W5Ko7qk-yxvgxLx3Wx7nTL6Kzzhyphenhyphenf9UDaw9ZA1fZcnaN9khtnlOUV37AWkIAKs5j7ssOAYHSuSknTa4fBGFscOP0JhwBiHRYgTmZKxv4IpnZAtfXD9qSB-2G25q2kv/s640/example15a+output.jpg" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
The value for <span style="font-family: 'Courier New',Courier,monospace;">'data'</span> is a list with two elements, one for each row in our filtered view. Each of those elements, in turn, has a list for the values of the columns in the row. As you'll see next, the first eight are metadata values. There are eight hidden columns each having metadata for a row.<br />
<br />
After the <span style="font-family: 'Courier New',Courier,monospace;">'data'</span> list, is <span style="font-family: 'Courier New',Courier,monospace;">'meta'</span> which contains metadata for the view. <span style="font-family: 'Courier New',Courier,monospace;">'view'</span> is a dictionary and one of its keys is <span style="font-family: 'Courier New',Courier,monospace;">'columns'</span> with a value of a list with elements for each column. Each of those list elements has the metadata for the column. We care about a couple of those metadata elements: 1) <span style="font-family: 'Courier New',Courier,monospace;">'id'</span> - if it is -1 then this is a metadata column; and 2) <span style="font-family: 'Courier New',Courier,monospace;">'name'</span> - the name of the column. Translation: we now know the names of each column in the view. We also know which of these columns are hidden from the web browser UI. Turns out the first eight columns are hidden.<br />
<br />
<span style="font-size: large;">Program to get filtered view column names and data </span><br />
We can use our understanding of the API response to create a small program, <span style="font-family: 'Courier New',Courier,monospace;">example15b.py</span>, to load up the JSON data for the filtered view and print out the column name along with its value for each row. Here's the code:<br />
<span style="font-size: small;"><br /></span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;">from urllib2 import urlopen</span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;">from json import load</span><br />
<span style="font-size: small;"><br /></span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;">url = 'http://datacatalog.cookcountyil.gov/api/views/2wek-2jap/rows.json'</span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;">u = urlopen(url)</span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;">response = load(u)</span><br />
<span style="font-size: small;"><br /></span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;">rowNum = 1</span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;">for rowData in response['data']:</span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print "\n===== Row %d ====" % (rowNum)</span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;"> colNum = 0</span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;"> for columnMeta in response['meta']['view']['columns']:</span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;"> if columnMeta['id'] > 0:</span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print columnMeta['name'], ' = ', rowData[colNum]</span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;"> colNum += 1</span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;"> rowNum += 1</span><br />
<br />
The outer loop iterates through the two rows returned in <span style="font-family: 'Courier New',Courier,monospace;">'data'</span>. For each of those rows, the inner loop iterates through each column's metadata. If a column has a negative value for <span style="font-family: 'Courier New',Courier,monospace;">'id'</span>, then it's metadata and we won't print it. (Note, if you want to see the hidden metadata and its value for each column, just comment out that <span style="font-family: 'Courier New',Courier,monospace;">if </span>statement.) Otherwise, print the name of the column along with it's corresponding value from the <span style="font-family: 'Courier New',Courier,monospace;">'data'</span> array. Here's the output.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjM-fAo0lpwhM7yCHGNeUXmv7d9JRNEFRM2W9Z3ol4l3JiAuGeUjORkezCTItJcklRLenG_pU_a6uVO0rJSMprxGIHP9xcBRFVWxSbwxUBuQ1XM-TEjH1LtQQX7a4Taxk01gv_DuAl0EUrg/s1600/example15b+output.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="468" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjM-fAo0lpwhM7yCHGNeUXmv7d9JRNEFRM2W9Z3ol4l3JiAuGeUjORkezCTItJcklRLenG_pU_a6uVO0rJSMprxGIHP9xcBRFVWxSbwxUBuQ1XM-TEjH1LtQQX7a4Taxk01gv_DuAl0EUrg/s640/example15b+output.jpg" width="640" /></a></div>
<br />
This program lays the groundwork for you to create a program that could explore the column names and, on the fly, give the user column names and values. You have both the column metadata and the row data.<br />
<br />
<span style="font-size: large;">Program to get just the filtered data </span><br />
If the above is too much bother and you just want to get your hands on the data values. You can ask the API to send back just the data. Your code would then do things like hard code that the 11th element of the <span style="font-family: 'Courier New',Courier,monospace;">'data'</span>list is the <i>Company Name</i>. By specifying <span style="font-family: 'Courier New',Courier,monospace;">meta=false</span> parameter on your API request, the <span style="font-family: "Courier New",Courier,monospace;">views </span>service will return just the data. Here's the code:<br />
<br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;">from urllib2 import urlopen</span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;">from json import load</span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;">import pprint</span><br />
<span style="font-size: small;"><br /></span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;">url = 'http://datacatalog.cookcountyil.gov/api/views/2wek-2jap/rows.json?meta=false'</span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;">u = urlopen(url)</span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;">response = load(u)</span><br />
<span style="font-size: small;"><br /></span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;">pp = pprint.PrettyPrinter(indent=4)</span><br />
<span style="font-family: 'Courier New',Courier,monospace; font-size: small;">pp.pprint(response)</span><br />
<br />
Here's the output of just the data.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8ac8MsuX7J38xi9RiNd-tZjBQt_Ig1vq-VIxI-PDJaFwRlgFYetUoXbktT65vShuw1gzH8CvJuNUEYwavKXF2ds_1k2hdp02l8gojjg5FtKhowWFeIwbHuWqCK2UUqZ6CYwu1J4e459Wy/s1600/example15c+output.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="454" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8ac8MsuX7J38xi9RiNd-tZjBQt_Ig1vq-VIxI-PDJaFwRlgFYetUoXbktT65vShuw1gzH8CvJuNUEYwavKXF2ds_1k2hdp02l8gojjg5FtKhowWFeIwbHuWqCK2UUqZ6CYwu1J4e459Wy/s640/example15c+output.jpg" width="640" /></a></div>
<br />
<br />
<span style="font-size: x-large;">Summary</span><br />
This is one approach to running a complex query. You can use the web browser UI to create your filtered view and save it. You can write a simple program to get back the data. You can write a more complicated program to plow through the metadata and data to respond to the user's needs.<br />
<br />
With a filtered view, you can't change the specifics of the query. If I wanted to find construction services over $10,000 or if I wanted to find consulting services over $1m, I'd have to create a new filter, get it's <span style="font-family: "Courier New",Courier,monospace;">viewID</span>, and put it in the code.<br />
<br />
The next example shows how to do a query without a filtered view. It gives you the ability to create a query or change the parameters of a query in your code.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-43843371215716429822011-08-31T15:05:00.000-07:002011-09-06T18:39:08.098-07:00Example #14 - Error handlingFor a real app, you'll need to do some error checking and handling on your API requests. This example shows one approach.<br />
<br />
You need to check that the <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">urlopen()</span> call worked. If it didn't, you either couldn't reach the web site or you had a bad service request. Reasons you might not reach the web site include: you misspelled the data portal URL, the portal is down, or you aren't connected to the network. A bad service request would typically be you misspelled it or (rare but I suppose it could happen) that service is down.<br />
<br />
We'll use Python's exception for the <span style="font-family: "Courier New",Courier,monospace;">urllib2 urlopen()</span> function. <span style="font-family: "Courier New",Courier,monospace;">urlopen()</span> sends back errors by raising exceptions and providing us some details to start the debugging. You'll see we will try the<span style="font-family: "Courier New",Courier,monospace;"> urlopen() </span>and check for two different exceptions: <span style="font-family: "Courier New",Courier,monospace;">HTTPError</span> and <span style="font-family: "Courier New",Courier,monospace;">URLError</span>.<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">HTTPError</span> tells us something is wrong with what we sent the server. The server got the request but had problems with it. You can find details on the various error codes about halfway down on this <a href="http://docs.python.org/library/httplib.html">Python reference page</a>. There's a table of the values and a link to more info.<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">URLError</span> tells us something is wrong with the URL we are using. The server didn't get the request. It didn't get that far.<br />
<br />
<span class="Apple-style-span" style="font-size: x-large;">Example Code</span><br />
Four snippets are provided: the first without error and the next three have different sorts of errors in the API request.<br />
<br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">from urllib2 import urlopen, URLError, HTTPError</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"><br />
</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"># 1 - Good URL and service name</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">print '\nHere\'s what happens with a good URL.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">url = 'http://data.cityofchicago.org/api/views.json'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">try:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> u = urlopen(url)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">except HTTPError, e:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'The server could not handle the request.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'Error code: ', e.code</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">except URLError, e:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'We failed to reach a server.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'Reason: ', e.reason</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">else:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'OK'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"><br />
</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"># 2 - Bad URL</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">print '\nHere\'s what happens with a bad URL.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">url = 'http://XYZ.cityofchicago.org/api/views.json'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">try:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> u = urlopen(url)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">except HTTPError, e:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'The server could not handle the request.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'Error code: ', e.code</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">except URLError, e:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'We failed to reach a server.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'Reason: ', e.reason</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">else:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'OK'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"><br />
</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"># 3 - Bad service name</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">print '\nHere\'s what happens with a good URL but bad API service name.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">url = 'http://data.cityofchicago.org/api/XYZ.json'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">try:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> u = urlopen(url)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">except HTTPError, e:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'The server could not handle the request.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'Error code: ', e.code</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">except URLError, e:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'We failed to reach a server.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'Reason: ', e.reason</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">else:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'OK'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"><br />
</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"># 4 - Bad parameter</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">print '\nHere\'s what happens with a good URL and good API service name but bad parameter.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">url = 'http://data.cityofchicago.org/api/views.json?XYZ=2'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">try:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> u = urlopen(url)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">except HTTPError, e:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'The server could not handle the request.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'Error code: ', e.code</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">except URLError, e:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'We failed to reach a server.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'Reason: ', e.reason</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;">else:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace; font-size: small;"> print 'OK'</span><br />
<br />
<span class="Apple-style-span" style="font-size: x-large;">Explanations of each snippet</span><br />
For each of the four snippets, here's their few lines of code, commentary, and output.<br />
<br />
<span class="Apple-style-span" style="font-size: large;">Good URL</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"># 1 - Good URL and service name</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">print '\nHere\'s what happens with a good URL.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">url = 'http://data.cityofchicago.org/api/views.json'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">try:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> u = urlopen(url)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">except HTTPError, e:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'The server could not handle the request.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'Error code: ', e.code</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">except URLError, e:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'We failed to reach a server.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'Reason: ', e.reason</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">else:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'OK'</span><br />
This is the normal case. You've seen this URL several times before. We're requesting all the views from the city of Chicago data portal. We try to open the URL and since the output shows, <i>OK</i>, we skipped the exceptions - so, no errors.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNphhZ36Z6YrSuqToK3bDcZ8hef8Jg8MXcTB0ava9yPEOwfcyzZoCmzmDOUQJ85qdY_wcdBpXZvr2bcivLQuS9XM0TihbTHQBKN1RZwcLoCgxzy2Silej5_bPgGjgSK-cCEsf-B1e6VuhB/s1600/Snippet+1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="90" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNphhZ36Z6YrSuqToK3bDcZ8hef8Jg8MXcTB0ava9yPEOwfcyzZoCmzmDOUQJ85qdY_wcdBpXZvr2bcivLQuS9XM0TihbTHQBKN1RZwcLoCgxzy2Silej5_bPgGjgSK-cCEsf-B1e6VuhB/s640/Snippet+1.jpg" width="640" /></a></div>
<br />
<span class="Apple-style-span" style="font-size: large;">Bad URL</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"># 2 - Bad URL</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">print '\nHere\'s what happens with a bad URL.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">url = 'http://XYZ.cityofchicago.org/api/views.json'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">try:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> u = urlopen(url)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">except HTTPError, e:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'The server could not handle the request.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'Error code: ', e.code</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">except URLError, e:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'We failed to reach a server.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'Reason: ', e.reason</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">else:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'OK'</span><br />
Here, the portal URL is wrong. There is no web site at <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">XYZ.cityofchicago.org</span>. Since this is a garbage URL, we end up with a <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">URLError</span> exception and we print out our own error message followed by the reason code from the Python <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">urllib2</span> module.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkJmes_Rs-SHo9qjZxB86nArPTI3nAl9DQX5_RUtP0irtn4CfAd_hX2VRKlkzmntKfupLoSDDAhXAftwM0-vpl4dNgUuKkPRqQktnoa0PBIhBh_1WhkNya16yYnN7JIpfO37qgWkgC3dhc/s1600/Snippet+2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="46" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkJmes_Rs-SHo9qjZxB86nArPTI3nAl9DQX5_RUtP0irtn4CfAd_hX2VRKlkzmntKfupLoSDDAhXAftwM0-vpl4dNgUuKkPRqQktnoa0PBIhBh_1WhkNya16yYnN7JIpfO37qgWkgC3dhc/s640/Snippet+2.jpg" width="640" /></a></div>
<br />
<span class="Apple-style-span" style="font-size: large;">Bad service name</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"># 3 - Bad service name</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">print '\nHere\'s what happens with a good URL but bad API service name.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">url = 'http://data.cityofchicago.org/api/XYZ.json'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">try:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> u = urlopen(url)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">except HTTPError, e:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'The server could not handle the request.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'Error code: ', e.code</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">except URLError, e:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'We failed to reach a server.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'Reason: ', e.reason</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">else:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'OK</span><br />
The URL is correct but the API service name is wrong. There is no service called <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">/api/XYZ</span>. When this runs, the Chicago portal gets the request. However, it balks because it doesn't know what to do for <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">/api/XYZ</span>. We have an <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">HTTPError</span>. The server doesn't know what to do with the HTTP we sent. We print out our error message followed by the error code from the Python <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">urllib2</span> module.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBYQsUskmkpdsWrR10ZF7SQnMNvu1KgNczPXJg9Z0siF7QI4RciM0feBGVZPLU0t3G0ZE1BVPocXdnv4f2RefZiaZxmqbini_Qzoq8hDALYQD-1KPROe5FGSqnVy5s_LJogoAdDZFK-dpd/s1600/Snippet+3.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="50" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBYQsUskmkpdsWrR10ZF7SQnMNvu1KgNczPXJg9Z0siF7QI4RciM0feBGVZPLU0t3G0ZE1BVPocXdnv4f2RefZiaZxmqbini_Qzoq8hDALYQD-1KPROe5FGSqnVy5s_LJogoAdDZFK-dpd/s640/Snippet+3.jpg" width="640" /></a></div>
<br />
<span class="Apple-style-span" style="font-size: large;">Bad parameter</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"># 4 - Bad parameter</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">print '\nHere\'s what happens with a good URL and good API service name but bad parameter.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">url = 'http://data.cityofchicago.org/api/views.json?XYZ=2'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">try:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> u = urlopen(url)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">except HTTPError, e:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'The server could not handle the request.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'Error code: ', e.code</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">except URLError, e:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'We failed to reach a server.'</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'Reason: ', e.reason</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">else:</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print 'OK'</span><br />
Last thing to check - what happens when you send a bad parameter in an API request? Here, we send <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">/api/views</span> a name/value parameter of <span style="font-family: "Courier New",Courier,monospace;">XYZ =2</span>. <span style="font-family: "Courier New",Courier,monospace;">XYZ</span> is bogus. As the output shows, this does not cause any exception. The URL is good. The service name is good. As far as <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">urllib2.urlopen()</span> is concerned everything worked properly. The service would need to send back an error message to us complaining about an unknown name. The API just ignores the bad parameter. (Want to see? Add one more line to the code,<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"> print u.read()</span>, and you will see in the output there's no indication of an error.) <br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhAI-8xLVWihULSixQhvazCcPZ479iM-GlanlhCx-8iMu6D0EwKhzj_2u3UXhlXDYiwZJE4yfLAgZ4rh7syEE6kiUWfkGz0rmjKCHf4sf2aVMv-0vmkOx6miG2O8GKDBwjkqYr-pmmFOw8/s1600/Snippet+4.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="38" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhAI-8xLVWihULSixQhvazCcPZ479iM-GlanlhCx-8iMu6D0EwKhzj_2u3UXhlXDYiwZJE4yfLAgZ4rh7syEE6kiUWfkGz0rmjKCHf4sf2aVMv-0vmkOx6miG2O8GKDBwjkqYr-pmmFOw8/s640/Snippet+4.jpg" width="640" /></a></div>
<br />
<span class="Apple-style-span" style="font-size: x-large;">Summary</span><br />
You've seen how to write some error checking code and how the different sorts of errors are reported. For a production app, this error checking is just a start. You'd need to be more thorough and helpful to your users. In addition, note that nothing will tell you about a bad parameter being sent to <span style="font-family: "Courier New",Courier,monospace;">/api/views</span>. Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-79879989895062285552011-08-31T13:16:00.000-07:002011-09-06T18:44:23.699-07:00Example #13 - Simple queries using GETGoing a bit deeper into the API. Let's look at code a few simple queries. Some of this will look familiar. We'll combine some of the queries we did in Example #10 with the code from Example #12 to create code snippets that could be used by a simple app.<br />
<br />
The SODA API is a RESTful API. You can read more about that on <a href="http://en.wikipedia.org/wiki/Representational_state_transfer">Wikipedia </a>or <a href="http://rest.elkstein.org/">one</a> of many simple tutorials.<br />
<br />
The SODA reference page for the <a href="http://opendata.socrata.com/api/docs/views#beta">Views API</a> shows that all uses of this service are done with a GET. When we open a URL without any parameters, that is a GET. This is what we've done in all examples so far and will do in this example.<br />
<br />
This talk about REST and GET is just background. You may want to dig deeper and if you do, you'll start to care about these terms. <br />
<br />
So, on with the example. We're going to progressively add parameters to do different queries and show how the returned information differs. The output from each request shows the number of views and the name of the view.<br />
<br />
<span style="font-size: x-large;">Example Code</span><br />
A few overall comments:<br />
<ul>
<li>I'm using a different style of importing. When you import <span style="font-family: "Courier New",Courier,monospace;">from</span> a module, you don't have to put in the name of the module when you use a function. In past examples, I would <span style="font-family: 'Courier New',Courier,monospace;">import urllib2</span> and then call <span style="font-family: 'Courier New',Courier,monospace;">urllib2.urlopen()</span>. This example uses <span style="font-family: 'Courier New',Courier,monospace;">from urllib2 import urlopen</span> and then calls <span style="font-family: 'Courier New',Courier,monospace;">urlopen()</span>. It makes the code look a little cleaner.</li>
<li>If you are wondering about the actions of the code, look back at Example 12 and your favorite Python book or website.</li>
</ul>
<br />
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: small;">from urllib2 import urlopen</span><br />
<span style="font-size: small;">from json import load</span><br />
<span style="font-size: small;"><br /></span><br />
<span style="font-size: small;"># 1 - Ask for all views</span><br />
<span style="font-size: small;">url = 'http://data.cityofchicago.org/api/views.json'</span><br />
<span style="font-size: small;">u = urlopen(url)</span><br />
<span style="font-size: small;">views = load(u)</span><br />
<span style="font-size: small;">numViews = len(views)</span><br />
<span style="font-size: small;">print '\nAsk for all views. Number of views returned = %s' % numViews</span><br />
<span style="font-size: small;">for view in views:</span><br />
<span style="font-size: small;"> print ' ', view['name']</span><br />
<span style="font-size: small;"><br /></span><br />
<span style="font-size: small;"># 2 - Ask for views with 'police' in the name</span><br />
<span style="font-size: small;">url = 'http://data.cityofchicago.org/api/views.json?name=police'</span><br />
<span style="font-size: small;">u = urlopen(url)</span><br />
<span style="font-size: small;">views = load(u)</span><br />
<span style="font-size: small;">numViews = len(views)</span><br />
<span style="font-size: small;">print '\nAsk for \"police\" in name. Number of views returned = %s' % numViews</span><br />
<span style="font-size: small;">for view in views:</span><br />
<span style="font-size: small;"> print ' ', view['name']</span><br />
<span style="font-size: small;"><br /></span><br />
<span style="font-size: small;"># 3 - Ask for views with 'police station' in the name</span><br />
<span style="font-size: small;">url = 'http://data.cityofchicago.org/api/views.json?name=police+station'</span><br />
<span style="font-size: small;">u = urlopen(url)</span><br />
<span style="font-size: small;">views = load(u)</span><br />
<span style="font-size: small;">numViews = len(views)</span><br />
<span style="font-size: small;">print '\nAsk for \"police station\" in name. Number of views returned = %s' % numViews</span><br />
<span style="font-size: small;">for view in views:</span><br />
<span style="font-size: small;"> print ' ', view['name']</span><br />
<span style="font-size: small;"><br /></span><br />
<span style="font-size: small;"># 4 - Ask for views with 'police' in the name but set a limit of just returning two views</span><br />
<span style="font-size: small;">url = 'http://data.cityofchicago.org/api/views.json?name=police&limit=2'</span><br />
<span style="font-size: small;">u = urlopen(url)</span><br />
<span style="font-size: small;">views = load(u)</span><br />
<span style="font-size: small;">numViews = len(views)</span><br />
<span style="font-size: small;">print '\nAsk for \"police\" in name and limit to two views. Number of views returned = %s' % numViews</span><br />
<span style="font-size: small;">for view in views:</span><br />
<span style="font-size: small;"> print ' ', view['name']</span></div>
<br />
<span style="font-size: x-large;">Explanations of each snippet</span><br />
For each of the four snippets, here's the few lines of code, an explanation, and the portion of the output generated by that snippet. <br />
<br />
<span style="font-size: large;">API request with no parameters</span><br />
<div style="font-family: "Courier New",Courier,monospace;">
# 1 - Ask for all views<br />
url = 'http://data.cityofchicago.org/api/views.json'<br />
u = urlopen(url)<br />
views = load(u)<br />
numViews = len(views)<br />
print '\nAsk for all views. Number of views returned = %s' % numViews<br />
for view in views:<br />
print ' ', view['name']</div>
This is a straightforward request from the <span style="font-family: 'Courier New',Courier,monospace;">views</span> service to return all the views (aka datasets) in the city of Chicago data portal. The output shows we get back the default of 50 views and lists their names.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8U5cL3E0EiVjoSVErUSCjCrcmzTU8-OlOP1mAPxIpbRv2Mwc-tBaqA7IBNDaHlLH5YRDSt-1OvaNEJCeHwqIDep_rKHpk9XmiXgcYKS4RuwMBLFZuuAN8AjcM9RgWHAN7WCAxbgV_mMYG/s1600/Snippet+1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="628" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8U5cL3E0EiVjoSVErUSCjCrcmzTU8-OlOP1mAPxIpbRv2Mwc-tBaqA7IBNDaHlLH5YRDSt-1OvaNEJCeHwqIDep_rKHpk9XmiXgcYKS4RuwMBLFZuuAN8AjcM9RgWHAN7WCAxbgV_mMYG/s640/Snippet+1.jpg" width="640" /></a></div>
<br />
<span style="font-size: large;">API request with one parameter</span><br />
<span style="font-family: 'Courier New',Courier,monospace;"># 2 - Ask for views with 'police' in the name</span><br />
<span style="font-family: 'Courier New',Courier,monospace;">url = 'http://data.cityofchicago.org/api/views.json?name=police'</span><br />
<span style="font-family: 'Courier New',Courier,monospace;">u = urlopen(url)</span><br />
<span style="font-family: 'Courier New',Courier,monospace;">views = load(u)</span><br />
<span style="font-family: 'Courier New',Courier,monospace;">numViews = len(views)</span><br />
<span style="font-family: 'Courier New',Courier,monospace;">print '\nAsk for \"police\" in name. Number of views returned = %s' % numViews</span><br />
<span style="font-family: 'Courier New',Courier,monospace;">for view in views:</span><br />
<span style="font-family: 'Courier New',Courier,monospace;"> print ' ', view['name']</span><br />
This shows how to add one parameter by adding on the <i>?</i> along with a name/value pair. This time we get back the eight views that have the word <i>police</i> in their name.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgCsXJSjQRFAV3yj8SK7ITXtMRfpMixGAE2pBZ_lw_GVB9O3MOyp56tMgqYq2lGex6mE6iBW4tJM6TFHGE9eKipKOx73p13c_p-jmD-cV0OzXoR5NDZydLSE6SbpwXt_OwZJ3UZDhHVw2M/s1600/Snippet+2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="146" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgCsXJSjQRFAV3yj8SK7ITXtMRfpMixGAE2pBZ_lw_GVB9O3MOyp56tMgqYq2lGex6mE6iBW4tJM6TFHGE9eKipKOx73p13c_p-jmD-cV0OzXoR5NDZydLSE6SbpwXt_OwZJ3UZDhHVw2M/s640/Snippet+2.jpg" width="640" /></a></div>
<br />
<span style="font-size: large;">API request with a multi-word parameter</span><br />
<div style="font-family: "Courier New",Courier,monospace;">
# 3 - Ask for views with 'police station' in the name<br />
url = 'http://data.cityofchicago.org/api/views.json?name=police+station'<br />
u = urlopen(url)<br />
views = load(u)<br />
numViews = len(views)<br />
print '\nAsk for \"police station\" in name. Number of views returned = %s' % numViews<br />
for view in views:<br />
print ' ', view['name']</div>
The <i>+</i> character is treated as a space. This is a request to search for <i>police station</i> in the name. We get back three datasets.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxNpAFsWXpy_qDsG9YsyTFjn-399HeY733sLqMCK8jCuK3yNqXzWs1o0dChgYdRrU_xzXPZkPjEBHWiex1lUS_ABA3-r34R54P1wiapQzEAkQRta3z9UPQKZrO_mAZBpGKAGOwGQ4ZKP24/s1600/Snippet+3.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="66" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxNpAFsWXpy_qDsG9YsyTFjn-399HeY733sLqMCK8jCuK3yNqXzWs1o0dChgYdRrU_xzXPZkPjEBHWiex1lUS_ABA3-r34R54P1wiapQzEAkQRta3z9UPQKZrO_mAZBpGKAGOwGQ4ZKP24/s640/Snippet+3.jpg" width="640" /></a></div>
<br />
<span style="font-size: large;">API request with multiple parameters</span><br />
<div style="font-family: "Courier New",Courier,monospace;">
# 4 - Ask for views with 'police' in the name but set a limit of just returning two views<br />
url = 'http://data.cityofchicago.org/api/views.json?name=police&limit=2'<br />
u = urlopen(url)<br />
views = load(u)<br />
numViews = len(views)<br />
print '\nAsk for \"police\" in name and limit to two views. Number of views returned = %s' % numViews<br />
for view in views:<br />
print ' ', view['name']</div>
Using the <i>&</i> character between name/value pairs, you can string together multiple parameters for your service request. Here we ask for the views with <i>police </i>in the name but with a limit of returning just two views. Instead of getting the eight views that have <i>police</i> in the name, we get just the first two.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhy29cej9ZaO16EQeqT22W4ZuLPEJVw8OWUn2Ac5SycW2jx3h7hrjWghc2h6Owou3IR6gUS-tRJROxtND9IF4vOcl2RMCaj1Ru-CdCAJd9DEUqgP-MlYw9qZDUnoQWx1fVj0WOM-VUvJdDQ/s1600/Snippet+4.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="48" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhy29cej9ZaO16EQeqT22W4ZuLPEJVw8OWUn2Ac5SycW2jx3h7hrjWghc2h6Owou3IR6gUS-tRJROxtND9IF4vOcl2RMCaj1Ru-CdCAJd9DEUqgP-MlYw9qZDUnoQWx1fVj0WOM-VUvJdDQ/s640/Snippet+4.jpg" width="640" /></a></div>
<br />
<span style="font-size: x-large;">Generalizing</span><br />
You can use this type of code to make all the requests detailed on the reference card in Example #10. Just plug in the URL you wish to use. You can use the <i>?</i> to string together name/value parameters shown on the reference card. If any of your parameters have multiple words in the value, use the <i>+</i>.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-81761064208281660092011-08-26T18:35:00.000-07:002011-09-08T18:08:58.866-07:00Example #12 - Pull data out of an API responseLet's take a look at how you might access specific values from the JSON-formatted response from an API request.<br />
<br />
To get some data to play around with, we'll send one API request for metadata for the views in the Chicago data portal. The response, in JSON, is a set of metadata structured as a list. This is a SODA <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">View[]</span>. Each element of the list contains the metadata for one view. That metadata (a SODA <span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">View</span>), in turn, is represented as a number of key-value pairs. Some of the values are themselves a set of nested key-value pairs. You can look at the output from the snippet #4 to see the metadata for one view.<br />
<br />
The example code does eight actions to illustrate some of the ways to access and use the data returned from an API request. You can generalize these to accessing data from other API requests.<br />
<br />
<span class="Apple-style-span" style="font-size: x-large;">Example Code</span><br />
<br />
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">#</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"># Example12.py - Shows code to access data in an API response</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">#</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">import json</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">import urllib2</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">import pprint</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">import time</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"><br />
</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"># Get metadata for views in the Chicago data portal</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">fileHandle = urllib2.urlopen("http://data.cityofchicago.org/api/views.json")</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">views = json.load(fileHandle)</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"><br />
</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"># 1 - Get the number of views that were returned</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">numViews = len(views)</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">print '\nNUMBER OF VIEWS RETURNED = %s\n' % numViews</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"><br />
</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"># 2 - Access one of the metadata values. Get the name of the first view.</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">viewName = views[0]['name']</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">print 'NAME OF FIRST VIEW = %s\n' % viewName</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"><br />
</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"># 3 - Iterate through the views and print the name of each view</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">print 'NAMES OF ALL %s VIEWS DOWNLOADED:' % numViews</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">for view in views:</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>print view['name']</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"><br />
</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"># 4 - Print all metadata for first view, both the keys and values, including all nested pairs </span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">print '\nFORMATTED DUMP OF THE FIRST VIEW SHOWING ALL KEYS AND VALUES:'</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">pp = pprint.PrettyPrinter(indent=4)</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">pp.pprint(views[0])</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"><br />
</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"># 5 - Iterate thru first view and print the keys. Does not print nested keys</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">print '\nHIGH-LEVEL FIELDS IN A VIEW:'</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">for key in views[0].iterkeys():</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>print key</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"><br />
</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"># 6 - Access a nested key-value pair. Print the name of the view owner.</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">ownerName = views[0]['owner']['displayName']</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">print '\nNAME OF THE OWNER OF THE FIRST VIEW %s\n' % ownerName</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"><br />
</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"># 7 - Get and print a time value. It's in seconds since the epoch. </span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">secsOwnerProfileModified = views[0]['owner']['profileLastModified']</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">print 'OWNER OF FIRST VIEW LAST MODIFIED HIS/HER USER PROFILE %s SECONDS SINCE JANUARY 1, 1970\n' % secsOwnerProfileModified</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"><br />
</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"># 8 - Now print the time value in local timezone.</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">localtimeOwnerProfileModified = time.ctime(secsOwnerProfileModified)</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">print 'OWNER LAST MODIFIED HIS/HER USER PROFILE ON %s\n' %localtimeOwnerProfileModified</span></div>
<br />
<span class="Apple-style-span" style="font-size: x-large;">Explanations of each snippet</span><br />
By the numbers, you'll see the few lines of code, an explanation, and the portion of the output generated by this section of the code.<br />
<br />
<span class="Apple-style-span" style="font-size: large;">Convert JSON and load an array</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;"># Get metadata for views in the Chicago data portal</span><br />
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">fileHandle = urllib2.urlopen("http://data.cityofchicago.org/api/views.json")</span></div>
<div>
<span class="Apple-style-span" style="font-family: 'Courier New',Courier,monospace;">views = json.load(fileHandle)</span></div>
To start, get that array of metadata of views. First, make the API request to the city of Chicago data portal and ask for the metadata for all views. Second, use <span style="font-family: "Courier New",Courier,monospace;">json.load()</span> to take data coming back from the API, parse the JSON, and place the elements the <span style="font-family: "Courier New",Courier,monospace;">views</span> list. The remaining eight numbered snippets work with that list.<br />
<br />
<div>
<span style="font-size: large;">Get the number of views returned</span><br />
<span style="font-family: 'Courier New',Courier,monospace;"># 1 - Get the number of views that were returned</span><br />
<span style="font-family: 'Courier New',Courier,monospace;">numViews = len(views)</span><br />
<span style="font-family: 'Courier New',Courier,monospace;">print '\nNUMBER OF VIEWS RETURNED = %s\n' % numViews</span><br />
How many individual views were sent back? We can get the length of the list to find out. The result shows there are 50 elements. Hey, first discovery - with no parameters, the API defaults to return 50 of the 603 currently available views<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgLF22iNW3yEye53jyI1FpDPvcjdnvkMlTR1wkv5QUSMvRHNzDalUnHr4w8aYJn2DpVAZtfFxM-nIjdWq8_Ba2Z57CfL_wvTl8jHp-cpMth3qkoNkdnTS7KYWrMtOim1jBVXfgqT0Q2dOm0/s1600/Snippet+1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="54" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgLF22iNW3yEye53jyI1FpDPvcjdnvkMlTR1wkv5QUSMvRHNzDalUnHr4w8aYJn2DpVAZtfFxM-nIjdWq8_Ba2Z57CfL_wvTl8jHp-cpMth3qkoNkdnTS7KYWrMtOim1jBVXfgqT0Q2dOm0/s640/Snippet+1.jpg" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<span style="font-size: large;">Get a value from one view</span><br />
<div style="font-family: "Courier New",Courier,monospace;">
# 2 - Access one of the metadata values. Get the name of the first view.<br />
viewName = views[0]['name']<br />
print 'NAME OF FIRST VIEW = %s\n' % viewName</div>
This may be something you'll be doing for an app. You can pull out the individual strings that are the values for different pieces of the metadata. Here, we are asking for the value associated with the key called <span style="font-family: 'Courier New',Courier,monospace;">name</span> for the zeroeth element of the <span style="font-family: "Courier New",Courier,monospace;">views</span> list. Translation: get the name of the first view.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFu7SxDItMRcHq68C8ADpNXvIc1XwhN4_SQgRjWRrpeMuaypU1h4Dt6KPUiaW9C5DAx_IlKvcYbnUrE236EbCYt5uMf89lRD3fKR0fKGcGF93zJU5fWyOihbJbJJ6bhScEQhpXw0szpGZu/s1600/Snippet+2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="28" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFu7SxDItMRcHq68C8ADpNXvIc1XwhN4_SQgRjWRrpeMuaypU1h4Dt6KPUiaW9C5DAx_IlKvcYbnUrE236EbCYt5uMf89lRD3fKR0fKGcGF93zJU5fWyOihbJbJJ6bhScEQhpXw0szpGZu/s640/Snippet+2.jpg" width="640" /></a></div>
<br />
<span style="font-size: large;">Iterate through all of the views</span><br />
<div style="font-family: "Courier New",Courier,monospace;">
# 3 - Iterate through the views and print the name of each view<br />
print 'NAMES OF ALL %s VIEWS DOWNLOADED:' % numViews<br />
for view in views:<br />
print view['name']</div>
Here's how you can step through the list of views and get a value for the <span style="font-family: "Courier New",Courier,monospace;">name</span> key in each view. The output is a nice summary of the views from the portal.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVflMtWi-IdMd-4u6SvZQL6UXVuDODoMn7l4BHm6dCfhLYW7dgJzOvkqxcZPLatCfAAIURvEiTShh7Ukx6sGkJxNRyWsjGa98xQRxYLSqtE1TRiY5oWZcTJjcutK8REkgC65HhPsHWN3sz/s1600/Snippet+3.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVflMtWi-IdMd-4u6SvZQL6UXVuDODoMn7l4BHm6dCfhLYW7dgJzOvkqxcZPLatCfAAIURvEiTShh7Ukx6sGkJxNRyWsjGa98xQRxYLSqtE1TRiY5oWZcTJjcutK8REkgC65HhPsHWN3sz/s640/Snippet+3.jpg" width="600" /></a></div>
<br />
<span style="font-size: large;">Print all metadata for a view</span><br />
<span style="font-family: 'Courier New',Courier,monospace;"># 4 - Print all metadata for first view, both the keys and values, including all nested pairs </span><br />
<span style="font-family: 'Courier New',Courier,monospace;">print '\nFORMATTED DUMP OF THE FIRST VIEW SHOWING ALL KEYS AND VALUES:'</span><br />
<span style="font-family: 'Courier New',Courier,monospace;">pp = pprint.PrettyPrinter(indent=4)</span><br />
<span style="font-family: 'Courier New',Courier,monospace;">pp.pprint(views[0])</span><br />
Want the details of the metadata for a view? Here's everything we have for the first view. <span style="font-family: 'Courier New',Courier,monospace;">pprint </span>outputs each key-value pair on its own line. The nested pairs are indented four spaces. You can use this for other API responses to see everything that you receive in a layout that is somewhat easy to read. The <span style="font-family: 'Courier New',Courier,monospace;">u'....'</span> means <span style="font-family: 'Courier New',Courier,monospace;">... </span>is a unicode string.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjEvGkK_3ujMkKR_SObAfP7GCyjCV40z8ziXn0QutvZntvYdFBIPmH1z7Wl1mjkIjxxYISLVTnMibq4-na_jHkHrWF_3kRg4Ekt_t-6jQYjy8_vqFqNat9WHfmrP0PHIHpo0CjJ_QQ3wILJ/s1600/Snippet+4.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="484" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjEvGkK_3ujMkKR_SObAfP7GCyjCV40z8ziXn0QutvZntvYdFBIPmH1z7Wl1mjkIjxxYISLVTnMibq4-na_jHkHrWF_3kRg4Ekt_t-6jQYjy8_vqFqNat9WHfmrP0PHIHpo0CjJ_QQ3wILJ/s640/Snippet+4.jpg" width="640" /></a></div>
<br />
<span style="font-size: large;">Iterate through one view</span><br />
<div style="font-family: "Courier New",Courier,monospace;">
# 5 - Iterate thru first view and print the keys. Does not print nested keys<br />
print '\nHIGH-LEVEL FIELDS IN A VIEW:'<br />
for key in views[0].iterkeys():<br />
print key</div>
We iterated through all views and printed one value from each view. Now, let's go inside one view and iterate through it. We'll step through all the key-value pairs and just print the key. We aren't going deeper into the nested key-value pairs. For example, the key <span style="font-family: 'Courier New',Courier,monospace;">owner </span>has as it's value a whole set of key-value pairs. We'll just print out owner and move on. The result is a nice list of all the data elements in the <span style="font-family: 'Courier New',Courier,monospace;">View </span>data type. Note that it differs from the reference material on the SODA site. That documentation does not match the API's operation.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4Ok0spWBX1bsdfmIx1pE2tgNXFIL6033aNOcMD8CHD1krbhVGZ7s4pR7c0P5fT02pwnM9LcY2IhXB6DKo2tyA881WUvL3beJsVvtQIBQCWu3jEcOLv3xr999tT8tSFbS7bVA2sVjuO1QH/s1600/Snippet+5.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4Ok0spWBX1bsdfmIx1pE2tgNXFIL6033aNOcMD8CHD1krbhVGZ7s4pR7c0P5fT02pwnM9LcY2IhXB6DKo2tyA881WUvL3beJsVvtQIBQCWu3jEcOLv3xr999tT8tSFbS7bVA2sVjuO1QH/s320/Snippet+5.jpg" width="249" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<span style="font-size: large;">Get a nested key-value pair</span><br />
<div style="font-family: "Courier New",Courier,monospace;">
# 6 - Access a nested key-value pair. Print the name of the view owner.<br />
ownerName = views[0]['owner']['displayName']<br />
print '\nNAME OF THE OWNER OF THE FIRST VIEW %s\n' % ownerName</div>
Now, let's dig into one of those nested dictionaries. The <span style="font-family: 'Courier New',Courier,monospace;">owner </span>key has a value that consists of nine key-value pairs. Here's how you can get one of those. The code goes to the first view and then to the <span style="font-family: 'Courier New',Courier,monospace;">owner</span> key and then the <span style="font-family: 'Courier New',Courier,monospace;">displayName </span>key of within the <span style="font-family: 'Courier New',Courier,monospace;">owner </span>key. Translation: what's the name of the owner of the first view?<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiALhGgr4kqpHKqhLSTxfhoTV54k60yAMmYqaTmNUcca-cCFoRCuJevWE9rWL2Ig05qCkIGOYKFmD2mdWNBZE2pKbEoCMS25a3w05gp66Tp4mOfptutYTEzbRJynKjRp4UHs-fa8sOviR1g/s1600/Snippet+6.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="48" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiALhGgr4kqpHKqhLSTxfhoTV54k60yAMmYqaTmNUcca-cCFoRCuJevWE9rWL2Ig05qCkIGOYKFmD2mdWNBZE2pKbEoCMS25a3w05gp66Tp4mOfptutYTEzbRJynKjRp4UHs-fa8sOviR1g/s640/Snippet+6.jpg" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<span style="font-size: large;">Get a time value</span><br />
<div style="font-family: "Courier New",Courier,monospace;">
# 7 - Get and print a time value. It's in seconds since the epoch. <br />
secsOwnerProfileModified = views[0]['owner']['profileLastModified']<br />
print 'OWNER OF FIRST VIEW LAST MODIFIED HIS/HER USER PROFILE %s SECONDS SINCE JANUARY 1, 1970\n' % secsOwnerProfileModified</div>
Some of the data elements are timestamps. This example and the next one look at those. We'll work with the element that represents the time when the owner of the first view last modified his or her user profile. These values are represented as the number of seconds since January 1, 1970, which is also called the epoch. This snippet prints out that value.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAnidwvZf2ISSdWLUrl7dMK6p8JlC7oOC-a_e_eJFDrrwJgU5r8n0bqPb4fJdJ3gTf8ml7RJc5Dw4w1PYcxM77yUIIciMZQmEuapSrAWi57qCzC7YZJiXPeO9cKcn6V2QzIrnboDLBMy5K/s1600/Snippet+7.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="24" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAnidwvZf2ISSdWLUrl7dMK6p8JlC7oOC-a_e_eJFDrrwJgU5r8n0bqPb4fJdJ3gTf8ml7RJc5Dw4w1PYcxM77yUIIciMZQmEuapSrAWi57qCzC7YZJiXPeO9cKcn6V2QzIrnboDLBMy5K/s640/Snippet+7.jpg" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<span style="font-size: large;">Convert time in seconds to local time</span><br />
<span style="font-family: 'Courier New',Courier,monospace;"># 8 - Now print the time value in local timezone.</span><br />
<span style="font-family: 'Courier New',Courier,monospace;">localtimeOwnerProfileModified = time.ctime(secsOwnerProfileModified)</span><br />
<span style="font-family: 'Courier New',Courier,monospace;">print 'OWNER LAST MODIFIED HIS/HER USER PROFILE ON %s\n' %localtimeOwnerProfileModified </span><br />
Seeing the total number of seconds isn't very useful to us (it is, however, very useful for comparing times or computing the difference between times). Here we'll use a method to create a string that has day, month, time of day, and year in the local timezone. Now, it's easier understand when the view's owner's profile was modified.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_V_wEns1RVIvgE7RhOcst2fCVnZRFfGMmIT2Ju8u5K-Zes0nGAKGzZNQImnbK3c6IcazFtDSp-Bz0vg52XqdW6iMQqMwxpc9Iogh3Sak-_7Cut8W0_eRTbT3oVQxGP_cGvKtqk1Kk42S6/s1600/Snippet+8.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="44" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_V_wEns1RVIvgE7RhOcst2fCVnZRFfGMmIT2Ju8u5K-Zes0nGAKGzZNQImnbK3c6IcazFtDSp-Bz0vg52XqdW6iMQqMwxpc9Iogh3Sak-_7Cut8W0_eRTbT3oVQxGP_cGvKtqk1Kk42S6/s640/Snippet+8.jpg" width="640" /></a></div>
<br />
<span class="Apple-style-span" style="font-size: large;">Final comments</span><br />
Remember, if you run this code, your output may differ from what's shown. When you run it, the portal may have a different sets of views with different values.</div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-73867617007490003722011-08-24T14:19:00.000-07:002011-09-06T18:57:42.631-07:00Example #11 - Create the simplest programExample #10 showed many uses of the API. Let's code up one. This will be the <b>Hello World!</b> example of using the API.<br />
<br />
The code examples will be written in Python. I'm brand new to Python and using this work as a great excuse to learn it. If you look at any of the code and know there's a better way, use it and ignore my novice approach.<br />
<br />
Let's pick the API request from Example #10 with the smallest amount of response data. It'll just keep things simpler. The winner is the API request to count the number of views in the Chicago portal since it just returns a number.<br />
<br />
<span style="font-size: x-large;">Code</span><br />
Here's the code for my program called, <span style="font-family: "Courier New",Courier,monospace;">example11.py</span>:<br />
<div style="font-family: "Courier New",Courier,monospace;">
import urllib2<br />
f = urllib2.urlopen("http://data.cityofchicago.org/api/views.json?count=true")<br />
response = f.read() <br />
print response</div>
<br />
<span style="font-size: large;">Explanation</span><br />
What's going on in this code? Here's a line by line description:<br />
<ol>
<li>Import the module that has the functions to open and get data from a URL.</li>
<li>The URL is the API request to return the number of views in the Chicago portal. (See Example #10 for more about how the URL is determined.) <span style="font-family: "Courier New",Courier,monospace;">f</span> is the filehandle to read the response from the API request. The API is invoked during this statement.</li>
<li>Using file I/O method, <span style="font-family: "Courier New",Courier,monospace;">read()</span>, to read what the URL returned, ie., the API response, into the string <span style="font-family: "Courier New",Courier,monospace;">response</span>.</li>
<li>Finally, print <span style="font-family: "Courier New",Courier,monospace;">response</span>.</li>
</ol>
If you are new to Python as well, you can try this yourself. First, go to the <a href="http://www.python.org/">Python site</a> and <a href="http://www.python.org/download/">download Python.</a> I'm using Python 2.7. Then, setup Python on your system. You can cut-and-paste the code into your editor, save it as <span style="font-family: "Courier New",Courier,monospace;">example11.py</span>, and run it with the command, <span style="font-family: "Courier New",Courier,monospace;">python example11.py</span>.<br />
<br />
Here's a screenshot when I ran the program and its output:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrT7fS-20lmSH-p-qjr4GKxWGmysLDHDa1niLP-TPJyUvtHFinsSYgGoSWOr2b5sHIVSpOQL4J-p26x2hKQxHAibupRxlM5DO6tMLjS37wGn_pFM1ID8YfuY-PKJG4CaCNS0J1x_NqEHuN/s1600/Running+example11.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="134" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrT7fS-20lmSH-p-qjr4GKxWGmysLDHDa1niLP-TPJyUvtHFinsSYgGoSWOr2b5sHIVSpOQL4J-p26x2hKQxHAibupRxlM5DO6tMLjS37wGn_pFM1ID8YfuY-PKJG4CaCNS0J1x_NqEHuN/s640/Running+example11.jpg" width="640" /></a></div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-16727791623783348372011-08-23T19:48:00.000-07:002011-09-08T18:08:03.459-07:00Example #10 - Explore the Views Service APIIt's time to get a better understanding of the API you can use for an app that searches for and retrieves data from the portal.<br />
<br />
The SODA API consists of four services. These are the types of interactions you can request from the portal. They are:<br />
<ol>
<li><span style="font-family: "Courier New",Courier,monospace;">authenticate</span> - this invokes a cookie-based authentication method for subsequent requests.</li>
<li><span style="font-family: "Courier New",Courier,monospace;">docs</span> - retrieves the reference documentation for the API services </li>
<li><span style="font-family: "Courier New",Courier,monospace;">users</span> - retrieves details of user profiles as well as let you manage your own profile.</li>
<li><span style="font-family: "Courier New",Courier,monospace;">views </span>- access views to retrieve metadata plus query, create, read, update, and delete rows.</li>
</ol>
We're going to ignore the first three services. They aren't needed for our simple apps.<br />
<br />
We'll study a portion of the <span style="font-family: "Courier New",Courier,monospace;">views</span> service. Our apps will just consume data. We won't publish any data on the portal. So, we'll look at retrieving metadata, query, and read operations. We'll ignore the create, update, and delete operations.<br />
<br />
If you want to study the other services, head over to the <a href="http://dev.socrata.com/">SODA Developers site</a>.<br />
<br />
If you want to study the details of the views service, check out the <a href="http://opendata.socrata.com/api/docs/views">SODA Developers reference</a> page.<br />
<br />
Beware that this API is still a beta. A number of the services are noted with the disclaimer that they are not finalized and subject to change. <br />
<br />
<span style="font-size: large;">Terms</span><br />
<div style="text-align: center;">
<i><span style="font-family: inherit;">"When I use a word," HumptyDumpty said, in a rather scornful tone,</span></i></div>
<div style="text-align: center;">
<i><span style="font-family: inherit;">"it means just what I choose it to mean - neither more nor less."</span></i></div>
<div style="text-align: center;">
<i><span style="font-family: inherit;"> </span><br style="font-family: inherit;" /><span style="font-family: inherit;">"The question is," said Alice,</span></i></div>
<div style="text-align: center;">
<i><span style="font-family: inherit;">"whether you can make words mean so many different things."</span></i></div>
<div style="text-align: right;">
<i><span style="font-family: inherit;">-- Alice in Wonderland</span></i></div>
<br />
In the land of Socrata, the terms view, table, and data set (or dataset) all mean the same thing. Yeah, I know, if you are a database purist, that's unsettling. Nevertheless, it seems as though the three terms are interchangeable. Plus, the terms view and views might get confusing. The <span style="font-family: "Courier New",Courier,monospace;">views</span> service does its work against a view in the portal. At times, you will specify a particular view for the <span style="font-family: "Courier New",Courier,monospace;">views</span> API to go after.<br />
<br />
When discussing the API, you make a <b>request </b>to a <b>service</b>. The <b>service </b>sends back a <b>response</b>. I tend to lapse into old school lingo of making a <b>call </b>to an <b>API </b>and getting back a <b>return</b>. There are even a few mentions on the SODA site of <b>methods </b>which is from the object-oriented culture and lines-up with request or call. For our purposes, different words - same concepts. <br />
<br />
<span style="font-size: x-large;">Reference card for views </span><br />
Here's a one page reference sheet to summarize the <span style="font-family: "Courier New",Courier,monospace;">views </span>service. Think of it as the magic decoder ring to create an API request. You can click on it to enlarge it.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeDgpuc1rHY87QPS58cetAJaewasSLlRUMPeMHSbjaCuvYfAWts3-UJUv8IH2AftfRQ7U8Yig3uXeUOScGl_MJllm4csns_2WSnb9j_2HiSO-ZOmKY6QWEfM1_iOiCjH0RltOC9x5x5JH2/s1600/Views+Reference+V1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeDgpuc1rHY87QPS58cetAJaewasSLlRUMPeMHSbjaCuvYfAWts3-UJUv8IH2AftfRQ7U8Yig3uXeUOScGl_MJllm4csns_2WSnb9j_2HiSO-ZOmKY6QWEfM1_iOiCjH0RltOC9x5x5JH2/s640/Views+Reference+V1.jpg" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhF-rusZ3WmMhHNdJ9lglnP8bFnZegnvyjVhpe-wGT6m6HmC1Dk5sYAV9u8lUxtGRUtJPSK1X24rrqgUgXOeV847RA1Dc7rHEFlwiHZmOyyiPF7FSa5d5leMTq8CfX2DT4nraUYR-_JANMx/s1600/Views+Reference+V1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><br /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
The bold <b>http...</b> line at the top is the API request that you send to a Socrata server. You have to fill in four blanks:<br />
<ol>
<li>The portal site you are going to use - either the city, county, or state. Or, the URL of some other Socrata site.</li>
<li>The <span style="font-family: "Courier New",Courier,monospace;">views </span>service you need. Your choices are in the <b>Service </b>column.</li>
<li>Tthe formatting of the response data. You can have the API send you JSON, XML, etc.</li>
<li>The parameters for the service from the <b>Parameters </b>column. These further define your request. </li>
</ol>
<br />
<span style="font-size: x-large;">Data types</span><br />
The <b>Response </b>column lists the data type you will receive from the service. There are six data types. When the '[ ]' appear it means you will receive an array of them - you'll get back one or more in an array. Here are the types with links that take you to the SODA reference document, if available, to show you the details:<br />
<ul>
<li><a href="http://opendata.socrata.com/api/docs/types#View" style="font-family: "Courier New",Courier,monospace;">View</a><span style="font-family: "Courier New",Courier,monospace;"> </span>- has all the information about a view such as description of the view, the columns in the view, etc.</li>
<li><span style="font-family: "Courier New",Courier,monospace;">ViewColumn </span>- has all the information about a column. (The SODA reference does not have a section describing this data type.)</li>
<li><span style="font-family: "Courier New",Courier,monospace;">File </span>- contents of a file. You can request a file that is attached to a view. For example, you can request a file from a dataset that has a GIS file in .KML format. (The SODA reference does not have a section describing this data type.)</li>
<li><span style="font-family: "Courier New",Courier,monospace;">Row </span>- contents of a row in a view. This is the data you see when you are looking at a dataset in your web browser. (The SODA reference does not have a section describing this data type.)</li>
<li><a href="http://opendata.socrata.com/api/docs/types#string" style="font-family: "Courier New",Courier,monospace;">String</a> - a string of characters.</li>
<li><span style="font-family: "Courier New",Courier,monospace;">UserTag </span>- what the user entered when they added a tag to a dataset. (The SODA reference does not have a section describing this data type.) </li>
</ul>
<span style="font-size: x-large;"> </span><br />
<span style="font-size: x-large;">Annotated examples</span><br />
We'll use the Chicago Ward Offices dataset. To start, any API request we make for this dataset will use the Chicago portal address (If you are stickler for details, it's the hostname) and most requests will use the<span style="font-family: "Courier New",Courier,monospace;"> viewID</span> of the dataset. <br />
<br />
Go to the Chicago Data Portal, search for "ward office", look through the results list for the "Ward Offices" entry, click on it, and you'll be looking at the table of data. (If you'd rather take a shortcut, here's the link to the<a href="http://data.cityofchicago.org/dataset/Ward-Offices/htai-wnw4"> ward office dataset</a>.) Now, take a look in your web browser URL bar. You'll see the URL for the dataset: <span style="font-family: "Courier New",Courier,monospace;">http://data.cityofchicago.org/dataset/Ward-Offices/htai-wnw4</span>. The portal site is <span style="font-family: "Courier New",Courier,monospace;">data.cityofchicago.org</span>. The <span style="font-family: "Courier New",Courier,monospace;">viewID</span> is <span style="font-family: "Courier New",Courier,monospace;">htai-wnw4</span>.<br />
<br />
Now, let's build some API requests. Try these out. You can cut-and-paste these into your own command prompt window and put <span style="font-family: "Courier New",Courier,monospace;">curl</span> in front of them or edit them slightly and use the Chicago console. See <a href="http://metrochicagoopendataexamples.blogspot.com/2011/08/example-9-introduction-to-soda-api.html">Example #9</a> for the how-to on both of those approaches.<br />
<br />
You'll see I'm favoring the <span style="font-family: "Courier New",Courier,monospace;">JSON</span> format for the response. Try replacing that string in one or two samples with <span style="font-family: "Courier New",Courier,monospace;">XML </span>to see the difference between a JSON-formatted response and an XML-formatted response. It seems that JSON is the default. If you omit the format, you'll get back JSON. I like showing it so you'll know where to slip in another format keyword if you want something besides JSON.<br />
<span style="font-size: large;"><br />
Heads up!</span><br />
If you are using <span style="font-family: "Courier New",Courier,monospace;">curl</span>, you might need to quote your <span style="font-family: "Courier New",Courier,monospace;">http</span> string. I just found one example that needed quotes. All other examples work without quotes. My lazy way is to let you know you probably need quotes if you string parameters together. I might dig into this in the future and revise all the examples as needed.<br />
<br />
This doesn't work:<br />
<span style="font-family: "Courier New",Courier,monospace;">curl http://data.cityofchicago.org/api/views.json?name=ward&limit=2 </span><br />
I get the error message, <span style="font-family: "Courier New",Courier,monospace;">'limit' is not recognized as an operable program or batch file. </span><br />
<div style="font-family: "Courier New",Courier,monospace;">
</div>
<br />
This works:<br />
<div style="font-family: "Courier New",Courier,monospace;">
curl "http://data.cityofchicago.org/api/views.json?name=ward&limit=2"</div>
<br />
<span style="font-size: large;">Get metadata for all views</span><br />
<div style="font-family: "Courier New",Courier,monospace;">
http://data.cityofchicago.org/api/views.json</div>
This will return a huge amount of detail on all the datasets in the portal. You probably won't use this for a simple app. You might use this in an app that allows the user to navigate to any dataset in a portal. For that, your first step might be to grab all this metadata and show the user the datasets in the portal to let the user pick what looks interesting.<br />
<br />
<span style="font-size: large;">Get number of views</span><br />
<span style="font-family: "Courier New",Courier,monospace;">http://data.cityofchicago.org/api/views.json?count=true</span><br />
This parameter will cause the API to return the number of views in the portal. You could use this to tell the user how many datasets there are in the portal. When I run this today, I get back a count of 603. This matches what I see in my web browser when I look at the Chicago portal main page. At the bottom of the page under the page numbers it has, "Showing 25 of 603". <br />
<br />
<span style="font-size: large;">Get metadata for a view</span><br />
<div style="font-family: "Courier New",Courier,monospace;">
http://data.cityofchicago.org/api/views/htai-wnw4.json</div>
This returns all the metadata about the Ward Offices dataset. You might want to use this. You can pull out the description of the dataset, when it was last updated by the city, how many times it has been viewed and other tidbits that might help your user. It also has the information on the columns of data in the table you can access. This might be really useful. Your app could parse through this to discover the names of columns and other helpful data about each column.<br />
<br />
<span style="font-size: large;">Get metadata for views with specific word in name</span><br />
<span style="font-family: "Courier New",Courier,monospace;">http://data.cityofchicago.org/api/views.json?name=ward</span><br />
You can add a parameter to return just the views that have the term, 'ward', in their view name. When I run this today, I get back four views. You could use this service to show the user all the views that have to do with wards. Note that you could use <span style="font-family: "Courier New",Courier,monospace;">...name=ward </span>or <span style="font-family: "Courier New",Courier,monospace;">...name='ward'</span>. They are equivalent. <br />
<br />
<span style="font-size: large;">Combine parameters to limit the views returned</span><br />
<span style="font-family: "Courier New",Courier,monospace;">http://data.cityofchicago.org/api/views.json?name=ward&limit=2</span><br />
Here we'll run the query above but only ask for a max of two views to be returned. Because of the limit only two views are returned. This is an example of combining two parameters with <span style="font-family: "Courier New",Courier,monospace;">&</span>. For this and other services with multiple parameters, you can combine several parameters in this fashion. See the <b>Heads Up!</b> found above. You may need to quote this string with <span style="font-family: "Courier New",Courier,monospace;">curl</span>.<br />
<br />
<span style="font-size: large;">Get metadata for views with specific words in name</span><br />
<span style="font-family: "Courier New",Courier,monospace;">http://data.cityofchicago.org/api/views.json?name=ward+office</span><br />
Like above but this time you get back the views that have both words, <i>ward</i> and <i>office</i>, in the name. When I run this, I get back two views. Note, the search just looks for both words. It does not look for them being adjacent like <i>ward office</i>. (I haven't figured out how to search for names with the phrase, <i>ward office,</i> yet.)<br />
<br />
<span style="font-size: large;">Get metadata for views with specific word in description</span><br />
<span style="font-family: "Courier New",Courier,monospace;">http://data.cityofchicago.org/api/views.json?description=ward</span><br />
This parameter will search the view descriptions and return the views with <i>ward</i> in the description. When I run this today, I get back seven views. <br />
<br />
<span style="font-size: large;">Get metadata for views with specific words in description</span><br />
<span style="font-family: "Courier New",Courier,monospace;">http://data.cityofchicago.org/api/views.json?description=ward+office</span><br />
This parameter will search the view descriptions and return the views with both <i>ward</i> and <i>office </i>in the description. When I run this today, I get back three views.<br />
<br />
<span style="font-size: large;">Get metadata for all columns</span><br />
<div style="font-family: "Courier New",Courier,monospace;">
http://data.cityofchicago.org/api/views/htai-wnw4/columns.json</div>
Returns the metadata about all columns in a view. This is a subset of the previous API request. In the one just above we get the metadata about the columns plus a whole lot more. If you just want to know about the columns in the view, use this one.<br />
<br />
<span style="font-size: large;">Get one column </span><br />
<div style="font-family: "Courier New",Courier,monospace;">
http://data.cityofchicago.org/api/views/htai-wnw4/columns/2607673.json</div>
Returns the data and the metadata for a specific column. I used the response from the previous API and found the <span style="font-family: "Courier New",Courier,monospace;">columnID</span>, 2607673, for the ALDERMAN column. This sample will return the ALDERMAN column.<br />
<br />
<span style="font-size: large;">Get all rows</span><br />
<div style="font-family: "Courier New",Courier,monospace;">
http://data.cityofchicago.org/api/views/htai-wnw4/rows.json</div>
Returns the data in the table. Now we're getting something really useful for a simple app. This pulls back all the data along with the row metadata for the table. If you study the output, you'll see that the metadata for the view is returned and after that, at the end of flood of info is the "data" field. There you'll find the metadata plus contents of each row.<br />
<br />
<span style="font-size: large;">Get the IDs of all rows </span><br />
<div style="font-family: "Courier New",Courier,monospace;">
http://data.cityofchicago.org/api/views/htai-wnw4/rows.json?row_ids_only=true</div>
You can request the set of all <span style="font-family: "Courier New",Courier,monospace;">rowID</span>s. This will have one ID for each row. Note that you'll get view metadata first and at the end of the response, in the <span style="font-family: "Courier New",Courier,monospace;">"data"</span> field, you will get the <span style="font-family: "Courier New",Courier,monospace;">rowID</span>s in an array. I'm trying to think of a good use for this array. There are other API services that will probably give you the information you really need.<br />
<br />
<span style="font-size: large;">Get one row (alternative #1)</span><br />
<span style="font-family: "Courier New",Courier,monospace;">http://data.cityofchicago.org/api/views/htai-wnw4/rows.json?ids=2</span><br />
Using the list of <span style="font-family: "Courier New",Courier,monospace;">rowID</span>s from the previous response, you can use them with the <span style="font-family: "Courier New",Courier,monospace;">ids</span> parameter to get a specific row. You could use the previous sample to get all <span style="font-family: "Courier New",Courier,monospace;">rowID</span>s and then use this service to pull them back one at a time. This sample retrieves the row with <span style="font-family: "Courier New",Courier,monospace;">rowID</span> = 2. The return from this call is very succinct. You will get back just the info for that row and no other metadata. (The SODA site says you can retrieve multiple rows with <span style="font-family: "Courier New",Courier,monospace;">...?ids=<rowID>&ids=<rowID></span> but I haven't been able to do that.)<br />
<br />
<span style="font-size: large;">Get one row (alternative #2)</span><br />
<span style="font-family: "Courier New",Courier,monospace;">http://data.cityofchicago.org/api/views/htai-wnw4/rows/2.json</span><br />
Pulls back one row. Here's another request you can use to pull back one row. This approach has slightly simpler syntax. <br />
<br />
<span style="font-size: large;">Search and get rows </span><br />
<div style="font-family: "Courier New",Courier,monospace;">
http://data.cityofchicago.org/api/views/htai-wnw4/rows.json?search="Burke"</div>
This one is <b>really important</b> for a simple app. It lets you search for rows for a text string. This example searches for Alderman Burke's ward office by searching the rows for <i>Burke</i>. You'll get back all the metadata, which is probably superfluous to you, and then at the end is the <span style="font-family: "Courier New",Courier,monospace;">"data"</span> for the one row about Alderman Burke's office. Seems like you can use <span style="font-family: "Courier New",Courier,monospace;">...search=Burke</span> or <span style="font-family: "Courier New",Courier,monospace;">...search="Burke"</span>. If you use <span style="font-family: "Courier New",Courier,monospace;">...search='Burke'</span>, it doesn't find the row.<br />
<br />
<span style="font-size: large;">More parameters</span><br />
The <span style="font-family: "Courier New",Courier,monospace;">views/<viewID>/rows</span> service has many parameters. Take a look at these sometime. You can limit the number of rows that are returned. You can just get a subset of rows by giving a starting row plus a number of follow-on rows to return; you can use this to page through the dataset. There are other parameters but they probably won't be of interest for a simple app.<br />
<br />
<span style="font-size: large;">Other <span style="font-family: "Courier New",Courier,monospace;">views</span> services</span><br />
I'm going to skip the requests that work with sub_columns, files, tags, and user tags. The reference card shows them and there's more info at the SODA site. They don't seem too pertinent for a simple app.<br />
<br />
The <span style="font-family: "Courier New",Courier,monospace;">INLINE</span> filtered search will be useful to us. You can do very complex searches with one request. Even better, you can do a geographic search to find rows that fall into a circle. You can specify a latitude/longitude and a radius. The service will return the rows with locations in that circle. It warrants it's own entry. I'll get to that one in a future example.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-24554948090713665382011-08-20T20:57:00.000-07:002011-08-20T21:02:51.198-07:00Results #8 - Publish a map of foreclosures and incomeBelow is a map of foreclosures in the first half of 2011 by zip code in Cook County along with the per capita income from 2009 in those zip codes.<br />
<br />
<script src="http://public.tableausoftware.com/javascripts/api/viz_v1.js" type="text/javascript">
</script><br />
<div class="tableauPlaceholder" style="height: 445px; width: 504px;"><noscript>&lt;a href="#"&gt;&lt;img alt="Foreclosures by Zip " src="http:&amp;#47;&amp;#47;public.tableausoftware.com&amp;#47;static&amp;#47;images&amp;#47;Co&amp;#47;CookCountyForeclosuresandIncomebyZipCode&amp;#47;ForeclosuresbyZip&amp;#47;1_rss.png" style="height: 100%; width: 100%; border: none" /&gt;&lt;/a&gt;</noscript><object class="tableauViz" height="445" style="display: none;" width="504"><param name="host_url" value="http%3A%2F%2Fpublic.tableausoftware.com%2F" /><param name="name" value="CookCountyForeclosuresandIncomebyZipCode/ForeclosuresbyZip" /><param name="tabs" value="yes" /><param name="toolbar" value="yes" /><param name="static_image" value="http://public.tableausoftware.com/static/images/Co/CookCountyForeclosuresandIncomebyZipCode/ForeclosuresbyZip/1.png" /><param name="animate_transition" value="yes" /><param name="display_static_image" value="yes" /><param name="display_spinner" value="yes" /><param name="display_overlay" value="yes" /></object></div><div style="color: black; font: normal 8pt verdana,helvetica,arial,sans-serif; height: 22px; padding: 0px 10px 0px 0px; width: 504px;"><div style="float: right; padding-right: 8px;"><a href="http://www.tableausoftware.com/public?ref=http://public.tableausoftware.com/views/CookCountyForeclosuresandIncomebyZipCode/ForeclosuresbyZip" target="_blank">Powered by Tableau</a></div></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-87323306625707427822011-08-20T20:37:00.000-07:002012-01-12T15:51:59.772-08:00Example #8 - Publish a map of foreclosures and per capita income by zip codeWe can use <a href="http://www.tableausoftware.com/public/">Tableau Public</a> to create more complex visualizations. Let's look at foreclosures along with per capita income in Cook County by zip code. As expected, you'll see that the lower income zip codes have a high number of foreclosures.<br />
<br />
There's more work involved in this visualization. Tableau Public (TP) runs on a Windows computer so you'll have to <a href="http://www.tableausoftware.com/public/download">download a free copy </a>and install it. Getting the data ready takes some effort. Start with the Cook County Data Portal foreclosure data set, download it,and edit it to clean it up. I used the text editor, gVim (a vi variant), to do the clean up. Four changes were needed:<br />
<ol>
<li>Change the <i>Location 1</i> field into a five-digit zip code. In the portal the field has zip+4 appended to a string of lat/lon.</li>
<li>Remove the rows that represent amended foreclosures since they duplicate the original foreclosures.</li>
<li>Remove rows that had null or zeroes for zip code.</li>
<li>Remove the <i>Considerations</i> column. </li>
</ol>
We're not quite done. TP wants an XLS file. So, load the cleaned up CSV file into the spreadsheet or your choice (I used Lotus Symphony) and then save it as .XLS. Crank up TP and connect it to that XLS file. Finally, we're set to make visualizations.<br />
<br />
Here's a nice sample of what TP can create. This map shows a blue circle for the foreclosures in each zip code in the county - the larger the circle, the more foreclosures. TP provides built-in layers for the map. Select the Per Capita Income layer. Tinker with the colors and circles scale a bit to get the map below.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNpzGcyP1VnHQ-y2p_ImJyCwRm_0i5rrriWgGbvjWdJeSgRHoUkL-fHubDFe1T_1iAeNOsEZ7bN5fik64z5giRNFChc93JCfvyI5yr9Zf3a8pSMnUdFhSckhxPLbXMqSvmQ6X75lVgJ4U2/s1600/Map.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="474" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNpzGcyP1VnHQ-y2p_ImJyCwRm_0i5rrriWgGbvjWdJeSgRHoUkL-fHubDFe1T_1iAeNOsEZ7bN5fik64z5giRNFChc93JCfvyI5yr9Zf3a8pSMnUdFhSckhxPLbXMqSvmQ6X75lVgJ4U2/s640/Map.jpg" width="640" /></a></div>
<br />
You can quickly generate other visualizations. Here's a chart showing foreclosures by month.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6G0z87Jm1nUsZmTv50qEmVZ8E9ODdYOWM31CGuvi7HmE2B2trFzuAHfOccKSomFwHATxfuEoWyB37PzROXq3KVPAXgJVzxHjSak9aMrxFFX1sW06eJBn1jBPjtyIyX17PbhM2kn_eECy5/s1600/Bar+chart.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6G0z87Jm1nUsZmTv50qEmVZ8E9ODdYOWM31CGuvi7HmE2B2trFzuAHfOccKSomFwHATxfuEoWyB37PzROXq3KVPAXgJVzxHjSak9aMrxFFX1sW06eJBn1jBPjtyIyX17PbhM2kn_eECy5/s640/Bar+chart.jpg" width="508" /></a></div>
<br />
<i>Note: </i> The drop in July is not due to economic conditions. The data set ends on 7/15/11. We only have half of the month's data.<br />
<br />
You can see an interact with both of these charts in the <a href="http://metrochicagoopendataexamples.blogspot.com/2011/08/example-8-publish-map-of-foreclosures.html">Results #8</a> blog entry. <br />
<br />
Ready to watch? Here's the link to the <a href="http://www.youtube.com/watch?v=EoVFSh3ndnI">YouTube video</a>. If you have trouble reading the small fonts used, try changing the YouTube resolution to 720p and expand the video to full screen. You'll find the controls to make those changes near the bottom of the YouTube video window when you hover your mouse over the video.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-73546942700056385922011-08-20T13:01:00.000-07:002011-08-20T13:08:26.298-07:00Results #7 - Publish a ManyEyes bubble chartHere's a bubble chart showing the number of employees in each job title. Only job titles with over 100 people in them are displayed.<br />
<br />
<script src="http://www-958.ibm.com/me/visualizations/c5aad26ccb6111e09d6a000255111976/comments/c5ae8a6acb6111e09d6a000255111976.js?width=425&height=350" type="text/javascript">
</script>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-19099055410248940472011-08-20T12:52:00.000-07:002011-09-08T17:23:37.806-07:00Example #7 - Publish a ManyEyes bubble chartLet's take a look at another web site that offers visualizations. We'll use IBM's <a href="http://www-958.ibm.com/software/data/cognos/manyeyes/">ManyEyes</a> to create a bubble chart showing the number of jobs in each job title in the city government.<br />
<br />
To build this, we'll use a roll-up and a sort to get the count of each job title (ie., the number of police officers, number of firefighters, etc.), download the results to a spreadsheet, copy the ones with more than 100 people in them, paste them into ManyEyes, and have ManyEyes create the bubble chart. Here's that visualization:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQ4DhVfhIbIdS2D082S5v8AEALuqthAvK0L1zOsZBupq5BV4YsbT_7_UrBpv8-0gZIIluE5N9Uebu2WpPaDPEdwppNBNge0IVWRd8pCD1PpnhyNDFYQM850EVOX1KZKBkAr7id7tVR5K_S/s1600/Bubble+chart.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="392" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQ4DhVfhIbIdS2D082S5v8AEALuqthAvK0L1zOsZBupq5BV4YsbT_7_UrBpv8-0gZIIluE5N9Uebu2WpPaDPEdwppNBNge0IVWRd8pCD1PpnhyNDFYQM850EVOX1KZKBkAr7id7tVR5K_S/s640/Bubble+chart.jpg" width="640" /></a></div>
<br />
You can publish this visualization on your own web page, too. See the <a href="http://metrochicagoopendataexamples.blogspot.com/2011/08/results-7-number-of-employees-by-job.html">Results #7 </a>entry for an interactive example.<br />
<br />
ManyEyes has more styles of visualizations. Once your data set is loaded, it's easy to try others.<br />
<br />
Ready to watch? Here's the <a href="http://www.youtube.com/user/snewell4?feature=mhee#p/u/1/X8-edeb9_eA">YouTube video</a>Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-5928026535874540705.post-78542962179537408342011-08-14T17:11:00.000-07:002011-09-08T18:04:54.577-07:00Example #6 - Create a bar chart using tabular dataWe'll create a bar chart in the data portal showing where the money goes from Cook County's checkbook. The bar chart will summarize the spending from December 2010 by totaling the amount for the biggest categories.<br />
<br />
You will see how to use the roll-up filter to create spending for each description (I think of them as categories), the sort filter to order the categories amounts from most to least, and filter this to show just the categories where $100,000 or more was spent. Next, use the bar chart visualization to create a chart that looks like this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgSqLwiXQtHlYC9JQp6qusdsNN4Nt9ZrxZQAiSrKFIBT5SmRUDKZPgNDJJba01jv0VJNg9Itu7CsIFx5jBPoqAlMI2eK9rIH015PcLld0WugsrNLhvFoZq2wDQXjykLYJZJdy0tJoPvw12k/s1600/Check+Register+chart.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="378" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgSqLwiXQtHlYC9JQp6qusdsNN4Nt9ZrxZQAiSrKFIBT5SmRUDKZPgNDJJba01jv0VJNg9Itu7CsIFx5jBPoqAlMI2eK9rIH015PcLld0WugsrNLhvFoZq2wDQXjykLYJZJdy0tJoPvw12k/s640/Check+Register+chart.jpg" width="640" /></a></div>
<br />
Ready to watch? Here's the link to the <a href="http://www.youtube.com/watch?v=FVHczNqm9K4">YouTube video.</a>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-30186804727696253752011-08-14T14:26:00.000-07:002011-09-08T18:03:54.509-07:00Example #5 - Combine three KML data sets and display in Google EarthWe'll create a map showing Cook County fishing lakes, Metra stations, and Metra lines. Let's say you want to go fishing. You can use the resulting map to look for lakes that are close to Metra stops.<br />
<br />
We'll download KML files from the data portal and load them into Google Earth to create the map. These are files that show the shapes of things. In the previous examples, we create Google Maps showing points of graffiti. In this map, we'll show the outline of the fishing lakes and the rail lines.<br />
<br />
Here's a screen shot of the map. The lake is in bright blue. The station is the red 'X'. The rail line is shown as the solid black line.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6JFELPBQbM1Stuw4-0qg574gijh48WRNgeoP8YATnvvGREjnSou8lxLccv5iO-ZqjbTgemJ53xE4zHCHNqSTYBlBEnAy9gV0g8kucy652zpwfNrBRZE5pkfiU7GFvyO-embU4GiHRaZ2M/s1600/Earth+image.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="378" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6JFELPBQbM1Stuw4-0qg574gijh48WRNgeoP8YATnvvGREjnSou8lxLccv5iO-ZqjbTgemJ53xE4zHCHNqSTYBlBEnAy9gV0g8kucy652zpwfNrBRZE5pkfiU7GFvyO-embU4GiHRaZ2M/s640/Earth+image.jpg" width="640" /></a></div>
<br />
This map doesn't display in your web browser. It is shown using your own copy of Google Earth. If you don't have a copy, you can download one from the <a href="http://www.google.com/earth/download/ge/agree.html">Google Earth download page</a>.<br />
<br />
Ready to watch? Here's a link to the <a href="http://www.youtube.com/watch?v=TwMjVU7RXQg">YouTube video</a>.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-9633309870860273302011-08-08T05:53:00.000-07:002011-08-08T06:10:44.795-07:00Results #4 - Publish a Google intensity mapWant to see the hot spots of graffiti are located? I've embedded a map of the locations of 311 service requests for graffiti removal below. You'll see by the intensity of color where the graffiti is located.<br />
<br />
<iframe width="500px" height="300px" scrolling="no" src="http://www.google.com/fusiontables/embedviz?viz=MAP&q=select+*+from+1259797+&h=true&lat=41.886432216946986&lng=-87.62351989746094&z=11&t=1&l=col11"></iframe>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5928026535874540705.post-84479211713525859322011-08-08T05:49:00.000-07:002011-09-08T18:02:48.870-07:00Example #4 - Publish a Google intensity mapWe'll use the intensity map created in Example #3 and publish it in this blog.<br />
<br />
This example will show you how to embed a Google fusion table map into your own web page. This approach can be used for any visualization you build using a fusion table.<br />
<br />
Here's what we'll create - a blog entry with the intensity map.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcaujn9CzbmqsU9SUxxApMvtHDCdekfg1uzcoMVGAayV7kIhuyeVdszeZZVjNxQ3ffbfgQXHrI-iOCLXex09CjFTxfVOYunDvDzW59M2-5zxr7Aj7MzJZAfSbYMMu-MDIR2o8rrbrMJcDT/s1600/blog+image.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="630" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcaujn9CzbmqsU9SUxxApMvtHDCdekfg1uzcoMVGAayV7kIhuyeVdszeZZVjNxQ3ffbfgQXHrI-iOCLXex09CjFTxfVOYunDvDzW59M2-5zxr7Aj7MzJZAfSbYMMu-MDIR2o8rrbrMJcDT/s640/blog+image.jpg" width="640" /></a></div>
<br />
This is a dynamic map. You can interact with it to zoom in/out or move it around.<br />
<br />
The<a href="http://metrochicagoopendataexamples.blogspot.com/2011/08/results-4-publish-google-intensity-map.html"> Results #4</a> blog entry is the working example.<br />
<br />
Ready to watch? Here's the link to the <a href="http://www.youtube.com/watch?v=Upnelk5eMJ4">YouTube video</a>.Unknownnoreply@blogger.com0