I’m going to go a little meta and discuss the aspects of this project behind the project itself.

Why would someone want to blog every day?

• To educate those who don’t know what you do
• To learn more about a topic you already know – regardless of what you think of your knowledge you will be doing LOTS of research and experimentation along the way.
• To practice writing and communicating on technical topics. This isn’t easy, and I’ve been “writing” for a while.

What does it take?

• Interest in the topic you’re writing about. A little (or lot!) knowledge helps too.
• Discipline – nearly every day there’s some degree of “meh I do NOT want to do this, or at least there was for me. Once I reached the 3/4 mark though it switched to “meh I do NOT want to do this, but I’m so dang close to succeeding!”
• Basic blog software. Like I said at the beginning of the month, my time was better spent writing content that taught than hacking on some blog I could then use. You’ll bikeshed the crap out of a blog you build yourself, don’t fall in this trap.

My advice

• Take it small – I would up about 500 words (according to WordPress) a day by accident. Wasn’t a goal, but that’s about where things felt “done”
• Work a little every day. I thought taking weekends off would give me a chance to pre-fill the buffer with entries and leave weeknights free – nope. I was too busy enjoying NOT writing and doing other things.
• Pre-plan: I didn’t get to everything I put on my list, and things came up that I didn’t even think about. But when crunch time came to prep an entry for the next day, the list gave me a clear direction of a sequence that made sense and if all else failed gave me an idea I could run with on a short deadline. Ask people what they want to know about your topic. In my case I asked several experienced Django devs I know what they needed to know about GeoDjango, mixed in the stuff I thought they should know too and wrote them.
• Don’t be afraid to cut into pieces, or walk through a “project”. There seem to be a lot of people showing up to read entries, but a lot of the time they’re coming a couple days after publication. And now the entries will be out “forever” so those that find it can continue at their convenience.
• DO IT. You’ll learn a lot, you’ll practice even more, and you’ll help people who don’t know things just like others helped you when you didn’t know things.
• Don’t shoot for more than a month.
• Give yourself time off – weekends, Monday/Wednesday/Friday, anything.

It’s been fun, and this blog isn’t going anywhere – the topic sheet still has a bunch there. But I’ve got some life, and a mess of housework, to catch up on.

As great as the Django Admin is, it’s not for most of your users. So what do we do to allow user-submitted geometries?

Cue django-floppyforms which contains handy widgets we can use with our own forms and modelforms to allow geographic input.

Consider this model:

class MyArea(models.Model):
    name = models.CharField(max_length=64, unique=True)
    point = models.PointField(srid=4326)
    polygon = models.MultiPolygonField(srid=4326)

Throw this in views.py

import floppyforms as forms

from django.http import HttpResponseRedirect
from django.shortcuts import render_to_response
from django.template import RequestContext

from formstuff.myarea.models import MyArea

class OSMPointWidget(forms.gis.PointWidget, forms.gis.BaseOsmWidget):
    map_srid = 900913

class OSMMultiPolygonWidget(forms.gis.MultiPolygonWidget, forms.gis.BaseOsmWidget):
    map_srid = 900913

class AreaForm(forms.ModelForm):
    point = forms.gis.PointField(widget=OSMPointWidget)
    polygon = forms.gis.MultiPolygonField(widget=OSMMultiPolygonWidget)

    class Meta:
        model = MyArea

def editing(request):
    form = AreaForm(request.POST or None)
    if form.is_valid():
        form.save()
        return HttpResponseRedirect('/')

    return render_to_response('area.html', {
        'form': form,
    }, context_instance=RequestContext(request))

and users can drop points and polygons on your model, which will be saved into the database. Do you want the Google basemap instead? Extend BaseGMapWidget instead of BaseOsmWidget, remove the srid attrib, and you’re set. It’s amazing how easy those widgets make it – you don’t even have to know you’re dealing with a geometry in your save method.

The Django admin saves us mountains of time. Back when Django still wasn’t quite Django and it was an internal project it was common for the team to leave a meeting, scribble a schema on a napkin and have the admin ready for data entry in the matter of a couple hours.

Models are still my first stop for any project – build the basic frame, and start gathering data with the admin. But over the years we’ve all come to expect more from applications and Django has gotten much better tools for it too. Need to reorder fields? Hide the stuff that rarely gets entered? Use a custom form? It’s in the docs. With the GeoDjango admin though, things aren’t as well documented. So let’s look at some of the simplest things that can make an application better.

Changing the default map

Two modes are available, admin.GeoModelAdmin and admin.OSMGeoAdmin.  GeoModelAdmin uses very simple tiles from OpenLayers that have state lines, rivers / water sources and a few other unmarked areas. I can’t imagine how it would be useful because there’s nothing to reference from, but if you have your own set of WMS tiles you can set wms_url and it will use them.

By default when you load the admin you’ll find yourself looking in the Atlantic off the coast of Africa. If you’re US-centric, try

class CampgroundAdmin(admin.GeoModelAdmin):
    default_lon = -98
    default_lat = 38.5
    default_zoom = 3

Which will center you in a spot where you can see the entire US. default_zoom of 4 is a closer view if you don’t need to see New England. For Kansas, default_zoom of 6 gets you the whole state. Experiment with these three values and you can set up your web app to be “optimized” for entry anywhere. As even a neogeographer can tell, they’re in decimal degrees like you get from your GPS.

This works in OSMGeoAdmin as well (the admin I use the most). Just as it says on the tin, OSMGeoAdmin uses OpenStreetMap so there’s a LOT of data available on the basemap you edit on top of. BUT this piece of the docs means one thing – “that uses a spherical mercator projection” – we can’t just put decimal degrees in and it work. But we have the tools to make it work.

>>> from django.contrib.gis.geos import Point
>>> center = Point((-98, 38.5), srid=4326)
>>> center.transform(900913)
>>> center.y
4650301.836738959
>>> center.x
-10909310.097740812

We put these coordinates on the admin object, and get a street map in the admin for editing.

class CampgroundOSMAdmin(admin.OSMGeoAdmin):
    default_lon = -10909310
    default_lat = 4650301
    default_zoom = 6

Other options in the admin class are available but some of the ones I could see using don’t seem to work – like max_zoom / min_zoom. “modifiable” set to false will force geometries not to be editable, but that’s not particularly useful in the admin.

Adding More Layers

We can add additional WMS layers to our maps, too – in this example we’ll add the current NEXRAD radar composite and a Topographical map. (additional layer info can be found at http://trac.osgeo.org/openlayers/wiki/AvailableWMSServices but many of them did not work for me)

Add the map_template attribute to our admin class:

class CampgroundAdmin(admin.GeoModelAdmin):
    default_lon = -98
    default_lat = 38.5
    default_zoom = 6
    max_zoom = 18
    min_zoom = 6
    map_template = 'gis/admin/openlayers_extralayers.html'

Now we need to create the template, and the javascript that will initialize the additional layers.

In gis/admin/openlayers_extralayers.html we change which file is included:

{% extends "openlayers.html" %}

{% block openlayers %}{% include "gis/admin/openlayers_extralayers.js" %}{% endblock %}

And in openlayers_extralayers.js we define the new layers:

{% extends "gis/admin/openlayers.js" %}

{% block extra_layers %}
    topo_layer = new OpenLayers.Layer.WMS( "USA Topo", "http://terraservice.net/ogcmap.ashx", {layers: 'DRG'} );
    {{ module }}.map.addLayer(topo_layer);
    nexrad_layer = new OpenLayers.Layer.WMS( "NEXRAD", "http://mesonet.agron.iastate.edu/cgi-bin/wms/nexrad/n0r.cgi?", {layers:"nexrad-n0r",transparent:"true",format:'image/png'} );
    {{ module }}.map.addLayer(nexrad_layer);
{% endblock extra_layers %}

IF you want this to be the default behavior for ALL of your admins, just name the file openlayers.html and openlayers.js instead of openlayers_extralayers.html/js. NOTE: If you do this you must copy/paste the entirety of openlayers.html and openlayers.js into YOUR openlayers.html and openlayers.js and put your code inside of the appropriate blocks. If you use my example, the template will extend itself, cause an infinite recursion and the dev server or your server process will promptly quit with no warning. Ask me how I know!! 🙂

You can download openlayers_extralayers.html and openlayers_extralayers.js for use in your own projects.

Note this won’t work with OSMGeoAdmin immediately unless you use the openlayers.html / openlayers.js naming and follow the above concerns. If you don’t want to do that (and I would try to avoid it because it could get you stuck on an old version of the javascript in the future) create your own osm.html:

{% extends "gis/admin/openlayers_extralayers.html" %}
{% block openlayers %}{% include "gis/admin/osm_extralayers.js" %}{% endblock %}

and osm_extralayers.js:

{% extends "gis/admin/openlayers_extralayers.js" %}
{% block base_layer %}new OpenLayers.Layer.OSM.Mapnik("OpenStreetMap (Mapnik)");{% endblock %}

Since we named it osm.html it will take over for all OpenStreetMap admins. If that’s not what you want, name it something else and override map_template on the admin class.

Another note: the Topo layer used doesn’t understand the OSM projection so it won’t work in OSMGeoAdmin.

Occasionally you just need a quick map, and it’s not going to be used for interaction or even necessarily display on the internet (say a Keynote/PowerPoint presentation, for example). We could build it as a Google map, get a gigantic computer monitor and fullscreen the window and take a screenshot, and have it look like every other web map out there – or we can reach into the toolbox of those who are NOT neogeographers and borrow a few things.

For me this tool is still qGIS (for Mac check out KyngChaos for binary builds), though there are alternatives like GRASS GIS and TileMill, to name only a couple. (TileMill is actually designed to create tiles similar to how Google Maps / OpenStreetMap do it, but you can shoehorn it to do “print” maps)

Of course, it bears repeating that the results from doing this are not survey quality. Don’t use my recommendations / guidance to determine where not to dig to avoid buried lines, where to build a road, or anything else. But zoomed back a ways where the dot on the map is easily the size of a 40,000 population town, it’s close enough.

Since this is a process with no code, I’m switching format a bit and creating a screencast, which you can see here as a QuickTime file.

Mentioned files you may want for your own maps:

• “land_ocean_ice” original image: http://earthobservatory.nasa.gov/Features/BlueMarble/BlueMarble_2002.php
• “Blue marble” tagged GeoTIFF: http://www.samueltoepke.com/projects/olympianengine/public/tutorials/geotiff/index.html (see section “4/ 12 A World Map”

Now we make our code restrict to things near the user when they allow geolocation…but the results aren’t really what we’re shooting for.

def earthquake_display(request):
    search_origin = False
    if request.GET.get('lat') and request.GET.get('lon'):
        search_origin = Point(float(request.GET.get('lon')), float(request.GET.get('lat')))

    weekago = datetime.datetime.now() - datetime.timedelta(days=7)
    quakes = Quake.objects.filter(datetime__gte=weekago).order_by('-datetime')

    last_check = cache.get('usgs-poll-last-finished', datetime.datetime(2000, 1, 1))
    checking = cache.get('usgs-poll-in-progress', False)
    if not checking:
        cache.set('usgs-poll-in-progress', True)
        latest_quake_ago = datetime.datetime.now() - quakes[0].datetime
        latest_check_ago = datetime.datetime.now() - last_check
        if latest_quake_ago > datetime.timedelta(minutes=90) and latest_check_ago > datetime.timedelta(minutes=90):
            from django.core import management
            management.call_command('load_quakes')
            cache.set('usgs-poll-last-finished', datetime.datetime.now())
            # and refresh the QuerySet to get the new data
            quakes = Quake.objects.filter(datetime__gte=weekago).order_by('-datetime')
        cache.delete('usgs-poll-in-progress')
        checking = False

    if search_origin:
        quakes = quakes.distance(search_origin).order_by('distance', '-datetime')

    quakes = quakes[:25]

    return render_to_response('earthquakes.html', {
        'object_list': quakes,
        'checking': checking,
    }, context_instance=RequestContext(request))

<html>
<head>
  <script src='http://media.adamfast.com/QueryData.compressed.js'></script>
  <script language='JavaScript'>

    function checkLocation() {
      var getData = new QueryData();
      if ('lat' in getData) { }
      else {
        if (navigator.geolocation) {
          navigator.geolocation.getCurrentPosition(
            function (ppos) {
              window.location.href = window.location.href + '?lat=' + ppos.coords.latitude + '&lon=' + ppos.coords.longitude;
            },
            function (err) {
              switch(err.code) {

                case err.TIMEOUT:
                  alert('Attempts to retrieve location timed out.')
                  break;

                case err.POSITION_UNAVAILABLE:
                  alert("Your browser doesn't know where you are.")
                  break;

                case err.PERMISSION_DENIED:
                  alert('You have to give us permission!')
                  break;

                case err.UNKNOWN_ERROR:
                  alert('Unknown error returned.')
                  break;

                default:
                  alert(err + ' ' + err.code)

              }
            }
          );
        }
      }
    }
  </script>
</head>
<body{% block body_override %} onLoad="javascript:checkLocation();"{% endblock body_override %}>
{% block content %}
    {% for object in object_list %}
    {{ object }}<br>
    {% endfor %}
{% endblock content %}
</body>
</html>

This is returning results with too much emphasis on distance and not enough on how long ago the activity at that distance was. So I tried a few things:

• Restrict to activity only in the past 4 hours instead of a week (not bad! helps quite a bit.)
• Restrict to activity within 1000 miles (maybe restricts too much? This left me with no results for Missouri where I write this, but there are no quakes recently near here.

def earthquake_display(request):
    search_origin = False
    if request.GET.get('lat') and request.GET.get('lon'):
        search_origin = Point(float(request.GET.get('lon')), float(request.GET.get('lat')))
        search_origin.transform(900913)
        search_poly = search_origin.buffer(1000 * 2172.344)
        search_poly.transform(4326)
        search_origin.transform(4326)

#    weekago = datetime.datetime.now() - datetime.timedelta(days=7)
#    quakes = Quake.objects.filter(datetime__gte=weekago).order_by('-datetime')
    last_4_hrs = datetime.datetime.now() - datetime.timedelta(hours=4)
    quakes = Quake.objects.filter(datetime__gte=last_4_hrs).order_by('-datetime')

    last_check = cache.get('usgs-poll-last-finished', datetime.datetime(2000, 1, 1))
    checking = cache.get('usgs-poll-in-progress', False)
    if not checking:
        cache.set('usgs-poll-in-progress', True)
        latest_quake_ago = datetime.datetime.now() - quakes[0].datetime
        latest_check_ago = datetime.datetime.now() - last_check
        if latest_quake_ago > datetime.timedelta(minutes=90) and latest_check_ago > datetime.timedelta(minutes=90):
            from django.core import management
            management.call_command('load_quakes')
            cache.set('usgs-poll-last-finished', datetime.datetime.now())
            # and refresh the QuerySet to get the new data

#            quakes = Quake.objects.filter(datetime__gte=weekago).order_by('-datetime')
            quakes = Quake.objects.filter(datetime__gte=last_4_hrs).order_by('-datetime')

        cache.delete('usgs-poll-in-progress')
        checking = False

    if search_origin:
        quakes = quakes.filter(point__within=search_origin)
        quakes = quakes.distance(search_origin).order_by('distance', '-datetime')

    quakes = quakes[:10]

    return render_to_response('earthquakes.html', {
        'object_list': quakes,
        'checking': checking,
    }, context_instance=RequestContext(request))

In the spirit of the holiday (in the US we’re celebrating Thanksgiving) I thought I’d offer my appreciation / thanks to the projects I use and lean on to make my applications work. This isn’t a canonical list, I’m sure I’ve forgotten libraries here and there. And regardless of whether I use your code right now or not, thanks for releasing it where I can someday if it does something I need. I contribute everything I can when I can because everybody else has contributed so much that makes me more productive I want to follow suit.

If you blog, I’d ask you to consider something alone these lines of your own. As software authors there are lots of instances where our code is doing things we have no idea about until we’re told.

• Python language
• Django
• django-socialregistration (I’ve got my own fork of this but it existing saved me a bunch of time!)
• dateutil
• PostgreSQL
• PostGIS
• Coverage.py
• django-coverage
• GeoPy
• python-twitter
• Metar
• Arduino – platform, software and hardware. I love these things and enjoy prototyping with them.

This is in addition to all the OTHER stuff I use besides libraries – GitX, VLC, Audacity, QGIS, TileMill and others I’m sure.

Open source software authors, give yourselves a pat on the back. Open source users, thank the authors who give you something to use in the first place.

We’ve got our quakes app installed (update it if you’re following along at home, I committed a bunch of code changes for this entry) and our initial data imported.

However, we need this data to be updated somewhat regularly for the service to be useful. Sean shipped django-quakes with Celery support (which I commented out of the requirements file because I’m running this app on the neat ep.io auto-scaling webapp service‘s free tier and don’t have Celery access. If you run Celery just re-enable it, if you don’t use Celery or even know what it is (check it out!) you can use cron to schedule it just as well.

To avoid polling too often and bugging the USGS I’d set the interval at hourly or more under normal circumstances. We only care about quakes when our users do. I’d run it at least daily though so you keep history.

Since we care when our users do and earthquakes can’t be convinced to ONLY hit at certain minutes after the hour, what do we do? How about we check the latest quake we have and see if it’s beyond a certain age to force a refresh?

This is more of that “please don’t do this” code – yes, it works, but it’s bad. If you’re like me and want the site to auto-retrieve at a faster schedule when users are using it, you might be tempted to do this:

latest_quake_ago = datetime.datetime.now() - quakes[0].datetime
if latest_quake_ago > datetime.timedelta(minutes=5):
    from django.core import management
    management.call_command('load_quakes')

This is bad. Why? Our webapp isn’t a single-file line. When there’s an earthquake, people start going nuts. The resulting traffic will be somewhat of a flood, so imagine a bunch of these views all running at once. USGS will not be happy with us, and we’re doing all kinds of extra processing that just gets thrown out. (NOTE: If you’re trying this in the default dev server, it won’t work – it’s single-threaded so both will never be going at once.)

So what’s our next thought? Lock files (virtually, in cache instead of filesystem)!

checking = cache.get('usgs-poll-in-progress', False)
if not checking:
    print('spawning check')
    cache.set('usgs-poll-in-progress', True)
    latest_quake_ago = datetime.datetime.now() - quakes[0].datetime
    if latest_quake_ago > datetime.timedelta(minutes=5):
        from django.core import management
        management.call_command('load_quakes')
    cache.delete('usgs-poll-in-progress')
    print('check done.')
else:
    print('in progress')

We’re still not quite there. After all, at the time of this writing there hasn’t been an earthquake in 45 minutes – so every access kicks off a poll. Let’s add caching to store the last time a poll was done.

last_check = cache.get('usgs-poll-last-finished', datetime.datetime(2000, 1, 1))
checking = cache.get('usgs-poll-in-progress', False)
if not checking:
    cache.set('usgs-poll-in-progress', True)
    latest_quake_ago = datetime.datetime.now() - quakes[0].datetime
    latest_check_ago = datetime.datetime.now() - last_check
    if latest_quake_ago > datetime.timedelta(minutes=5) and latest_check_ago > datetime.timedelta(minutes=5):
        print('spawning check')
        from django.core import management
        management.call_command('load_quakes')
        cache.set('usgs-poll-last-finished', datetime.datetime.now())
        print('check done.')
    cache.delete('usgs-poll-in-progress')
else:
    print('in progress')

But what about people who hit the site while a poll is in progress? How about a time.sleep(0.5) loop?

Did you just throw your mouse? I dodged it. Yeah, that’s a bad idea. After all, it does tie up our Python processes at the moment when we’ve got hordes of eager, shaken (pun only slightly intended) users ringing the doorbell.

So what’s a programmer to do? Let’s set a context flag in the template and use it to trigger something in the template about “we just heard, we’re checking into it!”with what we already have.

def earthquake_display(request):
    weekago = datetime.datetime.now() - datetime.timedelta(days=7)
    quakes = Quake.objects.filter(datetime__gte=weekago).order_by('-datetime')

    last_check = cache.get('usgs-poll-last-finished', datetime.datetime(2000, 1, 1))
    checking = cache.get('usgs-poll-in-progress', False)
    if not checking:
        cache.set('usgs-poll-in-progress', True)
        latest_quake_ago = datetime.datetime.now() - quakes[0].datetime
        latest_check_ago = datetime.datetime.now() - last_check
        if latest_quake_ago > datetime.timedelta(minutes=5) and latest_check_ago > datetime.timedelta(minutes=5):
            from django.core import management
            management.call_command('load_quakes')
            cache.set('usgs-poll-last-finished', datetime.datetime.now())
        cache.delete('usgs-poll-in-progress')
        checking = False

    return render_to_response('earthquakes.html', {
        'object_list': quakes,
        'checking': checking,
    }, context_instance=RequestContext(request))

We’re getting there! We still need geolocation, geographic filtering, and a frontend.

We’ve discussed all sorts of things this month, now it’s time to start putting them together more completely than snippet-for-this, snippet-for-that. I’m not going to inline command + output on everything but I am going to spell out all of the steps – if you have trouble, let me know in the comments.

To accomplish this we’re going to build a simple web service I’ve snagged “isthatanearthquake.com” for.

First order of business is creating the project.

Because virtualenv and virtualenvwrapper rocks (`pip install virtualenv virtualenvwrapper`), I started with `mkvirtualenv isthatanearthquake`. My global virtualenv `postactivate` script includes the line `cd $VIRTUAL_ENV` which drops me in the env whenever I activate it. Next I created a `src` directory, created and cd’ed into `isthatanearthquake-git`, created `templates` and created requirements.txt with these contents:

django==1.3.1

psycopg2==2.4.2

-e git+git://github.com/adamfast/django-quakes.git@4d9fcdb4978a28f35d1e32406a81f3b8b5d58845#egg=quakes

pip install -r requirements.txt gets me Django, Postgres and the reusable app django-quakes (thanks Sean Bleier, who I’ve forked the library from for this example) installed in my virtualenv.

Create the Django project with `python $VIRTUAL_ENV/lib/python2.?/site-packages/django/bin/django-admin.py startproject isthatanearthquake` (note you will need to specify your python version in this command)

`add2virtualenv isthatanearthquake` will make the new project show up on your PYTHONPATH.

Open isthatanearthquake/settings.py and `import os` at the top. Somewhere in the file (I tend to put it nearer the bottom define

PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__))

Set up a database: DATABASES[‘default’][‘ENGINE’] should be ‘django.contrib.gis.db.backends.postgis’ and [‘NAME’] should be whatever you want to call the new database.

Set your template dirs to use the templates folder created just outside the project from earlier:

TEMPLATE_DIRS = (
    os.path.join(PROJECT_ROOT, "../templates"),
)

Add ‘django.contrib.gis’, and ‘quakes’, to INSTALLED_APPS. Uncomment ‘django.contrib.admin’ (and ‘django.contrib.admindocs’, though that one isn’t strictly required it’s just my habit.) We’re done with settings.py for now.

Uncomment the line in urls.py including admin.site.urls. If you uncommented django.contrib.admindocs in INSTALLED_APPS, uncomment its include statement as well.

Create the database you configured for with `createdb -T template_postgis <dbname>`

`python manage.py syncdb` will create the necessary tables, create a superuser of your choice, and `python manage.py load_quakes` will grab the last seven days of earthquakes from USGS.

`python manage.py runserver` will pop up the development server, and now with a web browser we can hit http://localhost:8000/admin/, login and choose “Quakes” to see a list of what came in.

One of the hardest parts of doing geo projects is getting the data you need to do it in the first place. In the US at least there are mountains of data at the federal level, some at the state level and who knows what at the local level. There isn’t a single place I can go to for anything outside of census-type stuff. Which school for a given grade level would a child at this point attend? Maybe it’s published, maybe it’s not.

Most cities or counties will have a GIS department. Some of them will be great, and helpful towards your goal as a developer. Others won’t. Our world of the tools and technologies are leapfrogging traditional methods.

So here’s some of my favorite sources of different kinds of data, worth looking at for your next project.

TIGER/LINE: Gigabytes of shapefiles of things the US Federal government collects. I tend to look here first if I need basic location data.

SimpleGeo Places DB: SimpleGeo put a dataset of 21 million places (12ish million in the US) into the public domain last summer, and links to the file here. I have played with the data some and it’s pretty clean but often categorized inconsistently (but hey, it’s free for all to use) There are discussions online about various ways of getting it imported, shapefile conversion didn’t work for me and neither did wrapping it in a feature collection in GeoJSON – it’s just too big – nearly 8GB of JSON. The method I got to actually work was taking it line by line, deserializing and then processing. I split into a TON of files containing 20,000 places each and ran several processes to get it imported. It’s a BIG database, and pretty slow in PostGIS so be warned. I have no plans to put anything in production from it so speed isn’t that big of an issue. As a side note, people importing this dataset are probably their best sales tactic toward paying for their SAAS version.

Flickr Shapefiles: Also public domain, potentially useful if you need to bring photos into the mix.

Timezones: Full shapefile of timezones of the world. Useful for auto-detecting your users’ time by their location and auto-shifting times to them. Public domain.

Free IP/Location database: Similar to GeoIP but community built. Likely not as comprehensive, it had no idea where I was for instance.

data.gov: It’s hard to find what you want and sometimes hard to figure out how to use the format it’s in, but data.gov is as close to a one-stop-shop as you may come.

data.nasa.gov: Datasets published by NASA. This one’s still a work in progress, but has a better search than data.gov and includes most NASA data listed on data.gov.

Ordnance Survey: Open (see licenses) data for the UK.

FCC GIS: Information on communications systems you can import.

FCC APIs: APIs to interact with the FCC’s data through their systems, includes data gathered for the national broadband map.

FreeGIS: Links to other datasets and software related to geo

NAIP Imagery: Satellite photos from flyovers during growing seasons.

Land Cover: Determining classification of land – forested, open, urban, etc.

National Hydrography Dataset: Information on our water.

Cartographic Boundaries: Outlines of various types of drawn boundaries.

Have you got a favorite or just plain useful dataset missed here? Leave a comment and let us know about them!

I stopped short of displaying an equivalent overlay in Google Maps v3 yesterday, but wanted to circle back and show that it’s doable as well. It’s far more verbose than the magic-y “pass it a polygon and it does the rest”, but you have more control too.

View, pretty much just like the one we used for points only with a model that has a polygon:

def gmap3_poly(request):
    states = State.objects.filter(mpoly__distance_lte=(Point((-94, 37)), D(mi=100)))

    return render_to_response('gmap3.html', {
        'object_list': states,
    }, context_instance=RequestContext(request))

For the template, I actually modified yesterday’s to auto-detect the geometry coming through and just do the right thing.

{% extends "gmap3base.html" %}

{% block page-title %}Requested Items{% endblock %}

{% block head_override %}
<script type="text/javascript">
  var bounds = new google.maps.LatLngBounds();

  function buildMarker(map, latitude, longitude, name, color) {
    var latlng = new google.maps.LatLng(latitude, longitude);
    var marker = new google.maps.Marker({
          position: latlng,
          map: map,
          title:name,
    });
    marker.setIcon('http://maps.google.com/mapfiles/ms/icons/' + color + '-dot.png');
    bounds.extend(latlng);
    return marker
  }

  function mapInitialize() {
    var myOptions = {
      zoom: 6,
      center: new google.maps.LatLng(0, 0),
      mapTypeId: google.maps.MapTypeId.SATELLITE
    };
    var map = new google.maps.Map(document.getElementById("map_canvas"),
        myOptions);

    {% for object in object_list %}
    {% if object.point %}marker{{ object.pk }} = buildMarker(map, {{ object.point.y }}, {{ object.point.x }}, "{{ object }}", 'red');
    {% endif %}
    {% if object.mpoly %}
    var polygon{{ object.pk }}Coords = [
    {% with object.mpoly.0.0 as the_polygon %}
    {% for point in the_polygon %}
        new google.maps.LatLng({{ point.1 }}, {{ point.0 }}),
    {% endfor %}
    {% endwith %}
    ];

    var the_polygon{{ object.pk }} = new google.maps.Polygon({
      paths: polygon{{ object.pk }}Coords,
      strokeColor: "#FF0000",
      strokeOpacity: 0.8,
      strokeWeight: 2,
      fillColor: "#FF0000",
      fillOpacity: 0.35
    });

    for (i = 0; i < polygon{{ object.pk }}Coords.length; i++) {
      bounds.extend(polygon{{ object.pk }}Coords[i]);
    }

    the_polygon{{ object.pk }}.setMap(map);
    {% endif %}
    {% endfor %}

    map.fitBounds(bounds);
  }

</script>
{% endblock head_override %}

{% block body_override %} onload="mapInitialize()"{% endblock body_override %}

{% block content %}
<div id="map_canvas" style='width:600px;height:400px;'></div>
{% endblock content %}

Rendered out, the (4.4MB) file looks like this.

But we want our apps to be more interactive – so let’s quickly add popup windows when something is clicked on. The blog isn’t going to handle a diff very well, and that’s the best way to show it, so check out this gist. It’s not very difficult at all. Now we get popups when somebody clicks a marker, without writing a single line of view code. I tend to build methods on my models for map_display_html() or something similar. If I’m really returning HTML I’ll put it in a template and use the template language to make it work, if it’s just a name and hyperlink or something similar I just leave it in the model.