Smathermather's Weblog

Remote Sensing, GIS, Ecology, and Oddball Techniques

Archive for July, 2012

Independence– Free as in Freedom, part II

Posted by smathermather on July 21, 2012

A friend of mine read my first post “Independence– Free as in Freedom” and said this:
“awesome blog and great post. did jack respond?”

My response:

“Nooo.  I think I’m unknown in the ESRI world ….  But it was a heartfelt post.  I think Jack has changed the world, but recognize it has changed him.  I’m pulling for that organization to radically re-invent itself.  The reality is, with their hosting (ahem.  cloud) services they very well could, but only if they also let go of code control.  Then they’ll spin off a hundred little ecosystems of GIS economy and themselves grow again and thrive again.

“They should be looking to a model like CartoDB, open source software on a hosted format.  It creates competition, fosters creativity, but the risk averse will still aim to have ESRI hosting, as they are the experts, meanwhile thought leaders, risk takers, and startups will deploy their own instances and contribute code and ideas back to the ecosystem.  Right now, all the cool and really talented geospatial developers are using FOSSGIS partially or entirely, and ESRI isn’t getting those ideas and developments back into their work.

“The real value in the GIS ecosystem isn’t government contracts.  The real value is in the trade of thoughts, ideas, and methods.  Until ESRI dives into that ecosystem, they can’t recoup that value.  Which is almost unfair, as they played such a large role in creating that ecosystem in the first place.

Hmm.  That might be another post.

Posted in Other | Tagged: , | Leave a Comment »

Using OSM Nominatim to search for geographic objects

Posted by smathermather on July 21, 2012

Just playing a little today.  I’m at Cleveland Give Camp today, working on a project for Ohio City Writers, a non-profit youth creative writing center.  I’m deploying some maps for the registration site, that will be filterable, using CartoDB as a host for the data.  As I don’t know what we’re going to filter, or where the page will go yet, I was trolling around trying to find alternate data sources for things like locations of schools.

OpenStreetMap has a search service called Nominatim that allows for search of geographic objects across the world.  This is one of the tools that used on the OSM landing page when you search for a location.

This post won’t be much more than a reflector for the documentation, but it’s a nice easy to use service, i.e.:<query&gt;?<params>

and e.g.:,cleveland,oh

By default, it returns with HTML.  If we want xml or json, that’s not so hard:,cleveland,oh?format=json


"place_id" : "3095185",
"licence" : "Data Copyright OpenStreetMap Contributors, Some Rights Reserved. CC-BY-SA 2.0.",
"osm_type" : "node",
"osm_id" : "357479537",
"boundingbox" : [41.482427825928, 41.502431640625, -81.717878112793, -81.697870483398],
"lat" : "41.4924303",
"lon" : "-81.707872",
"display_name" : "Urban Community School, W 25th St, Detroit-Shoreway, Cleveland, Cuyahoga, Ohio, 44109, United States of America",
"class" : "amenity",
"type" : "school",
"icon" : "http:\/\/\/images\/mapicons\/education_school.p.20.png"
}, {
"place_id" : "3100746",
"licence" : "Data Copyright OpenStreetMap Contributors, Some Rights Reserved. CC-BY-SA 2.0.",
"osm_type" : "node",
"osm_id" : "357508524",
"boundingbox" : [41.451994171143, 41.47199798584, -81.720136413574, -81.70012878418],
"lat" : "41.4619957",
"lon" : "-81.7101311",
"display_name" : "Blessed Sacrament School (historical), Trowbridge, Cleveland, Cuyahoga, Ohio, 44109, United States of America",
"class" : "amenity",
"type" : "school",
"icon" : "http:\/\/\/images\/mapicons\/education_school.p.20.png"
//... etc.

I may have to use this for a future geographic search application.  I so often have to append OpenStreetMap’s Copyright on my applications anyway… .

Posted in OpenStreetMap | Tagged: , , | Leave a Comment »

Serving and filtering #GeoJSON from #GeoServer

Posted by smathermather on July 19, 2012

The nice thing about setting up something like GeoServer, which is so feature rich, is when you need to pivot based on the demands of a new project, the technical infrastructure is already there, just waiting to be configured or turned on.

The case today: feature services.  Someone wants to use my existing infrastructure in a new application.  I typically serve tile services.  One checkbox to enable WFS; a couple clicks to see an example GeoJSON request, and


Ah.  That wasn’t so bad.  But wait, we need to filter the request!  GeoServer offers CQL and ECQL:

http://localhost:8080/geoserver/cm/ows?service=WFS&version=1.0.0&request=GetFeature&typeName=cm:parks&outputFormat=json&cql_filter=(res=’Mildreds Favorite Park”)

Oops, I also need it reprojected on the fly to Google Mercator:

http://localhost:8080/geoserver/cm/ows?service=WFS&version=1.0.0&request=GetFeature&typeName=cm:parks&outputFormat=json&cql_filter=(res=’Mildreds Favorite Park”)&srsName=EPSG:3857

Not too bad for a feature I hadn’t needed until today… .

Posted in GeoServer | Tagged: , , , , | 1 Comment »

WordPress Country Stats, a cartographers complaint

Posted by smathermather on July 14, 2012

In the category of moderately new features in WordPress, I really appreciate the new WordPress Country Stats, with the nice touch of flags of the nations, and a map.  As an inveterate map snob, however, the map drives me crazy:

Why?  Because choropleth maps are bad for showing count data, unless all the polygons are the same size.  Look at the map above– France and the US have the same number of hits, but it is easier to notice the US, because of its larger land area.  It makes for a confusing or even misleading map.

So, how to fix it?  Cartograms are one solution.  But, they can be confusing, and many really don’t like them (they are often quite ugly).  Another solution is variable sized symbols, for example this map:

from WordPress year in review 2011.  This would require some thought relative to how to show this within the limited form factor of the WordPress stats map, but not a prohibitive amount of thought.   For example, symbol clustering to the continent level would work for large clusters of hits from Europe.

Anyway, my 2 cents… .  Still love that it’s there, even if it does drive me crazy.  🙂

Posted in Cartography, Other | Tagged: , , | 3 Comments »

Using Quantum GIS for real work…

Posted by smathermather on July 7, 2012

I have long lived in the desktop realm of ESRI, and expect ESRI products to be a part of my workflow for a long time to come.  But, the time has come to delve in deeply with Quantum GIS.  I’ve read from many sources that it has come of age as a desktop GIS.  I tried it 4 years ago, along with its competitors in the Open Source desktop GIS world, and was unimpressed.  Consider me convinced.

So why now?  Why try QGIS again?  I had a unpaid side project (church volunteer work).  Unpaid one-off projects inspire the use of FOSS tools.  That said, the tool is likely to now be part of my paid work flow as well.  It was easy to use, easy to compose a decent looking map in very little time, had nice tools for labeling and line rendering, as well as sane and useful defaults.  The one flaw I found was for a large format print (I wanted 24″x36″) the exported PDF was prohibitively large (158 Mb).  As I was working on a Mac, I simply printed to PDF with fine results and a reasonable file size (1.6 Mb).

Posted in Cartography, QGIS | Tagged: , , , | 1 Comment »

Independence– Free as in Freedom

Posted by smathermather on July 5, 2012

I recently gave a talk on a really interesting project I’m working on, and in the process of talking to a non-technical crowd introduced Open Source using some of my favorite (borrowed) phrases, i.e. “Free not as in free beer, but free as in freedom, free as in ‘free puppy'”, which resulting in a surprisingly satisfying chuckle from the audience.

What’s great about the project is that we’re doing some really cool stuff that will be useful to others, and because we’re building it on top of open source software, we can just share it, without licensing restrictions, without pay-to-play start-up for other users.

Today, I write a short post about “free software” and free as in freedom, in the spirit of the US holiday “Independence Day” or colloquially, “4th of July”.

ESRI recently had an article in both of the magazines they send me on Open Source, Open Standards, Open Data, Open APIs, etc.. ESRI catches a lot of flack from the Open Source geospatial community. We can quote Paul Ramsey’s recent blog post: “Thought experiment: if ESRI truly embraced interoperability, at some point would one not expect it to cease to be a noteworthy topic?”   I use ESRI software almost exclusively for desktop, and Open Source software exclusively for server, and love when there’s more and better integration.

And… ESRI has made some recent headway. While not well-manned, they had a booth at FOSS4G in 2011 promoting e.g. their work creating an OpenStreetMap editor for ArcGIS 10.x. They also had a booth at FOSS4G-NA this year, which was well-manned. I had the pleasure of walking past their booth where one of their machines had a blue screen of death. I was kind enough to not take a picture.  🙂 The folks manning the booth blamed Windows, so I suggested they get working on a serious Linux port… .

ESRI has also done some good work in the standards realm, in addition to the venerable shapefile standard, they’ve added open web APIs, and opened up the file geodatabase with an API to read and write 10.x geodatabases.

But alas, to date, no special sauce. No write access to any of these:

  • Annotation and Dimension feature classes
  • Relationship Classes
  • Networks (GN and ND)
  • Topologies
  • Terrains
  • Representations
  • Parcel Fabrics

Which is where the really cool (and largely patented) work that ESRI has done in the last few years.  Jack Dangermond often talks about changing the world for the better with revolutionary software and ideals.  And he has.  You’ve had an excellent Act I and Act II.  Finish the revolution, Jack.  Open up.

So, on this day of freedom (ahem, ok, today is the 5th).  ESRI.  Jack.  It’s time for Act III.

Posted in Other | Tagged: , | 1 Comment »