Smathermather's Weblog

Remote Sensing, GIS, Ecology, and Oddball Techniques

Archive for April, 2012

Building simple clients for MapFish — cURL as a client

Posted by smathermather on April 25, 2012

I have two previous posts on using MapFish (in this case, the GeoServer version) to allow for printing to hi-resolution PDF maps from the browser.  Here we use a command-line browser (cURL) to post our json to the MapFish service in order to retrieve our PDF.

I did not keep any notes from before on making json posts to the MapFish server as a means by which to test any manual configuration of the json file, so I had to rediscover this approach from pages like this.

The “@” sign below is so that curl knows I’m feeding it a file instead of the actual json to post:

curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST --data @mapfish_landscape.json http://localhost:8080/geoserver/pdf/create.json

Posted in Database, GeoExt, GeoExt, GeoServer, MapFish, PostGIS, PostgreSQL, SQL | Tagged: , , , , , , , , , | 3 Comments »

EcoHackNYC, version 2.0

Posted by smathermather on April 22, 2012

I had the pleasure this weekend of attending EcoHackNYC for the second time.  It was a generous, brilliant, and fun group to join for a day of hacking.  Sadly, this time, as last, I had public transit issues, so I did not make it for the Friday night ignite talks.  There were a number of talks and topics; swing over to twitter and search for #ecohacknyc to get a tweety view of it.

I presented a project idea on Saturday AM re: putting together mobile-ready field forms for performing plant community assessment protocol (PCAP).  In short, plant community assessments allow us to make qualitative assessments of plant communities, so that we can distinguish between high quality, medium quality, and low quality habitats.  The process stems out of the tradition of wetland monitoring programs from the State of Ohio, recently adapted for terrestrial ecosystems as well.

The interesting thing about these protocols, relative to simple diversity indices, is that alike communities are compared.  So, for example in wetland assessment, the wetland hydrogeomorphic (HGM) class is considered, meaning a high-quality fen will be compared with other fens, but not to a coastal wetland.  This prevents high quality ecosystems of naturally lower diversity, lower plant sensitivity, etc. from being scored relative to systems that are naturally more diverse, and/or have more or a greater number of sensitive species.

What I proposed at EcoHackNYC is that a group of us work on putting together tablet-ready field sheets to help streamline the data collection process for these datasets.  Presenting at the last minute, as I had missed the ignite sessions, I got no nibbles on working on the project.

So, I sat with a group building this:

An interface for communicating local demand for community supported agriculture (CSA) through a simple map interface.  It’s meant to be a tool to aid in the development of new CSA drop-offs based on local demand.  You can follow the project on twitter @csabuilder.  It’s built on Heroku for hosting, MapBox for map tiles, Leaflet for the mapping interface, jQuery for most everything else, and leveraging CartoDB for PostGIS goodness.

In the interest of full disclosure, I wrote not a lick of code for this projects– I mostly just kibitzed and made snide remarks, while working on getting Open Data Kit (ODK) to bend to my will in developing field forms for the aforementioned PCAP project.  I may have a forthcoming post on ODK, which was fun and easy(ish) to set up and deploy for creating Android form management.

Posted in Conferences, EcoHackNYC, Mobile, ODK | Tagged: , , | Leave a Comment »

Quick narrative on the distinction between leadership and power… .

Posted by smathermather on April 20, 2012

Leadership vs. Power:

Off topic here a bit.  I think this post will end up categorized as “other”… .

A while back I met a fellow we’ll call Bob, who has spent a long time in the tech world, implementing, for example, some of the first closed circuit TVs for video feeds at a statehouse in the 1970s.

As he tells the story, the closed circuit TVs were so that House members would be able to keep track of developments on the House floor (e.g. know when it was time to vote) from their offices, therefore making them more efficient.  Bob was the contractor managing the installation and decided it would be better for the democratic process if the governor also had access to that feed.  So, he ran coax cable up to the governor’s office, installed a television, and turned the feed on.

Several hours later, he got a call from the Speaker of the House demanding that he turn off the feed to the governor.  His answer: “Yes Maam.”

Two hours later, the Speaker called back, “Bob.  Turn the feed back on.”

“Yes Maam.”

“And Bob.”

“Yes Maam?”

“You played me.”

“Yes Maam.”

Bob told this story as a parable of the distinction between power and leadership to his grandson.  The Speaker of the House had power.  Bob demonstrated leadership.  I don’t know if his grandson was listening, but I know I was.

Posted in Other | Leave a Comment »

Kicking the tires of PostGIS 2.0 — Testing ST_MakeValid

Posted by smathermather on April 20, 2012

The feature in PostGIS 2.0 that excited me most was not topology support, raster support, or 3D functions.  Ok, raster was near the top of my list.  But what I was really excited by was the ST_MakeValid function.  Sad, isn’t it?  Lack of vision probably– excited to try to solve recurring technical snafus in a computationally inexpensive way, rather than being more excited by the shiny new toys the PostGIS team has brought us for applying to new problems.

Never shy to try the impossible before trying the practical, I threw a really nasty 4 million acre, 3 meter vectorized landscape position dataset a colleague of mine put together.  It’s a nice dendritic, complicated mess, with polygon validity issues:

Always one for doing tests, I compared the speed with which ST_MakeValid fixed the polygons vs. Horst Duester’s cleanGeometry pgsql function, which I posted about earlier.  Sadly ST_MakeValid was much slower, and completed with error.  On the other hand, the cleanGeometry function dropped polygons, although I haven’t had time to look into why.  I don’t like my functions dropping data… .

Having posted to the PostGIS forum, Martin Davis poked a bit at the data and concluded that my problem was “self-touching polygons”, and suggested I try buffering the data a zero distance.  Ahh– it makes sense that this is the issue– it’s polygonized raster data in shapefile form.  Self-touching polygons are valid in the ESRI world but not in the simple features world (see Paul Ramsey’s presentation on PostGIS for Power Users from Foss4G 2011 for more on this).

So, ST_Buffer(the_geom, 0), and we’re good to go in half the time of the cleanGeometry function, without losing data.

Stay tuned for an ST_MakeValid wrapper which will do this trick for self-touching and self-intersecting polygons before trying to fix with ST_MakeValid.  Code below.

psql -U postgres -d test -f "C:\Program Files\PostgreSQL\9.1\share\contrib\postgis-2.0\legacy.sql"
 SELECT gid, id, gridcode, "class name", cleanGeometry(geom) AS the_geom
 FROM tpi;

 SELECT gid, id, gridcode, "class name", ST_MakeValid(geom) AS the_geom
 FROM tpi;
  SELECT gid, id, gridcode, "class name", ST_Buffer(geom, 0) AS the_geom
  FROM tpi;



Posted in Database, Ecology, Landscape Position, PostGIS, PostgreSQL | Tagged: , , , , , , , , | Leave a Comment »

Cartography and USGS — Fake Building Footprints in PostGIS now with distance operator

Posted by smathermather on April 17, 2012

Quick and fun post tonight.  Remember in USGS quads all the little building footprints that represented civilization?  We (me and my colleague John Stein) were contemplating how to pull off something similar with address points.  Here was our first attempt:

It looks ok, but may be a little crude to be considered cartography (click on it to see it bigger– you’ll see those buildings don’t look quite right… ).  So, what we wanted to do of course was to rotate these buildings relative to the road network.  With ST_ClosestPoint in PostGIS (1.5 and beyond) this is easier than advertised:

ALTER TABLE jstein.address_points_all ADD COLUMN azimuth integer;
UPDATE jstein.address_points_all addr
SET azimuth = degrees(ST_Azimuth(addr.the_geom, ST_ClosestPoint(road.the_geom, addr.the_geom)))
FROM jstein.street_centerlines road;

and Walla! (or voila, or something):

Ahh, now that, with some additional work, should result in some very nice cartography.  The buildings now follow the angle of the road (so long as we always use a square… ).  USGS quads are dead.  Long live USGS quads!

Posted in Analysis, Cartography, Database, PostGIS, PostgreSQL, SQL | Tagged: , , , , , | 2 Comments »

G. I. O.: A Real American Hero (FOSS4G-NA)

Posted by smathermather on April 14, 2012

Whew! What a conference! All the classic FOSS4G stuff was there, from why the breaks in version for PostGIS (serialization + more…) = 2.0 rather than 1.6, to “here are the new combinations of existing technology stacks”, to the “tribes” of Open Source GIS.

I will confess: when I went to FOSS4G 2011 in Denver, I was a little terrified to actually meet and interact with the minds and persons behind the ecosystem of Open Source tools that I use and follow. This time, FOSS4G North America, I was a little more comfortable in my own skin. I had some nice conversations with lead developers and other cognoscenti on some of my favorite projects, including GeoServer, JTS, PostGIS, and CartoDB. This remains a remarkably accessible and down-to-earth group, in spite of the rapid growth of Open Source Software in the GeoSpatial sector, with warm and knowledgeable personalities.

Subtitle: FOSS4G-NA– send in the Bureaucrats!

The thing that was different about this conference was the preponderance of bureaucrats. Making the list were Michael Byrne from the FCC and Keith Barber, from the NGA, plus a few from DOD that were at the vendor session on Thursday. Later I’ll create a blog, likely called Straw-Hat-Hacker on my thoughts on Peace, Social Justice, and Anabaptism and my thoughts on the interaction between technology, FOSS, FOSS4G and the military industrial complex. For now, I’ll just state there is a strong connection, and that connection was represented at the start and end of the conference.

Speaking-wise, the highlight from The Bureaucrats was Michael Byrne, the Geographic Information Officer (GIO) for the Federal Communications Commission. It was an entertaining, well structured, and informative presentation, advocating for RESTFul, transparent, paperless, as well as cost-effective governance, with tangible examples from his own work within the FCC, including getting his hands dirty with a MacBook Pro and MapBox TileMill for quickly provisioning great “enterprisey” products for decision-makers and the public. I look forward to re-watching it when the videos become available online.

Given all the politics we watch from outside The Beltway, it was really nice to see that genuine effort goes into informing decision makers by very talented and genuine characters like the FCC’s GIO. Bravo.

Posted in FOSS4G-NA | Leave a Comment »