Smathermather's Weblog

Remote Sensing, GIS, Ecology, and Oddball Techniques

Posts Tagged ‘GeoExt’

OpenLayers, GeoExt, GeoServer, and GetFeatureInfo

Posted by smathermather on May 6, 2012

I wrote an earlier post on using GetFeatureInfo through OpenLayers to bring back a formatted html document with pictures, and formated tables, etc.  It wasn’t sophisticated, but got the job done.  Since around that time, as I’ve been building out our services, the speed with which a GetFeatureInfo request returns has g o t t  e  n  p    r     o     g       r       e        s         s            i             v              e             l            y     slower, to the point where the feature is essentially unusable.  Why?  Poor client coding.  Here’s my getfeatureinfo codeblock for OpenLayers:


////// WMS GetFeatureInfo
var info = new OpenLayers.Control.WMSGetFeatureInfo({
             drillDown : false,
queryVisible : true,
panMapIfOutOfView : false,
             url : GeoserverWMS,
layerUrls : [GeowebcacheURL],
eventListeners : {
getfeatureinfo : function (event) {
popup = new OpenLayers.Popup.FramedCloud(
                             "popinfo",
map.getLonLatFromPixel(event.xy),
null,
event.text,
null,
                             true);
map.addPopup(popup, true);
}
}
});

map.addControl(info);
info.activate();

//  end of popup code

The problem?  This code will query all visible layers.   Lots of visible layers results in a GetFeatureInfo request that looks like this:
localhost:8080/geoserver/wms?&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetFeatureInfo&LAYERS=summer_aerial_2,summer_aerial_1,reservation_boundaries_public_private_cm_dissolved_mask_gradien,odot_interstate,odot_us_routes,odot_state_routes,reservation_bounds,detailed_hydro_view,cm_bridge_view,cm_trails,impervious_update,cm_buildings,cm_buildings_outline,golf_view,nhd_lake_erie,supplementary_shields,planet_osm_line,cuyahoga_street_centerlines_labels,planet_osm_line_outside_cuy,detailed_hydro_labels,facilities_cm,facility_areas_cm&QUERY_LAYERS=summer_aerial_2,summer_aerial_1,reservation_boundaries_public_private_cm_dissolved_mask_gradien,odot_interstate,odot_us_routes,odot_state_routes,reservation_bounds,detailed_hydro_view,cm_bridge_view,cm_trails,impervious_update,cm_buildings,cm_buildings_outline,golf_view,nhd_lake_erie,supplementary_shields,planet_osm_line,cuyahoga_street_centerlines_labels,planet_osm_line_outside_cuy,detailed_hydro_labels,facilities_cm,facility_areas_cm&STYLES=,,,,,,,,,,,,,,,,,,,,,&BBOX=2232424.250515%2C625357.428203%2C2233363.463113%2C625592.231353&FEATURE_COUNT=10&HEIGHT=213&WIDTH=852&FORMAT=image%2Fpng&INFO_FORMAT=text%2Fhtml&SRS=EPSG%3A3734&X=320&Y=91
My naive attempts to fix this were simply adding a layers key/value pair, e.g.:

////// WMS GetFeatureInfo
     var info = new OpenLayers.Control.WMSGetFeatureInfo({
drillDown : false,
queryVisible : true,
panMapIfOutOfView : false,
             url : GeoserverWMS,
layers : [reservation_bounds],
layerUrls : [GeowebcacheURL],
eventListeners : {
getfeatureinfo : function (event) {
                     popup = new OpenLayers.Popup.FramedCloud(
                             "popinfo",
map.getLonLatFromPixel(event.xy),
null,
event.text,
null,
                             true);
map.addPopup(popup, true);
}
}
});

map.addControl(info);
info.activate();

//  end of popup code
… but for whatever reason, maybe the way I instantiated the OpenLayers.Layer.WMS instances using Ext.each (ahem, the way I modified a vendor’s cleverly written code…), the layers aren’t recognized as objects by the name I might expect.  So, I hack and create a duplicate layer by a different name.  I’ll have a non-hack version here soon (I hope) which will use the existing array of WMS instances, loop through them, and only add the appropriate ones to the layers array.  In the mean time, this should work.
var reservation_bounds = new OpenLayers.Layer.WMS("Reservation Boundaries", GeoserverWMS,
{'layers': 'base:reservation_bounds', transparent: true, format: 'image/png'},
             {isBaseLayer: false}
);

////// WMS GetFeatureInfo
var info = new OpenLayers.Control.WMSGetFeatureInfo({
drillDown : false,
queryVisible : true,
panMapIfOutOfView : false,
             url : GeoserverWMS,
layers : [reservation_bounds],
layerUrls : [GeowebcacheURL],
eventListeners : {
getfeatureinfo : function (event) {
popup = new OpenLayers.Popup.FramedCloud(
                             "popinfo",
map.getLonLatFromPixel(event.xy),
null,
event.text,
null,
                             true);
map.addPopup(popup, true);
}
}
});

map.addControl(info);
info.activate();

//  end of popup code

Posted in GeoExt, GeoServer, Javascript, OpenLayers | Tagged: , , , , | Leave a Comment »

Building simple clients for MapFish — cURL as a client

Posted by smathermather on April 25, 2012

I have two previous posts on using MapFish (in this case, the GeoServer version) to allow for printing to hi-resolution PDF maps from the browser.  Here we use a command-line browser (cURL) to post our json to the MapFish service in order to retrieve our PDF.

I did not keep any notes from before on making json posts to the MapFish server as a means by which to test any manual configuration of the json file, so I had to rediscover this approach from pages like this.

The “@” sign below is so that curl knows I’m feeding it a file instead of the actual json to post:


curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST --data @mapfish_landscape.json http://localhost:8080/geoserver/pdf/create.json

Posted in Database, GeoExt, GeoExt, GeoServer, MapFish, PostGIS, PostgreSQL, SQL | Tagged: , , , , , , , , , | 3 Comments »

Building simple clients for MapFish — Beginnings of a PL/pgSQL function

Posted by smathermather on March 10, 2012

I’ve had a couple of other posts (1 and 2 and 3 and) on simple clients for MapFish.  I like the client server infrastructure for MapFish– with the client end of things built up in GeoExt, it makes for a really elegant combo.  But I’d like articulate my vision for simple clients for MapFish a little further.  One thing that seems quite feasible is to embed the JSON for the MapFish requests in a PostgreSQL table.  Why there and not just within our client?  Well, we can use PostGIS to construct really clever multi-page prints if we want to, build into PostGIS the logic to decide the orientation, number of pages, scale, and other information needed to decide how best to print this object, and we can access that JSON through a GetFeatureInfo Request through any WMS compliant server (e.g. GeoServer).  In this way, we can use the GetFeatureInfo bubble as a place where we have links (enhanced with a little javascript) to post the JSON to our MapFish service and return a PDF.

Any object we want exposed through our interface could have a link associated with it that generates a pdf map of that object.  Let’s start with the functionality we want in our PostgreSQL function and figure out what it needs to generate the JSON we want. Here’s what we want our JSON to look like, at least for a very simple example:

{
	"units" : "ft",
	"srs" : "EPSG:3734",
	"layout" : "1) LETTER 8.5x11 Portrait",
	"dpi" : 300,
	"serviceParams" : {
		"locale" : "en_US"
	},
	"resourcesUrl" : "http://maps/geoserver/www/printing",
	"layersMerging" : true,
	"preferredIntervalFractions" : [0.1, 0.2, 0.4],
	"metaTitle" : "Title Here Please! GIS Print",
	"metaAuthor" : "Title Here Please!",
	"metaSubject" : "Title Here Please! GIS Print",
	"metaKeywords" : "",
	"outputFilename" : "cm_gis",
	"legends" : [],
	"layers" : [{
			"baseURL" : "http://maps/geoserver/wms?",
			"opacity" : 1,
			"singleTile" : false,
			"type" : "WMS",
			"layers" : ["cuy_bridge_decks", "planet_osm_line_outside_cuy_map", "cuy_roads_poly", "cuy_street_centerlines", "reservation_bounds_solid"],
			"format" : "image/png",
			"styles" : [""],
			"customParams" : {
				"TILED" : "false",
				"TRANSPARENT" : true
			}
		}
	],
	"pages" : [{
			"center" : [2160649.7795275, 597547.8687664],
			"scale" : 6000,
			"rotation" : 0,
			"mapTitle" : "Title Here Please!"
		}
	]
}

As a starting point, we can split this into two sections, the global parameters, i.e. everything except “pages” (pages is what we want postgis to calculate for us).  In the most generic sense, we would want to pass all of the parameters in the global section to the function, plus the geometry of the object over which we want to print the extent, plus the actual print size of the printable area for the desired layout have it return the json, with a population of pages section done by a little PostGIS magic. PL/pgSQL to come… .

Posted in Database, GeoExt, GeoExt, GeoServer, MapFish, PostGIS, PostgreSQL, SQL | Tagged: , , , , , , , , | Leave a Comment »

Building simple clients for MapFish — Underlying Infrastructure

Posted by smathermather on February 10, 2012

In order to build simple clients for the MapFish print service, we have to understand what the protocols are that are invoked and how they function.  To do this we can read the MapFish Print Module Doc, and then modify and vamp from there.  While I was going to joke that this would be an excellent cure for insomnia, the joke fell apart when I actually read the Print Module Doc, and it was clear, concise, readable, and frankly at least as interesting as most of my blog entries (no comments here please).

But, I did that after the fact.  In other words, I did it the hard way– just like you do when you bring home that really cool electronic toy, and play with it for a couple hours before (maybe, if ever) picking up the manual.  So for me, instead of reading the excellent documentation, I sniffed the protocol using a GeoExt interface to a GeoServer/MapFish combo with Firebug.  (quick aside– if you write web stuff and are new to it, then you should know you should use Firebug to write it better and test it on the fly– or if you’re a cool kid with horn-rimmed glasses and an ironic t-shirt, do it in Google Chrome’s Javascript Console, it makes no real difference as far as outcomes, just a difference in style).

In sniffing the protocol, I saw that my request for a PDF was a POST request to the server, with a JSON object as the request.  In short, the interface converts my form information into a bunch of text (a javascript object) which it pushes to the server.  The server location in this case is: http://localhost:8080/geoserver/pdf/create.json.  The text it’s pushing is a file which reads something like this:

{
	"units" : "ft",
	"srs" : "EPSG:3734",
	"layout" : "1) LETTER 8.5x11 Portrait",
	"dpi" : 300,
	"serviceParams" : {
		"locale" : "en_US"
	},
	"resourcesUrl" : "http://maps/geoserver/www/printing",
	"layersMerging" : true,
	"preferredIntervalFractions" : [0.1, 0.2, 0.4],
	"metaTitle" : "Title Here Please! GIS Print",
	"metaAuthor" : "Title Here Please!",
	"metaSubject" : "Title Here Please! GIS Print",
	"metaKeywords" : "",
	"outputFilename" : "cm_gis",
	"legends" : [],
	"layers" : [{
			"baseURL" : "http://maps/geoserver/wms?",
			"opacity" : 1,
			"singleTile" : false,
			"type" : "WMS",
			"layers" : ["cuy_bridge_decks", "planet_osm_line_outside_cuy_map", "cuy_roads_poly", "cuyahoga_street_centerlines", "reservation_bounds_solid"],
			"format" : "image/png",
			"styles" : [""],
			"customParams" : {
				"TILED" : "false",
				"TRANSPARENT" : true
			}
		}
	],
	"pages" : [{
			"center" : [2160649.7795275, 597547.8687664],
			"scale" : 6000,
			"rotation" : 0,
			"mapTitle" : "Title Here Please!"
		}
	]
}

I’ll call your attention to the last little bit of code in the object:


"pages" : [{
		"center" : [2160649.7795275, 597547.8687664],
		"scale" : 6000,
		"rotation" : 0,
		"mapTitle" : "Title Here Please!"
	}

]

I was proud of myself for recognizing (through the haze of a guy who’s modified a lot of javascript, but never learned it proper-like) that this is a javascript array with just one object.  Which means, we can make it an array with more than one object.  Eureka!  multi-page pdfs with just 5-6 more lines of code:


"pages" : [{
		"center" : [2160649.7795275, 597547.8687664],
		"scale" : 6000,
		"rotation" : 0,
		"mapTitle" : "Title Here Please!"
	}, {
		"center" : [2216902.0734907, 596701.84251968],
		"scale" : 1800,
		"rotation" : 0,
		"mapTitle" : "Title Here Please!"
	}
]

Now, just to figure out how to test this out without building a web page to do it.  curl is our friend here, just a few extra flags for telling the server what we are doing with the json file (thanks to this post):

curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d @test.json http://localhost:8080/geoserver/pdf/create.json

And now for all the json fit to print:

{
	"units" : "ft",
	"srs" : "EPSG:3734",
	"layout" : "1) LETTER 8.5x11 Portrait",
	"dpi" : 300,
	"serviceParams" : {
		"locale" : "en_US"
	},
	"resourcesUrl" : "http://maps/geoserver/www/printing",
	"layersMerging" : true,
	"preferredIntervalFractions" : [0.1, 0.2, 0.4],
	"metaTitle" : "Title Here Please! GIS Print",
	"metaAuthor" : "Title Here Please!",
	"metaSubject" : "Title Here Please! GIS Print",
	"metaKeywords" : "",
	"outputFilename" : "cm_gis",
	"legends" : [],
	"layers" : [{
			"baseURL" : "http://maps/geoserver/wms?",
			"opacity" : 1,
			"singleTile" : false,
			"type" : "WMS",
			"layers" : ["cuy_bridge_decks", "planet_osm_line_outside_cuy_map", "cuy_roads_poly", "cuyahoga_street_centerlines", "reservation_bounds_solid"],
			"format" : "image/png",
			"styles" : [""],
			"customParams" : {
				"TILED" : "false",
				"TRANSPARENT" : true
			}
		}
	],
	"pages" : [{
			"center" : [2160649.7795275, 597547.8687664],
			"scale" : 6000,
			"rotation" : 0,
			"mapTitle" : "Title Here Please!"
		}, {
			"center" : [2216902.0734907, 596701.84251968],
			"scale" : 1800,
			"rotation" : 0,
			"mapTitle" : "Title Here Please!"
		}
	]
}

*Updated with better formatted sourcecode, thanks to notepad++’s JSMin plugin

Posted in Database, GeoExt, GeoExt, GeoServer, MapFish, PostGIS, PostgreSQL, SQL | Tagged: , , , , , , , , | 1 Comment »

Ditching GeoExt– building simple clients for MapFish

Posted by smathermather on February 8, 2012

I’ve been enamored with the GeoExt interface for grabbing MapFish based print services since I first saw it. It’s a slick little interface, and can even been extended for multi-page print layouts pretty easily, ala http://tinyurl.com/mapfishmultipageprint. But as I’ve started to give thought not to what an organization full of professionals needs but what a public interface should looks (and probably those interfaces for professional organizations as well, only they tend to be more tolerant of poor design…), I’ve begun to realize that there are some clever ways we can bypass the GeoExt interface for generating the print documents at all. The actual request for a document is a simple post with a JSON object that has certain properties. We can construct this object all sorts of ways.

So, in the interest of making the snake eat its tail, my objective over the next few posts is to create an entry in a PostGIS database that has a view that automatically parses the “best” multi-page print for the geometry, feeds that back like a good API to the client script, which then requests a pdf print based on those criteria through the GeoServer Mapfish extension. Are we clear as a computer program, usually running over the Internet, that allows multiple users to participate in virtual-reality role-playing games? Good.

Posted in Database, GeoExt, GeoExt, GeoServer, MapFish, PostGIS, PostgreSQL, SQL | Tagged: , , , , , , , , | 5 Comments »

PostgreSQL Views within GeoServer, GetFeatureInfo with Freemarker Templates, etc.

Posted by smathermather on August 20, 2011

GeoServer now has the ability to consume database views from PostGIS, not just raw tables. I say it “now” has that ability– I think that came online with GeoServer 2.x series, but I’m just “now” starting to take advantage of it. You can also create views on the fly within GeoServer, but I prefer to apply the logic at the database level, just in case I use something instead of or supplemental to GeoServer in the future, the application logic is built in at the PostGIS/PostgreSQL level.

To this end, we have an infrastructure database that is maintained in Access, the records of which I’d like a copy of in PostgreSQL. Long term, we will probably move the records over to PostgreSQL and link them, but for now we’ll retain two copies– a master copy in the Access database and a slave copy in PostgreSQL.

The tables in question were exported from Access initially (to get their schema just right) through an ODBC connection, and then re-added as linked tables through an ODBC connection. I decided that synchronization should happen only when I’m the user, so I don’t slow down other users’ experiences, so the code for this is embedded in the main form and looks like this (just a little VBA):

 Private Sub Form_Close() 'Test for user If (Environ("username") = "smathermather") Then 'Disable user confirmation warnings DoCmd.SetWarnings False 'Remove all records in building tables DoCmd.RunSQL "DELETE FROM [public_Building Photos]" DoCmd.RunSQL "DELETE FROM [public_Building Statistics]" DoCmd.RunSQL "DELETE FROM [public_Building Utilities]" DoCmd.RunSQL "DELETE FROM [public_Master Building Inventory]" 'Reload all records in building tables DoCmd.RunSQL "INSERT INTO [public_Building Photos] SELECT [Building Photos].* FROM [Building Photos]" DoCmd.RunSQL "INSERT INTO [public_Building Statistics] SELECT [Building Statistics].* FROM [Building Statistics]" DoCmd.RunSQL "INSERT INTO [public_Building Utilities] SELECT [Building Utilities].* FROM [Building Utilities]" DoCmd.RunSQL "INSERT INTO [public_Master Building Inventory] SELECT [Master Building Inventory].* FROM [Master Building Inventory]" 'Reenable user confirmation warnings DoCmd.SetWarnings True End If End Sub 

Great, now the architecture database dumps into mine. It runs whenever I close the form, although this could be tied to a button or some other mechanism as well. Ideally, I suppose it would be simple enough to create a separate database with linked tables from each, have it run on start, and then schedule it to run periodically.

So, what to do next? Well this is a rich database with all sorts of information– we’ll need curate that information (is this an abuse of that word?). It has all sorts of building stats built in, a photo tied to the database etc.. This can all be curated with a database view.

 CREATE OR REPLACE VIEW public.cm_buildings AS SELECT build.name, build."Year Constructed", build."Square Footage", build."Construction Materials", build."Use Group", build."Architect", build."Structural Engineer", build."General Contractor", build."Site Location", build."Street Address", build."City", build."Phone", build."Capacity", build.link, build.the_geom, res."RES_LINK" AS vs_link FROM public."Building Reservable" res RIGHT JOIN ( SELECT photo."Building Name" AS name, stats."Year Constructed", stats."Square Footage", stats."Construction Materials", stats."Use Group", stats."Architect", stats."Structural Engineer", stats."General Contractor", master."Site Location", master."Street Address", master."City", master."Phone", master."Notations" AS "Capacity", regexp_replace(regexp_replace((string_to_array(photo."txtStockGraph", '\\'))[4], ' ', '_'), '.jpg', '') AS link, foot.the_geom FROM public."Master Building Inventory" master, public."Building Statistics" stats, public."Building Photos" photo, public.facilities_footprints_cm foot WHERE master."Building" = stats."Building" AND master."Building" = photo."Building Name" AND (master."Amenities" = 'BLD' OR master."Amenities" = 'CCF') AND master."Building Code" = foot."Building Code") build ON res.b_name_ar = build.name; ALTER TABLE public.cm_buildings OWNER TO postgres; GRANT ALL ON TABLE public.cm_buildings TO postgres; 

Now here’s the fun part. The paths to the images were several directories deep, etc., but I wanted to dump them all in a single directory. So the interesting part of the query is reformating the relative paths for the image locations and just grabbing the portion with the image name. In this case, they are all at the same depth in the directory structure, so I convert the string to an array using the backslash as my data separator, grab the 4th element, and then trim off the extension “.jpg” so I can manipulate the name further– like take advantage of some thumbnails I created with the same name but a “*.png” extension. That bit of (Postgres enhanced) SQL is as follows (it’s also in the above CREATE VIEW code, excerpted here for clarity):

 regexp_replace(regexp_replace((string_to_array(photo."txtStockGraph", '\\'))[4], ' ', '_'), '.jpg', '') AS link 

So, now we have a table with fields that can inform the user and serve as links to images available on the server. GeoServer allows for some really cool HTML templating with Freemarker Templates that we can take advantage of here. For a simple example, see the tutorial on GeoServer’s site.

For our example, we vamp a little (though not much):


<#--
Body section of the GetFeatureInfo template, it's provided with one feature collection, and
will be called multiple times if there are various feature collections
-->
<table>
  <tr>
<#list type.attributes as attribute>
  <#if !attribute.isGeometry>
    <#if attribute.name!="link">
    <#if attribute.name!="vs_link">
      <th>${attribute.name}</th>
   </#if>
   </#if>
  </#if>
</#list>
  </tr>

<#assign odd = false>
<#list features as feature>
  <#if odd>
    <tr>
  <#else>
    <tr>
  </#if>
  <#assign odd = !odd>
  <#list feature.attributes as attribute>
    <#if !attribute.isGeometry>
     <#if attribute.name!="link">
     <#if attribute.name!="vs_link">
       <td>${attribute.value}</td>
     </#if>
     </#if>
    </#if>
  </#list>
  <#if feature.link.value!="">
      <img src="arch/img/${feature.link.value}.png"><br><br/>
      <a href="www/arch/img/${feature.link.value}.jpg" target="_blank">Full Size Image</a> <br><br/>
  </#if>
  <#if feature.vs_link.value!="">
        <a href="http://external_link.com/${feature.vs_link.value}" target="_blank">Reservable</a> 
  </#if>
  </tr>
</#list>
</table>
<br/>

 

 

(Confession: I haven’t double checked my HTML yet to make sure that it’s perfect… ). Now we have to have a way for the user to get this info back with a GetFeatureInfo query, good old WMS standby (and essentially a subset of WFS features):

 var info = new OpenLayers.Control.WMSGetFeatureInfo({ drillDown: false, queryVisible: true, panMapIfOutOfView: false, url: GeoserverWMS, layerUrls: [GeowebcacheURL], eventListeners: { getfeatureinfo: function(event) { popup = new OpenLayers.Popup.FramedCloud( "popinfo", map.getLonLatFromPixel(event.xy), null, event.text, null, true ); map.addPopup(popup, true); } } }); map.addControl(info); info.activate(); 

And the results? Splendid:

Map of picnic area with GetFeatureInfo enabled on (brown) picnic structure.

Picnic area with GetFeatureInfo triggered

Next trick will be to start building out tabbed interfaces inside the GetFeatureInfo bubble to consolidate and streamline the information as we add the available info.

Posted in Database, GeoServer, OpenLayers, PostGIS, SQL | Tagged: , , , , , , , , , , | 5 Comments »

Really loooong WMS requests

Posted by smathermather on August 3, 2011

We have a GeoExt/Mapfish/GeoServer/PostGIS stack in house that allows us to print nice maps through a web interface. There has been a ceiling, however, as far as size– 140dpi and 22×34 has been the largest we’ve been able to render maps, and even then, sometimes the layers are too complicated.

For a long time I was under the naïve impression that it was a memory issue with JAITools. The GeoServer users group set me straight on that– it was a timeout issue. A couple more posts to GeoServer Users and Mapfish, and I really got set straight– in my case it was a client side timeout issue (I had already allowed for long WMS requests in GeoServer and in Mapfish’s config.yaml). Since the stack includes GeoExt, and therefore leverages ExtJS, the solution is simple. I now have the following line at the start of my javascript document:


Ext.Ajax.timeout = 120000; //2 minutes

Now we can do 300dpi images/maps at 36 inches x 48 inches (about A0 for the folks on the International system).

Posted in GeoExt, GeoServer, MapFish | Tagged: , , , , | Leave a Comment »