Smathermather's Weblog

Remote Sensing, GIS, Ecology, and Oddball Techniques

Archive for the ‘Photogrammetry’ Category

Finding peace, finding ground: Drone flights for hydrologic modeling

Posted by smathermather on October 15, 2016

Š-L-M

Lately I’ve been helping with project to predict flooding in Dar es Salaam (“Home of Peace”), the capital former capital of Tanzania in East Africa. This is a really fun project for me, in part because most of the hard work is already done, but there still remain some tricky problems to solve.

This project is a part of Dar Ramani Huria, a “community-based mapping project… training university students and local community members to create highly accurate maps of the most flood-prone areas of the city using OpenStreetMap.”

Dar es Salaam (or Dar for short), is the fastest growing city in Africa, making the identification of flood prone zones during rainy seasons absolutely critical. The Ramani Huria crew has mapped the streams, ditches, and other waterways of Dar, as well as flown imagery (via drone) over much of the city critical parts of the city.

screen-shot-2016-10-15-at-3-01-04-pm

Problem Space:

The results are stunning, but using drone imagery and photogrammetric point clouds (instead of LiDAR) has it’s limitations.

One problem, that I’m not going to get into in any depth today, is the difficulty of conflating (vertically and horizontally) different datasets flown at different times with commodity GPS.

Another problem is the difficulty of turning photogrammetrically derived point clouds into Digital Terrain Models. There is proprietary software that does this well (e.g. LasTools and others), but we seek a free and open source alternative. Let’s visualize the problem. See below a photogrammetrically-derived point cloud (processed in Pix4D, visualized at https://plas.io):

screen-shot-2016-10-15-at-3-15-44-pm

We can directly turn this into a digital surface model (DSM):

screen-shot-2016-10-15-at-3-24-23-pm

5

We can see the underlying topography, but buildings and trees are much of the signal in the surface model. If we, for example, calculate height above the nearest stream as a proxy for flooding, we get a noisy result.

OSM to the Rescue

If you recall at the beginning of this post, I said this is “a really fun project for me” because “most of the hard work is already done”. Dar is a city with pretty much every building digitized. We can take advantage of this work to remove buildings from our point cloud. We’ll use the utilities associated with the Point Data Abstraction Layer (PDAL) library. Specifically, we’ll use PDAL’s ability to clip point cloud data with 2D OGR compatible geometries (a great tutorial on this here).

First thing we need to do is extract the buildings from OSM into shapefiles. For this, and easy way is to use Mapzen’s Metro Extracts:

screen-shot-2016-10-15-at-3-58-35-pm

I chose Datasets grouped into individual layers by OpenStreetMap tags (IMPOSM) as this gives me the capacity to just choose “Buildings”. I loaded those buildings into a PostGIS database for the next steps. What I seek is a shapefile of all the areas which are not buildings, essentially from this:

building

To this:

no_building

For this we’ll need to use ST_Difference + ST_ConvexHull. ST_ConvexHull will create a shape of the extent of our data, and we’ll use ST_Difference to cookie-cutter out all the building footprints:

DROP TABLE IF EXISTS "salaam-buff-diff-split1";
CREATE TABLE "salaam-buff-diff-split1" AS
WITH subset AS (
	SELECT ST_Transform(geom, 32737) AS geom FROM "salaam-buildings" LIMIT 20000
	),
extent AS (
	SELECT ST_ConvexHull(ST_Union(geom)) AS geom FROM subset
	),
unioned AS (
	SELECT ST_Union(geom) AS geom FROM subset
	),
differenced AS (
	SELECT ST_Difference(a.geom, b.geom) AS geom FROM
	extent a, unioned b
	)
SELECT 1 AS id, geom FROM differenced;

This gives us a reasonable result. But, when we use this in PDAL, (I assume) we want a subdivided shape to ensure we don’t have to access the entire point cloud at any given time to do our clipping. We’ll add to this process ST_SubDivide, which will subdivide our shape into parts not to exceed a certain number of nodes. In this case we’ll choose 500 nodes per shape:

DROP TABLE IF EXISTS "salaam-buff-diff-split1";
CREATE TABLE "salaam-buff-diff-split1" AS
WITH subset AS (
	SELECT ST_Transform(geom, 32737) AS geom FROM "salaam-buildings" LIMIT 20000
	),
extent AS (
	SELECT ST_ConvexHull(ST_Union(geom)) AS geom FROM subset
	),
unioned AS (
	SELECT ST_Union(geom) AS geom FROM subset
	),
differenced AS (
	SELECT ST_Difference(a.geom, b.geom) AS geom FROM
	extent a, unioned b
	)
SELECT 1 AS id, ST_Subdivide(geom, 500) AS geom FROM
differenced;

subdivided

Finally, if we want to be sure to remove the points from the edges of buildings (we can assume the digitized buildings won’t perfectly match our point clouds), then we should buffer our shapes:

DROP TABLE IF EXISTS "salaam-buff-diff-split1";
CREATE TABLE "salaam-buff-diff-split1" AS
WITH subset AS (
	SELECT ST_Transform(geom, 32737) AS geom FROM "salaam-buildings" LIMIT 20000
	),
buffered AS (
	SELECT ST_Buffer(geom, 2, 'join=mitre mitre_limit=5.0') AS geom FROM subset
	),
extent AS (
	SELECT ST_ConvexHull(ST_Union(geom)) AS geom FROM subset
	),
unioned AS (
	SELECT ST_Union(geom) AS geom FROM Buffered
	),
differenced AS (
	SELECT ST_Difference(a.geom, b.geom) AS geom FROM
	extent a, unioned b
	)
SELECT 1 AS id, ST_Subdivide(geom, 500) AS geom FROM
differenced;

If we recall our before point cloud:

screen-shot-2016-10-15-at-3-15-44-pm

screen-shot-2016-10-15-at-4-38-53-pm

Now we have filtered out most buildings:

screen-shot-2016-10-15-at-4-36-15-pm

screen-shot-2016-10-15-at-4-36-31-pm

In order to do this filtering in PDAL, we do two things. First we create a json file that defines the filter:

{
  "pipeline":[
    "/data/2015-05-20_tandale_merged_densified_point_cloud_part_1.las",
    {
      "type":"filters.attribute",
      "dimension":"Classification",
      "datasource":"/data/salaam-buff-diff-split.shp",
      "layer":"salaam-buff-diff-split",
      "column":"id"
    },
    {
      "type":"filters.range",
      "limits":"Classification[1:1]"
    },
    "/data/2015-05-20_tandale_merged_densified_point_cloud_part_1_nobuild_buff.las"
  ]
}

Then we use these json definitions to apply the filter:

nohup sudo docker run -v /home/gisuser/docker/:/data pdal/pdal:1.3 pdal pipeline /data/shape-clip.json

Problems continue

This looks pretty good, but as we interrogate the data, we can see the artifacts of trees and some buildings still linger in the dataset.

screen-shot-2016-10-15-at-4-37-38-pm

screen-shot-2016-10-15-at-4-38-03-pm

 

Enter the Progressive Morphological Filter

It is possible to use the shape of the surface model to filter out buildings and trees. To do so, we start with the assumption that the buildings and vegetation shapes distinctive from the shape of the underlying ground. We have already used the hard work of the OpenStreetMap community to filter most of the buildings, but we still have some buildings and plenty of trees. PDAL has another great tutorial for applying this filter which we’ll leverage.

Again we need a JSON file to define our filter:

{
  "pipeline": {
    "name": "Progressive Morphological Filter with Outlier Removal",
    "version": 1.0,
    "filters": [{
        "name": "StatisticalOutlierRemoval",
        "setMeanK": 8,
        "setStddevMulThresh": 3.0
      }, {
        "name": "ProgressiveMorphologicalFilter",
        "setCellSize": 1.5
    }]
  }
}

And then use that filter to remove all the morphology and statistical outliers we don’t want:

nohup sudo docker run -v /home/gisuser/docker/:/data pdal/pdal:1.3 pdal pcl -i /data/2015-05-20_tandale_merged_densified_point_cloud_part_1_nobuild_buff.las -o /data/nobuild_filtered2.las -p /data/sor-pmf.json

This command will remove our colorization for the points, so we’ll see the colorization according to height only:

screen-shot-2016-10-15-at-4-53-55-pm

screen-shot-2016-10-15-at-4-54-49-pm

 

What remains is to interpolate this into digital terrain model. That is a project for another day.

Posted in 3D, Drone, Other, PDAL, Photogrammetry, UAS | Tagged: , , , | 1 Comment »

Viewing Sparse Point Clouds from OpenDroneMap — GeoKota

Posted by smathermather on June 30, 2016

This is a post about OpenDroneMap, an opensource project I am a maintainer for. ODM is a toolchain for post-processing drone imagery to create 3D and mapping products. It’s currently in beta and under pretty heavy development. If you’re interested in contributing to the project head over here. The Problem So for most of the […]

via Viewing Sparse Point Clouds from OpenDroneMap — GeoKota

Posted in 3D, Image Processing, OpenDroneMap, OpenDroneMap, Optics, Other, Photogrammetry | Tagged: , , , | Leave a Comment »

OpenDroneMap — Paris Code Sprint

Posted by smathermather on February 29, 2016

I failed to make it to the Paris Code Sprint. It just wasn’t in the cards. But, my colleague Dakota and I sprinted anyway, with some help and feedback from the OpenDroneMap community.

So, what did we do? Dakota did most of the work. He hacked away at the cmake branch of ODM, a branch set up by Edgar Riba to substantially improve the installation process for ODM.

  • Fixed odm_orthophoto in the branch so that it produces geotiffs
  • Fixed PMVS so that it is multithreaded again
  • Added rerun-all and rerun-from function
  • Integrated @lupas78’s additions for an xyz point cloud output
  • Added benchmarking which is an important soft number for when we have code changes
  • (Technically before the sprint) wrote the first test for OpenDroneMap
  • Cleaned code
What did I do? Mostly, I got caught up with the project. I haven’t been very hands on since the python port, let alone the cmake branch, so I became a little more pythonistic by just trying to successfully modify the code.
  • I also added PDAL to the build processs
  • And I inserted PDAL into the point cloud translation process.

Currently, this means we’ve dropped support for LAZ output, as I haven’t successfully built PDAL with LAZ support, but it stages the project for LAZ support through PDAL, and allows us to tap into additional PDAL functionality in the future.

It was an intensive couple of days that would have been improved with French wine, but we were in Parma (Ohio). So, a shout out to the coders in Paris at the same time, and cheers to all.

Posted in 3D, Drone, OpenDroneMap, OpenDroneMap, Photogrammetry, PMVS, UAS | Tagged: , | Leave a Comment »

MSF Canada Drone Day follow-up

Posted by smathermather on July 13, 2015

Dirk’s MSF Canada Drone Day is officially the first blog post I have “re-blogged”. Please read: https://smathermather.wordpress.com/2015/07/13/msf-canada-drone-day/

or better yet here: http://dirkgorissen.com/2015/07/14/msf-canada-drone-day/

I had the pleasure of co-presenting with Dirk and Ivan, and the rest is well covered in Dirk’s post. It came together as an excellent day and I think you would be hard pressed to have had a better introduction to drones.

The day was valuable to me as an emerging practitioner. I learned more about the state of the art in hardware, software, regulations, philosophy, and RC control from this day, and it was inspiring to inhabit the same space with such dedicated practitioners for a short time.

Beyond the value of the workshop to the participants, the outcomes were the following, this quoted from Dirk’s post:

As a first milestone we are looking to pull together a proposal to the Humanitarian Innovation Fund in collaboration with OpenDroneMap and supported by the Missing Maps consortium.

I love the extension of ODM into this space. This is the real value of open source, the opportunity to collaborate across the world, across industries and use cases, and across organizations. Expect to see improvements to ODM in usability, performance, and output qualities from this initiative. More on this later.

Another outcome / learning for me was observing Ivan’s OpenUAV. From his repo:

This is intended to be a repository for design files, instructions, photos, documentation, and everything else needed for people wishing to build a and operate UAV (drone) in a low-income, resource-poor environment. This is not about cutting-edge UAVs, it’s about democratizing the technology and getting it into the hands of more people, particularly in poorer countries and humanitarian settings.

Photo of OpenUAV example

Ivan undersells it. This is a pro quality quad copter on a very nice price diet — a brilliant piece of pragmatic engineering.

This little quad copter will make its way into drone building workshops I’ll be offering in Cleveland and Columbus Ohio and Seoul, South Korea in August and September. More details forthcoming.

If you are in Cleveland, plan to be at FOSS4G Seoul, or Ohio GIS, come build Ivan’s capable quad.

 

(BTW, Ivan says with a couple of 4C 8000mAh batteries, this sucker flies for 50 minutes… .)

Posted in 3D, Bundler, Camera Calibration, FOSS4G, Image Processing, OpenDroneMap, OpenDroneMap, Optics, Photogrammetry, PMVS | Tagged: , , , , , , , , , | Leave a Comment »

OpenAerialMap, OpenImageryNetwork, MapKnitter, OpenTerrain, and OpenDroneMap (cont. 1)

Posted by smathermather on June 7, 2015

Citing my previous post, let’s move on to more specifics on my thoughts regarding the integration of OpenAerialMap, OpenDroneMap, and MapKnitter as projects.

Image from kite over Seneca Golf Course

OpenDroneMap ❤ OpenAerialMap.

OpenAerialMap will become a platform by which drone users can share their imagery under an open license.

So, as the metadata spec for OpenAerialMap and OpenImageryNetwork matures, and as soon as a publicly available place for drone users to push their data comes online, ODM will write appropriate metadata and geotiffs to go into OIN to be indexed by OAM. Probably as an added bonus, ODM should be able to optionally auto-upload outputs from to the appropriate node on the OpenImageryNetwork.

Lincoln Peak Vinyard

OpenDroneMap ❤ MapKnitter.

MapKnitter / ODM integration is pretty straight forward in my mind too. There are ways that MapKnitter complements ODM, and vice versa. ODM does not have a graphical user interface at this time. MapKnitter promises to fill that role in a future OpenDroneMap implementation. MapKnitter has no image blending or auto-matching tools. OpenDroneMap will soon have both.

  • Ways MapKnitter may help OpenDroneMap:
    • MapKnitter’s clever use of Leaflet to handle affine transformation of images is really exciting, and may help with improving final georeferencing for ODM datasets.
    • Regarding the above, one really useful thing for fliers launching balloons, drones, and kites without GPS would be the ability to quickly and easily perform really approximate georeferencing. I would envision a workflow where a user moves an image to its approximate position and size relative to a background aerial. ODM would be able to take advantage of this approximate georeferencing to optimize matching.
  • Ways OpenDroneMap could benefit MapKnitter
    • For large image datasets, matching images can be very tedious. Automatic feature extraction and matching can help. OpenDroneMap could be adapted to serve back match information to Mapknitter to ease this process. This will become increasingly important as MapKnitter raises the ~60 image limit on images that it can process.
    • A near future version of ODM will have image blending / smoothing / radiometric matching. For the server portion of the MapKnitter infrastructure, this feature could be a really useful addition for production of final mosaics.

These projects (plus OpenTerrain…) are really exciting in their own right. Together, they represent amazing opportunities to foster, cultivate, process, and serve a large community of imagery providers, from individuals and small entities capturing specific datasets using kites, drones, and balloons, to satellite imagery providers hosting their own “image buckets” of open imagery data. Exciting times.

Image over Groth Memorial from kite

Posted in 3D, Bundler, Camera Calibration, FOSS4G, Image Processing, OpenDroneMap, Optics, Photogrammetry, PMVS | Tagged: , , , , , , , , , , | Leave a Comment »

OpenAerialMap, OpenImageryNetwork, MapKnitter, OpenTerrain, and OpenDroneMap

Posted by smathermather on May 29, 2015

This tweet:

is the beginning of some fruitful discussion, I suspect. There are some really awesome projects gaining momentum. I’ll give an overview of them as best I am able.

Kite aerial photography image over bridle riding ring.

Kite aerial photography image over bridle riding ring.

Let’s start with the one nearest and dearest to my heart (if you’ve been reading my blog, you can skip this part): OpenDroneMap. OpenDroneMap is an open source toolkit for processing drone, balloon, kite imagery into geographic data. It does this by using fully automated feature-matching between images, which create a 3D point cloud. From that, we can create a 3D surface (mesh), textured mesh, and orthophoto. This guy says it better:

But, it’s just a stand alone, Linux (Ubuntu)-based tool. It requires some geekiness to run, and it does not (at least not yet) act as a platform.  By that I mean, generically, you can’t just upload images to it and get the wonderful output from a service, and we don’t have a place to store and share all this wonderful data that comes from and will be coming from drones and other aerial platforms. This is where (from my selfish perspective) the other projects are so well timed… .

Screenshot of DevelopmentSeed's introductory post for OpenAerialMap

Let’s start with OpenAerialMap. From the Development Seed blog post on it (yes, you should follow the link. Don’t worry, I’ll wait until you return):

OpenAerialMap is a set of tools for searching, sharing, and using open satellite and drone imagery. This initial release includes the core infrastructure to catalog petabytes of open imagery. It also includes an extremely usable API and an elegant web interface to submit, search and download available imagery.

This is a reboot of a couple of previous attempts at solving this problem space, and it’s really exciting to watch passionate and brilliant work take place to make this happen. Also, this is not an easy problem space, and is being really thoughtfully simplified and implemented.

(As a side note, I’m not going to get into the distinction between OpenAerialMap and OpenImageryNetwork — not today anyway)

Preview of Open Terrain tumblr page

Preview of Open Terrain tumblr page

 

Open Terrain is a project for which a portion of its scope is to do for terrain models what OpenAerialMap and OpenImageryNetwork will do for open aerial datasets. The projects are informing each other and growing together, which is awesome collaboration to observe.

Finally, Mapknitter has recently been rebooted too, and it’s now a really elegant tool for taking a few aerial images and knitting them into a usable map (ok, it always was p. cool — now it’s even more elegant). What’s great about MapKnitter is it specifically addresses the problem of georeferencing balloon, kite, or drone images in a simple-to-use interface in the browser.

Snapshot of MapKnitter landing page

So, back to the question:

Bravo, yes. Lets. I have been thinking about, talking to people, discerning the strengths, overlaps, and complementary fittings of these projects as they have emerged. We are headed toward some really great things… . More specific thoughts to come.

Posted in 3D, Bundler, Camera Calibration, FOSS4G, Image Processing, OpenDroneMap, Optics, Photogrammetry, PMVS | Tagged: , , , , , , , , , , | 1 Comment »

Moar kite flight pics

Posted by smathermather on April 27, 2015

        

Posted in 3D, Bundler, Image Processing, OpenDroneMap, OpenDroneMap, Optics, Photogrammetry, PMVS, UAS | Tagged: , , , , , , , , , | Leave a Comment »

Kite flight (too windy for balloons, ahem “aerostats”)

Posted by smathermather on April 20, 2015

Inflation of aerostat

The aerostat hanger.

The end of the string.

The 9-footer is just so stable. But not enough wind to lift the cameras this day.

And so we send up the 16-foot workhorse. See that little dot? That’s the camera array.

The 16-footer flew nice and vertical, but pulled really hard. Processed images to follow soon.

Canon S100s from Kaptery — the silver one is an NIR adapted one; the black one is RGB color.


Edit: forgot the camera array:

CIR image from balloon:

IR image from the flight.

IR image from the flight.

Posted in 3D, Bundler, Image Processing, OpenDroneMap, OpenDroneMap, Photogrammetry, PMVS, UAS | Tagged: , , , , , , , , , | Leave a Comment »

OpenDroneMap Video for FOSS4GNA

Posted by smathermather on March 6, 2015

Thanks to some brilliant work by our Interpretive Technology Technician, we now have a little video for FOSS4GNA on OpenDroneMap. Come see my presentation next week in San Francisco, Weds. at 13:30 (1:30PM):

odm_screenshot

Posted in 3D, Bundler, Camera Calibration, FOSS4G, Image Processing, OpenDroneMap, Optics, Photogrammetry, PMVS | Tagged: , , , , , , , | 6 Comments »

OpenDroneMap — GCP file format

Posted by smathermather on March 1, 2015

Introduction:

So much documentation to catch up on for OpenDroneMap. Probably the most important missing element is documenting the ground control point file format. This will be a file name gcp_list.txt to be placed in the root of the image directory that you are processing.

Objective:

The ground control point file associates locations in the input raw images with geographic positions, allowing for the output of an orthophoto and georeferenced mesh. It can augment or replace the need for geographic coordinates in the exif headers of the input images.

Format

I mention this file briefly in a previous post, but let’s get into the weeds a bit on how to create one. The format is a simple text file. The header, or first line, is a descriptor of the coordinate system. This is not a formal thing, so don’t worry about format for this line.

The subsequent lines contain the x, y, and z of the ground control location in your local coordinate system, the pixel and line number of the location in the image, and the imagename itself. A minimum of 3 GCPs is necessary:

x y z pixelx pixely imagename

So, for example, if I mock up a version for the Langley dataset, I get something like this:

WGS84 UTM 10N
544256.7 5320919.9 5 3044 2622 IMG_0525.jpg
544157.7 5320899.2 5 4193 1552 IMG_0585.jpg
544033.4 5320876.0 5 1606 2763 IMG_0690.jpg

Note that my elevations are all the same. I was lazy, and assumed that there’s minimal topography and Langley, and that we are 5 meters above sea level. It is far better to put real values here, derived from an elevation dataset.

More on the how:

So, how to get these numbers? I loaded my raw unprocessed jpegs into QGIS. As I mouse over the image, it gives me coordinates, like “3044 -2622”. These are pixel values. I ignore the sign on the Y value.

For the X and Y values, I just used a Leaflet slippy map with a lat/lon pop-up: http://smathermather.github.io/leaflet-test/index.html. It would be better if I had GPS values from the field (preferably map grade) or survey values, but this quick and dirty adaptation of http://leafletjs.com/examples/quick-start.html will at least get my map in the correct part of the world. I plugged the lat/lon values I get into Montana State’s UTM converter, though this is a slow method if I am doing a lot of points.

Screenshot of coordinate collection process

Screenshot of coordinate collection process

Screenshot of coordinate collection process -- a total of 3 points have now been collected.

Screenshot of coordinate collection process — a total of 3 points have now been collected.

Finally…
Finally, I save my text file as gcp_list.txt, save it in the directory that contains my raw images, and run the run.pl script, and sit back and get some coffee. Output will be a subdirectory with a name similar to reconstruction-with-image-size-2400-results.

Edit:
And now I’ve added this to the wiki: https://github.com/OpenDroneMap/OpenDroneMap/wiki/Running-OpenDroneMap

Posted in 3D, Bundler, Camera Calibration, FOSS4G, Image Processing, OpenDroneMap, Optics, Photogrammetry, PMVS | Tagged: , , , , , , , | Leave a Comment »