Smathermather's Weblog

Remote Sensing, GIS, Ecology, and Oddball Techniques

Archive for March, 2016

OpenDroneMap — texturing improvements

Posted by smathermather on March 27, 2016

Great news on OpenDroneMap. We now have a branch that has MVS-Texturing integrated, thanks to continuing work by Spotscale, and of course continuing integration work by @dakotabenjamin.

The MVS-Texturing branch isn’t fully tested yet, nor fully integrated, but the initial results are promising. MVS-Texturing itself handles the problems of choosing the best photos for a given facet on a textured model in order to do a great job texturing a complex scene. This bears the promise of vastly improved textured models and very nice orthophotos.. It seems an ideal drop in for the texturing limitations of OpenDroneMap. From the project site:

Our method addresses most challenges occurring in such reconstructions: the large number of input images, their drastically varying properties such as image scale, (out-of-focus) blur, exposure variation, and occluders (e.g., moving plants or pedestrians). Using the proposed technique, we are able to texture datasets that are several orders of magnitude larger and far more challenging than shown in related work.

When we apply this approach to one of our more difficult datasets, which was taken on a partially cloud part of the day…


we get very promising results:


Posted in 3D, OpenDroneMap | 1 Comment »

Taking Slices from LiDAR data: Part VI

Posted by smathermather on March 19, 2016

I finally got PDAL properly compiled with Point Cloud Library (PCL) baked in. Word to the wise — CLANG is what the makers are using to compile. The PDAL crew were kind enough to revert the commit which broke GCC support, but why swim upstream? If you are compiling PDAL yourself, use CLANG. (Side note, the revert to support GCC was really helpful for ensuring we could embed PDAL into OpenDroneMap without any compiler changes for that project.)

With a compiled version of PDAL with the PCL dependencies built in, I can bypass using the docker instance. When I was spawning tens of threads of Docker and then killing them, recovery was a problem (it would often hose my docker install completely). I’m sure there’s some bug to report there, or perhaps spawning 40 docker threads is ill advised for some grander reason, but regardless, running PDAL outside a container has many benefits, including simpler code. If you recall our objectives with this script, we want to:

  • Calculate relative height of LiDAR data
  • Slice that data into bands of heights
  • Load the data into a PostgreSQL/PostGIS/pgPointCloud database.

The control script without docker becomes as follows:


# readlink gets us the full path to the file. This is necessary for docker
readlinker=`readlink -f $1`
# returns just the directory name
pathname=`dirname $readlinker`
# basename will strip off the directory name and the extension
name=`basename $1 .las`

# PDAL must be built with PCL.
# See

pdal translate "$name".las "$name".bpf height --writers.bpf.output_dims="X,Y,Z,Intensity,ReturnNumber,NumberOfReturns,ScanDirectionFlag,EdgeOfFlightLine,Classification,ScanAngleRank,UserData,PointSourceId,HeightAboveGround"

# Now we split the lidar data into slices of heights, from 0-1.5 ft, etc.
# on up to 200 feet. We're working in the Midwest, so we don't anticipate
# trees much taller than ~190 feet
for START in 0:1.5 1.5:3 3:6 6:15 15:30 30:45 45:60 60:105 105:150 150:200
        # We'll use the height classes to name our output files and tablename.
        # A little cleanup is necessary, so we're removing the colon ":".
        nameend=`echo $START | sed s/:/-/g`

        # Name our output

        # Implement the height range filter
        pdal translate $name.bpf $bpfname -f range --filters.range.limits="HeightAboveGround[$START)"

        # Now we put our data in the PostgreSQL database.
        pdal pipeline -i pipeline.xml --writers.pgpointcloud.table='pa_layer_'$nameend --readers.bpf.filename=$bpfname --writers.pgpointcloud.overwrite='false'

We still require our pipeline xml in order to set our default options as follows:

<?xml version="1.0" encoding="utf-8"?>
<Pipeline version="1.0">
  <Writer type="writers.pgpointcloud">
    <Option name="connection">
      host='localhost' dbname='user' user='user' password=‘password’
    <Option name="table">54001640PAN_heightasz_0-1.5</Option>
    <Option name="compression">dimensional</Option>
    <Filter type="filters.chipper">
      <Option name="capacity">400</Option>
      <Reader type="readers.bpf">
      <Option name="filename">54001640PAN_heightasz_0-1.5.bpf</Option>

And as before, we can use parallel to make this run a little lot faster:

find . -name '*.las' | parallel -j20 ./

For the record, I found out through testing that my underlying host only has 20 processors (though more cores). No point in running more processes than that… .

So, when point clouds get loaded, they’re broken up in to “chips” or collections of points. How many chips do we have so far?:

user=# SELECT COUNT(*) FROM "pa_layer_0-1.5";
(1 row)

Now, how many rows is too many in a PostgreSQL database? Answer:

In other words, your typical state full of LiDAR (Pennsylvania or Ohio for example) are not too large to store, retrieve, and analyze. If you’re in California or Texas, or have super dense stuff that’s been flown recently, you will have to provide some structure in the form of partitioning your data into separate tables based on e.g. geography. You could also modify your “chipper” size in the XML file. I have used the default 400 points per patch (for about 25,765,414,000 points total), which is fine for my use case as then I do not exceed 100 million rows once the points are chipped:

      <Option name="capacity">400</Option>

Posted in 3D, Database, Docker, LiDAR, Other, PDAL, pointcloud, PostGIS, PostgreSQL | Tagged: , , , , , | 3 Comments »

The ultimate kite?

Posted by smathermather on March 18, 2016

Loving the new kite we acquired at work. With a year or so of flying under our belts, we’ve learned to love and hate the wind. It seems it is often either too slow or too fast. If it’s too slow, that spells frustration and time lost. If it’s too fast, the same is true, but a fast wind can cause costly equipment damage as well.

It turns out, there are some really nice kites made in New Zealand for kite fishing that solve the problem of too much wind handily.

This kite will fish winds of 6 to 50 knots and can be tacked up to 60 degrees off the wind.

Anecdotally, the kite in question will fly in even faster winds safely, but just to be safe, it’s probably best to fly within specs.

Another frustrating aspect of kite flying with an objective of collecting data is putting the kite right where you want it, irrespective of wind direction. Fortunately, that’s a solved and well documented problem:

Flies like a dream. Happy flying!

Posted in Kite Aerial Photography, Other | Tagged: | Leave a Comment »