Smathermather's Weblog

Remote Sensing, GIS, Ecology, and Oddball Techniques

Archive for February, 2017

Taking Slices from ~~LiDAR~~ OpenDroneMap data: Part X

Posted by smathermather on February 23, 2017

Part 10 of N… , wait. This is a lie. This post is actually about optical drone data, not LiDAR data. This is about next phase features fro OpenDroneMap — automated and semiautomation of the point clouds, creation of DTMs and other fun such stuff.

To date, we’ve only extracted Digital Surface Models from ODM — the top surface of everything in the scene. As it is useful for hydrological modeling and other purposes to have a Digital Terrain Model estimated, we’ll be including PDAL’s Progressive Morphological Filter for the sake of DEM extraction. Here’s a small preview:

Posted in 3D, Docker, OpenDroneMap, PDAL | Tagged: , , | Leave a Comment »

Taking Slices from LiDAR data: Part IX

Posted by smathermather on February 20, 2017

Part 9 of N… , see e.g. my previous post on the topic.

We’ve been working to reduce the effect of overlapping samples on statistics we run on LiDAR data, and to do so, we’ve been using PDAL’s filters.sample approach. One catch: this handles the horizontal sampling problem well, but we might want to intentionally retain samples from high locations — after all, I want to see the trees for the forest and vice versa. So, it might behoove us to sample within each of our desired height classes to retain as much vertical information as possible.

Posted in 3D, Database, Docker, LiDAR, Other, PDAL, pointcloud, PostGIS, PostgreSQL | Tagged: , , , , , | Leave a Comment »

Taking Slices from LiDAR data: Part VIII

Posted by smathermather on February 18, 2017

Part 8 of N… , see e.g. my previous post on the topic.

I didn’t think my explanation of sampling problems with LiDAR data in my previous post was adequate. Here are a couple more figures for clarification.

We can take this dataset over trees, water, fences, and buildings that is heavily sampled in some areas and sparsely sampled in others and use PDAL’s filters.sample (Poisson dart-throwing) to create an evenly sampled version of the dataset.

Figure showing overlap of LiDAR scanlines

Figure showing overlap of LiDAR scanlines

Figure showing data resampled for eveness

Figure showing data resampled for evenness

An extra special thanks to the PDAL team for not only building such cool software, but being so responsive to questions!

Posted in 3D, Database, Docker, LiDAR, Other, PDAL, pointcloud, PostGIS, PostgreSQL | Tagged: , , , , , | Leave a Comment »

Quantitative analysis of gorilla and monkey movement and R-Stat (part 3)

Posted by smathermather on February 16, 2017

This blog post is from a series of posts on gorilla and biodiversity research in Rwanda. I have introduced the people, the place, and a little on the beasties there. Now we’ll talk some R-code for doing home range estimation.

Home range estimation is a pretty deep and also abstract concept. Heuristically, it is the process looking at where an animal or group of animals move in the world. If one were to create a home range map for me, it’d be a pretty simple bimodal map of home and work.

Where it gets funky, is what does one do with the unusual places that an animal travels. So, for my home range, I am mostly at work, home, church on Sundays, yoga, and the grocery store. But I did spend 2 weeks in Rwanda, one week in Tanzania, one week in Belgium and the Netherlands, one week in Seattle, one in Raleigh, etc. etc.. Should these places be part of my home range?

All this to say usually home ranges are calculated with some means to not include the less common places. I would be happy if East Africa were part of my home range, but I think it’s arguable that it is not yet so.

Also, depending on the approach we use, travel between places may or may not be considered part of the home range. Back to my home range: ideally even if we concluded that 2 weeks living in Rwanda expanded my home range to include Musanze, the flight there and back probably shouldn’t be included in my home range. For our work today, we are not going to be excluding travel from our home range calculations, but understand that it can be relevant to some home range calculations.

For our home range calculation today, the following assumptions will be made:

  • We won’t be explicitly excluding travel from our home range calculations.
  • We’ll use simple techniques to exclude ephemeral portions of the home range

For basic home range analysis, we’ll use R’s adehabitat home range (adehabitatHR) package.

# Load the adehabitatHR library
# Load appropriate libraries for loading and manipulating data
library(sp) # Spatial data objects in R
library(rgdal) # Geospatial Data Abstraction Library
library(adehabitatHR) # Adehabitat HomeRange
library(readr) # File read capacity
library(rgeos) # Geometric calculation to be used later
library(maptools) # more spatial stuff

Now that we have every library we need loaded (and maybe then some) let’s load the data.

# We need to add a data filter here... .
# For now, we assign just the columns we need for HR calculation
loc_int_totf <- loc_int_tot[,c('X','Y', 'id')]

We’ll want to explicitly turn these data into geospatial data.

# Use sp library to assign coordinates and projection
coordinates(loc_int_totf) <- c("X", "Y")
# Our projection is UTM Zone 35S
# proj4string acquired at
proj4string(loc_int_totf) <- CRS("+proj=utm +zone=35 +south +ellps=WGS84 +datum=WGS84 +units=m +no_defs") 

Now we are ready to do some home range calculations. We’ll use the kernelUD function to convert our data into a surface representing our home range estimate. The total of all the pixels in this surface (as represented by a raster) will total to 1, or 100 of the home range.

# Estimating the utilization distribution using "reference" bandwidth
kud <- kernelUD(loc_int_totf)

# Display the utilization distribution

Kernel density map

Recall that the total of all the pixel values here is 1. This means that this image represents 100 percent of the calculated range of the Golden Monkeys. If we want to calculate the 70% homerange (what we estimate the golden monkeys spend 70% of their time in) we would do so as follows:

# Estimate the homerange from the utilization distribution
homerange <- getverticeshr(kud, 70)

Outline of 70% home range

Now it would be useful to convert this to a data frame so that we can further manipulate and understand the data. For example, what is the home range size for any given percentile?

# Calculate home range sizes

# Calculate home range sizes for every 5% from 50-95%
ii <- kernel.area(kud, percent=seq(1, 99, by=1))

Plot comparing home range % vs hectares

Finally, it would be nice to be able to get these data out of R and display alongside other GIS data. We’ll use writeOGR as part of RGDAL to do so.

# Write out data
writeOGR(homerange, getwd(),
         "homerange", driver="ESRI Shapefile")

That’s it for today. This bit of R will serve as the core code for a range of different analyses. Stay tuned!

Posted in Gorillas, Karisoke, R | Tagged: , , , , | Leave a Comment »

Taking Slices from LiDAR data: Part VII

Posted by smathermather on February 15, 2017

Part 7 of N… , see e.g. my previous post on the topic.

More work on taking LiDAR slices. This time, the blog post is all about data preparation. LiDAR data, in its raw form, often has scan line effects when we look at density of points.


This can affect statistics we are running, as our sampling effort is not even. To ameliorate this affect a bit, we can decimate our point cloud before doing further work with it. In PDAL, we have three choices for decimation: filters.decimation, which samples every Nth point from the point cloud; filters.voxelgrid, which does volumetric pixel based resampling; and filters.sample or “Poisson sampling via ‘Dart Throwing'”.

filters.decimation won’t help us with the above problem. Voxelgrid sampling could help, but it’s very regular, so I reject this on beauty grounds alone. This leaves filters.sample.

The nice thing about both the voxelgrid and the poisson sampling is that they retain much of the shape of the point cloud while down sampling the data:



We will execute the poisson sampling in PDAL. As many things in PDAL are best done with a (json) pipeline file, we construct a pipeline file describing the filtering we want to do, and then call that from the command line:

We can slice our data up similar to previous posts, and then look at the point density per slice. R-code for doing this forthcoming (thanks to Chris Tracey at Western Pennsylvania Conservancy and the LidR project), but below is a graphic as a teaser. For the record, we will probably pursue a fully PDAL solution in the end, but really interesting results in the interim:


More to come. Stay tuned.

Posted in 3D, Database, Docker, LiDAR, Other, PDAL, pointcloud, PostGIS, PostgreSQL | Tagged: , , , , , | 1 Comment »