Smathermather's Weblog

Remote Sensing, GIS, Ecology, and Oddball Techniques

Archive for the ‘OpenDroneMap’ Category

Viewing Sparse Point Clouds from OpenDroneMap — GeoKota

Posted by smathermather on June 30, 2016

This is a post about OpenDroneMap, an opensource project I am a maintainer for. ODM is a toolchain for post-processing drone imagery to create 3D and mapping products. It’s currently in beta and under pretty heavy development. If you’re interested in contributing to the project head over here. The Problem So for most of the […]

via Viewing Sparse Point Clouds from OpenDroneMap — GeoKota

Posted in 3D, Image Processing, OpenDroneMap, OpenDroneMap, Optics, Other, Photogrammetry | Tagged: , , , | Leave a Comment »

OpenDroneMap — texturing improvements

Posted by smathermather on March 27, 2016

Great news on OpenDroneMap. We now have a branch that has MVS-Texturing integrated, thanks to continuing work by Spotscale, and of course continuing integration work by @dakotabenjamin.

The MVS-Texturing branch isn’t fully tested yet, nor fully integrated, but the initial results are promising. MVS-Texturing itself handles the problems of choosing the best photos for a given facet on a textured model in order to do a great job texturing a complex scene. This bears the promise of vastly improved textured models and very nice orthophotos.. It seems an ideal drop in for the texturing limitations of OpenDroneMap. From the project site:

Our method addresses most challenges occurring in such reconstructions: the large number of input images, their drastically varying properties such as image scale, (out-of-focus) blur, exposure variation, and occluders (e.g., moving plants or pedestrians). Using the proposed technique, we are able to texture datasets that are several orders of magnitude larger and far more challenging than shown in related work.

When we apply this approach to one of our more difficult datasets, which was taken on a partially cloud part of the day…

IMG_1347_RGB.jpg

we get very promising results:

 

Posted in 3D, OpenDroneMap | 1 Comment »

OpenDroneMap — Paris Code Sprint

Posted by smathermather on February 29, 2016

I failed to make it to the Paris Code Sprint. It just wasn’t in the cards. But, my colleague Dakota and I sprinted anyway, with some help and feedback from the OpenDroneMap community.

So, what did we do? Dakota did most of the work. He hacked away at the cmake branch of ODM, a branch set up by Edgar Riba to substantially improve the installation process for ODM.

  • Fixed odm_orthophoto in the branch so that it produces geotiffs
  • Fixed PMVS so that it is multithreaded again
  • Added rerun-all and rerun-from function
  • Integrated @lupas78’s additions for an xyz point cloud output
  • Added benchmarking which is an important soft number for when we have code changes
  • (Technically before the sprint) wrote the first test for OpenDroneMap
  • Cleaned code
What did I do? Mostly, I got caught up with the project. I haven’t been very hands on since the python port, let alone the cmake branch, so I became a little more pythonistic by just trying to successfully modify the code.
  • I also added PDAL to the build processs
  • And I inserted PDAL into the point cloud translation process.

Currently, this means we’ve dropped support for LAZ output, as I haven’t successfully built PDAL with LAZ support, but it stages the project for LAZ support through PDAL, and allows us to tap into additional PDAL functionality in the future.

It was an intensive couple of days that would have been improved with French wine, but we were in Parma (Ohio). So, a shout out to the coders in Paris at the same time, and cheers to all.

Posted in 3D, Drone, OpenDroneMap, OpenDroneMap, Photogrammetry, PMVS, UAS | Tagged: , | Leave a Comment »

OpenDroneMap — the future that awaits (part 이)

Posted by smathermather on October 25, 2015

In my previous post, ODM — the future that awaits, I start to chart out OpenDroneMap beyond the toolchain. Here’s a bit more, in outline form. More narrative and breakdown to come. (this is the gist)

Objectives:

Take OpenDroneMap from simple toolchain to an online processing tool + open aerial dataset. This would be distinct from and complementary to OpenAerialMap:

  1. Explicitly engage and provide a platform for drone enthusiasts to contribute imagery in return for processing services.
  2. Address and serve:
    • Aerial imagery
    • Point clouds
    • Surface models
    • Terrain models
  3. Additionally, as part of a virtuous circle, digitizing to OSM from this aerial imagery would refine the surface models and thus the final aerial imagery
    • More on this later: in short digitizing OSM on this dataset would result in 3D photogrammetric breaklines which would in turn refine the quality of surface and terrain models and aerial imagery.

Outputs / Data products:

  • Aerial basemap
    • (ultimately with filter for time / season?)
  • Point cloud (see e.g. http://plas.io)
  • Digital surface model (similar to Open Terrain)
  • Digital elevation model (in conjunction with Open Terrain)

Likely Software / related projects

Back of the envelope calculations — Mapping a city with drones

If ODM is to take submissions of large portions of data, data collection campaigns may come into play. Here are some back-of-the-envelope calculations for flying whole cities, albeit the medium size cities of San Francisco and Cleveland. This ignores time needed for all sort so things, including coordinating with local air traffic control. As such, this is a best case scenario set of estimates.

Drone Flight Height Pixel Overlap Per flight City Name City Area Total # of Flights Total Flight time
E384 400 ft 3 cm 60% 1.5 sq mile San Francisco 50 sq miles 33 flights 66 hours
E384 400 ft 5 cm 90 % 0.5 sq mile San Francisco 50 sq miles 100 flights 200 hours
E384 400 ft 3 cm 60 % 0.5 sq mile Cleveland 80 sq miles 54 flights 108 hours
E384 400 ft 5 cm 90 % 0.5 sq mile Cleveland 80 sq miles 160 flights 320 hours
Iris+ 400 ft 3 cm 60% 0.078 sq mile San Francisco 50 sq miles 640 flights 213 hours
Iris+ 400 ft 5 cm 90% 0.026 sq mile San Francisco 50 sq miles 1920 flights 640 hours

Posted in 3D, OpenDroneMap, OpenDroneMap | Tagged: , | Leave a Comment »

OpenDroneMap — Improvements Needed

Posted by smathermather on October 18, 2015

Talking about the future sometimes requires critiquing the present. The wonderful thing about an open source project is we can be quite open about limitations, and discuss ways forward. OpenDroneMap is a really interesting and captivating project… and there’s more work to do.

To understand what work needs done, we need to understand OpenDroneMap / structure from motion in general. Some of the limitations endemic to ODM are specific to its maturity as a project. Some of the limitations to ODM are extant in commercial closed-source industry leaders. I’ll highlight each as I do the walk through.

A simplified version of Structure from Motion (SfM) workflows as they apply to drone image processing are as follows:

Find features & Match features –> Find scene structure / camera positions –> Create dense point cloud –> Create mesh –> Texture mesh –> Generate orthophoto and other products

This misses some steps, but gives the major themes. Let’s visualize these as drawings and screen shots. (In the interest of full disclosure, the screen shots are from a closed source solution so that I can demonstrate the problems endemic across all software I have tested to date.)

Diagrams / screenshots of the toolchain parts:

Find features & Match features --> Find scene structure

Find features & Match features –> Find scene structure


 

Create Dense Point Cloud

Create Dense Point Cloud


 

Create mesh

Create mesh


 

Texture mesh

Texture mesh


 

And then generate orthophoto and secondary products (no diagram)

Problem space:

Of these, let’s highlight in bold known deficiencies in ODM:

Find features & Match features –> Find scene structure / camera positions –> Create dense point cloud –> Create mesh –> Texture mesh –> Generate orthophoto and other products

(These highlights assume that our new texturing engine that’s being written will address deficiencies there. Time and testing will tell… . This also assumes that the inclusion of OpenSfM in the toolchain fixes the scene structure /camera issues. This assumption also requires more testing.)

Each portion of the pipeline is dependent upon the next, if for example the camera positions are poor, point cloud won’t be great, and the texturing will be very problematic. If the dense point cloud isn’t as dense as possible, features will be lost, and the mesh, textured mesh, orthophoto, and other products will be degraded as well. For example, see these two different densities of point clouds:

Create Dense Point Cloud

More sparse point cloud


 

Less sparse (dense) point cloud

Less sparse point cloud


It becomes clear that the density and veracity of that point cloud lays the groundwork for the remainder of the pipeline.

ODM Priority 1: Improve density / veracity of point cloud

So what about the mesh issues? The meshing process for ODM and its closed source siblings (with possible exceptions) is problematic. Take for example this mesh of a few buildings:

Textured mesh of building

Textured mesh of building

The problems with this mesh become quite apparent when we view the un-textured counterpart:

Un-textured mesh of buildings

Un-textured mesh of buildings

We can see many issues with this mesh. This is a problem with all drone image processing tools I have tested to date — geometric surfaces are not treated as planar, meshing processes treat vegetation, ground, built environment equally, and thus don’t model any of them well.

ODM Priority 2: Improve meshing process

Priority 2 is difficult space, probably requires automated or semi-automated classification of the point cloud &/or input imagery, and while simple in the case of buildings, may be quite complicated in the case of vegetation. Old-school photogrammetry would have hand digitized hard and soft breaklines for built environments. How we handle this for ODM is an area we have yet to explore.

Conclusions

I am optimistic that ODM’s Find features & Match features –> Find scene structure / camera positions step is much improved with the integration of OpenSfM (please comment if you’ve found otherwise and have test cases to demonstrate). I am hopeful that the upcoming Texture mesh –> Generate orthophoto improvements will be a good solution. Where we need to improve will be in the near future is in the Create dense point cloud step. Where every software I have tested needs improvement, closed source and open source, is in the Create mesh step.

Posted in 3D, OpenDroneMap | Tagged: , | 1 Comment »

Humanitarian UAV Experts Meeting — first blush.

Posted by smathermather on October 13, 2015


UAViators, MIT Lincoln Labs, UNOCHA, and others organized and hosted the UAViators Experts Meeting on MIT’s campus this weekend. It was a remarkable event, if only for the thoughtfulness and knowledge base of the people in the room. The meeting brought together UAV operators, manufacturers, humanitarians, and a few folks at the intersection of these.

For me, it was relevatory with respect to all the non-mapping specific drone applications, from the advancement of technologies for last mile logistics to basic tactical / observational applications.

What was also interesting was some of the insights into regulatory issues and questions that are on the horizon.

I gave a short presentation on OpenDroneMap and therefore much of my time was spent thinking and listening in order to understand the potential application of OpenDroneMap in the humanitarian and development space. This extends a bit my understanding of its role and potential role in environmental applications.

More soon!

Posted in 3D, OpenDroneMap | Leave a Comment »

MSF Canada Drone Day follow-up

Posted by smathermather on July 13, 2015

Dirk’s MSF Canada Drone Day is officially the first blog post I have “re-blogged”. Please read: https://smathermather.wordpress.com/2015/07/13/msf-canada-drone-day/

or better yet here: http://dirkgorissen.com/2015/07/14/msf-canada-drone-day/

I had the pleasure of co-presenting with Dirk and Ivan, and the rest is well covered in Dirk’s post. It came together as an excellent day and I think you would be hard pressed to have had a better introduction to drones.

The day was valuable to me as an emerging practitioner. I learned more about the state of the art in hardware, software, regulations, philosophy, and RC control from this day, and it was inspiring to inhabit the same space with such dedicated practitioners for a short time.

Beyond the value of the workshop to the participants, the outcomes were the following, this quoted from Dirk’s post:

As a first milestone we are looking to pull together a proposal to the Humanitarian Innovation Fund in collaboration with OpenDroneMap and supported by the Missing Maps consortium.

I love the extension of ODM into this space. This is the real value of open source, the opportunity to collaborate across the world, across industries and use cases, and across organizations. Expect to see improvements to ODM in usability, performance, and output qualities from this initiative. More on this later.

Another outcome / learning for me was observing Ivan’s OpenUAV. From his repo:

This is intended to be a repository for design files, instructions, photos, documentation, and everything else needed for people wishing to build a and operate UAV (drone) in a low-income, resource-poor environment. This is not about cutting-edge UAVs, it’s about democratizing the technology and getting it into the hands of more people, particularly in poorer countries and humanitarian settings.

Photo of OpenUAV example

Ivan undersells it. This is a pro quality quad copter on a very nice price diet — a brilliant piece of pragmatic engineering.

This little quad copter will make its way into drone building workshops I’ll be offering in Cleveland and Columbus Ohio and Seoul, South Korea in August and September. More details forthcoming.

If you are in Cleveland, plan to be at FOSS4G Seoul, or Ohio GIS, come build Ivan’s capable quad.

 

(BTW, Ivan says with a couple of 4C 8000mAh batteries, this sucker flies for 50 minutes… .)

Posted in 3D, Bundler, Camera Calibration, FOSS4G, Image Processing, OpenDroneMap, OpenDroneMap, Optics, Photogrammetry, PMVS | Tagged: , , , , , , , , , | Leave a Comment »

Moar kite flight pics

Posted by smathermather on April 27, 2015

        

Posted in 3D, Bundler, Image Processing, OpenDroneMap, OpenDroneMap, Optics, Photogrammetry, PMVS, UAS | Tagged: , , , , , , , , , | Leave a Comment »

Kite flight (too windy for balloons, ahem “aerostats”)

Posted by smathermather on April 20, 2015

Inflation of aerostat

The aerostat hanger.

The end of the string.

The 9-footer is just so stable. But not enough wind to lift the cameras this day.

And so we send up the 16-foot workhorse. See that little dot? That’s the camera array.

The 16-footer flew nice and vertical, but pulled really hard. Processed images to follow soon.

Canon S100s from Kaptery — the silver one is an NIR adapted one; the black one is RGB color.


Edit: forgot the camera array:

CIR image from balloon:

IR image from the flight.

IR image from the flight.

Posted in 3D, Bundler, Image Processing, OpenDroneMap, OpenDroneMap, Photogrammetry, PMVS, UAS | Tagged: , , , , , , , , , | Leave a Comment »

ICCM — International Conference of Crisis Mappers

Posted by smathermather on November 9, 2014

Screen shot of Crisis Mappers landing page

Screen shot of Crisis Mappers landing page

The mark of a great conference is one that not only is well run and orchestrated, interesting from a content perspective, and full of bright minds, but also one that experiments with elements of interactions to maximize the value delivered to attendees. By these measures, the International Conference of Crisis Mappers (ICCM) succeeds. Normally, I struggle with the call to attend sessions vs. the conference hallway conversations which can be of such great value. ICCM provided enough context and structure for enjoying both, plus a number of other participatory structures in which to interact with and learn from other ICCM attendees.

For attending ICCM, I had plans to only hallway chat about http://OpenDroneMap.org, in order to get a measure of the culture and needs, and also of the interest for such a project from digital humanitarians in an effort to get enough interest to begin to garnish users and contributors. Thanks to some last minute slots opening up, and on the kind recommendation of Jen Ziemke, I was able to show OpenDroneMap at the Tech & Analysis Fair at Google at the start of the conference and also do a well attended deep dive session on Saturday. In each, I got great questions and recommendations, did a brainstorming session on use in the deep dive session, and had a thoroughly great time.

More later, but I will end with this — if you like technology, have empathy and a desire to apply empathetic design, development, and sweat equity to humanitarian needs, then get signed up at http://crisismappers.net, become a participant in HOT, join the Standby Task Force, and / or all the other work done by digital humanitarians, and make your way to Boston next year for the next ICCM.

Bravo, all.

Posted in 3D, International Conference of Crisis Mappers, OpenDroneMap | Tagged: , , | Leave a Comment »