Smathermather's Weblog

Remote Sensing, GIS, Ecology, and Oddball Techniques

Posts Tagged ‘Windows Command Prompt’

GDAL tools of the trade: GDAL, MrSid, and nearblack revisited

Posted by smathermather on February 11, 2016

internet_archiveBack in 2011 I wrote a post about GDAL, translating MrSID files and trimming off the artifacts so mosaics work. I’m revisiting some similar work this week and realized I had lost my copy of FWTools that can deal with MrSIDs. The Internet Archive came to the rescue.

At some point LizardTech licensed a DLL that allowed for compilation compatible with FWTools licensing so we could do things like use gdal_translate to covert a MrSID file to a tiff. Later on, the licensing changed, but binaries created prior to the change hold that original license. The WayBack Machine archived one such copy here: https://web.archive.org/web/20130127013649/http://home.gdal.org/fwtools/FWTools202.exe .

Download, install, and now for some Batch scripting fun… .

W:\Aircraft\naip_ortho>for %I in (*.sid) do gdal_translate %I %~nI.tif

W:\Aircraft\naip_ortho>gdal_translate ortho_1-1_1n_s_oh035_2015_1.sid ortho_1-1_
1n_s_oh035_2015_1.tif
Input file size is 58222, 49140
0…10…20.

Posted in GDAL, Image Processing | Tagged: , , , | 2 Comments »

gdal_calc.py– Raster Calculator using Numby Functions

Posted by smathermather on December 1, 2011

gdal_calc is a python script that makes it easy to do band math and logical operations with gdal/numby. This ranges from basic arithemtic operations to logical functions. And while gdal_calc.py has been around since 2008, it is but is a recent and revelatory discovery for me. I had just today resigned myself to properly learning python so as to use the gdal bindings. But, my laziness may continue unabated. Not learning Python. Now that is lazy.

Anyway, want to do band math? It’s as simple as follows (straight from the python script itself):

# example 1 - add two files together
gdal_calc.py -A input1.tif -B input2.tif --outfile=result.tif --calc="A+B"

# example 2 - average of two layers
gdal_calc.py -A input.tif -B input2.tif --outfile=result.tif --calc="(A+B)/2"

# example 3 - set values of zero and below to null
gdal_calc.py -A input.tif --outfile=result.tif --calc="A*(A>0)" --NoDataValue=0

So, I had little modification to average my set of images from PovRay:

gdal_calc.py -A input.tif -B input2.tif -C input3.tif -D input4.tif -E input5.tif --outfile=result.tif --calc="(A+B+C+D+E)/5"

I might write some bash wrappers to create built in functions for basic things I’ll do repeatedly, like average a set of images based on a directory listing. Of course, a little voice inside says “You should just do that in Python.” We’ll see.

Posted in Analysis, GDAL, Image Processing | Tagged: , , , , | 2 Comments »

GDAL, MrSid, and nearblack

Posted by smathermather on July 29, 2011

Translating MrSid lossy compressed files into uncompressed imagery has its drawbacks, including licensing and artifacts.
Old versions of fwtools, which includes the GDAL utilities (and more), were compiled with a license that allowed for the translation of MrSid into e.g. Erdas Imagine images or GeoTiff. The licensing changed on that library, so FWTools and MS4W don’t do this anymore.

If you have a compiled version of the GDAL utilities with a legal license for converting, you still run into the issue of artifacts, e.g.

Nearly, but not quite black pixels at the edge of MrSid lossy compressed image

GDAL’s nearblack is a great utility for correcting this problem. If we wanted to batch this up on a windows system, it might look something like this:


FOR %f IN (*.sid) DO nearblack -o %~nf.img %f

Posted in GDAL, Image Processing | Tagged: , , , | 5 Comments »

Historical USGS Quads

Posted by smathermather on March 4, 2011

This will be a very quick post today.  We wanted to use some historical USGS quads for analysis.  We had a ready source, used ArcGIS to georeference them to a grid, but then wanted something approaching a seamless product.

Enter gdal.

First step, create an image to put the mosaic into:


gdal_merge -o usgs_merge.tif -createonly input1.tif input2.tif ...

Then mosaic into that.  Since we have a shapefile with the shape of the bounds of each of the images in question, we can use a cutline to perform the mosaic, e.g.

gdalwarp -cutline quadrangle_index_24k.shp -cwhere "quadname_ = West_View" "West_View.tif" usgs_merge.tif

In all honesty, this was scripted in Excel for expediency, but could easily be done in Windows Command with for-in-do, or BASH at a UNIX machine.

Final result:

Posted in Other | Tagged: , , , | 1 Comment »

Batch processing resampling of DEMs

Posted by smathermather on January 4, 2010

Yet again, working with that Ohio DEM dataset, I need some reduced resolution versions of the 2.5 foot DEMs.  At a grand total of ~600 DEMs for a given county, it was time to batch, this time in Windows Command prompt.  The neat thing is the variable substitution in the for-in-do command in the command prompt, where, e.g. for a file variable %f, the file name, minus extension can be invoked with %~nf.  As I write this,  I realize that I didn’t strictly need that variable for this dataset, as they were ESRI Grids, so they have no extension to trim off… .  Hehe.  I think I was being stupidly clever.  But, well, trim off the /D flag, and you can use this with any file extension… .  So, ya.  This is the more generic solution… .

I use bilinear resampling below for each tier from half the resolution.  Why bilinear when there are fancy resampling methods available?  I trust bilinear to do exactly what I want, and that is create an averaged tier that can be overlayed with the original data 100% (knock on wood… .).  I can quantify it’s behavior at each tier.  In other words, I’m fussy, and I understand bilinear, and haven’t invested the time to find out if I like anything else… .

for /D %f in (n*) do gdalwarp -tr 5 5 -r bilinear %f resampled\%~nf_05.tif
for /D %f in (n*) do gdalwarp -tr 10 10 -r bilinear resampled\%~nf_05.tif resampled\%~nf_10.tif
for /D %f in (n*) do gdalwarp -tr 20 20 -r bilinear resampled\%~nf_10.tif resampled\%~nf_20.tif
for /D %f in (n*) do gdalwarp -tr 40 40 -r bilinear resampled\%~nf_20.tif resampled\%~nf_40.tif
for /D %f in (n*) do gdalwarp -tr 80 80 -r bilinear resampled\%~nf_40.tif resampled\%~nf_80.tif
for /D %f in (n*) do gdalwarp -tr 160 160 -r bilinear resampled\%~nf_80.tif resampled\%~nf_160.tif

Posted in GDAL | Tagged: , , , , | Leave a Comment »