Extended Depth-Of-Field Using Panorama Tools
Focus stacking software continues to improve and proliferate.
At present, my personal favorite is a new program, Zerene Stacker 1.0.
It offers uniquely good automatic handling of tough
"extreme macro" problems, such as bristles and low contrast subjects
shot in deep
stacks. At the same time,
it's easy to use, its retouching capabilities are first-rate, and it
works just fine on simpler problems too.
That's why my links page
describes it as "better images, less work".
Check it out!
November 25, 2009
As of this writing [now over 4 years
ago], recent updates to two easily used software packages
make most of this page "overcome by events". That is a Good Thing!
Be sure to check them out: Helicon
Focus 3.00 and CombineZ5.
See also John
Hollenberg's review of these packages.
The discussion below includes some work using earlier versions of these
packages. The new versions are much improved. For most
subjects, both packages can now produce better results with a lot less
work than using Panorama Tools. Most readers should think of this
page as providing historical perspective and possibly some interesting
test cases and discussion.
March 13, 2005
This extended-depth-of-field image was assembled from 55 separate
source images, using modified open-source Panorama Tools
techniques as described below. (Click on the image for a higher
resolution version in a new window.)
This apparently fearsome beast is just a common Ten-Lined June
Beetle. You can search
Google for pictures and stories about the insect.
An extended-depth-of-field capability has been incorporated into recent
versions of the open-source Panorama Tools library "pano12.dll".
This capability works much like commercially available extended
depth-of-field software. It automatically determines, for every point
in the picture, which of numerous overlapping images is focused the
best. Then it generates masks that select only the best focused
image areas to be visible in the final picture.
The images produced by this pano12.dll are similar to those produced by
other extended-depth-of-field software, but can be easier to edit
visibility masks can be preserved in the output.
Panorama Tools and related software are extremely good at adjusting
individual input images' scale and position so that the finest details
are properly aligned across the whole sequence of input
images. This alignment contributes a great deal to the high
quality assembly illustrated above.
The extended-depth-of-field capability described here is now standard
with pano12.dll versions 220.127.116.11 and above. Useful downloads are:
Members of the Yahoo PanoTools group should check that group's
Files area at http://groups.yahoo.com/group/PanoTools/files/PanoTools/
for updated binaries.
Note: the extended depth-of-field capability contains known problems as
Please read those before using.
To invoke extended depth-of-field, just add a "z" control line
the PTStitcher script, using your favorite GUI's "show/edit script"
button. For example:
# script file for ptStitcher created by
p w825 h558 f0 v50 u2 n"PSD_mask"
<more lines not shown>
z m2 f4 s4
Parameters on the "z" line are:
Extended depth-of-field is computed whenever feathering is selected and
line is present.
mN mask type
m0 hard-edged masks,
m1 hard-edged masks, stack
of nested masks
m2 blended masks, stack of
(m2 is default & strongly
this option includes a
smoothing computation that seems to help a lot.)
fN focus estimation window size, N = halfwidth of window.
Recommended value is
0.5% of image width, e.g. 4 pixels for an 800-pixel image.
Computation cost for focus estimation increases
proportional to N^2. Default
sN smoothing window size, N = halfwidth of window.
Recommended value is 0.5% of
image width, e.g. 4 pixels for an 800-pixel image.
Computation cost for smoothing increases
proportional to N^2. Default
The computed masks are present in "Photoshop with feather" (PSD_mask)
"Multi-image feathered TIFF" (TIFF_mask) outputs. The effects of
the masks are shown in all single-image output, such as TIFF and JPEG.
In PTGui, the
results are visible in a
Preview image, but not in the Panorama Editor window.
How it Works
This code uses the classical variance method of estimating focus.
An array of "best source" image numbers is computed, recording at each
pixel position the PTStitcher image number that has the largest
variance of pixel values
within a focus-estimation window. Then masks are computed as
follows. For mask type m0, at each pixel, the mask value is 255
best image and 0 for all others. For mask type m1, at each pixel,
value is 255 for the best image and all lower-numbered images, and 0
for all higher-numbered images. For mask type m2, the array of
"best source" image numbers is smoothed by a simple averaging
filter. Then at each pixel position, the mask value is computed
as 255 if the image number is less than or equal to the average, 0 if
the image number is greater than the average+1, and linearly
interpolated between 255 and 0 as the image number is between average
and average+1. I make no claim that this is a
particularly good way to do the job, but it was quick to get working
and generates perhaps surprisingly good results.
Expected Results -- Example #1
To illustrate the extended depth-of-field feature, I will use the
discussed in Max Lyons' forum at http://www.tawbaware.com/forum2/viewtopic.php?t=759
, aligned and registered as discussed there by JohnH. Very
briefly, these are a set of 8 images that sweep focus from top to
bottom of a small beaded deer art object. The top and bottom
focus images are these:
With default parameters (z m2 f4 s4), the computed image is this
(displayed at 50%, click to expand):
The masks, as shown in the Photoshop Layers palette, look like this:
masks, nested stack)
(blended masks, nested stack)
Here are three microscopy images, the computed masks, and the composite
image, created with default parameters. The source images were
downloaded from http://bigwww.epfl.ch/demo/edf/
For comparison, the following results are computed without masks, using
an algorithm based on complex wavelets, as described at http://bigwww.epfl.ch/demo/edf/
A More Challenging Example
The preceding two examples seem to be relatively "easy", in the sense
that all software tested so far on them does pretty well.
A more challenging example is presented by the lovely Columbine flower
photographed by Karl Gohl. Karl's original montage, viewable at http://www.pbase.com/image/28622076/large,
was generated entirely by manual editing of masks in a Photoshop stack,
as he describes at http://www.tawbaware.com/forum2/viewtopic.php?t=759
There are two main reasons why this example is more challenging than
the first two:
To attack this problem, I created a PTGui project containing the
original 7 full-size images. I defined control points and
optimized pitch/roll/yaw/fov to register visible details in the flower
and its leaves as well as possible
between frames. Then, based on visual inspection of the images, I
decided to place the backmost image at the top of the Photoshop stack,
because I anticipated having to do the most painting in that
mask. To accomplish that, I re-ordered the source images in the
PTGui project to be in front-to-back order. (This is because
pano12.dll constructs the Photoshop stack from bottom to top.)
Then I rendered, using PTGui's "show script" option to insert the "z"
line into the PTStitcher script.
seven source images for this montage cannot be registered as
perfectly as in our first two examples. Due to natural movement
flower stem, the flower rotates slightly between shots and also shifts
across the background. Here are all seven frames, before and
registration, animated as a film loop.
- Much of the image is background, which is not well focused in any frame. Thus, in
much of the image, an algorithm has essentially no information
conveniently available to tell it which frame to select.
Quite frankly, I was surprised at how well the current simple algorithm
handled this problem. Coming straight out of the algorithm, the
flower looked pretty good. The background, as expected, was sort
of randomly selected from various images based on noise in the original
images. However, that aspect was easily cleaned up by painting
white in the background image's mask -- the top one in the Photoshop
To facilitate comparison with other available software, I then backed
up and tweaked the PTGui project to produce a set of registered images
at a convenient test/demo resolution that I could use as a standard
input set for all software to be compared. Then I ran the
test/demo size registered images
through four different extended depth-of-field packages:
These test images and outputs can be downloaded here: ColumbineDemoProject.zip
- This experimental pano12.dll
- CombineZ (http://www.hadleyweb.pwp.blueyonder.co.uk/CZ4Docs/pages/introduction.htm)
- Syncroscopy Auto_Montage Essentials version 5.01.0006 ES DEMO (http://www.syncroscopy.com/syncroscopy/am.asp)
- Helicon Focus 2.03 Lite (http://helicon.com.ua)
Following is a summary of the results. (You can click on any of
these images to see them full-size
in new browser windows.)
This experimental pano12.dll (raw output, no editing)
Mottled background, some softness in the image, particularly in the
transition between backmost two images.
CombineZ, using default parameters and the single command Special | Do
Hard edged streaky artifacts in background, doubled edges along some
petals and throughout the yellow stamens.
I don't know how to adjust the parameters on CombineZ to make this
better. Suggestions, anyone?
Syncroscopy Auto_Montage Essentials version 5.01.0006 ES DEMO (Scan
Montage, Method: Fixed, Patch Size: 10).
Mottled background, otherwise quite good -- crisp rendering and correct
selection of source images..
Helicon Focus 2.03 Lite contrast estimation radius 10, smoothing 4,
background dim-out 0.
Extremely good automated result. With these parameters, image
suffers only from a slight smoothness, visible for example in the stem
With smoothing=0 (not
shown), image details are crisp but background goes mottled.
This experimental pano12.dll, after editing just the topmost Photoshop
Smooth background (due to mask editing), some softness in areas that
fall "between" two images.
What I see here is a spectrum of capabilities and costs, with no single
perfect solution. Aside from the slight smoothness, the Helicon
Focus image is extraordinary. However, this tool has no image
registration or image editing capabilities of its own.
Syncroscopy Auto-Montage has both, but it is rather expensive and the
editing function is sluggish for large images. CombineZ is
extremely easy to use -- four mouseclicks to align and combine all the
images -- but for this example, its output image is significantly lower
quality than the others. The experimental pano12 is still a bit
clunky, but its output image and editable format has some attraction if
you are going for highest quality. And of course, it's free.
Your mileage may vary. I am not expert with any of these codes,
including my own. Suggestions for improvement will be appreciated.
June Beetle Example
This is a more recent example, designed to see how this Panorama Tools
technology behaves when pushed.
The June Beetle was imaged using a Canon Digital Rebel camera with a
Sigma 100 mm macro lens at 1:1, f/11. 55 input frames were
captured at size 3072x2048 pixels, stepping the subject-to-lens
distance by 0.010 inches in depth to guarantee that every
point was well focused in some image. (This increment was smaller
than necessary. Stepping by 0.030 inches would have yielded
almost the same quality.) These 55 input frames were entered into
a PTGui project (version 3.7beta1, from http://www.ptgui.com).
Control points were generated automatically using autopano_v103 (http://www.le-geo.com/kolor/autopano/),
invoked from PTGui. A couple of iterations were performed, using
PTGui optimization for fov/pitch/yaw/roll and the APClean utility (http://www.fsoft.it/panorama/APClean.htm)
to remove out-of-concensus control points, eventually leaving in place
3042 control points with an rms error of 0.06% of image width (1.95
pixels out of 3072).
An extended-depth-of-field output image was rendered
direct to JPEG using "z s7 f7" at 3072x2048 pixels.
required light editing to clean up the background and to improve upon
automatic determination of best focus within the foreground
antenna. To enable this editing, a second rendering run was done,
identical to the first except that the output image format was changed
to "Multi-image TIFF". This rendering run generated 55 separate
output images, each properly registered against the single
extended-depth-of-field image. Then, using Photoshop CS,
visually selected portions of a few of these separate images were
manually merged into the extended-depth-of-field image. This
merging was done by repeatedly overlaying the extended-depth-of-field
image with one of the separate images,
adding a "hide all" layer mask, painting the layer mask white
to use the selected image's pixels instead of the algorithm's output,
then flattening the two layers into a single improved
extended-depth-of-field image. (Unlike using Photoshop's "clone" tool,
working with the mask image allows one to edit nondestructively until
the two layers are flattened.)
this application, it would not
have been effective to have Panorama Tools generate a .psd file
because the large image size and large number of input images would
have produced an unmanageably large .psd file. It was much more
practical to let Panorama Tools generate a single flattened
extended-depth-of-field image, then do the minor editing one image at a
For comparison purposes, I also tried running the June Beetle example
through Helicon Focus to compute the extended-depth-of-field image from
the registered "Multi-image TIFF" files generated by Panorama
Tools. As with the Columbine flower example discussed above, the
Helicon Focus output was very good other than some overall softness
that I could not eliminate without introducing visibility
defects. It seems again that the tradeoff is between highest
final quality and ease of use. For routine use, an attractive
workflow is to use Panorama Tools to generate properly registered
intermediate images, and feed those to Helicon Focus for
However, quality of output can be further improved by combining the
extended depth-of-field outputs of Panorama Tools and Helicon
Focus. To use this workflow, run Panorama Tools once to generate
an extended depth-of-field image, and a second time to generate
registered intermediate images which get run through Helicon
Focus. Then load both extended depth-of-field images into
Photoshop as two layers with masking, and manually paint the mask to
reveal the best parts of both images. In my testing, it seems
typical that Panorama Tools generates a sharper image over most of the
area, while Helicon Focus does a better job around the edges of
objects. (Note: it's probably best not to use the free version of
Helicon Focus with this method, since then their required message gets
more than a bit misleading.)
Known Bugs & Limitations
1. This version does not work if the red channel is saturated.
[At present, the pano12.dll code uses only the red channel for
focus. If the red channel saturates, then the code will think
the image has no contrast at that position and will make
correspondingly bad decisions about focus,
even though the green and blue channels contain perfectly good
This limitation is only because I had to hack the algorithm into
pano12.dll (since PTStitcher source code is not available), and at the
best place that I could find to tap in, the only available image data
is the red channel.]
2. This version correctly handles only those areas of the picture that
are covered by all images. If any image does not cover some area
of the picture, then incorrect masks are likely to be computed in that
area, particularly if blended masks option m2 (default) is used.
It is not immediately clear how to remove this restriction, due to
difficulty defining a smoothing algorithm that is both correct and
useful even around image edges.
3. No support for 16-bit images -- will not work properly and will not
diagnose the problem.
4. The recommended settings for fN and sN are preliminary and
have been developed with small images (<1Kx1K) from a digital
SLR. For larger and/or noisier images, larger window sizes will
be required. Processing large images with large windows can be
time consuming -- the columbine image at 3K x 2K and f19 s19 took close
to an hour at 2.8 GHz. The computation time for the current
implementation scales as the square of image size times the square of
window size. (The surrounding window contents are evaluated for
each pixel from scratch. There are standard ways to speed this up
a lot by incremental
updates, if there is sufficient demand.)
5. No support for smoothed masks except with m2 (stacked blended
masks). For example there is no concept of exclusive masks with
soft edges, equivalent to painting masks in Photoshop with a soft-edged
6. Does not work if the Panorama Tools options for color and/or
brightness correction are selected. Be sure to disable these
corrections in your GUI. If you need these corrections, make them
in a separate step to generate new image files, then use the corrected
files as input to the extended depth-of-field code.
7. The current version of autopano (v103) sometimes generates incorrect
control points for images that contain out-of-focus regions. This
problem is being investigated.
I would like to hear feedback about this work. Please let me know
successes, failures, problems, or other comments. Here is current
information on how to reach me.
Page last modified November 25, 2009. Page
modification April 20, 2005. Previous major
October 23, 2004.