ScanMultiPro with Vuescan "Long exposure pass"

[UrbanVoyeur]

With great trepidation at what I started, I'm going to post a new
observation regarding the effect of long exposures on my ScanMultiPro:

Since I found that checking "Exposure Lock" gives me manual control of
scanning exposure I have made some single scans at different levels of
exposure and experimented with HDRShop. It looks like (shudder!) both
sides are right. High level exposures (8 and above) yield
artifact-filled images and Vuescan makes them even worse by the method
it uses to combine them when "Long Exposure Scan" is checked.

I now pull my head down into the trench and hope the shrapnel blows
over in a month or two.

Back to the topic at hand: if anybody wants to pick up this thread in a
relevant, polite and strictly factual manner, please email me directly
for JPEGs of sample scans; if there is enough interest (if you've read
this far through the previous 39 posts, I'll count that as a definite
YES!) I'll post them publicly and maybe start a new thread.

Frank

Thank you.

 

[Don]

Don rarely states just the facts, objective or otherwise

No, Don always *starts* by merely stating objective facts. (The key
words here is *starts*!)

- he often accompanies them with a snipe
at the other person's knowledge or competence, e.g. "That shows that you do
not understand the most basic concept of HDR:" and "That shows, again, that
you don't really understand HDR." etc, etc.

You're talking things out of context. Please go back to the beginning,
don't just jump in the middle of the thread. That's the key problem
here. People just jump in and cherry pick.

That was only *after* J *repeatedly* accused me of "not understanding"
what blooming is and Bart cowardly sniping from the sidelines and then
running to hide behind a mythical "filter". Not only do I know what
blooming is, but it's totally irrelevant in the current context.

Please read the *whole* thread!

Yet your posts declare
absolute certainty - no room here for doubt - it's all objective fact.

Now, if *I* wrote the above sentence I'd be accused of everything from
"not understanding" to "bashing" and, from "being offensive" to
"arrogance"!

I do *not* declare things with absolute certainly. I merely state
known facts.

*If* the other side would also state the facts, then - in case the
facts are contradictory - we can objectively evaluate both sets and
decide which facts are correct.

The problem is, the response is usually "it works for me" or "I kinda,
sorta, see colors", etc... or they go off on an irrelevant tangent!

Therefore, until the other side does come up with the facts, the
original statement stands unchallenged. That's all!

Yet
people are questionning whether what you say is, indeed, objective fact. A
that point, you have to accept a difference of opinion

And that's the key! This is *not* about opinion, but about fact.

It is precisely because the other side puts up *subjective opinion*
against *objective fact* that's causing the problem. As well as the
failure to graps the difference between fact and opinion.

To assert
that your knowledge is superior is likely to be seen, at least by some, to
be arrogant.

Which, as repeatedly explained, is a gross misunderstanding and
distortion. I have *never* stated my knowledge as superior. Quite the
contrary!

Please quote a *single* occurrence, ===> *in context* <===!

If, for example, I met Ed Hamrick
in a pub and to his face and in public called him 'incompetent' - I would
expect to wake up in hospital.

Absolutely!

*But* that's not the case here!

If he *repeatedly* spread misinformation, and you patiently provided
facts to the contrary. Yet he continues, and them contradicts himself
even in the same thread. And then you politely quote these
contradictions. Etc... etc... (This goes for others as well.)

At some point you'd say that his/their refusal to accept these
*objective* facts without bothering to counter them - except with
"subjective opinion" - is, indeed, incompetence, if not worse.

If someone else now jumps in at that point, they may get a totally
wrong impressions of the situation and consider that the patient
person is the villain for "calling people names".

That's exactly what many here are doing constantly, without bothering
to check the context or the facts!

You seem reluctant to
concede anyone else's argument

Again, I'm sorry, but that's a totally biased and one-sided opinion
without any basis in fact.

Please state a *single* occurrence of this, ===> *in context* <===!

The observations of a neutral observer - and one with all his teeth intact

I don't know about neutral, but I do take your word about the dental
work. ;o)

Don.

 

[Don]

With great trepidation at what I started, I'm going to post a new
observation regarding the effect of long exposures on my ScanMultiPro:

Oh, don't pay attention to our hand-bag spat... ;o) Focus on the
information.

Since I found that checking "Exposure Lock" gives me manual control of
scanning exposure I have made some single scans at different levels of
exposure and experimented with HDRShop. It looks like (shudder!) both
sides are right. High level exposures (8 and above) yield
artifact-filled images and Vuescan makes them even worse by the method
it uses to combine them when "Long Exposure Scan" is checked.

The key difference is that VueScan does not align the images before
combining them, while HDR Shop does. Therefore - proving conclusivelly
- that VueScan's main problem is the misalignment. Just like I said at
the start... :-(

It's not clear from the above how many exposures you combined in HDR
Shop. If it's only two, then that's the problem. If you see "blooming"
that means you have to increase the series i.e. the number of scans,
until the artefacts disappear - and they will disappear.

On the other hand, if you already do have a series and artefacts are
still there, it might be for one of two reasons.

You may need to increase the series (go to half a stop, or even less)
and they will disappear.

The other thing is, what do you actually mean by artifacts. If you are
simply viewing the image in HDR Shop and increasing exposure, then
what you see is perfectly normal and to be expected.

If, however, you combine the images and export them as a single image
then there should be no artifacts in the final results. To do this use
the "tone mapping" plug-in also available on the site.

Finally, if the resulting image does not have enough detail in shadows
or the highlights are too bright, don't forget that HDR Shop allows
you to edit the image at any point! If you do that, then there will be
no artefacts in the final result.

I now pull my head down into the trench and hope the shrapnel blows
over in a month or two.

Wishfull thinking... ;o)

While I'm at it, I suggest, Don, that you setup a Vuescan Bugs website
and use that as a forum for providing useful information about the
software's problems in a form that may actually help Vuescan users who,
like me, are simply unwilling to wade through the crap to find any real
nuggets of information that these offensive posts may contain.

That's something even a semi-reputable software manufacturer would do
themselves!!

In its absence all you need to do is search the archives here. Many
VueScan bugs are recurring, pretending to disappear in one version and
then reappearing in another. Nothing to do with Don. These complaints
are regularly posted by exasperated VueScan users themselves.

As I have repeatedly stated, but since you're new you may not have
seen it:

If you find a version of VueScan than "works" for you, *don't upgrade*
just for the sake of upgrading! And if you feel you must, definitely
*keep* the version you are replacing! There have been countless posts
of unsuspecting victims "upgrading" and then scrambling to find the
old version again because the new one was too buggy.

Finally - and this is the key - a couple of questions worth
considering:

- Who provided you with the link to HDR Shop?
- How useful was this link?
- Has the VueScan crowd done the same or were they more (only?)
concerned with rabidly attacking Don rather then helping?

Food for though... Tells volumes, too!

Don.

 

[UrbanVoyeur]

Please post examples of this "blooming" including:

- the series used to generate the HDR image
- parameters and/or settings in the HDR program
- the final result

so that everyone can independently and objectively evaluate the data.

So to pick things up where we left off, I was of the opinion that
blooming affected nearby areas of the sensor in way that were not
necessarily correctable by either changing the clipping threshold in an
HDR merge or by radiance mapping.

Don believes that an HDR can be constructed to correct all of that, and
point out to an article that discusses radiance mapping among other
things: http://www.debevec.org/Research/HDR/debevec-siggraph97.pdf

Now, I'm not an expert, but from what I understand, radiance (tone)
mapping it is a method of measuring a sensor's linear response range. It
is useful in HDR because values found to be outside of the radiance
mapped values for that sensor area can be considered a non-linear
response and discarded.

Now, why doesn't this correct all of blooming? after all, you are
discarding the non-linear pixels responses (those overdriven by too much
light) and you are setting the clipping threshold to define where
highlights begin.

Well, as I see it, blooming, or the "spill over" effect of a sensor
receiving too much light does not drive all of it neighbors to either
clipping or non-linearity. It does increase their exposure, but not
enough to be outside their luminance mapped linear response range, nor
enough to for them to be considered "overblown highlights" above the
threshold.

Does that mean that HDR is not a valid means of extending the dynamic
range of scanned film. Certainly not. But in some cases where bright
highlight areas are adjacent to dark areas, there will be color shifts
that are not easily corrected.

As requested, I have posted a few images to illustrate.

I scanned a Kodak Ektachrome calibration target using Vuescan and a
Nikon LS-4000. These are single pass 4 x multisampled to reduce noise.
No sharpening or post processing.

These are all single pass, unmerged imaged. They are not HDR. They
simply illustrate blooming.

small, but full frame:
http://www.urbanvoyeur.com/bloom/small_vs_1_gain_correct_exp.jpg

I scanned it at 3 different analog gains, which in Vuescan with the
Nikon LS-4000 is just increasing the exposure time.

Here are the relevant crops from the lower right hand corner of the
slide, near the dark end of the gray scale.

Normal, zero unity gain scan:
http://www.urbanvoyeur.com/bloom/crop_vs_1_gain_correct_exp.jpg
R, G, B gain at +20 :
http://www.urbanvoyeur.com/bloom/crop_vs_20_gain_bloom.jpg
R, G, B gain at +100:
http://www.urbanvoyeur.com/bloom/crop_vs_100_gain_bloom.jpg

The full res scans are here:
http://www.urbanvoyeur.com/bloom/large_vs_1_gain_correct_exp.jpg
http://www.urbanvoyeur.com/bloom/large_vs_20_gain_bloom.jpg
http://www.urbanvoyeur.com/bloom/large_vs_100_gain_bloom.jpg


Notice the width of the white line - it grows with in each of the two
ever exposures. That's blooming.

Notice the edges of the white line. Even in the most overexposed, they
vary in color an luminance. That is the part of blooming that is not
easy to correct - it is not outside of the linear range of those sensor
areas and the darker parts are not elevated to the range of highlights.

When layered in HDR, these regions may in some images show up as color
fringing unrelated to alignment.

 

[Don]

These are all single pass, unmerged imaged. They are not HDR. They
simply illustrate blooming.

We all know what blooming is, J. That was never the question.

The questions was: To demonstrate your assertion that blooming affects
the resulting, merged image - which is what was used as an excuse for
VueScan's poor results in order to hide the real cause i.e. VueScan's
failure to align before merging, tone map, etc.

Therefore, in order to demonstrate your assertion, what we need - and
I'll try to make it very exact - is:

- the series of images used to generate the HDR image (so they can be
independently evaluated)
- parameters and/or settings in the HDR program (so the process can
be independently duplicated)
- the final, merged, tone mapped, etc. result showing this alleged
"blooming" (so the actual artifact you perceive as "blooming" can be
identified and its cause established)

Don.

 

[UrbanVoyeur]

We all know what blooming is, J. That was never the question.

The questions was: To demonstrate your assertion that blooming affects
the resulting, merged image - which is what was used as an excuse for
VueScan's poor results in order to hide the real cause i.e. VueScan's
failure to align before merging, tone map, etc.

Therefore, in order to demonstrate your assertion, what we need - and
I'll try to make it very exact - is:

- the series of images used to generate the HDR image (so they can be
independently evaluated)
- parameters and/or settings in the HDR program (so the process can
be independently duplicated)
- the final, merged, tone mapped, etc. result showing this alleged
"blooming" (so the actual artifact you perceive as "blooming" can be
identified and its cause established)

Don.

Don

Not everyone knows what blooming is, and not everyone has been following
this discussion from the beginning. Moreover, there has been quite a bit
of back an forth and few side threads. I wanted to lay out clearly where
we were.

The reason I started with these images is to reduce the number of
variables. HDR merging involves a number of discrete decisions and I did
not want a discussion of those to unnecessarily distract from the
central issues around blooming. And as this is a dialog, I get to impose
a few twists of my own in answering your question.

Before we go any further, lets just address what's in these images.

Do you agree that these demonstrate blooming?
Do you agree that alignment is not an issue in these scans?
Do you agree that one of the effects of blooming is changing the
exposure & color cast of adjacent areas but not necessarily driving them
to non-linearity or into the highlight range?

And while you are at it, am I or am I not correct in asserting that
radiance mapping is a method of measuring the linear response range of a
sensor or a part of a sensor and that its role in HDR is to more easily
identify and discard non-linear responses?

One more thing, Don:

I only used Vuescan to produce these initial scans. I do not believe I
have once in this discussion made a positive or negative statement about
Vuescan's ability to merge scans to produce an HDR. What I have been
discussing is the effects of blooming and how they relate to HDR.

Since we are not discussing the relative merits of Vuescan, I think
bashing the program is gratuitous.

 

[Don]

Not everyone knows what blooming is, and not everyone has been following
this discussion from the beginning. Moreover, there has been quite a bit
of back an forth and few side threads. I wanted to lay out clearly where
we were.

Fair enough. Even though there are plenty of examples of blooming on
the net, I'm all for clarity so, good start!

(You see, I thought that was it, but if it's only a start, well done!
Establishing both the context and the scope is very important!)

The reason I started with these images is to reduce the number of
variables. HDR merging involves a number of discrete decisions and I did
not want a discussion of those to unnecessarily distract from the
central issues around blooming. And as this is a dialog, I get to impose
a few twists of my own in answering your question.

Provided that it's relevant, then: Twist away!

Before we go any further, lets just address what's in these images.

Do you agree that these demonstrate blooming?
Yes.

Do you agree that alignment is not an issue in these scans?

I don't know.

You need to correlate the original files (not jpegs!) to see how much
misalignment there is (if any).

Do note also that misalignment is not necessary the same across the
whole image! It may be different in different parts of the scan.
(That's what's causing me bold spots right now! ;o)) Depending on the
scanner the image may need to be rotated or even scaled. HDR Shop will
handle all of those cases and align them automatically but it won't
say how much misalignment there was.

Also, the amount of misalignment will depend on the (sub-pixel) method
used. This is not a problem as long as the same method is used to
actually align the images afterwards.

Nevertheless, you can go ahead and merge them anyway, because there's
a clear distinction between blooming and misalignment. So the final
result should show that. I mean, I can think of possible complications
but, usually, the difference is quite clear.

Another possibility (as I mentioned before) is that you may actually
have seen blooming, in which case a number of things could have been
wrong - as already indicated - from choosing the point at which to
tone map to insufficient series, etc...? I just assumed you did the
merge correctly, in which case there should have been no blooming in
the histogram compresed result and, then, the prime suspect is
misalignment.

Do you agree that one of the effects of blooming is changing the
exposure & color cast of adjacent areas but not necessarily driving them
to non-linearity or into the highlight range?

Yup!

(It's not relevant in context of HDR, but "Yup" in the above narrow
context.)

And while you are at it, am I or am I not correct in asserting that
radiance mapping is a method of measuring the linear response range of a
sensor or a part of a sensor and that its role in HDR is to more easily
identify and discard non-linear responses?

Yes, but that's not really relevant because you're only looking at a
single instance (as you change the exposure). The above only addresses
what happens within that single point.

The thing is, regardless of how the HDR was generated, the key point
is that the areas of the image which are "out of bounds" are thrown
away in the final result. Radiance mapping is only the front end of
the process (and there are other methods too) but "histogram
compression" at the back end is where undesirable data is discarded.
And there are a bunch of those too! From linear to an infinite number
of "perception weighted" methods, a new one seemingly "discovered"
every day.

Therefore, how that data is arrived at is really not important if that
data is later thrown away when producing the final result. So, we
shouldn't really digress into this right now...

One more thing, Don:

I only used Vuescan to produce these initial scans. I do not believe I
have once in this discussion made a positive or negative statement about
Vuescan's ability to merge scans to produce an HDR. What I have been
discussing is the effects of blooming and how they relate to HDR.

Not quite. By agreeing with VueScan's excuse (i.e. by only blaming
blooming) you are, in effect, backing up VueScan.

This whole discussion started because blooming was given as the sole
excuse for VueScan's failure to merge the scans properly - without any
mention of alignment or tone mapping... etc., none of which VueScan
does and, all of which negatively affect the result!

Therefore, VueScan simply blaming "blooming" for everything while
hiding failure to align or tone map... etc. is a cop out. By agreeing
without any mention of what VueScan fails to do, you are defending it.

Since we are not discussing the relative merits of Vuescan, I think
bashing the program is gratuitous.

I'm *not* bashing it! Merely stating its limitations which the author
omitted. Even Frank has acknowledged the improvement once he started
using HDR Shop but you don't accuse Frank of "bashing VueScan"?

Speaking of which, one problem you may have with VueScan is when the
time comes to merge the series into an HDR image. (I'm just mentioning
this now so you can think about it beforehand.)

In order to merge you normally need to specify the interval used for
the series. This is usually expressed in EV. VueScan doesn't do EV but
it uses it's own non-standard "multiplier units". So, you will have to
convert these *correctly* into EV!

HDR Shop can automatically determine the interval but it's not very
accurate. Normally, that may not be a serious problem, but if we are
going to dissect the result at 1600% magnification, incorrect interval
will have a major impact!

So, another possibility is that what you perceived as "blooming" may
have actually been a case of an incorrect interval? But instead of my
pointless speculation... carry on!

Don.

 

[mlgh]

Don wrote:


J. >>Do you agree that alignment is not an issue in these scans?

Don.> I don't know.

You need to correlate the original files (not jpegs!) to see how much
misalignment there is (if any).

How can there be any misalignment if they are all individual single
scans?
......Or have I lost the plot?

mlgh

 

[UrbanVoyeur]

Don wrote:


J. >>Do you agree that alignment is not an issue in these scans?

Don.> I don't know.




How can there be any misalignment if they are all individual single
scans?
.....Or have I lost the plot?

You are correct mlgh. There is no misalignment because they are all
single pass scans without any merging.

I'll point that out in a lengthier direct reply to Don's post.

 

[UrbanVoyeur]

I don't know.

Just so everyone is clear,

Do you mean that alignment is *not* a issue with these individual scans
themselves, but *may* be an issue when they are recombined in a HDR
merge/blend.

Or

Alignment is an issue with these individual scans? That is, these
individual scans themselves may be out of alignment.

(I bring this up to clarify a question from another poster.)

Bear in mind that these scans as they are shown now are all single pass,
no blending.

You need to correlate the original files (not jpegs!) to see how much
misalignment there is (if any).

Will do. I just posted JPEG's 'cause they were easier for people to
download.

Yup!

(It's not relevant in context of HDR, but "Yup" in the above narrow
context.)

We'll come back to how it's relevant to the issue later.

Suffice it to say for now that this effect of blooming can produce a
colored "halo" around the bloomed area, as seen in the detail of these
single pass scans.

Yes, but that's not really relevant because you're only looking at a
single instance (as you change the exposure). The above only addresses
what happens within that single point.

The thing is, regardless of how the HDR was generated, the key point
is that the areas of the image which are "out of bounds" are thrown
away in the final result. Radiance mapping is only the front end of
the process (and there are other methods too) but "histogram
compression" at the back end is where undesirable data is discarded.
And there are a bunch of those too! From linear to an infinite number
of "perception weighted" methods, a new one seemingly "discovered"
every day.

Ahh, but what is considered linear vs non-linear is essence of
determining what is "out of bound" and is driver for the decision of
what to discard during an HDR merge.

If the area under consideration is behaving linearly and is below the
threshold for highlights, it is not discarded, correct?

Therefore, how that data is arrived at is really not important if that
data is later thrown away when producing the final result. So, we
shouldn't really digress into this right now...

Yes, we sill undoubtedly come back to this later.

Not quite. By agreeing with VueScan's excuse (i.e. by only blaming
blooming) you are, in effect, backing up VueScan.

I am laying out a position independent of Vuescan. I have not taken a
position on Vuescan's HDR performance so please don't assign me one.

This whole discussion started because blooming was given as the sole
excuse for VueScan's failure to merge the scans properly - without any
mention of alignment or tone mapping... etc., none of which VueScan
does and, all of which negatively affect the result!

Therefore, VueScan simply blaming "blooming" for everything while
hiding failure to align or tone map... etc. is a cop out. By agreeing
without any mention of what VueScan fails to do, you are defending it.

You are pointing out perceived shortcomings in Vuescan. Just to be
clear, I am avoiding that discussion an taking on an issue directly
related to HDR. I am neither defending nor attacking Vuescan.

 

[Don]

Don wrote:


J. >>Do you agree that alignment is not an issue in these scans?

Don.> I don't know.


How can there be any misalignment if they are all individual single
scans?
.....Or have I lost the plot?

That's how I read it at first, too...

But then, on second reading, I thought he can't be asking something
that obvious, and maybe J was "thinking ahead" in preparation for the
merge.

Don.

 

[Don]

Do you mean that alignment is *not* a issue with these individual scans
themselves, but *may* be an issue when they are recombined in a HDR
merge/blend.

That's right. Obviously, if there are only individual scans, then
there's nothing to compare to.

Actually, that's how I understood it the first time I read it, and
wrote a reply to that effect. But then I though, you can't be asking
something that obvious and that you were probably "thinking ahead".

Bear in mind that these scans as they are shown now are all single pass,
no blending.

That's fine. I just thought you were referring to the merge ahead.

Will do. I just posted JPEG's 'cause they were easier for people to
download.
Good!

Suffice it to say for now that this effect of blooming can produce a
colored "halo" around the bloomed area, as seen in the detail of these
single pass scans.

Yup. We all agreed on that from the start.

Ahh, but what is considered linear vs non-linear is essence of
determining what is "out of bound" and is driver for the decision of
what to discard during an HDR merge.

The whole point of HDR is to the "cafeteria" principle. You only pick
what you like. So if a particular exposure has areas you don't like
you leave them out. So if something is out of range of a particular
scan you leave it out. Now, whether what you leave out is linear or
not, doesn't really matter. What's important is what you take from it.

If the area under consideration is behaving linearly and is below the
threshold for highlights, it is not discarded, correct?

Linear or not is, strictly speaking, not the issue. All you care for
(in this case) is proper exposure without artifacts.

Indeed, most (if not all?) HDR software internally works at linear
gamma because the math is easier and there is least corruption of
data.

I can also conceive a method where you could use an "HDR-like" merge
to combine multiple scans with the *same* exposure, only each with a
different focus to get around curved film! In that case you would
discard areas out of focus, similarly to discarding clipped and noisy
areas, respectively, when contrast blending.

I am laying out a position independent of Vuescan. I have not taken a
position on Vuescan's HDR performance so please don't assign me one.

It's not the position I'm assign to you but the fact that you did
agree that the "reason" for VueScan's failures (as reported by Frank)
was due to blooming.

You are pointing out perceived shortcomings in Vuescan. Just to be
clear, I am avoiding that discussion an taking on an issue directly
related to HDR. I am neither defending nor attacking Vuescan.

By agreeing with the excuse put forth by the author you are implicitly
defending it. But, I agree, let's not digress...

Don.