Scanner woes

I'm new to this scanner malarkey, so please bear with me:

I'm scanning some negatives, using Nikon Scan and a Coolscan V. What I
want to achieve is to batch scan the film strips with the minimum of
user intervention, both to save time, and also to enable me to do
something else whilst scanning. I only want to handle the negs once, so
I scan them at the highest settings, in 14bit and with ICE & GEM set on,
saving them as uncropped, 'flat' TIFFs.

So far, so good. Until I noticed some of my scans had a colour cast.
Mostly blue, but in one extreme example, a deep blue sky was rendered
green. When looking at the individual RGB levels in Photoshop, I notice
that they are bunched up in the centre. Low contrast I think. Picking
the shadow and highlight points for all 3 channels more or less removes
the colour cast. I seem to be able to achieve the same thing using the
midpoint picker in the Nikon Scan curves tool.

What I what to know is, is there a difference in using these two
solutions? As I said earlier, I want to batch scan as much as possible
and I don't mind having to tweak the scans at a later date, using PS6.
I'm concerned about changing any setting in NS from 'default' as using
these settings, many of my scans seem to have correct colour and wonder
if these would then 'go the other way', IYSWIM.

Essentially, what I'm asking is: Is there any difference between
tweaking levels / curves in Photoshop, and getting the scan 'right' in
the first place?

TIA.
--
Alex Wilde

Are you letting NS set the exposure ?

Yes there is a difference between bunching up all of the density range
of the neg over a few bits of the A/D converter ( think using 6 bits out
of the 14 !) and using all 14 bits. You will loose the distinctions in
tone of the image. Whether it is visible to you are not depends on how
much of the density range the neg used for the image to begin with. If
it was reasonably exposed , then you should see the difference in the
two scans, at least partially on the CRT. If your image is only using 8
bits of density range or less , then you will not likely see any
difference in the CRT (8 bit /pixel device) between the two scans. The
extra "headroom" bits will help when you are adjusting the image in PS.
We have worked with 8 bit per pixel images for years and done quite
well, you just had to get the scan correct (setting white and black
points) to begin with and then limit the size of any curve edits to the
image.


Alex Wilde wrote:
> I'm new to this scanner malarkey, so please bear with me:
>
> I'm scanning some negatives, using Nikon Scan and a Coolscan V. What I
> want to achieve is to batch scan the film strips with the minimum of
> user intervention, both to save time, and also to enable me to do
> something else whilst scanning. I only want to handle the negs once, so
> I scan them at the highest settings, in 14bit and with ICE & GEM set on,
> saving them as uncropped, 'flat' TIFFs.
>
> So far, so good. Until I noticed some of my scans had a colour cast.
> Mostly blue, but in one extreme example, a deep blue sky was rendered
> green. When looking at the individual RGB levels in Photoshop, I notice
> that they are bunched up in the centre. Low contrast I think. Picking
> the shadow and highlight points for all 3 channels more or less removes
> the colour cast. I seem to be able to achieve the same thing using the
> midpoint picker in the Nikon Scan curves tool.
>
> What I what to know is, is there a difference in using these two
> solutions? As I said earlier, I want to batch scan as much as possible
> and I don't mind having to tweak the scans at a later date, using PS6.
> I'm concerned about changing any setting in NS from 'default' as using
> these settings, many of my scans seem to have correct colour and wonder
> if these would then 'go the other way', IYSWIM.
>
> Essentially, what I'm asking is: Is there any difference between
> tweaking levels / curves in Photoshop, and getting the scan 'right' in
> the first place?
>
> TIA.

In message <miMeg.102885$dW3.36597@newssvr21.news.prodigy.com>, degrub
<degrub@nospamflash.net> writes
>Are you letting NS set the exposure ?

Yes.
>
>Yes there is a difference between bunching up all of the density range
>of the neg over a few bits of the A/D converter ( think using 6 bits
>out of the 14 !) and using all 14 bits. You will loose the distinctions
>in tone of the image. Whether it is visible to you are not depends on
>how much of the density range the neg used for the image to begin with.
>If it was reasonably exposed , then you should see the difference in
>the two scans, at least partially on the CRT. If your image is only
>using 8 bits of density range or less , then you will not likely see
>any difference in the CRT (8 bit /pixel device) between the two scans.
>The extra "headroom" bits will help when you are adjusting the image in
>PS. We have worked with 8 bit per pixel images for years and done quite
>well, you just had to get the scan correct (setting white and black
>points) to begin with and then limit the size of any curve edits to the image.
>
I think I understand that. Given what you said the your last sentence,
is trying to archive negative scans 'unattended' a futile task? Maybe my
workflow needs rethinking.

What I can't understand is why some scans are bunched up as described,
and others okay - both seemingly of very similar contrast / exposure.
This is why I am worried about applying a 'global' curves setting to the
scans. In making the some right, surely I'm making others worse? If this
is the case, then my goal of almost uninterrupted scanning of negs is a
none starter.


--
Alex Wilde

Alex Wilde wrote:

> In message <miMeg.102885$dW3.36597@newssvr21.news.prodigy.com>, degrub
> <degrub@nospamflash.net> writes
>
>> Are you letting NS set the exposure ?
>
>
> Yes.
>
>>
>> Yes there is a difference between bunching up all of the density range
>> of the neg over a few bits of the A/D converter ( think using 6 bits
>> out of the 14 !) and using all 14 bits. You will loose the
>> distinctions in tone of the image. Whether it is visible to you are
>> not depends on how much of the density range the neg used for the
>> image to begin with. If it was reasonably exposed , then you should
>> see the difference in the two scans, at least partially on the CRT. If
>> your image is only using 8 bits of density range or less , then you
>> will not likely see any difference in the CRT (8 bit /pixel device)
>> between the two scans. The extra "headroom" bits will help when you
>> are adjusting the image in PS. We have worked with 8 bit per pixel
>> images for years and done quite well, you just had to get the scan
>> correct (setting white and black points) to begin with and then limit
>> the size of any curve edits to the image.
>>
> I think I understand that. Given what you said the your last sentence,
> is trying to archive negative scans 'unattended' a futile task? Maybe my
> workflow needs rethinking.
>
> What I can't understand is why some scans are bunched up as described,
> and others okay - both seemingly of very similar contrast / exposure.
> This is why I am worried about applying a 'global' curves setting to the
> scans. In making the some right, surely I'm making others worse? If this
> is the case, then my goal of almost uninterrupted scanning of negs is a
> none starter.

Hi Alex...

Experienced much of what you're seeing as well (archiving negs
that span many, many years)

I'm pretty much convinced that what's happening is that our
brain is doing a *lot* of pre or post or somewhere in between
of what the eye actually sees and what our brain thinks it should
see.

IOW, the eye can be fooled, but not the scanner.

FWIW, I'm three years into scanning negs and slides, and still
a little of half way through the pile... doing 'em one by one

Take care.

Ken

Another method of attack is to batch them and then go back and manually
scan the ones that are not good enough.


Ken Weitzel wrote:
>
>
> Alex Wilde wrote:
>
>> In message <miMeg.102885$dW3.36597@newssvr21.news.prodigy.com>, degrub
>> <degrub@nospamflash.net> writes
>>
>>> Are you letting NS set the exposure ?
>>
>>
>>
>> Yes.
>>
>>>
>>> Yes there is a difference between bunching up all of the density
>>> range of the neg over a few bits of the A/D converter ( think using 6
>>> bits out of the 14 !) and using all 14 bits. You will loose the
>>> distinctions in tone of the image. Whether it is visible to you are
>>> not depends on how much of the density range the neg used for the
>>> image to begin with. If it was reasonably exposed , then you should
>>> see the difference in the two scans, at least partially on the CRT.
>>> If your image is only using 8 bits of density range or less , then
>>> you will not likely see any difference in the CRT (8 bit /pixel
>>> device) between the two scans. The extra "headroom" bits will help
>>> when you are adjusting the image in PS. We have worked with 8 bit per
>>> pixel images for years and done quite well, you just had to get the
>>> scan correct (setting white and black points) to begin with and then
>>> limit the size of any curve edits to the image.
>>>
>> I think I understand that. Given what you said the your last sentence,
>> is trying to archive negative scans 'unattended' a futile task? Maybe
>> my workflow needs rethinking.
>>
>> What I can't understand is why some scans are bunched up as described,
>> and others okay - both seemingly of very similar contrast / exposure.
>> This is why I am worried about applying a 'global' curves setting to
>> the scans. In making the some right, surely I'm making others worse?
>> If this is the case, then my goal of almost uninterrupted scanning of
>> negs is a none starter.
>
>
> Hi Alex...
>
> Experienced much of what you're seeing as well (archiving negs
> that span many, many years)
>
> I'm pretty much convinced that what's happening is that our
> brain is doing a *lot* of pre or post or somewhere in between
> of what the eye actually sees and what our brain thinks it should
> see.
>
> IOW, the eye can be fooled, but not the scanner.
>
> FWIW, I'm three years into scanning negs and slides, and still
> a little of half way through the pile... doing 'em one by one
>
> Take care.
>
> Ken
>

On Mon, 29 May 2006 17:51:20 +0100, Alex Wilde
<awilde@cizeta.demon.co.uk> wrote:

>So far, so good. Until I noticed some of my scans had a colour cast.
>Mostly blue, but in one extreme example, a deep blue sky was rendered
>green. When looking at the individual RGB levels in Photoshop, I notice
>that they are bunched up in the centre. Low contrast I think. Picking
>the shadow and highlight points for all 3 channels more or less removes
>the colour cast. I seem to be able to achieve the same thing using the
>midpoint picker in the Nikon Scan curves tool.
>
>What I what to know is, is there a difference in using these two
>solutions? As I said earlier, I want to batch scan as much as possible
>and I don't mind having to tweak the scans at a later date, using PS6.
>I'm concerned about changing any setting in NS from 'default' as using
>these settings, many of my scans seem to have correct colour and wonder
>if these would then 'go the other way', IYSWIM.

A couple of things here. First of all, it's normal to have the
histograms bunched up in the middle in case of negatives. The dynamic
range (the difference between brightest and darkest points in an
image) is compressed on negatives. Your scanner has much more dynamic
range than that so the negatives end up only occupying a portion (the
middle) of this full scanner range.

So, if you don't do any image editing during scanning i.e. scan "raw"
the histograms are bunched up in the middle (or elsewhere, depending
on the exposure).

>Essentially, what I'm asking is: Is there any difference between
>tweaking levels / curves in Photoshop, and getting the scan 'right' in
>the first place?

No. That's a common misconception. Scanner software is actually
composed of two *unrelated* components: the actual scanning (i.e.
getting data from the scanner) and the editing tools.

So, in theory, it doesn't matter if you let the scanner software do
everything in one big gulp (i.e. scan and edit together) or scan "raw"
and do you edits later.

However, in practice scanner software editing tools are usually a poor
substitute (i.e. a very limited quick-and-dirty subset) of standalone
external editors like Photoshop. They exist only for marketing reasons
and people for whom that's good enough and they can't justify the cost
of a dedicated editor or wish to spend the extra time.

In case of NikonScan and Photoshop *6* there is one arcane
complication. This is really "out there" in the pedantic land, so you
can skip it. But here's the mind twister for completeness:

The 16-bit editing mode in Photoshop *6* is actually only 15-bit!
However, since CoolScan V is 14-bit that shouldn't be a problem,
right? Well, it is because gamma and calibration of this raw 14-bit
data is done using the full 16-bits in NikonScan. So, in theory you
may lose some data. But wait, there's more! ;o) Since you're scanning
negatives, they have a (vastly) reduced dynamic range, you will never
push at the limits of 14-bit let alone 16.

So, the bottom line in case of negatives is, you can skip that last
paragraph! ;o)

Don.

In message <pf0p72ta4pm6v6egi5bp212j0i2udo14j4@4ax.com>, Don
<phoney.email@yahoo.com> writes
>No. That's a common misconception. Scanner software is actually
>composed of two *unrelated* components: the actual scanning (i.e.
>getting data from the scanner) and the editing tools.
>
>So, in theory, it doesn't matter if you let the scanner software do
>everything in one big gulp (i.e. scan and edit together) or scan "raw"
>and do you edits later.
>
So what you are saying is that no setting in the editing tools of Nikon
Scan has any effect on the data pulled from the hardware? It only
effects the output, and I can do that better in PS?

>However, in practice scanner software editing tools are usually a poor
>substitute (i.e. a very limited quick-and-dirty subset) of standalone
>external editors like Photoshop. They exist only for marketing reasons
>and people for whom that's good enough and they can't justify the cost
>of a dedicated editor or wish to spend the extra time.
>
I understand this, and would rather edit in PS. I won't edit all the
images, just the ones that I want to print, but I thought I might as
well scan them all as best I could, then they can always be edited at a
later date.

>
>The 16-bit editing mode in Photoshop *6* is actually only 15-bit!
>However, since CoolScan V is 14-bit that shouldn't be a problem,
>right? Well, it is because gamma and calibration of this raw 14-bit
>data is done using the full 16-bits in NikonScan. So, in theory you
>may lose some data. But wait, there's more! ;o) Since you're scanning
>negatives, they have a (vastly) reduced dynamic range, you will never
>push at the limits of 14-bit let alone 16.
>
>So, the bottom line in case of negatives is, you can skip that last
>paragraph! ;o)

I usually have to convert the images I've edited to 8-bit anyway, but I
leave the original in 14-bit so I can maybe take more advantage of them
if I later get more up to date software. If I'm gonna scan all these
boxes of negatives, I only wanna do it once.
--
Alex Wilde

If i use a 3 bit A/D i will have 8 buckets for data. If i use an 8 bit
A/D i will have 256 buckets. i will get very different results for a
continuous tone image if i use the 3 bit A/D and stretch it to a 8 bit
range (using levels in PS for example). i will have 8 different tones
with gaps between in the 256 different tone levels! (This is what
happens during leveling when you see gaps open up in the histogram of
the image where before there were values in a narrow band only). Using
the 8 bit A/D i can more or less get a continuous gradation from "white"
to "black". If the image only has 3 bits worth of range, then Don is
correct and it will not make any difference.

A practical example of this is a "line art" scan of a typed sheet of
paper - a 1 bit A/D implementation. Everything is "black" or "white".
OCR software does a little bit better if the scan is done in "grey
scale" - 8 bit, 16 bit whatever, so that it can see some of the smudges
(and half tone patterns) that connect the dots on the page and
interpolate the pattern more or less correctly to make a letter.

BTW, anyone heard from Kennedy lately ?

regards,

Alex Wilde wrote:
> In message <pf0p72ta4pm6v6egi5bp212j0i2udo14j4@4ax.com>, Don
> <phoney.email@yahoo.com> writes
>
>> No. That's a common misconception. Scanner software is actually
>> composed of two *unrelated* components: the actual scanning (i.e.
>> getting data from the scanner) and the editing tools.
>>
>> So, in theory, it doesn't matter if you let the scanner software do
>> everything in one big gulp (i.e. scan and edit together) or scan "raw"
>> and do you edits later.
>>
> So what you are saying is that no setting in the editing tools of Nikon
> Scan has any effect on the data pulled from the hardware? It only
> effects the output, and I can do that better in PS?
>
>> However, in practice scanner software editing tools are usually a poor
>> substitute (i.e. a very limited quick-and-dirty subset) of standalone
>> external editors like Photoshop. They exist only for marketing reasons
>> and people for whom that's good enough and they can't justify the cost
>> of a dedicated editor or wish to spend the extra time.
>>
> I understand this, and would rather edit in PS. I won't edit all the
> images, just the ones that I want to print, but I thought I might as
> well scan them all as best I could, then they can always be edited at a
> later date.
>
>>
>> The 16-bit editing mode in Photoshop *6* is actually only 15-bit!
>> However, since CoolScan V is 14-bit that shouldn't be a problem,
>> right? Well, it is because gamma and calibration of this raw 14-bit
>> data is done using the full 16-bits in NikonScan. So, in theory you
>> may lose some data. But wait, there's more! ;o) Since you're scanning
>> negatives, they have a (vastly) reduced dynamic range, you will never
>> push at the limits of 14-bit let alone 16.
>>
>> So, the bottom line in case of negatives is, you can skip that last
>> paragraph! ;o)
>
>
> I usually have to convert the images I've edited to 8-bit anyway, but I
> leave the original in 14-bit so I can maybe take more advantage of them
> if I later get more up to date software. If I'm gonna scan all these
> boxes of negatives, I only wanna do it once.

On Tue, 30 May 2006 22:44:19 +0100, Alex Wilde
<awilde@cizeta.demon.co.uk> wrote:

>>No. That's a common misconception. Scanner software is actually
>>composed of two *unrelated* components: the actual scanning (i.e.
>>getting data from the scanner) and the editing tools.
>>
>>So, in theory, it doesn't matter if you let the scanner software do
>>everything in one big gulp (i.e. scan and edit together) or scan "raw"
>>and do you edits later.
>>
>So what you are saying is that no setting in the editing tools of Nikon
>Scan has any effect on the data pulled from the hardware? It only
>effects the output, and I can do that better in PS?

Exactly! The only thing that actually does affect the data is Analog
Gain i.e. exposure. But curves and friends only work on that data
after it has been retrieved from the scanner.

Now, there's a complication (as usual) and that's ICE. This is part
hardware and part software. The hardware part is the data from the
infrared LED. The software part is applying this to the image data.

In theory, there's no reason why the software part of ICE couldn't
also be done later. However, for marketing reasons Nikon was forced to
incorporate ICE in NikonScan and it doesn't give you an option of
saving this infrared data separately.

Finally, there's GEM, ROC and DEE. All that is software and in theory
could also be done later. But, again, for marketing reasons Nikon was
forced to incorporate it in NikonScan.

However, you can buy them as Photoshop plug-ins which are actually
much more flexible and powerful than the NikonScan built-in versions.

You can download all three - GEM (Grain Equalization Management), ROC
(Restoration Of Color) and SHO (Shadow & Highlight Optimized) - from:

http://www.asf.com

They (asf = Applied Science Fiction) are also the "inventors" of ICE
but were later bought out by Kodak.

The try-out versions put a watermark on the final image.

>>However, in practice scanner software editing tools are usually a poor
>>substitute (i.e. a very limited quick-and-dirty subset) of standalone
>>external editors like Photoshop. They exist only for marketing reasons
>>and people for whom that's good enough and they can't justify the cost
>>of a dedicated editor or wish to spend the extra time.
>>
>I understand this, and would rather edit in PS. I won't edit all the
>images, just the ones that I want to print, but I thought I might as
>well scan them all as best I could, then they can always be edited at a
>later date.

Yes, that's the gist of scanning "raw". You get the most out of the
scanner, archive that as your "digital negative" and then work on a
copy i.e. to print or make JPGs for screen viewing, reducing size as
required.

However, since you have the originals, once you get a bigger monitor
or higher resolution printer you can go back and create another print
or JPG to accommodate your new peripherals by making the most of them.

But the biggest benefit of this "digital negative" is that it is
fixed. The original (analog) film will continue to deteriorate and you
can't make a lossless copy. Digital data is frozen in time and you can
make as many lossless copies as you want. Of course, you have to make
sure you refresh and check your archives at regular intervals but
that's another story.

>I usually have to convert the images I've edited to 8-bit anyway, but I
>leave the original in 14-bit so I can maybe take more advantage of them
>if I later get more up to date software. If I'm gonna scan all these
>boxes of negatives, I only wanna do it once.

Exactly! That's precisely the point I was making above! Should've read
until the end before replying... ;o)

Basically, I would only use Analog Gain, ICE (if appropriate or
desired) and set gamma to 2.2 (in case of Windows). Strictly speaking
the data from the scanner comes as gamma 1.0 (called "linear gamma")
but since monitors are set to 2.2 there's little point in using 1.0.

Finally, I would do all the edits in 16-bit and convert to 8-bit as
the very last step before printing or converting to JPG or whatever.

Don.

Don wrote:
> On Tue, 30 May 2006 22:44:19 +0100, Alex Wilde
> <awilde@cizeta.demon.co.uk> wrote:
>
> >>However, in practice scanner software editing tools are usually a poor
> >>substitute (i.e. a very limited quick-and-dirty subset) of standalone
> >>external editors like Photoshop. They exist only for marketing reasons
> >>and people for whom that's good enough and they can't justify the cost
> >>of a dedicated editor or wish to spend the extra time.
> >>
> >I understand this, and would rather edit in PS. I won't edit all the
> >images, just the ones that I want to print, but I thought I might as
> >well scan them all as best I could, then they can always be edited at a
> >later date.
>
> Yes, that's the gist of scanning "raw". You get the most out of the
> scanner, archive that as your "digital negative" and then work on a
> copy i.e. to print or make JPGs for screen viewing, reducing size as
> required.
>
> However, since you have the originals, once you get a bigger monitor
> or higher resolution printer you can go back and create another print
> or JPG to accommodate your new peripherals by making the most of them.
>
> But the biggest benefit of this "digital negative" is that it is
> fixed. The original (analog) film will continue to deteriorate and you
> can't make a lossless copy. Digital data is frozen in time and you can
> make as many lossless copies as you want. Of course, you have to make
> sure you refresh and check your archives at regular intervals but
> that's another story.
>
> >I usually have to convert the images I've edited to 8-bit anyway, but I
> >leave the original in 14-bit so I can maybe take more advantage of them
> >if I later get more up to date software. If I'm gonna scan all these
> >boxes of negatives, I only wanna do it once.
>
> Exactly! That's precisely the point I was making above! Should've read
> until the end before replying... ;o)
>
> Basically, I would only use Analog Gain, ICE (if appropriate or
> desired) and set gamma to 2.2 (in case of Windows). Strictly speaking
> the data from the scanner comes as gamma 1.0 (called "linear gamma")
> but since monitors are set to 2.2 there's little point in using 1.0.

While slide film is engineered to reproduce the scene accurately (in
some cases with an intended bias towards saturation), transforming a
negative into an accurate color file takes a little more: profiling.
The simplest profiling will correct for colors being mixed-up instead
of pure (Matrix) , and having different gammas. Sure, in principle, you
can edit gammas in PS. With SelectiveCorrection, you can even correct
for the unwanted response of (say) the blue-sensitive layer to green,
etc... That is 9 possible adjustment parameters. The chances of hitting
the right values by just looking at the screen are slim. Matrix-Gamma
profiling will take care of that for you. Creative editing is somthing
else that should not, imho, be mixed with correcting for the media's
little defects. Then again, you could (having determined the film
profile with appropriate methods) apply the profile in PS. But why not
do that right at the scanning stage?
(Note: to be fair, you can also do it right after scanning, with a
command-line batch procedure using the jpegicc utility).
And, with the technical rendering issues being (mostly) taken care of,
one may go one step further, and tell the scanner software that you
want the white and black points at x% of the histogram, with x
conservatively small (for me white 0.1%, black 1%, you choices may be
different). Now the range of available numerical values is
efficiently used, and, depending on personal judgement, you might
consider that a moderate-compression (quality 90+, or 10 in PS scale)
jpeg is adequate and still robust enough for further creative editing.
One film in 16-bit tiff will fill the better part of a DVD. In
high-quality jpeg, the storage is of order 1/20 of 16-bit tiff.
Do note that I do *not* advocate to give the scanner software *any*
freedom to "interpret" the colors, as in "AutoWhiteBalance" or
"AutoLevels", etc... Just to take care of the repetitive part of image
processing.

Bernard