Handles and Kernel Object Reference Counts

[Norman Bullen]

I've always believed that, with respect to Kernel objects, reference
counting meant that the Kernel kept track of the number of handles
(unique 32-bit numbers) that it issued for each Kernel object. In most
cases, when that handle comes back to the Kernel in a call to
CloseHandle() or similar, the count is decremented and, if now zero, the
Kernel object is destroyed.

A recent experience seems to indicate that this belief is incorrect.

Please observe the following code. It's part of the preamble to a call
to CreateProcess() using pipes to receive the output of a command line
program. Two pipes are created and the handles of the input ends are
duplicated to make them inheritable. DUPLICATE_CLOSE_SOURCE should cause
the original non-inheritable handle to be closed.
CreatePipe(&hpipeStdOut, &hPipe, NULL, BUFFER_SIZE);
DuplicateHandle(hProcess, hPipe, hProcess, &startupInfo.hStdOutput,
0, TRUE, DUPLICATE_CLOSE_SOURCE|DUPLICATE_SAME_ACCESS
);
assert(!CloseHandle(hPipe));
CreatePipe(&hpipeStdErr, &hPipe, NULL, BUFFER_SIZE);
DuplicateHandle(hProcess, hPipe, hProcess, &startupInfo.hStdError,
0, TRUE, DUPLICATE_CLOSE_SOURCE|DUPLICATE_SAME_ACCESS
);
assert(!CloseHandle(hPipe));

Originally, it was written without the assert() statements and, as far
as I can tell, worked correctly.

I added the assert() statements as part of an effort to assure myself
that the program was no leaking handles. (The application of which this
a small part will pass through this code many times during its execution.)

I was surprised to find that the assertions failed, meaning that
CloseHandle() was returning a non-zero value indicating success--it was
able to the handles being passed and that in turn meant that the
original handles were _not_ closed by DuplicateHandle().

Further, I found that when my application attempted to read from the
output end of the pipes the ReadFile() failed with ERROR_BROKEN_PIPE
indicating that the input handle had been closed even though the command
line program had not had a chance to terminate. It looks like
CloseHandle() with the original handle closes both the original handle
and the duplicated handle.

I moved the assert() statement to a point after the call to
CreateProcess() and after the calls to CloseHandle() that close the two
startupInfo handles. (I don't need them anymore since they've been
inherited into the command line program by this point.) Now I get a
first chance exception from the CloseHandle() in the assert() and the
assertion fails; assert() pops up a message box.

Here's what I now believe to be happening: the Kernel is not counting
the number of unique handles that have been passed out, but is instead
counting the number of processes to which those handles have been
passed. The Kernel treats any handle owned by a process as equivalent,
at least in the context of CloseHandle(). (Any handle passed to
CloseHandle() closes all handles to that Kernel object that are owned by
the calling process.) I may, if I can find time, do some more
investigation to see whether all handles are treated as equivalent with
respect to access and inheritance.

Any thoughts on this? Is this behavior documented somewhere?

Norm

 

[Sergei Zhirikov]

Any handle passed to

CloseHandle() closes all handles to that Kernel object that are owned by
the calling process.

I doubt very much that this could possibly be true.

It seems to me that you are jumping into conclusions. For example, I can
offer an alternative explanation. (Note, it is purely theoretical, not based
on any knowledge of kernel internals, and I didn't check anything of it in
practice, so it is just a speculation.) What could happen is that when you
tell the kernel that you won't be needing the old handle (by setting
DUPLICATE_CLOSE_SOURCE) the kernel might decide to reuse the handle value.
In that case the handle produced by DuplicateHandle (the one put into
startupinfo) would have the same value as the original handle (but it
wouldn't be the same handle though). Then your call to CloseHandle would
close the new (duplicated) handle.

Does this make any sense to you? Of course, I'm sure one can think of more
possible explanations for what you are observing.

 

[Carl Woodward]

I've always believed that, with respect to Kernel objects, reference
counting meant that the Kernel kept track of the number of handles (unique
32-bit numbers) that it issued for each Kernel object. In most cases, when
that handle comes back to the Kernel in a call to CloseHandle() or
similar, the count is decremented and, if now zero, the Kernel object is
destroyed.

Not quite, kernel objects can also be referenced by kernel mode modules,
this also increases the reference count. It is not necessarily true that
NtCloseHandle will cause the object to be destroyed, but it will remove it
from the process handle table for the process context in which you called
NtCloseHandle.

Carly

 

[Norman Bullen]

I doubt very much that this could possibly be true.

It seems to me that you are jumping into conclusions. For example, I can
offer an alternative explanation. (Note, it is purely theoretical, not based
on any knowledge of kernel internals, and I didn't check anything of it in
practice, so it is just a speculation.) What could happen is that when you
tell the kernel that you won't be needing the old handle (by setting
DUPLICATE_CLOSE_SOURCE) the kernel might decide to reuse the handle value.
In that case the handle produced by DuplicateHandle (the one put into
startupinfo) would have the same value as the original handle (but it
wouldn't be the same handle though). Then your call to CloseHandle would
close the new (duplicated) handle.

Does this make any sense to you? Of course, I'm sure one can think of more
possible explanations for what you are observing.

I had an opportunity today to re-run this in the debugger and found that
Sergei is correct; DuplicateHandle() is returning the same numeric value
for the new handle as was passed for the original handle that was to be
copied and closed.

So when I added the the assert(!CloseHandel(hPipe)) statement it was
actually closing the new handle (and thus the kernel object) before it
could be inherited by the new process.

Sorry for the confusion.

Norm

 

[Lucian Wischik]

So when I added the the assert(!CloseHandel(hPipe)) statement it was
actually closing the new handle (and thus the kernel object) before it
could be inherited by the new process.

Just as an aside, I think it's VERY bad practice to have side-effects
inside an assert expression!

 

[Alun Jones]

Just as an aside, I think it's VERY bad practice to have side-effects
inside an assert expression!

Agreed, and here's why:

#ifdef NDEBUG
#define assert(exp) ((void)0)
#else

So, in the debug version, you close the handle, and in the release version,
you don't close the handle. Makes you wonder why you'd want to test and debug
a version of the program that has different behaviours from the one you give
to your customers!

Alun.
~~~~

[Please don't email posters, if a Usenet response is appropriate.]