Jet and Active Directory

[David W. Fenton]

Someone just challenged me on the assertion I've been making for
ages that Jet is used as the data store for Active Directory, and
cited a quote that says it's the format of a variant of the Jet Blue
engine (i.e., the one used in Exchange) rather than the Jet Red
engine (our beloved Jet) that the AD data store is in. This raises
the question of why the Windows development team took over
responsibility for Jet 4 after the release of Access 2000 if it's
not Jet 4 that's being used for AD's data store.

Some googling about produced a reference to a Jet Blue-ISAM,
suggesting that perhaps the Jet Red engine is used for access to a
Jet Red data store using that ISAM.

Can anyone confirm/deny this? Or provide any pointers to some
documentation of what's going on? My reputation on StackOverflow.com
might be in the balance!

 

[Wolfgang Kais]

Hello David.

Someone just challenged me on the assertion I've been making for
ages that Jet is used as the data store for Active Directory, and
cited a quote that says it's the format of a variant of the Jet Blue
engine (i.e., the one used in Exchange) rather than the Jet Red
engine (our beloved Jet) that the AD data store is in. This raises
the question of why the Windows development team took over
responsibility for Jet 4 after the release of Access 2000 if it's
not Jet 4 that's being used for AD's data store.

I think, that "someone" is right. afaik AD and Exchange both use the
ESE (Extensible Storage Engine, also known as JET Blue).

Some googling about produced a reference to a Jet Blue-ISAM,
suggesting that perhaps the Jet Red engine is used for access to a
Jet Red data store using that ISAM.

There where two different ISAMs: BC7 (JET Red) and DAE (JET Blue) that
share a common API specification: JET (Joint Engine Technology).

Can anyone confirm/deny this? Or provide any pointers to some
documentation of what's going on? My reputation on StackOverflow.com
might be in the balance!

Maybe this helps:
http://en.wikipedia.org/wiki/Extensible_Storage_Engine#History
As you yan see in the comparison section, ESE does not have it's own
query interface. Both Exchange and the AD service provide access to
their databases through the LDAP protocol, a possible query interface
is ADSI. There are also OleDB providers that can be used:
ExOLEDB (Exchange) and ADsDSOObject (Active Directory).

 

[David W. Fenton]

I think, that "someone" is right. afaik AD and Exchange both use
the ESE (Extensible Storage Engine, also known as JET Blue).

But why is Jet 4 (Jet Red) what ships with Windows? Is it used with
some interface to ESE?

There where two different ISAMs: BC7 (JET Red) and DAE (JET Blue)
that share a common API specification: JET (Joint Engine
Technology).

But they don't have anything at all to do with each other any
longer. Jet (red) can read Exchange data with the proper drivers,
but it doesn't do it natively. It was explained to me long ago that
the forking of Jet into Red and Blue happened so long ago that they
just have zilch to do with each other any longer except for the
name.

Maybe this helps:
http://en.wikipedia.org/wiki/Extensible_Storage_Engine#History
As you yan see in the comparison section, ESE does not have it's
own query interface. Both Exchange and the AD service provide
access to their databases through the LDAP protocol, a possible
query interface is ADSI. There are also OleDB providers that can
be used: ExOLEDB (Exchange) and ADsDSOObject (Active Directory).

None of that explains why MS is including Jet 4 (i.e., Jet Red) in
ever copy of Windows since 2000 was released. The explanation I have
been depending on came from Michael Kaplan, i.e., that it was used
by Active Directory. But if ESE is the actual data store, why is the
Jet 4 engine being shipped with Windows instead of the Exchange
version of Jet?

 

[Wolfgang Kais]

Hello David.

But why is Jet 4 (Jet Red) what ships with Windows? Is it used with
some interface to ESE?

As far as I know, IIS, Wins and DHCP use Jet (Blue) databases. I don't
think that Jet Red is shipped with Windows. Jet Red is installed with
MS Office and other developper tools, and it was contained in earlier
versions if MDAC (2.5-).

[...]
But they don't have anything at all to do with each other any
longer. Jet (red) can read Exchange data with the proper drivers,
but it doesn't do it natively. It was explained to me long ago that
the forking of Jet into Red and Blue happened so long ago that they
just have zilch to do with each other any longer except for the
name.

As I understood it, the two ISAMs where separately developped, and
later both project teams (BC7 and DAE) where joined together using
only one name: JET.
Jet (red) can for sure not read Exchange data, but it might be able
to send requests to a service that accesses the Exchange database.

None of that explains why MS is including Jet 4 (i.e., Jet Red) in
ever copy of Windows since 2000 was released. The explanation I have
been depending on came from Michael Kaplan, i.e., that it was used
by Active Directory. But if ESE is the actual data store, why is the
Jet 4 engine being shipped with Windows instead of the Exchange
version of Jet?

In this case, the great Michael Kaplan was wrong. Definitely, Echange
and AD both use Jet Blue (ESE with transaction log files), not Red.
More information on Jet Blue can be found here:
http://msdn.microsoft.com/en-us/library/ms684493(VS.85).aspx
I have googled, too, trying to find out if at all any implementation
of JET (Red) is shipped with Windows and did not find any official
statement. How can you know this? If it was true, then: good question.
Maybe it's for searching and indexing the file system.

Sorry for being not much help for you.

 

[David W. Fenton]

I have googled, too, trying to find out if at all any
implementation of JET (Red) is shipped with Windows and did not
find any official statement. How can you know this?

How can I know that it's part of Windows? Because the developers of
Access 2007 talked about the fact that Jet 4 was owned by the
Windows group, and that they forked the Jet codebase to produce the
ACE, which the Access groups owns.

Also, MSJET40.DLL is installed on every copy of Windows from 2000
on, and is maintained with up-to-date patches by Windows Update.

I may have misinterpreted what MichKa said about Jet being the
datastore for AD (he may have meant Jet Blue, whereas I interpreted
him to be referring to *our* version of Jet, Jet Red), but I am
pretty sure it was from him that I got this tidbit.

If it's not used in some way with AD, then that means it's a much
less crucial component, and not one that we can rely on as much to
be in all future versions of Windows.

 

[James A. Fortune]

Hello David.

But why is Jet 4 (Jet Red) what ships with Windows? Is it used with
some interface to ESE?

As far as I know, IIS, Wins and DHCP use Jet (Blue) databases. I don't
think that Jet Red is shipped with Windows. Jet Red is installed with
MS Office and other developper tools, and it was contained in earlier
versions if MDAC (2.5-).

[...]
But they don't have anything at all to do with each other any
longer. Jet (red) can read Exchange data with the proper drivers,
but it doesn't do it natively. It was explained to me long ago that
the forking of Jet into Red and Blue happened so long ago that they
just have zilch to do with each other any longer except for the
name.

As I understood it, the two ISAMs where separately developped, and
later both project teams (BC7 and DAE) where joined together using
only one name: JET.
Jet (red) can for sure not read Exchange data, but it might be able
to send requests to a service that accesses the Exchange database.

None of that explains why MS is including Jet 4 (i.e., Jet Red) in
ever copy of Windows since 2000 was released. The explanation I have
been depending on came from Michael Kaplan, i.e., that it was used
by Active Directory. But if ESE is the actual data store, why is the
Jet 4 engine being shipped with Windows instead of the Exchange
version of Jet?

In this case, the great Michael Kaplan was wrong. Definitely, Echange
and AD both use Jet Blue (ESE with transaction log files), not Red.
More information on Jet Blue can be found here:
http://msdn.microsoft.com/en-us/library/ms684493(VS.85).aspx
I have googled, too, trying to find out if at all any implementation
of JET (Red) is shipped with Windows and did not find any official
statement. How can you know this? If it was true, then: good question.
Maybe it's for searching and indexing the file system.

Sorry for being not much help for you.

I posted some links that contain, among other things, information about
the transition from MDAC as a stand-alone redistributable package to a
core component of Windows:

http://groups.google.com/group/microsoft.public.access/msg/f2041a04a501df59

especially:

http://support.microsoft.com/kb/892854

which states:

This release strategy limits the occurrence of DLL conflicts that may
occur when a customer has multiple versions of MDAC installed.

Of course Microsoft might have other reasons for having it there that
they're not telling us about .

James A. Fortune
(e-mail address removed)

 

[Wolfgang Kais]

Hello David.

How can I know that it's part of Windows? Because the developers
of Access 2007 talked about the fact that Jet 4 was owned by the
Windows group, and that they forked the Jet codebase to produce
the ACE, which the Access groups owns.

Also, MSJET40.DLL is installed on every copy of Windows from 2000
on, and is maintained with up-to-date patches by Windows Update.

We also have learned from all our research that "blue" and "red" are
different ISAMs. I think that Jet can be used to access all kind of
data that an ISAM is installed for, so MSJET40.dll might be a wrapper
dll that controls the use of the installed ISAMs.
Or do you know if MSJET40.dll especially refers to the "red" ISAM?
DAO is probably just a library that publishes Objects that use the
"red" parts.

I may have misinterpreted what MichKa said about Jet being the
datastore for AD (he may have meant Jet Blue, whereas I interpreted
him to be referring to *our* version of Jet, Jet Red), but I am
pretty sure it was from him that I got this tidbit.

If it's not used in some way with AD, then that means it's a much
less crucial component, and not one that we can rely on as much to
be in all future versions of Windows.

As said before: Maybe the dll mentioned above is a wrapper that is
also used for Jet Blue, and Jet Blue is used for many data stores in
the Windows operating systems: AD, DHCP, WINS, Secedit, Certificate
Services, Microsoft Update Client, Windows Desktop Search and also
Windows Mail (says wikipedia).
Also found this, which is interesting but also won't answer your
question: http://tinyurl.com/6gk6ta

 

[David W. Fenton]

We also have learned from all our research that "blue" and "red"
are different ISAMs. I think that Jet can be used to access all
kind of data that an ISAM is installed for, so MSJET40.dll might
be a wrapper dll that controls the use of the installed ISAMs.
Or do you know if MSJET40.dll especially refers to the "red" ISAM?

It's definitely the core DLL for Jet Red. But if there's an ISAM for
Jet Blue that it can use, that would certainly allow it to be used
to access the AD data store.

If you look back at my original post in this thread, that's exactly
what I suggested might be going on, and was looking for
confirmation.

DAO is probably just a library that publishes Objects that use the
"red" parts.

I'm pretty sure of that.

As said before: Maybe the dll mentioned above is a wrapper that is
also used for Jet Blue, and Jet Blue is used for many data stores
in the Windows operating systems: AD, DHCP, WINS, Secedit,
Certificate Services, Microsoft Update Client, Windows Desktop
Search and also Windows Mail (says wikipedia).
Also found this, which is interesting but also won't answer your
question: http://tinyurl.com/6gk6ta

Interesting article, though it just confirms what I already knew.
Aside from the wrong-headed note about record-level locking in Jet
Red being prone to crashes (which a commenter takes issue with), the
only thing that confuses me is that both Jet Red and Jet Blue are
referred to as ISAMs.

My understanding of this was that an ISAM is something that Jet can
load and communicate with. Certainly many of the data types that can
be accessed with linked tables use ISAMs, with Jet hosting those
ISAMs. That was the model I was thinking of when I first posted the
question, suggesting that perhaps "our" Jet was being used to host
the Jet Blue ISAM. This would make a certain amount of sense given
one piece of information from that article, i.e., that there is no
SQL interface provided for Jet Blue, but that you have to write your
own. Jet Red (and MSJET40.DLL) comes with a SQL engine and that
might explain the role that Jet Red plays in interacting with these
Jet Blue data stores.

But it's all speculation, of course.

Too bad someone from MS can't chime in and straighten things out.

 

[David W. Fenton]

I posted some links that contain, among other things, information
about the transition from MDAC as a stand-alone redistributable
package to a core component of Windows:

http://groups.google.com/group/microsoft.public.access/msg/f2041a04
a501df59

Could you please explain exactly what you think the MDAC has to do
with the inclusion of MSJET40.DLL in all versions of Windows since
Windows 2000? The URLs you cited in that post (at least the ones I
looked at) are all old and outdated. I'm not interested in reading
through them to dig out the nuggets, since I assume you've already
done so or wouldn't be citing them. Are there hints in those
articles of the role Jet Red plays in Windows? Or are they just
completely off topic from this thread?

 

[James A. Fortune]

Could you please explain exactly what you think the MDAC has to do
with the inclusion of MSJET40.DLL in all versions of Windows since
Windows 2000? The URLs you cited in that post (at least the ones I
looked at) are all old and outdated. I'm not interested in reading
through them to dig out the nuggets, since I assume you've already
done so or wouldn't be citing them. Are there hints in those
articles of the role Jet Red plays in Windows? Or are they just
completely off topic from this thread?

Good question. Maybe MSJET40.DLL is included in Windows so that
multiple threads can use JET replication to handle distributed
processing . Although Jet is no longer part of MDAC, I had a reason
for posting each link. For example, the first link was one of the few
references I found that actually stated what some of the DLL's are for.
You're right that this thread is not that thread, but keeping the
number of different DLL versions/variants low is one possibility for
including the MSJET40.DLL in the way that they did.

From:

http://www.microsoft.com/technet/security/bulletin/MS08-028.mspx

"The Microsoft Jet Database Engine provides data access to applications
such as Microsoft Access, Microsoft Visual Basic, and many third party
applications. Jet can also be used by Internet applications that require
database functionality."

So Microsoft says it's there to help applications that need database
functionality. We certainly don't want to hamper database
functionality, but with MSJET40.DLL being so problematic from a security
standpoint, why should Microsoft insist that everyone have it? Some
sites have the claim that if MSJET40.DLL is deleted, it comes back. Is
there data in the registry that requires the OS to use MSJET40.DLL?
Were there lots of things using JET Red, making it burdensome, or even
problematic, to choose JET Red or Blue exclusively for the OS? I don't
really suspect anything sinister since I've had Access databases that
got copied and maintained separately for other purposes without ever
intending for them to remerged at some point in the future.

James A. Fortune
(e-mail address removed)

http://www.lafn.org/webconnect/mentor/startup/PENINDEX.HTM

-- Identify processes, from Turbocharge Windows XP, Robert Strohmeyer

 

[David W. Fenton]

Good question. Maybe MSJET40.DLL is included in Windows so that
multiple threads can use JET replication to handle distributed
processing .

Jet Replication has its own set of DLLs, and those are *not*
included by default with Windows. So, no, I don't think that's the
answer.

Although Jet is no longer part of MDAC,

Perhaps precisely because Jet is part of the OS?

I had a reason
for posting each link. For example, the first link was one of the
few references I found that actually stated what some of the DLL's
are for.
You're right that this thread is not that thread, but keeping
the
number of different DLL versions/variants low is one possibility
for including the MSJET40.DLL in the way that they did.

From:

http://www.microsoft.com/technet/security/bulletin/MS08-028.mspx

"The Microsoft Jet Database Engine provides data access to
applications such as Microsoft Access, Microsoft Visual Basic, and
many third party applications. Jet can also be used by Internet
applications that require database functionality."

It fails to mention that Windows is using it for something or other,
or it wouldn't be included with Windows even when you don't have any
of those apps installed.

So Microsoft says it's there to help applications that need
database functionality. We certainly don't want to hamper
database functionality, but with MSJET40.DLL being so problematic
from a security standpoint, why should Microsoft insist that
everyone have it?

I don't think that's the reason it's installed with Windows at all,
since before Win2K, any app that needed it could install it as
needed. This worked just fine and dandy before Win2K, so I really
can't see how that would be any justification for including it with
Windows, let alone having the Windows development team take over the
maintenance of the Jet 4 db engine's codebase and development.

Some
sites have the claim that if MSJET40.DLL is deleted, it comes
back.

That's correct, because of file protection in place since Win2K. Any
number of Windows DLLs that you delete will reappear because of this
feature (and it's a good one, in my opinion).

Is
there data in the registry that requires the OS to use
MSJET40.DLL? Were there lots of things using JET Red, making it
burdensome, or even problematic, to choose JET Red or Blue
exclusively for the OS? I don't really suspect anything sinister
since I've had Access databases that got copied and maintained
separately for other purposes without ever intending for them to
remerged at some point in the future.

I would say that Jet Red would not have been folded into the Windows
team for maintenance and development if it were not used by some
part of Windows such that it *had* to be present. I can see no other
justification for both including it in all copies of Windows *and*
make its maintenance part of the responsibility of the Windows
development team.

 

[James A. Fortune]

I would say that Jet Red would not have been folded into the Windows
team for maintenance and development if it were not used by some
part of Windows such that it *had* to be present. I can see no other
justification for both including it in all copies of Windows *and*
make its maintenance part of the responsibility of the Windows
development team.

I see no fault in your logic. Is there a way to determine if a given
application uses Jet Red or not? If not, is there any other way that
can be used to narrow down the list of suspects? It seems that some
cleanup programs can tell when certain DLL's aren't being used by any
programs. Those DLL files coming back, though protective, work to limit
diagnostic techniques such as seeing which programs break when the DLL
is removed. Is Microsoft trying to be mysterious about the reason it is
included? Do the Resource Kits offer any explanation? What application
would you use for it? BTW, I thought that Microsoft's idea of using
local Active Directory information to validate a federated identity
system was quite clever.

James A. Fortune
(e-mail address removed)