> maybe he can successfully shed some more light on this subject.

Here are some more basics, just so that we're all on the same page:

* In theory one can have multiple I/O APICs in a system. In practice,
on the sorts of systems we're talking about here, there's only going to
be one. It lives in the PCI-to-ISA bridge part of what is variously
called an "I/O Controller Hub" (ICH) or a "PCI ISA IDE Xcelerator"
(PIIX) by Intel or a "South Bridge" by VIA.

* The difference between MPS (Intel's "Multi-Processor Specification")
version 1.1 tables and version 1.4 tables is that the version 1.4 tables
contain various extensions, dealing with things like multiple I/O APICs.
Again, this will be largely irrelevant here.

* For most, perhaps all (certainly all that I have datasheets for),
chipsets, there is a control bit of some form in configuration space
that enables or disables the I/O APIC. If there are no local APICs
enabled on the other end of the APIC bus, however, it doesn't really
matter what state the I/O APICs are in. Nothing will be listening to
their messages. Indeed, in a more extreme case, the APICCLK signal may
be simply tied to ground, and no I/O APIC messages will happen on the
APIC bus at all, because its clock is frozen.

* There are two ways to turn off a local APIC: the hard way and the easy
way. Both involve setting flags in CPU registers. If a local APIC is
turned off the hard way, it cannot be turned back on again without
potentially losing interrupts and confusing the entire APIC bus
arbitration scheme. A local APIC turned off the easy way can be re-enabled.

* There are two ways for firmware to report I/O APIC configuration
information to an operating system: the MPS table, and ACPI tables.
They aren't quite the same. The configuration information reported
states among other things which ISA devices and which PCI IRQ lines are
connected to which I/O APIC inputs. There's no requirement, after all,
that every motherboard manufacturer connect the ISA IRQ #7 signal to I/O
APIC INTIN pin #7.

* In addition to deciding how to program the enable control bits for the
I/O APIC and all of the local APICs, firmware also gets to decide what
it reports in the MPS and ACPI tables. It might decide to lie, for
example, and deny the existence of I/O APICs or local APICs in the
machine. The idea of this would be to force an operating system, whose
primary source of this hardware information is supposed to be the MPS
and ACPI tables, to still "see" an ACPI system, but one that doesn't
have APICs; thereby forcing it to fall back to whatever dual-8259 mode
of operation it has, whilst still retaining other unrelated
ACPI-provided information such as (say) system reset and environment
control capabilities.

* An OS/2 Platform-Specific Driver (PSD) is not a Windows NT Hardware
Abstraction Layer (HAL). A HAL abstracts away quite a lot of the
details of interrupt processing and low-level inter-processor
synchronization and communication. A PSD does not. There are two
particulars of note. First: The OS/2 kernel has fallback code that
knows how to talk to dual 8259s, should a PSD not implement certain
optional capabilities, and operate what is essentially an *asymmetric*
multiprocessor system. Second: An OS/2 PSD has no responsibility for
implementing spinlocks. So a system where the PSD omits the optional
features does not devolve to being identical to a uniprocessor OS/2 system.

* The idea that I/O APICs increase the number of available interrupts is
a bit of a swizz. The number of PIRQ signal lines on the PCI bus
doesn't magically change. Some internal devices, built in to the
southbridge, gain the ability to use extra interrupt signals that don't
exist on the real PCI bus (On some VIA southbridges, for example, the
internal PCI-to-ATA bridge gets to use PCI INT #E.). But, really,
stating that the point of an I/O APIC is to "gain more interrupts" is
mis-selling it. I/O APICs provide better ways to manage and control the
*same* set of PCI and ISA interrupt signals, not more of them.

On Wed, 10 Aug 2011 14:51:07 UTC, Jonathan de Boyne Pollard
<J.deBoynePollard-(E-Mail Removed)> wrote:

> > maybe he can successfully shed some more light on this subject.
>
> Here are some more basics, just so that we're all on the same page:


....snip...snip...a ton of absolutely WEALTHY WONDERFUL info...thank you!!!


> * An OS/2 Platform-Specific Driver (PSD) is not a Windows NT Hardware
> Abstraction Layer (HAL). A HAL abstracts away quite a lot of the
> details of interrupt processing and low-level inter-processor
> synchronization and communication. A PSD does not. There are two
> particulars of note. First: The OS/2 kernel has fallback code that
> knows how to talk to dual 8259s, should a PSD not implement certain
> optional capabilities, and operate what is essentially an *asymmetric*
> multiprocessor system. Second: An OS/2 PSD has no responsibility for
> implementing spinlocks. So a system where the PSD omits the optional
> features does not devolve to being identical to a uniprocessor OS/2 system.
>
> * The idea that I/O APICs increase the number of available interrupts is
> a bit of a swizz. The number of PIRQ signal lines on the PCI bus
> doesn't magically change. Some internal devices, built in to the


OK...so are there any tools currently in existence on our platform that we could
use to really understand the 'setup' that our BIOS has attempted to implement?

I found this interesting IOAPIC note
(http://www.o3one.org/tutorials/apicarticle.txt), it explains the functionality
in accessible terms.

Now, a question on my part as I've been trying to understand the options for the
OS2APIC.PSD driver...what exactly do the following mean (as best as you can
estimate):

/apic
/pic
/nmi
/prec
....and directly from a reference post:

There are three (3) possible entries that can describe a route:
route to the I/O APIC
route to the Local APIC
route described by argument(s) to the PSD

For a typical single user, multi core system, what are the possible
'combinations' of these options to use/try? I understand that this depends
heavily on the hardware itself, but is there an overall direction one should be
trying to pursue while setting up the system?

Thanks again!

Hallo,

since OS2APIC.PSD is pretty old I would think that it exclusively follows
Intel's multiprocessor specification Version 1.4 and completely ignores what
the ACPI multiprocessor table has to offer
(which eventually will replace the MP tables):

http://www.intel.com/design/pentium/datashts/242016.htm

Get that document to gain a better understanding. In particular it has some
nice diagrams
that will tell you more than a thousand words.

Then, switches for OS2APIC.PSD:

/apic: set up the system as described in chapter 3.6.2.3 in said spec
Please note that SMP for Warp Server (this is what we use today if
we have a
multiprocessor system) only routes interrupts to CPU1. OS/2 SMP
version 2.1
used to route interrupts to all CPUs. There were reasons to change
this
with SMP for Warp Server (avoid "parallel" paths in device
drivers).

/pic: set up the system as described in chapter 3.6.2.2 of said spec
Note: yes, the system runs in "virtual wire" mode as a "minimum".
It will never run as defined in chapter 3.6.2.1. That also means
OS2APIC.PSD will only work if you have a CPU with a local APIC.
Of course that is not a problem since the first generation Pentiums
came out.
You will need to run in this mode if your system DOES NOT contain
an IO-APIC.

/nmi: basically defines on what local APIC input pin the NMI signal line is
routed across.
You will only need this if the MP tables on your system are broken
or incomplete
If you look at what the MP tables contain, you will see that the
NMI signal line is
routed via the local APIC just as the interrupt lines are.
See chapter 4.3.5 of said spec. See table 4-11 for the types of
"interrupts" that are
routed across a local APIC.

/prec: as far as I understand if the NMI signal line is routed over more
than one local APIC
input, you can choose with this switch what the preferrable route
is. Again you
will only need this if the MP tables on your system are broken

Don't ask me about MSIs (message signalled interrupts).
Maybe it's the "messages" sent via the ICC-bus that the MP spec talks about
but I am not sure.


If I ever find the time, I could write a utility that dumps the MP tables
content (if you have Linux I would
guess that utility already exists). That would also help to find out if your
MP tables are broken or incomplete
(for example, if the NMI is not routed via any local APIC pin, then NMIs
will obviously not work).

But be aware that the MP tables will become obsolete with the multiprocessor
ACPI table that replaces them.
In other words: if your system no longer contains MP tables, then
OS2APIC.PSD will cease to work as it will
be looking for them.


Lars


"Dariusz Piatkowski" <dariusz@_NO-SPAM_mnsi.net> schrieb im Newsbeitrag
news:Qg5I6Bo2seGy-pn2-dR4CcEgq6dDB@neurobox...
> On Wed, 10 Aug 2011 14:51:07 UTC, Jonathan de Boyne Pollard
> <J.deBoynePollard-(E-Mail Removed)> wrote:
>
>> > maybe he can successfully shed some more light on this subject.
>>
>> Here are some more basics, just so that we're all on the same page:
>
>
> ...snip...snip...a ton of absolutely WEALTHY WONDERFUL info...thank you!!!
>
>
>> * An OS/2 Platform-Specific Driver (PSD) is not a Windows NT Hardware
>> Abstraction Layer (HAL). A HAL abstracts away quite a lot of the
>> details of interrupt processing and low-level inter-processor
>> synchronization and communication. A PSD does not. There are two
>> particulars of note. First: The OS/2 kernel has fallback code that
>> knows how to talk to dual 8259s, should a PSD not implement certain
>> optional capabilities, and operate what is essentially an *asymmetric*
>> multiprocessor system. Second: An OS/2 PSD has no responsibility for
>> implementing spinlocks. So a system where the PSD omits the optional
>> features does not devolve to being identical to a uniprocessor OS/2
>> system.
>>
>> * The idea that I/O APICs increase the number of available interrupts is
>> a bit of a swizz. The number of PIRQ signal lines on the PCI bus
>> doesn't magically change. Some internal devices, built in to the
>
>
> OK...so are there any tools currently in existence on our platform that we
> could
> use to really understand the 'setup' that our BIOS has attempted to
> implement?
>
> I found this interesting IOAPIC note
> (http://www.o3one.org/tutorials/apicarticle.txt), it explains the
> functionality
> in accessible terms.
>
> Now, a question on my part as I've been trying to understand the options
> for the
> OS2APIC.PSD driver...what exactly do the following mean (as best as you
> can
> estimate):
>
> /apic
> /pic
> /nmi
> /prec
> ...and directly from a reference post:
>
> There are three (3) possible entries that can describe a route:
> route to the I/O APIC
> route to the Local APIC
> route described by argument(s) to the PSD
>
> For a typical single user, multi core system, what are the possible
> 'combinations' of these options to use/try? I understand that this depends
> heavily on the hardware itself, but is there an overall direction one
> should be
> trying to pursue while setting up the system?
>
> Thanks again!

Jonathan de Boyne Pollard wrote:
> * The idea that I/O APICs increase the number of available interrupts is
> a bit of a swizz. The number of PIRQ signal lines on the PCI bus
> doesn't magically change. Some internal devices, built in to the
> southbridge, gain the ability to use extra interrupt signals that don't
> exist on the real PCI bus (On some VIA southbridges, for example, the
> internal PCI-to-ATA bridge gets to use PCI INT #E.). But, really,
> stating that the point of an I/O APIC is to "gain more interrupts" is
> mis-selling it. I/O APICs provide better ways to manage and control the
> *same* set of PCI and ISA interrupt signals, not more of them.

My motherboard manual shows the interrupt assignments. It shows 2 PCI
slots using INT #F and #G and the other one sharing #A IIRC.
Dave

> My motherboard manual shows the interrupt assignments. It shows 2 PCI
> slots using INT #F and #G and the other one sharing #A IIRC.

That's unusual. What chipset does it have?

Jonathan de Boyne Pollard wrote:
>> My motherboard manual shows the interrupt assignments. It shows 2 PCI
>> slots using INT #F and #G and the other one sharing #A IIRC.
>
> That's unusual. What chipset does it have?

Northbridge: Intel® 865G
Southbridge: Intel® ICH5

One PCI slot is shared with the on board audio on #B, The USB 2
controller and on board LAN share int #H. Nothing uses #E and no mention
is made of the on board Pata, Sata or what I guess are connected to the
ISA bus, mouse, keyboard, serial and parallel.
Dave

>> That's unusual. What chipset does it have?
>>
> Northbridge: Intel® 865G
>
> Southbridge: Intel® ICH5
>
Was it you who had me looking at the ICH7? The ICH7 is a little more
versatile, but also a little more restricted in its "mobile" flavour, in
that it only supports four PCI interrupt signals internally rather than
eight.

> One PCI slot is shared with the on board audio on #B, The USB 2
> controller and on board LAN share int #H. Nothing uses #E and no
> mention is made of the on board Pata, Sata or what I guess are
> connected to the ISA bus, mouse, keyboard, serial and parallel.
>

Your mainboard manual is misleading you somewhat. These aren't actual
PCI slots on your motherboard. All of these are internal devices that
exist on the ICH chip itself, at fixed PCI addresses.

What your mainboard manual should be telling you is how the PIRQ A, B,
C, and D signals from the actual PCI slots on the mainboard are wired up
to the PIRQ A, B, C, and D inputs to the ICH5 chip. This is
mainboard-specific. The idea is that every card uses the PIRQ A signal
for preference, and that a wiring matrix on the mainboard distributes
everyone's PIRQ A amongst PIRQs A, B, C, and D by the time that the
signal reaches the ICH chip. Mainboard manufacturers are free to choose
the matrix, and must lay out a table, the so-called "$PIR" table, in
firmware memory that describes it to software (so that software knows
what PCI interrupt a card that reports "I'm using PIRQ A!" is actually
using). The signals after they have been processed through the wiring
matrix are what the ICH5 treats as PIRQ A, B, C, and D, to which it adds
E, F, G, and H.

According to the ICH5 datasheet, the PCI interrupt signals used by the
built-in devices on the ICH itself are as follows:

PIRQ A: USB UHCI Controller #1, USB UHCI Controller #4
PIRQ B: AC ’97 , MC '97, SMbus
PIRQ C: USB UHCI Controller #3, PCI-to-ATA bridge in native mode
PIRQ D: USB UHCI Controller #2
PIRQ E: LAN
PIRQ F: (nothing)
PIRQ G: (nothing)
PIRQ H: USB EHCI Controller

SCI, TCO, and HPETs #0, #1, and #2 are all configurable between PCI IRQs
E, F, G, and H.

As usual for southbridges, these PCI interrupts are both routed directly
to I/O APIC INTIN signals #16 to #24 and routed indirectly, via a
programmable PCI IRQ router that controls what PCI IRQ maps to what INT
pin, to the (emulated) dual 8259s. As I said before, I/O APICs don't
give one more interrupts, as is so often claimed. As you can see, only
two devices (leaving aside the HPETs et al.) even use PIRQs #E to #H.
What using I/O APICs gives is (amongst other things) separation between
PCI and ISA interrupts, so that there's no risk of an edge-triggered
active-high ISA interrupt having to share a signal with a
level-triggered active-low PCI interrupt. They also free the systems
programmer from all of the PCI IRQ steering hooplah, which would be a
fourth layer of mapping and indirection in a mechanism that already has
three such layers for the systems programmer to deal with.

An i8042 keyboard and mouse, UART, parallel port, infrared port, MIDI
interface, environment controller, and 82078 floppy disc controller will
be connected to a separate "Super I/O" chip that is on the LPC bus. (If
you look at your mainboard you'll see it somewhere. It's usually a
WinBond or ITE chip. I'm interested in knowing which particular one you
have.) As such, they issue ISA interrupts, not PCI ones. The ISA
interrupts are, of course, routed in the traditional way to the
(emulated) dual 8259s, and are routed in a somewhat different
arrangement to the I/O APIC INTIN signals #0 to #15. This routing is
hardwired within the ICH5.

The interrupts issued by a PCI-to-ATA bridge are somewhat special. This
is because PCI-to-ATA bridges have two modes: "ISA compatibility" and
"PCI native". (The former makes a PCI-to-ATA bridge behave like an
ISA-to-ATA bridge. The latter makes it behave more like a proper PCI
device should, with reconfigurable base address registers, PCI INT pin
advertisements, and whatnot.) In ISA compatibility mode, the PCI-to-ATA
bridge always uses two interrupt signals, one per channel, and they are
both ISA interrupt signals: ISA IRQs #14 and #15. In PCI native mode,
the PCI-to-ATA bridge issues *one* interrupt signal, for *both*
channels, and it is a PCI interrupt.

Needless to say: That, coupled with the facts that each individual
channel on the bridge is theoretically individually switchable between
ISA compatibility and PCI native mode and that there's no "I'm the one
interrupting you!" flag in certain I/O transaction modes, makes
interrupt handling and Plug-and-Play resource management for Intel
PCI-to-ATA bridges somewhat ... erm ... interesting. (It's less
interesting for VIA PCI-to-ATA bridges, which have an additional "I'm
the one interrupting!" flag that's valid in all I/O modes. Although they
do add quirks of their own that, in compensation, make Plug-and-Play
resource management *more* interesting: In ISA compatibility mode the
ISA IRQs are not fixed as they are for Intel but configurable among ISA
IRQs #10, #11, #14, and #15. However, this uses the configuration
registers of a *different* PCI device, the PCI-to-ISA bridge. Using
someone else's configuration space isn't really in the spirit of PCI.)

> I found this interesting IOAPIC note
> (http://www.o3one.org/tutorials/apicarticle.txt), it explains the functionality
> in accessible terms.

It gets three things quite wrong, however:

* There is a means of querying the firmware for the mainboard's PCI
interrupt wiring matrix. It's the "$PIR" table.

* Many more than just "usually only [...] multiprocessor boards" have
I/O APICs. This is because an I/O APIC is built in to several
southbridges. A VIA VT8235M(CD) has a built-in I/O APIC, for example.
The mainboard manufacturer may well not have wired it up to the APIC
bus, but it's there.

* The swapping of INTIN #0 and INTIN #2 on the I/O APIC relative to IRQ
#0 and IRQ #2 at the dual 8259s isn't universal. It happens for some
southbridges that have built-in I/O APICs. But for southbridges that
use external I/O APIC chips the wiring is up to the mainboard
manufacturer, who can choose whatever wiring xe likes. How the I/O APIC
is wired up is something that has to be read out of the MPS or ACPI
tables, constructed by the mainboard manufacturer and supplied in
firmware. (I'm sure that M. Rieker discovered this when running xyr
program under Bochs. Bochs emulates one of the chipsets that requires
an external I/O APIC chip. It emulates such a chip being present, but
wires it up *without* swapping INTIN #0 and INTIN #2: a fact that system
software has to read the Bochs firmware-supplied MPS table to discover.)

> I've been trying to understand the options for the OS2APIC.PSD driver...

Don't go reading the copies of its documentation that exist on the WWW,
such as at the EDM/2 CONFIG.SYS Documentation Project for example. They
all seem to have a single source, that was mis-transcribed from the
original documentation. Where the WWW documents say

> The argument is "int" or "lint".

the original document actually said

> The argument is "int<n>" or "lint<n>".

which of course makes more sense when one reads what follows.

For a better description of OS2APIC.PSD's command-line arguments, see
IBM Technical Document #10322931. It has the advantage of being more up
to date than OS/2 version 2.11, as well. (-:

> For a typical single user, multi core system, what are the possible
> 'combinations' of these options to use/try?

The main use of the /NMI, /PREC, and /PIC options is to compensate for
faulty MPS tables. You've already seen that your firmware's SETUP
utility has the wrong help text attached to the "IOAPIC function"
option. This is not the only way that a mainboard manufacturer can
stuff up building a firmware image. A manufacturer can stuff up the MPS
tables, too.

It was common enough in years gone by that operating system vendors had
to implement workarounds like this. DOS+Windows 9x users wouldn't
notice that MPS tables were wrong. Not even most OS/2 users would.
Only a few people with Windows NT and other SMP operating systems would.
Now that operating systems that actually care about such things are
pretty much in the majority, the situation is somewhat different
(although the operating systems that care also tend to prefer ACPI
information to MPS).

Of course, if your MPS tables are *not* faulty, and correctly describe
how your system is wired up, you won't need to fiddle with these options
at all.

Jonathan de Boyne Pollard wrote:
>>> That's unusual. What chipset does it have?
>>>
>> Northbridge: Intel® 865G
>>
>> Southbridge: Intel® ICH5
>>
> Was it you who had me looking at the ICH7? The ICH7 is a little more
> versatile, but also a little more restricted in its "mobile" flavour, in
> that it only supports four PCI interrupt signals internally rather than
> eight.

Yes, I had a ICH7 based computer that unfortunately died


>
>> One PCI slot is shared with the on board audio on #B, The USB 2
>> controller and on board LAN share int #H. Nothing uses #E and no
>> mention is made of the on board Pata, Sata or what I guess are
>> connected to the ISA bus, mouse, keyboard, serial and parallel.
>>
>
> Your mainboard manual is misleading you somewhat. These aren't actual
> PCI slots on your motherboard. All of these are internal devices that
> exist on the ICH chip itself, at fixed PCI addresses.

Well, it claims to have 3 PCI buses, so perhaps slots 1 and 2 are
physically tied to #F and #G. The point of the manual was that only in
slot 3 did you have to worry about sharing interrupts.

>
> What your mainboard manual should be telling you is how the PIRQ A, B,
> C, and D signals from the actual PCI slots on the mainboard are wired up
> to the PIRQ A, B, C, and D inputs to the ICH5 chip. This is
> mainboard-specific. The idea is that every card uses the PIRQ A signal
> for preference, and that a wiring matrix on the mainboard distributes
> everyone's PIRQ A amongst PIRQs A, B, C, and D by the time that the
> signal reaches the ICH chip. Mainboard manufacturers are free to choose
> the matrix, and must lay out a table, the so-called "$PIR" table, in
> firmware memory that describes it to software (so that software knows
> what PCI interrupt a card that reports "I'm using PIRQ A!" is actually
> using). The signals after they have been processed through the wiring
> matrix are what the ICH5 treats as PIRQ A, B, C, and D, to which it adds
> E, F, G, and H.

The one PCI card I have installed is reported by pci.exe to use PCI
bus#2, IRQ#3, INT#A
[...]
Dave