board plug-in/add-on question (not quite sure how to describe it)

Just a random question that came to my mind reading about the FB-DIMM
thing.

With the added chip, it allows subsequent ram tech to use the same
interface and suppose to allow greater number of DIMM modules to be
used.

Some time ago, I remember reading here that having pieces of plug-in
ram is more difficult than having them soldered on due to some
"loading" issue so boards are generally limited in the number of slots
they have.

Does having a buffer chip means this loading issue goes away? Or is it
simply a case where the board still can't have more than say 4 slots
connected directly, but the chip presents effectively 1 slot to the
board and itself controls up to the max of 4 slots connected to it.
Hence with a limit of 4 slots, putting some kind of buffer chip into
each will allow 16 usable slots?

Would this also mean that with some kind of buffer chip, we could in
theory add unlimited ram capacities to the board, ignoring physical
constraints i.e.

board has 4 DIMM slots. Plug into each, a card with 4 DIMM slots and
an buffer chip, effectively creating 16 slots. Now plug such cards
into each of the expansion card slots, we get 64 slots. Or is it
impossible to do this due to the "loading" issue or something? This is
ignoring the physical impossibilities and assuming timings are
increased to accept the longer distance.

It's just a general question, not strictly FB-DIMM so pardon me if it
sounds weird. Just for curiousity's sake, thanks! :P

--
A Lost Angel, fallen from heaven
Lost in dreams, Lost in aspirations,
Lost to the world, Lost to myself

On Wed, 14 Dec 2005 03:16:14 GMT, a?n?g?e?l@lovergirl.lrigrevol.moc.com
(The little lost angel) wrote:

>Just a random question that came to my mind reading about the FB-DIMM
>thing.
>
>With the added chip, it allows subsequent ram tech to use the same
>interface and suppose to allow greater number of DIMM modules to be
>used.
>
>Some time ago, I remember reading here that having pieces of plug-in
>ram is more difficult than having them soldered on due to some
>"loading" issue so boards are generally limited in the number of slots
>they have.
>
>Does having a buffer chip means this loading issue goes away? Or is it
>simply a case where the board still can't have more than say 4 slots
>connected directly, but the chip presents effectively 1 slot to the
>board and itself controls up to the max of 4 slots connected to it.
>Hence with a limit of 4 slots, putting some kind of buffer chip into
>each will allow 16 usable slots?
>
>Would this also mean that with some kind of buffer chip, we could in
>theory add unlimited ram capacities to the board, ignoring physical
>constraints i.e.

The buffer chips in question, the AMBs, have a receiver and transmitter in
both the northbound and southbound directions so it would appear that no
external termination is needed for the signals - IOW in theory it would be
possible to extend the bus as far as you want *BUT* all downstream data
transfers pass through the upstream AMBs so the primary AMB is going to
pass every single bit of data - maybe why it needs a heatpipe for cooling.

Also, there is a variable latency mode, which I'm not sure is currently
viable/implemented, and without it you have to use the same programmed
latency for all DIMMs (remember DRDRAM ?) so the primary DIMM, next to the
chipset, has to be set to the same latency as the furthest one
downstream... so eventually the performance losses would get excessive.

>board has 4 DIMM slots. Plug into each, a card with 4 DIMM slots and
>an buffer chip, effectively creating 16 slots. Now plug such cards
>into each of the expansion card slots, we get 64 slots. Or is it
>impossible to do this due to the "loading" issue or something? This is
>ignoring the physical impossibilities and assuming timings are
>increased to accept the longer distance.

If that works it would seem like a way to mitigate the excessive latency
due to distance as a linear bus gets longer and longer with multiple DIMMs.
Whether it's possible I dunno.:-).. but if daytripper is reading, he'd be
the one to have the answer.

--
Rgds, George Macdonald

On Wed, 14 Dec 2005 08:53:21 -0500, George Macdonald
<fammacd=!SPAM^nothanks@tellurian.com> wrote:

>On Wed, 14 Dec 2005 03:16:14 GMT, a?n?g?e?l@lovergirl.lrigrevol.moc.com
>(The little lost angel) wrote:
>
>>Just a random question that came to my mind reading about the FB-DIMM
>>thing.
>>
>>With the added chip, it allows subsequent ram tech to use the same
>>interface and suppose to allow greater number of DIMM modules to be
>>used.
>>
>>Some time ago, I remember reading here that having pieces of plug-in
>>ram is more difficult than having them soldered on due to some
>>"loading" issue so boards are generally limited in the number of slots
>>they have.
>>
>>Does having a buffer chip means this loading issue goes away? Or is it
>>simply a case where the board still can't have more than say 4 slots
>>connected directly, but the chip presents effectively 1 slot to the
>>board and itself controls up to the max of 4 slots connected to it.
>>Hence with a limit of 4 slots, putting some kind of buffer chip into
>>each will allow 16 usable slots?
>>
>>Would this also mean that with some kind of buffer chip, we could in
>>theory add unlimited ram capacities to the board, ignoring physical
>>constraints i.e.
>
>The buffer chips in question, the AMBs, have a receiver and transmitter in
>both the northbound and southbound directions so it would appear that no
>external termination is needed for the signals - IOW in theory it would be
>possible to extend the bus as far as you want *BUT* all downstream data
>transfers pass through the upstream AMBs so the primary AMB is going to
>pass every single bit of data - maybe why it needs a heatpipe for cooling.
>
>Also, there is a variable latency mode, which I'm not sure is currently
>viable/implemented, and without it you have to use the same programmed
>latency for all DIMMs (remember DRDRAM ?) so the primary DIMM, next to the
>chipset, has to be set to the same latency as the furthest one
>downstream... so eventually the performance losses would get excessive.
>
>>board has 4 DIMM slots. Plug into each, a card with 4 DIMM slots and
>>an buffer chip, effectively creating 16 slots. Now plug such cards
>>into each of the expansion card slots, we get 64 slots. Or is it
>>impossible to do this due to the "loading" issue or something? This is
>>ignoring the physical impossibilities and assuming timings are
>>increased to accept the longer distance.
>
>If that works it would seem like a way to mitigate the excessive latency
>due to distance as a linear bus gets longer and longer with multiple DIMMs.
>Whether it's possible I dunno.:-).. but if daytripper is reading, he'd be
>the one to have the answer.

I think I understand his question, he's building a switched fbdimm memory
subsystem, instead of daisy-chaining, to beat the accumulated latency problem
(which, btw, Intel says "Is Not A Problem" - aka "Don't pay attention to that"
;-)

Yeah, you could probably do that, but you'd have to be implementing a really
exciting amount of memory to get much if any payback, and then there'd still
be conditions - and challenges. Assuming the SI and skew issues getting
through the active switch components and the extra connector layer(s) didn't
kill you dead....from a practical view, all the hard-wired shite inside the
only fbdimm-capable chipsets I've worked with to date (and at that, using
semi-functional AMBs - which, btw, will brown a burger better than a George
Foreman Grill can), would run out of capabilities way before any latency
improvement would be realized.

Not enough hardware to track all those fbdimms and DRAM banks, command fields
aren't wide enough, etc. Unless one had control of the host memory controller
design to broaden and deepen control structures, and was also willing to take
the platform cost hit to implement and amortize the switch components in a
near-commoditized marketplace for a likely-to-be-very-modest performance
improvement, I don't think this idea could gain traction...

Cheers

/daytripper

On Wed, 14 Dec 2005 16:44:28 -0500, daytripper
<day_trippr@REMOVEyahoo.com> wrote:

>I think I understand his question, he's building a switched fbdimm memory
>subsystem, instead of daisy-chaining, to beat the accumulated latency problem
>(which, btw, Intel says "Is Not A Problem" - aka "Don't pay attention to that"
>;-)

Actually having no engineering training or background, I was just
building a flight of curiousity as a result of having seen "memory
expanders" which are like cards with DIMM slots on them, reading that
the FB-DIMM buffer effectively will allow the same interface to the
chip to be used for different future memory types and the light bulb
suddenly burst on the thought of an infinitely upgradable motherboard
:P

>Yeah, you could probably do that, but you'd have to be implementing a really
>exciting amount of memory to get much if any payback, and then there'd still
>be conditions - and challenges. Assuming the SI and skew issues getting

What's SI in this context?

>through the active switch components and the extra connector layer(s) didn't
>kill you dead....from a practical view, all the hard-wired shite inside the
>only fbdimm-capable chipsets I've worked with to date (and at that, using
>semi-functional AMBs - which, btw, will brown a burger better than a George
>Foreman Grill can), would run out of capabilities way before any latency
>improvement would be realized.

What do you mean by ran of capabilities? Does it mean that it takes
way too much processing power to handle that many DIMMs?

>Not enough hardware to track all those fbdimms and DRAM banks, command fields
>aren't wide enough, etc. Unless one had control of the host memory controller
>design to broaden and deepen control structures, and was also willing to take

Why are all these needed? My non-skilled idea is that

For normal situations
board <-A-> buffer_Chip <-B-> DIMM

For expanded
board <-A-> buffer_Chip <-B-> expander_Chip/thing <-B-> DIMM

So the board doesn't care what happens AFTER <-A->, it will always use
<-A-> to talk to the "DIMM" whereas the buffer_Chip will always
provide a virtual "DIMM" of the same structure to the board. Something
like how the BIOS gives sector/cylinder/heads despite that it might be
addressing the drive through LBA kind of thing. The rest of the system
doesn't care what is actually there, they just see 4 DIMM from the
buffer_Chip. Then the buffer chip only sees a "DIMM" in the next step,
which is "emulated" by the next buffer_Chip.

Kinda like board wants to write data to location #1234567 and tells
the first buffer. Then it waits for the data and doesn't care if
#124567 is coming from an actual DIMM connected to the first buffer
chip or it's coming from a DIMM sitting behind a second buffer chip on
an expansion card as long as the first controller knows where to find
#1234567

>the platform cost hit to implement and amortize the switch components in a
>near-commoditized marketplace for a likely-to-be-very-modest performance
>improvement, I don't think this idea could gain traction...

Not that I thought it would, was just wondering about if something
like that could be done, if can why not done, if cannot, why not?

Thanks for the responses! :P

--
A Lost Angel, fallen from heaven
Lost in dreams, Lost in aspirations,
Lost to the world, Lost to myself