Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 02 Feb 2004 12:45:18 +0100
From:      Andreas Braukmann <braukmann@tse-online.de>
To:        obrien@freebsd.org
Cc:        amd64@freebsd.org
Subject:   Re: Dual processor, AMD 64 machine freezing.
Message-ID:  <2147483647.1075725918@[192.168.111.140]>
In-Reply-To: <20040202005326.GB60117@dragon.nuxi.com>
References:  <B1D77424948FD611A3B80000C0109EEF023B4D3D@SYNCRO> <20040202005326.GB60117@dragon.nuxi.com>

next in thread | previous in thread | raw e-mail | index | archive | help
--On Sonntag, 1. Februar 2004 16:53 Uhr -0800 David O'Brien <obrien@freebsd.org> wrote:

> On Mon, Feb 02, 2004 at 12:51:34AM +0100, Andreas Braukmann wrote:
>> > P.S. For best performance, I think you really want to run 4x 512 MB.
>> > Running with two DIMMs means either you only get 64-bit memory access (not
>> > 128-bit) or else you need to put both DIMMs into the CPU1 memory slots
>> > (which means CPU2 will have to access those through hypertransport).
>>
>> That's theory. ;-)
>> Since the allocators don't know about the numa-like architecture
>> memory would be accessed through hyptertransport (statistically)
>> more or less "half of the time". (CPU0 ---> HT ---> MEM1 ;
>> CPU1 ---> HT ---> MEM0)
>
> Its not theory, its fact -- even w/o a NUMA aware OS.  Statistically, 1/2
> the accesses by a CPU are to local memory, 1/2 to distant memory.  If you
> put all them memory on a single CPU then you've got two processors trying
> to access memory, saturating the memory controller on the single CPU with
> memory -- thus giving you less BW.

You are right, of course.
I have two nearly identical systems. One is equipped with two 1 GByte
Dimms both connected to CPU0, the second is equipped with four 512 MByte
Dimms (2 at CPU0 and 2 at CPU1). My "real world" benchmarks (no micro-
benchmarking involved) showed, that the system with memory on both CPUs
performs minimal better (<< 5%) than the system having its complete RAM
on one CPU.

Choosing a configuration that allows to make use of both memory controller
channels (even using only one CPU's controller) pays of significantly
(20% - 30%).

Especially for the K8S (having only 4 + 2 DIMM-slots) I would rather
buy 2 * 1 GByte Dimms (to be able to simply add two more).

>  Your diagram above leaves out the  memory controller (and its request buffer).

Point taken. My remark was over simplified and not to the point.

-Andreas





Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2147483647.1075725918>