Date: Fri, 03 Jun 2005 17:53:18 -0400 From: Stephan Uphoff <ups@tree.com> To: Matthew Dillon <dillon@apollo.backplane.com> Cc: freebsd-hackers@freebsd.org Subject: Re: Possible instruction pipelining problem between HT's on the same die ? Message-ID: <1117835598.27369.12036.camel@palm> In-Reply-To: <200506032057.j53KvOFw062012@apollo.backplane.com> References: <200506032057.j53KvOFw062012@apollo.backplane.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 2005-06-03 at 16:57, Matthew Dillon wrote: > I've been tracking down a crash one of our users gets occassionally. > He has a quad Intel(R) XEON(TM) CPU 2.00GHz (1996.61-MHz 686-class CPU) > system. > > After getting a few of these crashes he pulled three of the four cpus > out. But with just one physical cpu, with HTT turned on (so two > logical cpus), he is still getting these crashes. > > This is the sequence that causes the bad data: > > cpu #0 write A > write B > > (HT)cpu #1 read B > if (B) > read A <---- gets OLD data in A, not new data > > Now I was depending on the presumed write ordering, so if a foreign > cpu sees that B is updated it can assume that A has also been updated. > > But I'm beginning to think that it isn't working as advertised. I've > read the manuals over and over again and they seem to only guarentee > write ordering between physical cpus, not between logical HT cpus, and > even then it appears that a cpu can do a speculative read and > thus get an old value for A even after getting a new value for B. > > I looked at the various SFENCE/LFENCE/MFENCE instructions and they > do not seem to guarentee ordering for speculative accesses at all. > They all say that they do not protect against speculative reads. > Bus-locked instructions don't seem to avoid speculative reads either. > > I'm even more confused because this bug is occuring between two logical > cpus on the same physical die. Is write ordering not guarenteed with > respect to the other logical cpu? Can one logical cpu prefetch data > early then then becomes obsolete by the time the instruction is actually > run? Or perhaps its a pipeline bug... I just don't know. But it's > damn annoying. > > The only solution I see is to use an actual serializing instruction > like cpuid. I really do not want to have to use cpuid :-(. > > So, has anyone seen anything similar? This is normal behaviour. Take a look at IA-32 Intel Developers ... Vol 3, Section: 7.2.2 for details + solutions. Stephan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1117835598.27369.12036.camel>