Date: Sun, 13 Dec 2009 09:25:32 -0600 From: Robert Noland <rnoland@FreeBSD.org> To: Chris H <chris#@1command.com> Cc: freebsd-stable@freebsd.org Subject: Re: MTRR failure revisited (nVidia) 8-STABLE/RELEASE Message-ID: <1260717932.2281.73.camel@balrog.2hip.net> In-Reply-To: <3a3d4fb449dab5da292f47d4de1b6b58.HRCIM@webmail.1command.com> References: <d498c3f0038a2415580a54898d78a4cd.HRCIM@webmail.1command.com> <1260628594.2281.30.camel@balrog.2hip.net> <7f052646787bb8c47f847ecd3176e706.HRCIM@webmail.1command.com> <1260713097.2281.61.camel@balrog.2hip.net> <3a3d4fb449dab5da292f47d4de1b6b58.HRCIM@webmail.1command.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, 2009-12-13 at 06:53 -0800, Chris H wrote: > On Sun, December 13, 2009 6:04 am, Robert Noland wrote: > > On Sat, 2009-12-12 at 20:08 -0800, Chris H wrote: > > > >> On Sat, December 12, 2009 6:36 am, Robert Noland wrote: > >> > >>> On Sat, 2009-12-12 at 03:47 -0800, Chris H wrote: > >>> > >>> > >>>> Greetings, > >>>> I brought this same error to the list back in May 2009. > >>>> Under: failed to set mtrr: invalid argument. > >>>> Well, I'm back using the same card: > >>>> GeForce4 MX 440-SE - VideoRam 65536 - BusID PCI:1:3:0. > >>>> The driver is different, I'm using: nvidia-driver-96.43.13 out of ports on > >>>> a custom 8-STABLE kernel. Xorg starts up, and produces a desktop. But it's > >>>> "dog > >>>> slow" > >> UPDATE: > >> Disabling HAL /greatly/ increased performance > >> eg; hal_enable="YES" --> hal_enable="NO" in /etc/rc.conf More specifically, > >> response times are now closer to what one would anticipate/ expect now that > >> HAL has been dis-abled in rc.conf. > >> > >>>> , and the nvidia driver emits the following error: NVIDIA: failed to set > >>>> MTRR 0xf0000000, 0M (write-combining) > >>>> several times. I understand John Baldwin provided some "invaluable" help > >>>> some time ago: > >>>> http://lists.freebsd.org/pipermail/freebsd-hackers/2006-June/016995.html > >>>> and I was wondering if anyone has gained any further "insight" with these > >>>> cards, and how to better "interface" them in BSD. Last I spoke on the > >>>> topic, I was informed that the memory was basically "untouchable" - or > >>>> perhaps in other words; can't be manipulated. Has this changed? Surely > >>>> someone else has had to deal with this besides me. It seems crazy to spend > >>>> a "boat load" of $$ on these high performers, and not be able to use them > >>>> on a high performing OS - no? :) Sure, the one I'm working with now is > >>>> "legacy". But I have 3 near new, > >>>> top of their line cards, and thus far it appears that if I ever hope to > >>>> use them, I'll be forced to... hack, choke.. spin up a WIN CD. :( > >>>> > >>>> Thank you for all your time, consideration, and insight. > >>>> > >>>> > >> Greetings Robert, and thank you for taking the time to respond. > >> > >>> > >>> The mtrr issue has to do with the system / bios, not the graphics card. > >>> While I've not used the blob driver, the issue in Nouveau and other drm > >>> drivers is that on many systems if you run "memcontrol list", you will see a > >>> line something like this: > >>> > >>> 0x0/0x100000000 BIOS write-back set-by-firmware active > >>> > >> I see the following (condensed for brevity): > >> 0x0/0x10000 BIOS write-back fixed-base fixed-length set-by-firmware active > >> 0x10000/0x10000-0x70000/0x10000 BIOS write-back fixed-base fixed-length > >> set-by-firmware active 0x80000/0x4000 BIOS-0x9c000/0x4000 write-back fixed-base > >> fixed-length set-by-firmware active 0xa0000/0x4000-0xbc000/0x4000 BIOS > >> uncacheable fixed-base fixed-length set-by-firmware active > >> 0xc0000/0x10000xc7000/0x1000 BIOS write-protect fixed-base fixed-length > >> set-by-firmware active 0xc8000/0x1000-0xff000/0x1000 BIOS uncacheable > >> fixed-base fixed-length set-by-firmware active 0x0/0x40000000 BIOS write-back > >> set-by-firmware active > Hello Robert, and thank you for your thoughtful response. > > > > The above entry is the one that causes setting write-combine MTRR to > > fail. > > > >> 0xe0000000/0x20000000 BIOS uncacheable set-by-firmware active > >> > >> > >> While I could pull the BIOS out of it's socket after POST. I don't suppose > >> I could read it's contents to file, and then allow manipulation of the > >> regions currently "off limits"? > > > > This is more easily achieved in our MTRR code I expect, certainly than > > BIOS hacking all of the effected machines. Frankly, all of my more > > modern machines have this issue. > > > Ahhh, OK. > >>> > >>> This says that all of memory defaults to write-back. We aren't allowed > >>> to overlap write-combined on top of write-back, so the setting of mtrr > >>> fails. > >> Isn't it /best/ to choose write-back, so as to mark the memory dirty? I > >> /could/ > >> choose write-ahead, or write-through. > >>> I've looked at ways to try to fix this in the past, but > >>> generally found it more practical to use PAT than try to override/fix bios > >>> behavior. > >> Marius Nünnerich also mentioned this in a response to this thread. Would you > >> be willing to share any additional information, based on your experiences > >> using PAT? > > > > PAT and MTRR both allow for the setting of cache attributes for a range > > of memory. For MTRR, there are a certain number of variable range registers in a > > given CPU which allow the setting of global cache attributes for a given memory > > range. There are a number of rules about what memory types may overlap though, > > so having that entry that covers all of memory means that setting MTRR to > > anything other than uncached will always fail. > > > > PAT is less restrictive in this regard and a table exists in the intel > > documentation showing the effective cache method for various combinations of > > MTRR/PAT. The difficulty with PAT is that it is an > > attribute of the individual page mapping and so all mappings of the same physical > > region of memory need to have the same PAT type set. jhb@, alc@ and I (my > > contribution has mostly consisted of whining and testing) have recently (over > > the last year at least) been adding the ability to handle allowing userspace > > mappings of memory regions to be mapped with consistent PAT attributes. This > > was also one of the key features that nvidia wanted before they would deliver an > > amd64 blob driver. Much of the needed infrastructure is in place now, however > > there is still some work to be done on bus_dma to make use of the features. I > > have some local hacks to bus_dma that I am using in my trees to allow me to more > > easily manipulate the PAT attributes, but they aren't practical to commit. We > > need to revisit this topic and decide what is the correct way to proceed. > If I may be so bold... can we anticipate this in 8? Or are we looking at 9? > One last thing, it would appear that you use the Nouveau driver as your preferred > nVidia driver. Overall, would you say this one has more advantages than the blob > drivers? Well, it isn't without issues... My current nouveau patch will get you EXA and Xv acceleration on i386 and amd64. Until about a week or so ago, it was the only solution for amd64. The nouveau project has removed the fake buffer object support that we use in the current patch in favor of TTM, which is preventing me from updating the code right now. TTM is one of two drm memory managers, the other being GEM. I'm just not inclined to use a closed source driver, when every other vendor is actively providing code and docs. ATI/AMD is currently the most friendly when it comes to supporting open drivers on anything besides linux. Intel does provide both docs and code, however they continue to frustrate me by ripping out code that supported the traditional drm in favor of all of their new linux code. VIA is also providing docs and some code, but they don't quite have the hang of working with the drm/Xorg crowd yet, so much of the code that they have published hasn't gone anywhere. robert. > Thank you again for your thoughtful, and informative response. > > --Chris H > > > > > robert. > > > >>> I've been told that linux does apply some BIOS fixups to address this > >>> issue, which I might look into again, but I make no promises. > >> Is there anything I could do that would help you in this regard? > >> There are also a > >> > >>> very limited number of variable mtrr registers (7 on most hardware, iirc) > >>> for managing caching. > >> > >> Thank you again for taking the time to respond. > >> > >> > >> --Chris H > >> > >>> > >>> robert. > >>> > >>>> --Chris H > >>>> > >>>> > >>>> > >>>> > >>>> _______________________________________________ > >>>> freebsd-stable@freebsd.org mailing list > >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable > >>>> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > >>>> > >>>> > >>> -- > >>> Robert Noland <rnoland@FreeBSD.org> > >>> FreeBSD > >>> > >>> > >>> > >>> _______________________________________________ > >>> freebsd-stable@freebsd.org mailing list > >>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable > >>> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > >>> > >>> > >>> > >> > >> > > -- > > Robert Noland <rnoland@FreeBSD.org> > > FreeBSD > > > > > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > > > > > > -- Robert Noland <rnoland@FreeBSD.org> FreeBSD
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1260717932.2281.73.camel>