From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 6 04:28:05 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25DB210656D5 for ; Mon, 6 Sep 2010 04:28:05 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from argol.doit.wisc.edu (argol.doit.wisc.edu [144.92.197.212]) by mx1.freebsd.org (Postfix) with ESMTP id EBF268FC1E for ; Mon, 6 Sep 2010 04:28:04 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from avs-daemon.smtpauth3.wiscmail.wisc.edu by smtpauth3.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) id <0L8B00M005QSCX00@smtpauth3.wiscmail.wisc.edu>; Sun, 05 Sep 2010 23:28:04 -0500 (CDT) Received: from comporellon.tachypleus.net ([unknown] [76.210.68.10]) by smtpauth3.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) with ESMTPSA id <0L8B00JRP5QQU410@smtpauth3.wiscmail.wisc.edu>; Sun, 05 Sep 2010 23:28:03 -0500 (CDT) Date: Sun, 05 Sep 2010 23:28:02 -0500 From: Nathan Whitehorn In-reply-to: To: mdf@FreeBSD.org Message-id: <4C846DD2.4000507@freebsd.org> X-Spam-Report: AuthenticatedSender=yes, SenderIP=76.210.68.10 X-Spam-PmxInfo: Server=avs-10, Version=5.6.0.2009776, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2010.9.6.41815, SenderIP=76.210.68.10 X-Enigmail-Version: 1.0.1 References: <4C844609.9050505@freebsd.org> User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.11) Gecko/20100729 Thunderbird/3.0.6 Cc: freebsd-hackers@freebsd.org Subject: Re: UMA allocations from a specific physical range X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Sep 2010 04:28:05 -0000 On 09/05/10 22:51, mdf@FreeBSD.org wrote: > On Mon, Sep 6, 2010 at 1:38 AM, Nathan Whitehorn wrote: > >> PowerPC hypervisors typically provided a restricted range on memory when >> the MMU is disabled, as it is when initially handling exceptions. In >> order to restore virtual memory, the powerpc64 code needs to read a data >> structure called the SLB cache, which is currently allocated out of a >> UMA zone, and must be mapped into wired memory, ideally 1:1 >> physical->virtual address. Since this must be accessible in real mode, >> it must have a physical address in a certain range. I am trying to >> figure out the best way to do this. >> >> My first run at this code uses a custom UMA allocator that calls >> vm_phys_alloc_contig() to get a memory page. The trouble I have run into >> is that I cannot figure out a way to free the page. Marking the zone >> NOFREE is a bad solution, vm_page_free() panics the kernel due to >> inconsistent tracking of page wiring, and vm_phys_free_pages() causes >> panics in vm_page_alloc() later on ("page is not free"). What is the >> correct way to deallocate these pages? Or is there a different approach >> I should adopt? >> > I assume this is for the SLB flih? > > What AIX did was to have a 1-1 simple esid to vsid translation for > kernel addresses, reserve the first 16 SLB entries for various uses, > including one for the current process's process private segment, and > if the slb miss was on a process address we'd turn on translation and > look up the answer, the tables holding the answer being in the process > private segment effective address space so we wouldn't take another > slb miss. This required one level deep recursion in the slb slih, in > case there was a miss on kernel data with xlate on in the SLB slih. > Yes, that's correct. FreeBSD has the same 1-to-1 translation for the kernel, but the entire address space is switched out for user processes (no part of the kernel is mapped into user processes), so the code to load the user SLB entries has to be able to execute with the MMU off, lest it disappear underneath itself. > For historical reasons due to the per-process segment table for > POWER3, we also had a one-page hashed lookup table per process that we > stored the real address of in the process private segment, so the > assembly code in the flih looked here before turning on MSR_DR IIRC. > I was trying to find ways to kill this code when I left IBM, since > we'd ended support for POWER3 a few years earlier. > > I haven't had the time to look at FreeBSD ppc64 sources; how large are > the uma-allocated slb entries and what is stored in them? The struct > and filename is sufficient, though I don't have convenient access to > sources until Tuesday. > The entries are each 1 KB, and there is one for each pmap. Each consists of 64 16-byte SLBE/SLBV pairs. These buffers are just a carbon copy of what should be in the SLB after a context switch to that map. > V=R space is rather limited (well, depending on a lot of factors; for > AIX on Power5 and later the hypervisor only gave us 128M, though for > ppc64 on a Mac G4 I assume all of memory can be mapped V=R if desired) > so it was best to find a non V=R solution if possible. Turning on > translation in the flih after some setup and recursion stopping is one > of the easier ways, and also has the advantage of not needing to > either have separate code or macro access to data structures used in > both V and R modes. > On the PS3 (the target in this case), the hypervisor also limits us to 128 MB. The one and only kernel data structure that needs to be used in this mode is this SLB cache object, so I was hoping for a simple solution to just put them all in the real-mode accessible region. -Nathan