Date: Wed, 31 Dec 2008 08:32:15 -0600 (CST) From: Mark Tinguely <tinguely@casselton.net> To: gjb@semihalf.com, xcllnt@mac.com Cc: arm@freebsd.org, embedded@freebsd.org Subject: Re: Multiple virtual mappings considered harmful on ARM Message-ID: <200812311432.mBVEWF6O047499@casselton.net> In-Reply-To: <495B5531.8040604@semihalf.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> Marcel Moolenaar wrote: > > > > On Dec 19, 2008, at 6:07 AM, Grzegorz Bernacki wrote: > > > >> 2. Root cause. > >> The root cause of the problem is additional virtual mapping of read/write > >> buffers at cluster read/write (sys/kern/vfs_cluster.c, cluster_rbuild(), > >> cluster_wbuild(). Buffers for sequential read/write operation are > >> concatenated > >> and sent to device as one big buffer. Concatenation of buffers uses > >> pmap_qenter(), which puts *additional* mapping in the KVA for physical > >> area > >> already mapped. For each buffer we extract pages it contains and then > >> all the > >> pages from all the buffers are mapped into new virtual address of new > >> buffer. > >> So we end up with at least two virtual addresses for each page. > > > > Could this also affect I-cache coherency by virtue of not > > flushing the D-cache properly before synchronizing the > > I-cache, as you mention reading? > > > > I am not sure. I can't think of scenario which might lead to I-cache incoherency. > Have you experienced any issues with I-cache which might be related described problem? > > pozdrawiam, > Grzesiek If would be surprised for you to see an I-cache problems. pmap_qenter() writes back any managed cache pages (already under the control of a pv_entry) and then calls pmap_kenter_internal(), which tries to writeback and invalid any previously mapped pages. At the end of the pmap_qenter() the caches are clean. The problem is the new mapping is added with cache turned on, and the old mapping's page table caching is not modified either, so if either side modifies the page, there can be this caching problem. I think that the pmap_kenter_internal() should become a special kind of managed page (controlled by a pv_entry and a special flag, (I called PVF_UNMAN) that runs through the pmap_fix_cache(), so the mappings for the shared page should have the cache turned off. There are special checks The less obvious question is should PG_UNMANAGED pages be simularly managed? Or put another way, could the PG_UNMANAGED page be remapped by the I/O caching programs? My guts says yes. Doing so also forces BUS_DMA_COHERENT pages to turn off the caches on the share page. I have a crude workup of my idea, but I have to admit, I don't have the equipment to even try to compile it. There concerns about pv_entry use increase, there is lock issues, performance, and even could we cause a problem trying get a pv_entry at the wrong time, such as interrupt. Maybe the kernel pv_entrys need to come from a special pool. I have some crude ideas to flush the caches less often in the ARM11's virtual index / physical tag caches and ideas for the multicore physical index / physical tag cache. --Mark Tinguely
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200812311432.mBVEWF6O047499>