Date: Sat, 29 Nov 2014 00:36:40 +0100 From: Svatopluk Kraus <onwahe@gmail.com> To: Adrian Chadd <adrian@freebsd.org> Cc: "freebsd-arm@freebsd.org" <freebsd-arm@freebsd.org>, Ian Lepore <ian@freebsd.org> Subject: Re: Another Test Run with Alternative pmap Implementation Message-ID: <CAFHCsPUhAmc4TVhr9-j-xdV-LvWcXeHZEACHg2YKSZ=rVPtZqw@mail.gmail.com> In-Reply-To: <CAJ-Vmondax_%2BqGgb_DcdpDjbJvmpHACFiLRh7rLur-%2BfM0Eu2A@mail.gmail.com> References: <20141113125236.b16cd4e5f0e339eac0494cd4@ulrich-grey.de> <C6FED1A5-490C-47BE-B071-484271ED370E@me.com> <20141115143444.5ad037548e06f289d2532fb7@ulrich-grey.de> <CAFHCsPUJ1HhLqAjitPg6mPzhMYSui64Xmu4omO7Pkp%2B0kPZnAA@mail.gmail.com> <20141119225903.81fbbc7809093a0e6e0de9d5@ulrich-grey.de> <CAFHCsPXnSFY_X-O73M%2Bh0xO_XJ0cTmkRwtu-o4omPndnfbEhmg@mail.gmail.com> <20141120151900.a68c6d8316b96a62cb65d17a@ulrich-grey.de> <CAFHCsPWTnU7j0MC7YSHFFDE97%2B%2BBrnkJKGnK9zkxVGemaa6nAw@mail.gmail.com> <20141121115941.54d4e36b103341c3adf7eb36@ulrich-grey.de> <20141124132733.4e96b906f0d1ab69969dddd9@ulrich-grey.de> <1416840814.1147.380.camel@revolution.hippie.lan> <20141125225451.924a5df4bdb4753db273b8c5@ulrich-grey.de> <CAFHCsPXPEN3U%2B0=AtAJ4dL5g7jGuyW6=u%2B-tbHf3xH1QdJYyhQ@mail.gmail.com> <20141126125806.78f2df97328e807d12746ae3@ulrich-grey.de> <519fde5db60e4fc594956a600c6cad4e@e15be-01.zdv.Uni-Mainz.DE> <1417108193.1055.2.camel@revolution.hippie.lan> <CAFHCsPWL94b5JFL87__MxGfGFrA4q0c3QgPwZf7MGzJ9i6hsaQ@mail.gmail.com> <CAJ-Vmok=yN4EqMucLCz=ZkoUv8pGnp8FHTzLo%2BTuXFE2_K1CMg@mail.gmail.com> <CAFHCsPWg=A-DvCqm0if13z=GM3EK5gbgC1Dww_90hQBFbixW9w@mail.gmail.com> <CAJ-Vmondax_%2BqGgb_DcdpDjbJvmpHACFiLRh7rLur-%2BfM0Eu2A@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Nov 28, 2014 at 11:17 PM, Adrian Chadd <adrian@freebsd.org> wrote: > On 28 November 2014 at 11:49, Svatopluk Kraus <onwahe@gmail.com> wrote: > > Thanks for explanation. In this case, however, wrapping by sched_pin() > and > > sched_unpin() calls would be enough. Nevertheless, as our pmap is based > on > > i386 one, I asked Alan Cox for the meaning of this warning and created > patch > > according to his answer. > > > > More about PCPU stuff could be found in [1]. ;) > > As long as nothing else can preempt that thread on that CPU that'll > mess with your PCPU data, everything is fine. > > Doing sched_pin() just prevents your thread from migrating to another > core whilst it's running; it doesn't prevent something else at a > higher priority from preempting you and also trying to get access to > the same PCPU data. > > (I hit this stuff when playing with the flowtable code which makes > heavy use of pcpu data..) > > > -a > > Generally, it's correct. However, I said in this case. And in this case, it's about curpmap. And curpmap is always same from curthread. So only thing which must be ensured is that curpmap is read from correct PCPU. In other words, curthread should not migrate between reading of cpu number and getting curpmap from associated PCPU. Thus wrapping it by sched_pin() and sched_unpin() is enough. Svata > > > > Svata > > > > [1] > > > http://lists.freebsd.org/pipermail/freebsd-current/2013-February/039858.h= tml > > > > > > > > On Fri, Nov 28, 2014 at 6:50 PM, Adrian Chadd <adrian@freebsd.org> > wrote: > >> > >> Hi, > >> > >> PCPU_GET()ed things aren't atomic. Unless you're in a critical > >> section, you can be pre-empted at any time and migrated to another > >> CPU. There aren't explicit "migration points" that kernel code can be > >> migrated - it can be pre-empted and migrated to other CPUs whenever > >> something else at a higher priority comes along. > >> > >> So if you're not wrapping your PCPU_GET() and subsequent work inside a > >> critical section - if you're using the PCPU_GET()'ed data outside of > >> the critical section, then you're in for a world of trouble. > >> > >> > >> > >> -adrian > >> > >> > >> On 28 November 2014 at 01:31, Svatopluk Kraus <onwahe@gmail.com> wrote= : > >> > I think that the pmap_remove_page warning is very likely due to not > >> > atomic > >> > PCPU_GET(). Can you please try attached patch. > >> > > >> > Svata > >> > > >> > > >> > > >> > On Thu, Nov 27, 2014 at 6:09 PM, Ian Lepore <ian@freebsd.org> wrote: > >> > > >> >> On Wed, 2014-11-26 at 22:18 +0000, Wei=C3=9F, Dr. J=C3=BCrgen wrote= : > >> >> > I made a testrun with the updated source tree and the patches for > >> >> > the jetson tk1 platform. With > >> >> > > >> >> > options ARM_NEW_PMAP > >> >> > options DEBUG > >> >> > options DIAGNOSTIC > >> >> > options INVARIANTS # Enable calls of extra > >> >> > sanity > >> >> checking > >> >> > options INVARIANT_SUPPORT # Extra sanity checks of > >> >> internal structures, required by INVARIAN > >> >> > > >> >> > and no special sysctl settings. > >> >> > > >> >> > A make -j6 buildworld finishes successfully after 2h15m. There is > >> >> > one kernel message > >> >> > kernel: warning: pmap_remove_pages called with non-current pmap > >> >> > > >> >> > /usr/src and /usr/obj over nfs, /tmp on tmpfs > >> >> > > >> >> > Regards > >> >> > >> >> That's similar to my results. I changed to -j20 to see if that wou= ld > >> >> recreate the problems that Ulrich is seeing, but buildworld runs fi= ne > >> >> for me, in about 2 hours. I've never seen the non-current pmap > warning > >> >> on the system that uses a usb ssd drive as root, but I've seen it > with > >> >> nfs root. > >> >> > >> >> BTW, the DIAGNOSTIC option adds a LOT of performance overhead to an > arm > >> >> system without adding a lot of value. I usually leave it off, > >> >> sometimes > >> >> turn it on when I encounter a problem to see if it generates more > info > >> >> (usually it doesn't). > >> >> > >> >> -- Ian > >> >> > >> >> > >> >> > >> > > >> > _______________________________________________ > >> > freebsd-arm@freebsd.org mailing list > >> > http://lists.freebsd.org/mailman/listinfo/freebsd-arm > >> > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.or= g > " > > > > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFHCsPUhAmc4TVhr9-j-xdV-LvWcXeHZEACHg2YKSZ=rVPtZqw>