Date: Tue, 01 Jul 2008 02:15:21 +0200 From: Kris Kennaway <kris@FreeBSD.org> To: alex@schnarff.com Cc: questions@freebsd.org Subject: Re: Too Much Context Switching? - FIXED Message-ID: <48697719.40101@FreeBSD.org> In-Reply-To: <20080630200456.uf01ro1obms40cok@mail.schnarff.com> References: <20080630165205.GA3033@lpthe.jussieu.fr> <48691D7C.2090804@FreeBSD.org> <20080630181755.GA3327@lpthe.jussieu.fr> <48692DE7.3020502@FreeBSD.org> <20080630192154.nj1sns26kg44w4w8@mail.schnarff.com> <48696EB0.6000906@FreeBSD.org> <20080630200456.uf01ro1obms40cok@mail.schnarff.com>
next in thread | previous in thread | raw e-mail | index | archive | help
alex@schnarff.com wrote: > Quoting Kris Kennaway <kris@FreeBSD.org>: > >> alex@schnarff.com wrote: >>> Quoting Kris Kennaway <kris@FreeBSD.org>: >>> >>>> Michel Talon wrote: >>>>> On Mon, Jun 30, 2008 at 07:53:00PM +0200, Kris Kennaway wrote: >>>>>> Yep, it could be that -- what confuses me though is that it is >>>>>> claimed that performance suddenly regressed. If so then this >>>>>> cannot be the underlying cause. >>>>> >>>>> It may be that the load has augmented to the point that contention >>>>> imposes a rapid regression on throughput. >>>> >>>> Yes, it could be that. I don't know off-hand whether multiple threads >>>> are counted separately by vmstat (at a guess I'd say no), but >>>> ps/top/etc >>>> should show how many are active in the python process. >>> >>> Just ran ktrace, and a bit of Googling seems to confirm my initial >>> suspicion that the results I'm seeing are abnormal. The first several >>> screenfulls of output look like this: >>> >>> 52929 python2.4 1214867016.469416 CALL kse_wakeup(0x811740c) >>> 52929 python2.4 0.000060 RET kse_wakeup 0 >>> 52929 python2.4 0.000008 RET kse_release 0 >>> 52929 python2.4 0.000040 CALL kse_release(0x811df4c) >>> 52929 python2.4 0.000515 CALL kse_wakeup(0x811740c) >>> 52929 python2.4 0.000012 RET kse_wakeup 0 >>> 52929 python2.4 0.000004 RET kse_release 0 >>> 52929 python2.4 0.000012 CALL kse_release(0x811df4c) >>> 52929 python2.4 0.000365 CALL kse_wakeup(0x811740c) >>> 52929 python2.4 0.000012 RET kse_wakeup 0 >>> 52929 python2.4 0.000003 RET kse_release 0 >>> 52929 python2.4 0.000010 CALL kse_release(0x811df4c) >>> 52929 python2.4 0.000413 CALL kse_wakeup(0x811740c) >>> 52929 python2.4 0.000011 RET kse_wakeup 0 >>> 52929 python2.4 0.000004 RET kse_release 0 >>> 52929 python2.4 0.000009 CALL kse_release(0x811df4c) >>> 52929 python2.4 0.000393 CALL kse_wakeup(0x811740c) >>> 52929 python2.4 0.000012 RET kse_wakeup 0 >>> 52929 python2.4 0.000004 RET kse_release 0 >>> 52929 python2.4 0.000009 CALL kse_release(0x811df4c) >>> >>> I may be mistaken, but it seems like that's a lot of unnecessary >>> activity managing the threads; the confirmation I found came from >>> http://arkiv.freebsd.se/?ml=freebsd-threads&a=2007-02&t=3178634. >>> >>> Am I correct that this is abnormal behavior? If so, any idea what I >>> may need to do to fix the issue? >> >> Looks exactly like the python thread problem Michel described. >> >> You will get some improvement by switching to libthr and/or updating to >> 7.0 as I discussed, but ultimately you're hitting limits of python, not >> FreeBSD. > > WOW...it's *amazing* how much of a difference a single sysctl can make. > > I went ahead and set kern.threads.virtual_cpu=1, as suggested in the > thread above, and the difference is ridiculous -- Zope is now faster > than I've ever seen. More importantly, my ktracing shows that all of the > kse_* garabage is now gone. > > I'll probably be upgrading to 7.0 in the next month or so, given that > this is obviously a thread issue and that that release has much improved > thread code. However, for the time being, the pressing issue is fixed, > and for anyone in my position stuck on 6.2...this is night & day. Seriously, try libthr. No matter what you do to libkse it is going to suck. That's why we removed it. Kris
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?48697719.40101>