Date: Mon, 2 Jan 2006 20:57:12 +0100 (CET) From: "Lars Kristiansen" <lars+lister.freebsd@adventuras.no> To: freebsd-stable@freebsd.org Subject: Re: Swapfile problem in 6? Message-ID: <50410.80.111.250.79.1136231832.squirrel@mail.adventuras.no> In-Reply-To: <200601021114.k02BEgOQ004377@gw.catspoiler.org> References: <20051125032845.73799.qmail@web36208.mail.mud.yahoo.com> <200601021114.k02BEgOQ004377@gw.catspoiler.org>
next in thread | previous in thread | raw e-mail | index | archive | help
> Attempting to catch up with my backlog of unread email, only 12K unread > messages to go ... > > On 24 Nov, Rob wrote: > >> I have cvsup'ed the sources to STABLE as of Nov. 23rd >> 2005. >> After recompiling/installing world and debug-kernel, >> I again get a kernel deadlock when using swapfile: >> http://surfion.snu.ac.kr/~lahaye/swapfile2.txt >> >> Previous deadlocks are still documented here >> http://surfion.snu.ac.kr/~lahaye/swapfile.txt >> >> I hope this is of use for fixing this bug in 6. >> If further investigation is needed, then please let me >> know. > > This is a deadlock caused by memory exhaustion. The pagedaemon only has > a limited number of bufs that it uses for writing dirty pages to swap to > prevent it from saturating the I/O subsystem with large numbers of > writes. In this case, pagedaemon is trying to free up memory by writing > dirty pages, and it has used up all of its bufs and is waiting for the > write requests to complete and the bufs the bufs to be returned to it. > This isn't happening because md0 is stuck waiting for memory. This is a > little bit suprising to me because it looks like writes to vnode backed > devices are done synchronously by default. > > If you have a chance to test this again, a stack trace of md0 in the > deadlock state would be interesting. I'd like to know where md0 is > getting stuck. > > I wonder if pagedaemon should scan ahead and more agressively discard > clean pages when it has run out of bufs to write dirty pages, especially > in low memory situations. Preventing the creation of more dirty pages > would be nice, but I don't know how to do that ... Just in case it can help. Do not have this machine available for testing at the moment but this is the last debuginfo I did get from it. Here is a trace from a situation when a possible idle system got stuck during the night and db showed only one locked vnode: db> show lockedvnods Locked vnodes 0xc1309330: tag ufs, type VREG usecount 1, writecount 1, refcount 154 mountedhere 0 flags () v_object 0xc12cb39c ref 0 pages 606 lock type ufs: EXCL (count 1) by thread 0xc126b900 (pid 178) ino 8155, on dev ad0s1f db> trace 178 Tracing pid 178 tid 100058 td 0xc126b900 sched_switch(c126b900,0,1) at 0xc066a4db = sched_switch+0x17b mi_switch(1,0) at 0xc065f49e = mi_switch+0x27e sleepq_switch(c09b2a98,c484bacc,c065f0e3,c09b2a98,0) at 0xc0677f00 = sleepq_switch+0xe0 sleepq_wait(c09b2a98,0,0,c08ad92d,37b) at 0xc0678100 = sleepq_wait+0x30 msleep(c09b2a98,c09b2d00,244,c08adb6a,0) at 0xc065f0e3 = msleep+0x333 vm_wait(c12cb39c,0,c08990f3,ad7,c06512a4) at 0xc07c6a71 = vm_wait+0x91 allocbuf(c28fa9d8,4000,354000,0,354000) at 0xc06a2f89 = allocbuf+0x4e9 getblk(c1309330,d5,0,4000,0) at 0xc06a29cb = getblk+0x4eb cluster_read(c1309330,10000000,0,d5,0) at 0xc06a5d65 = cluster_read+0xe5 ffs_read(c484bc9c) at 0xc07a631f = ffs_read+0x28f VOP_READ_APV(c09309a0,c484bc9c) at 0xc0838aab = VOP_READ_APV+0x7b mdstart_vnode(c1310800,c1634294,c1310820,1,c0566e10) at 0xc056688c = mdstart_vnode+0xec md_kthread(c1310800,c484bd38,c1310800,c0566e10,0) at 0xc0566f7f = md_kthread+0x16f fork_exit(c0566e10,c1310800,c484bd38) at 0xc0645618 = fork_exit+0xa8 fork_trampoline() at 0xc0816f3c = fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xc484bd6c, ebp = 0 --- -- Lars
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?50410.80.111.250.79.1136231832.squirrel>