Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 15 Jun 2007 23:40:07 -0400
From:      Kris Kennaway <kris@obsecurity.org>
To:        Benjamin Close <Benjamin.Close@clearchain.com>
Cc:        freebsd-current@freebsd.org, Kris Kennaway <kris@obsecurity.org>
Subject:   Re: Swapfile on ZFS & Deadlock
Message-ID:  <20070616034006.GA17514@rot13.obsecurity.org>
In-Reply-To: <46735745.5000401@clearchain.com>
References:  <4672945C.3060304@clearchain.com> <20070615182521.GB9619@rot13.obsecurity.org> <46735745.5000401@clearchain.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Jun 16, 2007 at 12:51:41PM +0930, Benjamin Close wrote:
> Kris Kennaway wrote:
> >On Fri, Jun 15, 2007 at 11:00:04PM +0930, Benjamin Close wrote:
> >  
> >>Hi All,
> >>   Whilst running out of memory compiling Xorg (scanPCI is a killer) I 
> >>discovered a quick way to deadlock the system:
> >>
> >>dd if=/dev/zero of=somefileonzfs bs=something count=something
> >>mdconfig -a -f something
> >>swapon /dev/md0
> >>
> >>Then do something that needs swap.. instant deadlock. The system is 
> >>still responsive but all disk access become hung.
> >>
> >>Known issue? If so is there a way we can warn users/prevent users from 
> >>doing it?
> >>    
> >
> >Enable DEBUG_VFS_LOCKS and DEBUG_LOCKS, then break to DDB when the
> >deadlock occurs and do 'show lockedvnods'.
> >  
> Ok, enabled the above and this time got:
> 
> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 312865, size: 16384
> 
> just before the deadlock:
> 
> Show locked vnods returns (hand transcribed)
> 
> 0xffffff002c3ab5d0: tagz zfs, type VREG
>    usecout 1, writecount 1, refcount 2 mountedhere 0
>    flags()
>    v_object 0xffffff002f047c80 ref 0 pages 0
>       lock type zfs: EXCL (count 1) by thread 0xffffff0030573360 (pid 1188)

What was needed was the backtrace that should have been also displayed
here.

> Pid 1188 is:
> 
> 1188   0   0   0   SL   zfs:(&zi   0xffffff000208fd58   [md0]

Also a trace of the current backtrace of this.  However based on the
single byte of information 'i' I predict that this is due to zil (ZFS
intent logging).  I have turned this off on my machines because of
deadlock conditions during low memory, which is also going to be true
on your system.  Try adding to /boot/loader.conf

vfs.zfs.zil_disable=1

> called doadump and though it went through the motions, savecore didn't 
> find anything saved.
> Not sure what you need debugging wise, let me know.
> 
> System is Intel Core 2 duo, running in SMP amd64, updated Friday 15th June.
> The box has 1G physical ram, 1G dedicated swap partition, but needs an 
> extra 300M to compile xf86ScanPCI.c (freaky!).

Or just use -O0 on this file.

Kris



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070616034006.GA17514>