From owner-freebsd-current Fri May 25 20:44: 9 2001 Delivered-To: freebsd-current@freebsd.org Received: from thuvia.demon.co.uk (thuvia.demon.co.uk [193.237.34.248]) by hub.freebsd.org (Postfix) with ESMTP id 5B61637B42C for ; Fri, 25 May 2001 20:43:45 -0700 (PDT) (envelope-from mark@thuvia.demon.co.uk) Received: (from mark@localhost) by thuvia.demon.co.uk (8.11.3/8.11.3) id f4Q3gSO03181 for freebsd-current@freebsd.org; Sat, 26 May 2001 04:42:28 +0100 (BST) (envelope-from mark) Date: Sat, 26 May 2001 04:42:28 +0100 (BST) From: Mark Valentine Message-Id: <200105260342.f4Q3gSO03181@thuvia.demon.co.uk> X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: freebsd-current@freebsd.org Subject: vnode pager panic with latest sources Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I started getting a panic like this immediately after bootup around the same time as everyone else's problems (although my last build was from March 1st, and I've only just started using INVARIANTS/WITNESS and ``makeoptions DEBUG=-g'' which were recently aded to GENERIC). The latest commits don't seem to have changed anything. During boot (interleaved with IPv6 stuff), I get this: lock order reversal 1st 0xc04a7380 mntvnode @ /usr/src/sys/ufs/ffs/ffs_vsops.c:1007 2nd 0xcd105aec vnode interlock @ /usr/src/sys/ufs/ffs/ffs_vfsops.c:106 Just around the time of the console login prompt (consistently), I panic: [warning: likely to contain hand transcription errors] -- BEGIN ---------------------------------------------------------------------- recursed on non-recursive lock (sleep mutex) vm @ /usr/src/sys/ufs/ufs/ufs_readwrite.c:420 first acquired @ /usr/src/sys/vm/vnode_pager.c:912 panic: recurse Debugger("panic") Stopped at Debugger+0x44: pushl %ebx db> show reg cs 0x8 ds 0xc0200010 getitimer+0xa0 es 0xcd0f0010 fs 0xc0460018 w_data+0x1418 ss 0x10 eax 0x12 ecx 0x20 edx 0xc00b8f00 ebx 0x202 esp 0xcd0fac1c ebp 0xcd0fac28 esi 0x100 edi 0xc0482f1c eip 0xc036a8dc Debugger+0x44 Debugger+0x44: pushl %ebx db> trace Debugger(c03b9b7b) at Debugger+0x44 panic(c03bc8c8,cd0fad0c,cd0fad0c,1,0) at panic+0x70 witness_lock(c04b3960,8,c03d7bc0.1a4) at witness_lock+0x348 ffs_write(cd0facf8) at ffs_write+0xba vnode_pager_generic_putpages(cd1196c0,cd0fadf4,1000,0,cd0fadc8) at vnode_pager_generic_putpages+0x198 vop_stdputpages(cd0fad7c,cd0fad60,c0331ab9,cd0fad7c,cd0fad9c) at vop_stdputpages+0x1a vop_defaultop(cd0fad7c,cd0fad9c,c034f35d,cd0fad7c,0) at vop_defaultop+0x15 ufs_vnoperate(cd0fad7c) at ufs_vnoperate+0x15 vnode_pager_putpages(cd120240,cd0fadf4,1,0,cd0fadc8,c04b3960,1,c03db941,91) at vnode_pager_putpages+0x1a9 vm_pageout_flush(cd0fadf4,1,0,c0c36404,c0466bc0) at vm_pageout_flush+0x12a vm_object_page_clean(cd120240,0,0,4) at vm_object_page_clean+0x430 vfa_msync(c125a600,2,cb67a43c,cb67a320,0) at vfs_msync+0x235 sync(cb67a320,cd0faf80,4,bfbffdcc,bfbffdcc) at sync+0xf4 syscall(2f,2f,2f,bfbffdcc,bfbffdcc) at syscall+0x689 syscall_with_err_pushed() at syscall_with_err_pushed+0x1b -- END ------------------------------------------------------------------------ src/sys/vm/vnode_pager.c is at revision 1.130 My kernel config file: -- BEGIN ---------------------------------------------------------------------- machine i386 cpu I686_CPU ident DOTAR maxusers 32 makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols options NO_F00F_HACK options INET #InterNETworking options INET6 #IPv6 communications protocols options IPSEC #IP security options IPSEC_ESP #IP security (crypto; define w/ IPSEC) options MROUTING # Multicast routing options NETGRAPH #netgraph(4) system options IPFIREWALL #firewall options IPFIREWALL_VERBOSE #print information about # dropped packets options IPFIREWALL_DEFAULT_TO_ACCEPT #allow everything by default options IPV6FIREWALL #firewall for IPv6 options IPV6FIREWALL_VERBOSE options IPV6FIREWALL_DEFAULT_TO_ACCEPT options IPDIVERT #divert sockets options FFS #Berkeley Fast Filesystem options MFS #Memory Filesystem options MD_ROOT #MD is a potential root device options NFS #Network Filesystem options NFS_ROOT #NFS usable as root device, NFS required options MSDOSFS #MSDOS Filesystem options CD9660 #ISO 9660 Filesystem options PROCFS #Process filesystem options SOFTUPDATES options COMPAT_43 #Compatible with BSD 4.3 [KEEP THIS!] options SCSI_DELAY=15000 #Delay (in ms) before probing SCSI options AUTO_EOI_1 options UCONSOLE #Allow users to grab the console options VESA options USERCONFIG #boot -c editor options VISUAL_USERCONFIG #visual boot -c editor options KTRACE #ktrace(1) support options PERFMON options SYSVSHM #SYSV-style shared memory options SYSVMSG #SYSV-style message queues options SYSVSEM #SYSV-style semaphores options DDB options INVARIANTS options INVARIANT_SUPPORT options WITNESS options P1003_1B options _KPOSIX_PRIORITY_SCHEDULING device isa device pci device fdc device ata device atadisk # ATA disk drives options ATA_STATIC_ID #Static dev number like old `wd' driver device ahc # AHA2940 and onboard AIC7xxx devices device sym # NCR/Symbios Logic (newer chipsets + those of `ncr') options AHC_ALLOW_MEMIO device scbus # SCSI bus (required) device da # Direct Access (disks) device sa # Sequential Access (tape etc) device cd # CD device pass # Passthrough device (direct SCSI access) device atkbdc 1 # At keyboard controller device atkbd # at keyboard device psm # psm mouse device vga # VGA screen device splash device sc 1 device npx device apm device pmtimer device pcm device midi device seq device pca device sio # 8250, 16[45]50 based serial ports device ppc device ppbus # Parallel port bus (required) device lpt # Printer device plip # TCP/IP over parallel device ppi # Parallel port interface device device joy device miibus # MII bus support device fxp # Intel EtherExpress PRO/100B (82557, 82558) device random # Entropy device device loop # Network loopback device ether # Ethernet support device disc # Discard device device sl 3 # Kernel SLIP device ppp 3 # Kernel PPP device tun # Packet tunnel. device gif 4 # IPv6 and IPv4 tunneling device faith 1 # IPv6-to-IPv4 relaying (translation) device stf # 6to4 IPv6 over IPv4 encapsulation device pty # Pseudo-ttys (telnet etc) device snp 3 # Snoop device - to look at pty/vty/etc.. device md # Memory "disks" device speaker # Play IBM BASIC-style noises out your speaker device splash device bpf #Berkeley packet filter options PPP_BSDCOMP #PPP BSD-compress support options PPP_DEFLATE #PPP zlib/deflate/gzip support options PPP_FILTER #enable bpf filtering (needs bpf) options COMPAT_LINUX -- END ------------------------------------------------------------------------ dmesg output from my working kernel (March 1st): -- BEGIN ---------------------------------------------------------------------- Copyright (c) 1992-2001 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 5.0-CURRENT #1: Thu Mar 1 02:04:02 GMT 2001 root@dotar-sojat:/usr/obj/usr/src/sys/DOTAR Timecounter "i8254" frequency 1193182 Hz Timecounter "TSC" frequency 601365847 Hz CPU: Pentium III/Pentium III Xeon/Celeron (601.37-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x681 Stepping = 1 Features=0x387f9ff real memory = 268419072 (262128K bytes) avail memory = 256643072 (250628K bytes) Preloaded elf kernel "kernel" at 0xc048c000. WARNING: size of kinfo_proc (648) should be 644!!! Pentium Pro MTRR support enabled VESA: v2.0, 32768k memory, flags:0x1, mode table:0xc00c6974 (c0006974) VESA: Matrox Graphics Inc. Using $PIR table, 8 entries at 0xc00f0e70 npx0: on motherboard npx0: INT 16 interface pcib0: at pcibus 0 on motherboard pci0: on pcib0 pcib1: at device 1.0 on pci0 pci1: on pcib1 pci1: at 0.0 (no driver attached) isab0: at device 4.0 on pci0 isa0: on isab0 atapci0: port 0xd800-0xd80f at device 4.1 on pci0 ata0: at 0x1f0 irq 14 on atapci0 pci0: at 4.2 (no driver attached) pci0: at 4.3 (no driver attached) csa0: mem 0xdf800000-0xdf8fffff,0xe0000000-0xe0000fff irq 9 at device 9.0 on pci0 pcm0: on csa0 sym0: <896> port 0xd000-0xd0ff mem 0xde800000-0xde801fff,0xdf000000-0xdf0003ff irq 10 at device 11.0 on pci0 sym0: Symbios NVRAM, ID 7, Fast-40, LVD, parity checking sym0: open drain IRQ line driver, using on-chip SRAM sym0: using LOAD/STORE-based firmware. sym0: handling phase mismatch from SCRIPTS. sym1: <896> port 0xb800-0xb8ff mem 0xdd800000-0xdd801fff,0xde000000-0xde0003ff irq 9 at device 11.1 on pci0 sym1: Symbios NVRAM, ID 7, Fast-40, SE, parity checking sym1: open drain IRQ line driver, using on-chip SRAM sym1: using LOAD/STORE-based firmware. sym1: handling phase mismatch from SCRIPTS. pci0: at 13.0 (no driver attached) atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 psm0: irq 12 on atkbdc0 psm0: model Generic PS/2 mouse, device ID 0 fdc0: at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 fdc0: FIFO enabled, 8 bytes threshold fd0: <1440-KB 3.5" drive> on fdc0 drive 0 joy0 at port 0x201 on isa0 pca0 at port 0x40 on isa0 pmtimer0 on isa0 ppc0: at port 0x378-0x37f irq 7 flags 0x40 on isa0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/9 bytes threshold ppi0: on ppbus0 plip0: on ppbus0 lpt0: on ppbus0 lpt0: Interrupt-driven port sc0: on isa0 sc0: VGA <16 virtual consoles, flags=0x200> sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 16550A sio1 at port 0x2f8-0x2ff irq 3 on isa0 sio1: type 16550A sio2 at port 0x3e8-0x3ef irq 5 on isa0 sio2: type 16550A vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 sc1: on isa0 sc1: MDA <16 virtual consoles, flags=0x0> WARNING: Driver mistake: repeat make_dev("consolectl") vga1: at port 0x3b0-0x3bb iomem 0xb0000-0xb7fff on isa0 unknown: can't assign resources unknown: can't assign resources unknown: can't assign resources unknown: can't assign resources unknown: can't assign resources unknown: can't assign resources pca1: at port 0x61 on isa0 WARNING: Driver mistake: repeat make_dev("pcaudio") WARNING: Driver mistake: repeat make_dev("pcaudioctl") IPv6 packet filtering initialized, default to accept, unlimited logging IP packet filtering initialized, divert enabled, rule-based forwarding disabled, default to accept, unlimited logging IPsec: Initialized Security Association Processing. ad0: 73308MB [148945/16/63] at ata0-master UDMA33 Waiting 15 seconds for SCSI devices to settle (noperiph:sym0:0:-1:-1): SCSI BUS reset delivered. (noperiph:sym1:0:-1:-1): SCSI BUS reset delivered. sa0 at sym1 bus 0 target 4 lun 0 sa0: Removable Sequential Access SCSI-CCS device sa0: 3.300MB/s transfers Mounting root from ufs:/dev/da0a da0 at sym0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-3 device da0: 80.000MB/s transfers (40.000MHz, offset 31, 16bit), Tagged Queueing Enabled da0: 17501MB (35843670 512 byte sectors: 255H 63S/T 2231C) da1 at sym0 bus 0 target 1 lun 0 da1: Fixed Direct Access SCSI-3 device da1: 80.000MB/s transfers (40.000MHz, offset 31, 16bit), Tagged Queueing Enabled da1: 17501MB (35843670 512 byte sectors: 255H 63S/T 2231C) cd0 at sym1 bus 0 target 2 lun 0 cd0: < > Fixed CD-ROM SCSI-0 device cd0: 3.300MB/s transfers cd0: Attempt to query device size failed: HARDWARE FAILURE, Ram failure cd1 at sym1 bus 0 target 3 lun 0 cd1: Removable CD-ROM SCSI-2 device cd1: 20.000MB/s transfers (20.000MHz, offset 8) cd1: Attempt to query device size failed: NOT READY, Medium not present - tray closed da2 at sym1 bus 0 target 0 lun 0 da2: Removable Direct Access SCSI-2 device da2: 10.000MB/s transfers (10.000MHz, offset 15) da2: Attempt to query device size failed: NOT READY, Medium not present -- END ------------------------------------------------------------------------ What changes should I start to make first to narrow this down? (I'll try to get kernel dumps going, though I haven't had much success with these in the past.) Cheers, Mark. -- Mark Valentine, Thuvia Labs "Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses "We're kind of stupid that way." *munch* *munch* and endorses FreeBSD -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message