Date: Tue, 30 Mar 1999 11:09:42 -0600 (CST) From: bob@pmr.com To: FreeBSD-gnats-submit@freebsd.org Subject: kern/10872: Panic in soreceive() in 3.1-stable running amanda Message-ID: <199903301709.LAA33066@luke.pmr.com>
next in thread | raw e-mail | index | archive | help
>Number: 10872
>Category: kern
>Synopsis: Panic in sorecieve() due to NULL mbuf pointer
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Tue Mar 30 09:20:00 PST 1999
>Closed-Date:
>Last-Modified:
>Originator: Bob Willcox
>Release: FreeBSD 3.1-STABLE i386
>Organization:
Power Micro Research
>Environment:
FreeBSD deathstar.pmr.com 3.1-STABLE FreeBSD 3.1-STABLE #4: Tue Mar 30 08:59:32 CST 1999 bob@deathstar.pmr.com:/usr/src/sys/compile/DEATHSTAR i386
>Description:
A panic occurs on this system during my nightly amanda backups (this is
my amanda backup server). The panic is the result of the sb_mb pointer
being NULL in soreceive when loaded into m at line 642 in uipc_socket.c.
At the time of the panic amanda is loading the system pretty well with
5 dumps running (from 5 different systems on the network) and writing to
the Mammoth tape drive.
Note that this problem suddenly started happening (last Friday morning).
Prior to that I had not changed this system (deathstar) for several
weeks, though the client systems had changed (I don't have a precise
record of those changes). I have since changed deathstar (upgraded to
more recent 3.1-stable and modified the kernel configuration) in a (so
far) futile attempt to work-arround the problem.
Some (hopefully helpful) info from the crash dump:
#0 boot (howto=260) at ../../kern/kern_shutdown.c:285
285 dumppcb.pcb_cr3 = rcr3();
(kgdb) where
#0 boot (howto=260) at ../../kern/kern_shutdown.c:285
#1 0xf014e705 in panic (fmt=0xf0233f4c "from debugger")
at ../../kern/kern_shutdown.c:446
#2 0xf012aab1 in db_panic (addr=-266261713, have_addr=0, count=-1,
modif=0xf4224d5c "") at ../../ddb/db_command.c:432
#3 0xf012aa51 in db_command (last_cmdp=0xf0251e64, cmd_table=0xf0251cc4,
aux_cmd_tablep=0xf0267acc) at ../../ddb/db_command.c:332
#4 0xf012ab16 in db_command_loop () at ../../ddb/db_command.c:454
#5 0xf012ce67 in db_trap (type=3, code=0) at ../../ddb/db_trap.c:71
#6 0xf021290a in kdb_trap (type=3, code=0, regs=0xf4224e4c)
at ../../i386/i386/db_interface.c:157
#7 0xf021c0b4 in trap (frame={tf_es = 16, tf_ds = 16, tf_edi = -202329632,
tf_esi = 256, tf_ebp = -199078256, tf_isp = -199078284,
tf_ebx = -266105266, tf_edx = -266043248, tf_ecx = -267680032,
tf_eax = 18, tf_trapno = 3, tf_err = 0, tf_eip = -266261713, tf_cs = 8,
tf_eflags = 598, tf_esp = -266043264, tf_ss = -266111117})
at ../../i386/i386/trap.c:548
#8 0xf0212b2f in Debugger (msg=0xf0237773 "panic")
at ../../i386/i386/db_interface.c:317
#9 0xf014e6fc in panic (fmt=0xf0238e4e "receive 1")
at ../../kern/kern_shutdown.c:444
#10 0xf01667d3 in soreceive (so=0xf3f0b1e0, psa=0x0, uio=0xf4224f40, mp0=0x0,
controlp=0x0, flagsp=0x0) at ../../kern/uipc_socket.c:659
#11 0xf015c6d4 in soo_read (fp=0xf1026540, uio=0xf4224f40, cred=0xf0f2a180)
at ../../kern/sys_socket.c:69
#12 0xf01591ed in read (p=0xf418f3c0, uap=0xf4224f94)
at ../../kern/sys_generic.c:121
#13 0xf021c8c3 in syscall (frame={tf_es = -272695257, tf_ds = -272695257,
tf_edi = -272638492, tf_esi = 64, tf_ebp = -272638364,
tf_isp = -199077916, tf_ebx = 0, tf_edx = 82768, tf_ecx = 6, tf_eax = 3,
tf_trapno = 7, tf_err = 7, tf_eip = 537674705, tf_cs = 31,
tf_eflags = 514, tf_esp = -272638820, tf_ss = 39})
at ../../i386/i386/trap.c:1100
#14 0x200c43d1 in ?? ()
#15 0x1f64 in ?? ()
#16 0x1099 in ?? ()
(kgdb) up 10
#10 0xf01667d3 in soreceive (so=0xf3f0b1e0, psa=0x0, uio=0xf4224f40, mp0=0x0,
controlp=0x0, flagsp=0x0) at ../../kern/uipc_socket.c:659
Source file is more recent than executable.
659 KASSERT(m != 0 || !so->so_rcv.sb_cc, ("receive 1"));
(kgdb) list
654 if (m == 0 || (((flags & MSG_DONTWAIT) == 0 &&
655 so->so_rcv.sb_cc < uio->uio_resid) &&
656 (so->so_rcv.sb_cc < so->so_rcv.sb_lowat ||
657 ((flags & MSG_WAITALL) && uio->uio_resid <= so->so_rcv.sb_hiwat)) &&
658 m->m_nextpkt == 0 && (pr->pr_flags & PR_ATOMIC) == 0)) {
659 KASSERT(m != 0 || !so->so_rcv.sb_cc, ("receive 1"));
660 if (so->so_error) {
661 if (m)
662 goto dontblock;
663 error = so->so_error;
(kgdb) print *so
$1 = {so_zone = 0xf0f0ef00, so_type = 1, so_options = 0, so_linger = 0,
so_state = 2, so_pcb = 0xf400bea0 "", so_proto = 0xf0259294, so_head = 0x0,
so_incomp = {tqh_first = 0x0, tqh_last = 0xf3f0b1f8}, so_comp = {
tqh_first = 0x0, tqh_last = 0xf3f0b200}, so_list = {tqe_next = 0x0,
tqe_prev = 0x0}, so_qlen = 0, so_incqlen = 0, so_qlimit = 0, so_timeo = 0,
so_error = 0, so_sigio = 0x0, so_oobmark = 0, so_rcv = {sb_cc = 4380,
sb_hiwat = 17520, sb_mbcnt = 6528, sb_mbmax = 140160, sb_lowat = 1,
sb_mb = 0x0, sb_sel = {si_pid = 0, si_flags = 0}, sb_flags = 1,
sb_timeo = 0}, so_snd = {sb_cc = 0, sb_hiwat = 17520, sb_mbcnt = 0,
sb_mbmax = 140160, sb_lowat = 2048, sb_mb = 0x0, sb_sel = {si_pid = 0,
si_flags = 0}, sb_flags = 0, sb_timeo = 0}, so_upcall = 0,
so_upcallarg = 0x0, so_uid = 90, so_gencnt = 3716}
(kgdb) print m
$2 = (struct mbuf *) 0x0
(kgdb) print *uio
$3 = {uio_iov = 0xf4224f38, uio_iovcnt = 1, uio_offset = 0xffffffffffffffff,
uio_resid = 820, uio_segflg = UIO_USERSPACE, uio_rw = UIO_READ,
uio_procp = 0xf418f3c0}
Dmesg output:
Copyright (c) 1992-1999 FreeBSD Inc.
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
FreeBSD 3.1-STABLE #4: Tue Mar 30 08:59:32 CST 1999
bob@deathstar.pmr.com:/usr/src/sys/compile/DEATHSTAR
Timecounter "i8254" frequency 1193182 Hz
Timecounter "TSC" frequency 199309847 Hz
CPU: Pentium Pro (199.31-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x616 Stepping=6
Features=0xf9ff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV>
real memory = 33554432 (32768K bytes)
avail memory = 29958144 (29256K bytes)
Preloaded elf kernel "kernel" at 0xf02cd000.
Probing for devices on PCI bus 0:
chip0: <Intel 82440FX (Natoma) PCI and memory controller> rev 0x02 on pci0.0.0
chip1: <Intel 82371SB PCI to ISA bridge> rev 0x01 on pci0.1.0
ahc0: <Adaptec 2940 SCSI adapter> rev 0x00 int a irq 12 on pci0.10.0
ahc0: aic7870 Single Channel A, SCSI Id=7, 16/255 SCBs
fxp0: <Intel EtherExpress Pro 10/100B Ethernet> rev 0x01 int a irq 10 on pci0.11.0
fxp0: Ethernet address 00:a0:c9:31:e6:21
ncr0: <ncr 53c810 fast10 scsi> rev 0x01 int a irq 11 on pci0.12.0
ncr1: <ncr 53c875 fast20 wide scsi> rev 0x03 int a irq 9 on pci0.13.0
Probing for devices on the ISA bus:
sc0 on isa
sc0: VGA color <16 virtual consoles, flags=0x0>
atkbdc0 at 0x60-0x6f on motherboard
atkbd0 irq 1 on isa
psm0 not found
sio0 at 0x3f8-0x3ff irq 4 flags 0x10 on isa
sio0: type 16550A
sio1 at 0x2f8-0x2ff irq 3 on isa
sio1: type 16550A
fdc0 at 0x3f0-0x3f7 irq 6 drq 2 on isa
fdc0: FIFO enabled, 8 bytes threshold
fd0: 1.44MB 3.5in
ppc0 at 0x378 irq 7 on isa
ppc0: W83877F chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/16 bytes threshold
nlpt0: <generic printer> on ppbus 0
nlpt0: Interrupt-driven port
ppi0: <generic parallel i/o> on ppbus 0
plip0: <PLIP network interface> on ppbus 0
vga0 at 0x3b0-0x3df maddr 0xa0000 msize 131072 on isa
npx0 on motherboard
npx0: INT 16 interface
Waiting 10 seconds for SCSI devices to settle
sa0 at ahc0 bus 0 target 1 lun 0
sa0: <EXABYTE EXB-89008E000204 V38b> Removable Sequential Access SCSI-2 device
sa0: 10.000MB/s transfers (10.000MHz, offset 15)
sa1 at ncr0 bus 0 target 5 lun 0
sa1: <WANGTEK 51000 SCSI 75F2> Removable Sequential Access SCSI-2 device
sa1: 4.807MB/s transfers (4.807MHz, offset 8)
changing root device to da0s1a
cd0 at ncr0 bus 0 target 4 lun 0
cd0: <TOSHIBA CD-ROM XM-3401TA 0283> Removable CD-ROM SCSI-2 device
cd0: 4.237MB/s transfers (4.237MHz, offset 8)
cd0: Attempt to query device size failed: NOT READY, Medium not present
da1 at ncr1 bus 0 target 1 lun 0
da1: <IBM DCAS-34330W S65A> Fixed Direct Access SCSI-2 device
da1: 40.000MB/s transfers (20.000MHz, offset 15, 16bit)
da1: 4134MB (8467200 512 byte sectors: 255H 63S/T 527C)
da2 at ncr1 bus 0 target 2 lun 0
da2: <IBM DDRS-39130D DC1B> Fixed Direct Access SCSI-2 device
da2: 40.000MB/s transfers (20.000MHz, offset 15, 16bit), Tagged Queueing Enabled
da2: 8715MB (17850000 512 byte sectors: 255H 63S/T 1111C)
da0 at ncr1 bus 0 target 0 lun 0
da0: < DFRSS2W 4B4B> Fixed Direct Access SCSI-2 device
da0: 20.000MB/s transfers (10.000MHz, offset 15, 16bit), Tagged Queueing Enabled
da0: 2150MB (4404489 512 byte sectors: 255H 63S/T 274C)
ch0 at ahc0 bus 0 target 0 lun 0
ch0: <EXABYTE EXB-210 5.00> Removable Changer SCSI-2 device
ch0: 3.300MB/s transfers
ch0: 11 slots, 1 drive, 1 picker, 0 portals
WARNING: / was not properly dismounted
ffs_mountfs: superblock updated for soft updates
ffs_mountfs: superblock updated for soft updates
ffs_mountfs: superblock updated for soft updates
ffs_mountfs: superblock updated for soft updates
ffs_mountfs: superblock updated for soft updates
ffs_mountfs: superblock updated for soft updates
ffs_mountfs: superblock updated for soft updates
link_elf: symbol splash_register undefined
Kernel config file:
#
# DEATHSTAR -- Configure file of the DEATHSTAR system
#
# For more information read the handbook part System Administration ->
# Configuring the FreeBSD Kernel -> The Configuration File.
# The handbook is available in /usr/share/doc/handbook or online as
# latest version from the FreeBSD World Wide Web server
# <URL:http://www.FreeBSD.ORG/>;
#
# An exhaustive list of options and more detailed explanations of the
# device lines is present in the ./LINT configuration file. If you are
# in doubt as to the purpose or necessity of a line, check first in LINT.
#
# $Id$
machine "i386"
cpu "I686_CPU"
ident DEATHSTAR
maxusers 40
options INET #InterNETworking
options FFS #Berkeley Fast Filesystem
options FFS_ROOT #FFS usable as root device [keep this!]
options MFS #Memory Filesystem
options NFS #Network Filesystem
options MSDOSFS #MSDOS Filesystem
options "CD9660" #ISO 9660 Filesystem
options "CD9660_ROOT" #CD-ROM usable as root. "CD9660" req'ed
options PROCFS #Process filesystem
options "COMPAT_43" #Compatible with BSD 4.3 [KEEP THIS!]
options SCSI_DELAY=10000 #Be pessimistic about Joe SCSI device
options UCONSOLE #Allow users to grab the console
options FAILSAFE #Be conservative
options USERCONFIG #boot -c editor
options VISUAL_USERCONFIG #visual boot -c editor
options SOFTUPDATES #enable soft updates support
#options "NMBCLUSTERS=4096"
config kernel root on da0
controller isa0
controller pci0
controller fdc0 at isa? port "IO_FD1" bio irq 6 drq 2
disk fd0 at fdc0 drive 0
# A single entry for any of these controllers (ncr, ahb, ahc) is
# sufficient for any number of installed devices.
controller ncr0
controller ahc0
controller scbus0
device da0
device sa0
device pass0
device cd0
device ch0
# atkbdc0 controlls both the keyboard and the PS/2 mouse
controller atkbdc0 at isa? port IO_KBD tty
device atkbd0 at isa? tty irq 1
device psm0 at isa? tty irq 12
device vga0 at isa? port ? conflicts
# splash screen/screen saver
#pseudo-device splash
# syscons is the default console driver, resembling an SCO console
device sc0 at isa? tty
device npx0 at isa? port IO_NPX irq 13
# Serial ports
device sio0 at isa? port "IO_COM1" flags 0x10 tty irq 4
device sio1 at isa? port "IO_COM2" tty irq 3
# Parallel port
device ppc0 at isa? port? net irq 7
controller ppbus0
device nlpt0 at ppbus?
device plip0 at ppbus?
device ppi0 at ppbus?
#controller vpo0 at ppbus?
# Order is important here due to intrusive probes, do *not* alphabetize
# this list of network interfaces until the probes have been fixed.
# Right now it appears that the ie0 must be probed before ep0. See
# revision 1.20 of this file.
device de0
device fxp0
pseudo-device loop
pseudo-device ether
pseudo-device sl 2
pseudo-device ppp 2
pseudo-device tun 2
pseudo-device pty 64
pseudo-device gzip # Exec gzipped a.out's
#
# Enable debug support
#
options KTRACE #kernel tracing
options DDB #kernel debugger
options INVARIANTS #extra sanity checks
options INVARIANT_SUPPORT #needed for INVARIANTS
#
# These three options provide support for System V Interface
# Definition-style interprocess communication, in the form of shared
# memory, semaphores, and message queues, respectively.
#
options SYSVSHM
options SYSVSEM
options SYSVMSG
# The `bpfilter' pseudo-device enables the Berkeley Packet Filter. Be
# aware of the legal and administrative consequences of enabling this
# option. The number of devices determines the maximum number of
# simultaneous BPF clients programs runnable.
pseudo-device bpfilter 4 #Berkeley packet filter
>How-To-Repeat:
All I have to do is run amanda and wait for about an hour and a half (that's
how long it takes to fail).
>Fix:
Wish I had one to offer.
>Release-Note:
>Audit-Trail:
>Unformatted:
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-bugs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199903301709.LAA33066>