From owner-freebsd-current@FreeBSD.ORG Fri Oct 3 11:55:50 2003 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5E45216A4B3 for ; Fri, 3 Oct 2003 11:55:50 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) by mx1.FreeBSD.org (Postfix) with ESMTP id 394D343FEC for ; Fri, 3 Oct 2003 11:55:47 -0700 (PDT) (envelope-from larse@ISI.EDU) Received: from isi.edu (c-24-130-112-121.we.client2.attbi.com [24.130.112.121]) by boreas.isi.edu (8.11.6p2+0917/8.11.2) with ESMTP id h93Itib12358; Fri, 3 Oct 2003 11:55:44 -0700 (PDT) Message-ID: <3F7DC62E.8090707@isi.edu> Date: Fri, 03 Oct 2003 11:55:42 -0700 From: Lars Eggert Organization: USC Information Sciences Institute User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20030916 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Kris Kennaway References: <20031003054326.GA51359@rot13.obsecurity.org> <3F7DAD7C.2040505@isi.edu> <20031003173649.GA54540@rot13.obsecurity.org> In-Reply-To: <20031003173649.GA54540@rot13.obsecurity.org> Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms040803000809040802020702" cc: current@FreeBSD.org Subject: Re: NFS corruption on p4 machines (please test) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Oct 2003 18:55:50 -0000 This is a cryptographically signed message in MIME format. --------------ms040803000809040802020702 Content-Type: multipart/mixed; boundary="------------070900030808020003040106" This is a multi-part message in MIME format. --------------070900030808020003040106 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Kris Kennaway wrote: > On Fri, Oct 03, 2003 at 10:10:20AM -0700, Lars Eggert wrote: > >>Kris, >> >>Kris Kennaway wrote: >> >> >>>For some months now I have been experiencing NFS corruption on the >>>three machines in the dosirak.kr package cluster - these are SMP >>>pentium 4 machines that run -CURRENT. Setting DISABLE_PSE and >>>DISABLE_PG_G does not fix these problems. I am able to easily >>>reproduce these problems using /usr/src/tools/regression/fsx on a >>>loopback nfs mount - they are not deterministic, but it blows up >>>within about 8000 operations (less than a minute of operation). In >>>fact sometimes it even manages to make fsx segfault, which is fairly >>>impressive :) >>> >>>Just mount something rw via loopback nfs, and run 'fsx foo' on the nfs >>>filesystem for a few minutes. >> >>I just ran an fsx cycle on my desktop machine over a TCP mount, and it >>seemed to work fine: > > > Thanks. What hardware specs? Attached. Lars -- Lars Eggert USC Information Sciences Institute --------------070900030808020003040106 Content-Type: text/plain; name="dmesg.boot" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="dmesg.boot" cam: using minimum scsi_delay (100ms) Copyright (c) 1992-2003 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 5.1-CURRENT #0: Tue Sep 30 10:11:59 PDT 2003 root@nik.isi.edu:/usr/obj/usr/src/sys/KERNEL-1.31 Preloaded elf kernel "/boot/kernel/kernel" at 0xc06ed000. Preloaded elf module "/boot/kernel/vesa.ko" at 0xc06ed21c. Preloaded elf module "/boot/kernel/md.ko" at 0xc06ed2c8. Preloaded elf module "/boot/kernel/linux.ko" at 0xc06ed370. Preloaded elf module "/boot/kernel/if_gif.ko" at 0xc06ed41c. Preloaded elf module "/boot/kernel/if_tun.ko" at 0xc06ed4c8. Preloaded elf module "/boot/kernel/ipfw.ko" at 0xc06ed574. Preloaded elf module "/boot/kernel/if_an.ko" at 0xc06ed620. Preloaded elf module "/boot/kernel/wlan.ko" at 0xc06ed6cc. Preloaded elf module "/boot/kernel/rc4.ko" at 0xc06ed778. Preloaded elf module "/boot/kernel/pccard.ko" at 0xc06ed820. Preloaded elf module "/boot/kernel/if_em.ko" at 0xc06ed8cc. Preloaded elf module "/boot/kernel/if_fxp.ko" at 0xc06ed978. Preloaded elf module "/boot/kernel/miibus.ko" at 0xc06eda24. Preloaded elf module "/boot/kernel/if_lnc.ko" at 0xc06edad0. Preloaded elf module "/boot/kernel/if_wi.ko" at 0xc06edb7c. Preloaded elf module "/boot/kernel/if_xl.ko" at 0xc06edc28. Preloaded elf module "/boot/kernel/snd_emu10k1.ko" at 0xc06edcd4. Preloaded elf module "/boot/kernel/snd_pcm.ko" at 0xc06edd84. Preloaded elf module "/boot/kernel/snd_es137x.ko" at 0xc06ede30. Preloaded elf module "/boot/kernel/snd_ich.ko" at 0xc06edee0. Preloaded elf module "/boot/kernel/snd_maestro3.ko" at 0xc06edf8c. Preloaded elf module "/boot/kernel/ugen.ko" at 0xc06ee040. Preloaded elf module "/boot/kernel/usb.ko" at 0xc06ee0ec. Preloaded elf module "/boot/kernel/uhid.ko" at 0xc06ee194. Preloaded elf module "/boot/kernel/ukbd.ko" at 0xc06ee240. Preloaded elf module "/boot/kernel/ulpt.ko" at 0xc06ee2ec. Preloaded elf module "/boot/kernel/ums.ko" at 0xc06ee398. Preloaded elf module "/boot/kernel/umass.ko" at 0xc06ee440. Preloaded elf module "/boot/kernel/umodem.ko" at 0xc06ee4ec. Preloaded elf module "/boot/kernel/ucom.ko" at 0xc06ee598. Preloaded elf module "/boot/kernel/bktr.ko" at 0xc06ee644. Preloaded elf module "/boot/kernel/bktr_mem.ko" at 0xc06ee6f0. Preloaded elf module "/boot/kernel/agp.ko" at 0xc06ee7a0. Preloaded elf module "/boot/kernel/random.ko" at 0xc06ee848. Preloaded elf module "/boot/kernel/ip_mroute.ko" at 0xc06ee8f4. Preloaded elf module "/boot/kernel/ip6fw.ko" at 0xc06ee9a4. Preloaded elf module "/boot/kernel/netgraph.ko" at 0xc06eea50. Preloaded elf module "/boot/kernel/dummynet.ko" at 0xc06eeb00. Preloaded elf module "/boot/kernel/radeon.ko" at 0xc06eebb0. Preloaded elf module "/boot/kernel/r128.ko" at 0xc06eec5c. Preloaded elf module "/boot/kernel/ahc.ko" at 0xc06eed08. Preloaded elf module "/boot/kernel/mpt.ko" at 0xc06eedb0. Preloaded elf module "/boot/kernel/fdc.ko" at 0xc06eee58. Preloaded elf module "/boot/kernel/cbb.ko" at 0xc06eef00. Preloaded elf module "/boot/kernel/exca.ko" at 0xc06eefa8. Preloaded elf module "/boot/kernel/cardbus.ko" at 0xc06ef054. Preloaded elf module "/boot/kernel/lpt.ko" at 0xc06ef100. Preloaded elf module "/boot/kernel/ubsa.ko" at 0xc06ef1a8. Preloaded elf module "/boot/kernel/firewire.ko" at 0xc06ef254. Preloaded elf module "/boot/kernel/sbp.ko" at 0xc06ef304. Preloaded elf module "/boot/kernel/smbus.ko" at 0xc06ef3ac. Preloaded elf module "/boot/kernel/intpm.ko" at 0xc06ef458. Preloaded elf module "/boot/kernel/smb.ko" at 0xc06ef504. Preloaded elf module "/boot/kernel/iicbus.ko" at 0xc06ef5ac. Preloaded elf module "/boot/kernel/iic.ko" at 0xc06ef658. Preloaded elf module "/boot/kernel/iicsmb.ko" at 0xc06ef700. Preloaded elf module "/boot/kernel/uart.ko" at 0xc06ef7ac. Preloaded elf module "/boot/kernel/acpi.ko" at 0xc06ef858. Timecounter "i8254" frequency 1193121 Hz quality 0 CPU: Intel(R) XEON(TM) CPU 2.40GHz (2372.81-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf24 Stepping = 4 Features=0x3febfbff Hyperthreading: 2 logical CPUs real memory = 1073180672 (1023 MB) avail memory = 1034924032 (986 MB) Changing APIC ID for IO APIC #0 from 0 to 4 on chip Programming 24 pins in IOAPIC #0 IOAPIC #0 intpin 2 -> irq 0 FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs cpu0 (BSP): apic id: 0, version: 0x00050014, at 0xfee00000 cpu1 (AP): apic id: 1, version: 0x00050014, at 0xfee00000 cpu2 (AP): apic id: 2, version: 0x00050014, at 0xfee00000 cpu3 (AP): apic id: 3, version: 0x00050014, at 0xfee00000 io0 (APIC): apic id: 4, version: 0x00178020, at 0xfec00000 bktr_mem: memory holder loaded Pentium Pro MTRR support enabled VESA: v2.0, 65536k memory, flags:0x1, mode table:0xc0410a22 (1000022) VESA: ATI RADEON RV250 npx0: on motherboard npx0: INT 16 interface acpi0: on motherboard pcibios: BIOS version 2.10 Using $PIR table, 12 entries at 0xc00fb9a0 acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 acpi_cpu0: on acpi0 acpi_cpu1: on acpi0 acpi_cpu2: on acpi0 acpi_cpu3: on acpi0 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 IOAPIC #0 intpin 19 -> irq 2 IOAPIC #0 intpin 17 -> irq 13 IOAPIC #0 intpin 23 -> irq 16 agp0: mem 0xe8000000-0xefffffff at device 0.0 on pci0 pcib1: at device 1.0 on pci0 pci1: on pcib1 IOAPIC #0 intpin 16 -> irq 17 drm0: port 0xec00-0xecff mem 0xff8f0000-0xff8fffff,0xf4000000-0xf7ffffff irq 17 at device 0.0 on pci1 info: [drm] AGP at 0xe8000000 128MB info: [drm] Initialized radeon 1.9.0 20020828 on minor 0 pci1: at device 0.1 (no driver attached) pcib2: at device 2.0 on pci0 pci2: on pcib2 pcib3: at device 31.0 on pci2 pci3: on pcib3 IOAPIC #0 intpin 20 -> irq 18 IOAPIC #0 intpin 21 -> irq 19 pci3: at device 0.0 (no driver attached) mpt0: port 0xdc00-0xdcff mem 0xff6a0000-0xff6bffff,0xff6c0000-0xff6dffff irq 18 at device 12.0 on pci3 mpt1: port 0xd800-0xd8ff mem 0xff660000-0xff67ffff,0xff680000-0xff69ffff irq 19 at device 12.1 on pci3 em0: mem 0xff6e0000-0xff6effff,0xff640000-0xff65ffff irq 19 at device 13.0 on pci3 em0: Speed:1000 Mbps Duplex:Full pcib4: at device 30.0 on pci0 pci4: on pcib4 IOAPIC #0 intpin 18 -> irq 20 xl0: <3Com 3c905C-TX Fast Etherlink XL> port 0xcc80-0xccff mem 0xff3ffc00-0xff3ffc7f irq 16 at device 11.0 on pci4 xl0: Ethernet address: 00:06:5b:bd:ee:48 miibus0: on xl0 ukphy0: on miibus0 ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto fwohci0: mem 0xff3f8000-0xff3fbfff,0xff3ff000-0xff3ff7ff irq 17 at device 12.0 on pci4 fwohci0: [MPSAFE] fwohci0: OHCI version 1.0 (ROM=0) fwohci0: No. of Isochronous channel is 4. fwohci0: EUI64 80:5b:06:00:48:ee:bd:00 fwohci0: Phy 1394a available S400, 2 ports. fwohci0: Link S400, max_rec 2048 bytes. firewire0: on fwohci0 sbp0: on firewire0 fwohci0: Initiate bus reset fwohci0: BUS reset fwohci0: node_id=0xc800ffc0, gen=1, CYCLEMASTER mode firewire0: 1 nodes, maxhop <= 0, cable IRM = 0 (me) firewire0: bus manager 0 (me) em1: port 0xcc40-0xcc7f mem 0xff3a0000-0xff3bffff,0xff3c0000-0xff3dffff irq 13 at device 13.0 on pci4 em1: Speed:N/A Duplex:N/A pcm0: port 0xcc20-0xcc3f irq 20 at device 14.0 on pci4 pcm0: isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0xffa0-0xffaf at device 31.1 on pci0 ata0: at 0x1f0 irq 14 on atapci0 ata0: [MPSAFE] ata1: at 0x170 irq 15 on atapci0 ata1: [MPSAFE] uhci0: port 0xff80-0xff9f irq 2 at device 31.2 on pci0 usb0: on uhci0 usb0: USB revision 1.0 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhub1: NMB Dell USB Keyboard Hub, class 9/0, rev 1.10/0.01, addr 2 uhub1: 3 ports with 2 removable, bus powered ukbd0: NMB Dell USB 7HK Keyboard, rev 1.10/0.01, addr 3, iclass 3/1 kbd0 at ukbd0 ums0: Logitech USB Receiver, rev 1.10/9.10, addr 4, iclass 3/1 ums0: 5 buttons and Z dir. pci0: at device 31.3 (no driver attached) uhci1: port 0xff60-0xff7f irq 16 at device 31.4 on pci0 usb1: on uhci1 usb1: USB revision 1.0 uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub2: 2 ports with 2 removable, self powered fdc0: port 0x3f7,0x3f0-0x3f5 irq 6 drq 2 on acpi0 fdc0: FIFO enabled, 8 bytes threshold fd0: <1440-KB 3.5" drive> on fdc0 drive 0 atkbdc0: port 0x64,0x60 irq 1 on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 ppc0 port 0x778-0x77f,0x378-0x37f irq 7 drq 1 on acpi0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/8 bytes threshold ppbus0: on ppc0 lpt0: on ppbus0 lpt0: Interrupt-driven port orm0: