From owner-freebsd-stable Tue May 28 11:16:48 2002 Delivered-To: freebsd-stable@freebsd.org Received: from mx1.inoc.net (mx1.inoc.net [64.246.131.30]) by hub.freebsd.org (Postfix) with ESMTP id 0468737BA59 for ; Tue, 28 May 2002 11:12:39 -0700 (PDT) Received: from nimbus (unverified [10.0.0.111]) by mx1.inoc.net (Vircom SMTPRS 5.2.204) with ESMTP id for ; Tue, 28 May 2002 14:12:24 -0400 From: "Robert Blayzor" To: Subject: Swap_pager error Date: Tue, 28 May 2002 14:12:23 -0400 Organization: INOC, LLC Message-ID: <008201c20673$37ac9c60$6f00000a@z0.inoc.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG We have a Dell PowerEdge 2550 server. It's running FreeBSD4-stable (up'd just a couple of weeks ago). It's an SMP box, 1GB of RAM, two 3com Tigon2 Gigabit NIC cards and a PERC3/QC controller. We have two logical drives. One is a RAID1 set of two 9GB drives which holds the operating system only. The other is a 300GB RAID10 array. The box had been running fine for months when suddenly the box got hosed as we received tons of these errors on the console. (nothing logged to /var/log/messages) swap_pager: indefinite wait buffer: #amrd/0x20001, blkno:272, size:4096 The box only runs as an NFS/Samba server and nothing else. It eventually just became useless and we had to reset the box hard. We ran FSCK and it reported no errors and the box came up normally. We were considering running scanning on the OS disk containing the swap, but feel there really is no need to as the RAID controller is reporting no problems as well. Anyone have any suggestions on where to start looking for this problem? We've had this unit in service almost six months and this is the first time we've seen this. Is there a way to "test" swap space in production other than writing something to gobble up memory and forcing the box to swap? Kernel dmesg: Copyright (c) 1992-2002 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 4.5-RELEASE-p4 #0: Mon Apr 22 21:37:02 EDT 2002 root@goliath:/usr/obj/src/sys/GOLIATH Timecounter "i8254" frequency 1193182 Hz CPU: Pentium III/Pentium III Xeon/Celeron (997.46-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x68a Stepping = 10 Features=0x383fbff real memory = 1073676288 (1048512K bytes) avail memory = 1041707008 (1017292K bytes) Changing APIC ID for IO APIC #0 from 0 to 2 on chip Changing APIC ID for IO APIC #1 from 0 to 3 on chip Programming 16 pins in IOAPIC #0 IOAPIC #0 intpin 2 -> irq 0 Programming 16 pins in IOAPIC #1 FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 1, version: 0x00040011, at 0xfee00000 cpu1 (AP): apic id: 0, version: 0x00040011, at 0xfee00000 io0 (APIC): apic id: 2, version: 0x000f0011, at 0xfec00000 io1 (APIC): apic id: 3, version: 0x000f0011, at 0xfec01000 Preloaded elf kernel "kernel" at 0xc0350000. Pentium Pro MTRR support enabled md0: Malloc disk Using $PIR table, 8 entries at 0xc00fc270 npx0: on motherboard npx0: INT 16 interface pcib0: on motherboard IOAPIC #1 intpin 6 -> irq 2 IOAPIC #1 intpin 2 -> irq 5 pci0: on pcib0 pcib2: at device 2.0 on pci0 IOAPIC #1 intpin 5 -> irq 10 pci1: on pcib2 pcib5: at device 0.0 on pci1 IOAPIC #1 intpin 4 -> irq 11 pci2: on pcib5 amr0: mem 0xf0000000-0xf7ffffff irq 11 at device 0.0 on pci2 amr0: Firmware 1.57, BIOS 3.13, 128MB RAM pci1: (vendor=0x1077, dev=0x1216) at 1.0 irq 10 pci1: (vendor=0x1077, dev=0x1216) at 2.0 irq 2 ti0: <3Com 3c985-SX Gigabit Ethernet> mem 0xfe404000-0xfe407fff irq 2 at device 4.0 on pci0 ti0: Ethernet address: 00:60:08:f6:f5:92 ti1: <3Com 3c985-SX Gigabit Ethernet> mem 0xfe400000-0xfe403fff irq 5 at device 8.0 on pci0 ti1: Ethernet address: 00:60:08:f6:f5:93 pci0: at 14.0 isab0: at device 15.0 on pci0 isa0: on isab0 atapci0: port 0x8b0-0x8bf at device 15.1 on pci0 ata0: at 0x1f0 irq 14 on atapci0 pcib4: on motherboard IOAPIC #1 intpin 0 -> irq 13 pci4: on pcib4 pcib6: at device 2.0 on pci4 IOAPIC #1 intpin 15 -> irq 16 IOAPIC #1 intpin 14 -> irq 17 pci5: on pcib6 ahc0: port 0xbc00-0xbcff mem 0xfe7ff000-0xfe7fffff irq 16 at device 4.0 on pci5 aic7899: Ultra160 Wide Channel A, SCSI Id=7, 32/255 SCBs ahc1: port 0xb800-0xb8ff mem 0xfe7fe000-0xfe7fefff irq 17 at device 4.1 on pci5 aic7899: Ultra160 Wide Channel B, SCSI Id=7, 32/255 SCBs fxp0: port 0xacc0-0xacff mem 0xfe500000-0xfe5fffff,0xfe900000-0xfe900fff irq 13 at device 4.0 on pci4 fxp0: Ethernet address 00:06:5b:1a:5c:e9 inphy0: on miibus0 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto pcib1: on motherboard IOAPIC #1 intpin 1 -> irq 18 pci3: on pcib1 pci3: (vendor=0x14e4, dev=0x1644) at 8.0 irq 18 pcib3: on motherboard pci6: on pcib3 orm0: