From owner-freebsd-questions@FreeBSD.ORG Thu Jul 12 10:29:23 2007 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1EAD316A421 for ; Thu, 12 Jul 2007 10:29:23 +0000 (UTC) (envelope-from digital9ja@gmail.com) Received: from nz-out-0506.google.com (nz-out-0506.google.com [64.233.162.224]) by mx1.freebsd.org (Postfix) with ESMTP id BD80313C459 for ; Thu, 12 Jul 2007 10:29:20 +0000 (UTC) (envelope-from digital9ja@gmail.com) Received: by nz-out-0506.google.com with SMTP id l8so80717nzf for ; Thu, 12 Jul 2007 03:29:20 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:user-agent:mime-version:to:subject:x-enigmail-version:openpgp:content-type:content-transfer-encoding:from; b=j9v4WDomeuTe1WUKZjTTkI4XWViL4HepmldNZknvtePqkgwK5gdGG76vfAwCa1re+Lk0Ff8MfboiGQTmn74EWaSMZGm8sAgN0wzew6gGyFqIiu0e9LT+0v6yFyE2CDgnmZ2uvTcVf9+nmmv6m0S6OCEFhdvkJlDLcjsPB40F1V0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:user-agent:mime-version:to:subject:x-enigmail-version:openpgp:content-type:content-transfer-encoding:from; b=aOBt/jqTmJK3GK8zG8vY7BBuY08Lz+9JjoUyqgIIQisJoapVaQw7NP7mvKWNJzThucJyRFeMrNgpKLrPm8mpa0g+ypDzjANfVsjYzVAXL3prGbGsv7qkbT0xpQ/v/MiKFaK7JHEjSeSXP2RiN3RfB14j7iNDMK+Zh/Aj/WDTvLI= Received: by 10.114.173.15 with SMTP id v15mr421091wae.1184234555782; Thu, 12 Jul 2007 03:02:35 -0700 (PDT) Received: from ?192.168.1.102? ( [69.50.223.224]) by mx.google.com with ESMTP id n37sm34240417wag.2007.07.12.03.02.34 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 12 Jul 2007 03:02:35 -0700 (PDT) Message-ID: <4695FC36.4000107@cox.net> Date: Thu, 12 Jul 2007 03:02:30 -0700 User-Agent: Thunderbird 1.5.0.12 (X11/20070604) MIME-Version: 1.0 To: freebsd-questions@freebsd.org X-Enigmail-Version: 0.94.2.0 OpenPGP: id=17353058 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit From: Steven Wagner Subject: 6.2 Freezes X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jul 2007 10:29:23 -0000 Our server is running for awhile (sometimes 1 day, sometimes less than an hour) then ssh sessions hang and disconnect, web server times out, console allows us to give input to the login prompt, but after typing root and hitting enter the password prompt never appears. After rebooting and an fsck the server comes back on-line. We were experiencing this about every 10-12 hours but then we disabled APIC in the BIOS and entered the following: hint.apic.0.disabled="1" to /boot/loader.conf and to /boot/defaults/loader.conf The server was up for about a day and a half, then last night went down twice within an hour. Here is the output of top at the time of the freeze: last pid: 5967; load averages: 0.20, 0.42, 0.37 up 0+00:48:58 00:11:41 124 processes: 1 running, 123 sleeping CPU states: 0.0% user, 0.0% nice, 0.0% system, 0.4% interrupt, 99.6% idle Mem: 1186M Active, 949M Inact, 142M Wired, 128K Cache, 112M Buf, 731M Free Swap: 8192M Total, 8192M Free PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 896 mysql 10 20 0 59116K 32668K kserel 2:46 0.00% mysqld 1035 root 1 -4 0 10904K 9636K ufs 0:12 0.00% perl5.8.8 1016 root 1 -4 0 10904K 9636K ufs 0:12 0.00% perl5.8.8 1041 root 1 -4 0 10904K 9636K ufs 0:12 0.00% perl5.8.8 1037 root 1 -4 0 10904K 9636K ufs 0:12 0.00% perl5.8.8 1033 root 1 -4 0 10904K 9636K ufs 0:12 0.00% perl5.8.8 1036 root 1 -4 0 10904K 9636K ufs 0:12 0.00% perl5.8.8 1034 root 1 -4 0 10904K 9636K ufs 0:12 0.00% perl5.8.8 1038 root 1 -4 0 10904K 9636K ufs 0:12 0.00% perl5.8.8 1040 root 1 -4 0 10904K 9636K ufs 0:12 0.00% perl5.8.8 1039 root 1 -4 0 10900K 9632K ufs 0:11 0.00% perl5.8.8 1101 root 1 96 0 9972K 9040K select 0:05 0.00% named 2170 root 1 -4 0 80996K 79172K ufs 0:05 0.00% perl5.8.8 1936 root 1 -4 0 81092K 79196K ufs 0:05 0.00% perl5.8.8 1987 root 1 -4 0 81092K 79196K ufs 0:04 0.00% perl5.8.8 2132 root 1 -4 0 81272K 79444K ufs 0:04 0.00% perl5.8.8 1860 root 1 -4 0 81292K 79236K ufs 0:04 0.00% perl5.8.8 1481 root 1 -4 0 80968K 79112K ufs 0:04 0.00% perl5.8.8 2208 root 1 -4 0 81272K 79440K ufs 0:04 0.00% perl5.8.8 2027 root 1 -4 0 81032K 79076K ufs 0:04 0.00% perl5.8.8 1712 root 1 -4 0 80908K 79060K ufs 0:04 0.00% perl5.8.8 1675 root 1 -4 0 80956K 79120K ufs 0:04 0.00% perl5.8.8 1583 root 1 -4 0 80980K 79140K ufs 0:04 0.00% perl5.8.8 1749 root 1 -4 0 80988K 79148K ufs 0:04 0.00% perl5.8.8 1637 root 1 -4 0 81216K 79212K ufs 0:04 0.00% perl5.8.8 1786 root 1 -4 0 81152K 79316K ufs 0:04 0.00% perl5.8.8 1897 root 1 -4 0 80884K 79044K ufs 0:04 0.00% perl5.8.8 1391 root 1 -4 0 80900K 78928K ufs 0:04 0.00% perl5.8.8 1523 root 1 -4 0 80840K 79016K ufs 0:04 0.00% perl5.8.8 1434 root 1 -4 0 80840K 79016K ufs 0:04 0.00% perl5.8.8 We aren't getting any kind of clues from the log files and there isn't anything relevant on-screen at the time of the freeze. I say freeze rather than crash because no core dumps are getting generated and the server still pings even in the frozen state. The hardware was tested for a month before installing the OS and going live with this server. If anyone has any ideas on what might be causing this or a suggestion as to how I can capture more information at the time of a crash it's very much appreciated. In case it helps, here's the output of /var/log/dmesg: Copyright (c) 1992-2007 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 6.2-RELEASE-p4 #0: Thu Apr 26 17:55:55 UTC 2007 root@i386-builder.daemonology.net:/usr/obj/usr/src/sys/SMP WARNING: MPSAFE network stack disabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(R) CPU X5355 @ 2.66GHz (2666.68-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x6f7 Stepping = 7 Features=0xbfebfbff Features2=0x4e3bd,CX16,,,> AMD Features=0x20100000 AMD Features2=0x1 Cores per package: 4 real memory = 3220611072 (3071 MB) avail memory = 3150569472 (3004 MB) kbd1 at kbdmux0 ath_hal: 0.9.17.2 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) cpu0 on motherboard pcib0: pcibus 0 on motherboard pir0: on motherboard pci0: on pcib0 pcib1: at device 2.0 on pci0 pci1: on pcib1 pcib2: irq 9 at device 0.0 on pci1 pci2: on pcib2 pcib3: irq 9 at device 0.0 on pci2 pci3: on pcib3 pcib4: at device 0.0 on pci3 pci4: on pcib4 aac0: mem 0xd8200000-0xd83fffff,0xd8000000-0xd81fffff,0xc0000000-0xcfffffff irq 9 at device 1.0 on pci4 aac0: New comm. interface enabled aac0: Adaptec Raid Controller 2.0.0-1 aacp0: on aac0 aacp1: on aac0 pcib5: at device 0.2 on pci3 pci5: on pcib5 rl0: port 0x2000-0x20ff mem 0xd8400000-0xd84000ff irq 9 at device 1.0 on pci5 miibus0: on rl0 rlphy0: on miibus0 rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto rl0: Ethernet address: 00:40:f4:50:f0:4e rl0: [GIANT-LOCKED] pcib6: irq 11 at device 2.0 on pci2 pci6: on pcib6 em0: port 0x3000-0x301f mem 0xd8500000-0xd851ffff irq 11 at device 0.0 on pci6 em0: Ethernet address: 00:30:48:33:a6:12 em0: [GIANT-LOCKED] em1: port 0x3020-0x303f mem 0xd8520000-0xd853ffff irq 10 at device 0.1 on pci6 em1: Ethernet address: 00:30:48:33:a6:13 em1: [GIANT-LOCKED] pcib7: at device 0.3 on pci1 pci7: on pcib7 pcib8: at device 4.0 on pci0 pci8: on pcib8 pcib9: at device 6.0 on pci0 pci9: on pcib9 pcib10: irq 5 at device 28.0 on pci0 pci10: on pcib10 uhci0: port 0x1800-0x181f irq 5 at device 29.0 on pci0 uhci0: [GIANT-LOCKED] usb0: on uhci0 usb0: USB revision 1.0 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhci1: port 0x1820-0x183f irq 10 at device 29.1 on pci0 uhci1: [GIANT-LOCKED] usb1: on uhci1 usb1: USB revision 1.0 uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0x1840-0x185f irq 11 at device 29.2 on pci0 uhci2: [GIANT-LOCKED] usb2: on uhci2 usb2: USB revision 1.0 uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub2: 2 ports with 2 removable, self powered ehci0: mem 0xd8a00000-0xd8a003ff irq 5 at device 29.7 on pci0 ehci0: [GIANT-LOCKED] usb3: waiting for BIOS to give up control usb3: timed out waiting for BIOS usb3: EHCI version 1.0 usb3: companion controllers, 2 ports each: usb0 usb1 usb2 usb3: on ehci0 usb3: USB revision 2.0 uhub3: Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1 uhub3: 6 ports with 6 removable, self powered pcib11: at device 30.0 on pci0 pci11: on pcib11 pci11: at device 1.0 (no driver attached) isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1860-0x186f at device 31.1 on pci0 ata0: on atapci0 ata1: on atapci0 pci0: at device 31.3 (no driver attached) pmtimer0 on isa0 orm0: at iomem 0xc0000-0xcafff,0xcb000-0xcffff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] fdc0: at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 fdc0: [FAST] ppc0: at port 0x378-0x37f irq 7 on isa0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/9 bytes threshold ppbus0: on ppc0 plip0: on ppbus0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 16550A sio1 at port 0x2f8-0x2ff irq 3 on isa0 sio1: type 16550A vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 unknown: can't assign resources (port) unknown: can't assign resources (memory) unknown: can't assign resources (memory) unknown: can't assign resources (port) unknown: can't assign resources (port) unknown: can't assign resources (port) unknown: can't assign resources (port) Timecounter "TSC" frequency 2666679432 Hz quality 800 Timecounters tick every 1.000 msec acd0: DMA limited to UDMA33, controller found non-ATA66 cable acd0: DVDROM at ata0-slave UDMA33 aacd0: on aac0 aacd0: 120393MB (246564864 sectors) ses0 at aacp0 bus 0 target 8 lun 0 ses0: Fixed unknown SCSI-2 device ses0: 3.300MB/s transfers ses0: SAF-TE Compliant Device ses1 at aacp1 bus 0 target 8 lun 0 ses1: Fixed unknown SCSI-2 device ses1: 3.300MB/s transfers ses1: SAF-TE Compliant Device pass0 at aacp0 bus 0 target 0 lun 0 pass0: Fixed unknown SCSI-3 device pass0: 3.300MB/s transfers pass1 at aacp0 bus 0 target 1 lun 0 pass1: Fixed unknown SCSI-3 device pass1: 3.300MB/s transfers pass2 at aacp0 bus 0 target 2 lun 0 pass2: Fixed unknown SCSI-3 device pass2: 3.300MB/s transfers pass3 at aacp0 bus 0 target 3 lun 0 pass3: Fixed unknown SCSI-3 device pass3: 3.300MB/s transfers pass4 at aacp0 bus 0 target 4 lun 0 pass4: Fixed unknown SCSI-3 device pass4: 3.300MB/s transfers pass5 at aacp0 bus 0 target 5 lun 0 pass5: Fixed unknown SCSI-3 device pass5: 3.300MB/s transfers pass6 at aacp0 bus 0 target 6 lun 0 pass6: Fixed unknown SCSI-3 device pass6: 3.300MB/s transfers pass8 at aacp0 bus 0 target 9 lun 0 pass8: Fixed unknown SCSI-3 device pass8: 3.300MB/s transfers pass9 at aacp1 bus 0 target 0 lun 0 pass9: Fixed unknown SCSI-3 device pass9: 3.300MB/s transfers pass10 at aacp1 bus 0 target 1 lun 0 pass10: Fixed unknown SCSI-3 device pass10: 3.300MB/s transfers pass11 at aacp1 bus 0 target 2 lun 0 pass11: Fixed unknown SCSI-3 device pass11: 3.300MB/s transfers pass12 at aacp1 bus 0 target 3 lun 0 pass12: Fixed unknown SCSI-3 device pass12: 3.300MB/s transfers pass13 at aacp1 bus 0 target 4 lun 0 pass13: Fixed unknown SCSI-3 device pass13: 3.300MB/s transfers pass14 at aacp1 bus 0 target 5 lun 0 pass14: Fixed unknown SCSI-3 device pass14: 3.300MB/s transfers pass15 at aacp1 bus 0 target 6 lun 0 pass15: Fixed unknown SCSI-3 device pass15: 3.300MB/s transfers pass17 at aacp1 bus 0 target 9 lun 0 pass17: Fixed unknown SCSI-3 device pass17: 3.300MB/s transfers Trying to mount root from ufs:/dev/aacd0s1a rl0: link state changed to UP