Date: Sat, 11 Nov 2006 18:27:31 +0100 From: Gergely CZUCZY <phoemix@harmless.hu> To: freebsd-hackers@freebsd.org Subject: system randomly freezes Message-ID: <20061111172731.GB39999@harmless.hu>
next in thread | raw e-mail | index | archive | help
--TRYliJ5NKNqkz5bu Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello, I have sent this as a PR, but Remko Lodder suggested that it should be discuessed here. I have seen several other examples to very similar unstability issues in the mailing lists previously. When i have posted this to -questions@ there also was a person who said he has fimilar behaviours sometimes. so, here it goes: >Environment: System: FreeBSD marvin.harmless.hu 6.1-RELEASE-p10 FreeBSD 6.1-RELEASE-p10 = #0: Mon Oct 2 14:54:14 CEST 2006 root@marvin.harmless.hu:/usr/obj/usr/src/s= ys/MARVIN i386 >Description: We run hosting services on this FreeBSD box, and we encountered strange problems. Sometimes, in 4-6 weeks' time the system simply freezes. I use the term "freeze", because i feel it the proper one. The console is blank, we cannot do network communication with the box, there is no crash dump left of the event, the system logs are empty. We've got absolutely no clue on this at the moment. This a supermicro 5013C-MT: http://www.supermicro.com/products/system/1U/5013/SYS-5013C-MT.cfm with a P4-2.4GHz, and with 4x512MB of DDR RAM. We are using the 6300ESB SATA controller, because the marvell one is buggy (i have already filed a bug report on that). All the other hardware spec will be in the later pciiconf output and dmesg. The config. We are running ftp, web, email, postgresql services mostly on this box. We have 6 jails on it with the pf packet filter. if anything is else is needed, please ask for the info. Dmesg: --- chop here --- Copyright (c) 1992-2006 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 6.1-RELEASE-p10 #0: Mon Oct 2 14:54:14 CEST 2006 root@marvin.harmless.hu:/usr/obj/usr/src/sys/MARVIN ACPI APIC Table: <IntelR AWRDACPI> Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Pentium(R) 4 CPU 2.80GHz (2795.24-MHz 686-class CPU) Origin =3D "GenuineIntel" Id =3D 0xf29 Stepping =3D 9 Features=3D0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PG= E,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=3D0x4400<CNTX-ID,<b14>> Logical CPUs per core: 2 real memory =3D 2146304000 (2046 MB) avail memory =3D 2096168960 (1999 MB) ioapic0: Changing APIC ID to 2 ioapic0 <Version 2.0> irqs 0-23 on motherboard ioapic1 <Version 2.0> irqs 24-47 on motherboard kbd1 at kbdmux0 acpi0: <IntelR AWRDACPI> on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 cpu0: <ACPI CPU> on acpi0 acpi_button0: <Power Button> on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 pcib1: <ACPI PCI-PCI bridge> at device 3.0 on pci0 pci1: <ACPI PCI bus> on pcib1 em0: <Intel(R) PRO/1000 Network Connection Version - 3.2.18> port 0xc000-0x= c01f mem 0xfd500000-0xfd51ffff irq 18 at device 1.0 on pci1 em0: Ethernet address: 00:30:48:80:aa:3e pcib2: <ACPI PCI-PCI bridge> at device 28.0 on pci0 pci2: <ACPI PCI bus> on pcib2 atapci0: <Marvell 88SX5041 SATA150 controller> mem 0xfd400000-0xfd47ffff ir= q 27 at device 4.0 on pci2 ata2: <ATA channel 0> on atapci0 ata3: <ATA channel 1> on atapci0 ata4: <ATA channel 2> on atapci0 ata5: <ATA channel 3> on atapci0 pcib3: <ACPI PCI-PCI bridge> at device 30.0 on pci0 pci3: <ACPI PCI bus> on pcib3 pci3: <display, VGA> at device 9.0 (no driver attached) em1: <Intel(R) PRO/1000 Network Connection Version - 3.2.18> port 0xd100-0x= d13f mem 0xfc000000-0xfc01ffff irq 19 at device 10.0 on pci3 em1: Ethernet address: 00:30:48:80:aa:3f isab0: <PCI-ISA bridge> at device 31.0 on pci0 isa0: <ISA bus> on isab0 atapci1: <Intel 6300ESB UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x= 177,0x376,0xf000-0xf00f at device 31.1 on pci0 ata0: <ATA channel 0> on atapci1 ata1: <ATA channel 1> on atapci1 atapci2: <Intel 6300ESB SATA150 controller> port 0xe200-0xe207,0xe300-0xe30= 3,0xe400-0xe407,0xe500-0xe503,0xe600-0xe60f at device 31.2 on pci0 ata6: <ATA channel 0> on atapci2 ata7: <ATA channel 1> on atapci2 pci0: <serial bus, SMBus> at device 31.3 (no driver attached) acpi_tz0: <Thermal Zone> on acpi0 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acp= i0 sio0: type 16550A sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] pmtimer0 on isa0 orm0: <ISA Option ROM> at iomem 0xc0000-0xc7fff on isa0 sc0: <System console> at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=3D0x300> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounter "TSC" frequency 2795239320 Hz quality 800 Timecounters tick every 1.000 msec acd0: CDROM <CD-224E/1.9A> at ata0-master UDMA33 ad12: 190782MB <HDT722520DLA380 V44OA9BA> at ata6-master SATA150 ad14: 190782MB <HDT722520DLA380 V44OA96A> at ata7-master SATA150 GEOM_MIRROR: Device gm0 created (id=3D2141307500). GEOM_MIRROR: Device gm0: provider ad12 detected. GEOM_MIRROR: Device gm0: provider ad14 detected. GEOM_MIRROR: Device gm0: provider ad14 activated. GEOM_MIRROR: Device gm0: provider ad12 activated. GEOM_MIRROR: Device gm0: provider mirror/gm0 launched. Trying to mount root from ufs:/dev/mirror/gm0s1a WARNING: / was not properly dismounted WARNING: /var was not properly dismounted /var: mount pending error: blocks 36 files 3 em1: link state changed to UP --- chop here --- pciconf -lv: --- chop here --- hostb0@pci0:0:0: class=3D0x060000 card=3D0x518015d9 chip=3D0x25788086 rev= =3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82875P/E7210 DRAM Controller / Host-Hub Interface' class =3D bridge subclass =3D HOST-PCI pcib1@pci0:3:0: class=3D0x060400 card=3D0x00000000 chip=3D0x257b8086 rev=3D= 0x02 hdr=3D0x01 vendor =3D 'Intel Corporation' device =3D '82875P/E7210 PCI to CSA Bridge' class =3D bridge subclass =3D PCI-PCI pcib2@pci0:28:0: class=3D0x060400 card=3D0x00000050 chip=3D0x25ae8086 rev= =3D0x02 hdr=3D0x01 vendor =3D 'Intel Corporation' device =3D '6300ESB Hub Interface to PCI-X Bridge' class =3D bridge subclass =3D PCI-PCI pcib3@pci0:30:0: class=3D0x060400 card=3D0x00000000 chip=3D0x244e8086 rev= =3D0x0a hdr=3D0x01 vendor =3D 'Intel Corporation' device =3D '82801BA/CA/DB/DBL/EB/ER/FB (ICH2/3/4/4/5/5/6), 6300ESB Hu= b Interface to PCI Bridge' class =3D bridge subclass =3D PCI-PCI isab0@pci0:31:0: class=3D0x060100 card=3D0x00000000 chip=3D0x25a18086 rev= =3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '6300ESB LPC Interface Bridge' class =3D bridge subclass =3D PCI-ISA atapci1@pci0:31:1: class=3D0x01018a card=3D0x518015d9 chip=3D0x25a28086 rev= =3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '6300ESB IDE Controller' class =3D mass storage subclass =3D ATA atapci2@pci0:31:2: class=3D0x01018f card=3D0x518015d9 chip=3D0x25a38086 rev= =3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '6300ESB Serial ATA Controller' class =3D mass storage subclass =3D ATA none0@pci0:31:3: class=3D0x0c0500 card=3D0x518015d9 chip=3D0x25a48086 rev= =3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '6300ESB SMBus Controller' class =3D serial bus subclass =3D SMBus em0@pci1:1:0: class=3D0x020000 card=3D0x10758086 chip=3D0x10758086 rev=3D0x= 00 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82547EI Gigabit Ethernet Controller' class =3D network subclass =3D ethernet atapci0@pci2:4:0: class=3D0x010000 card=3D0x504111ab chip=3D0x504111ab rev= =3D0x03 hdr=3D0x00 vendor =3D 'Marvell Semiconductor (Was: Galileo Technology Ltd)' device =3D 'MV88SX5041 4-port SATA PCI-X Controller' class =3D mass storage subclass =3D SCSI none1@pci3:9:0: class=3D0x030000 card=3D0x518015d9 chip=3D0x47521002 rev=3D= 0x27 hdr=3D0x00 vendor =3D 'ATI Technologies Inc' device =3D 'Rage XL PCI' class =3D display subclass =3D VGA em1@pci3:10:0: class=3D0x020000 card=3D0x10768086 chip=3D0x10768086 rev=3D0= x00 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82547EI Gigabit Ethernet Controller' class =3D network subclass =3D ethernet --- chop here --- Kernel config: --- chop here --- machine i386 cpu I686_CPU ident MARVIN maxusers 64 device npx # i will have this removed soonely: options CPU_FASTER_5X86_FPU # To statically compile in device wiring instead of /boot/device.hints #hints "GENERIC.hints" # Default places to look for devices. #makeoptions DEBUG=3D-g # Build kernel with gdb(1) debug symbols #options SCHED_ULE # ULE scheduler options SCHED_4BSD # 4BSD scheduler options PREEMPTION # Enable kernel thread preemption options INET # InterNETworking #options INET6 # IPv6 communications protocols options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists options UFS_DIRHASH # Improve performance on big directories #options MD_ROOT # MD is a potential root device options NFSCLIENT # Network Filesystem Client options NFSSERVER # Network Filesystem Server #options NFS_ROOT # NFS usable as /, requires NFSCLIENT options MSDOSFS # MSDOS Filesystem options CD9660 # ISO 9660 Filesystem options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework options GEOM_GPT # GUID Partition Tables. options COMPAT_43 # Compatible with BSD 4.3 [KEEP THIS!] options COMPAT_FREEBSD4 # Compatible with FreeBSD4 # i will have this removed soonely: options COMPAT_FREEBSD5 # Compatible with FreeBSD5 options SCSI_DELAY=3D5000 # Delay (in ms) before probing SCSI options KTRACE # ktrace(1) support options SYSVSHM # SYSV-style shared memory options SYSVMSG # SYSV-style message queues options SYSVSEM # SYSV-style semaphores # i will have this removed soonely: options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions options KBD_INSTALL_CDEV # install a CDEV entry in /dev # i will have this removed soonely: options AHC_REG_PRETTY_PRINT # Print register bitfields in debug # output. Adds ~128k to driver. # i will have this removed soonely: options AHD_REG_PRETTY_PRINT # Print register bitfields in debug # output. Adds ~215k to driver. options ADAPTIVE_GIANT # Giant mutex is adaptive. device apic # I/O APIC # Bus support. device isa device eisa device pci # Floppy drives #device fdc # ATA and ATAPI devices device ata device atadisk # ATA disk drives # i will have this removed soonely: device ataraid # ATA RAID drives device atapicd # ATAPI CDROM drives #device atapifd # ATAPI floppy drives #device atapist # ATAPI tape drives options ATA_STATIC_ID # Static device numbering # atkbdc0 controls both the keyboard and the PS/2 mouse device atkbdc # AT keyboard controller device atkbd # AT keyboard device kbdmux # keyboard multiplexer device vga # VGA video card driver # syscons is the default console driver, resembling an SCO console device sc # Enable this for the pcvt (VT220 compatible) console driver device vt #options XSERVER # support for X server on a vt console #options FAT_CURSOR # start with block cursor # i will have this removed soonely: device agp # support several AGP chipsets # Power management support (see NOTES for more options) # i will have this removed soonely: device apm # Add suspend/resume support for the i8254. # i will have this removed soonely: device pmtimer # Serial (COM) ports device sio # 8250, 16[45]50 based serial ports # PCI Ethernet NICs. device em # Intel PRO/1000 adapter Gigabit Ethernet Card # PCI Ethernet NICs that use the common MII bus controller code. # NOTE: Be sure to keep the 'device miibus' line in order to use these NICs! device miibus # MII bus support device fxp # Intel EtherExpress PRO/100B (82557, 82558) device sk # SysKonnect SK-984x & SK-982x gigabit Ethernet device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'') # Pseudo devices. device loop # Network loopback device random # Entropy device device ether # Ethernet support # i will have this removed soonely: device sl # Kernel SLIP # i will have this removed soonely: device ppp # Kernel PPP device tun # Packet tunnel. device pty # Pseudo-ttys (telnet etc) device md # Memory "disks" # The `bpf' device enables the Berkeley Packet Filter. # Be aware of the administrative consequences of enabling this! # Note that 'bpf' is required for DHCP. device bpf # Berkeley packet filter device pf device pflog device pfsync options ALTQ options ALTQ_CBQ options ALTQ_RED options ALTQ_RIO options ALTQ_HFSC options ALTQ_PRIQ options ALTQ_NOPCC device tap options COMPAT_LINUX # i will have this removed soonely: device acpi device nullfs device geom_mirror device geom_label options DEVICE_POLLING --- chop here --- Bye, Gergely Czuczy mailto: gergely.czuczy@harmless.hu --=20 Weenies test. Geniuses solve problems that arise. --TRYliJ5NKNqkz5bu Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (FreeBSD) owHNWb1vI0l2X9/awKkBB5c6qpsFPBKOH/1BUhQ9M16KpCRiRJFmU7OzXiw0ze4i 1af+2q5uSdzAkQMHB8PYxMEFlzg1cMHB/gOMSw9wZBjOnDp14sj+vepms0lp5+N2 F7Bml2R3vffq/V69eh9Vf/fHH3/0o5/87te/+eJnv/jml3/wj41/mVf8NEmCZdW3 4ls3qGqqqlWNI/xVNaPaOjTUueO0m4Y9t5sLffDqxx/3wiDhQVKdrSLeYQm/T+qR Z7nBnzH72ooFT56nyaLaVtZ0fVdEoXATNww6zA08N+DF2Cy2ArHgcXUQ2KHjBssO +yoNE+5Uo9gNEmvucUU5454XVhRlyK6tW84EGFly7Qpm4T82mVbYPE3YlPs3ITsP HYfHTKTLJReQA0IrUdyEiesw9Rw258xxhZ1yITB4zWNeY4VcHuDjlseWx8IEYwq/ t/zI44IlIcP7FROu73pWzNJAQDnXc5MVc4WAOCDDXJz5Ft4GS8VzRSJYFPNbN0yF t6qxz64h383mgkEy5Vwpu/oVJJCBxKcKTcyZ5YmQ3UmAEY9FGLC765AJyyWlIUOw Ra7KnEMi5oiFIkKfJ67PRU3B74qEx4B9GXLRUV4Mgls3DgMf9uso5goa+B12EnN+ bPZZtvw1rKAPwKJ2nbJWTatOB+eDrjmoRppakO6+f658onbYCDqO7YTpTGt0mo2O 1mC9gTljuqq2WByGyacPp+jUUxHXw/nP5beI7bp4rqxEfdSdvhpeMNdotxRFedHn wo7dSHqQ8hlncRqwa1gQdsaCQagN+4dBZs61lvPwvsKswGF3XOHwrhQeF8PmIoHP LbEEcQjv8slW5tpuFVrFRrUFHn4jnjJ6KVdVSGvR8kfeii0wxddk5SFLRUYA2T57 kg08gUNy26Ihly0492gNiAhTYjGhKa8pMzzbWPDQA5Vgc88KbiqYl9lWEIQJc0IW 8OQujG9A5vtp4NoWGUC5c5NrKU0CzLwFAoKQ2bElrpmT+hHz+CJh4ULSwaODRFIq OQwvXMKxwMf9KCHP5E/hk0vMas2hUJpwgCSBXsoLu2IbSf8OyX9gtJl8iZ0GSL5r xyFrqprRq45mHeU6SaJOvX53d1fbjNeAow4TOKmdiHqmSl27rBNb3fzcrK75a/bC z2Bidzeqeq1xevZ1vpb0tnHf1PTRMeHr96ds2h1BHbgFIUoFOQUp2jJUdWAeM7M7 65Klkzj0PB5v1ibbrfEtoouCJZGrgLCxYvv5JrW8mFsOltv14DcWjbKYR2GcZEax koOa0vU8KUkGDPDFzh3pISJuQ1sMIuLkocGz4CUssl0X6ixYmCZRmigEy4HvLaVN pVMsXDzkgODrAUFaJBG5x7yCRUOIqcgQsoy5+Mrb7AEf77B0+ZIp8BBa3AxNi/0c fHKfwB0LL4oWLLLsG54QTOgHLdwFbL2CBEwLo3BPZB7GucMdTOxxC28sccMWYSyF uMEiBGOfYHSUarWKVBBGWfzBE8J9tIrd5XXC9u0Dph0d6VUZFgjver9O4vDn3IZn 7RIfHlXw2VblpyE/W/KzLT/l6JEmP3X5KWmOGsoeiZ/yJfxVrDfDZeAikAuK23jT szwXGALXqjFaSDmtwCKTRblTU74t5JUD3qMRT2H4e/+oV8Q8pdubDFl3MuyxGWW/ Dns2RODypqz72bRPgy+UGaJSHs/YE7etNxtPKCQhhwT2imnakaG1dXb2NVKpJTOU qvQmlx0mBe1PD9gEFnFTn35C5ckl02ttFXuM7euHR82a3qiO8NBqt6q2ZwlBJAcA NIZ54MrPjT57csqDFIlcinzC2NCRr9X7hX7EmJnwKCLvoXdH4DzhVpLCqnhW7+cL Pl/MF4tnJ5PLyqvRoNIfVCbmoDIze5WROa1MuoPKqDeo9F63K2SIijmYVEazKUZO nys01q30RuNXIJwRo9Gq9M5Pzi/Ns0p/ZlbIRpXR6HXl5DWEmRCM/3V8VM5ms8ps VJkcD16UdNKlUo2Gqj7rXcxeV4f9yrO51nhBNOfhEmHXIwMIysPYnDGWRFcQGDzm cz9ETSBR6loDAQdCVBhRbbTY6PhAsW6x49Zkkko9ammt9lELVHDRI0nlhlbk2nCo 3jUSE5lNLv+wT5WBvh5mz16R38Ln9Jr6grnxV4KpVd2g/ezL4DMPEXxycu0xcqxr 43CX/mbuaIjtDN9+eq8qlh256iNet8uX003CO5jlGCUkxvcX7j13DrYdlLirC0sk ZSc1mnC0RrPspCg8s9mvKOvGpAMUnlPqpGfS0ahlbPDOF0zGYSyc2q7S55wUlEop dpQSt9xJWLkXmxEpfi6VJYqy8iUqROh5IeAMIbU66Q3ZPHadJS/mtRc0r71YbPEV bJIjFVKqlCelaqXhLaEA53CK4cCo5jySRfs2gZrC/WKd5KaejutkQ3aRlw2osQME VPKBtS9UIV6vae0NCjAAxnPFVrUFeSptYaepqtl7/NQW+CP/YVq7pKZWqJkrMiDX QMXCLMfBrhIdpqodQ+002p222rGsjsGlCfR3m0Bvb9tA/zYb6IqVWLnVR1kmZ+22 +bqpNjSZ97WmWkr9LwqAjQ3AxmEO8LnyFdMPS2o0Ci3kPFILKiWwSwPUdGrmMZkC RGDsEGi7BI0dAn2XoLlDYGwREGLjPRxI3bae8W3WM9ajaIjQwK0q7NVptyzpCIL2 UQY6MeVMDCSWfU37m/vad/Y8R8s9z9GMwvNstVgYcsjC847KnrfBZ2SavNv1Foor LLmnyW5Ds/uY3bSS3UAuNxdR5iaTEnKHK+AXVeZlf9QFoi1vy5FqCwKkLQ4r6r2x aOFTO8yga4fy3SG9W+S4VXWxrZRWKIW51bc4oUYE2lucUMu11x9o/9heybXnutQL X6QrN7InQ32uGPTcyJ4b2Wgze2qqcqyVPbV2EellRK23IJL77vAtiPR1zEW95iIj Y60qzBwd05JtTWk87shZuvmaJKBWjH2I+Eu0AqVcAOelUa3VbKrdKjqYCF0Y6jLW G4+kiQpDGTIhGLnLNtjCs9BlYbHVXNpzpZCXrCI4spRJb7T3m0GXM+j5DMaWktq2 UCtBPqea4tlLvpIJu7S2aHDaakM/KAS3VCxXq5HttVK2JCGZx7G1mBclomwOqiBU snZGXnB9cTrsXsyq5+Pey0H/SyXys7SebyVVCWN/vcXGsrVn0/FILpsbZuHAzmOB TSG64BMSVnZ8se6gJdvG4Oo2McIa2ZfdunGCYmPNBFeRLLL+g1+/UG6XcoOhtoU7 2YxUW4fE9SLbpJDhLAolrVzJ+aKs5FYFhMK2XPtQja0bR4aulsuftrrFJVD12DfU u6N0RGQCJl9wGx5Ltu31YSv2rNev6npjUNdqR5mWFCGqPmotTEsRyTAUy9Gw37Uj 9bBNPfOzs/7sUNebuto/7xptlb1qNMbdo+OCv7Xmz2MCCWi8W0CrEHC4K+B0MB5d jYbT6XjaYf1sSy59BBsU0XQItu86WAIU0JqhHiKCoKn+FpYOnaDcunTAR7CwvxOk GerT3ouh8eEMFrLYrfX+HPqHcfhuHIdxnazhWWlAQammzOKVPMMIUW/DF2QXCfcJ fZYuRKeOoFbf8AnNUj7rTi+GF6cdVpcnhnR6lJ00eSs67JRSuFMiu7Xit1ISQSef PeIBncsyTjN22NwL7RvBjJY8FcGPLAV7bnDDRALcMk4v6WwzZJeTR84CKGTL84+q d/vYWQEd7M3VT2Vgp397sgeVm1Rt0WZjNuKQfG5qbVVrOkeQ4Ebyjd48bLfVdgst /O1zJWPS2bUTZz9V2ZbfAlIYs6xxe5plwl4YY4vLE7ankijPHTlRW28fNif1waGO cN6fdkdU46zDaT1rEs7SuWyx44Vl80xI1j7nQrKSQ74X6TwbovdnY3NGtVzWIWTA jQfAG2Xgav63BXxeADf6z5Vt4Nr3ApwqSCxrD3HxWIL5AJR5wZr1ABlIvf0+KJtb KC3+luX9vVGu66CtJSSopPXr7wbWyFf0oS+/a0kbjQdgre9nSVXtuFvvdev9Y/x3 Xh8c1wfT+skx2x/2zvS6UW/gXxP/WgcVtrHNc+WhdX5/28h6OreN9sA22rvc3dJ+ iH2+Bns+6ZWg/l4YUT2su4UCpbaFUlO1tvX2aGbpPyTKYX9QimOP4PPpt0gg6jGU yO/rjqJAqD9AuHgHQuOHRGhmfQH1D98RaIC+YOOuxhZMG0HqHWnJavygMKnleTvC TYf0EJ9kp2OcT2Vvu7sb9e2sq6FKk2AKeMWbLPlQawuG7yPrNhuHgyE7dZcWHQQW jf5bkebXZg9h8px9fWr0qeyFGztotZ0aQ21ommbNN2iLN6XFNN6Odn04ZXKfah+6 AcPg/meW6LBT1P8eD9mM29dB6IXLFTtPnIPH7DF6VZxuNaqyI5GXW1mS+g7+bfbM oXRwGamMztGOSYx3lV2NwybqA7j0pvrQD99uku5suIHsopIcBvZjmKfQmL0+J5CP 4MqPrh5CQt9GlWmGR9vNvQ89uvXAo1tlj36u3P+/9uhHyuyXNOLl14mPFdq+BbgB 39tzjXaLDs339oatduuqN7lU0J0Eyd5eficF0vsUAUTstRpKBmhvL4juFeUT5ma3 nPKKUd4Tx9wPb1H/ixAO5a06Sii7e7G3B8FXJ11zNpheNV9johNMBAmzUHYOdNGC NoQOQLAf6M40N92dG8ubyABdpeXQtV19jpaong3XgCERyifya2/vyengYjAd9rLX T/b2PkHntbBSD52MZ9EdKeoWLwyzu8tMhKhBDd+64WtN+4Pjy1PYt7ok/uPU9Rx2 k5lT3psunfm+dgBuuhEWK38eegIicna2Z/bOBv2ry/MBseOLCersUiyuskPTODb7 RETfj1FNpoPBaDIbji+IahDQleBaleSarqfRwHHuS/oN2/BiMNsjDlnD4IHcB0Ys 6UgkrYxmctsis/tpgDXIhtEVJqFNsAqGkxNTkh/z+IZ7fMVO0OazE+oC5VlMCdr4 ZHY56XdnA7OkNfjhE4uEpZGDPhHhKI0ohm34Lk/Mq27vXM5iZoNylSwbayTWB1jo NUUitrn6w+lZ1zyTaHzqrjnd1oHXtwJ4UBiwubtEsIg5BV5Em5IhRv2r6XicmWvU p4tui0VhQlekyJey+c7cZDPlxYnZOx8OLmbEsz703liC9TwX7Fv05mD6ajD9FnqT bpvjkk5gyJUCPeyWCmlCtOz1CqODJCARrFBjM9PI7I/NfKHk70cXqNc/arXUbPHN MaOHR+km03EvFzaJQ7kIi43W+4UiE3Nw2R+fmAcl1vyV5BU8dcJqiXURWz6XIa1g kIclpxMJ+fRyiBraihNXngvOCDw26Ub/8WjSnV01DCLubU5M5eakndSoGeyLl4PB hM3OhuZPv3zAeoJ9BcLGYwJOsKFo7EOC267g5lsEN8tBwBxe9Qfn3c8RbejWLYtX yGhsHwHQFwdszuHGnDbknKKgTNUF/8vZtNsbyAW6SWKENwpLD/aV+bn5yjwbZfsK v6siWUEncW3FgJJdT29Tj8zTXWofq0+p+KuUp1zsCB88FM59K7qG5uLDzHj1cjI2 h6+vJtPheDqcfX4l4+Tl+fDilMGTaIxNUGoYV9oxxFhelY6VGb/HhqWLppJmL4/7 V8MLc9Y9P7/q9QevoCDlEGQZ7HB6wbBJ4xVlGsolH6Zn96x3NR2cQs/BDFpCXUQD 2iYundjxpSuPQJHiFy73HJGlM6QLZW9PmipMkyhNaox1HYz+laa3bygzZTcUtQ/V pf996qJrzS1dNvP0u0hFrwZX8mRfblTXwhR+mvB7GTgdC7S3HPl0XSVYkWtngaY+ Zt3JsEcZn5qV3EtrBaUrrOI3Lz+ghCOmEy+MolWmFZxqPbpwbBqlOtgKHPqeDNd5 faNFYpV/o2q8IZ2IiX4XQt/D6iUxseU6azHTLkJWLqZEAvRrEqiVHdvvIpBkixLZ 4nGkkk4kGzo88zXVZpFm3Sv4/GzYuxr2aU/K0mpdSwWpP+dUTpHN8gucdWIVbB4i SqGkRIWRXxqRSenFxKzrdCAseAkdcWfabBg2t0zbhDt0xSCG/PReBrC1CB/Fmht5 /L4k43ZpEQ3d5NDJeShr99xDFcKCxELXOuSFpK+T13z5VU9OSbkTgWnuUSS1AgTT 8ZqimElId8prFukCVIKQzMi+Tdj+q5muq2xzTXewM8dG5aSU0F9v8r8oVTavqS2n K0m6TmO3hb4lzhMkld7l1BxPiRdOl2TJRB7EMzuNRRi/Xx1erMcyKush6I6JzkhO J7L/ETyhYpZNwju6obACRH0fkbJg2Becs4sxqjuJwaf0lKt78GGKRD7tXAdDqaB7 hjqWJ0U0L5uILO9S2/R+QbEIGtl1IwHJz4D2kaEP5D3eZociY5ApIF6tMK31RaP5 ZVNlc0uQyIwtYyB79Iabdu1i2BOb0MX9da3tMdRMdTpCyIIhLPiw1SP/f0wgIFko jwWXoKkih1uMhkM6tClf39qhw8katAYd1OOwVyzPY284jyTv03y7+64L3qd0PSM7 qjCmWydQ5pPgkyb+aYEkY5DVYz7vupoo4u19tAEr1R/cowVBaZgjP2b7sGfzsEJm bbYPNsaWMddciZdhEKAQZ+bL6lG7cc/+NPul37Pljq0K3nuPeA1UVMywj9R7tv/m zXEYYoWtYPn0aYW9edNb2R6c4OnTA2lcWXNuGry1ILR+UbkKp+e5Zd8UBBDohH7W t8DgFIez6r9YbNJNjq/Xbm2iD/FPIQHlPbp5Ppx8mHdHUYl9MpkUI0kayJIbkKAZ nkCwQR8lq1JBniQrwfaxjgSCJ/ZmqXwZr0eyMmRPKEWKJ7JVh2+9mUeLp+t8wmWc zGJu0Rjmk6OjSLJaBi5q3VkUJhaS0nJ8N0Aqo3OSWy6DHloJHlB7DhIpVF6Bwgg/ JUdHO5Ztjqdycmka2Xs4Mkb0z3qTDUiQbPWpUabOQqqzqUyiRemnFy5LT2IVIAsU 5xbd89lfbD9d9Y5330wH/d03w/HOmzM0bDuvUKztSroYT3q9jZ7I8iVd8jYD9fDl 6w8M+3bkbqQGqectNoFwyUP/KrtV3n7nWXPuleZH2TzsDa4m43MqyR87d1KOV7yi KKc8XmJ+1vs6tb9eKb4F84cdtsxe12z5+lO0ID78R9SuUwXCnuuq8hnnAZ0HJlyg Jj3FA4IV4lDo3WZtkMf9PFRasSsQCf/mzz/+o48Gr3788e9+/ZsvfvaLb375kx/d /dtHvzr/2+Cb/71TPn7z7/9884s//O1f+2/+ofPRr269xm//de+//v7Ff/a/+e/2 P/3yP/6n+yf/Bw== =jd+e -----END PGP SIGNATURE----- --TRYliJ5NKNqkz5bu--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20061111172731.GB39999>