From owner-freebsd-stable@FreeBSD.ORG Wed Nov 22 20:01:50 2006 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D39DC16A417 for ; Wed, 22 Nov 2006 20:01:50 +0000 (UTC) (envelope-from quetzal@zone3000.net) Received: from mx1.sitevalley.com (sitevalley.com [209.67.60.43]) by mx1.FreeBSD.org (Postfix) with SMTP id 10FF043FA6 for ; Wed, 22 Nov 2006 19:56:38 +0000 (GMT) (envelope-from quetzal@zone3000.net) Received: from unknown (HELO localhost) (217.144.69.37) by 209.67.61.254 with SMTP; 22 Nov 2006 19:56:21 -0000 Date: Wed, 22 Nov 2006 21:55:49 +0200 From: Nikolay Pavlov To: freebsd-stable@freebsd.org Message-ID: <20061122195549.GA57018@zone3000.net> Mail-Followup-To: Nikolay Pavlov , freebsd-stable@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-Operating-System: FreeBSD 6.1-RELEASE-p10 Subject: deadlock in "zoneli" state on 6.2-PRERELEASE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Nov 2006 20:01:50 -0000 Hi. It seems i have a deadlock on 6.2-PRERELEASE. This is squid server in accelerator mode. I can easily trigger it with a high rate of requests. Squid is locked on some "zoneli" state, i am not sure what it is. Also i can't KILL proccess even with SIGKILL. In addition one of sshd proccess is locked too. Is there any additional information that i could provide? last pid: 1197; load averages: 0.00, 0.00, 0.00 up 0+01:54:58 14:46:40 31 processes: 1 running, 29 sleeping, 1 zombie CPU states: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 704M Active, 629M Inact, 447M Wired, 12K Cache, 112M Buf, 2109M Free Swap: 4070M Total, 4070M Free PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 671 squid 1 -16 0 688M 688M zoneli 6:32 0.00% squid ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 680 root 1 96 0 6628K 4760K select 0:02 0.00% snmpd 1170 root 1 96 0 2332K 1588K RUN 0:00 0.00% top 698 root 1 -8 0 7768K 7288K piperd 0:00 0.00% perl5.8.8 634 root 1 96 0 2984K 1808K select 0:00 0.00% ntpd 362 _pflogd 1 -58 0 1600K 1144K bpf 0:00 0.00% pflogd 1097 quetzal 1 96 0 6220K 3220K select 0:00 0.00% sshd 709 root 1 96 0 3464K 2796K select 0:00 0.00% sendmail 1100 root 1 20 0 5036K 3064K pause 0:00 0.00% tcsh 551 root 1 96 0 1352K 996K select 0:00 0.00% syslogd 1085 root 1 4 0 6232K 3204K sbwait 0:00 0.00% sshd 1095 root 1 4 0 6232K 3204K sbwait 0:00 0.00% sshd 1088 quetzal 1 6 0 4724K 2952K ttywai 0:00 0.00% tcsh 719 root 1 8 0 1364K 1060K nanslp 0:00 0.00% cron 1098 quetzal 1 20 0 4704K 2932K pause 0:00 0.00% tcsh 1087 quetzal 1 -16 0 6220K 3220K zoneli 0:00 0.00% sshd ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 654 root 1 96 0 1264K 804K select 0:00 0.00% usbd 692 root 1 96 0 3504K 2656K select 0:00 0.00% sshd 713 smmsp 1 20 0 3364K 2728K pause 0:00 0.00% sendmail 358 root 1 4 0 1536K 1092K sbwait 0:00 0.00% pflogd 769 root 1 5 0 1320K 896K ttyin 0:00 0.00% getty 773 root 1 5 0 1320K 896K ttyin 0:00 0.00% getty 772 root 1 5 0 1320K 896K ttyin 0:00 0.00% getty 771 root 1 5 0 1320K 896K ttyin 0:00 0.00% getty 770 root 1 5 0 1320K 896K ttyin 0:00 0.00% getty 775 root 1 5 0 1320K 896K ttyin 0:00 0.00% getty 774 root 1 5 0 1320K 896K ttyin 0:00 0.00% getty 776 root 1 5 0 1320K 896K ttyin 0:00 0.00% getty 497 root 1 114 0 528K 388K select 0:00 0.00% devd 128 root 1 20 0 1228K 680K pause 0:00 0.00% adjkerntz Also there is some interesting fstat info: root@accel1:~# fstat -p 671 -v | head -n 40 can't read vnode at 0x0 for pid 671 can't read vnode at 0x0 for pid 671 can't read vnode at 0x0 for pid 671 can't read vnode at 0x0 for pid 671 can't read vnode at 0x0 for pid 671 USER CMD PID FD MOUNT INUM MODE SZ|DV R/W squid squid 671 root / 2 drwxr-xr-x 512 r squid squid 671 wd /usr 1908230 drwxr-x--- 512 r squid squid 671 text /usr 1887228 -r-xr-xr-x 638296 r squid squid 671 0 - - error - squid squid 671 1 - - error - squid squid 671 2 - - error - squid squid 671 3 - - error - squid squid 671 4 /var 47121 -rw-r--r-- 2935342 rw squid squid 671 5* internet dgram udp c96205a0 squid squid 671 6 /var 47131 -rw-r--r-- 48909168 w squid squid 671 7* pipe c9551198 <-> c9551250 3 rw squid squid 671 8 /cache 7 -rw-r--r-- 91506636 w squid squid 671 9* internet stream tcp d2f17ae0 squid squid 671 10* pipe c9551a48 <-> c9551990 0 rw squid squid 671 11* internet stream tcp c971e3a0 squid squid 671 12* internet dgram udp c9620000 squid squid 671 13 - - error - squid squid 671 14* internet stream tcp squid squid 671 15* internet stream tcp d6b211d0 squid squid 671 16* internet stream tcp cf29c740 squid squid 671 17* internet stream tcp d0c9cae0 squid squid 671 18* internet stream tcp c9ebc570 squid squid 671 19* internet stream tcp d49c9000 squid squid 671 20* internet stream tcp d262eae0 squid squid 671 21 /cache 4031491 -rw-r--r-- 2037934 r squid squid 671 22* internet stream tcp ca1941d0 squid squid 671 23* internet stream tcp d5330570 squid squid 671 24* internet stream tcp d53fe000 squid squid 671 25* internet stream tcp d53a83a0 squid squid 671 26* internet stream tcp squid squid 671 27* internet stream tcp d2657910 squid squid 671 28* internet stream tcp d964b740 squid squid 671 29* internet stream tcp squid squid 671 30* internet stream tcp squid squid 671 31* internet stream tcp ca1d31d0 squid squid 671 32* internet stream tcp d83203a0 squid squid 671 33* internet stream tcp ca24b910 squid squid 671 34 /cache 3984782 -rw-r--r-- 1981893 r squid squid 671 35* internet stream tcp d7c0b3a0 -- ====================================================================== - Best regards, Nikolay Pavlov. <<<----------------------------------- ======================================================================