From owner-freebsd-net@FreeBSD.ORG Mon Aug 8 21:31:46 2005 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ADA9016A41F for ; Mon, 8 Aug 2005 21:31:46 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [204.156.12.53]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5B6F843D46 for ; Mon, 8 Aug 2005 21:31:46 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by cyrus.watson.org (Postfix) with ESMTP id 908F446B13; Mon, 8 Aug 2005 17:31:45 -0400 (EDT) Date: Mon, 8 Aug 2005 22:34:53 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Ed Maste In-Reply-To: <20050808190024.GA4843@sandvine.com> Message-ID: <20050808223106.P67598@fledge.watson.org> References: <20050808190024.GA4843@sandvine.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-net@freebsd.org Subject: Re: Multicast locking LOR X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Aug 2005 21:31:46 -0000 On Mon, 8 Aug 2005, Ed Maste wrote: > I built a vanilla (i.e. without local patches) kernel with > the recent multicast locking changes, and got the following > LOR: Ed, Could you add a hard-coded entry to WITNESS to place the udpinp lock before the in_multi_mtx in the lock order, and let me know which path resulted in the opposite order from this one? In theory, pcb locks should appear before the inet multicast address lock. The only case I'm aware of right now that might cause problems is IGMP's call into ip_output() to send an IGMP message, and ip_output() could hit ipfw/ pf/etc acquiring an inpcb lock. There might also be recursion back into ip_input() due to multicast, which is the case I'm curious about. I had been wondering if we need to defer the ip_output() call for IGMP to a separate worker context, and assuming you're hitting a case related to that, I think that we'll need to. There was already a warning about potential recursion there in the IGMP source. Thanks, Robert N M Watson > > Lock order reversal > 1st 0xa26ffaec inp (udpinp) @ /d2/emaste/cvs_mcast/src/sys/netinet/udp_usrreq.c:762 > 2nd 0xa07ebf60 in_multi_mtx (in_multi_mtx) @ d2/emaste/cvs_mcast/src/sys/netinet/ip_output.c:294 > KDB: stack backtrace: > kdb_backtrace(0,ffffffff,a07ab3b8,a07ab480,a075e1e0) at 0xa0586a95 = kdb_backtrace+0x29 > witness_checkorder(a07ebf60,9,a0729936,126) at 0xa0590c60 = witness_checkorder+0x564 > _mtx_lock_flags(a07ebf60,0,a0729936,126,0) at 0xa0566610 = _mtx_lock_flags+0x60 > ip_output(a2706a00,0,c8463b08,20,a26a9500) at 0xa05f7d8a = ip_output+0x3fe > udp_output(a26ffa5c,a2706a00,0,0,a29b9900) at 0xa060754f = udp_output+0x4a7 > udp_send(a29b2ca0,0,a2911900,0,0) at 0xa0607abe = udp_send+0x1a > sosend(a29b2ca0,0,c8463cbc,a2911900,0) at 0xa05a8287 = sosend+0x5e3 > soo_write(a26d79d8,c8463cbc,a2605800,0,a29b9900) at 0xa059810a = soo_write+0x46 > dofilewrite(a29b9900,c,a26d79d8,c8463cbc,ffffffff) at 0xa0592903 = dofilewrite+0x77 > kern_writev(a29b9900,c,c8463cbc,842b12a,0) at 0xa05927a7 = kern_writev+0x3b > write(a29b9900,c8463d04,3,0,282) at 0xa05926cd = write+0x45 > syscall(685a003b,804003b,9fbf003b,821c400,f6) at 0xa06c75b3 = syscall+0x25b > Xint0x80_syscall() at 0xa06b4ccf = Xint0x80_syscall+0x1f > --- syscall (4, FreeBSD ELF32, write), eip = 0x681e31af, esp = 0x9f9ff9fc, ebp = 0x9f9ffa18 --- > > -- > Ed Maste, Sandvine Incorporated > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >