From owner-freebsd-current@FreeBSD.ORG Thu Sep 9 18:08:53 2010 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 943AF1065670; Thu, 9 Sep 2010 18:08:53 +0000 (UTC) (envelope-from weongyo.jeong@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id EB5AE8FC0A; Thu, 9 Sep 2010 18:08:52 +0000 (UTC) Received: by fxm4 with SMTP id 4so1297214fxm.13 for ; Thu, 09 Sep 2010 11:08:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:mail-followup-to:references :mime-version:content-type:content-disposition:in-reply-to :user-agent:organization:x-operation-sytem; bh=M+IKw3sJL0pgZ7n5k/U4zWRx6B8xly5A8jvke2vTaXA=; b=tDR9ld0C9ehZEx1vDInOW+1ecaGUDpLg8m8UMfMXTjMc3BqlJdLz8ISm01CnatnBFe atZ4YEJl3MCIDPURfFj3A7C1jXHQ8aguQ5yElNZ1tZYgKa3TLVZvYrtqN8Nvgqut0ICl kc0OgWtjEhfUnCU8z302XiBOyyasBLDGOmjqk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:mail-followup-to :references:mime-version:content-type:content-disposition :in-reply-to:user-agent:organization:x-operation-sytem; b=EFrJwzadxM7vosmURzALG5U+zYO2mAH1bu1nkJNPr+bLiPTtWau1Lq6+Hxy9wg2B8p wg3s7XH4QOf0OC8N8fTaVituZButQmBm8AedgXqASyxtTddKUseP6vPxYHXvn3B5x9a2 pd2OGJjhqYG7mk9qR9MASAOf7OlsPBJTeV5Gs= Received: by 10.223.121.7 with SMTP id f7mr91370far.13.1284054106236; Thu, 09 Sep 2010 10:41:46 -0700 (PDT) Received: from weongyo ([174.35.1.224]) by mx.google.com with ESMTPS id b36sm848601faq.11.2010.09.09.10.41.43 (version=SSLv3 cipher=RC4-MD5); Thu, 09 Sep 2010 10:41:45 -0700 (PDT) Received: by weongyo (sSMTP sendmail emulation); Thu, 9 Sep 2010 10:41:46 -0700 From: Weongyo Jeong Date: Thu, 9 Sep 2010 10:41:46 -0700 To: John Baldwin Message-ID: <20100909174146.GG1328@weongyo> Mail-Followup-To: John Baldwin , freebsd-current@freebsd.org, current@freebsd.org References: <20100908201419.GF1328@weongyo> <201009090936.19412.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201009090936.19412.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i Organization: CDNetworks. X-Operation-Sytem: FreeBSD Cc: freebsd-current@freebsd.org, current@freebsd.org Subject: Re: about in_multi_mtx @ netinet/in_mcast.c:1095 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Weongyo Jeong List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Sep 2010 18:08:53 -0000 On Thu, Sep 09, 2010 at 09:36:19AM -0400, John Baldwin wrote: > On Wednesday, September 08, 2010 4:14:19 pm Weongyo Jeong wrote: > > Hello, > > > > I have a question about IN_MULTI_LOCK() because it uses MTX_DEF flag > > when it's initialized so I always encounters the following LOR > > > > lock order reversal: (sleepable after non-sleepable) > > 1st 0xffffffff80d0b560 in_multi_mtx (in_multi_mtx) @ > netinet/in_mcast.c:1095 > > 2nd 0xffffff00014e3850 USB device SX lock (USB device SX lock) @ > /usr/home/freebsd_usb/sys/modules/usb/usb/../../../dev/usb/usb_request.c:441 > > KDB: stack backtrace: > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a > > _witness_debugger() at _witness_debugger+0x2e > > witness_checkorder() at witness_checkorder+0x807 > > _sx_xlock() at _sx_xlock+0x55 > > usbd_do_request_flags() at usbd_do_request_flags+0xe5 > > axe_cmd() at axe_cmd+0xc7 > > axe_setmulti_locked() at axe_setmulti_locked+0x70 > > axe_setmulti() at axe_setmulti+0x3e > > axe_ioctl() at axe_ioctl+0x132 > > if_addmulti() at if_addmulti+0x19b > > in_joingroup_locked() at in_joingroup_locked+0x1bc > > in_joingroup() at in_joingroup+0x52 > > in_control() at in_control+0x1144 > > ifioctl() at ifioctl+0x1118 > > kern_ioctl() at kern_ioctl+0xbe > > ioctl() at ioctl+0xfd > > syscallenter() at syscallenter+0x1aa > > syscall() at syscall+0x4c > > Xfast_syscall() at Xfast_syscall+0xe2 > > > > when I uses the following code at driver's ioctl routine: > > > > case SIOCADDMULTI: > > case SIOCDELMULTI: > > axe_setmulti(sc, 0); > > break; > > > > It means that USB driver always should defer SIOCADDMULTI / > > SIOCDELMULTI handling to the other process context to avoid LOR. > > > > My question is that is it safe if the multicasting operations for USB > > device happens without IN_MULTI_LOCK? Or is there any race cases if the > > task is deferred? > > Why is USB using an sx lock instead of a mutex? Frankly speaking I also don't know why hps@ uses sx lock. That is one of things I'd like to change it. Just looking the comment at usb_request.c@441: /* * Grab the default sx-lock so that serialisation * is achieved when multiple threads are involved: */ sx_xlock(&udev->ctrl_sx); I think he might want to hold the lock even if the thread is going into sleep. It might be for serialization. However even if we succeed to change the lock from sx to mutex, it's hard to avoid the requests going into the sleep. It means USB stack should call like below: mtx_sleep(chan, IN_MULTI_LOCK, ...); to avoid the kernel's complain (would be `sleep with holding non-sleepable lock'). What I'd like to say is that the sleeping is big problem if mutex is used that it'd be worse when multiple mutex locks are used. So I'm looking for a fundamental solution to solve this problem. Welcomes any ideas. regards, Weongyo Jeong