From owner-freebsd-geom@freebsd.org Fri Mar 23 10:38:49 2018 Return-Path: Delivered-To: freebsd-geom@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B24DF5DCBE; Fri, 23 Mar 2018 10:38:49 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf0-f50.google.com (mail-lf0-f50.google.com [209.85.215.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9DA876C543; Fri, 23 Mar 2018 10:38:48 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf0-f50.google.com with SMTP id o102-v6so17555142lfg.8; Fri, 23 Mar 2018 03:38:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=RnkJwCPMB8DlCuQU/E3+vvPcKP34ZPwz99FZCPBfc5c=; b=fhZXjHrLMS6SluBMOJu7DGQEm0zjmpPithEAssHBs4f58mDbyVDlP1yuftHL0C4QM4 oqa5hQT3gSs45DXyHzhI+fYrneepWhZ4INxX5QC7qVABUIeDKR4W+73+bxfdkrZmq1h6 au1hfva9iK/0v3qnMrVBZ3M1P6zOLQ6B3cu7uNiF6B2Zot6z4sxAsAa5q/u9dlr325Eu 3UeDlWZxULq+z6LqjZLhdgeAliWinl51pluMjBA9oT/YmTdvlGCVAO1aR9nQVOkMNG6d DRsMb7WpriQi+7hXf/4DdJ6bxLXQ1vvL74S3OsJxuBBbshvMHKiO5ngR0+9Ugzjx5LkI ZhoA== X-Gm-Message-State: AElRT7GNLQIs+fiW5eVxZoEiEx2KLGRqWGqXSeLcTKads72GFC2xjzAO hrLh1kFrQUD5YRewNES5OAzwdJcTbb8= X-Google-Smtp-Source: AG47ELudZztuJ+KMhwsk6ykow8i8drgXxR2h2TslPx9uij9f0syZ2JYJwqaDPyFJjGgcdQI4fMWVaQ== X-Received: by 2002:a19:1754:: with SMTP id n81-v6mr16898162lfi.113.1521801526843; Fri, 23 Mar 2018 03:38:46 -0700 (PDT) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id a9-v6sm2100794lfk.80.2018.03.23.03.38.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Mar 2018 03:38:45 -0700 (PDT) Subject: Re: geom->access problem and workaround To: Poul-Henning Kamp , Warner Losh Cc: "freebsd-arch@freebsd.org" , freebsd-geom@freebsd.org References: <809d9254-ee56-59d8-69a4-08838e985cea@FreeBSD.org> <56619.1520878022@critter.freebsd.dk> From: Andriy Gapon Message-ID: <5e416eb6-0e79-1419-f09a-eb747215dc28@FreeBSD.org> Date: Fri, 23 Mar 2018 12:38:44 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <56619.1520878022@critter.freebsd.dk> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 10:38:49 -0000 On 12/03/2018 20:07, Poul-Henning Kamp wrote: > If we want to have an architectural sound way to do slow operations > before any "user-I/O" is initiated, the right way to do so is to > define new BIO_OPEN and BIO_CLOSE operation, and insist via asserts > than all BIO_{READ|WRITE|DELETE} are wrapped in these. In support of this proposal I want to add another example. The problem is not only with doing I/O in access, but also with doing blocking I/O in the normal I/O path. The following happened when a userland program tried to read from a CD-ROM while its tray was open: panic: sleepq_add: td 0xfffff80008e1c000 to sleep on wchan 0xfffff801e58b8048 with sleeping prohibited cpuid = 0 curthread: 0xfffff80008e1c000 time = 1521652565 KDB: stack backtrace: db_trace_self_wrapper() at 0xffffffff804670bb = db_trace_self_wrapper+0x2b/frame 0xfffffe07e1886750 kdb_backtrace() at 0xffffffff806d4c79 = kdb_backtrace+0x39/frame 0xfffffe07e1886800 vpanic() at 0xffffffff80699da6 = vpanic+0x166/frame 0xfffffe07e1886840 kassert_panic() at 0xffffffff80699adb = kassert_panic+0x16b/frame 0xfffffe07e18868b0 sleepq_add() at 0xffffffff806e1902 = sleepq_add+0x342/frame 0xfffffe07e1886910 _sleep() at 0xffffffff806a41f7 = _sleep+0x2b7/frame 0xfffffe07e18869b0 cam_periph_ccbwait() at 0xffffffff802b53ad = cam_periph_ccbwait+0x5d/frame 0xfffffe07e18869e0 cam_periph_runccb() at 0xffffffff802b51d8 = cam_periph_runccb+0xb8/frame 0xfffffe07e1886a40 cdrunccb() at 0xffffffff802d52fc = cdrunccb+0x3c/frame 0xfffffe07e1886a60 cdprevent() at 0xffffffff802d4beb = cdprevent+0x7b/frame 0xfffffe07e1886aa0 cdcheckmedia() at 0xffffffff802d48b2 = cdcheckmedia+0x22/frame 0xfffffe07e1886af0 cdstrategy() at 0xffffffff802d2a36 = cdstrategy+0x56/frame 0xfffffe07e1886b20 g_disk_start() at 0xffffffff80608330 = g_disk_start+0x170/frame 0xfffffe07e1886b70 g_io_schedule_down() at 0xffffffff8060c1db = g_io_schedule_down+0x1eb/frame 0xfffffe07e1886b90 g_down_procbody() at 0xffffffff8060cbdd = g_down_procbody+0x6d/frame 0xfffffe07e1886ba0 fork_exit() at 0xffffffff8065e410 = fork_exit+0xd0/frame 0xfffffe07e1886bf0 fork_trampoline() at 0xffffffff808e4aee = fork_trampoline+0xe/frame 0xfffffe07e1886bf0 But maybe I am overgeneralizing. Maybe it's just silly that cdstrategy tries perform media manipulations instead of just failing a request. Apparently the device was already opened, but the prevent-allow bit was changed by something else (e.g. via the pass device) and that allowed to get the tray opened. What do you think? -- Andriy Gapon From owner-freebsd-geom@freebsd.org Fri Mar 23 10:47:42 2018 Return-Path: Delivered-To: freebsd-geom@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5BE93F5E928; Fri, 23 Mar 2018 10:47:42 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id EEF9B6CDD9; Fri, 23 Mar 2018 10:47:41 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.55.3]) by phk.freebsd.dk (Postfix) with ESMTP id C993C273B0; Fri, 23 Mar 2018 10:47:34 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.15.2/8.15.2) with ESMTPS id w2N9U4NX000890 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 23 Mar 2018 09:30:04 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.15.2/8.15.2/Submit) id w2N9U3gJ000889; Fri, 23 Mar 2018 09:30:03 GMT (envelope-from phk) To: Andriy Gapon cc: Warner Losh , "freebsd-arch@freebsd.org" , freebsd-geom@freebsd.org Subject: Re: geom->access problem and workaround In-reply-to: <5e416eb6-0e79-1419-f09a-eb747215dc28@FreeBSD.org> From: "Poul-Henning Kamp" References: <809d9254-ee56-59d8-69a4-08838e985cea@FreeBSD.org> <56619.1520878022@critter.freebsd.dk> <5e416eb6-0e79-1419-f09a-eb747215dc28@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <887.1521797403.1@critter.freebsd.dk> Content-Transfer-Encoding: quoted-printable Date: Fri, 23 Mar 2018 09:30:03 +0000 Message-ID: <888.1521797403@critter.freebsd.dk> X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 10:47:42 -0000 -------- In message <5e416eb6-0e79-1419-f09a-eb747215dc28@FreeBSD.org>, Andriy Gapo= n writes: >On 12/03/2018 20:07, Poul-Henning Kamp wrote: >> If we want to have an architectural sound way to do slow operations >> before any "user-I/O" is initiated, the right way to do so is to >> define new BIO_OPEN and BIO_CLOSE operation, and insist via asserts >> than all BIO_{READ|WRITE|DELETE} are wrapped in these. >What do you think? I don't see that changing anything... GEOM rests on a set of assumptions, if you violate them, you get panics. -- = Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe = Never attribute to malice what can adequately be explained by incompetence= . From owner-freebsd-geom@freebsd.org Fri Mar 23 13:26:22 2018 Return-Path: Delivered-To: freebsd-geom@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EE7ACF6C2B7 for ; Fri, 23 Mar 2018 13:26:21 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x232.google.com (mail-io0-x232.google.com [IPv6:2607:f8b0:4001:c06::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 87E8873CF8 for ; Fri, 23 Mar 2018 13:26:21 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x232.google.com with SMTP id y128so15156397iod.4 for ; Fri, 23 Mar 2018 06:26:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=ugh28Ml5qpKKmSDdaCpdU+52DDJneW1msRSYr9yW60Y=; b=USaYwdcmbdBUAjTgZ1T1k2A3ewI99nFLKk2iELNp4+WtP6lKqm5+eSCwFoHJRiqH7r jgjH64GrO8dvT4/X4mIVZtg4DaWRD969ONAGO23mMUr6AKw4Fo1vNfqIqeSdnVCzH9N0 IEUNsce6KWqlIFQmywG1eGCKSxHiqpLxc6wd42iTTOHG6O48BT++KhfjQoudfsztQBcp rqBIV+MLBL48oAxVXXsYB/P8q5rCRC07ay6ilBBO5/sQeUms2W8mJZk+uMajJ6uMQTQB OXZHIzLdV5lySPiFDOiyUt1SKoOa7XEdOcqlNLzus17etj4luHKPxwC4z0Nj4bPW5+ZF a3SA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=ugh28Ml5qpKKmSDdaCpdU+52DDJneW1msRSYr9yW60Y=; b=Y2vLy61h3fRcEEIUGq3iArjpmDGX3cdgQA4asA7WUPNWflW8q0DdUjCkDQVdw9lMg+ lgwu7GXIukglO0leU3tyfHlV36dnA7GPwTlVZW01Do0o/VrDb9ZIyzPkkSWwLBbcudNc 6Rbz0eTcnZLRPbxQX55PT8mSah/rBhfYQIIXygUlT19swDiVOBFd7c6s5deyjo7iJ+0G RuhE26E4T8RKdCCj4HjJGqyOAu52oJ5Jwtb0rWoc/3u8Fs+8rSpGa1HEMYwlgR08qoUM bztr/bQpOLWuwHYeVcK5wENqMx51SMff7LSgii59WtNYsFl0pE4Dn6gvqS/RgS+sF6+B NJkw== X-Gm-Message-State: AElRT7Gpe+cb+pjO8qH6zJm+FbfifjL4O7YEMpsawywir7sA2gmbKLMb AdMY/WPdcM9wnQjnCcb94E3QDzoz9rltFlsZlCdNgQ== X-Google-Smtp-Source: AG47ELtCk3iCOOuDXu2mLkifesZ0MzlzShtKwTfvrFXjdQRN8Ff7fmBO7ND//xvJf+6Gt3MewFBIbz/UFgp6KJr/6kM= X-Received: by 10.107.144.11 with SMTP id s11mr30319147iod.37.1521811580519; Fri, 23 Mar 2018 06:26:20 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.203.196 with HTTP; Fri, 23 Mar 2018 06:26:19 -0700 (PDT) X-Originating-IP: [2603:300b:6:5100:18a2:a4f7:170:8dd9] In-Reply-To: <888.1521797403@critter.freebsd.dk> References: <809d9254-ee56-59d8-69a4-08838e985cea@FreeBSD.org> <56619.1520878022@critter.freebsd.dk> <5e416eb6-0e79-1419-f09a-eb747215dc28@FreeBSD.org> <888.1521797403@critter.freebsd.dk> From: Warner Losh Date: Fri, 23 Mar 2018 07:26:19 -0600 X-Google-Sender-Auth: RTDvmECwPqKjp3mie1_rAEW99j8 Message-ID: Subject: Re: geom->access problem and workaround To: Poul-Henning Kamp Cc: Andriy Gapon , "freebsd-arch@freebsd.org" , freebsd-geom@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 13:26:22 -0000 On Fri, Mar 23, 2018 at 3:30 AM, Poul-Henning Kamp wrote: > -------- > In message <5e416eb6-0e79-1419-f09a-eb747215dc28@FreeBSD.org>, Andriy > Gapon writes: > >On 12/03/2018 20:07, Poul-Henning Kamp wrote: > >> If we want to have an architectural sound way to do slow operations > >> before any "user-I/O" is initiated, the right way to do so is to > >> define new BIO_OPEN and BIO_CLOSE operation, and insist via asserts > >> than all BIO_{READ|WRITE|DELETE} are wrapped in these. > > >What do you think? > > I don't see that changing anything... > > GEOM rests on a set of assumptions, if you violate them, you get panics. I agree. The cd panic is a problem in the cd driver where it bogusly uses runccb in a context that can't sleep. Warner From owner-freebsd-geom@freebsd.org Fri Mar 23 13:31:16 2018 Return-Path: Delivered-To: freebsd-geom@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B0E1BF6C883 for ; Fri, 23 Mar 2018 13:31:16 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x22f.google.com (mail-io0-x22f.google.com [IPv6:2607:f8b0:4001:c06::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 20DDD7407D for ; Fri, 23 Mar 2018 13:31:16 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x22f.google.com with SMTP id b20so15184887iof.5 for ; Fri, 23 Mar 2018 06:31:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=AoVRnoozLtO9SyxnFjtFKt13Uy2kljmz15rSsGuHkb0=; b=JUYg5+AjbNM+JV/xPuewpGnEofVeWRqGHOiZYyXJ5Im/D6lbeol3IZXBvv2XEJVDnh ERXGwmNBSORK4Tn32MgZw6nWU+JuugIqHrB/KtzZ0CccnrSNbFe9lVWE4bQlSKs7CzC/ KWMsodcPxxJ5SxHF/y03QuTvaHuwF8jAjWZVM/8EDhEjmRxF/MTfJ0Rr+UzmM99sC1oq yAF9NwRTa20ZdyPgV8V5zKJEhLkSe3p0UsAjD7daeFcvm81m8kEDvOev+V/A3RL06WhI Zb1vkEL2AVjrNW8Fdf+1Jjpy25DGvHajP48UOhGmXpgQ7wTZXtkQxvFEoPS0N3x27S/K YwRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=AoVRnoozLtO9SyxnFjtFKt13Uy2kljmz15rSsGuHkb0=; b=Z0531P0wSWra6cMbu8KuhwUZK3CMM96S0M6y/wl9FaF0XLz4dCHlniDnG1nzbEzQ9n 9EFL0AkJg6wdNBu7Zs3gpr2Maz0T7JY3YTTdf8wFVIGYwk8nMxLwr0f0Q/VhtQy+pfPd ckmLnrJpgmBqrxZqTHcAACugppw8wMG1NO6AftrndHC4Fi+r4d3vZ6jhSCNrr0/cHoUP dEIkO0Dl7AbZHtRES4FZMxQktwjl1LGuyrp6eIdXyCgYe0yDKo5uCA181nVW1vt0RVRj BZDZdd6RnDtTaSYRJ6zgjYwy9nJji7+php6ceHB70YvgXaGTE6FDW5Moiw1HA9Y/s0PG c5yg== X-Gm-Message-State: AElRT7GMRoNng9gCo+Y9y61a3wdzquHgGwvaveWIwYoFKqtTOkMXIHXY jso266HT/Ns5NjzvHz81+dsGDgaYBVpLGmONlCG4iw== X-Google-Smtp-Source: AG47ELtXbkgrg7zfkIR7FsHvb9H9FB9VjaN8Xz4UXcIoJ5IrzzbJ663ui6agaZnp03sVFaZjug+dqFm7/rl58o8nWHQ= X-Received: by 10.107.58.134 with SMTP id h128mr29447583ioa.299.1521811875502; Fri, 23 Mar 2018 06:31:15 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.203.196 with HTTP; Fri, 23 Mar 2018 06:31:14 -0700 (PDT) X-Originating-IP: [2603:300b:6:5100:18a2:a4f7:170:8dd9] In-Reply-To: <5e416eb6-0e79-1419-f09a-eb747215dc28@FreeBSD.org> References: <809d9254-ee56-59d8-69a4-08838e985cea@FreeBSD.org> <56619.1520878022@critter.freebsd.dk> <5e416eb6-0e79-1419-f09a-eb747215dc28@FreeBSD.org> From: Warner Losh Date: Fri, 23 Mar 2018 07:31:14 -0600 X-Google-Sender-Auth: kpEyGCL0IYYV3dHyl3t97cIGIjk Message-ID: Subject: Re: geom->access problem and workaround To: Andriy Gapon Cc: Poul-Henning Kamp , "freebsd-arch@freebsd.org" , freebsd-geom@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 13:31:17 -0000 On Fri, Mar 23, 2018 at 4:38 AM, Andriy Gapon wrote: > On 12/03/2018 20:07, Poul-Henning Kamp wrote: > > If we want to have an architectural sound way to do slow operations > > before any "user-I/O" is initiated, the right way to do so is to > > define new BIO_OPEN and BIO_CLOSE operation, and insist via asserts > > than all BIO_{READ|WRITE|DELETE} are wrapped in these. > > > In support of this proposal I want to add another example. > The problem is not only with doing I/O in access, but also with doing > blocking > I/O in the normal I/O path. > The following happened when a userland program tried to read from a CD-ROM > while > its tray was open: > > panic: sleepq_add: td 0xfffff80008e1c000 to sleep on wchan > 0xfffff801e58b8048 > with sleeping prohibited > cdstrategy shouldn't be sleeping. It can start I/O, but it can't wait for it to finish. strategy has been a no-sleep-zone since at least v6. The fact it's waiting for prevent is the bug here. Waner