From owner-freebsd-arch@freebsd.org Mon Sep 11 19:32:04 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3EB3BE1F8C0 for ; Mon, 11 Sep 2017 19:32:04 +0000 (UTC) (envelope-from marcel@xcllnt.net) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 277E46F712 for ; Mon, 11 Sep 2017 19:32:04 +0000 (UTC) (envelope-from marcel@xcllnt.net) Received: by mailman.ysv.freebsd.org (Postfix) id 26B52E1F8BF; Mon, 11 Sep 2017 19:32:04 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 264E3E1F8BE for ; Mon, 11 Sep 2017 19:32:04 +0000 (UTC) (envelope-from marcel@xcllnt.net) Received: from mail.xcllnt.net (mail.xcllnt.net [50.0.150.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EF6606F711; Mon, 11 Sep 2017 19:32:03 +0000 (UTC) (envelope-from marcel@xcllnt.net) Received: from marcels-mbp.int.brkt.com (cerberus.brkt.com [208.185.168.138]) (authenticated bits=0) by mail.xcllnt.net (8.15.2/8.15.2) with ESMTPSA id v8BJ58Pi080647 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 11 Sep 2017 12:05:09 -0700 (PDT) (envelope-from marcel@xcllnt.net) From: Marcel Moolenaar Message-Id: <26458208-EC4B-4647-8271-DF480EDD57DF@xcllnt.net> Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: ELF auxiliary vector tags Date: Mon, 11 Sep 2017 12:05:02 -0700 In-Reply-To: Cc: arch@freebsd.org To: John Baldwin References: X-Mailer: Apple Mail (2.3273) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (mail.xcllnt.net [50.0.150.214]); Mon, 11 Sep 2017 12:05:09 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Sep 2017 19:32:04 -0000 > On Sep 8, 2017, at 10:36 AM, John Baldwin wrote: > I know Justin changed time_t to 64-bit on 32-bit powerpc which = effectively broke 32-bit powerpc earlier, but this change would break = both 32-bit and 64-bit powerpc and is probably more disruptive (in = theory some binaries might have worked with a wrong time_t, but renumber = AT_STACKPROT, etc. will probably break every binary). That probably depends on the byte order. I would think widening time_t on a big-endian machine is a lot more disruptive than doing it on a little-endian machine. That said and along the lines of what @imp said: Maybe add a a MD macro (e.g. NO_MI_AUX_VECTORS) whose existence suppresses the MI definitions of AT_* so that MD headers can define their own? --=20 Marcel Moolenaar marcel@xcllnt.net From owner-freebsd-arch@freebsd.org Tue Sep 12 15:12:16 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7CCA3E0DF7B for ; Tue, 12 Sep 2017 15:12:16 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 682A17592B for ; Tue, 12 Sep 2017 15:12:16 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 6770AE0DF7A; Tue, 12 Sep 2017 15:12:16 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 670D4E0DF79 for ; Tue, 12 Sep 2017 15:12:16 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4484E75929 for ; Tue, 12 Sep 2017 15:12:15 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by mail.baldwin.cx (Postfix) with ESMTPSA id 2C9C110A82E; Tue, 12 Sep 2017 11:12:13 -0400 (EDT) From: John Baldwin To: Marcel Moolenaar Cc: arch@freebsd.org Subject: Re: ELF auxiliary vector tags Date: Mon, 11 Sep 2017 15:45:08 -0700 Message-ID: <5184520.CTdkFHEYDQ@ralph.baldwin.cx> User-Agent: KMail/4.14.10 (FreeBSD/11.1-STABLE; KDE/4.14.30; amd64; ; ) In-Reply-To: <26458208-EC4B-4647-8271-DF480EDD57DF@xcllnt.net> References: <26458208-EC4B-4647-8271-DF480EDD57DF@xcllnt.net> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Tue, 12 Sep 2017 11:12:13 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Sep 2017 15:12:16 -0000 On Monday, September 11, 2017 12:05:02 PM Marcel Moolenaar wrote: > > > On Sep 8, 2017, at 10:36 AM, John Baldwin wrote: > > > > I know Justin changed time_t to 64-bit on 32-bit powerpc which effectively broke 32-bit powerpc earlier, but this change would break both 32-bit and 64-bit powerpc and is probably more disruptive (in theory some binaries might have worked with a wrong time_t, but renumber AT_STACKPROT, etc. will probably break every binary). > > That probably depends on the byte order. I would think widening > time_t on a big-endian machine is a lot more disruptive than > doing it on a little-endian machine. > > That said and along the lines of what @imp said: > Maybe add a a MD macro (e.g. NO_MI_AUX_VECTORS) whose existence > suppresses the MI definitions of AT_* so that MD headers can > define their own? Going forward I would like to standardize on common values for new vectors added. The current implementation of 'info auxv' for GDB assumes they are the same on all architectures (and judging by the binutils / gdb bits for Linux, Linux uses the same AT_* values on all platforms). Are you running powerpc binaries yourself? The only person who I know is who has replied (Justin) is fine with just pulling the tier-2 card on powerpc to bring it inline with all the other platforms (which are already identical). One suggestion was to set a value in AT_FLAGS (currently always set to zero) that binaries could then use to detect the new layout on ppc. -- John Baldwin From owner-freebsd-arch@freebsd.org Tue Sep 12 15:43:06 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6432BE0F487 for ; Tue, 12 Sep 2017 15:43:06 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 3E6D976D07 for ; Tue, 12 Sep 2017 15:43:06 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mailman.ysv.freebsd.org (Postfix) id 3D863E0F486; Tue, 12 Sep 2017 15:43:06 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3CE28E0F485 for ; Tue, 12 Sep 2017 15:43:06 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x22a.google.com (mail-io0-x22a.google.com [IPv6:2607:f8b0:4001:c06::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F33B076D05 for ; Tue, 12 Sep 2017 15:43:05 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x22a.google.com with SMTP id d16so49638407ioj.3 for ; Tue, 12 Sep 2017 08:43:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=hWOIIeuPNIMIMfsXwmJcr6Mgsl7rft22ZvV4EF1f/FM=; b=i0I8T9V82zHB8GnGglIBInecrd5odCVDLDlpKvDnGHAQ6aO8qq27DE9rXAP+fOdEZ8 3u7tVWj0hiYurimq1tkrpLq0uNaH6HmDRCfb1NkAemSs4C3UjdHWPoOMKK8tdIVbap0+ PtoWqFWDLzx9RRnEuGeURYGgXVHsQJUpDR2D7a+glYqKnskJXU6mEc6T79VUnF/gDRWm qbfJcPrVVYiMnYzbyBgZPnnR6omj2Vn+6M1X9TOGC2h0iHDAnfTapU/m7sU0acQndXFe dG0XHkJvmO+uQvtnQ0ibYerQyWcLYzSzjquOS69OsHtKzJFCij7utx6UcFIWZ4TnGhxX R6fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=hWOIIeuPNIMIMfsXwmJcr6Mgsl7rft22ZvV4EF1f/FM=; b=Qb772FLcWV5mvV99/hfNHy1uSYVv8hVzOmr7Kl6tiQCVH9AoSXSrGAdHGnuVQweDf5 5aMhKAZ/EX08hlVwcKYXFKFXWztn1laVqz801kEk/hHRnTNwhMCPMdB5vKbzMI1R3xrY p/dt+Z2NBrBOV0Qb/PJM5dcOAU2Q8MVFSMxG9kocn1HD/PJXNvq/R65bqPebozt/Tt8T wr6RSALyy3m3lCXinxgiUVNKcFAcQ8OYKTAacX8jM3XjFhuV7c0kQOQwrXBbKhoqcczt sXDXpePIxfHZuSMs1nusdaJwMKkc6kGcpKwsrK7YnX2u2bnb3GOvZbmGtzI6BgM17pz0 m1Zw== X-Gm-Message-State: AHPjjUiWeyDeEQObdJJkNBHkNOafKTrR2SOgtNXW3QC+EqHes83wNNCh p5Zjc06dnRUoU6dVJpEK4c4qlc/xpu0a X-Google-Smtp-Source: ADKCNb4Blk5pWyG9T9kcEH8fUMtYEZeixCTlS5QesXKFwU6HCchdak2mWSh0SHx+FN292CmGBuBs3PqkMhHduDHN9p8= X-Received: by 10.107.133.92 with SMTP id h89mr13560679iod.208.1505230985114; Tue, 12 Sep 2017 08:43:05 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.10.71 with HTTP; Tue, 12 Sep 2017 08:43:04 -0700 (PDT) X-Originating-IP: [2603:300b:6:5100:809b:5e24:6d84:c8fa] In-Reply-To: <5184520.CTdkFHEYDQ@ralph.baldwin.cx> References: <26458208-EC4B-4647-8271-DF480EDD57DF@xcllnt.net> <5184520.CTdkFHEYDQ@ralph.baldwin.cx> From: Warner Losh Date: Tue, 12 Sep 2017 09:43:04 -0600 X-Google-Sender-Auth: HjIXfWCVfs36aHckbUai-USgD_M Message-ID: Subject: Re: ELF auxiliary vector tags To: John Baldwin Cc: Marcel Moolenaar , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Sep 2017 15:43:06 -0000 On Mon, Sep 11, 2017 at 4:45 PM, John Baldwin wrote: > On Monday, September 11, 2017 12:05:02 PM Marcel Moolenaar wrote: > > > > > On Sep 8, 2017, at 10:36 AM, John Baldwin wrote: > > > > > > > I know Justin changed time_t to 64-bit on 32-bit powerpc which > effectively broke 32-bit powerpc earlier, but this change would break both > 32-bit and 64-bit powerpc and is probably more disruptive (in theory some > binaries might have worked with a wrong time_t, but renumber AT_STACKPROT, > etc. will probably break every binary). > > > > That probably depends on the byte order. I would think widening > > time_t on a big-endian machine is a lot more disruptive than > > doing it on a little-endian machine. > > > > That said and along the lines of what @imp said: > > Maybe add a a MD macro (e.g. NO_MI_AUX_VECTORS) whose existence > > suppresses the MI definitions of AT_* so that MD headers can > > define their own? > > Going forward I would like to standardize on common values for new vectors > added. The current implementation of 'info auxv' for GDB assumes they > are the same on all architectures (and judging by the binutils / gdb bits > for Linux, Linux uses the same AT_* values on all platforms). Are you > running powerpc binaries yourself? The only person who I know is who has > replied (Justin) is fine with just pulling the tier-2 card on powerpc to > bring it inline with all the other platforms (which are already identical). > I think aligning is the right thing, regardless of tier status. The question about 'adding compat' is a separate issue, imho. > One suggestion was to set a value in AT_FLAGS (currently always set to > zero) > that binaries could then use to detect the new layout on ppc. > This is a path forward if we want to maintain upgradability across this incident. I don't think it's incumbent on John to do it (unless he wants to). It's incumbent on the powerpc folks if they need the compatibility. If this were arm the calculous would be different (since lots of people do binary upgrades and it's de-facto nearly tier 1) or even mips (where some people do binary upgrades, despite being definitely not tier 1). OTOH, if John did this to arm and didn't do compat shims, myself or some other arm user would do them. The only question would be how much snark would make its way into the commit message :) It's also a reasonable fallback plan should more users than just Justin be affected given the bumpy nature of -current at time. Warner From owner-freebsd-arch@freebsd.org Tue Sep 12 17:07:42 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BA18AE12DCC for ; Tue, 12 Sep 2017 17:07:42 +0000 (UTC) (envelope-from marcel@xcllnt.net) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id A2F847DA3C for ; Tue, 12 Sep 2017 17:07:42 +0000 (UTC) (envelope-from marcel@xcllnt.net) Received: by mailman.ysv.freebsd.org (Postfix) id A2620E12DCB; Tue, 12 Sep 2017 17:07:42 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A206AE12DCA for ; Tue, 12 Sep 2017 17:07:42 +0000 (UTC) (envelope-from marcel@xcllnt.net) Received: from mail.xcllnt.net (mail.xcllnt.net [50.0.150.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 84C7A7DA3B; Tue, 12 Sep 2017 17:07:42 +0000 (UTC) (envelope-from marcel@xcllnt.net) Received: from marcels-mbp.int.brkt.com (cerberus.brkt.com [208.185.168.138]) (authenticated bits=0) by mail.xcllnt.net (8.15.2/8.15.2) with ESMTPSA id v8CH7ewZ085603 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 12 Sep 2017 10:07:41 -0700 (PDT) (envelope-from marcel@xcllnt.net) From: Marcel Moolenaar Message-Id: <3EB61FE4-7EA8-434C-92B1-2243725C25A3@xcllnt.net> Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: ELF auxiliary vector tags Date: Tue, 12 Sep 2017 10:07:34 -0700 In-Reply-To: <5184520.CTdkFHEYDQ@ralph.baldwin.cx> Cc: arch@freebsd.org To: John Baldwin References: <26458208-EC4B-4647-8271-DF480EDD57DF@xcllnt.net> <5184520.CTdkFHEYDQ@ralph.baldwin.cx> X-Mailer: Apple Mail (2.3273) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (mail.xcllnt.net [50.0.150.214]); Tue, 12 Sep 2017 10:07:41 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Sep 2017 17:07:42 -0000 > On Sep 11, 2017, at 3:45 PM, John Baldwin wrote: > Are you running powerpc binaries yourself? No, not anymore. > The only person who I know is who has replied (Justin) is fine with = just pulling the tier-2 card on powerpc to bring it inline with all the = other platforms (which are already identical). I think you=E2=80=99re good to go with the unification then! --=20 Marcel Moolenaar marcel@xcllnt.net From owner-freebsd-arch@freebsd.org Tue Sep 12 18:45:44 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 95FC3E170CD for ; Tue, 12 Sep 2017 18:45:44 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 8272580B1A for ; Tue, 12 Sep 2017 18:45:44 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: by mailman.ysv.freebsd.org (Postfix) id 819F4E170CC; Tue, 12 Sep 2017 18:45:44 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8120CE170CB for ; Tue, 12 Sep 2017 18:45:44 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: from mail.soaustin.net (mail.soaustin.net [192.108.105.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.soaustin.net", Issuer "StartCom Class 2 IV Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6835E80B19; Tue, 12 Sep 2017 18:45:43 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: from lonesome.com (bones.soaustin.net [192.108.105.22]) by mail.soaustin.net (Postfix) with ESMTPSA id D75DB669; Tue, 12 Sep 2017 13:45:36 -0500 (CDT) Date: Tue, 12 Sep 2017 13:45:35 -0500 From: Mark Linimon To: Marcel Moolenaar Cc: John Baldwin , arch@freebsd.org Subject: Re: ELF auxiliary vector tags Message-ID: <20170912184535.GA8741@lonesome.com> References: <26458208-EC4B-4647-8271-DF480EDD57DF@xcllnt.net> <5184520.CTdkFHEYDQ@ralph.baldwin.cx> <3EB61FE4-7EA8-434C-92B1-2243725C25A3@xcllnt.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3EB61FE4-7EA8-434C-92B1-2243725C25A3@xcllnt.net> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Sep 2017 18:45:44 -0000 On Tue, Sep 12, 2017 at 10:07:34AM -0700, Marcel Moolenaar wrote: > > > On Sep 11, 2017, at 3:45 PM, John Baldwin wrote: > > > Are you running powerpc binaries yourself? > > No, not anymore. I run them but will do the work. You need to talk to swills@ as well. mcl From owner-freebsd-arch@freebsd.org Tue Sep 12 19:00:14 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 45278E17E34 for ; Tue, 12 Sep 2017 19:00:14 +0000 (UTC) (envelope-from andreast@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 2F59F81547 for ; Tue, 12 Sep 2017 19:00:14 +0000 (UTC) (envelope-from andreast@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 2EBAEE17E31; Tue, 12 Sep 2017 19:00:14 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2E4B4E17E30 for ; Tue, 12 Sep 2017 19:00:14 +0000 (UTC) (envelope-from andreast@FreeBSD.org) Received: from smtp.imp.ch (smtp.imp.ch [IPv6:2001:4060:1:1001::13:196]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E9DB281545; Tue, 12 Sep 2017 19:00:13 +0000 (UTC) (envelope-from andreast@FreeBSD.org) Received: from [192.168.225.14] (dhclient-91-190-10-49.flashcable.ch [91.190.10.49]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by fgznet.ch (Postfix) with ESMTPSA id 7EFC1C0517; Tue, 12 Sep 2017 21:00:02 +0200 (CEST) Subject: Re: ELF auxiliary vector tags To: John Baldwin Cc: arch@freebsd.org References: <26458208-EC4B-4647-8271-DF480EDD57DF@xcllnt.net> <5184520.CTdkFHEYDQ@ralph.baldwin.cx> From: Andreas Tobler Message-ID: Date: Tue, 12 Sep 2017 20:59:53 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <5184520.CTdkFHEYDQ@ralph.baldwin.cx> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: de-CH Content-Transfer-Encoding: 7bit X-Scanned-By: Asterix Submit on 127.0.1.1 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Sep 2017 19:00:14 -0000 On 12.09.17 00:45, John Baldwin wrote: > On Monday, September 11, 2017 12:05:02 PM Marcel Moolenaar wrote: >> >>> On Sep 8, 2017, at 10:36 AM, John Baldwin wrote: >> >> >>> I know Justin changed time_t to 64-bit on 32-bit powerpc which effectively broke 32-bit powerpc earlier, but this change would break both 32-bit and 64-bit powerpc and is probably more disruptive (in theory some binaries might have worked with a wrong time_t, but renumber AT_STACKPROT, etc. will probably break every binary). >> >> That probably depends on the byte order. I would think widening >> time_t on a big-endian machine is a lot more disruptive than >> doing it on a little-endian machine. >> >> That said and along the lines of what @imp said: >> Maybe add a a MD macro (e.g. NO_MI_AUX_VECTORS) whose existence >> suppresses the MI definitions of AT_* so that MD headers can >> define their own? > > Going forward I would like to standardize on common values for new vectors > added. The current implementation of 'info auxv' for GDB assumes they > are the same on all architectures (and judging by the binutils / gdb bits > for Linux, Linux uses the same AT_* values on all platforms). Are you > running powerpc binaries yourself? The only person who I know is who has > replied (Justin) is fine with just pulling the tier-2 card on powerpc to > bring it inline with all the other platforms (which are already identical). +1 from my side. Andreas From owner-freebsd-arch@freebsd.org Wed Sep 13 00:55:55 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5B820E026BD for ; Wed, 13 Sep 2017 00:55:55 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2910B6BB4D for ; Wed, 13 Sep 2017 00:55:51 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (106-69-250-203.dyn.iinet.net.au [106.69.250.203]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id v8D0tl9i025216 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Tue, 12 Sep 2017 17:55:50 -0700 (PDT) (envelope-from julian@freebsd.org) Subject: Fwd: Re: extending the maximum filename length (pointer to patch)[request for input] References: To: "freebsd-arch@FreeBSD.ORG" From: Julian Elischer X-Forwarded-Message-Id: Message-ID: <3e85f646-1787-98dd-30e2-1dae78fa9962@freebsd.org> Date: Wed, 13 Sep 2017 08:55:41 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Sep 2017 00:55:55 -0000 is this a topic for arch? Julian -------- Forwarded Message -------- On 12/9/17 2:17 pm, Conrad Meyer wrote: > On Sat, Sep 9, 2017 at 9:09 AM, Julian Elischer wrote: >> maybe we could get it into -current. >> It'd be silly to have to have people re-inventing hte wheel all the time. >> How about you put those changes into the reviews.freebsd.org and we can get >> some general consensus on them. >> We'll have to do similar for the Asian customers and anyone who uses UTF-8. >> So it >> would be silly to have to develop it all again (but subtly different of >> course). >> >> The key issue is how many system calls and other APIs would be broken, >> and how many would be broken in a non backwards compatible way? >> >> We would need it in a stable/10 and 11 branch but if the patch is isolated >> enough we could carry it forward until we get to 12. >> >> One has to allow people to do whatever they are used to with Windows. >> And in this case the issue is serving files over samba to windows machines. > Hey Julian, > > I've thrown the patch up at https://reviews.freebsd.org/D12330 . I > haven't actually tested it on FreeBSD, but it does compile. We also > have some patches against contrib/pjdfstest to fix those tests against > long file names, but I think we can hold off on those changes until > we've nailed down what the architectural change will be (if any). thanks! that looks a lot like a proof -of concept patch we derived a while back but never really tested. The issue for us is that using UTF-8 the filenames become too short for common usage in China and Japan. Apparently they routinely nae files with the contents of a small novella. e.g. “这是一个测试多字符文件名长度的文件目的是命名一个文件用中文或者日文或者韩文字符并且要求字符长度超过八十五个字符然后拷贝该文件到我们的共享文件夹看看是否能够拷贝该文件到我.txt” (I have no idea what that says but apparently it's a real filename from a windows machine that blew up when written via samba.) Does anyone else have any thoughts about whether FreeBSD 12 might grow longer path/filename support?  (I'm told Linux uses 1K and 4096 for filenames and path length.) Julian > > It's quite possible this accidentally breaks even more APIs than > expected and we should do some fine tuning to reduce the damage. Our > $WORK product mostly doesn't care about ABI, so we may not have > noticed some ABI breakage. > > If anyone else is interested, please subscribe or add yourself as a > reviewer on the phabricator revision. > > Best, > Conrad > _______________________________________________ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-arch@freebsd.org Wed Sep 13 05:23:29 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 01420E0CD23 for ; Wed, 13 Sep 2017 05:23:29 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from c.mail.sonic.net (c.mail.sonic.net [64.142.111.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D47DB734BA for ; Wed, 13 Sep 2017 05:23:28 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from zeppelin.tachypleus.net (cpe-45-48-160-106.socal.res.rr.com [45.48.160.106]) (authenticated bits=0) by c.mail.sonic.net (8.15.1/8.15.1) with ESMTPSA id v8D5BXPB026869 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT) for ; Tue, 12 Sep 2017 22:11:33 -0700 Subject: Re: ELF auxiliary vector tags To: freebsd-arch@freebsd.org References: <26458208-EC4B-4647-8271-DF480EDD57DF@xcllnt.net> <5184520.CTdkFHEYDQ@ralph.baldwin.cx> From: Nathan Whitehorn Message-ID: <565a4072-f95d-3d2e-1405-7bfc0286120b@freebsd.org> Date: Tue, 12 Sep 2017 22:11:33 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Sonic-CAuth: UmFuZG9tSVZsp7BpRFEetMBFxqJ7f1fnreMVI69N5mhYHfdkLhrNyWJlC6R4lQA93NF1A6CwIDKVX1AhuVzkOe7Oe2Cm1bjk9UJRkL61vTQ= X-Sonic-ID: C;SPVvAkKY5xGBqcGbEi49jA== M;vpm+AkKY5xGBqcGbEi49jA== X-Spam-Flag: No X-Sonic-Spam-Details: 0.0/5.0 by cerberusd X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Sep 2017 05:23:29 -0000 On 09/12/17 08:43, Warner Losh wrote: > On Mon, Sep 11, 2017 at 4:45 PM, John Baldwin wrote: > >> On Monday, September 11, 2017 12:05:02 PM Marcel Moolenaar wrote: >>>> On Sep 8, 2017, at 10:36 AM, John Baldwin wrote: >>> >>>> I know Justin changed time_t to 64-bit on 32-bit powerpc which >> effectively broke 32-bit powerpc earlier, but this change would break both >> 32-bit and 64-bit powerpc and is probably more disruptive (in theory some >> binaries might have worked with a wrong time_t, but renumber AT_STACKPROT, >> etc. will probably break every binary). >>> That probably depends on the byte order. I would think widening >>> time_t on a big-endian machine is a lot more disruptive than >>> doing it on a little-endian machine. >>> >>> That said and along the lines of what @imp said: >>> Maybe add a a MD macro (e.g. NO_MI_AUX_VECTORS) whose existence >>> suppresses the MI definitions of AT_* so that MD headers can >>> define their own? >> Going forward I would like to standardize on common values for new vectors >> added. The current implementation of 'info auxv' for GDB assumes they >> are the same on all architectures (and judging by the binutils / gdb bits >> for Linux, Linux uses the same AT_* values on all platforms). Are you >> running powerpc binaries yourself? The only person who I know is who has >> replied (Justin) is fine with just pulling the tier-2 card on powerpc to >> bring it inline with all the other platforms (which are already identical). >> > I think aligning is the right thing, regardless of tier status. The > question about 'adding compat' is a separate issue, imho. > > >> One suggestion was to set a value in AT_FLAGS (currently always set to >> zero) >> that binaries could then use to detect the new layout on ppc. >> > This is a path forward if we want to maintain upgradability across this > incident. I don't think it's incumbent on John to do it (unless he wants > to). It's incumbent on the powerpc folks if they need the compatibility. If > this were arm the calculous would be different (since lots of people do > binary upgrades and it's de-facto nearly tier 1) or even mips (where some > people do binary upgrades, despite being definitely not tier 1). OTOH, if > John did this to arm and didn't do compat shims, myself or some other arm > user would do them. The only question would be how much snark would make > its way into the commit message :) It's also a reasonable fallback plan > should more users than just Justin be affected given the bumpy nature of > -current at time. > We just broke PPC compatibility anyway, so this seems like the perfect time to push forward and not worry about compat. -Nathan From owner-freebsd-arch@freebsd.org Wed Sep 13 10:10:33 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 27391E18315 for ; Wed, 13 Sep 2017 10:10:33 +0000 (UTC) (envelope-from repeatable_compression@yahoo.com) Received: from sonic317-36.consmr.mail.ne1.yahoo.com (sonic317-36.consmr.mail.ne1.yahoo.com [66.163.184.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EA94A80990 for ; Wed, 13 Sep 2017 10:10:32 +0000 (UTC) (envelope-from repeatable_compression@yahoo.com) X-YMail-OSG: fa7upfoVM1nl8iWzSx0nDQob3jj8WG8SevkqzxdBk6jIvhdNCbhblOqsDKOwJ09 X9uSlEO.TbTJm7wCUrnA4p1uiuMgzVWjC4TEJgi7VbSpM5zGvu3qMH16IiZmN0K5rdyNwnB5wbiT ccbt5NX_SbqobznCgMKa3eJVAEatF6dCVBw.dAw7fzRab99AJ8Fn.Edv7lvxZZrBX8Toxb14FxPt n2J6PhOecWsWr3.8d5ppGDRgUjDcDGHu.6.idI650m2mIesZ74tdeNdSD0pf7uJuzYwZf6wBeR9w aB6jebeG2U4ef5IyIYn2QJRhCcorpWdyjh1TS_loPfp8t9xALStF5nk2E90Kzlp0P.7g8Yq9HwCl 42W9oFG2Fh4csNfq2sc68dOaZV7PdIJjTlXBsJATKJKN7ZvJsJW.XLnpA6rN.J2sPE5vlva_C8yV hiPEFhhOvIEHGDGyvS2Px8AJHfezsaBtyO1gd5alYyoZFhachZ.GYZ1pIkYSNq_c4E6CO6c0wXNi kSemkQu52TUSGWKCbOSdrMZ_jUwa5gbIP Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.ne1.yahoo.com with HTTP; Wed, 13 Sep 2017 10:10:25 +0000 Received: from [127.0.0.1] by smtp219.mail.ne1.yahoo.com with NNFMP; 13 Sep 2017 10:10:23 -0000 X-Yahoo-Newman-Id: 280334.33333.bm@smtp219.mail.ne1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: fa7upfoVM1nl8iWzSx0nDQob3jj8WG8SevkqzxdBk6jIvhd NCbhblOqsDKOwJ09X9uSlEO.TbTJm7wCUrnA4p1uiuMgzVWjC4TEJgi7VbSp M5zGvu3qMH16IiZmN0K5rdyNwnB5wbiTccbt5NX_SbqobznCgMKa3eJVAEat F6dCVBw.dAw7fzRab99AJ8Fn.Edv7lvxZZrBX8Toxb14FxPtn2J6PhOecWsW r3.8d5ppGDRgUjDcDGHu.6.idI650m2mIesZ74tdeNdSD0pf7uJuzYwZf6wB eR9waB6jebeG2U4ef5IyIYn2QJRhCcorpWdyjh1TS_loPfp8t9xALStF5nk2 E90Kzlp0P.7g8Yq9HwCl42W9oFG2Fh4csNfq2sc68dOaZV7PdIJjTlXBsJAT KJKN7ZvJsJW.XLnpA6rN.J2sPE5vlva_C8yVhiPEFhhOvIEHGDGyvS2Px8AJ HfezsaBtyO1gd5alYyoZFhachZ.GYZ1pIkYSNq_c4E6CO6c0wXNikSemkQu5 2TUSGWKCbOSdrMZ_jUwa5gbIP X-Yahoo-SMTP: KDkTLsqswBBCmUTAOzBaZ_hLyVQzFsoqgrhYGNK2rJDiXlA- Subject: Re: Fwd: Re: extending the maximum filename length (pointer to patch)[request for input] To: freebsd-arch@freebsd.org References: <3e85f646-1787-98dd-30e2-1dae78fa9962@freebsd.org> From: Jules Gilbert Message-ID: <71c41b0d-f2a5-b556-cae8-f19e930604d0@yahoo.com> Date: Wed, 13 Sep 2017 06:10:21 -0400 User-Agent: Mozilla/5.0 (X11; Linux i686; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <3e85f646-1787-98dd-30e2-1dae78fa9962@freebsd.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Sep 2017 10:10:33 -0000 First, I'm just an old guy, I don't maintain any systems (maybe a couple of laptop's,) but no one holds me responsible if their work stations fail. However very suddenly I have a contribution... Do not worry about windoz or compatibility with windoz. I assure you, billygates and his gang never stayed up late worrying about how they could keep the 'BSD boys happy. Truth, the historical record is that Microsoft made changes in their OS in order to break the programs of alternative ISD'ers. One example, Wordstar, Micropro was selling a version for MS-DOS and when MS brought out their own editor/word-processor, suddenly MS-DOS was changed in a way that, too bad, the Micropro product wouldn't work on MS systems. Indeed, Micropro failed and eventually MS faced DOJ sanctions for such behavior. So folks, just design and build the best you can. Recently I got my wife a laptop running W10. She's not a computer person, in fact she hasn't used a computer for about fifteen years. Now she can. So I discovered how it is, running W10. (Bloated, and little more than a vehicle for delivering advertisements to the user.) If you want to support windoz, one last question, do you remember when billy went around the country giving speeches telling people how wrong it was to use open source programs? I don't post here often. So now I'm going to break the rules go off-topic. First, friend up with the guys from OpenBSD. Yes they can be, well not jerks, just not overly social. They have a lot to contribute. Second, this business of giving chips the appearance of multiple cores is nice if that's all you need/want. But I think the world is looking for an OS that goes way beyond, say, a mere hypercube. What is needed are operating system architectures that can make use of many millions of internet connected machines. And the people behind FreeBSD are the right people to put something like this together. On 09/12/2017 08:55 PM, Julian Elischer wrote: > is this a topic for arch? > > > Julian > > > > -------- Forwarded Message -------- > > > On 12/9/17 2:17 pm, Conrad Meyer wrote: >> On Sat, Sep 9, 2017 at 9:09 AM, Julian Elischer >> wrote: >>> maybe we could get it into -current. >>> It'd be silly to have to have people re-inventing hte wheel all the >>> time. >>> How about you put those changes into the reviews.freebsd.org and we >>> can get >>> some general consensus on them. >>> We'll have to do similar for the Asian customers and anyone who uses >>> UTF-8. >>> So it >>> would be silly to have to develop it all again (but subtly different of >>> course). >>> >>> The key issue is how many system calls and other APIs would be broken, >>> and how many would be broken in a non backwards compatible way? >>> >>> We would need it in a stable/10 and 11 branch but if the patch is >>> isolated >>> enough we could carry it forward until we get to 12. >>> >>> One has to allow people to do whatever they are used to with Windows. >>> And in this case the issue is serving files over samba to windows >>> machines. >> Hey Julian, >> >> I've thrown the patch up at https://reviews.freebsd.org/D12330 . I >> haven't actually tested it on FreeBSD, but it does compile. We also >> have some patches against contrib/pjdfstest to fix those tests against >> long file names, but I think we can hold off on those changes until >> we've nailed down what the architectural change will be (if any). > > thanks! > > that looks a lot like a proof -of concept patch we derived a while > back but never really tested. > The issue for us is that using UTF-8 the filenames become too short > for common usage in China and Japan. > Apparently they routinely nae files with the contents of a small novella. > > e.g. > > “这是一个测试多字符文件名长度的文件目的是命名一个文件用中文或者日文或者韩文字符并且要求字符长度超过八十五个字符然后拷贝该文件到我们的共享文件夹看看是否能够拷贝该文件到我.txt” > > (I have no idea what that says but apparently it's a real filename > from a windows machine that blew up when written via samba.) > > Does anyone else have any thoughts about whether FreeBSD 12 might grow > longer path/filename support? > (I'm told Linux uses 1K and 4096 for filenames and path length.) > > Julian > >> >> It's quite possible this accidentally breaks even more APIs than >> expected and we should do some fine tuning to reduce the damage. Our >> $WORK product mostly doesn't care about ABI, so we may not have >> noticed some ABI breakage. >> >> If anyone else is interested, please subscribe or add yourself as a >> reviewer on the phabricator revision. >> >> Best, >> Conrad >> > > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" . From owner-freebsd-arch@freebsd.org Wed Sep 13 16:34:41 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A60F8E0495E for ; Wed, 13 Sep 2017 16:34:41 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8B8DA6D074 for ; Wed, 13 Sep 2017 16:34:41 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: 76f30345-98a1-11e7-950d-03a3531dacf2 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.ore.mailhop.org (Halon) with ESMTPSA id 76f30345-98a1-11e7-950d-03a3531dacf2; Wed, 13 Sep 2017 16:34:51 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id v8DGYYER017835; Wed, 13 Sep 2017 10:34:34 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1505320474.32063.120.camel@freebsd.org> Subject: hardware RNG harvest rate From: Ian Lepore To: "freebsd-arch@FreeBSD.org" Date: Wed, 13 Sep 2017 10:34:34 -0600 Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Sep 2017 16:34:41 -0000 While looking into updating the hardware rng driver for RPi 3 I discovered that on the original wimpy RPi we're feeding random_harvest_queue() with 16 bytes of generated data, 82 times a second. That seems like a lot of work for a wimpy processor, and probably more than needed for good random numbers. I looked at some other rng drivers, and I see them feeding in a few bytes at a time at rates ranging from 100 Hz to once every 5 seconds. So no clear concensus there. Some have this comment: /* MarkM: FIX!! Check that this does not swamp the harvester! */ A comment block above random_harvest_queue() warns, in part: * It is also illegal (and morally reprehensible) to insert any * high-rate data here. So is there any simple "X bytes per second is good and more than Y is wasted/harmful" advice available? -- Ian From owner-freebsd-arch@freebsd.org Wed Sep 13 16:41:42 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 647B5E04F54 for ; Wed, 13 Sep 2017 16:41:42 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 49B7C6D645 for ; Wed, 13 Sep 2017 16:41:41 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (220-253-182-172.dyn.iinet.net.au [220.253.182.172]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id v8DGfave029092 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 13 Sep 2017 09:41:39 -0700 (PDT) (envelope-from julian@freebsd.org) Subject: Re: Fwd: Re: extending the maximum filename length (pointer to patch)[request for input] To: Jules Gilbert , freebsd-arch@freebsd.org References: <3e85f646-1787-98dd-30e2-1dae78fa9962@freebsd.org> <71c41b0d-f2a5-b556-cae8-f19e930604d0@yahoo.com> From: Julian Elischer Message-ID: Date: Thu, 14 Sep 2017 00:41:30 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <71c41b0d-f2a5-b556-cae8-f19e930604d0@yahoo.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Sep 2017 16:41:42 -0000 On 13/9/17 6:10 pm, Jules Gilbert via freebsd-arch wrote: > First, I'm just an old guy, I don't maintain any systems (maybe a > couple of laptop's,) but no one holds me responsible if their work > stations fail. > > However very suddenly I have a contribution... Hi Jules. I was not saying we want to compete with microsoft, just that Microsoft has set the bar as t o how long Asian users think a filename can be. And as we are selligna file-server which is sending files to windows, we need to be able to handle any filenames that the Windows client sends to us. > > Do not worry about windoz or compatibility with windoz.  I assure > you, billygates and his gang never stayed up late worrying about how > they could keep the 'BSD boys happy.  Truth, the historical record > is that Microsoft made changes in their OS in order to break the > programs of alternative ISD'ers.  One example, Wordstar, Micropro > was selling a version for MS-DOS and when MS brought out their own > editor/word-processor, suddenly MS-DOS was changed in a way that, > too bad, the Micropro product wouldn't work on MS systems.   Indeed, > Micropro failed and eventually MS faced DOJ sanctions for such > behavior. > > So folks, just design and build the best you can. > > Recently I got my wife a laptop running W10.  She's not a computer > person, in fact she hasn't used a computer for about fifteen years.  > Now she can.  So I discovered how it is, running W10. (Bloated, and > little more than a vehicle for delivering advertisements to the user.) > > If you want to support windoz, one last question, do you remember > when billy went around the country giving speeches telling people > how wrong it was to use open source programs? > > I don't post here often.  So now I'm going to break the rules go > off-topic. > > First, friend up with the guys from OpenBSD.  Yes they can be, well > not jerks, just not overly social.  They have a lot to contribute. > > Second, this business of giving chips the appearance of multiple > cores is nice if that's all you need/want.  But I think the world is > looking for an OS that goes way beyond, say, a mere hypercube. What > is needed are operating system architectures that can make use of > many millions of internet connected machines. > > And the people behind FreeBSD are the right people to put something > like this together. > > > On 09/12/2017 08:55 PM, Julian Elischer wrote: >> is this a topic for arch? >> >> >> Julian >> >> >> >> -------- Forwarded Message -------- >> >> >> On 12/9/17 2:17 pm, Conrad Meyer wrote: >>> On Sat, Sep 9, 2017 at 9:09 AM, Julian Elischer >>> wrote: >>>> maybe we could get it into -current. >>>> It'd be silly to have to have people re-inventing hte wheel all >>>> the time. >>>> How about you put those changes into the reviews.freebsd.org and >>>> we can get >>>> some general consensus on them. >>>> We'll have to do similar for the Asian customers and anyone who >>>> uses UTF-8. >>>> So it >>>> would be silly to have to develop it all again (but subtly >>>> different of >>>> course). >>>> >>>> The key issue is how many system calls and other APIs would be >>>> broken, >>>> and how many would be broken in a non backwards compatible way? >>>> >>>> We would need it in a stable/10 and 11 branch but if the patch is >>>> isolated >>>> enough we could carry it forward until we get to 12. >>>> >>>> One has to allow people to do whatever they are used to with >>>> Windows. >>>> And in this case the issue is serving files over samba to windows >>>> machines. >>> Hey Julian, >>> >>> I've thrown the patch up at https://reviews.freebsd.org/D12330 .  I >>> haven't actually tested it on FreeBSD, but it does compile. We also >>> have some patches against contrib/pjdfstest to fix those tests >>> against >>> long file names, but I think we can hold off on those changes until >>> we've nailed down what the architectural change will be (if any). >> >> thanks! >> >> that looks a lot like a proof -of concept patch we derived a while >> back but never really tested. >> The issue for us is that using UTF-8 the filenames become too short >> for common usage in China and Japan. >> Apparently they routinely nae files with the contents of a small >> novella. >> >> e.g. >> >> “这是一个测试多字符文件名长度的文件目的是命名一个文件用中文或者日文或者韩文字符并且要求字符长度超过八十五个字符然后拷贝该文件到我们的共享文件夹看看是否能够拷贝该文件到我.txt” >> >> (I have no idea what that says but apparently it's a real filename >> from a windows machine that blew up when written via samba.) >> >> Does anyone else have any thoughts about whether FreeBSD 12 might >> grow longer path/filename support? >>  (I'm told Linux uses 1K and 4096 for filenames and path length.) >> >> Julian >> >>> >>> It's quite possible this accidentally breaks even more APIs than >>> expected and we should do some fine tuning to reduce the damage.  Our >>> $WORK product mostly doesn't care about ABI, so we may not have >>> noticed some ABI breakage. >>> >>> If anyone else is interested, please subscribe or add yourself as a >>> reviewer on the phabricator revision. >>> >>> Best, >>> Conrad >>> >> >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to >> "freebsd-current-unsubscribe@freebsd.org" >> >> >> _______________________________________________ >> freebsd-arch@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-arch >> To unsubscribe, send any mail to >> "freebsd-arch-unsubscribe@freebsd.org" > > > . > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > > From owner-freebsd-arch@freebsd.org Thu Sep 14 12:46:12 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DB6C6E15B99; Thu, 14 Sep 2017 12:46:12 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id DA38472BC7; Thu, 14 Sep 2017 12:46:11 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id PAA18631; Thu, 14 Sep 2017 15:46:03 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1dsTWt-00007d-5E; Thu, 14 Sep 2017 15:46:03 +0300 Subject: Re: mount / unmount and mountcheckdirs() To: Kirk McKusick Cc: freebsd-arch@FreeBSD.org, freebsd-fs References: <201605220640.u4M6enEo017327@chez.mckusick.com> From: Andriy Gapon Message-ID: <134c7c6e-f4f1-ef38-cc50-0e56c27c9fb8@FreeBSD.org> Date: Thu, 14 Sep 2017 15:45:07 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <201605220640.u4M6enEo017327@chez.mckusick.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Sep 2017 12:46:13 -0000 On 22/05/2016 09:40, Kirk McKusick wrote: > I added the checkdirs functionality in the mount direction only > (I actually did it in 4.4BSD-Lite and it got swept in with commit > 22521). The reason is that when a directory that is not empty is > mounted on, the expectation is that the entries in that directory > should no longer be present; rather they should be replaced by the > entries in the newly mounted directory. Thus all processes sitting > in the mounted on directory should see the newly mounted directory > as if they had come to it using a lookup after the mount had been > done. If a process had proceeded through the mounted on directory > into one of its other entries, then they are left alone until such > time as they chdir back into the mount point directory through ".." > at which time they will be passed up to the mounted directory using > the same mechanism that would put them there if they traversed into > the mount point from above it in the tree. I believe this is the > correct behavior, is not a security threat, and should be left alone. I almost dropped a ball on this issue, but I am now picking it up again. At the moment I am moving forward with the dounmount change as it seems to be non-contentious and rather simple to do and test. Regarding the mount part, I am not sure that I completely agree with you. Even if mountcheckdirs() does not cause any problems in the mount path, I still fail to see its usefulness. Specifically, I still do not see any significant difference between the covered directory and any directory below it. So, if we leave the lower directories alone, while bother with the covered directory... The covered directory: - absolute paths work correctly - relative paths with enough ".." (one) can access the actual namespace - other relative paths operate on the shadowed sub-tree of the original filesystem The lower directories: - absolute paths work correctly - relative paths with enough ".." (> 1) can access the actual namespace - other relative paths operate on the shadowed sub-tree of the original filesystem The only difference I can think of is that the root of the mounted filesystem cannot be reached with just ".."-s from the covered directory. But is this difference of any significance? Mateusz also raised some interesting points. On the other hand, it seems that illumos and probably Solaris has interesting parallels to the FreeBSD behavior. It does not allow to mount over a directory that is a current directory for any process ("Device busy"), but does not object against processes in directories below the mount point. So, probably it's just I who misses something about that scenario :-) > I was not aware that the functionality had been added at unmount > time, and I do not believe that it should have been done. Normally > an unmount will not succeed if any vnodes are busy (for example, if > any directory in the filesystem is a current directory). The only > way that it can succeed in such a case is if a forcible unmount is > done. The forcible unmount will effectively do a revoke(2) on all > current directory vnodes in the unmounted filesystem. Further attempts > to access them will fail with "." not found errors. The only way to > get a valid current directory is to chdir to an absolute pathname. > Gratuitously fixing this if you happen to be in the former root of > the filesystem is wrong. And as you note can lead to unintensionally > giving an escape path from a prison. So I concur with your removing > this added functionality. -- Andriy Gapon From owner-freebsd-arch@freebsd.org Thu Sep 14 12:59:37 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62B6EE168E5; Thu, 14 Sep 2017 12:59:37 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 7A5AD73231; Thu, 14 Sep 2017 12:59:35 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id PAA18650; Thu, 14 Sep 2017 15:59:34 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1dsTjy-00008D-4Q; Thu, 14 Sep 2017 15:59:34 +0300 Subject: Re: mount / unmount and mountcheckdirs() To: Mateusz Guzik Cc: freebsd-arch@FreeBSD.org, freebsd-fs References: <5c01bf62-b7b2-2e1d-bca5-859e6bf1f0e5@FreeBSD.org> <20160515165332.GA27836@dft-labs.eu> From: Andriy Gapon Message-ID: <69d80f6d-da25-7b94-7774-564cecd0a8c2@FreeBSD.org> Date: Thu, 14 Sep 2017 15:58:13 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20160515165332.GA27836@dft-labs.eu> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Sep 2017 12:59:37 -0000 On 15/05/2016 19:53, Mateusz Guzik wrote: > On Sun, May 15, 2016 at 04:37:05PM +0300, Andriy Gapon wrote: >> >> I am curious about the purpose of mountcheckdirs() called when mounting and >> unmounting a filesystem. >> >> The function is described as such: >> /* >> * Scan all active processes and prisons to see if any of them have a current >> * or root directory of `olddp'. If so, replace them with the new mount point. >> */ >> and it seems to be used to "lift" processes and jails to a root of a new >> filesystem when it is mounted and to "lower" them onto a covered vnode (if any) >> when a filesystem is unmounted. >> >> What's the purpose of those actions? >> It's strange that the machinations are done at all, but it is stranger that they >> are applied only to processes and jails at exactly a covered vnode and a root >> vnode. Anything below in a filesystem's tree is left alone. Is there anything >> so very special about being at exactly those points? >> >> IMO, the machinations can have unexpected security consequences. >> > > I don't know why this was implemented. It is also being done in NetBSD. > It is not done in Solaris nor Linux. I've done some testing on illumos and while it indeed does not do any "lifting", it does not permit to mount over a directory which is a current directory for any process. I haven't tested with zones, but suspect that it would also be prohibited. > Replacement is buggy in at least 2 ways: > 1. the process vs jail vnode replacement leaves a time window where > these 2 don't match, which screws up with the look up > 2. on fork we can have a 'struct filedesc' object copied but not > assigned to the new process yet, so it ends up with the old vnode > > And indeed, interested parties still have access to old vnodes by means > of having a file descriptor. Yeah. > That said, this likely needs to be simply changed to /deny/ mount > operations which would alter jail roots. That's probably what we should do. Or rather, Someone Should Do It. -- Andriy Gapon From owner-freebsd-arch@freebsd.org Fri Sep 15 03:11:00 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E2B7FE20030; Fri, 15 Sep 2017 03:11:00 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (chez.mckusick.com [70.36.157.235]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B2D4A74F22; Fri, 15 Sep 2017 03:11:00 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (localhost [IPv6:::1]) by chez.mckusick.com (8.15.2/8.15.2) with ESMTP id v8F3Ea6B085072; Thu, 14 Sep 2017 20:14:36 -0700 (PDT) (envelope-from mckusick@mckusick.com) Message-Id: <201709150314.v8F3Ea6B085072@chez.mckusick.com> From: Kirk McKusick To: Andriy Gapon Subject: Re: mount / unmount and mountcheckdirs() Cc: Konstantin Belousov , freebsd-arch@FreeBSD.org, freebsd-fs In-reply-to: <134c7c6e-f4f1-ef38-cc50-0e56c27c9fb8@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <85070.1505445276.1@chez.mckusick.com> Content-Transfer-Encoding: quoted-printable Date: Thu, 14 Sep 2017 20:14:36 -0700 X-Spam-Status: No, score=0.1 required=5.0 tests=MISSING_MID, UNPARSEABLE_RELAY autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on chez.mckusick.com X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 03:11:01 -0000 > To: Kirk McKusick > Subject: Re: mount / unmount and mountcheckdirs() > Cc: freebsd-arch@FreeBSD.org, freebsd-fs > From: Andriy Gapon > Date: Thu, 14 Sep 2017 15:45:07 +0300 > = > On 22/05/2016 09:40, Kirk McKusick wrote: >> I added the checkdirs functionality in the mount direction only >> (I actually did it in 4.4BSD-Lite and it got swept in with commit >> 22521). The reason is that when a directory that is not empty is >> mounted on, the expectation is that the entries in that directory >> should no longer be present; rather they should be replaced by the >> entries in the newly mounted directory. Thus all processes sitting >> in the mounted on directory should see the newly mounted directory >> as if they had come to it using a lookup after the mount had been >> done. If a process had proceeded through the mounted on directory >> into one of its other entries, then they are left alone until such >> time as they chdir back into the mount point directory through ".." >> at which time they will be passed up to the mounted directory using >> the same mechanism that would put them there if they traversed into >> the mount point from above it in the tree. I believe this is the >> correct behavior, is not a security threat, and should be left alone. > = > I almost dropped a ball on this issue, but I am now picking it up again. > At the moment I am moving forward with the dounmount change as it seems = to be > non-contentious and rather simple to do and test. > = > Regarding the mount part, I am not sure that I completely agree with you= . > Even if mountcheckdirs() does not cause any problems in the mount path, = I still > fail to see its usefulness. Specifically, I still do not see any signif= icant > difference between the covered directory and any directory below it. So= , if we > leave the lower directories alone, while bother with the covered directo= ry... > = > The covered directory: > - absolute paths work correctly > - relative paths with enough ".." (one) can access the actual namespace > - other relative paths operate on the shadowed sub-tree of the original > filesystem > = > = > The lower directories: > - absolute paths work correctly > - relative paths with enough ".." (> 1) can access the actual namespace > - other relative paths operate on the shadowed sub-tree of the original > filesystem > = > The only difference I can think of is that the root of the mounted files= ystem > cannot be reached with just ".."-s from the covered directory. But is t= his > difference of any significance? > = > Mateusz also raised some interesting points. > = > On the other hand, it seems that illumos and probably Solaris has > interesting parallels to the FreeBSD behavior. It does not allow > to mount over a directory that is a current directory for any process > ("Device busy"), but does not object against processes in directories > below the mount point. > = > So, probably it's just I who misses something about that scenario :-) > = >> I was not aware that the functionality had been added at unmount >> time, and I do not believe that it should have been done. Normally >> an unmount will not succeed if any vnodes are busy (for example, if >> any directory in the filesystem is a current directory). The only >> way that it can succeed in such a case is if a forcible unmount is >> done. The forcible unmount will effectively do a revoke(2) on all >> current directory vnodes in the unmounted filesystem. Further attempts >> to access them will fail with "." not found errors. The only way to >> get a valid current directory is to chdir to an absolute pathname. >> Gratuitously fixing this if you happen to be in the former root of >> the filesystem is wrong. And as you note can lead to unintensionally >> giving an escape path from a prison. So I concur with your removing >> this added functionality. > -- = > Andriy Gapon I had to dig back through some *really* old emails to find out what triggered the addition of mountcheckdirs(). The problem that it was specifically solving was that as part of the startup script a minimal root directory was replaced by the real root directory. The shell running the startup script needed to be moved to the new mounted-on root so that the rest of the script would not fail. That disaster of a hack has been replaced with the much more functional code that deals with setting up the root and the devfs filesystem on /dev. So the need for which it was designed no longer exists. But I still believe that it is the correct thing to do. For example, if you are using automount code and chdir into your home directory triggering an auto-mount, you should just be in your home directory after the mount rather than having to do cd ../$USER to get there. ~Kirk From owner-freebsd-arch@freebsd.org Fri Sep 15 09:20:13 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 29CB1E092B9; Fri, 15 Sep 2017 09:20:13 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C2FE58378C; Fri, 15 Sep 2017 09:20:12 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v8F9K1Ai048801 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 15 Sep 2017 12:20:01 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v8F9K1Ai048801 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v8F9K1Ar048800; Fri, 15 Sep 2017 12:20:01 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 15 Sep 2017 12:20:01 +0300 From: Konstantin Belousov To: Kirk McKusick Cc: Andriy Gapon , freebsd-fs , freebsd-arch@FreeBSD.org Subject: Re: mount / unmount and mountcheckdirs() Message-ID: <20170915092001.GK78693@kib.kiev.ua> References: <134c7c6e-f4f1-ef38-cc50-0e56c27c9fb8@FreeBSD.org> <201709150314.v8F3Ea6B085072@chez.mckusick.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201709150314.v8F3Ea6B085072@chez.mckusick.com> User-Agent: Mutt/1.9.0 (2017-09-02) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 09:20:13 -0000 On Thu, Sep 14, 2017 at 08:14:36PM -0700, Kirk McKusick wrote: > > To: Kirk McKusick > > Subject: Re: mount / unmount and mountcheckdirs() > > Cc: freebsd-arch@FreeBSD.org, freebsd-fs > > From: Andriy Gapon > > Date: Thu, 14 Sep 2017 15:45:07 +0300 > > > > On 22/05/2016 09:40, Kirk McKusick wrote: > >> I added the checkdirs functionality in the mount direction only > >> (I actually did it in 4.4BSD-Lite and it got swept in with commit > >> 22521). The reason is that when a directory that is not empty is > >> mounted on, the expectation is that the entries in that directory > >> should no longer be present; rather they should be replaced by the > >> entries in the newly mounted directory. Thus all processes sitting > >> in the mounted on directory should see the newly mounted directory > >> as if they had come to it using a lookup after the mount had been > >> done. If a process had proceeded through the mounted on directory > >> into one of its other entries, then they are left alone until such > >> time as they chdir back into the mount point directory through ".." > >> at which time they will be passed up to the mounted directory using > >> the same mechanism that would put them there if they traversed into > >> the mount point from above it in the tree. I believe this is the > >> correct behavior, is not a security threat, and should be left alone. > > > > I almost dropped a ball on this issue, but I am now picking it up again. > > At the moment I am moving forward with the dounmount change as it seems to be > > non-contentious and rather simple to do and test. > > > > Regarding the mount part, I am not sure that I completely agree with you. > > Even if mountcheckdirs() does not cause any problems in the mount path, I still > > fail to see its usefulness. Specifically, I still do not see any significant > > difference between the covered directory and any directory below it. So, if we > > leave the lower directories alone, while bother with the covered directory... > > > > The covered directory: > > - absolute paths work correctly > > - relative paths with enough ".." (one) can access the actual namespace > > - other relative paths operate on the shadowed sub-tree of the original > > filesystem > > > > > > The lower directories: > > - absolute paths work correctly > > - relative paths with enough ".." (> 1) can access the actual namespace > > - other relative paths operate on the shadowed sub-tree of the original > > filesystem > > > > The only difference I can think of is that the root of the mounted filesystem > > cannot be reached with just ".."-s from the covered directory. But is this > > difference of any significance? > > > > Mateusz also raised some interesting points. > > > > On the other hand, it seems that illumos and probably Solaris has > > interesting parallels to the FreeBSD behavior. It does not allow > > to mount over a directory that is a current directory for any process > > ("Device busy"), but does not object against processes in directories > > below the mount point. > > > > So, probably it's just I who misses something about that scenario :-) > > > >> I was not aware that the functionality had been added at unmount > >> time, and I do not believe that it should have been done. Normally > >> an unmount will not succeed if any vnodes are busy (for example, if > >> any directory in the filesystem is a current directory). The only > >> way that it can succeed in such a case is if a forcible unmount is > >> done. The forcible unmount will effectively do a revoke(2) on all > >> current directory vnodes in the unmounted filesystem. Further attempts > >> to access them will fail with "." not found errors. The only way to > >> get a valid current directory is to chdir to an absolute pathname. > >> Gratuitously fixing this if you happen to be in the former root of > >> the filesystem is wrong. And as you note can lead to unintensionally > >> giving an escape path from a prison. So I concur with your removing > >> this added functionality. > > -- > > Andriy Gapon > > I had to dig back through some *really* old emails to find out what > triggered the addition of mountcheckdirs(). The problem that it was > specifically solving was that as part of the startup script a minimal > root directory was replaced by the real root directory. The shell > running the startup script needed to be moved to the new mounted-on > root so that the rest of the script would not fail. If the mountcheckdirs() code not going away, please add your spelunking results as a comment before the function. This theme is recurring, and it would be highly beneficial to not loose the non-trivial reasoning behind the code existence. > > That disaster of a hack has been replaced with the much more functional > code that deals with setting up the root and the devfs filesystem on > /dev. So the need for which it was designed no longer exists. But I > still believe that it is the correct thing to do. For example, if you > are using automount code and chdir into your home directory triggering > an auto-mount, you should just be in your home directory after the > mount rather than having to do cd ../$USER to get there. I believe that the current autofs does not allow a process to get into this situation at all. In fact, the behavior implemented by mountcheckdirs() is surprising as well. For instance, I did expected that the system would operate as if mountcheckdirs() does not exist, and it caused me some head-scratching when I see it first time. From owner-freebsd-arch@freebsd.org Fri Sep 15 10:08:32 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ECB19E0B8A8; Fri, 15 Sep 2017 10:08:32 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-ua0-x22e.google.com (mail-ua0-x22e.google.com [IPv6:2607:f8b0:400c:c08::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A1CB613CB; Fri, 15 Sep 2017 10:08:32 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-ua0-x22e.google.com with SMTP id k63so989334uak.12; Fri, 15 Sep 2017 03:08:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=WvemsmAcbVVWcb6anF3UZZ+QeMxdOBfA88FTxcKo+ws=; b=VZWi7kTq2IAgUQE+gA0YYcj9cdAMsVCfx1cDT1J8zY8k3U6U3Ja43x4dQADan18dE7 c2Tfl9wUktOd1EbBgArhs1b7jsI2F5PUu61chWeBzMxgu5Qju4/7sZptGmlKJS+eXoJ3 lLQ3wJ1IDvVNQhES1oA0NctUW+wyC0IRCxhinpfQMlcylg4kQJUI64S07UIoQcEGk+e1 07SkwVvhJ6qhPw3oPe9pRdFYvuTl4exFJ5mIe/LSsW8tgXoqkFcrqX4Uu++JVh03PkiX AM0paIIQ5et3rAMav01kUNEgkkiRF5qQgcSDV0oPvm1UrRVpWBF1GmaMtqaWoUfyqmzw QOcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=WvemsmAcbVVWcb6anF3UZZ+QeMxdOBfA88FTxcKo+ws=; b=kOSaB8SMIZXmB6iz4d0pCoJHwzuFFRpa54LJILC96AvRjr90PdHwjIO0hqZXZm4zTQ Mz1mOBG2RVj3HyYJr4NkdgINqkpacatNxsMG7czaeiswE3ujdtDU2E0s8YbQaCAMSH+2 +CMhdTNWoRGO7LkyNiDQ6D2d/Uo8ojPYl4VsSOONo2EVzMAq+qdFG+XDOwATByQJD5b0 BDni/6fU9DA6zNEDVpkh76/vKmtK1cPvx/tRteqNvef5ZHyeOFWAGOlw+ZsmkYB4YOBm 01aaWv6I3dRRvxJ1CCJwelrz1J+4Xeyv29eNWMQIqWPEfrsbA3iLrd1AZNAIHOVTjgOr lCfw== X-Gm-Message-State: AHPjjUhyxv4vqZQrdVzzBepZ+KLzfhr9mS8sySZKYcG6P5EvOTwWDlRd WhZFeTIB8rrKPPww8p4O8fEz4mZUKi1bp3pe278= X-Google-Smtp-Source: ADKCNb5R0j+c+YzO6Bucq8sonl/3eRgkdBN5qOndAXN9iO1TjCIxwv02Qlx/4fY9BcOqHCn/6mX3lR7JqVL1Kyeh2E0= X-Received: by 10.176.21.109 with SMTP id p42mr12368078uae.127.1505470111627; Fri, 15 Sep 2017 03:08:31 -0700 (PDT) MIME-Version: 1.0 Sender: etnapierala@gmail.com Received: by 10.159.51.232 with HTTP; Fri, 15 Sep 2017 03:08:31 -0700 (PDT) In-Reply-To: <20170915092001.GK78693@kib.kiev.ua> References: <134c7c6e-f4f1-ef38-cc50-0e56c27c9fb8@FreeBSD.org> <201709150314.v8F3Ea6B085072@chez.mckusick.com> <20170915092001.GK78693@kib.kiev.ua> From: Edward Napierala Date: Fri, 15 Sep 2017 11:08:31 +0100 X-Google-Sender-Auth: q1_4RPx-uknuYeYqYQVra7HLIj8 Message-ID: Subject: Re: mount / unmount and mountcheckdirs() To: Konstantin Belousov Cc: Kirk McKusick , freebsd-fs , Andriy Gapon , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 10:08:33 -0000 2017-09-15 10:20 GMT+01:00 Konstantin Belousov : > On Thu, Sep 14, 2017 at 08:14:36PM -0700, Kirk McKusick wrote: > > > To: Kirk McKusick > > > Subject: Re: mount / unmount and mountcheckdirs() > > > Cc: freebsd-arch@FreeBSD.org, freebsd-fs > > > From: Andriy Gapon > > > Date: Thu, 14 Sep 2017 15:45:07 +0300 > > > > > > On 22/05/2016 09:40, Kirk McKusick wrote: > > >> I added the checkdirs functionality in the mount direction only > > >> (I actually did it in 4.4BSD-Lite and it got swept in with commit > > >> 22521). The reason is that when a directory that is not empty is > > >> mounted on, the expectation is that the entries in that directory > > >> should no longer be present; rather they should be replaced by the > > >> entries in the newly mounted directory. Thus all processes sitting > > >> in the mounted on directory should see the newly mounted directory > > >> as if they had come to it using a lookup after the mount had been > > >> done. If a process had proceeded through the mounted on directory > > >> into one of its other entries, then they are left alone until such > > >> time as they chdir back into the mount point directory through ".." > > >> at which time they will be passed up to the mounted directory using > > >> the same mechanism that would put them there if they traversed into > > >> the mount point from above it in the tree. I believe this is the > > >> correct behavior, is not a security threat, and should be left alone. > > > > > > I almost dropped a ball on this issue, but I am now picking it up > again. > > > At the moment I am moving forward with the dounmount change as it > seems to be > > > non-contentious and rather simple to do and test. > > > > > > Regarding the mount part, I am not sure that I completely agree with > you. > > > Even if mountcheckdirs() does not cause any problems in the mount > path, I still > > > fail to see its usefulness. Specifically, I still do not see any > significant > > > difference between the covered directory and any directory below it. > So, if we > > > leave the lower directories alone, while bother with the covered > directory... > > > > > > The covered directory: > > > - absolute paths work correctly > > > - relative paths with enough ".." (one) can access the actual namespace > > > - other relative paths operate on the shadowed sub-tree of the original > > > filesystem > > > > > > > > > The lower directories: > > > - absolute paths work correctly > > > - relative paths with enough ".." (> 1) can access the actual namespace > > > - other relative paths operate on the shadowed sub-tree of the original > > > filesystem > > > > > > The only difference I can think of is that the root of the mounted > filesystem > > > cannot be reached with just ".."-s from the covered directory. But is > this > > > difference of any significance? > > > > > > Mateusz also raised some interesting points. > > > > > > On the other hand, it seems that illumos and probably Solaris has > > > interesting parallels to the FreeBSD behavior. It does not allow > > > to mount over a directory that is a current directory for any process > > > ("Device busy"), but does not object against processes in directories > > > below the mount point. > > > > > > So, probably it's just I who misses something about that scenario :-) > > > > > >> I was not aware that the functionality had been added at unmount > > >> time, and I do not believe that it should have been done. Normally > > >> an unmount will not succeed if any vnodes are busy (for example, if > > >> any directory in the filesystem is a current directory). The only > > >> way that it can succeed in such a case is if a forcible unmount is > > >> done. The forcible unmount will effectively do a revoke(2) on all > > >> current directory vnodes in the unmounted filesystem. Further attempts > > >> to access them will fail with "." not found errors. The only way to > > >> get a valid current directory is to chdir to an absolute pathname. > > >> Gratuitously fixing this if you happen to be in the former root of > > >> the filesystem is wrong. And as you note can lead to unintensionally > > >> giving an escape path from a prison. So I concur with your removing > > >> this added functionality. > > > -- > > > Andriy Gapon > > > > I had to dig back through some *really* old emails to find out what > > triggered the addition of mountcheckdirs(). The problem that it was > > specifically solving was that as part of the startup script a minimal > > root directory was replaced by the real root directory. The shell > > running the startup script needed to be moved to the new mounted-on > > root so that the rest of the script would not fail. > If the mountcheckdirs() code not going away, please add your spelunking > results as a comment before the function. This theme is recurring, and > it would be highly beneficial to not loose the non-trivial reasoning > behind the code existence. > > > > > That disaster of a hack has been replaced with the much more functional > > code that deals with setting up the root and the devfs filesystem on > > /dev. So the need for which it was designed no longer exists. But I > > still believe that it is the correct thing to do. For example, if you > > are using automount code and chdir into your home directory triggering > > an auto-mount, you should just be in your home directory after the > > mount rather than having to do cd ../$USER to get there. > I believe that the current autofs does not allow a process to get into > this situation at all. > It does. For example: [trasz@v2:~]% cd /media/md0 [trasz@v2:/media/md0]% mount /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) devfs on /dev (devfs, local, multilabel) map -hosts on /net (autofs) map -media on /media (autofs) [trasz@v2:/media/md0]% ls [trasz@v2:/media/md0]% mount /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) devfs on /dev (devfs, local, multilabel) map -hosts on /net (autofs) map -media on /media (autofs) /dev/md0 on /media/md0 (ufs, local, noatime, nosuid, automounted) Getting rid of mountcheckdirs() in the unmount path should be fine, I think. From owner-freebsd-arch@freebsd.org Fri Sep 15 10:30:43 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D9DBE0C9CC; Fri, 15 Sep 2017 10:30:43 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 125A12012; Fri, 15 Sep 2017 10:30:42 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v8FAUb7f065316 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 15 Sep 2017 13:30:37 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v8FAUb7f065316 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v8FAUbZt065314; Fri, 15 Sep 2017 13:30:37 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 15 Sep 2017 13:30:37 +0300 From: Konstantin Belousov To: Edward Napierala Cc: Kirk McKusick , freebsd-fs , Andriy Gapon , "freebsd-arch@freebsd.org" Subject: Re: mount / unmount and mountcheckdirs() Message-ID: <20170915103037.GM78693@kib.kiev.ua> References: <134c7c6e-f4f1-ef38-cc50-0e56c27c9fb8@FreeBSD.org> <201709150314.v8F3Ea6B085072@chez.mckusick.com> <20170915092001.GK78693@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.0 (2017-09-02) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 10:30:43 -0000 On Fri, Sep 15, 2017 at 11:08:31AM +0100, Edward Napierala wrote: > 2017-09-15 10:20 GMT+01:00 Konstantin Belousov : > > I believe that the current autofs does not allow a process to get into > > this situation at all. > > > > It does. For example: > > [trasz@v2:~]% cd /media/md0 > [trasz@v2:/media/md0]% mount > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > devfs on /dev (devfs, local, multilabel) > map -hosts on /net (autofs) > map -media on /media (autofs) > [trasz@v2:/media/md0]% ls > [trasz@v2:/media/md0]% mount > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > devfs on /dev (devfs, local, multilabel) > map -hosts on /net (autofs) > map -media on /media (autofs) > /dev/md0 on /media/md0 (ufs, local, noatime, nosuid, automounted) > > Getting rid of mountcheckdirs() in the unmount path should be fine, I think. How the example proves that mountcheckdirs() can be removed ? How can we see which directory content was printed by ls, the covered or mounted ? From owner-freebsd-arch@freebsd.org Fri Sep 15 10:49:19 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1409EE0D7EE; Fri, 15 Sep 2017 10:49:19 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-ua0-x22a.google.com (mail-ua0-x22a.google.com [IPv6:2607:f8b0:400c:c08::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BCCC029EE; Fri, 15 Sep 2017 10:49:18 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-ua0-x22a.google.com with SMTP id t5so1033493uad.13; Fri, 15 Sep 2017 03:49:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=JvUxFc3iQf2tK20/Xg7cXWHxAE9QNx105L4hLj9hTJU=; b=tBeEV1zKSsrO7PT47ysShzYNIolFkAW0CfJylCSFmnEhDY6ZT9kgNiz2XR0l48c4Ts Bmv5hig88Eb9MiSxhSNF2xmtEsEH5AsoZmUsx5O6HdLMcOd9dp3nBDJmDY90mB9OG6TJ KbnTBQVHOGsT+bIYJR3zis1TH1y2gkgE6BAkjjJ1jKCywNr6YwUde/tpZKBld1Lw23bM PZWKNDWyGl+nc9uKKZJjkFuAplB27knxVh9cTJy6qr1Yfbw/7C8665qCheO6Cpw3Ymuf iy1RVYuvWYvo3L1KvvcpSmHEscbMhRubOmNAJc9HjgXDEboebCSHRwpYMSohMxTtu58M qMuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=JvUxFc3iQf2tK20/Xg7cXWHxAE9QNx105L4hLj9hTJU=; b=V2ONm50Kj2WHgWwlSXwq8YLS2nl6pEgpBfBCyNUUsgdN6F/v3OovJjV9UMf8K+xAty IAXSp15WeqvyDc80PuJda4AdzUVgaFbYFVHcwyjTaLu9svO/od7+hIHuNBHjItzRBEYn NsMTtBEZKAD8/eUrTRt8eBqp6V1FNUjdL8RTchSDQ8R9UazXmKZ/sMGN6srHm+/OEfys ZKncfxKtxRMjpZovSHmVFxQaeniKivzFeKMcQStL+frfaXk1sTXXRT8mQaD8xAEFFQFB 5lZylaB703Dp66/Zk+hsMh2UkluJIvpa+Cyc+JYd0e+46u4beXy/EG7k9nBCN9VZhts+ qcHw== X-Gm-Message-State: AHPjjUgqlD46efSK1UR/oACLLeCSz97JF8xanz+1SINtbdxZVce+58V/ JKyqxqJmNx0XtwOAcbzqTiv8ytXyAKhCOnB1fRw= X-Google-Smtp-Source: AOwi7QAKe5oLEoNiGKpgmJKxqY5j9fk7JoVcuwqj+WtI4dflqwHlboZ0ayKkTBSAuTTquJTl41gK506818mwW6KPt78= X-Received: by 10.176.83.123 with SMTP id y56mr21175614uay.172.1505472557839; Fri, 15 Sep 2017 03:49:17 -0700 (PDT) MIME-Version: 1.0 Sender: etnapierala@gmail.com Received: by 10.159.51.232 with HTTP; Fri, 15 Sep 2017 03:49:17 -0700 (PDT) In-Reply-To: <20170915103037.GM78693@kib.kiev.ua> References: <134c7c6e-f4f1-ef38-cc50-0e56c27c9fb8@FreeBSD.org> <201709150314.v8F3Ea6B085072@chez.mckusick.com> <20170915092001.GK78693@kib.kiev.ua> <20170915103037.GM78693@kib.kiev.ua> From: Edward Napierala Date: Fri, 15 Sep 2017 11:49:17 +0100 X-Google-Sender-Auth: wZ_JDot4INtpYyYVBEu9HHbFkw4 Message-ID: Subject: Re: mount / unmount and mountcheckdirs() To: Konstantin Belousov Cc: Kirk McKusick , freebsd-fs , Andriy Gapon , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 10:49:19 -0000 2017-09-15 11:30 GMT+01:00 Konstantin Belousov : > On Fri, Sep 15, 2017 at 11:08:31AM +0100, Edward Napierala wrote: > > 2017-09-15 10:20 GMT+01:00 Konstantin Belousov : > > > I believe that the current autofs does not allow a process to get into > > > this situation at all. > > > > > > > It does. For example: > > > > [trasz@v2:~]% cd /media/md0 > > [trasz@v2:/media/md0]% mount > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > devfs on /dev (devfs, local, multilabel) > > map -hosts on /net (autofs) > > map -media on /media (autofs) > > [trasz@v2:/media/md0]% ls > > [trasz@v2:/media/md0]% mount > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > devfs on /dev (devfs, local, multilabel) > > map -hosts on /net (autofs) > > map -media on /media (autofs) > > /dev/md0 on /media/md0 (ufs, local, noatime, nosuid, automounted) > > > > Getting rid of mountcheckdirs() in the unmount path should be fine, I > think. > How the example proves that mountcheckdirs() can be removed ? In the unmount path? It doesn't; I just don't think autofs would be affected, since having a current working directory in a mountpoint prevents the unmount (unless it's forced). In the mount path? See below. > How can > we see which directory content was printed by ls, the covered or mounted ? > Well, it would be pretty useless if you got the covered directory. But ok, here's a better example, one that actually shows that: [trasz@v2:/media/md0]% mount /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) devfs on /dev (devfs, local, multilabel) map -hosts on /net (autofs) map -media on /media (autofs) [trasz@v2:/media/md0]% ls -al total 9 drwxr-xr-x 3 root wheel 512 Sep 10 10:16 . drwxr-xr-x 3 root wheel 512 Sep 8 11:42 .. drwxrwxr-x 2 root operator 512 Sep 10 10:16 .snap [trasz@v2:/media/md0]% mount /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) devfs on /dev (devfs, local, multilabel) map -hosts on /net (autofs) map -media on /media (autofs) /dev/md0 on /media/md0 (ufs, local, noatime, nosuid, automounted) Notice the .snap/ directory; it's an empty UFS filesystem. From owner-freebsd-arch@freebsd.org Fri Sep 15 11:00:38 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E806BE0E2AD; Fri, 15 Sep 2017 11:00:38 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8EB9E3131; Fri, 15 Sep 2017 11:00:38 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v8FB0XC3072038 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 15 Sep 2017 14:00:34 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v8FB0XC3072038 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v8FB0XmQ072037; Fri, 15 Sep 2017 14:00:33 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 15 Sep 2017 14:00:33 +0300 From: Konstantin Belousov To: Edward Napierala Cc: Kirk McKusick , freebsd-fs , Andriy Gapon , "freebsd-arch@freebsd.org" Subject: Re: mount / unmount and mountcheckdirs() Message-ID: <20170915110033.GO78693@kib.kiev.ua> References: <134c7c6e-f4f1-ef38-cc50-0e56c27c9fb8@FreeBSD.org> <201709150314.v8F3Ea6B085072@chez.mckusick.com> <20170915092001.GK78693@kib.kiev.ua> <20170915103037.GM78693@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.0 (2017-09-02) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 11:00:39 -0000 On Fri, Sep 15, 2017 at 11:49:17AM +0100, Edward Napierala wrote: > 2017-09-15 11:30 GMT+01:00 Konstantin Belousov : > > > On Fri, Sep 15, 2017 at 11:08:31AM +0100, Edward Napierala wrote: > > > 2017-09-15 10:20 GMT+01:00 Konstantin Belousov : > > > > I believe that the current autofs does not allow a process to get into > > > > this situation at all. > > > > > > > > > > It does. For example: > > > > > > [trasz@v2:~]% cd /media/md0 > > > [trasz@v2:/media/md0]% mount > > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > > devfs on /dev (devfs, local, multilabel) > > > map -hosts on /net (autofs) > > > map -media on /media (autofs) > > > [trasz@v2:/media/md0]% ls > > > [trasz@v2:/media/md0]% mount > > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > > devfs on /dev (devfs, local, multilabel) > > > map -hosts on /net (autofs) > > > map -media on /media (autofs) > > > /dev/md0 on /media/md0 (ufs, local, noatime, nosuid, automounted) > > > > > > Getting rid of mountcheckdirs() in the unmount path should be fine, I > > think. > > How the example proves that mountcheckdirs() can be removed ? > > > In the unmount path? It doesn't; I just don't think autofs would be > affected, > since having a current working directory in a mountpoint prevents the > unmount > (unless it's forced). > > In the mount path? See below. > > > > How can > > we see which directory content was printed by ls, the covered or mounted ? > > > > Well, it would be pretty useless if you got the covered directory. This is exactly what is being discussed. > But ok, here's > a better example, one that actually shows that: > > [trasz@v2:/media/md0]% mount > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > devfs on /dev (devfs, local, multilabel) > map -hosts on /net (autofs) > map -media on /media (autofs) > [trasz@v2:/media/md0]% ls -al > total 9 > drwxr-xr-x 3 root wheel 512 Sep 10 10:16 . > drwxr-xr-x 3 root wheel 512 Sep 8 11:42 .. > drwxrwxr-x 2 root operator 512 Sep 10 10:16 .snap > [trasz@v2:/media/md0]% mount > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > devfs on /dev (devfs, local, multilabel) > map -hosts on /net (autofs) > map -media on /media (autofs) > /dev/md0 on /media/md0 (ufs, local, noatime, nosuid, automounted) > > Notice the .snap/ directory; it's an empty UFS filesystem. And the .snap is from the /dev/md0 volume, right ? If yes, then this demonstration indeed proves that autofs behaves as intended on mount. From owner-freebsd-arch@freebsd.org Fri Sep 15 11:31:07 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 77BF2E0F8FB; Fri, 15 Sep 2017 11:31:07 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-ua0-x229.google.com (mail-ua0-x229.google.com [IPv6:2607:f8b0:400c:c08::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 299D33F35; Fri, 15 Sep 2017 11:31:07 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-ua0-x229.google.com with SMTP id 72so1070216uas.8; Fri, 15 Sep 2017 04:31:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=BCI+uCZ99PDTwjj4fnECwhYjMbmHlI4kV7UO85WAj8k=; b=U0vg7NkxyyqeGXIDr/C6/jmM6AJ5gVozWuK9AgDgydTm6oHMOZiKNM4eVkfPnqtw96 ahfiOjiFXo+n35bbiJufPvHGgCO35W/el8DTt7syEWMmmhuYgc1A9KA8iBXnk+PcY0wH lqc6z8XdwC/Yx5DuM8GP8DHUqejEyExjexGrW+8J6Lwblkv59vT4O/XsJrS8SCEzpQNt QaPLHpbPWBYq8rMr/cnS0M74x1cyiLoewU6DLrFC3C+kZ+ooU2KKK9VRaw7AhJ/CyYQx UUnlxAMGeta+eczBC6nx+WjFVl1AcGIAlaStYXQnBdIo01i9jPTmIzngkRpozi5koJG0 F0eQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=BCI+uCZ99PDTwjj4fnECwhYjMbmHlI4kV7UO85WAj8k=; b=hL+wHv79DnMU3zQFG9+BJALOlB9x/m1CAJkYow0jz8HUh6c74eZuJfR+XADos/YhiT 97CwOKNgjkHrqURfNGOQ+MKU+4hlMHyA+yLv/+ESrZsNPazVkZagW6/OqJHxakjOvR9t /I+5O9Dycog+MRZ59IUN7BEEoVUYdEa4m2LTvfIbeJlufdjtl+9EmczU56s7eE/rm1UR /IR6uXSZmmMCGMeEtsGN7m1adCmUkAQ2WFd0L7wENSMrStEDSomA2sxb19sc8QZp+5No TMaf/P7J8fnrCpt4U1ZRYiSSBqmOCyxhJa8qzXgyyjgXL/LJIRB4us/LgkK0trrAJllS IEJQ== X-Gm-Message-State: AHPjjUiM2BPESQUGSOPP6JSuHolU1n4MZSyMoJ1KZqsQGYMkF4YH/Xfn GckE8S7Z5Bpbp/w+WMYibtQ4g0y4n9H0t9hy1bQ= X-Google-Smtp-Source: ADKCNb5aTm0GnEozTU1lPnkNjCT3fdmbp/nT0ji7/w9OI6+wGuezGPLbtCkGeU02divGOZokZ3ZkYIurzFdL7sIU3W8= X-Received: by 10.176.92.111 with SMTP id a47mr21213593uag.111.1505475065951; Fri, 15 Sep 2017 04:31:05 -0700 (PDT) MIME-Version: 1.0 Sender: etnapierala@gmail.com Received: by 10.159.51.232 with HTTP; Fri, 15 Sep 2017 04:31:05 -0700 (PDT) In-Reply-To: <20170915110033.GO78693@kib.kiev.ua> References: <134c7c6e-f4f1-ef38-cc50-0e56c27c9fb8@FreeBSD.org> <201709150314.v8F3Ea6B085072@chez.mckusick.com> <20170915092001.GK78693@kib.kiev.ua> <20170915103037.GM78693@kib.kiev.ua> <20170915110033.GO78693@kib.kiev.ua> From: Edward Napierala Date: Fri, 15 Sep 2017 12:31:05 +0100 X-Google-Sender-Auth: MfR-gur5i6VTgT8RqaOUE7DyMKA Message-ID: Subject: Re: mount / unmount and mountcheckdirs() To: Konstantin Belousov Cc: Kirk McKusick , freebsd-fs , Andriy Gapon , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 11:31:07 -0000 2017-09-15 12:00 GMT+01:00 Konstantin Belousov : > On Fri, Sep 15, 2017 at 11:49:17AM +0100, Edward Napierala wrote: > > 2017-09-15 11:30 GMT+01:00 Konstantin Belousov : > > > > > On Fri, Sep 15, 2017 at 11:08:31AM +0100, Edward Napierala wrote: > > > > 2017-09-15 10:20 GMT+01:00 Konstantin Belousov >: > > > > > I believe that the current autofs does not allow a process to get > into > > > > > this situation at all. > > > > > > > > > > > > > It does. For example: > > > > > > > > [trasz@v2:~]% cd /media/md0 > > > > [trasz@v2:/media/md0]% mount > > > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > > > devfs on /dev (devfs, local, multilabel) > > > > map -hosts on /net (autofs) > > > > map -media on /media (autofs) > > > > [trasz@v2:/media/md0]% ls > > > > [trasz@v2:/media/md0]% mount > > > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > > > devfs on /dev (devfs, local, multilabel) > > > > map -hosts on /net (autofs) > > > > map -media on /media (autofs) > > > > /dev/md0 on /media/md0 (ufs, local, noatime, nosuid, automounted) > > > > > > > > Getting rid of mountcheckdirs() in the unmount path should be fine, I > > > think. > > > How the example proves that mountcheckdirs() can be removed ? > > > > > > In the unmount path? It doesn't; I just don't think autofs would be > > affected, > > since having a current working directory in a mountpoint prevents the > > unmount > > (unless it's forced). > > > > In the mount path? See below. > > > > > > > How can > > > we see which directory content was printed by ls, the covered or > mounted ? > > > > > > > Well, it would be pretty useless if you got the covered directory. > This is exactly what is being discussed. > > > But ok, here's > > a better example, one that actually shows that: > > > > [trasz@v2:/media/md0]% mount > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > devfs on /dev (devfs, local, multilabel) > > map -hosts on /net (autofs) > > map -media on /media (autofs) > > [trasz@v2:/media/md0]% ls -al > > total 9 > > drwxr-xr-x 3 root wheel 512 Sep 10 10:16 . > > drwxr-xr-x 3 root wheel 512 Sep 8 11:42 .. > > drwxrwxr-x 2 root operator 512 Sep 10 10:16 .snap > > [trasz@v2:/media/md0]% mount > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > devfs on /dev (devfs, local, multilabel) > > map -hosts on /net (autofs) > > map -media on /media (autofs) > > /dev/md0 on /media/md0 (ufs, local, noatime, nosuid, automounted) > > > > Notice the .snap/ directory; it's an empty UFS filesystem. > And the .snap is from the /dev/md0 volume, right ? > Yes. > If yes, then this demonstration indeed proves that autofs behaves as > intended on mount. > Exactly. And, from what I understand, that would break if mountcheckdirs() got removed from the mount path. From owner-freebsd-arch@freebsd.org Fri Sep 15 11:45:39 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 872F4E105CB; Fri, 15 Sep 2017 11:45:39 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2C80F63B1C; Fri, 15 Sep 2017 11:45:39 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v8FBjXxI081996 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 15 Sep 2017 14:45:33 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v8FBjXxI081996 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v8FBjX85081994; Fri, 15 Sep 2017 14:45:33 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 15 Sep 2017 14:45:33 +0300 From: Konstantin Belousov To: Edward Napierala Cc: Kirk McKusick , freebsd-fs , Andriy Gapon , "freebsd-arch@freebsd.org" Subject: Re: mount / unmount and mountcheckdirs() Message-ID: <20170915114533.GS78693@kib.kiev.ua> References: <134c7c6e-f4f1-ef38-cc50-0e56c27c9fb8@FreeBSD.org> <201709150314.v8F3Ea6B085072@chez.mckusick.com> <20170915092001.GK78693@kib.kiev.ua> <20170915103037.GM78693@kib.kiev.ua> <20170915110033.GO78693@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.0 (2017-09-02) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 11:45:39 -0000 On Fri, Sep 15, 2017 at 12:31:05PM +0100, Edward Napierala wrote: > 2017-09-15 12:00 GMT+01:00 Konstantin Belousov : > > > On Fri, Sep 15, 2017 at 11:49:17AM +0100, Edward Napierala wrote: > > > 2017-09-15 11:30 GMT+01:00 Konstantin Belousov : > > > > > > > On Fri, Sep 15, 2017 at 11:08:31AM +0100, Edward Napierala wrote: > > > > > 2017-09-15 10:20 GMT+01:00 Konstantin Belousov > >: > > > > > > I believe that the current autofs does not allow a process to get > > into > > > > > > this situation at all. > > > > > > > > > > > > > > > > It does. For example: > > > > > > > > > > [trasz@v2:~]% cd /media/md0 > > > > > [trasz@v2:/media/md0]% mount > > > > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > > > > devfs on /dev (devfs, local, multilabel) > > > > > map -hosts on /net (autofs) > > > > > map -media on /media (autofs) > > > > > [trasz@v2:/media/md0]% ls > > > > > [trasz@v2:/media/md0]% mount > > > > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > > > > devfs on /dev (devfs, local, multilabel) > > > > > map -hosts on /net (autofs) > > > > > map -media on /media (autofs) > > > > > /dev/md0 on /media/md0 (ufs, local, noatime, nosuid, automounted) > > > > > > > > > > Getting rid of mountcheckdirs() in the unmount path should be fine, I > > > > think. > > > > How the example proves that mountcheckdirs() can be removed ? > > > > > > > > > In the unmount path? It doesn't; I just don't think autofs would be > > > affected, > > > since having a current working directory in a mountpoint prevents the > > > unmount > > > (unless it's forced). > > > > > > In the mount path? See below. > > > > > > > > > > How can > > > > we see which directory content was printed by ls, the covered or > > mounted ? > > > > > > > > > > Well, it would be pretty useless if you got the covered directory. > > This is exactly what is being discussed. > > > > > But ok, here's > > > a better example, one that actually shows that: > > > > > > [trasz@v2:/media/md0]% mount > > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > > devfs on /dev (devfs, local, multilabel) > > > map -hosts on /net (autofs) > > > map -media on /media (autofs) > > > [trasz@v2:/media/md0]% ls -al > > > total 9 > > > drwxr-xr-x 3 root wheel 512 Sep 10 10:16 . > > > drwxr-xr-x 3 root wheel 512 Sep 8 11:42 .. > > > drwxrwxr-x 2 root operator 512 Sep 10 10:16 .snap > > > [trasz@v2:/media/md0]% mount > > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > > devfs on /dev (devfs, local, multilabel) > > > map -hosts on /net (autofs) > > > map -media on /media (autofs) > > > /dev/md0 on /media/md0 (ufs, local, noatime, nosuid, automounted) > > > > > > Notice the .snap/ directory; it's an empty UFS filesystem. > > And the .snap is from the /dev/md0 volume, right ? > > > > Yes. > > > > If yes, then this demonstration indeed proves that autofs behaves as > > intended on mount. > > > > Exactly. And, from what I understand, that would break if mountcheckdirs() > got removed from the mount path. Even for autofs ? Don't autofs postpone lookup() completion until the mount occurs ? From owner-freebsd-arch@freebsd.org Fri Sep 15 11:57:56 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC065E10F4E; Fri, 15 Sep 2017 11:57:56 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id CE7C464151; Fri, 15 Sep 2017 11:57:54 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA20952; Fri, 15 Sep 2017 14:57:52 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1dspFo-000185-Dr; Fri, 15 Sep 2017 14:57:52 +0300 Subject: Re: mount / unmount and mountcheckdirs() To: Edward Napierala , Konstantin Belousov Cc: Kirk McKusick , freebsd-fs , "freebsd-arch@freebsd.org" References: <134c7c6e-f4f1-ef38-cc50-0e56c27c9fb8@FreeBSD.org> <201709150314.v8F3Ea6B085072@chez.mckusick.com> <20170915092001.GK78693@kib.kiev.ua> From: Andriy Gapon Message-ID: Date: Fri, 15 Sep 2017 14:56:55 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 11:57:56 -0000 On 15/09/2017 13:08, Edward Napierala wrote: > It does.  For example: > > [trasz@v2:~]% cd /media/md0 > [trasz@v2:/media/md0]% mount > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > devfs on /dev (devfs, local, multilabel) > map -hosts on /net (autofs) > map -media on /media (autofs) Interesting, I thought that /dev/md0 would get mounted as soon as /media/md0 is looked up. But maybe that would be sub-optimal for some common scenarios... FWIW, ZFS snapshots get auto-mounted under .zfs as soon as there is a lookup. > [trasz@v2:/media/md0]% ls > [trasz@v2:/media/md0]% mount > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > devfs on /dev (devfs, local, multilabel) > map -hosts on /net (autofs) > map -media on /media (autofs) > /dev/md0 on /media/md0 (ufs, local, noatime, nosuid, automounted) > -- Andriy Gapon From owner-freebsd-arch@freebsd.org Fri Sep 15 11:59:01 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5BA8CE11125; Fri, 15 Sep 2017 11:59:01 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 725D3643EC; Fri, 15 Sep 2017 11:59:00 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA20960; Fri, 15 Sep 2017 14:58:58 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1dspGs-000189-PW; Fri, 15 Sep 2017 14:58:58 +0300 Subject: Re: mount / unmount and mountcheckdirs() To: Kirk McKusick Cc: Konstantin Belousov , freebsd-arch@FreeBSD.org, freebsd-fs References: <201709150314.v8F3Ea6B085072@chez.mckusick.com> From: Andriy Gapon Message-ID: <8a8e6eb6-2a80-de22-4944-38820c327a76@FreeBSD.org> Date: Fri, 15 Sep 2017 14:58:23 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <201709150314.v8F3Ea6B085072@chez.mckusick.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 11:59:01 -0000 On 15/09/2017 06:14, Kirk McKusick wrote: > I had to dig back through some *really* old emails to find out what > triggered the addition of mountcheckdirs(). The problem that it was > specifically solving was that as part of the startup script a minimal > root directory was replaced by the real root directory. The shell > running the startup script needed to be moved to the new mounted-on > root so that the rest of the script would not fail. > > That disaster of a hack has been replaced with the much more functional > code that deals with setting up the root and the devfs filesystem on > /dev. So the need for which it was designed no longer exists. But I > still believe that it is the correct thing to do. For example, if you > are using automount code and chdir into your home directory triggering > an auto-mount, you should just be in your home directory after the > mount rather than having to do cd ../$USER to get there. Kirk, thank you very much both for the historical reference and for the auto-mount example. -- Andriy Gapon From owner-freebsd-arch@freebsd.org Fri Sep 15 15:19:17 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 888AEE1CA31; Fri, 15 Sep 2017 15:19:17 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1C1DB6B9B8; Fri, 15 Sep 2017 15:19:17 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id v8FFJFVt065087; Fri, 15 Sep 2017 08:19:15 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id v8FFJFHK065086; Fri, 15 Sep 2017 08:19:15 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201709151519.v8FFJFHK065086@pdx.rh.CN85.dnsmgr.net> Subject: Re: mount / unmount and mountcheckdirs() In-Reply-To: To: Edward Napierala Date: Fri, 15 Sep 2017 08:19:15 -0700 (PDT) CC: Konstantin Belousov , Kirk McKusick , freebsd-fs , Andriy Gapon , "freebsd-arch@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 15:19:17 -0000 > 2017-09-15 10:20 GMT+01:00 Konstantin Belousov : > > > On Thu, Sep 14, 2017 at 08:14:36PM -0700, Kirk McKusick wrote: > > > > To: Kirk McKusick > > > > Subject: Re: mount / unmount and mountcheckdirs() > > > > Cc: freebsd-arch@FreeBSD.org, freebsd-fs > > > > From: Andriy Gapon > > > > Date: Thu, 14 Sep 2017 15:45:07 +0300 > > > > > > > > On 22/05/2016 09:40, Kirk McKusick wrote: > > > >> I added the checkdirs functionality in the mount direction only > > > >> (I actually did it in 4.4BSD-Lite and it got swept in with commit > > > >> 22521). The reason is that when a directory that is not empty is > > > >> mounted on, the expectation is that the entries in that directory > > > >> should no longer be present; rather they should be replaced by the > > > >> entries in the newly mounted directory. Thus all processes sitting > > > >> in the mounted on directory should see the newly mounted directory > > > >> as if they had come to it using a lookup after the mount had been > > > >> done. If a process had proceeded through the mounted on directory > > > >> into one of its other entries, then they are left alone until such > > > >> time as they chdir back into the mount point directory through ".." > > > >> at which time they will be passed up to the mounted directory using > > > >> the same mechanism that would put them there if they traversed into > > > >> the mount point from above it in the tree. I believe this is the > > > >> correct behavior, is not a security threat, and should be left alone. > > > > > > > > I almost dropped a ball on this issue, but I am now picking it up > > again. > > > > At the moment I am moving forward with the dounmount change as it > > seems to be > > > > non-contentious and rather simple to do and test. > > > > > > > > Regarding the mount part, I am not sure that I completely agree with > > you. > > > > Even if mountcheckdirs() does not cause any problems in the mount > > path, I still > > > > fail to see its usefulness. Specifically, I still do not see any > > significant > > > > difference between the covered directory and any directory below it. > > So, if we > > > > leave the lower directories alone, while bother with the covered > > directory... > > > > > > > > The covered directory: > > > > - absolute paths work correctly > > > > - relative paths with enough ".." (one) can access the actual namespace > > > > - other relative paths operate on the shadowed sub-tree of the original > > > > filesystem > > > > > > > > > > > > The lower directories: > > > > - absolute paths work correctly > > > > - relative paths with enough ".." (> 1) can access the actual namespace > > > > - other relative paths operate on the shadowed sub-tree of the original > > > > filesystem > > > > > > > > The only difference I can think of is that the root of the mounted > > filesystem > > > > cannot be reached with just ".."-s from the covered directory. But is > > this > > > > difference of any significance? > > > > > > > > Mateusz also raised some interesting points. > > > > > > > > On the other hand, it seems that illumos and probably Solaris has > > > > interesting parallels to the FreeBSD behavior. It does not allow > > > > to mount over a directory that is a current directory for any process > > > > ("Device busy"), but does not object against processes in directories > > > > below the mount point. > > > > > > > > So, probably it's just I who misses something about that scenario :-) > > > > > > > >> I was not aware that the functionality had been added at unmount > > > >> time, and I do not believe that it should have been done. Normally > > > >> an unmount will not succeed if any vnodes are busy (for example, if > > > >> any directory in the filesystem is a current directory). The only > > > >> way that it can succeed in such a case is if a forcible unmount is > > > >> done. The forcible unmount will effectively do a revoke(2) on all > > > >> current directory vnodes in the unmounted filesystem. Further attempts > > > >> to access them will fail with "." not found errors. The only way to > > > >> get a valid current directory is to chdir to an absolute pathname. > > > >> Gratuitously fixing this if you happen to be in the former root of > > > >> the filesystem is wrong. And as you note can lead to unintensionally > > > >> giving an escape path from a prison. So I concur with your removing > > > >> this added functionality. > > > > -- > > > > Andriy Gapon > > > > > > I had to dig back through some *really* old emails to find out what > > > triggered the addition of mountcheckdirs(). The problem that it was > > > specifically solving was that as part of the startup script a minimal > > > root directory was replaced by the real root directory. The shell > > > running the startup script needed to be moved to the new mounted-on > > > root so that the rest of the script would not fail. > > If the mountcheckdirs() code not going away, please add your spelunking > > results as a comment before the function. This theme is recurring, and > > it would be highly beneficial to not loose the non-trivial reasoning > > behind the code existence. > > > > > > > > That disaster of a hack has been replaced with the much more functional > > > code that deals with setting up the root and the devfs filesystem on > > > /dev. So the need for which it was designed no longer exists. But I > > > still believe that it is the correct thing to do. For example, if you > > > are using automount code and chdir into your home directory triggering > > > an auto-mount, you should just be in your home directory after the > > > mount rather than having to do cd ../$USER to get there. > > I believe that the current autofs does not allow a process to get into > > this situation at all. > > > > It does. For example: > > [trasz@v2:~]% cd /media/md0 This process should be blocked before the cd is completed and resumed once the automouter has done its job. Even per the autofs man page that is what should be happening: DESCRIPTION The autofs driver is the kernel component of the automounter infrastructure. Its job is to pass mount requests to the automountd(8) daemon, and pause the processes trying to access the automounted ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ filesystem until the mount is completed. It is mounted by the ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ automount(8). > [trasz@v2:/media/md0]% mount > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > devfs on /dev (devfs, local, multilabel) > map -hosts on /net (autofs) > map -media on /media (autofs) > [trasz@v2:/media/md0]% ls This looks like a fail of the autofs code to block the process in cd before it changes the cwd. > [trasz@v2:/media/md0]% mount > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > devfs on /dev (devfs, local, multilabel) > map -hosts on /net (autofs) > map -media on /media (autofs) > /dev/md0 on /media/md0 (ufs, local, noatime, nosuid, automounted) > > Getting rid of mountcheckdirs() in the unmount path should be fine, I think. -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-arch@freebsd.org Fri Sep 15 15:35:03 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EE8A7E1DCF5; Fri, 15 Sep 2017 15:35:03 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9D7796C659; Fri, 15 Sep 2017 15:35:03 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id v8FFAP2N065054; Fri, 15 Sep 2017 08:10:25 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id v8FFAPcg065053; Fri, 15 Sep 2017 08:10:25 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201709151510.v8FFAPcg065053@pdx.rh.CN85.dnsmgr.net> Subject: Re: mount / unmount and mountcheckdirs() In-Reply-To: <20170915092001.GK78693@kib.kiev.ua> To: Konstantin Belousov Date: Fri, 15 Sep 2017 08:10:25 -0700 (PDT) CC: Kirk McKusick , freebsd-fs , Andriy Gapon , freebsd-arch@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 15:35:04 -0000 > On Thu, Sep 14, 2017 at 08:14:36PM -0700, Kirk McKusick wrote: > > > To: Kirk McKusick > > > Subject: Re: mount / unmount and mountcheckdirs() > > > Cc: freebsd-arch@FreeBSD.org, freebsd-fs > > > From: Andriy Gapon > > > Date: Thu, 14 Sep 2017 15:45:07 +0300 > > > > > > On 22/05/2016 09:40, Kirk McKusick wrote: > > >> I added the checkdirs functionality in the mount direction only > > >> (I actually did it in 4.4BSD-Lite and it got swept in with commit > > >> 22521). The reason is that when a directory that is not empty is > > >> mounted on, the expectation is that the entries in that directory > > >> should no longer be present; rather they should be replaced by the > > >> entries in the newly mounted directory. Thus all processes sitting > > >> in the mounted on directory should see the newly mounted directory > > >> as if they had come to it using a lookup after the mount had been > > >> done. If a process had proceeded through the mounted on directory > > >> into one of its other entries, then they are left alone until such > > >> time as they chdir back into the mount point directory through ".." > > >> at which time they will be passed up to the mounted directory using > > >> the same mechanism that would put them there if they traversed into > > >> the mount point from above it in the tree. I believe this is the > > >> correct behavior, is not a security threat, and should be left alone. > > > > > > I almost dropped a ball on this issue, but I am now picking it up again. > > > At the moment I am moving forward with the dounmount change as it seems to be > > > non-contentious and rather simple to do and test. > > > > > > Regarding the mount part, I am not sure that I completely agree with you. > > > Even if mountcheckdirs() does not cause any problems in the mount path, I still > > > fail to see its usefulness. Specifically, I still do not see any significant > > > difference between the covered directory and any directory below it. So, if we > > > leave the lower directories alone, while bother with the covered directory... > > > > > > The covered directory: > > > - absolute paths work correctly > > > - relative paths with enough ".." (one) can access the actual namespace > > > - other relative paths operate on the shadowed sub-tree of the original > > > filesystem > > > > > > > > > The lower directories: > > > - absolute paths work correctly > > > - relative paths with enough ".." (> 1) can access the actual namespace > > > - other relative paths operate on the shadowed sub-tree of the original > > > filesystem > > > > > > The only difference I can think of is that the root of the mounted filesystem > > > cannot be reached with just ".."-s from the covered directory. But is this > > > difference of any significance? > > > > > > Mateusz also raised some interesting points. > > > > > > On the other hand, it seems that illumos and probably Solaris has > > > interesting parallels to the FreeBSD behavior. It does not allow > > > to mount over a directory that is a current directory for any process > > > ("Device busy"), but does not object against processes in directories > > > below the mount point. > > > > > > So, probably it's just I who misses something about that scenario :-) > > > > > >> I was not aware that the functionality had been added at unmount > > >> time, and I do not believe that it should have been done. Normally > > >> an unmount will not succeed if any vnodes are busy (for example, if > > >> any directory in the filesystem is a current directory). The only > > >> way that it can succeed in such a case is if a forcible unmount is > > >> done. The forcible unmount will effectively do a revoke(2) on all > > >> current directory vnodes in the unmounted filesystem. Further attempts > > >> to access them will fail with "." not found errors. The only way to > > >> get a valid current directory is to chdir to an absolute pathname. > > >> Gratuitously fixing this if you happen to be in the former root of > > >> the filesystem is wrong. And as you note can lead to unintensionally > > >> giving an escape path from a prison. So I concur with your removing > > >> this added functionality. > > > -- > > > Andriy Gapon > > > > I had to dig back through some *really* old emails to find out what > > triggered the addition of mountcheckdirs(). The problem that it was > > specifically solving was that as part of the startup script a minimal > > root directory was replaced by the real root directory. The shell > > running the startup script needed to be moved to the new mounted-on > > root so that the rest of the script would not fail. > If the mountcheckdirs() code not going away, please add your spelunking > results as a comment before the function. This theme is recurring, and > it would be highly beneficial to not loose the non-trivial reasoning > behind the code existence. > > > > > That disaster of a hack has been replaced with the much more functional > > code that deals with setting up the root and the devfs filesystem on > > /dev. So the need for which it was designed no longer exists. But I > > still believe that it is the correct thing to do. For example, if you > > are using automount code and chdir into your home directory triggering > > an auto-mount, you should just be in your home directory after the > > mount rather than having to do cd ../$USER to get there. > I believe that the current autofs does not allow a process to get into > this situation at all. I believe the ancient automountd did not allow a process to get into this situation either, the process got blocked before the completion of the cd that triggers the automount, and continued after the (auto)mount completed. > > In fact, the behavior implemented by mountcheckdirs() is surprising as > well. For instance, I did expected that the system would operate as if > mountcheckdirs() does not exist, and it caused me some head-scratching > when I see it first time. > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-arch@freebsd.org Fri Sep 15 16:02:05 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B37B8E1F52A; Fri, 15 Sep 2017 16:02:05 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-ua0-x229.google.com (mail-ua0-x229.google.com [IPv6:2607:f8b0:400c:c08::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 67B346D643; Fri, 15 Sep 2017 16:02:05 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-ua0-x229.google.com with SMTP id c27so1476635uah.2; Fri, 15 Sep 2017 09:02:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=aTqGj5uk1GG7tfE+vU6w9obFTF6Vdz5wtzR1inaQaVY=; b=vgo6jRb5lRk1OnaEsc2HMv3ESvQlsN0/VAwAFwXf1mQF2/aRnlZ3OmlISE3MdzStv9 6ecG6KzvzH8hIv68FGTeD8bmYPpTg9ElFp61BaIDQpTThxoZxbYEPoID37w9GXpXVu7f s+llTtQ24FpZQ6lT030x5CzYfqpVo2nnQ2t7fMMmUgd/VEs5uxkbQlrma6564u+F+K0U XIyu1Jrt1p6DaM7sAxa/G1pNB1R5H/zbbXWdMLmmX9Rbm0rI7GcHxZLgrGowsSSrzxc3 O3loVa4oG6xnXd4jKBrGv4T4Qt9B1btcafBajlmK0n9dlECRQ/mB/fHtoFwhsfUnbj6k m6dQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=aTqGj5uk1GG7tfE+vU6w9obFTF6Vdz5wtzR1inaQaVY=; b=eu+3xhS50dzGwzPryvgQdSO9vc/nONUd6N29mMFTHkyD+GaWfkkJS3sNiIT5nkh5cT d7RS++gaVwZRhg/N8kwfoBfl4dVIxA+ANmO1Wd8rqXaMSd1WZ0Bpqf3fs+xtdcki7xd3 imQwM7OtWDMqLpqRWgTauhg2b+cJuBmbJ/60dCS4wh0SYJa0QXYEfEf0a3I4jxA7vSiR plUv3R4E3pPXfqsLekJJeTYjq4x37i+S1x9cTu1AGLFLcIdWVJtbo6eMTvy2RB8U9ytP QO93QHrFdbMuq4XxwaFPkxLI3PO5PvcBFvpIOaDwFk8u0ejkQd1+BgTh/6tdbaj1NqHv br7Q== X-Gm-Message-State: AHPjjUiEBQxY8ilnCc35oYiIZ7+XoC9KSANlt0AFKKWppJgSAl7leSYd fuVCR7nPtH2/X6AFJbgFR0OROR1QO1eDCQonfhI= X-Google-Smtp-Source: ADKCNb7RmgZhGgKJq+HyxU7oDSIZ0S6E1vdo7E1BCrBeM64NEjco6mS1JxJNzvam1JIxl5DqhVGWwI/1eonwk5Rww40= X-Received: by 10.176.92.111 with SMTP id a47mr22014144uag.111.1505491324300; Fri, 15 Sep 2017 09:02:04 -0700 (PDT) MIME-Version: 1.0 Sender: etnapierala@gmail.com Received: by 10.159.51.232 with HTTP; Fri, 15 Sep 2017 09:02:03 -0700 (PDT) In-Reply-To: References: <134c7c6e-f4f1-ef38-cc50-0e56c27c9fb8@FreeBSD.org> <201709150314.v8F3Ea6B085072@chez.mckusick.com> <20170915092001.GK78693@kib.kiev.ua> From: Edward Napierala Date: Fri, 15 Sep 2017 17:02:03 +0100 X-Google-Sender-Auth: YQoXymOHcUVy7BkjINTxt8_Zdv0 Message-ID: Subject: Re: mount / unmount and mountcheckdirs() To: Andriy Gapon Cc: Konstantin Belousov , Kirk McKusick , freebsd-fs , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 16:02:05 -0000 2017-09-15 12:56 GMT+01:00 Andriy Gapon : > On 15/09/2017 13:08, Edward Napierala wrote: > > It does. For example: > > > > [trasz@v2:~]% cd /media/md0 > > [trasz@v2:/media/md0]% mount > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > devfs on /dev (devfs, local, multilabel) > > map -hosts on /net (autofs) > > map -media on /media (autofs) > > Interesting, I thought that /dev/md0 would get mounted as soon as > /media/md0 is > looked up. But maybe that would be sub-optimal for some common > scenarios... > FWIW, ZFS snapshots get auto-mounted under .zfs as soon as there is a > lookup. You really don't want it to work that way. If you have a directory with thousands of mountpoints inside, you don't want to mount all of them every time you do "ls" in that directory. Same problem with mounting on "cd". That's why autofs triggers in three cases: in VOP_READDIR(9), VOP_GETATTR(9), and VOP_LOOKUP(9) with the vnode as a parent dir. From owner-freebsd-arch@freebsd.org Fri Sep 15 16:06:11 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E53A3E1F972; Fri, 15 Sep 2017 16:06:11 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-vk0-x232.google.com (mail-vk0-x232.google.com [IPv6:2607:f8b0:400c:c05::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 97E2D6D958; Fri, 15 Sep 2017 16:06:11 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-vk0-x232.google.com with SMTP id z187so1243413vkd.13; Fri, 15 Sep 2017 09:06:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=t+d19FruIl6nVb34JPaYdPppoSC01L1YhVjmk3RJQMY=; b=FF0lVHJRCb1Kw6zki8WJ5nGbHOZ/oKmnoLXdHSoEThD1bvEEq6P22ljBlNyHnk9jAY ZR0CF8NKdmwHNNB/hej6hcxyrIm+7J2gi9z//k4/RPp5l+SFx9jxDwx2vCwu68zJR27C JkzZ5ZqHhK23cvUKW3uNPXAeNsOvF2GbITOW9xE88kAQwUxleEN0bW38sCES4FMVJRty EM71jMtzNy83UM0HlpvbNOb2cznfP1P2YxtofXHepzAv+TD/lPp+5NAuvTVA1L8JiKu6 LmAKyClmXvWQoco5MKJVdDuGMIEkPYetitU4keeNS0b28rRyMI0Yt1mONcGozjA3Rh1h lzNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=t+d19FruIl6nVb34JPaYdPppoSC01L1YhVjmk3RJQMY=; b=d28UWVLA/XHBC8aj+zYDAsZ7Owq1t9vM7C7j4ON+muBwY3UZId4pOwuOmJgXy9XywR Dkqav7F1n6dgnyU03MTLwW3pDWw8cGi98j8fm9GIata1713BeV0w38dfTbBSJ1vA0qim pqGqCXnPZQRnwjGE9uOgPkpkgZxr7P77cPA9Ea2f9QmTF+vz4F0fd4VgATimcJ2ZaAXE IrK3MvIoSGFhTLY2tE0Uoo5M8YNDHoBCTIIBa1Ogp0e0yh5svT4TBhApwYZkh2u4EsI3 7vE5U2j2TrhVHVdc64adl43avwZYLeQOEz9AD/oRvBbLVAX8yWC9q5onruXBos29v4UY SoyA== X-Gm-Message-State: AHPjjUgBnA8eUdjhhwWVIpQFavXOhBZjMDUpCB1/sUY8yCy7bQHA3qiq xafQtX1C9NZB1P5mAKgeBiS0mkgKLIL5VxOO38I= X-Google-Smtp-Source: AOwi7QDwvvBDFnW6Qpoq4ZO6sbhzScbi/yn3lwPPjOk3fQcug7RaUchHdqsoXLEghhIwY5R97ksiEtIb3Ubh+9nRXu8= X-Received: by 10.31.10.19 with SMTP id 19mr19326393vkk.3.1505491570639; Fri, 15 Sep 2017 09:06:10 -0700 (PDT) MIME-Version: 1.0 Sender: etnapierala@gmail.com Received: by 10.159.51.232 with HTTP; Fri, 15 Sep 2017 09:06:10 -0700 (PDT) In-Reply-To: <20170915114533.GS78693@kib.kiev.ua> References: <134c7c6e-f4f1-ef38-cc50-0e56c27c9fb8@FreeBSD.org> <201709150314.v8F3Ea6B085072@chez.mckusick.com> <20170915092001.GK78693@kib.kiev.ua> <20170915103037.GM78693@kib.kiev.ua> <20170915110033.GO78693@kib.kiev.ua> <20170915114533.GS78693@kib.kiev.ua> From: Edward Napierala Date: Fri, 15 Sep 2017 17:06:10 +0100 X-Google-Sender-Auth: qCK8B-Up8hPQ1H4SxiHOdnDAhy0 Message-ID: Subject: Re: mount / unmount and mountcheckdirs() To: Konstantin Belousov Cc: Kirk McKusick , freebsd-fs , Andriy Gapon , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 16:06:12 -0000 2017-09-15 12:45 GMT+01:00 Konstantin Belousov : > On Fri, Sep 15, 2017 at 12:31:05PM +0100, Edward Napierala wrote: > > 2017-09-15 12:00 GMT+01:00 Konstantin Belousov : > > > > > On Fri, Sep 15, 2017 at 11:49:17AM +0100, Edward Napierala wrote: > > > > 2017-09-15 11:30 GMT+01:00 Konstantin Belousov >: > > > > > > > > > On Fri, Sep 15, 2017 at 11:08:31AM +0100, Edward Napierala wrote: > > > > > > 2017-09-15 10:20 GMT+01:00 Konstantin Belousov < > kostikbel@gmail.com > > > >: > > > > > > > I believe that the current autofs does not allow a process to > get > > > into > > > > > > > this situation at all. > > > > > > > > > > > > > > > > > > > It does. For example: > > > > > > > > > > > > [trasz@v2:~]% cd /media/md0 > > > > > > [trasz@v2:/media/md0]% mount > > > > > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > > > > > devfs on /dev (devfs, local, multilabel) > > > > > > map -hosts on /net (autofs) > > > > > > map -media on /media (autofs) > > > > > > [trasz@v2:/media/md0]% ls > > > > > > [trasz@v2:/media/md0]% mount > > > > > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > > > > > devfs on /dev (devfs, local, multilabel) > > > > > > map -hosts on /net (autofs) > > > > > > map -media on /media (autofs) > > > > > > /dev/md0 on /media/md0 (ufs, local, noatime, nosuid, automounted) > > > > > > > > > > > > Getting rid of mountcheckdirs() in the unmount path should be > fine, I > > > > > think. > > > > > How the example proves that mountcheckdirs() can be removed ? > > > > > > > > > > > > In the unmount path? It doesn't; I just don't think autofs would be > > > > affected, > > > > since having a current working directory in a mountpoint prevents the > > > > unmount > > > > (unless it's forced). > > > > > > > > In the mount path? See below. > > > > > > > > > > > > > How can > > > > > we see which directory content was printed by ls, the covered or > > > mounted ? > > > > > > > > > > > > > Well, it would be pretty useless if you got the covered directory. > > > This is exactly what is being discussed. > > > > > > > But ok, here's > > > > a better example, one that actually shows that: > > > > > > > > [trasz@v2:/media/md0]% mount > > > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > > > devfs on /dev (devfs, local, multilabel) > > > > map -hosts on /net (autofs) > > > > map -media on /media (autofs) > > > > [trasz@v2:/media/md0]% ls -al > > > > total 9 > > > > drwxr-xr-x 3 root wheel 512 Sep 10 10:16 . > > > > drwxr-xr-x 3 root wheel 512 Sep 8 11:42 .. > > > > drwxrwxr-x 2 root operator 512 Sep 10 10:16 .snap > > > > [trasz@v2:/media/md0]% mount > > > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > > > devfs on /dev (devfs, local, multilabel) > > > > map -hosts on /net (autofs) > > > > map -media on /media (autofs) > > > > /dev/md0 on /media/md0 (ufs, local, noatime, nosuid, automounted) > > > > > > > > Notice the .snap/ directory; it's an empty UFS filesystem. > > > And the .snap is from the /dev/md0 volume, right ? > > > > > > > Yes. > > > > > > > If yes, then this demonstration indeed proves that autofs behaves as > > > intended on mount. > > > > > > > Exactly. And, from what I understand, that would break if > mountcheckdirs() > > got removed from the mount path. > > Even for autofs ? Don't autofs postpone lookup() completion until the > mount occurs ? > It does, and that case might work - I'm not sure, but it might. But what happens afterwards, after the syscall that triggered the mount completes? You would still have a shell with cwd in that dir, now covered by another filesystem. Autofs doesn't do anything to "lift" the process to the newly mounted filesystem root. From owner-freebsd-arch@freebsd.org Fri Sep 15 16:28:02 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B3923E20D75; Fri, 15 Sep 2017 16:28:02 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5B5DE6E387; Fri, 15 Sep 2017 16:28:02 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v8FGRrrV045173 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 15 Sep 2017 19:27:53 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v8FGRrrV045173 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v8FGRqJq045172; Fri, 15 Sep 2017 19:27:52 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 15 Sep 2017 19:27:52 +0300 From: Konstantin Belousov To: Edward Napierala Cc: Kirk McKusick , freebsd-fs , Andriy Gapon , "freebsd-arch@freebsd.org" Subject: Re: mount / unmount and mountcheckdirs() Message-ID: <20170915162752.GX78693@kib.kiev.ua> References: <134c7c6e-f4f1-ef38-cc50-0e56c27c9fb8@FreeBSD.org> <201709150314.v8F3Ea6B085072@chez.mckusick.com> <20170915092001.GK78693@kib.kiev.ua> <20170915103037.GM78693@kib.kiev.ua> <20170915110033.GO78693@kib.kiev.ua> <20170915114533.GS78693@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.0 (2017-09-02) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 16:28:02 -0000 On Fri, Sep 15, 2017 at 05:06:10PM +0100, Edward Napierala wrote: > 2017-09-15 12:45 GMT+01:00 Konstantin Belousov : > > > On Fri, Sep 15, 2017 at 12:31:05PM +0100, Edward Napierala wrote: > > > 2017-09-15 12:00 GMT+01:00 Konstantin Belousov : > > > > > > > On Fri, Sep 15, 2017 at 11:49:17AM +0100, Edward Napierala wrote: > > > > > 2017-09-15 11:30 GMT+01:00 Konstantin Belousov > >: > > > > > > > > > > > On Fri, Sep 15, 2017 at 11:08:31AM +0100, Edward Napierala wrote: > > > > > > > 2017-09-15 10:20 GMT+01:00 Konstantin Belousov < > > kostikbel@gmail.com > > > > >: > > > > > > > > I believe that the current autofs does not allow a process to > > get > > > > into > > > > > > > > this situation at all. > > > > > > > > > > > > > > > > > > > > > > It does. For example: > > > > > > > > > > > > > > [trasz@v2:~]% cd /media/md0 > > > > > > > [trasz@v2:/media/md0]% mount > > > > > > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > > > > > > devfs on /dev (devfs, local, multilabel) > > > > > > > map -hosts on /net (autofs) > > > > > > > map -media on /media (autofs) > > > > > > > [trasz@v2:/media/md0]% ls > > > > > > > [trasz@v2:/media/md0]% mount > > > > > > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > > > > > > devfs on /dev (devfs, local, multilabel) > > > > > > > map -hosts on /net (autofs) > > > > > > > map -media on /media (autofs) > > > > > > > /dev/md0 on /media/md0 (ufs, local, noatime, nosuid, automounted) > > > > > > > > > > > > > > Getting rid of mountcheckdirs() in the unmount path should be > > fine, I > > > > > > think. > > > > > > How the example proves that mountcheckdirs() can be removed ? > > > > > > > > > > > > > > > In the unmount path? It doesn't; I just don't think autofs would be > > > > > affected, > > > > > since having a current working directory in a mountpoint prevents the > > > > > unmount > > > > > (unless it's forced). > > > > > > > > > > In the mount path? See below. > > > > > > > > > > > > > > > > How can > > > > > > we see which directory content was printed by ls, the covered or > > > > mounted ? > > > > > > > > > > > > > > > > Well, it would be pretty useless if you got the covered directory. > > > > This is exactly what is being discussed. > > > > > > > > > But ok, here's > > > > > a better example, one that actually shows that: > > > > > > > > > > [trasz@v2:/media/md0]% mount > > > > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > > > > devfs on /dev (devfs, local, multilabel) > > > > > map -hosts on /net (autofs) > > > > > map -media on /media (autofs) > > > > > [trasz@v2:/media/md0]% ls -al > > > > > total 9 > > > > > drwxr-xr-x 3 root wheel 512 Sep 10 10:16 . > > > > > drwxr-xr-x 3 root wheel 512 Sep 8 11:42 .. > > > > > drwxrwxr-x 2 root operator 512 Sep 10 10:16 .snap > > > > > [trasz@v2:/media/md0]% mount > > > > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > > > > devfs on /dev (devfs, local, multilabel) > > > > > map -hosts on /net (autofs) > > > > > map -media on /media (autofs) > > > > > /dev/md0 on /media/md0 (ufs, local, noatime, nosuid, automounted) > > > > > > > > > > Notice the .snap/ directory; it's an empty UFS filesystem. > > > > And the .snap is from the /dev/md0 volume, right ? > > > > > > > > > > Yes. > > > > > > > > > > If yes, then this demonstration indeed proves that autofs behaves as > > > > intended on mount. > > > > > > > > > > Exactly. And, from what I understand, that would break if > > mountcheckdirs() > > > got removed from the mount path. > > > > Even for autofs ? Don't autofs postpone lookup() completion until the > > mount occurs ? > > > > It does, and that case might work - I'm not sure, but it might. But what > happens afterwards, after the syscall that triggered the mount completes? > You would still have a shell with cwd in that dir, now covered by another > filesystem. Autofs doesn't do anything to "lift" the process to the newly > mounted filesystem root. The change of the current directory should have already triggered the action, because it was preceeded by lookup. At least this is my understanding of the flow. From owner-freebsd-arch@freebsd.org Fri Sep 15 16:37:08 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AD01CE21541; Fri, 15 Sep 2017 16:37:08 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 6B4836E84A; Fri, 15 Sep 2017 16:37:07 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA21439; Fri, 15 Sep 2017 19:37:00 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1dstbv-0001KC-SP; Fri, 15 Sep 2017 19:36:59 +0300 Subject: Re: mount / unmount and mountcheckdirs() To: Edward Napierala Cc: Konstantin Belousov , Kirk McKusick , freebsd-fs , "freebsd-arch@freebsd.org" References: <134c7c6e-f4f1-ef38-cc50-0e56c27c9fb8@FreeBSD.org> <201709150314.v8F3Ea6B085072@chez.mckusick.com> <20170915092001.GK78693@kib.kiev.ua> From: Andriy Gapon Message-ID: <59255d6c-07fc-19ab-0a8a-914ada69680a@FreeBSD.org> Date: Fri, 15 Sep 2017 19:36:04 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 16:37:08 -0000 On 15/09/2017 19:02, Edward Napierala wrote: > 2017-09-15 12:56 GMT+01:00 Andriy Gapon >: > > On 15/09/2017 13:08, Edward Napierala wrote: > > It does.  For example: > > > > [trasz@v2:~]% cd /media/md0 > > [trasz@v2:/media/md0]% mount > > /dev/ada0s1a on / (ufs, local, noatime, journaled soft-updates) > > devfs on /dev (devfs, local, multilabel) > > map -hosts on /net (autofs) > > map -media on /media (autofs) > > Interesting, I thought that /dev/md0 would get mounted as soon as /media/md0 is > looked up.  But maybe that would be sub-optimal for some common scenarios... > FWIW, ZFS snapshots get auto-mounted under .zfs as soon as there is a lookup. > > > You really don't want it to work that way.  If you have a directory with thousands > of mountpoints inside, you don't want to mount all of them every time you do "ls" > in that directory. Well, "ls" wouldn't trigger the mounting as it's just VOP_READDIR in the parent vnode, "ls -l" (or any other options that require file attributes) is a different story :-) > Same problem with mounting on "cd".  That's why autofs triggers > in three cases: in VOP_READDIR(9), VOP_GETATTR(9), and VOP_LOOKUP(9) > with the vnode as a parent dir. Well, "cd" is also a different story, you can't cd to hundreds of directories at once. Of course, the filesystem doesn't really know if it's "cd" or something else that triggered VOP_LOOKUP. Although VOP_ACCESS could be used as a hint, it is a rather weak hint. In any case, I prefer the mount-on-lookup approach as it is "more seamless". -- Andriy Gapon From owner-freebsd-arch@freebsd.org Fri Sep 15 16:41:01 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EE6CBE219FD; Fri, 15 Sep 2017 16:41:01 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A22376EB4B; Fri, 15 Sep 2017 16:41:00 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA21446; Fri, 15 Sep 2017 19:40:59 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1dstfn-0001KN-8a; Fri, 15 Sep 2017 19:40:59 +0300 Subject: Re: mount / unmount and mountcheckdirs() To: Konstantin Belousov , Edward Napierala Cc: Kirk McKusick , freebsd-fs , "freebsd-arch@freebsd.org" References: <134c7c6e-f4f1-ef38-cc50-0e56c27c9fb8@FreeBSD.org> <201709150314.v8F3Ea6B085072@chez.mckusick.com> <20170915092001.GK78693@kib.kiev.ua> <20170915103037.GM78693@kib.kiev.ua> <20170915110033.GO78693@kib.kiev.ua> <20170915114533.GS78693@kib.kiev.ua> <20170915162752.GX78693@kib.kiev.ua> From: Andriy Gapon Message-ID: <317b4617-5f0b-bc9a-316f-db1d8cf661fa@FreeBSD.org> Date: Fri, 15 Sep 2017 19:40:22 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20170915162752.GX78693@kib.kiev.ua> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 16:41:02 -0000 On 15/09/2017 19:27, Konstantin Belousov wrote: > On Fri, Sep 15, 2017 at 05:06:10PM +0100, Edward Napierala wrote: >> It does, and that case might work - I'm not sure, but it might. But what >> happens afterwards, after the syscall that triggered the mount completes? >> You would still have a shell with cwd in that dir, now covered by another >> filesystem. Autofs doesn't do anything to "lift" the process to the newly >> mounted filesystem root. > The change of the current directory should have already triggered the action, > because it was preceeded by lookup. At least this is my understanding of > the flow. >From the earlier example I can only conclude that autofs does NOT do a mount on lookup in a sense of *vpp being a root of the mounted filesystem. My interpretation is the mounting is done only if there is a VOP_LOOKUP (or VOP_READDIR) with a mount point as a dvp. -- Andriy Gapon From owner-freebsd-arch@freebsd.org Fri Sep 15 23:38:46 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D8D9E11D77 for ; Fri, 15 Sep 2017 23:38:46 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3D63C8219D for ; Fri, 15 Sep 2017 23:38:45 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by mail.baldwin.cx (Postfix) with ESMTPSA id 8EA2C10A82D for ; Fri, 15 Sep 2017 19:38:38 -0400 (EDT) From: John Baldwin To: "freebsd-arch@freebsd.org" Subject: UMA and zone alignment Date: Fri, 15 Sep 2017 16:38:28 -0700 Message-ID: <10066620.9raPCKhsf8@ralph.baldwin.cx> User-Agent: KMail/4.14.10 (FreeBSD/11.1-STABLE; KDE/4.14.30; amd64; ; ) MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Fri, 15 Sep 2017 19:38:38 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 23:38:46 -0000 I ran into an odd panic recently while running an MIPS n32 kernel under qemu and tried to use NFS. The panic was due to an unaligned 64-bit store here: (gdb) l *cache_enter_time+0xe8 0x80359414 is in cache_enter_time (/usr/home/john/work/git/mips_xbuild/sys/kern/vfs_cache.c:1621). 1616 if (vp == NULL) 1617 ncp->nc_flag |= NCF_NEGATIVE; 1618 ncp->nc_dvp = dvp; 1619 if (tsp != NULL) { 1620 n3 = (struct namecache_ts *)ncp; 1621 n3->nc_time = *tsp; The reason the store was unaligned is that the UMA zone that stores the namecache structures has an alignment of UMA_ALIGN_PTR. However, struct namecache_ts stores a time_t, so on 32-bit platforms with 64-bit time_t (such as 32-bit mips, powerpc, arm) UMA_ALIGN_PTR only requests 4 byte alignment. For most 32-bit architectures this doesn't really matter as they only use 32-bit loads and stores and have to load a 64-bit time_t as two operations. However, MIPS n32 uses 32-bit pointers with 64-bit registers and thus uses 64-bit loads and stores. As a result, when UMA allocated a structure that was 32-bit (but not 64-bit) aligned, the store to save the time_t in 'nc_time' faulted: db> tr Tracing pid 987 tid 100057 td 0x80a5f000 cache_enter_time+0xe8 (?,?,?,?) ra ffffffff801f41dc sp ffffffffe5943470 sz 144 nfscl_doiods+0x516c (ffffffff80945420,?,?,?) ra 0 sp ffffffffe5943500 sz 0 db> x/i $pc cache_enter_time+0xe8: sd v1,40(s4) db> p/x $s4 80aaa984 I pondered hacking UMA_ALIGN_PTR on N32 to be sizeof(uint64_t) instead of sizeof(void *), but instead I would rather we stop trying to guess what the alignment of a given structure should be and let the compiler do that via _Alignof(). We have a fallback __alignof() for pre-C11 environments, so I believe we are fine to use this always (well, if have a pre-GCC 2.95 compiler you can't use this, but if you have that you can't compile the kernel anyway). To that end I added a UMA_ALIGNOF() which wraps _Alignof(): https://github.com/freebsd/freebsd/commit/a9d94dba3d78cfce330175a44445efec8531286e and then used that to initialize the zones in vfs_cache.c: https://github.com/freebsd/freebsd/commit/64271272bb30cc3b00ff22c1c44ed20582bea8b9 and now I can use NFS with a MIPS n32 kernels without insta-panics. I think I'd like to add UMA_ALIGNOF() and encourage UMA zones to use UMA_ALIGNOF(type) instead of UMA_ALIGN_PTR, etc. going forward. This would mean UMA would then honor 'struct foo { ... } __aligned(XX)', etc. What do you all think? -- John Baldwin From owner-freebsd-arch@freebsd.org Sat Sep 16 06:55:20 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D5584E0497F for ; Sat, 16 Sep 2017 06:55:20 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: from mail-lf0-x229.google.com (mail-lf0-x229.google.com [IPv6:2a00:1450:4010:c07::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 82A096B3E5 for ; Sat, 16 Sep 2017 06:55:20 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: by mail-lf0-x229.google.com with SMTP id m199so4165862lfe.3 for ; Fri, 15 Sep 2017 23:55:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nuxi-nl.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=9V4nbTznLKqZ1oT7WnBNgtjvPpLZioSOd8iyM/Ghhfg=; b=Q9RPvrmwW0TK4P5dm1PqNyLEttvcxcvVS6CTAxAC1hchDG3iSoBB46g+rSifd/HRAT 2dMWlRjYPSEFV3+C3WtWBZ+II46aFf3+vwq2uXqLVMw114lv7R58utbOKkEdvcsKZs0K 4rDpGav+AY45q0TgS+lCPBO8qjsmeWqi11oPEQze14pOl96iF09iu1xvL1e2nELnZCbu 3RtQdhLiCLHx5ZDIymCDvcXLdQ9AGrnJVaH5EclhykRYKI33AhOheIQQU8nTCTrVpNfS AqJqZwF47grQ8QvrPEv6aaSm4e1OCB42FQAWEEzJuf2+5oDV80ebMSqP0NbpR+rnTwen LpqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=9V4nbTznLKqZ1oT7WnBNgtjvPpLZioSOd8iyM/Ghhfg=; b=cpkpIZns0LS0zw3jpuuUTUh6AYHEs3mRMS0i/8W5p/E8aE1o5SrqrrCB1mbUcd1dDY B5cP1quL95SrU7Tgj3NrFetJupv6xFTUn4JAJFT/HHVCEqrcQK83SI5P5WBmKghRRpEC UYmEc6zuGZnjfWlGAbbOlwGsN5k3JNW/7uHvUow8Q7PiPSf3smk47tPjsWvi0i+IgJyd zrHVyEP7K+jnBIgFr8yZegXIkCwQk1Z74DWUpYDpuXuS007x3wW/6jOKallDXp6etGtK r60Tj35Ie/ZfhkvqKCU7co/8P9QVy7ksaxO6l/bkOQZKD7/qczt3DsYaSg3Z5rGR/5Pv ay4A== X-Gm-Message-State: AHPjjUgueIfXtH9TsTF+8vu1SuPECBsuirTc0txfcyDmoTr76X4MiPfO nO1tTNshc6WH3EHlQsnchvu+2OJU5y/o8xoLmXzJ7K53 X-Google-Smtp-Source: AOwi7QAJI9q+qz7nRO7OdSzhfavlAFyrNfowttChqmNVxN0rG5ttj5aY5bLxpPZyghx17xaQXZuqWnpkDilj9Gc6Q1E= X-Received: by 10.25.92.68 with SMTP id q65mr1325370lfb.206.1505544918548; Fri, 15 Sep 2017 23:55:18 -0700 (PDT) MIME-Version: 1.0 Received: by 10.25.142.68 with HTTP; Fri, 15 Sep 2017 23:54:47 -0700 (PDT) In-Reply-To: <10066620.9raPCKhsf8@ralph.baldwin.cx> References: <10066620.9raPCKhsf8@ralph.baldwin.cx> From: Ed Schouten Date: Sat, 16 Sep 2017 07:54:47 +0100 Message-ID: Subject: Re: UMA and zone alignment To: John Baldwin Cc: "freebsd-arch@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Sep 2017 06:55:20 -0000 2017-09-16 0:38 GMT+01:00 John Baldwin : > I think I'd like to add UMA_ALIGNOF() and encourage UMA zones to use > UMA_ALIGNOF(type) instead of UMA_ALIGN_PTR, etc. going forward. This would > mean UMA would then honor 'struct foo { ... } __aligned(XX)', etc. What do > you all think? Awesome! Random remark: it looks like uma_zcreate() already takes the size of the objects to allocate without any wrapping; it's just sizeof(...). Wouldn't we be able to do this for alignment as well? More specifically, if all of the UMA_ALIGN_* definitions were incremented by one and uma_zcreate() decrements it, we could eliminate the UMA_ALIGNOF() macro entirely and simply use _Alignof() at the call site directly. If people don't like _Alignof()'s spelling, we could always consider making something like available in kernel space. -- Ed Schouten Nuxi, 's-Hertogenbosch, the Netherlands From owner-freebsd-arch@freebsd.org Sat Sep 16 16:30:56 2017 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9C769E2087A for ; Sat, 16 Sep 2017 16:30:56 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7B96B8000E for ; Sat, 16 Sep 2017 16:30:56 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by mail.baldwin.cx (Postfix) with ESMTPSA id 3CFBB10A7DB; Sat, 16 Sep 2017 12:30:55 -0400 (EDT) From: John Baldwin To: Ed Schouten Cc: "freebsd-arch@freebsd.org" Subject: Re: UMA and zone alignment Date: Sat, 16 Sep 2017 09:18:59 -0700 Message-ID: <4524093.e3VFLNUReg@ralph.baldwin.cx> User-Agent: KMail/4.14.10 (FreeBSD/11.1-STABLE; KDE/4.14.30; amd64; ; ) In-Reply-To: References: <10066620.9raPCKhsf8@ralph.baldwin.cx> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Sat, 16 Sep 2017 12:30:55 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Sep 2017 16:30:56 -0000 On Saturday, September 16, 2017 07:54:47 AM Ed Schouten wrote: > 2017-09-16 0:38 GMT+01:00 John Baldwin : > > I think I'd like to add UMA_ALIGNOF() and encourage UMA zones to use > > UMA_ALIGNOF(type) instead of UMA_ALIGN_PTR, etc. going forward. This would > > mean UMA would then honor 'struct foo { ... } __aligned(XX)', etc. What do > > you all think? > > Awesome! > > Random remark: it looks like uma_zcreate() already takes the size of > the objects to allocate without any wrapping; it's just sizeof(...). > Wouldn't we be able to do this for alignment as well? > > More specifically, if all of the UMA_ALIGN_* definitions were > incremented by one and uma_zcreate() decrements it, we could eliminate > the UMA_ALIGNOF() macro entirely and simply use _Alignof() at the call > site directly. Fixing this part of the UMA API is also on my todo list, but I haven't gotten to it yet (and this is quicker to fix). -- John Baldwin