From owner-freebsd-stable@freebsd.org Sun Nov 24 02:33:58 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CDDF11C8329 for ; Sun, 24 Nov 2019 02:33:58 +0000 (UTC) (envelope-from areilly@bigpond.net.au) Received: from nsstlmta30p.bpe.bigpond.com (nsstlmta30p.bpe.bigpond.com [203.38.21.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "", Issuer "Openwave Messaging Inc." (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47LDkW5V5Rz4kPP for ; Sun, 24 Nov 2019 02:33:54 +0000 (UTC) (envelope-from areilly@bigpond.net.au) Received: from smtp.telstra.com ([10.10.24.4]) by nsstlfep30p-svc.bpe.nexus.telstra.com.au with ESMTP id <20191124023349.TOXK6334.nsstlfep30p-svc.bpe.nexus.telstra.com.au@smtp.telstra.com> for ; Sun, 24 Nov 2019 13:33:49 +1100 X-RG-Spam: Unknown X-RazorGate-Vade: gggruggvucftvghtrhhoucdtuddrgedufedrudehjedggeejucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuuffpveftpgfvgffnuffvtfetpdfqfgfvnecuuegrihhlohhuthemucegtddtnecunecujfgurhephfgtgfgguffkfffvofesthejmhdthhdtvdenucfhrhhomheptehnughrvgifucftvghilhhlhicuoegrrhgvihhllhihsegsihhgphhonhgurdhnvghtrdgruheqnecukfhppeeitddrvddvjedrudeigedrudejleenucfrrghrrghmpehhvghloheplgdutddrtddrtddriegnpdhinhgvthepiedtrddvvdejrdduieegrddujeelpdhmrghilhhfrhhomhepoegrrhgvihhllhihsegsihhgphhonhgurdhnvghtrdgruheqpdhrtghpthhtohepoehfrhgvvggsshguqdhsthgrsghlvgesfhhrvggvsghsugdrohhrgheqnecuvehluhhsthgvrhfuihiivgeptd X-RazorGate-Vade-Verdict: clean 0 X-RazorGate-Vade-Classification: clean X-RG-VS-CLASS: clean X-Authentication-Info: Submitted using ID areilly@bigpond.net.au Received: from [10.0.0.6] (60.227.164.179) by smtp.telstra.com (5.8.418) (authenticated as areilly@bigpond.net.au) id 5D91EF6F16DF38CB for freebsd-stable@freebsd.org; Sun, 24 Nov 2019 13:33:49 +1100 From: Andrew Reilly Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3601.0.10\)) Subject: Long-shot: repeatable macOS samba share unmounting during Lightroom import Message-Id: <28504691-D08B-483B-B4C5-CA47F2C523ED@bigpond.net.au> Date: Sun, 24 Nov 2019 13:33:38 +1100 To: freebsd-stable@freebsd.org X-Mailer: Apple Mail (2.3601.0.10) X-Rspamd-Queue-Id: 47LDkW5V5Rz4kPP X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=bigpond.net.au; spf=pass (mx1.freebsd.org: domain of areilly@bigpond.net.au designates 203.38.21.30 as permitted sender) smtp.mailfrom=areilly@bigpond.net.au X-Spamd-Result: default: False [-2.40 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:203.38.21.0/24]; FREEMAIL_FROM(0.00)[bigpond.net.au]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_DN_NONE(0.00)[]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_POLICY_ALLOW(-0.50)[bigpond.net.au,none]; MV_CASE(0.50)[]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[30.21.38.203.list.dnswl.org : 127.0.5.1]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[bigpond.net.au]; ASN(0.00)[asn:1221, ipnet:203.36.0.0/14, country:AU]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.00)[ipnet: 203.36.0.0/14(-3.27), asn: 1221(-2.00), country: AU(0.00)]; RECEIVED_SPAMHAUS_PBL(0.00)[179.164.227.60.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Nov 2019 02:33:58 -0000 Hi all, This is a long-shot question, because it involves a lot of moving pieces, most of which are opaque commercial, poorly documented things. Never the less, it does involve FreeBSD-stable as one of the players, and my experience over the years has been that FreeBSD folk are both knowledgeable and helpful, so here's hoping. Herwith my tale of computer-induced irritation: The story takes place at home, where the FreeBSD system in question is a local network file server. The FreeBSD tracks -STABLE every week. It boots from ZFS on NVM flash and has four 4TB Hitachi ATA drives in a RAIDZ. The current motherboard has a Ryzen 7 1700 8-core locked at 3GHz by the bios to avoid a problem of going to sleep permanently by failing to come out of some sort of low-power state. It has 32G RAM. It has intel "PRO/1000 PCI-Express Network Driver" network connected to a simple gigabit switch, with both IPv4 and IPv6 configured and working. The other protagonist in this tale, also connected to the gigabit LAN, is an iMac running current-Catalina on APFS flash, mounting three filesystems over SMB, from Samba 4.10.10. After appropriate Samba tweaking this seems to be at least as reliable as it ever was with netatalk or NFS, and apparently better supported by Apple. I keep my Lightroom Classic catalogue on the mac's local (flash) drive, but the photo storage is on the network. The Import Backups directory is also on (a different) network drive. I use Lightroom's Import function to copy photos off SD cards using the mac's built-in SD card reader and register with the catalogue. So far so normal, I think. The problem arose about ?three or four? months ago: could be coincident with OS or Lightroom upgrades, I can't remember, but I haven't changed anything about the setup, configuration or workflow. Now, every single time Lightroom does an import, while it's doing the first scan of the SD card to identify photos that it's seen before, all three of the Samba filesystems unmount from the mac, silently. I can find no record of error in any of the logs, suggesting that the system thinks that it happened deliberately. Needless to say, this throws out the import workflow, although it manages to pick itself up OK if I just re-mount everything. Anyone have any similar experiences? Any thoughts of where I could poke it to find out why this might be happening? It feels like a time-out bug somewhere, but (a) there is no complaint, and (b) the network traffic is light at the time. Needless to say Apple documentation is useless. Probably another good reason to find an alternative to Lightroom... Cheers, Andrew Reilly M: 0409-824-272 areilly@bigpond.net.au From owner-freebsd-stable@freebsd.org Sun Nov 24 02:46:54 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 02A101C88C0 for ; Sun, 24 Nov 2019 02:46:54 +0000 (UTC) (envelope-from matt.garber@gmail.com) Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com [IPv6:2607:f8b0:4864:20::72c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47LF1S6nDYz4kwh for ; Sun, 24 Nov 2019 02:46:52 +0000 (UTC) (envelope-from matt.garber@gmail.com) Received: by mail-qk1-x72c.google.com with SMTP id h15so9713227qka.13 for ; Sat, 23 Nov 2019 18:46:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=bOBr4ndgeV+6J03uCFlGfewnJvpX0CAeQkq5GDwETw4=; b=caySLxgx4JouIZWBitE/ZD4922blGhCR5YpqOZ4fOWInBXFpKNVdEwvsc0HxF1ThfE LVc5UvMt4UCMDEQ/JaxlGHMhPkLQFzKGBmnVI7O7YXYN+KSMM4Ie05/EOMJvbOlaIWXN XxfDyp5REMqUGxPWS/6vVydBSdIk4niCBLuLLD6eGgvp3COgFacRF75QAt6JeIxL5i+c iESMfs82SVQwqe0bM+Cq+7L9CZ7Ik5XE6bPISqHrB3TthA5+zd/Cj+rNi9T5TQaoAzwA eSePzmiHG89kstlb4m5cqCWw6877sw5FqDBdkBK9Fkh1HkbvFSHQM4oSV7ui2lVMuJ/t hraA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=bOBr4ndgeV+6J03uCFlGfewnJvpX0CAeQkq5GDwETw4=; b=AT0eu//tQLPLfStx0Lj8EUnix8ZSsr6dtj2Ad0NIlIwxm/iLBGnE0kx3DgWOhAlbiH xNCK24M2JWfiAvzBbhqRWZznY195VGWQBlV0W/t7bwY/0JxfdzH/TO9f6KjSeNFzCtEs tX3XAYYeEZP8070mkgzRwzIXSn+EeMHyPHU1571lR7Th3JzjZuIGGROZggnoVK/CWOlr 2OK1o0ARySSa0+Lq+VZyjPmZrHG0EOX8c1goru50fsAK5cmE+TAYEOzRaYB5c1FRwZzI hAM37mqn8nFPV4Pr7QVFGVTUrDznWyGTvFdhtnHkXF2g+Qx996AV7Hbq2KxaDZlNEQ/q YymA== X-Gm-Message-State: APjAAAWMxB++yyEciibbGbSy9cylN88Gy65MF1eBiy3R0SFyhrGnL8Gy 0wKFL8AEzQeT+U9NcAAgs1J5VxdJnncXgX2VKZU= X-Google-Smtp-Source: APXvYqzu8uGXHQK3bfsRfiUhLd+j0btkVuSxaByFj5+bQanOexyuYEC2aWeuaBLJq0INSjouUBHHp0LwoUdeqmgYfA0= X-Received: by 2002:a37:9602:: with SMTP id y2mr12261694qkd.498.1574563611254; Sat, 23 Nov 2019 18:46:51 -0800 (PST) MIME-Version: 1.0 References: <28504691-D08B-483B-B4C5-CA47F2C523ED@bigpond.net.au> In-Reply-To: <28504691-D08B-483B-B4C5-CA47F2C523ED@bigpond.net.au> From: Matt Garber Date: Sat, 23 Nov 2019 21:46:40 -0500 Message-ID: Subject: Re: Long-shot: repeatable macOS samba share unmounting during Lightroom import To: Andrew Reilly Cc: freebsd-stable@freebsd.org X-Rspamd-Queue-Id: 47LF1S6nDYz4kwh X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=caySLxgx; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mattgarber@gmail.com designates 2607:f8b0:4864:20::72c as permitted sender) smtp.mailfrom=mattgarber@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FREEMAIL_TO(0.00)[bigpond.net.au]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (-9.29), ipnet: 2607:f8b0::/32(-2.28), asn: 15169(-1.96), country: US(-0.05)]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[c.2.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Nov 2019 02:46:54 -0000 On Sat, Nov 23, 2019 at 9:34 PM Andrew Reilly wrote: > > The other protagonist in this tale, also connected to the gigabit > LAN, is an iMac running current-Catalina on APFS flash, mounting > three filesystems over SMB, from Samba 4.10.10. After appropriate > Samba tweaking this seems to be at least as reliable as it ever was > with netatalk or NFS, and apparently better supported by Apple. Considering all of the other bugs and instability introduced (or reintroduced) in Catalina: did you have this same Lightroom import workflow configured in Mojave (or whichever other previous macOS version you were using), and if so, were you encountering the same issue? Thanks, =E2=80=94 Matt Garber From owner-freebsd-stable@freebsd.org Sun Nov 24 02:53:49 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4958E1C8C49 for ; Sun, 24 Nov 2019 02:53:49 +0000 (UTC) (envelope-from theron.tarigo@gmail.com) Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47LF9S1tztz4lLN for ; Sun, 24 Nov 2019 02:53:48 +0000 (UTC) (envelope-from theron.tarigo@gmail.com) Received: by mail-qt1-x842.google.com with SMTP id 14so12933805qtf.5 for ; Sat, 23 Nov 2019 18:53:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding:content-language; bh=hUnmkRbDEYtPasdjbFEc9W32jTJj5arxt1fMOlTf9rU=; b=RFRQ3ViuH/2/dmGlqoIl3vNhF2zdTYVwU50huIUz7LFTwl4kPLQ68omeRHD2LiJgjX GBwtRDVnhNjLl0uqmocKoO2RBwmOY81/km1kLzMGhtSTSehBLIPVvbb0O74nfMC9WDo8 zsRd6BTDHzBjUs74nnjUXfzDcbuqW33F5swBcxmkzvjmdcSwT7egl1N9l73+mcAaif/c hjUqubk7WBGN1tpMgIZ+etCk7X5Y4hCSlUIq+t5BTr+X2D/esLxvPWeuskaOGLXf0O/4 SBC4pEFucOymRRnxQ55yMzZpoytXNir6eVls/79+l9TKDb5xsO/tA8FlriF5OgLClI29 4xSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=hUnmkRbDEYtPasdjbFEc9W32jTJj5arxt1fMOlTf9rU=; b=WFDUpmTI9LN/ccl6S2WBV/UkvilNJksSol8RrKby9g3NZIrp83GWUgV+6Hui73k/xr 4d/xDy9otw1B3Dz01v2PiBGLRrAaplGamYqtX2333jkM1YHto2bN1xw6GSFDu+jAN23E uaip5GM51vCgFwj5Wbptzx1LRGOBfB+8rEwZtM7TxVlxMAhC8+usVNuE4povgZwEGckx p1cnQucuotWGtcezinbC3kl4srPMVBuMpgtJ7ZUUokHBKSAaUJ0aeb7s+kXLYqiY9w8f 06uPJmwz/MKEdATKC95ZKk0GWmN+CgW9vxDr/urLVfsYZd0p1cIsI+i2iTsV6CF5ryHo o+iA== X-Gm-Message-State: APjAAAUm3RceR4LJAbjhWVGZGiJkU1vbkimJn6/TNS1lb8fyrejNRh1A lT/KVs8d4ECQtED6URjlK9COme/b X-Google-Smtp-Source: APXvYqxrWEivwbd1AhOMLXzB187FmsAo6cs70ePYgr2umAUl5VKSeuFKFQgjDOT43/Th0/CYMdM20Q== X-Received: by 2002:ac8:7b3c:: with SMTP id l28mr15750175qtu.62.1574564026501; Sat, 23 Nov 2019 18:53:46 -0800 (PST) Received: from [155.41.28.29] (dhcp-wifi-8021x-155-41-28-29.bu.edu. [155.41.28.29]) by smtp.gmail.com with ESMTPSA id d18sm1247046qko.112.2019.11.23.18.53.45 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 23 Nov 2019 18:53:45 -0800 (PST) Sender: Theron Tarigo Subject: Re: Long-shot: repeatable macOS samba share unmounting during Lightroom import To: Andrew Reilly , freebsd-stable@freebsd.org References: <28504691-D08B-483B-B4C5-CA47F2C523ED@bigpond.net.au> From: Theron Message-ID: <30110cdf-39e1-6a19-3732-0cda76c1937a@gmail.com> Date: Sat, 23 Nov 2019 21:53:44 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.2.1 MIME-Version: 1.0 In-Reply-To: <28504691-D08B-483B-B4C5-CA47F2C523ED@bigpond.net.au> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Rspamd-Queue-Id: 47LF9S1tztz4lLN X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=RFRQ3Viu; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of therontarigo@gmail.com designates 2607:f8b0:4864:20::842 as permitted sender) smtp.mailfrom=therontarigo@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FREEMAIL_TO(0.00)[bigpond.net.au]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (2.18), ipnet: 2607:f8b0::/32(-2.28), asn: 15169(-1.96), country: US(-0.05)]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2.4.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Nov 2019 02:53:49 -0000 On 2019-11-23 21:33, Andrew Reilly wrote: > It feels like a > time-out bug somewhere, but (a) there is no complaint, and (b) the > network traffic is light at the time. Needless to say Apple > documentation is useless. Maybe there is a way to reproduce the Lightroom's file access pattern from a shell script in a way that reproduces the problem. Then the same access pattern could be done from a FreeBSD client to see whether it happens there as well. Apple's SMB client appears to be FreeBSD's smbfs, however extended by Apple to support protocol version 3. https://opensource.apple.com/source/smb/smb-759.40.1/kernel/smbfs/smbfs.h.auto.html (Off-topic: I wonder if anyone has looked at porting this back to FreeBSD?) Maybe the differences are too great for the bug to appear on both systems. From owner-freebsd-stable@freebsd.org Sun Nov 24 04:31:17 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DDC6A1CAC49 for ; Sun, 24 Nov 2019 04:31:17 +0000 (UTC) (envelope-from areilly@bigpond.net.au) Received: from nsstlmta25p.bpe.bigpond.com (nsstlmta25p.bpe.bigpond.com [203.38.21.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "", Issuer "Openwave Messaging Inc." (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47LHKv34NSz4pqR for ; Sun, 24 Nov 2019 04:31:14 +0000 (UTC) (envelope-from areilly@bigpond.net.au) Received: from smtp.telstra.com ([10.10.24.4]) by nsstlfep25p-svc.bpe.nexus.telstra.com.au with ESMTP id <20191124043108.TQCV6194.nsstlfep25p-svc.bpe.nexus.telstra.com.au@smtp.telstra.com>; Sun, 24 Nov 2019 15:31:08 +1100 X-RG-Spam: Unknown X-RazorGate-Vade: gggruggvucftvghtrhhoucdtuddrgedufedrudehjedgjedvucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuuffpveftpgfvgffnuffvtfetpdfqfgfvnecuuegrihhlohhuthemucegtddtnecunecujfgurheptggguffhjgffgffkfhfvofesthhqmhdthhdtvdenucfhrhhomheptehnughrvgifucftvghilhhlhicuoegrrhgvihhllhihsegsihhgphhonhgurdhnvghtrdgruheqnecukfhppeeitddrvddvjedrudeigedrudejleenucfrrghrrghmpehhvghloheplgdutddrtddrtddriegnpdhinhgvthepiedtrddvvdejrdduieegrddujeelpdhmrghilhhfrhhomhepoegrrhgvihhllhihsegsihhgphhonhgurdhnvghtrdgruheqpdhrtghpthhtohepoehfrhgvvggsshguqdhsthgrsghlvgesfhhrvggvsghsugdrohhrgheqpdhrtghpthhtohepoehthhgvrhhonhdrthgrrhhighhosehgmhgrihhlrdgtohhmqeenucevlhhushhtvghrufhiiigvpedt X-RazorGate-Vade-Verdict: clean 0 X-RazorGate-Vade-Classification: clean X-RG-VS-CLASS: clean X-Authentication-Info: Submitted using ID areilly@bigpond.net.au Received: from [10.0.0.6] (60.227.164.179) by smtp.telstra.com (5.8.418) (authenticated as areilly@bigpond.net.au) id 5D91EF6F16E70F17; Sun, 24 Nov 2019 15:31:08 +1100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3601.0.10\)) Subject: Re: Long-shot: repeatable macOS samba share unmounting during Lightroom import From: Andrew Reilly In-Reply-To: <30110cdf-39e1-6a19-3732-0cda76c1937a@gmail.com> Date: Sun, 24 Nov 2019 15:30:58 +1100 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <444ACA46-DD7C-4754-B996-DC2997931811@bigpond.net.au> References: <28504691-D08B-483B-B4C5-CA47F2C523ED@bigpond.net.au> <30110cdf-39e1-6a19-3732-0cda76c1937a@gmail.com> To: Theron X-Mailer: Apple Mail (2.3601.0.10) X-Rspamd-Queue-Id: 47LHKv34NSz4pqR X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=bigpond.net.au; spf=pass (mx1.freebsd.org: domain of areilly@bigpond.net.au designates 203.38.21.25 as permitted sender) smtp.mailfrom=areilly@bigpond.net.au X-Spamd-Result: default: False [-2.40 / 15.00]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[bigpond.net.au]; R_SPF_ALLOW(-0.20)[+ip4:203.38.21.0/24]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[bigpond.net.au,none]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_IN_DNSWL_LOW(-0.10)[25.21.38.203.list.dnswl.org : 127.0.5.1]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[bigpond.net.au]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:1221, ipnet:203.36.0.0/14, country:AU]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[179.164.227.60.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE_FREEMAIL(0.00)[]; R_DKIM_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; IP_SCORE(0.00)[ipnet: 203.36.0.0/14(-3.49), asn: 1221(-2.07), country: AU(0.00)]; RCVD_TLS_LAST(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Nov 2019 04:31:17 -0000 Hi Theron, > On 24 Nov 2019, at 13:53 , Theron wrote: >=20 > On 2019-11-23 21:33, Andrew Reilly wrote: >> It feels like a >> time-out bug somewhere, but (a) there is no complaint, and (b) the >> network traffic is light at the time. Needless to say Apple >> documentation is useless. > Maybe there is a way to reproduce the Lightroom's file access pattern = from a shell script in a way that reproduces the problem. Then the same = access pattern could be done from a FreeBSD client to see whether it = happens there as well. As I half-expected, now that I've reported the problem, I can't = reproduce it on demand, even using the same workflow. Perhaps it is = triggered by having a certain number of photos on the card? I'll need = to do some more experimentation. Cheers, Andrew Reilly M: 0409-824-272 areilly@bigpond.net.au From owner-freebsd-stable@freebsd.org Sun Nov 24 04:31:44 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 73B9B1CAE06 for ; Sun, 24 Nov 2019 04:31:44 +0000 (UTC) (envelope-from areilly@bigpond.net.au) Received: from nsstlmta29p.bpe.bigpond.com (nsstlmta29p.bpe.bigpond.com [203.38.21.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "", Issuer "Openwave Messaging Inc." (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47LHLQ07Kxz4pyZ for ; Sun, 24 Nov 2019 04:31:41 +0000 (UTC) (envelope-from areilly@bigpond.net.au) Received: from smtp.telstra.com ([10.10.24.4]) by nsstlfep29p-svc.bpe.nexus.telstra.com.au with ESMTP id <20191124043133.DEQZ6111.nsstlfep29p-svc.bpe.nexus.telstra.com.au@smtp.telstra.com>; Sun, 24 Nov 2019 15:31:33 +1100 X-RG-Spam: Unknown X-RazorGate-Vade: gggruggvucftvghtrhhoucdtuddrgedufedrudehjedgjedvucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuuffpveftpgfvgffnuffvtfetpdfqfgfvnecuuegrihhlohhuthemucegtddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpegtggfuhfgjfffgkfhfvffosehtqhhmtdhhtddvnecuhfhrohhmpeetnhgurhgvficutfgvihhllhihuceorghrvghilhhlhiessghighhpohhnugdrnhgvthdrrghuqeenucfkphepiedtrddvvdejrdduieegrddujeelnecurfgrrhgrmhephhgvlhhopegluddtrddtrddtrdeingdpihhnvghtpeeitddrvddvjedrudeigedrudejledpmhgrihhlfhhrohhmpeeorghrvghilhhlhiessghighhpohhnugdrnhgvthdrrghuqedprhgtphhtthhopeeofhhrvggvsghsugdqshhtrggslhgvsehfrhgvvggsshgurdhorhhgqedprhgtphhtthhopeeomhgrthhtrdhgrghrsggvrhesghhmrghilhdrtghomheqnecuvehluhhsthgvrhfuihiivgepud X-RazorGate-Vade-Verdict: clean 0 X-RazorGate-Vade-Classification: clean X-RG-VS-CLASS: clean X-Authentication-Info: Submitted using ID areilly@bigpond.net.au Received: from [10.0.0.6] (60.227.164.179) by smtp.telstra.com (5.8.418) (authenticated as areilly@bigpond.net.au) id 5D91EF6F16E713F9; Sun, 24 Nov 2019 15:31:33 +1100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3601.0.10\)) Subject: Re: Long-shot: repeatable macOS samba share unmounting during Lightroom import From: Andrew Reilly In-Reply-To: Date: Sun, 24 Nov 2019 15:31:32 +1100 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <56ACDCF1-6D0E-4D35-93B2-FB9544C9139C@bigpond.net.au> References: <28504691-D08B-483B-B4C5-CA47F2C523ED@bigpond.net.au> To: Matt Garber X-Mailer: Apple Mail (2.3601.0.10) X-Rspamd-Queue-Id: 47LHLQ07Kxz4pyZ X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=bigpond.net.au; spf=pass (mx1.freebsd.org: domain of areilly@bigpond.net.au designates 203.38.21.29 as permitted sender) smtp.mailfrom=areilly@bigpond.net.au X-Spamd-Result: default: False [-2.40 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:203.38.21.0/24:c]; FREEMAIL_FROM(0.00)[bigpond.net.au]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[bigpond.net.au,none]; FREEMAIL_TO(0.00)[gmail.com]; RECEIVED_SPAMHAUS_PBL(0.00)[179.164.227.60.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[bigpond.net.au]; ASN(0.00)[asn:1221, ipnet:203.36.0.0/14, country:AU]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[29.21.38.203.list.dnswl.org : 127.0.5.1]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; IP_SCORE(0.00)[ipnet: 203.36.0.0/14(-3.66), asn: 1221(-2.13), country: AU(0.00)]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE_FREEMAIL(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Nov 2019 04:31:44 -0000 Hi Matt, > On 24 Nov 2019, at 13:46 , Matt Garber wrote: >=20 > On Sat, Nov 23, 2019 at 9:34 PM Andrew Reilly = wrote: >=20 > The other protagonist in this tale, also connected to the gigabit > LAN, is an iMac running current-Catalina on APFS flash, mounting > three filesystems over SMB, from Samba 4.10.10. After appropriate > Samba tweaking this seems to be at least as reliable as it ever was > with netatalk or NFS, and apparently better supported by Apple. >=20 > Considering all of the other bugs and instability introduced (or = reintroduced) in Catalina: did you have this same Lightroom import = workflow configured in Mojave (or whichever other previous macOS version = you were using), and if so, were you encountering the same issue? Yes, I've been using the same workflow for "ever". Since before = Lightroom was a subscription product. I know that a lot of unhappiness = has been expressed on the net about Catalina, but I've personally = experienced no obvious problems, except perhaps this one. Even so, I'm = afraid that I didn't note when this problem started, so I can't say = whether it was before, after, or coincident with the Catalina upgrade. Cheers, Andrew Reilly M: 0409-824-272 areilly@bigpond.net.au From owner-freebsd-stable@freebsd.org Sun Nov 24 11:15:53 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2A1801AB7DF for ; Sun, 24 Nov 2019 11:15:53 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (constantine.ingresso.co.uk [31.24.6.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47LSJm03JGz3PXQ for ; Sun, 24 Nov 2019 11:15:51 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from [2001:470:6cc4:1:b17b:489b:9dfd:c09a] (helo=foula.drayhouse.twisted.org.uk) by constantine.ingresso.co.uk with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.92.3 (FreeBSD)) (envelope-from ) id 1iYprk-000Nnv-2A for freebsd-stable@freebsd.org; Sun, 24 Nov 2019 11:15:44 +0000 Subject: Re: Long-shot: repeatable macOS samba share unmounting during Lightroom import To: freebsd-stable@freebsd.org References: <28504691-D08B-483B-B4C5-CA47F2C523ED@bigpond.net.au> From: Pete French Message-ID: <89708db0-940e-43b3-c428-4fa980a3abbb@ingresso.co.uk> Date: Sun, 24 Nov 2019 11:15:44 +0000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:60.0) Gecko/20100101 Thunderbird/60.9.1 MIME-Version: 1.0 In-Reply-To: <28504691-D08B-483B-B4C5-CA47F2C523ED@bigpond.net.au> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 47LSJm03JGz3PXQ X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=ingresso.co.uk; spf=pass (mx1.freebsd.org: domain of petefrench@ingresso.co.uk designates 31.24.6.74 as permitted sender) smtp.mailfrom=petefrench@ingresso.co.uk X-Spamd-Result: default: False [-6.09 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:31.24.6.74]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-3.29)[ip: (-9.80), ipnet: 31.24.0.0/21(-4.90), asn: 16082(-1.68), country: GB(-0.08)]; DMARC_POLICY_ALLOW(-0.50)[ingresso.co.uk,none]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16082, ipnet:31.24.0.0/21, country:GB]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Nov 2019 11:15:53 -0000 I have a very similar setup to you for serving files to my Mac from a FreeBSD server. I haven't seen the unmount problem, but I di have a few oddities until I added the 'fruit' module on the Samba side, which helps with compatbiloty with the Mac. The appropriate bit of my config looks like this: vfs objects = fruit streams_xattr zfsacl fruit:resource = xattr fruit:encoding = private Don't ask me what they do anymore, I added them ages ago, but it does work very nicely for me. You may already have this of course, but worth pointing out just in case as it took me a few years to discover it! As someone else has said though, this may well be a Catalina bug. I am not running that (MacBook too old, and not buying another until the new keyboards are avilable n the replacement I want). -pete. From owner-freebsd-stable@freebsd.org Sun Nov 24 11:34:28 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8A4FC1ABE60 for ; Sun, 24 Nov 2019 11:34:28 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from mx0.gid.co.uk (mx0.gid.co.uk [194.32.164.250]) by mx1.freebsd.org (Postfix) with ESMTP id 47LSkB3tFrz3QQV for ; Sun, 24 Nov 2019 11:34:26 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from [194.32.164.30] ([194.32.164.30]) by mx0.gid.co.uk (8.14.2/8.14.2) with ESMTP id xAOBYDpr031421; Sun, 24 Nov 2019 11:34:13 GMT (envelope-from rb@gid.co.uk) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: Long-shot: repeatable macOS samba share unmounting during Lightroom import From: Bob Bishop In-Reply-To: <28504691-D08B-483B-B4C5-CA47F2C523ED@bigpond.net.au> Date: Sun, 24 Nov 2019 11:34:05 +0000 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: 7bit Message-Id: <4FFEDC27-7E08-404B-A277-0D24942763CF@gid.co.uk> References: <28504691-D08B-483B-B4C5-CA47F2C523ED@bigpond.net.au> To: Andrew Reilly X-Mailer: Apple Mail (2.3273) X-Rspamd-Queue-Id: 47LSkB3tFrz3QQV X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of rb@gid.co.uk designates 194.32.164.250 as permitted sender) smtp.mailfrom=rb@gid.co.uk X-Spamd-Result: default: False [-2.31 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[gid.co.uk]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[250.164.32.194.list.dnswl.org : 127.0.10.0]; IP_SCORE(-0.61)[ip: (-2.23), ipnet: 194.32.164.0/24(-1.11), asn: 42831(0.35), country: GB(-0.08)]; FREEMAIL_TO(0.00)[bigpond.net.au]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:42831, ipnet:194.32.164.0/24, country:GB]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Nov 2019 11:34:28 -0000 Hi, > On 24 Nov 2019, at 02:33, Andrew Reilly wrote: [tale of woe trimmed] > I use Lightroom's > Import function to copy photos off SD cards using the mac's built-in > SD card reader and register with the catalogue. [etc] Long shot, but try using an external SD card reader. -- Bob Bishop rb@gid.co.uk From owner-freebsd-stable@freebsd.org Sun Nov 24 20:01:04 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 20C661B9239 for ; Sun, 24 Nov 2019 20:01:04 +0000 (UTC) (envelope-from freebsd@theory14.net) Received: from bacon.theory14.net (bacon.theory14.net [45.55.200.27]) by mx1.freebsd.org (Postfix) with ESMTP id 47Lgyl1NJmz4PpP for ; Sun, 24 Nov 2019 20:01:02 +0000 (UTC) (envelope-from freebsd@theory14.net) Received: from remote.theory14.net (remote.theory14.net [72.66.31.190]) by bacon.theory14.net (Postfix) with ESMTPSA id C1003125F3A; Sun, 24 Nov 2019 15:01:01 -0500 (EST) Received: from grackle.int.theory14.net (grackle.int.theory14.net [192.168.10.52]) by remote.theory14.net (Postfix) with ESMTPS id 857EBBBC; Sun, 24 Nov 2019 15:01:01 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=theory14.net; s=mail; t=1574625661; bh=m5Hx1FHc2pxbBQ4DPPFTC0UnmK2HKl+hmgXxg3zYXE8=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=g5I0uRKnxEobQh8hhd/qIHuxiFXTx/H7AHLtsGSmeWDaRiBuF4yBQ9Te1fwjCEj1L Znokh929HntU1HCzh77BzL3uxKYHnkeMeyEyz3r2lg4yqvMlNiMoxI/vf7zEOSguwA q84DRr4ROdH7/Jygo/kQoQAn53ntjYmIA8Y3wFTg= Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3601.0.10\)) Subject: Re: Long-shot: repeatable macOS samba share unmounting during Lightroom import From: Chris Gordon In-Reply-To: <28504691-D08B-483B-B4C5-CA47F2C523ED@bigpond.net.au> Date: Sun, 24 Nov 2019 15:01:01 -0500 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <28504691-D08B-483B-B4C5-CA47F2C523ED@bigpond.net.au> To: Andrew Reilly X-Mailer: Apple Mail (2.3601.0.10) X-Rspamd-Queue-Id: 47Lgyl1NJmz4PpP X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=pass header.d=theory14.net header.s=mail header.b=g5I0uRKn; dmarc=pass (policy=none) header.from=theory14.net; spf=pass (mx1.freebsd.org: domain of freebsd@theory14.net designates 45.55.200.27 as permitted sender) smtp.mailfrom=freebsd@theory14.net X-Spamd-Result: default: False [1.79 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[theory14.net:s=mail]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_MEDIUM(0.34)[0.342,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[theory14.net:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[theory14.net,none]; NEURAL_SPAM_LONG(0.49)[0.488,0]; FREEMAIL_TO(0.00)[bigpond.net.au]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(1.36)[ipnet: 45.55.192.0/18(4.92), asn: 14061(1.91), country: US(-0.05)]; ASN(0.00)[asn:14061, ipnet:45.55.192.0/18, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[190.31.66.72.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.10] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Nov 2019 20:01:04 -0000 WARNING: Mostly deviating from a FreeBSD specific discussion. > On Nov 23, 2019, at 9:33 PM, Andrew Reilly = wrote: >=20 > Hi all, >=20 > This is a long-shot question, because it involves a lot of moving > pieces, most of which are opaque commercial, poorly documented > things. Never the less, it does involve FreeBSD-stable as one of > the players, and my experience over the years has been that FreeBSD > folk are both knowledgeable and helpful, so here's hoping. Herwith > my tale of computer-induced irritation: >=20 > The story takes place at home, where the FreeBSD system in question > is a local network file server. The FreeBSD tracks -STABLE every > week. It boots from ZFS on NVM flash and has four 4TB Hitachi ATA > drives in a RAIDZ. The current motherboard has a Ryzen 7 1700 > 8-core locked at 3GHz by the bios to avoid a problem of going to > sleep permanently by failing to come out of some sort of low-power > state. It has 32G RAM. It has intel "PRO/1000 PCI-Express Network > Driver" network connected to a simple gigabit switch, with both > IPv4 and IPv6 configured and working. >=20 > The other protagonist in this tale, also connected to the gigabit > LAN, is an iMac running current-Catalina on APFS flash, mounting > three filesystems over SMB, from Samba 4.10.10. After appropriate > Samba tweaking this seems to be at least as reliable as it ever was > with netatalk or NFS, and apparently better supported by Apple. >=20 > I keep my Lightroom Classic catalogue on the mac's local (flash) > drive, but the photo storage is on the network. The Import Backups > directory is also on (a different) network drive. I use Lightroom's > Import function to copy photos off SD cards using the mac's built-in > SD card reader and register with the catalogue. So far so normal, > I think. >=20 > The problem arose about ?three or four? months ago: could be > coincident with OS or Lightroom upgrades, I can't remember, but I > haven't changed anything about the setup, configuration or workflow. > Now, every single time Lightroom does an import, while it's doing > the first scan of the SD card to identify photos that it's seen > before, all three of the Samba filesystems unmount from the mac, > silently. I can find no record of error in any of the logs, > suggesting that the system thinks that it happened deliberately. > Needless to say, this throws out the import workflow, although it > manages to pick itself up OK if I just re-mount everything. >=20 > Anyone have any similar experiences? Any thoughts of where I could > poke it to find out why this might be happening? It feels like a > time-out bug somewhere, but (a) there is no complaint, and (b) the > network traffic is light at the time. Needless to say Apple > documentation is useless. >=20 > Probably another good reason to find an alternative to Lightroom... Other than the hardware specifics, I have the same exact workflow and = same players involved. Maybe an odd question, but how are you mounting the SMB shares? Since updating to Catalina, I've found lots of problems dealing with SMB = using the Finder window and the items under the Locations side bar. For = instance: - Mount a share. At some point overnight when nothing is using it, the = share is unmounted. I can't find anything in the logs to say why, when, = what, etc. Just unmounted. - With a fresh start of the Finder process and I can access the SMB = server/shares. After some time, activity, something, I only get = "Connection Failed" and can't access anything. What's really crazy, is = that I can't even unmount mounted shares from under Locations when it = gets in this state (I can unmount via right click on the desktop item, = CLI, etc). I see the share with the eject button, but just get useless = error message (if anything). - Killing (killall -HUP Finder) makes everything work again for a short = bit. If I mount the share via the Finder menubar (Go -> Connect to Server) = everything works and is rock solid. Mounts with no problems, no mystery = unmount, etc. I did test a Lightroom import and had no SMB issues when = the shared where mounted via the menu bar. I also store all of my music on the server and access it via SMB mounts. = I've noticed that the Apple Music app will automatically mount the SMB = share when I hit play. Unlike iTunes, I assume it's more aware of the = filesystem and mounts. Even with the Finder window off in la-la land, = the auto-mount by Music works fine. Now if this behavior could only be = exposed to every other app.... My completely un-researched guess is that when they removed the NetBIOS = support (one of the changes in Catalina), some bug was introduced or = uncovered causing the problematic behavior. My guess ins based on the = assumption that the Finder window (not sure what to call it) displays = all of the network browsing and discovery and is the code path that was = impacted by the NetBIOS change. Any mount tied in with this path gets = impacted, and when this goes south, it takes the mounts with it. = Mounting via other means doesn't bring in the associated path. I will check each new OS X update to see if this issue goes away. Until = then, I just mount my shares via "Go- > 'Connect to Server'". Also = note that you can re-enable NetBIOS support in Catalina = (https://medium.com/@gobinathm/how-to-access-smb-printer-shares-in-macos-c= atalina-10-15-17ea91d2c10b). I've not done nor tested that. Hopefully a minor tweak to how you mount the SMB share will save you the = huge hassle of changing out tools. Hope that helps. Chris= From owner-freebsd-stable@freebsd.org Sun Nov 24 22:13:59 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5605C1BD73E for ; Sun, 24 Nov 2019 22:13:59 +0000 (UTC) (envelope-from areilly@bigpond.net.au) Received: from nsstlmta07p.bpe.bigpond.com (nsstlmta07p.bpe.bigpond.com [203.38.21.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "", Issuer "Openwave Messaging Inc." (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47Lkw40vcdz4YjM for ; Sun, 24 Nov 2019 22:13:54 +0000 (UTC) (envelope-from areilly@bigpond.net.au) Received: from smtp.telstra.com ([10.10.24.4]) by nsstlfep07p-svc.bpe.nexus.telstra.com.au with ESMTP id <20191124221344.GDAP17667.nsstlfep07p-svc.bpe.nexus.telstra.com.au@smtp.telstra.com>; Mon, 25 Nov 2019 09:13:44 +1100 X-RG-Spam: Unknown X-RazorGate-Vade: gggruggvucftvghtrhhoucdtuddrgedufedrudehkedgudeiudcutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfupfevtfgpvffgnffuvffttedpqfgfvfenuceurghilhhouhhtmecugedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmnecujfgurheptggguffhjgffgffkfhfvofesthhqmhdthhdtvdenucfhrhhomheptehnughrvgifucftvghilhhlhicuoegrrhgvihhllhihsegsihhgphhonhgurdhnvghtrdgruheqnecuffhomhgrihhnpehfrhgvvggsshgurdhorhhgpdgthigsvghrtghithhirdgsihiipdhsrghmsggrrdhorhhgpdhorhgrtghlvgdrtghomhenucfkphepvddtfedruddutddrudefvddrleefnecurfgrrhgrmhephhgvlhhopegrphhrqdhprhhordguohhlsgihrdhnvghtpdhinhgvthepvddtfedruddutddrudefvddrleefpdhmrghilhhfrhhomhepoegrrhgvihhllhihsegsihhgphhonhgurdhnvghtrdgruheqpdhrtghpthhtohepoehfrhgvvggsshguqdhsthgrsghlvgesfhhrvggvsghsugdrohhrgheqpdhrtghpthhtohepoehpvghtvghfrhgvnhgthhesihhnghhrvghsshhordgtohdruhhkqeenucevlhhushhtvghrufhiiigvpedt X-RazorGate-Vade-Verdict: clean 0 X-RazorGate-Vade-Classification: clean X-RG-VS-CLASS: clean X-Authentication-Info: Submitted using ID areilly@bigpond.net.au Received: from apr-pro.dolby.net (203.110.132.93) by smtp.telstra.com (5.8.418) (authenticated as areilly@bigpond.net.au) id 5D8C2B0F18B6E1A6; Mon, 25 Nov 2019 09:13:44 +1100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: Long-shot: repeatable macOS samba share unmounting during Lightroom import From: Andrew Reilly In-Reply-To: <89708db0-940e-43b3-c428-4fa980a3abbb@ingresso.co.uk> Date: Mon, 25 Nov 2019 09:13:43 +1100 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <6912108E-1257-46DA-9140-380BC5208189@bigpond.net.au> References: <28504691-D08B-483B-B4C5-CA47F2C523ED@bigpond.net.au> <89708db0-940e-43b3-c428-4fa980a3abbb@ingresso.co.uk> To: Pete French X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 47Lkw40vcdz4YjM X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=bigpond.net.au; spf=pass (mx1.freebsd.org: domain of areilly@bigpond.net.au designates 203.38.21.7 as permitted sender) smtp.mailfrom=areilly@bigpond.net.au X-Spamd-Result: default: False [-2.40 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:203.38.21.0/24]; FREEMAIL_FROM(0.00)[bigpond.net.au]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[bigpond.net.au,none]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[7.21.38.203.list.dnswl.org : 127.0.5.1]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[bigpond.net.au]; ASN(0.00)[asn:1221, ipnet:203.36.0.0/14, country:AU]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.00)[ipnet: 203.36.0.0/14(-3.81), asn: 1221(-2.28), country: AU(0.00)]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Nov 2019 22:13:59 -0000 Hi Pete, Thanks for the suggestions. FWIW I've been running the fruit extensions and a few other tweaks for = quite a while. My current global configuration contains: vfs objects =3D catia fruit streams_xattr zfsacl fruit:resource =3D xattr fruit:locking =3D netatalk fruit:time machine =3D yes #dflt: fruit:resource =3D file #dflt: fruit:locking =3D none fruit:advertise_fullsync =3D yes nfs4:mode =3D special nfs4:acedup =3D merge nfs4:chown =3D yes # could be derived from this article: # https://blogs.oracle.com/marks/entry/zfs_acls =20 mangled names =3D no hide dot files =3D no # prevent created files having execute bit set? map archive =3D no # prevent SMB1, which seems to be bad: # = https://www.cyberciti.biz/faq/how-to-configure-samba-to-use-smbv2-and-disa= ble-smbv1-on-linux-or-unix/ min protocol =3D SMB2 # Seems to be required for Spotlight (tracker) support (see spotlight =3D = yes on shares, below) # see: https://wiki.samba.org/index.php/Spotlight rpc_server:mdssd =3D fork rpc_server:mdssvc =3D external socket options =3D IPTOS_LOWDELAY TCP_NODELAY browseable =3D yes No, I haven't actually got Spotlight to work on the samba shares. Any = hints on that front would be appreciated. The documentation on some of = those Gnome-desktop background things is possibly worse than the Apple = documentation... I find that quite a lot about modern computing resembles magical = thinking, or occult practices: there is almost no way to actually = understand what is going on, you just try things and stick to what seems = to work. Superstition over science. Sigh. I can't remember why I'm not using fruit:encoding =3D private, but I = think that it was because I wanted file names to be representable and = compatible on both sides of the share, at the expense of occasionally = having to "fix" names that turn out to be unrepresentable. I've bumped log level up to 2 and increased the size of the log files by = a factor of ten (to 5000 (k)). I suspect that one reason why I wasn't = seeing errors being reported could be that the logs were being turned = over too quickly. We'll see. Cheers, Andrew Reilly E: areilly@bigpond.net.au M: +61-409-824-272 > On 24 Nov 2019, at 22:15, Pete French = wrote: >=20 > I have a very similar setup to you for serving files to my Mac from a = FreeBSD server. I haven't seen the unmount problem, but I di have a few = oddities until I added the 'fruit' module on the Samba side, which helps = with compatbiloty with the Mac. The appropriate bit of my config looks = like this: >=20 > vfs objects =3D fruit streams_xattr zfsacl > fruit:resource =3D xattr > fruit:encoding =3D private >=20 > Don't ask me what they do anymore, I added them ages ago, but it does = work very nicely for me. You may already have this of course, but worth = pointing out just in case as it took me a few years to discover it! >=20 > As someone else has said though, this may well be a Catalina bug. I am = not running that (MacBook too old, and not buying another until the new = keyboards are avilable n the replacement I want). >=20 > -pete. > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Sun Nov 24 22:33:58 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 863F21BDEBB for ; Sun, 24 Nov 2019 22:33:58 +0000 (UTC) (envelope-from areilly@bigpond.net.au) Received: from nsstlmta22p.bpe.bigpond.com (nsstlmta22p.bpe.bigpond.com [203.38.21.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "", Issuer "Openwave Messaging Inc." (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47LlM72N7Nz4ZT6 for ; Sun, 24 Nov 2019 22:33:54 +0000 (UTC) (envelope-from areilly@bigpond.net.au) Received: from smtp.telstra.com ([10.10.24.4]) by nsstlfep22p-svc.bpe.nexus.telstra.com.au with ESMTP id <20191124223343.LSTJ6391.nsstlfep22p-svc.bpe.nexus.telstra.com.au@smtp.telstra.com>; Mon, 25 Nov 2019 09:33:43 +1100 X-RG-Spam: Unknown X-RazorGate-Vade: gggruggvucftvghtrhhoucdtuddrgedufedrudehkedgudeihecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfupfevtfgpvffgnffuvffttedpqfgfvfenuceurghilhhouhhtmecugedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmnecujfgurheptggguffhjgffgffkfhfvofesthhqmhdthhdtvdenucfhrhhomheptehnughrvgifucftvghilhhlhicuoegrrhgvihhllhihsegsihhgphhonhgurdhnvghtrdgruheqnecuffhomhgrihhnpehmvgguihhumhdrtghomhenucfkphepvddtfedruddutddrudefvddrleefnecurfgrrhgrmhephhgvlhhopegrphhrqdhprhhordguohhlsgihrdhnvghtpdhinhgvthepvddtfedruddutddrudefvddrleefpdhmrghilhhfrhhomhepoegrrhgvihhllhihsegsihhgphhonhgurdhnvghtrdgruheqpdhrtghpthhtohepoegthhhrihhssehthhgvohhrhidugedrnhgvtheqpdhrtghpthhtohepoehfrhgvvggsshguqdhsthgrsghlvgesfhhrvggvsghsugdrohhrgheqnecuvehluhhsthgvrhfuihiivgeptd X-RazorGate-Vade-Verdict: clean 0 X-RazorGate-Vade-Classification: clean X-RG-VS-CLASS: clean X-Authentication-Info: Submitted using ID areilly@bigpond.net.au Received: from apr-pro.dolby.net (203.110.132.93) by smtp.telstra.com (5.8.418) (authenticated as areilly@bigpond.net.au) id 5D8A7318197C070F; Mon, 25 Nov 2019 09:33:43 +1100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: Long-shot: repeatable macOS samba share unmounting during Lightroom import From: Andrew Reilly In-Reply-To: Date: Mon, 25 Nov 2019 09:33:33 +1100 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <28504691-D08B-483B-B4C5-CA47F2C523ED@bigpond.net.au> To: Chris Gordon X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 47LlM72N7Nz4ZT6 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=bigpond.net.au; spf=pass (mx1.freebsd.org: domain of areilly@bigpond.net.au designates 203.38.21.22 as permitted sender) smtp.mailfrom=areilly@bigpond.net.au X-Spamd-Result: default: False [-2.40 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:203.38.21.0/24]; FREEMAIL_FROM(0.00)[bigpond.net.au]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[bigpond.net.au,none]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[22.21.38.203.list.dnswl.org : 127.0.5.1]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[bigpond.net.au]; ASN(0.00)[asn:1221, ipnet:203.36.0.0/14, country:AU]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.00)[ipnet: 203.36.0.0/14(-3.92), asn: 1221(-2.33), country: AU(0.00)]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Nov 2019 22:33:58 -0000 Hi Chris, > On 25 Nov 2019, at 05:12, Chris Gordon wrote: >=20 > WARNING: Mostly deviating from a FreeBSD specific discion >=20 >> On Nov 23, 2019, at 9:33 PM, Andrew Reilly = wrote: >>=20 >> Hi all, >>=20 >> This is a long-shot question, because it involves a lot of moving >> pieces, most of which are opaque commercial, poorly documented >> things. Never the less, it does involve FreeBSD-stable as one of >> the players, and my experience over the years has been that FreeBSD >> folk are both knowledgeable and helpful, so here's hoping. Herwith >> my tale of computer-induced irritation: >>=20 >> The story takes place at home, where the FreeBSD system in question >> is a local network file server. The FreeBSD tracks -STABLE every >> week. It boots from ZFS on NVM flash and has four 4TB Hitachi ATA >> drives in a RAIDZ. The current motherboard has a Ryzen 7 1700 >> 8-core locked at 3GHz by the bios to avoid a problem of going to >> sleep permanently by failing to come out of some sort of low-power >> state. It has 32G RAM. It has intel "PRO/1000 PCI-Express Network >> Driver" network connected to a simple gigabit switch, with both >> IPv4 and IPv6 configured and working. >>=20 >> The other protagonist in this tale, also connected to the gigabit >> LAN, is an iMac running current-Catalina on APFS flash, mounting >> three filesystems over SMB, from Samba 4.10.10. After appropriate >> Samba tweaking this seems to be at least as reliable as it ever was >> with netatalk or NFS, and apparently better supported by Apple. >>=20 >> I keep my Lightroom Classic catalogue on the mac's local (flash) >> drive, but the photo storage is on the network. The Import Backups >> directory is also on (a different) network drive. I use Lightroom's >> Import function to copy photos off SD cards using the mac's built-in >> SD card reader and register with the catalogue. So far so normal, >> I think. >>=20 >> The problem arose about ?three or four? months ago: could be >> coincident with OS or Lightroom upgrades, I can't remember, but I >> haven't changed anything about the setup, configuration or workflow. >> Now, every single time Lightroom does an import, while it's doing >> the first scan of the SD card to identify photos that it's seen >> before, all three of the Samba filesystems unmount from the mac, >> silently. I can find no record of error in any of the logs, >> suggesting that the system thinks that it happened deliberately. >> Needless to say, this throws out the import workflow, although it >> manages to pick itself up OK if I just re-mount everything. >>=20 >> Anyone have any similar experiences? Any thoughts of where I could >> poke it to find out why this might be happening? It feels like a >> time-out bug somewhere, but (a) there is no complaint, and (b) the >> network traffic is light at the time. Needless to say Apple >> documentation is useless. >>=20 >> Probably another good reason to find an alternative to Lightroom... >=20 > Other than the hardware specifics, I have the same exact workflow and = same players involved. >=20 > Maybe an odd question, but how are you mounting the SMB shares? In the fallback instance, I mount them with Finder command-K (Go -> = Connect to Server). Usually they mount automatically when I log in, = because I have them in my Startup Items. >=20 > Since updating to Catalina, I've found lots of problems dealing with = SMB using the Finder window and the items under the Locations side bar. = For instance: > - Mount a share. At some point overnight when nothing is using it, = the share is unmounted. I can't find anything in the logs to say why, = when, what, etc. Just unmounted. I haven't experienced any unattended unmounts. Just during Lightroom = import. I _have_ noticed (in Catalina, not before), that the Finder Sidebar does = not necessarily "keep up" with the mounted state of shares: after = logging in, I have seen my shared folders appear on the desktop as they = open, but the server listed in the side bar does not show the Eject icon = that comes with being mounted. > - With a fresh start of the Finder process and I can access the SMB = server/shares. After some time, activity, something, I only get = "Connection Failed" and can't access anything. What's really crazy, is = that I can't even unmount mounted shares from under Locations when it = gets in this state (I can unmount via right click on the desktop item, = CLI, etc). I see the share with the eject button, but just get useless = error message (if anything). > - Killing (killall -HUP Finder) makes everything work again for a = short bit. I've never been game to kill Finder from the command line. If it comes = to that I just reboot the mac. I figure there are so many daemon = processes involved these days that the chances of them not getting = tangled on manual intervention are slim. >=20 > If I mount the share via the Finder menubar (Go -> Connect to Server) = everything works and is rock solid. Mounts with no problems, no mystery = unmount, etc. I did test a Lightroom import and had no SMB issues when = the shared where mounted via the menu bar. After writing my call for help, I took a couple of test photos and did = another import and nothing broke. So perhaps there is an issue of = import size, or an issue with the first import after starting Lightroom, = or something. I'll be doing more experimentation to see what specific = issue triggers it. >=20 > I also store all of my music on the server and access it via SMB = mounts. I've noticed that the Apple Music app will automatically mount = the SMB share when I hit play. Unlike iTunes, I assume it's more aware = of the filesystem and mounts. Even with the Finder window off in la-la = land, the auto-mount by Music works fine. Now if this behavior could = only be exposed to every other app.... That sounds ominous. I don't use the Apple Music app (Apple doesn't = understand FLAC), but a precedent of automatically mounting and (one = assumes) unmounting "media shares" could be a clue. >=20 > My completely un-researched guess is that when they removed the = NetBIOS support (one of the changes in Catalina), some bug was = introduced or uncovered causing the problematic behavior. My guess ins = based on the assumption that the Finder window (not sure what to call = it) displays all of the network browsing and discovery and is the code = path that was impacted by the NetBIOS change. Any mount tied in with = this path gets impacted, and when this goes south, it takes the mounts = with it. Mounting via other means doesn't bring in the associated path. That sounds sort of plausible too. FWIW I'm running Avahi on my FreeBSD system, and advertising the SAMBA = shares in the "apple-approved" way: /user/local/etc/avahi/services/smb.service: %h _smb._tcp 445 _device-info._tcp 0 model=3DRackMac So I don't think that my system has been relying on NetBIOS at all. >=20 > I will check each new OS X update to see if this issue goes away. = Until then, I just mount my shares via "Go- > 'Connect to Server'". = Also note that you can re-enable NetBIOS support in Catalina = (https://medium.com/@gobinathm/how-to-access-smb-printer-shares-in-macos-c= atalina-10-15-17ea91d2c10b). I've not done nor tested that. >=20 > Hopefully a minor tweak to how you mount the SMB share will save you = the huge hassle of changing out tools. >=20 > Hope that helps. >=20 > Chris Thanks for the suggestions. I'll keep my eyes on the logs the next time = I do a big Lightroom import. Cheers, Andrew Reilly E: areilly@bigpond.net.au M: +61-409-824-272 From owner-freebsd-stable@freebsd.org Mon Nov 25 07:14:24 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 283C61CAD02 for ; Mon, 25 Nov 2019 07:14:24 +0000 (UTC) (envelope-from ml@netfence.it) Received: from soth.netfence.it (net-2-44-121-52.cust.vodafonedsl.it [2.44.121.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mailserver.netfence.it", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47Lyvf6rtHz44bT for ; Mon, 25 Nov 2019 07:14:22 +0000 (UTC) (envelope-from ml@netfence.it) Received: from guardian.ventu (89-97-212-98.ip19.fastwebnet.it [89.97.212.98]) (authenticated bits=0) by soth.netfence.it (8.15.2/8.15.2) with ESMTPSA id xAP7E9Y7081504 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Mon, 25 Nov 2019 08:14:10 +0100 (CET) (envelope-from ml@netfence.it) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netfence.it; s=201911; t=1574666053; bh=qV1Sd1a0aC3PTKZCFaSo8mKvRAaNS8cNKSQXx7alHj4=; h=Subject:To:References:From:Date:In-Reply-To; b=JA28p08YJ08Aq3XhQkX45mJcG1siPvwA/07WaHyNvoRxFOw8CrLcq9KrMoJa4LfBr SUjEjmjjj0FMahCZkUdzSwDOSn2ThQVTwauzrvd3hH7/lYGz31epVm7ZnoTM5kOD7D 0nEpcHKPWODwMA6NqWjj0KfxsEUHEp7WN3IxzIis= X-Authentication-Warning: soth.netfence.it: Host 89-97-212-98.ip19.fastwebnet.it [89.97.212.98] claimed to be guardian.ventu Subject: Re: Long-shot: repeatable macOS samba share unmounting during Lightroom import To: freebsd-stable@freebsd.org References: <28504691-D08B-483B-B4C5-CA47F2C523ED@bigpond.net.au> From: Andrea Venturoli Message-ID: <5d52cd16-6e1d-966c-355d-017ab09d2f62@netfence.it> Date: Mon, 25 Nov 2019 08:14:09 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.83 X-Rspamd-Queue-Id: 47Lyvf6rtHz44bT X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=netfence.it header.s=201911 header.b=JA28p08Y; dmarc=pass (policy=none) header.from=netfence.it; spf=pass (mx1.freebsd.org: domain of ml@netfence.it designates 2.44.121.52 as permitted sender) smtp.mailfrom=ml@netfence.it X-Spamd-Result: default: False [-5.06 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[netfence.it:s=201911]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:2.44.121.52]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[netfence.it:+]; DMARC_POLICY_ALLOW(-0.50)[netfence.it,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-2.06)[ip: (-6.70), ipnet: 2.44.0.0/16(-3.35), asn: 30722(-0.29), country: IT(0.03)]; ASN(0.00)[asn:30722, ipnet:2.44.0.0/16, country:IT]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Nov 2019 07:14:24 -0000 On 2019-11-24 21:01, Chris Gordon wrote: Just my 2c... > Since updating to Catalina, I've found lots of problems dealing with SMB using the Finder window and the items under the Locations side bar. For instance: > - Mount a share. At some point overnight when nothing is using it, the share is unmounted. I can't find anything in the logs to say why, when, what, etc. Just unmounted. I've got a customer who apparently is hit by the same problem: he boots his Mac in the morning and says in the afternoon (some of?) the shares are disconnected with no apparent reasons. I've yet to go and investigate this, but: a) I *think* (not sure) he uses the side bar (although I told him to use Command-K); b) his Mac has NOT been upgraded to Catalina yet. bye av. From owner-freebsd-stable@freebsd.org Mon Nov 25 09:22:07 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D4D521CDF45 for ; Mon, 25 Nov 2019 09:22:07 +0000 (UTC) (envelope-from tschweikle@gmail.com) Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com [IPv6:2a00:1450:4864:20::230]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47M1l30Q9Sz4BqY for ; Mon, 25 Nov 2019 09:22:06 +0000 (UTC) (envelope-from tschweikle@gmail.com) Received: by mail-lj1-x230.google.com with SMTP id j6so5907125lja.2 for ; Mon, 25 Nov 2019 01:22:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=R0eTHtAp1uRp4ji/k1g32SmnI+gpTCO+5SkiU917Kzg=; b=JcYhsLN0TL8VPmA1zRSGKqaGQqAXGLaXrCht56LcHMeuy+/35ySwrETUSqUzfucpSA wVWXVuzgq+4PxP5IPLCZaAsZsmyQ12Uh2cSEBmyiP/sXTEQmldNIZVqWWIFabV4d/ifF jsMpcX0IVLch5OzNKmoTDJYFiPYmVWuMfH/BtV5VaAzXQOp7Ii6mAW0/xi14KJDIBiKC XdyzWzJRKLMbbkjO4O4FbhkLI+vniX8QK9Z2Diwvz1OMjr8KrGqxuHvb31oRpcOij7lK F1G2mSmzpZhTjXyNf3WIbnIhIRJ+9ZvZvfmhQ37/VXtkQ3gHCJmE7eJysAA7+SCbc/x9 hO2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=R0eTHtAp1uRp4ji/k1g32SmnI+gpTCO+5SkiU917Kzg=; b=eAty/qB/MGp21JvGe954iqQGVgpP0eiLZvzoXkDaebbxUo5Nc2El8AJb1+qNrk09bc a/irrS69WwuOSVGLlf+cDAkZrhC1/+52loRBDgwXUTpBHE3m3hHkpWnnB4dJbCd/a0gN YMH8WZ9VSa5N27HND9W4A3d4dgSMmXodbOsz2tH1zPRgJge1r3gEkCHCZ/ESDJ0WD8ZT DK9ubP8y/nufW3IuopncATszs1TMWbkWT28sVULKR1rfokgaJSu0/vvD7iuY+6fjVrBy n+srPgC1+BtQGyqjjnTOsufVgdXib3RsRdwIJ2BOSxgX3GZyRcLwQvsR+Qb2ncCe71Wc L7vQ== X-Gm-Message-State: APjAAAVFv3/QVQzOs3KmxYQ6Mg8AndXNRlUiMiUgfupTMt8XlgbBc9wz mE175gwdPjb7ZVP3jyeZP/iYja2vviX4MFIsKdy9/w== X-Google-Smtp-Source: APXvYqyDBDTXlBICgKkMJ3CAGBHJ0aEbj9h6T+4BtBOA0EuV1Jkxnzqk1IF2j1Nmg7GqdidVE+8l62rbEFPTlMjC9ek= X-Received: by 2002:a2e:22c4:: with SMTP id i187mr22149921lji.86.1574673724817; Mon, 25 Nov 2019 01:22:04 -0800 (PST) MIME-Version: 1.0 From: Thomas Schweikle Date: Mon, 25 Nov 2019 10:21:44 +0100 Message-ID: Subject: Boot hangs after exausting uhub2: 7 ports with 7 removable, self powered To: freebsd-stable@freebsd.org X-Rspamd-Queue-Id: 47M1l30Q9Sz4BqY X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=JcYhsLN0; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of tschweikle@gmail.com designates 2a00:1450:4864:20::230 as permitted sender) smtp.mailfrom=tschweikle@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_DN_NONE(0.00)[]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE(0.00)[ip: (-9.13), ipnet: 2a00:1450::/32(-2.70), asn: 15169(-1.96), country: US(-0.05)]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[0.3.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Nov 2019 09:22:07 -0000 Hi! since 12.1-ALPHA9 FreeBSD wont boot anymore into multiuser. 12.0-ALPHA8 is ok, but not 12.0-ALPHA9. Same for: - FreeBSD 11.3_RELEASE, 11_STABLE - FreeBSD 12.1_RELEASE, 12_STABLE - FreeBSD 13-HEAD All on zfs without any ufs slice/partition/disk. It is the same for amd-64 and arm, arm64 architectures. Looks like ufs only systems do not have any problems booting. No hints about any changes to boot processes. ZFS loads, finds zfs:zroot. After this message some usb related stuff is printed, the last line then is "usb2: 7 ports with 7 removable, self powered". Module opensolaris.ko is build and loads, right together with module zfs.ko. So zfs shall work. Any idea what is going wrong here? -- Thomas From owner-freebsd-stable@freebsd.org Mon Nov 25 11:58:15 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 877081A9BEC for ; Mon, 25 Nov 2019 11:58:15 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 47M5CC12kqz4LBw for ; Mon, 25 Nov 2019 11:58:15 +0000 (UTC) (envelope-from david@catwhisker.org) Received: by mailman.nyi.freebsd.org (Postfix) id 21BEF1A9BEB; Mon, 25 Nov 2019 11:58:15 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 217F21A9BEA for ; Mon, 25 Nov 2019 11:58:15 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mx.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47M5C95lzQz4LBv for ; Mon, 25 Nov 2019 11:58:13 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.15.2/8.15.2) with ESMTP id xAPBwAND011581 for ; Mon, 25 Nov 2019 11:58:10 GMT (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.15.2/8.15.2/Submit) id xAPBwAiO011580 for stable@freebsd.org; Mon, 25 Nov 2019 03:58:10 -0800 (PST) (envelope-from david) Date: Mon, 25 Nov 2019 03:58:10 -0800 From: David Wolfskill To: stable@freebsd.org Subject: Error building stable/12 (amd64) at r355087 Message-ID: <20191125115810.GD1373@albert.catwhisker.org> Reply-To: stable@freebsd.org Mail-Followup-To: stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="ylS2wUBXLOxYXZFQ" Content-Disposition: inline User-Agent: Mutt/1.12.2 (2019-09-21) X-Rspamd-Queue-Id: 47M5C95lzQz4LBv X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of david@catwhisker.org designates 198.144.209.73 as permitted sender) smtp.mailfrom=david@catwhisker.org X-Spamd-Result: default: False [-2.89 / 15.00]; ARC_NA(0.00)[]; HAS_REPLYTO(0.00)[stable@freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:198.144.209.73]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[stable@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[73.209.144.198.list.dnswl.org : 127.0.10.0]; SIGNED_PGP(-2.00)[]; DMARC_NA(0.00)[catwhisker.org]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:7961, ipnet:198.144.208.0/20, country:US]; RCVD_COUNT_TWO(0.00)[2]; IP_SCORE(-3.49)[ip: (-9.32), ipnet: 198.144.208.0/20(-4.46), asn: 7961(-3.62), country: US(-0.05)]; REPLYTO_EQ_TO_ADDR(5.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Nov 2019 11:58:15 -0000 --ylS2wUBXLOxYXZFQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable This is during a source-based update from r355048 to r355087, during "stage 4.3: building everything" (using META_MODE); meta file reads: # Meta data file /common/S3/obj/usr/src/amd64.amd64/usr.sbin/camdd/camdd.o.= meta CMD cc -target x86_64-unknown-freebsd12.1 --sysroot=3D/common/S3/obj/usr/sr= c/amd64.amd64/tmp -B/common/S3/obj/usr/src/amd64.amd64/tmp/usr/bin -O2 -pi= pe -std=3Dgnu99 -fstack-protector-strong -Wsystem-headers -Werror -Wall -= Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-proto= types -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -W= shadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-= externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -Wmissin= g-variable-declarations -Wno-empty-body -Wno-string-plus-int -Wno-unused-co= nst-variable -Qunused-arguments -c /usr/src/usr.sbin/camdd/camdd.c -o cam= dd.o CMD=20 CWD /common/S3/obj/usr/src/amd64.amd64/usr.sbin/camdd TARGET camdd.o -- command output -- In file included from /usr/src/usr.sbin/camdd/camdd.c:54: In file included from /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/ma= chine/bus.h:6: In file included from /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/x8= 6/bus.h:1043: In file included from /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/ma= chine/bus_dma.h:34: /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/x86/bus_dma.h:182:1: err= or: unknown type name 'bool' bool bus_dma_dmar_set_buswide(device_t dev); ^ /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/x86/bus_dma.h:182:31: er= ror: unknown type name 'device_t' bool bus_dma_dmar_set_buswide(device_t dev); ^ 2 errors generated. *** Error code 1 Peace, david --=20 David H. Wolfskill david@catwhisker.org Claiming that Ukraine meddled in the 2016 US election is aiding and abetting Putin -- who does not have the US's best interest at heart. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --ylS2wUBXLOxYXZFQ Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGTBAEBCgB9FiEE4owz2QxMJyaxAefyQLJg+bY2PckFAl3bwdJfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEUy OEMzM0Q5MEM0QzI3MjZCMTAxRTdGMjQwQjI2MEY5QjYzNjNEQzkACgkQQLJg+bY2 Pckk2Qf8C5OPF4SWOTmaTwxXRSYQ9k+1KZWxRSxAW3/eVmz/gFk+GayBdyVnEuqH 0pia4tYdlGSyBVKZNInXQHn2z6KbkTrd48sGeML6TYpAUuT16lk2M+pDBbq3U9R7 h7rGt6dyeiCBOopR0JYHicZ0eNpMHypJxmtptkAJ9Xi/o70Q1hnyePsL6TCo7d1F aItywSAnTQDsE1cWTyC8GLW9rpr1FZ9MkTgp7rtA7DTAW0drhSUoCXMbPVH3Kf6Q SEZpSIZnDFqZs3KeMuC+UOfp+XjkVRVHK1pfuDZtc5g/dqPh33Xswez3Vtr4F5Cw kZrKSnIiydmvTPEc9lBMDn19QIne8A== =rE/h -----END PGP SIGNATURE----- --ylS2wUBXLOxYXZFQ-- From owner-freebsd-stable@freebsd.org Mon Nov 25 14:21:55 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C36321AE13F for ; Mon, 25 Nov 2019 14:21:55 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47M8Nz3rjHz4TTx for ; Mon, 25 Nov 2019 14:21:55 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 8416F1AE13B; Mon, 25 Nov 2019 14:21:55 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 83E041AE139 for ; Mon, 25 Nov 2019 14:21:55 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47M8Nz270Pz4TTt for ; Mon, 25 Nov 2019 14:21:54 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id xAPELfl0095780 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Mon, 25 Nov 2019 16:21:44 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua xAPELfl0095780 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id xAPELffu095779 for stable@freebsd.org; Mon, 25 Nov 2019 16:21:41 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 25 Nov 2019 16:21:41 +0200 From: Konstantin Belousov To: stable@freebsd.org Subject: Re: Error building stable/12 (amd64) at r355087 Message-ID: <20191125142141.GC10580@kib.kiev.ua> References: <20191125115810.GD1373@albert.catwhisker.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20191125115810.GD1373@albert.catwhisker.org> User-Agent: Mutt/1.12.2 (2019-09-21) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-Rspamd-Queue-Id: 47M8Nz270Pz4TTt X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Nov 2019 14:21:55 -0000 On Mon, Nov 25, 2019 at 03:58:10AM -0800, David Wolfskill wrote: > This is during a source-based update from r355048 to r355087, during > "stage 4.3: building everything" (using META_MODE); meta file reads: > > # Meta data file /common/S3/obj/usr/src/amd64.amd64/usr.sbin/camdd/camdd.o.meta > CMD cc -target x86_64-unknown-freebsd12.1 --sysroot=/common/S3/obj/usr/src/amd64.amd64/tmp -B/common/S3/obj/usr/src/amd64.amd64/tmp/usr/bin -O2 -pipe -std=gnu99 -fstack-protector-strong -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -Wmissing-variable-declarations -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Qunused-arguments -c /usr/src/usr.sbin/camdd/camdd.c -o camdd.o > CMD > CWD /common/S3/obj/usr/src/amd64.amd64/usr.sbin/camdd > TARGET camdd.o > -- command output -- > In file included from /usr/src/usr.sbin/camdd/camdd.c:54: > In file included from /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/machine/bus.h:6: > In file included from /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/x86/bus.h:1043: > In file included from /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/machine/bus_dma.h:34: > /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/x86/bus_dma.h:182:1: error: unknown type name 'bool' > bool bus_dma_dmar_set_buswide(device_t dev); > ^ > /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/x86/bus_dma.h:182:31: error: unknown type name 'device_t' > bool bus_dma_dmar_set_buswide(device_t dev); > ^ > 2 errors generated. > > *** Error code 1 I hope that this is fixed by r355089. I did not tracked down how HEAD was immune to the problem. From owner-freebsd-stable@freebsd.org Mon Nov 25 14:52:03 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 15DFB1AF3AD for ; Mon, 25 Nov 2019 14:52:03 +0000 (UTC) (envelope-from peter.blok@bsd4all.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47M93k6My4z4W2L for ; Mon, 25 Nov 2019 14:52:02 +0000 (UTC) (envelope-from peter.blok@bsd4all.org) Received: by mailman.nyi.freebsd.org (Postfix) id DA9D21AF3AC; Mon, 25 Nov 2019 14:52:02 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D94E11AF3AA for ; Mon, 25 Nov 2019 14:52:02 +0000 (UTC) (envelope-from peter.blok@bsd4all.org) Received: from smtpq2.tb.mail.iss.as9143.net (smtpq2.tb.mail.iss.as9143.net [212.54.42.165]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47M93j0yVSz4W2H for ; Mon, 25 Nov 2019 14:52:00 +0000 (UTC) (envelope-from peter.blok@bsd4all.org) Received: from [212.54.42.136] (helo=smtp12.tb.mail.iss.as9143.net) by smtpq2.tb.mail.iss.as9143.net with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1iZFiY-0002gf-57; Mon, 25 Nov 2019 15:51:58 +0100 Received: from 94-209-85-88.cable.dynamic.v4.ziggo.nl ([94.209.85.88] helo=wan0.bsd4all.org) by smtp12.tb.mail.iss.as9143.net with esmtp (Exim 4.90_1) (envelope-from ) id 1iZFiX-00053g-Sj; Mon, 25 Nov 2019 15:51:57 +0100 Received: from newnas (localhost [127.0.0.1]) by wan0.bsd4all.org (Postfix) with ESMTP id 6EF1228B; Mon, 25 Nov 2019 15:51:57 +0100 (CET) X-Virus-Scanned: amavisd-new at bsd4all.org Received: from wan0.bsd4all.org ([127.0.0.1]) by newnas (newnas.bsd4all.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id e-h_BFRLbk35; Mon, 25 Nov 2019 15:51:56 +0100 (CET) Received: from [192.168.1.65] (unknown [192.168.1.65]) by wan0.bsd4all.org (Postfix) with ESMTPSA id A8237CD; Mon, 25 Nov 2019 15:51:56 +0100 (CET) From: peter.blok@bsd4all.org Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: Error building stable/12 (amd64) at r355087 Date: Mon, 25 Nov 2019 15:51:56 +0100 References: <20191125115810.GD1373@albert.catwhisker.org> <20191125142141.GC10580@kib.kiev.ua> To: Konstantin Belousov , stable@freebsd.org In-Reply-To: <20191125142141.GC10580@kib.kiev.ua> Message-Id: <704A2A7A-3FC5-421E-A593-2A75D315D3D1@bsd4all.org> X-Mailer: Apple Mail (2.3445.104.11) X-SourceIP: 94.209.85.88 X-Ziggo-spambar: / X-Ziggo-spamscore: 0.0 X-Ziggo-spamreport: CMAE Analysis: v=2.3 cv=Rvq70xuK c=1 sm=1 tr=0 a=LYXyOGYQqFYBMgK+Y6iqTg==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=MeAgGD-zjQ4A:10 a=pGLkceISAAAA:8 a=6I5d2MoRAAAA:8 a=W0ZYaHBNIVl8oGWSMxMA:9 a=CjuIK1q_8ugA:10 a=MI7dwjCrUmeX9BtVjt8A:9 a=-ektBcCgkg_P1-rZ:21 a=_W_S_7VecoQA:10 a=IjZwj45LgO3ly-622nXo:22 X-Ziggo-Spam-Status: No X-Spam-Status: No X-Spam-Flag: No X-Rspamd-Queue-Id: 47M93j0yVSz4W2H X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of peter.blok@bsd4all.org designates 212.54.42.165 as permitted sender) smtp.mailfrom=peter.blok@bsd4all.org X-Spamd-Result: default: False [-3.14 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[6]; RECEIVED_SPAMHAUS_PBL(0.00)[88.85.209.94.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; RCVD_TLS_LAST(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:smtp.ziggo.nl/16]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[bsd4all.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FROM_NO_DN(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; IP_SCORE(-1.24)[ipnet: 212.54.32.0/20(-3.95), asn: 33915(-2.27), country: NL(0.02)]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_IN_DNSWL_LOW(-0.10)[165.42.54.212.list.dnswl.org : 127.0.5.1]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:33915, ipnet:212.54.32.0/20, country:NL]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[] Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Nov 2019 14:52:03 -0000 I can confirm it has been fixed. > On 25 Nov 2019, at 15:21, Konstantin Belousov = wrote: >=20 > On Mon, Nov 25, 2019 at 03:58:10AM -0800, David Wolfskill wrote: >> This is during a source-based update from r355048 to r355087, during >> "stage 4.3: building everything" (using META_MODE); meta file reads: >>=20 >> # Meta data file = /common/S3/obj/usr/src/amd64.amd64/usr.sbin/camdd/camdd.o.meta >> CMD cc -target x86_64-unknown-freebsd12.1 = --sysroot=3D/common/S3/obj/usr/src/amd64.amd64/tmp = -B/common/S3/obj/usr/src/amd64.amd64/tmp/usr/bin -O2 -pipe -std=3Dgnu99= -fstack-protector-strong -Wsystem-headers -Werror -Wall -Wno-format-y2k = -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes = -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch = -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline = -Wnested-externs -Wredundant-decls -Wold-style-definition = -Wno-pointer-sign -Wmissing-variable-declarations -Wno-empty-body = -Wno-string-plus-int -Wno-unused-const-variable -Qunused-arguments -c = /usr/src/usr.sbin/camdd/camdd.c -o camdd.o >> CMD=20 >> CWD /common/S3/obj/usr/src/amd64.amd64/usr.sbin/camdd >> TARGET camdd.o >> -- command output -- >> In file included from /usr/src/usr.sbin/camdd/camdd.c:54: >> In file included from = /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/machine/bus.h:6: >> In file included from = /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/x86/bus.h:1043: >> In file included from = /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/machine/bus_dma.h:34: >> = /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/x86/bus_dma.h:182:1: = error: unknown type name 'bool' >> bool bus_dma_dmar_set_buswide(device_t dev); >> ^ >> = /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/x86/bus_dma.h:182:31: = error: unknown type name 'device_t' >> bool bus_dma_dmar_set_buswide(device_t dev); >> ^ >> 2 errors generated. >>=20 >> *** Error code 1 >=20 > I hope that this is fixed by r355089. I did not tracked down how HEAD > was immune to the problem. > _______________________________________________ > freebsd-stable@freebsd.org mailing = list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable = > To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org = " From owner-freebsd-stable@freebsd.org Mon Nov 25 14:58:26 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 99CBE1AF5B9 for ; Mon, 25 Nov 2019 14:58:26 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from dec.sakura.ne.jp (dec.sakura.ne.jp [210.188.226.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47M9C45hZFz4WMP for ; Mon, 25 Nov 2019 14:58:24 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from kalamity.joker.local (124-18-96-116.dz.commufa.jp [124.18.96.116]) (authenticated bits=0) by dec.sakura.ne.jp (8.15.2/8.15.2/[SAKURA-WEB]/20080708) with ESMTPA id xAPEwEon061488; Mon, 25 Nov 2019 23:58:14 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Mon, 25 Nov 2019 23:58:14 +0900 From: Tomoaki AOKI To: freebsd-stable@freebsd.org Cc: kostikbel@gmail.com, tschweikle@gmail.com Subject: Re: Error building stable/12 (amd64) at r355087 Message-Id: <20191125235814.6aea80894959b37c9b267f70@dec.sakura.ne.jp> In-Reply-To: <20191125142141.GC10580@kib.kiev.ua> References: <20191125115810.GD1373@albert.catwhisker.org> <20191125142141.GC10580@kib.kiev.ua> Reply-To: junchoon@dec.sakura.ne.jp Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; amd64-portbld-freebsd12.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 47M9C45hZFz4WMP X-Spamd-Bar: +++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of junchoon@dec.sakura.ne.jp has no SPF policy when checking 210.188.226.8) smtp.mailfrom=junchoon@dec.sakura.ne.jp X-Spamd-Result: default: False [5.16 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[junchoon@dec.sakura.ne.jp]; MV_CASE(0.50)[]; TO_DN_NONE(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; HAS_ORG_HEADER(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[116.96.18.124.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.10]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:9370, ipnet:210.188.224.0/19, country:JP]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sakura.ne.jp]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(1.00)[1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(1.00)[1.000,0]; IP_SCORE(1.76)[ipnet: 210.188.224.0/19(4.85), asn: 9370(3.93), country: JP(0.02)]; R_SPF_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FREEMAIL_CC(0.00)[gmail.com] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Nov 2019 14:58:26 -0000 r347836 (not MFC'ed) on head eliminates inclusion of machine/bus.h for usr.sbin/camdd/camdd.c. This whole commit introduces some bool functions, but head builds fine. I'm not shure applying camdd.c part alone is OK, so cherry-picked this revision excluding sys/compat/linuxkpi/common/src/linux_pci.c part and it fixes build. Not yet tested r355089, though. On Mon, 25 Nov 2019 16:21:41 +0200 Konstantin Belousov wrote: > On Mon, Nov 25, 2019 at 03:58:10AM -0800, David Wolfskill wrote: > > This is during a source-based update from r355048 to r355087, during > > "stage 4.3: building everything" (using META_MODE); meta file reads: > > > > # Meta data file /common/S3/obj/usr/src/amd64.amd64/usr.sbin/camdd/camdd.o.meta > > CMD cc -target x86_64-unknown-freebsd12.1 --sysroot=/common/S3/obj/usr/src/amd64.amd64/tmp -B/common/S3/obj/usr/src/amd64.amd64/tmp/usr/bin -O2 -pipe -std=gnu99 -fstack-protector-strong -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -Wmissing-variable-declarations -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Qunused-arguments -c /usr/src/usr.sbin/camdd/camdd.c -o camdd.o > > CMD > > CWD /common/S3/obj/usr/src/amd64.amd64/usr.sbin/camdd > > TARGET camdd.o > > -- command output -- > > In file included from /usr/src/usr.sbin/camdd/camdd.c:54: > > In file included from /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/machine/bus.h:6: > > In file included from /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/x86/bus.h:1043: > > In file included from /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/machine/bus_dma.h:34: > > /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/x86/bus_dma.h:182:1: error: unknown type name 'bool' > > bool bus_dma_dmar_set_buswide(device_t dev); > > ^ > > /common/S3/obj/usr/src/amd64.amd64/tmp/usr/include/x86/bus_dma.h:182:31: error: unknown type name 'device_t' > > bool bus_dma_dmar_set_buswide(device_t dev); > > ^ > > 2 errors generated. > > > > *** Error code 1 > > I hope that this is fixed by r355089. I did not tracked down how HEAD > was immune to the problem. > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" -- Tomoaki AOKI From owner-freebsd-stable@freebsd.org Mon Nov 25 15:55:44 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 92A681B0EFD for ; Mon, 25 Nov 2019 15:55:44 +0000 (UTC) (envelope-from peter.blok@bsd4all.org) Received: from smtpq1.tb.mail.iss.as9143.net (smtpq1.tb.mail.iss.as9143.net [212.54.42.164]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47MBTC0DRpz4ZX3 for ; Mon, 25 Nov 2019 15:55:42 +0000 (UTC) (envelope-from peter.blok@bsd4all.org) Received: from [212.54.42.135] (helo=smtp11.tb.mail.iss.as9143.net) by smtpq1.tb.mail.iss.as9143.net with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1iZGiC-0001Hd-Ah for freebsd-stable@freebsd.org; Mon, 25 Nov 2019 16:55:40 +0100 Received: from 94-209-85-88.cable.dynamic.v4.ziggo.nl ([94.209.85.88] helo=wan0.bsd4all.org) by smtp11.tb.mail.iss.as9143.net with esmtp (Exim 4.90_1) (envelope-from ) id 1iZGiC-0006JD-5z for freebsd-stable@freebsd.org; Mon, 25 Nov 2019 16:55:40 +0100 Received: from newnas (localhost [127.0.0.1]) by wan0.bsd4all.org (Postfix) with ESMTP id 66CE3217 for ; Mon, 25 Nov 2019 16:55:39 +0100 (CET) X-Virus-Scanned: amavisd-new at bsd4all.org Received: from wan0.bsd4all.org ([127.0.0.1]) by newnas (newnas.bsd4all.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id V9hI9cljkwJc for ; Mon, 25 Nov 2019 16:55:38 +0100 (CET) Received: from [192.168.1.65] (unknown [192.168.1.65]) by wan0.bsd4all.org (Postfix) with ESMTPSA id DDC0948 for ; Sun, 24 Nov 2019 16:19:32 +0100 (CET) From: peter.blok@bsd4all.org Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: Long-shot: repeatable macOS samba share unmounting during Lightroom import Date: Sun, 24 Nov 2019 16:19:32 +0100 References: <28504691-D08B-483B-B4C5-CA47F2C523ED@bigpond.net.au> <89708db0-940e-43b3-c428-4fa980a3abbb@ingresso.co.uk> To: FreeBSD Stable In-Reply-To: <89708db0-940e-43b3-c428-4fa980a3abbb@ingresso.co.uk> Message-Id: <3FD14E13-45A1-46BE-9559-74F63ED03719@bsd4all.org> X-Mailer: Apple Mail (2.3445.104.11) X-SourceIP: 94.209.85.88 X-Ziggo-spambar: / X-Ziggo-spamscore: 0.0 X-Ziggo-spamreport: CMAE Analysis: v=2.3 cv=GKl27dFK c=1 sm=1 tr=0 a=LYXyOGYQqFYBMgK+Y6iqTg==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=kj9zAlcOel0A:10 a=MeAgGD-zjQ4A:10 a=cM7xiVoAAAAA:8 a=6I5d2MoRAAAA:8 a=4Mb7YEFmdEMmLFW_hsEA:9 a=CjuIK1q_8ugA:10 a=c7CaBhpcVtvhwdizcjts:22 a=IjZwj45LgO3ly-622nXo:22 X-Ziggo-Spam-Status: No X-Spam-Status: No X-Spam-Flag: No X-Rspamd-Queue-Id: 47MBTC0DRpz4ZX3 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of peter.blok@bsd4all.org designates 212.54.42.164 as permitted sender) smtp.mailfrom=peter.blok@bsd4all.org X-Spamd-Result: default: False [-2.12 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[6]; RECEIVED_SPAMHAUS_PBL(0.00)[88.85.209.94.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; RCVD_TLS_LAST(0.00)[]; R_SPF_ALLOW(-0.20)[+a:smtp.ziggo.nl/16]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[bsd4all.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE(-1.22)[ipnet: 212.54.32.0/20(-3.90), asn: 33915(-2.23), country: NL(0.02)]; TO_DN_ALL(0.00)[]; FROM_NO_DN(0.00)[]; MV_CASE(0.50)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_IN_DNSWL_LOW(-0.10)[164.42.54.212.list.dnswl.org : 127.0.5.1]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:33915, ipnet:212.54.32.0/20, country:NL]; MID_RHS_MATCH_FROM(0.00)[]; DATE_IN_PAST(1.00)[]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Nov 2019 15:55:44 -0000 The fruit module forces avahi or mdns_responder to be compiled as well. = A share dispappearing could be due to some interaction with avahi. It = could be that the combination samba+fruit+avahi and samba+avahi is = having different behavior. Peter > On 24 Nov 2019, at 12:15, Pete French = wrote: >=20 > I have a very similar setup to you for serving files to my Mac from a = FreeBSD server. I haven't seen the unmount problem, but I di have a few = oddities until I added the 'fruit' module on the Samba side, which helps = with compatbiloty with the Mac. The appropriate bit of my config looks = like this: >=20 > vfs objects =3D fruit streams_xattr zfsacl > fruit:resource =3D xattr > fruit:encoding =3D private >=20 > Don't ask me what they do anymore, I added them ages ago, but it does = work very nicely for me. You may already have this of course, but worth = pointing out just in case as it took me a few years to discover it! >=20 > As someone else has said though, this may well be a Catalina bug. I am = not running that (MacBook too old, and not buying another until the new = keyboards are avilable n the replacement I want). >=20 > -pete. > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Mon Nov 25 17:01:18 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 42BD21B28A4 for ; Mon, 25 Nov 2019 17:01:18 +0000 (UTC) (envelope-from stable@lordcow.org) Received: from mail.lordcow.org (lordcow.org [IPv6:2c0f:fb18:402:5::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "devaux.za.net", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47MCwr0L9Sz4dZ3 for ; Mon, 25 Nov 2019 17:01:15 +0000 (UTC) (envelope-from stable@lordcow.org) Received: from lordcow.org (localhost [127.0.0.1]) by mail.lordcow.org (8.15.2/8.15.2) with ESMTPS id xAPH13em006244 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 25 Nov 2019 19:01:03 +0200 (SAST) (envelope-from lordcow@lordcow.org) X-Authentication-Warning: lordcow.org: Host localhost [127.0.0.1] claimed to be lordcow.org Received: (from lordcow@localhost) by lordcow.org (8.15.2/8.15.2/Submit) id xAPH0wmm006004 for freebsd-stable@freebsd.org; Mon, 25 Nov 2019 19:00:58 +0200 (SAST) (envelope-from lordcow) Date: Mon, 25 Nov 2019 19:00:58 +0200 From: Gareth de Vaux To: freebsd-stable@freebsd.org Subject: ZFS ghost files on 11.3-STABLE Message-ID: <20191125170058.GA10727@lordcow.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.12.2 (2019-09-21) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on lordcow.org X-Rspamd-Queue-Id: 47MCwr0L9Sz4dZ3 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of stable@lordcow.org designates 2c0f:fb18:402:5::2 as permitted sender) smtp.mailfrom=stable@lordcow.org X-Spamd-Result: default: False [-2.27 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.982,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2c0f:fb18:402:5::2/64]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; HAS_XAW(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.99)[-0.991,0]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[lordcow.org]; IP_SCORE(0.00)[country: ZA(0.00)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:37199, ipnet:2c0f:fb18::/32, country:ZA]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Nov 2019 17:01:18 -0000 Hi all, I deleted a file (rm filter) by mistake instead of its backup 'filter~', however it seems a remnant of a much older version of the file (given by its date and content) is half hanging around? $ ls -l filter ls: filter: No such file or directory $ ls -l filter* -rw-r--r-- 1 lordcow lordcow 18179 Mar 28 2019 filter -rw------- 1 lordcow lordcow 19092 Nov 20 23:36 filter~ $ stat filter stat: filter: stat: No such file or directory $ stat filter* 1525321853 1911959 -rw-r--r-- 1 lordcow lordcow 4294967295 18179 "Mar 28 17:07:34 2019" "Mar 28 17:08:07 2019" "Mar 28 17:08:07 2019" "Mar 28 17:07:34 2019" 18432 24 0x800 filter 1525321853 833819 -rw------- 1 lordcow lordcow 4294967295 19092 "Aug 10 22:31:53 2016" "Nov 20 23:36:40 2019" "Nov 24 23:08:13 2019" "Nov 20 23:36:40 2019" 19456 24 0x800 filter~ $ cat filter cat: filter: No such file or directory $ cat filter* (cats both files) A zpool scrub returns no errors I haven't tried to overwrite the file yet incase anyone wants some output. FreeBSD 11.3-STABLE #0 r353939 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) From owner-freebsd-stable@freebsd.org Mon Nov 25 17:25:28 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F299C1B31EF for ; Mon, 25 Nov 2019 17:25:28 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f180.google.com (mail-oi1-f180.google.com [209.85.167.180]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47MDSm0HpMz4fdX for ; Mon, 25 Nov 2019 17:25:27 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f180.google.com with SMTP id x21so6745548oic.0 for ; Mon, 25 Nov 2019 09:25:27 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Yr6838THtEKJuhX1HxLkbg1KeMm2JkT//Kv9vq56MQg=; b=Up03Aj+xVJrBzbro+kT8b4aKmf4XrTBOwpxbQT9QaSbZo+oX4DTwWsXAYpk43o6m7M a9ht/pa+D3st9xMO7RNPGm250pkI+5cAK89xCjbabhJIvIhLcLR+vrrBzzGkTd96e4hq zWQxMIdw4Q3aXWTuCkJ7YyEa5W1XsjeIxIsST32zfn2cW5uFpvp4P2INVxjkZkf4ENd1 M+nODsxqS5/79gfPn9SFBpox31d6xkNJkp9Lq6/Iq/vcY02+NKQB5/gO0shDCbF8VcGO FoQrMSxApZPW4LgprJcqjBAloYLadSJaxKdPNpV1kpsdTg6xk8jMelhO+oCFUCG+LqZk 9xhw== X-Gm-Message-State: APjAAAUwxkSEN+8k9X6GrEsxdph6nYh3pDfit709P/uv1wf5q0Ev762k WActxgIpauU+eaIyNStSEgHUF4VabQ+8SF7znU+NqaJt X-Google-Smtp-Source: APXvYqwbGMuZO6P8fZAFNsjcAF1HtsiWoM4oGJpztcqMKQDijoETYvtjuWQs0cCjSmYgoY72pgWQQ5K6mwWTSUbPmHc= X-Received: by 2002:aca:e108:: with SMTP id y8mr23879115oig.73.1574702726581; Mon, 25 Nov 2019 09:25:26 -0800 (PST) MIME-Version: 1.0 References: <20191125170058.GA10727@lordcow.org> In-Reply-To: <20191125170058.GA10727@lordcow.org> From: Alan Somers Date: Mon, 25 Nov 2019 10:25:16 -0700 Message-ID: Subject: Re: ZFS ghost files on 11.3-STABLE To: Gareth de Vaux Cc: FreeBSD X-Rspamd-Queue-Id: 47MDSm0HpMz4fdX X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.167.180 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-3.14 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[180.167.85.209.list.dnswl.org : 127.0.5.0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; IP_SCORE(-1.14)[ip: (-0.51), ipnet: 209.85.128.0/17(-3.17), asn: 15169(-1.96), country: US(-0.05)]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[180.167.85.209.rep.mailspike.net : 127.0.0.17]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Nov 2019 17:25:29 -0000 On Mon, Nov 25, 2019 at 10:01 AM Gareth de Vaux wrote: > Hi all, I deleted a file (rm filter) by mistake instead of its backup > 'filter~', > however it seems a remnant of a much older version of the file (given by > its > date and content) is half hanging around? > > $ ls -l filter > ls: filter: No such file or directory > > $ ls -l filter* > -rw-r--r-- 1 lordcow lordcow 18179 Mar 28 2019 filter > -rw------- 1 lordcow lordcow 19092 Nov 20 23:36 filter~ > > $ stat filter > stat: filter: stat: No such file or directory > > $ stat filter* > 1525321853 1911959 -rw-r--r-- 1 lordcow lordcow 4294967295 18179 "Mar 28 > 17:07:34 2019" "Mar 28 17:08:07 2019" "Mar 28 17:08:07 2019" "Mar 28 > 17:07:34 2019" 18432 24 0x800 filter > 1525321853 833819 -rw------- 1 lordcow lordcow 4294967295 19092 "Aug 10 > 22:31:53 2016" "Nov 20 23:36:40 2019" "Nov 24 23:08:13 2019" "Nov 20 > 23:36:40 2019" 19456 24 0x800 filter~ > > $ cat filter > cat: filter: No such file or directory > > $ cat filter* > (cats both files) > > A zpool scrub returns no errors > > I haven't tried to overwrite the file yet incase anyone wants some output. > > > FreeBSD 11.3-STABLE #0 r353939 > > ZFS filesystem version: 5 > ZFS storage pool version: features support (5000) > Oh Lord Cow, is there a directory beginning with the word filter and containing files named filter and filter~? Do "ls -ld filter*". -Alan From owner-freebsd-stable@freebsd.org Mon Nov 25 17:35:05 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D9E531B37C4 for ; Mon, 25 Nov 2019 17:35:05 +0000 (UTC) (envelope-from SRS0=C6/G=ZR=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47MDgr6ggMz4gKG for ; Mon, 25 Nov 2019 17:35:04 +0000 (UTC) (envelope-from SRS0=C6/G=ZR=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id B9CF128422; Mon, 25 Nov 2019 18:35:03 +0100 (CET) Received: from illbsd.quip.test (ip-62-24-92-232.net.upcbroadband.cz [62.24.92.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id D75322840C; Mon, 25 Nov 2019 18:35:01 +0100 (CET) Subject: Re: ZFS ghost files on 11.3-STABLE To: Gareth de Vaux , freebsd-stable@freebsd.org References: <20191125170058.GA10727@lordcow.org> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <86336ae2-5759-36b7-60d6-e9fe64cd4416@quip.cz> Date: Mon, 25 Nov 2019 18:35:01 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Firefox/52.0 SeaMonkey/2.49.3 MIME-Version: 1.0 In-Reply-To: <20191125170058.GA10727@lordcow.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 47MDgr6ggMz4gKG X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of SRS0=C6/G=ZR=quip.cz=000.fbsd@elsa.codelab.cz has no SPF policy when checking 94.124.105.4) smtp.mailfrom=SRS0=C6/G=ZR=quip.cz=000.fbsd@elsa.codelab.cz X-Spamd-Result: default: False [4.07 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; IP_SCORE(0.89)[ip: (0.40), ipnet: 94.124.104.0/21(0.20), asn: 42000(3.75), country: CZ(0.09)]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[quip.cz]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.99)[0.990,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[4.105.124.94.list.dnswl.org : 127.0.10.0]; NEURAL_SPAM_LONG(1.00)[0.996,0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[000.fbsd@quip.cz,SRS0=C6/G=ZR=quip.cz=000.fbsd@elsa.codelab.cz]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ]; FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz,SRS0=C6/G=ZR=quip.cz=000.fbsd@elsa.codelab.cz]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Nov 2019 17:35:05 -0000 Gareth de Vaux wrote on 2019/11/25 18:00: > Hi all, I deleted a file (rm filter) by mistake instead of its backup 'filter~', > however it seems a remnant of a much older version of the file (given by its > date and content) is half hanging around? > > $ ls -l filter > ls: filter: No such file or directory > > $ ls -l filter* > -rw-r--r-- 1 lordcow lordcow 18179 Mar 28 2019 filter > -rw------- 1 lordcow lordcow 19092 Nov 20 23:36 filter~ > > $ stat filter > stat: filter: stat: No such file or directory > > $ stat filter* > 1525321853 1911959 -rw-r--r-- 1 lordcow lordcow 4294967295 18179 "Mar 28 17:07:34 2019" "Mar 28 17:08:07 2019" "Mar 28 17:08:07 2019" "Mar 28 17:07:34 2019" 18432 24 0x800 filter > 1525321853 833819 -rw------- 1 lordcow lordcow 4294967295 19092 "Aug 10 22:31:53 2016" "Nov 20 23:36:40 2019" "Nov 24 23:08:13 2019" "Nov 20 23:36:40 2019" 19456 24 0x800 filter~ > > $ cat filter > cat: filter: No such file or directory > > $ cat filter* > (cats both files) > > A zpool scrub returns no errors > > I haven't tried to overwrite the file yet incase anyone wants some output. > > > FreeBSD 11.3-STABLE #0 r353939 > > ZFS filesystem version: 5 > ZFS storage pool version: features support (5000) Do you have snapshot of this dataset before you run the "rm filter"? What files are listed there? It is possible you have some non printable (invisible) character in the filename. It can be trailing space, newline or something else so in fact it has different filename than the one of deleted file but it looks the same. Miroslav Lachman From owner-freebsd-stable@freebsd.org Mon Nov 25 17:43:00 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 68EFD1B3CE3 for ; Mon, 25 Nov 2019 17:43:00 +0000 (UTC) (envelope-from spork@bway.net) Received: from smtp2.bway.net (smtp2.bway.net [216.220.96.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47MDrz4Dpdz3CKH for ; Mon, 25 Nov 2019 17:42:59 +0000 (UTC) (envelope-from spork@bway.net) Received: from frankentosh.sporklab.com (pool-173-70-93-30.nwrknj.fios.verizon.net [173.70.93.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: spork@bway.net) by smtp2.bway.net (Postfix) with ESMTPSA id F20399587A; Mon, 25 Nov 2019 12:42:49 -0500 (EST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: Long-shot: repeatable macOS samba share unmounting during Lightroom import From: Charles Sprickman In-Reply-To: <3FD14E13-45A1-46BE-9559-74F63ED03719@bsd4all.org> Date: Mon, 25 Nov 2019 12:42:49 -0500 Cc: FreeBSD Stable Content-Transfer-Encoding: quoted-printable Message-Id: <3C55BDE8-FF42-4141-9F91-354279515007@bway.net> References: <28504691-D08B-483B-B4C5-CA47F2C523ED@bigpond.net.au> <89708db0-940e-43b3-c428-4fa980a3abbb@ingresso.co.uk> <3FD14E13-45A1-46BE-9559-74F63ED03719@bsd4all.org> To: peter.blok@bsd4all.org X-Mailer: Apple Mail (2.3273) X-Rspamd-Queue-Id: 47MDrz4Dpdz3CKH X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.61 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[bway.net:s=mail]; RECEIVED_SPAMHAUS_PBL(0.00)[30.93.70.173.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.10]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:216.220.96.28/32]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(-0.01)[country: US(-0.05)]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; DWL_DNSWL_LOW(-1.00)[bway.net.dwl.dnswl.org : 127.0.5.1]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bway.net:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[bway.net,quarantine]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_IN_DNSWL_LOW(-0.10)[28.96.220.216.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8059, ipnet:216.220.96.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Nov 2019 17:43:00 -0000 > On Nov 24, 2019, at 10:19 AM, peter.blok@bsd4all.org wrote: >=20 > The fruit module forces avahi or mdns_responder to be compiled as = well. A share dispappearing could be due to some interaction with avahi. = It could be that the combination samba+fruit+avahi and samba+avahi is = having different behavior. You guys are making feel like my laziness in sticking with AFP is = actually paying off. :) Apple=E2=80=99s QA is really garbage these days. I have a hackintosh at = home that by some fluke of its uniqueness can cause a kernel panic on = any mac connecting to it via SMB. Can reproduce it all day. Connect, do = stuff, and after about 5 minutes idle, the connecting machine will = panic. Report it every time, existed since at least 10.12... Charles > Peter >=20 >=20 >=20 >> On 24 Nov 2019, at 12:15, Pete French = wrote: >>=20 >> I have a very similar setup to you for serving files to my Mac from a = FreeBSD server. I haven't seen the unmount problem, but I di have a few = oddities until I added the 'fruit' module on the Samba side, which helps = with compatbiloty with the Mac. The appropriate bit of my config looks = like this: >>=20 >> vfs objects =3D fruit streams_xattr zfsacl >> fruit:resource =3D xattr >> fruit:encoding =3D private >>=20 >> Don't ask me what they do anymore, I added them ages ago, but it does = work very nicely for me. You may already have this of course, but worth = pointing out just in case as it took me a few years to discover it! >>=20 >> As someone else has said though, this may well be a Catalina bug. I = am not running that (MacBook too old, and not buying another until the = new keyboards are avilable n the replacement I want). >>=20 >> -pete. >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Mon Nov 25 18:44:11 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7933F1B5E8F for ; Mon, 25 Nov 2019 18:44:11 +0000 (UTC) (envelope-from stable@lordcow.org) Received: from mail.lordcow.org (lordcow.org [IPv6:2c0f:fb18:402:5::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "devaux.za.net", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47MGCW2Lz7z3Hr1 for ; Mon, 25 Nov 2019 18:44:06 +0000 (UTC) (envelope-from stable@lordcow.org) Received: from lordcow.org (localhost [127.0.0.1]) by mail.lordcow.org (8.15.2/8.15.2) with ESMTPS id xAPIi0T8028410 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Nov 2019 20:44:01 +0200 (SAST) (envelope-from lordcow@lordcow.org) X-Authentication-Warning: lordcow.org: Host localhost [127.0.0.1] claimed to be lordcow.org Received: (from lordcow@localhost) by lordcow.org (8.15.2/8.15.2/Submit) id xAPIhri2028175; Mon, 25 Nov 2019 20:43:53 +0200 (SAST) (envelope-from lordcow) Date: Mon, 25 Nov 2019 20:43:53 +0200 From: Gareth de Vaux To: Miroslav Lachman <000.fbsd@quip.cz> Cc: freebsd-stable@freebsd.org Subject: Re: ZFS ghost files on 11.3-STABLE Message-ID: <20191125184353.GA21392@lordcow.org> References: <20191125170058.GA10727@lordcow.org> <86336ae2-5759-36b7-60d6-e9fe64cd4416@quip.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <86336ae2-5759-36b7-60d6-e9fe64cd4416@quip.cz> User-Agent: Mutt/1.12.2 (2019-09-21) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on lordcow.org X-Rspamd-Queue-Id: 47MGCW2Lz7z3Hr1 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of stable@lordcow.org designates 2c0f:fb18:402:5::2 as permitted sender) smtp.mailfrom=stable@lordcow.org X-Spamd-Result: default: False [-2.29 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.991,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2c0f:fb18:402:5::2/64:c]; NEURAL_HAM_LONG(-1.00)[-0.997,0]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; DMARC_NA(0.00)[lordcow.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(0.00)[country: ZA(0.00)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:37199, ipnet:2c0f:fb18::/32, country:ZA]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Nov 2019 18:44:11 -0000 On Mon 2019-11-25 (18:35), Miroslav Lachman wrote: > It is possible you have some non printable (invisible) character in the > filename. It can be trailing space, newline or something else so in fact Wow I'm an idiot, thought I'd spotted a bug, yes there was a trailing space, many thanks. From owner-freebsd-stable@freebsd.org Mon Nov 25 20:36:34 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EC45A1B9742 for ; Mon, 25 Nov 2019 20:36:34 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-qk1-x733.google.com (mail-qk1-x733.google.com [IPv6:2607:f8b0:4864:20::733]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47MJjF3PQjz3QV4 for ; Mon, 25 Nov 2019 20:36:33 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-qk1-x733.google.com with SMTP id e187so14101648qkf.4 for ; Mon, 25 Nov 2019 12:36:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=4yHhEzkgkgdVt+ZOowlUsjZcP8u+kMEocwZYlCK+gfI=; b=GGMKcFpZ8OoEv9FKQWu900WbDduDpvZ3J8bWaefTtK7U6+2F3RM4FjDDxWFksRe4ac S1r8IVcGQp8GW/LxEnSnRZ0jlxQMetvJCEfnO3hWmMOdBg2ONT/EZUJZPirC0KgG+Obr pP9LSyuWYJFBUm5cO1/zEh+Tg98u+sIg8qbShmu/gEV+5bHhHMfsNNiDayt2Yl39QVpP 4zjjaEuMy9fTltvsyMIkPisTFpAfEdUP2waPFDMhVQMmWBNlxp4b+hilk1N3Z2aAAqy6 49lqshfEV353V9AZilKcRq4wBoM/t2knJRoyQDJCNHnSasSFngpiFVv6zuo4ogJrGbSC Hu9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=4yHhEzkgkgdVt+ZOowlUsjZcP8u+kMEocwZYlCK+gfI=; b=AsrtvjT7zENbwPWldv4IhaqdoRYj2UI2iurEer81xqjP6wgOCmz37LwKkHiaAnBm+r +yByrD4KtpIn9eMFFMrxiqkzyaJk0umgpbIr9ANToGlY0ZnexX05Jk+PYB6masgVGKaG eMFxR6o4fblBKbysQnp3m+CSeJ3gf602QGrOE+uME3DXQ5tTONz6KHVGyXluyY4Z39bM hGbd/KQZBK99Fq7kVDzyMZjx2TYALvTrAFfd2hW/1yYNSJZ85nIUjRinvIEJRhrzFElh wNvo8RWiKcFI+C60Nb2wbjK6eq337OBODdr0erSfilAL2KAB4Vh19MCiJ1em+ZAe7vfv /XVQ== X-Gm-Message-State: APjAAAWofmXRnxEIjMoP/tDePvcl6l6aMimKFmXolHyDbGlbUFFcgCca JAmDU7aFoU+ZpAVW1aoXYC/JtDjjO4cn36UI8WtFzw== X-Google-Smtp-Source: APXvYqz+WlHyG91kWclAG92wwQCPar8Gbzkj6aBOLRuuA5qFR7gpWH0y9zzE1bUGE8Cx3URc0c45GA0JOjIbUukHo8w= X-Received: by 2002:a05:620a:133b:: with SMTP id p27mr11473396qkj.363.1574714192137; Mon, 25 Nov 2019 12:36:32 -0800 (PST) MIME-Version: 1.0 References: <20191119213319.GD38096@zxy.spb.ru> In-Reply-To: <20191119213319.GD38096@zxy.spb.ru> From: Ryan Stone Date: Mon, 25 Nov 2019 15:36:21 -0500 Message-ID: Subject: Re: Access to NETMAP from c++ program To: Slawa Olhovchenkov Cc: freebsd-stable@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 47MJjF3PQjz3QV4 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=GGMKcFpZ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rysto32@gmail.com designates 2607:f8b0:4864:20::733 as permitted sender) smtp.mailfrom=rysto32@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[3.3.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; IP_SCORE(0.00)[ip: (-9.25), ipnet: 2607:f8b0::/32(-2.27), asn: 15169(-1.96), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Nov 2019 20:36:35 -0000 Remove "using namespace std;" from your program. From owner-freebsd-stable@freebsd.org Tue Nov 26 13:53:05 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5FD971B6E95 for ; Tue, 26 Nov 2019 13:53:05 +0000 (UTC) (envelope-from salvadore@FreeBSD.org) Received: from MailHost (unknown [80.211.33.142]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47MljD55Hyz3M1P for ; Tue, 26 Nov 2019 13:53:04 +0000 (UTC) (envelope-from salvadore@FreeBSD.org) Received: from root (uid 0) (envelope-from salvadore@FreeBSD.org) id 229c by MailHost (DragonFly Mail Agent v0.11+); Tue, 26 Nov 2019 14:52:33 +0100 To: freebsd-hackers@FreeBSD.org Subject: FreeBSD Quarterly Status Report - Third Quarter 2019 Cc: freebsd-stable@FreeBSD.org Date: Tue, 26 Nov 2019 14:52:33 +0100 Message-Id: <5ddd2e21.229c.5acaaa0a@MailHost> From: X-Rspamd-Queue-Id: 47MljD55Hyz3M1P X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-1.80 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-0.91)[-0.912,0]; NEURAL_HAM_LONG(-0.89)[-0.885,0]; ASN(0.00)[asn:31034, ipnet:80.211.0.0/17, country:IT] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Nov 2019 13:53:05 -0000 FreeBSD Project Quarterly Status Report - Third Quarter 2019 Here is the third quarterly status report for 2019. This quarter the reports team has been more active than usual thanks to a better organization: calls for reports and reminders have been sent regularly, reports have been reviewed and merged quickly (I would like to thank debdrup@ in particular for his reviewing work). Efficiency could still be improved with the help of our community. In particular, the quarterly team has found that many reports have arrived in the last days before the deadline or even after. I would like to invite the community to follow the guidelines below that can help us sending out the reports sooner. Starting from next quarter, all quarterly status reports will be prepared the last month of the quarter itself, instead of the first month after the quarter's end. This means that deadlines for submitting reports will be the 1st of January, April, July and October. Next quarter will then be a short one, covering the months of November and December only and the report will probably be out in mid January. -- Lorenzo Salvadore __________________________________________________________________ FreeBSD Team Reports * Cluster Administration Team * Continuous Integration * FreeBSD Core Team * FreeBSD Foundation * FreeBSD Graphics Team status report * FreeBSD Release Engineering Team * FreeBSD Security Team Projects * FAT / msdosfs support for makefs(8) * FUSE * Google Summer of Code 2019 * GSoC'19 Project - MAC policy on IP addresses in Jail: mac_ipacl * Improving laptop support * NFS Version 4.2 implementation * Rockchip RK3399 SoC's eMMC support * syzkaller on FreeBSD * TPM2 Software Stack (TSS2) Kernel * Casueword(9) livelock * Kernel Mapping Protections * Kernel ZLIB Update * PROT_MAX mmap/mprotect maximum protections API * Randomized Top of Stack pointer * Signals delivered on unhandled Page Faults Architectures * Broadcom ARM64 SoC support * FreeBSD support for the forthcoming Arm Morello CPU, SoC, and board * FreeBSD/powerpc Project * NXP ARM64 SoC support Userland Programs * gets(3) retirement Ports * FreshPorts * Java on FreeBSD * KDE on FreeBSD * Ports Collection * XFCE 4.14 update Third-Party Projects * ClonOS: virtualization platform on top of FreeBSD Operating System * ENA FreeBSD Driver Update * Nomad pot driver - Orchestrating jails via nomad * sysctlinfo __________________________________________________________________ FreeBSD Team Reports Entries from the various official and semi-official teams, as found in the Administration Page. Cluster Administration Team Contact: Cluster Administration Team The FreeBSD Cluster Administration Team consists of the people responsible for administering the machines that the Project relies on for its distributed work and communications to be synchronised. In this quarter, the team has worked on the following: * Change IPv6 address in TWN site. * Solved hardware issues in KWC site (with hrs@). * Moved remaining infrastructure from the YSV (Yahoo!) site to NYI (New York Internet) (peter@). * YSV hosted most of FreeBSD.org between 2000 and 2019. Installed new machines for portmgr@ courtesy of the FreeBSD Foundation. Resolved outtages (thanks uqs@) with GitHub exporter, Bugzilla and hg-beta (thanks bapt@). PowerPC64 servers are online (power8) building pkgs and reference hosts. Ongoing systems administration work: * Creating accounts for new committers. * Backups of critical infrastructure. * Keeping up with security updates in 3rd party software. Work in progress: * Review the service jails and service administrators operation. * South Africa Mirror (JINX) in progress. * NVME issues on PowerPC64 Power9 blocking dual socket machine from being used as pkg builder. * Drive upgrade test for pkg builders (SSDs) courtesy of the FreeBSD Foundation. * Boot issues with Aarch64 reference machines. * New NYI.net sponsored colocation space in Chicago-land area. * Setup new host for CI staging environment. __________________________________________________________________ Continuous Integration Links FreeBSD Jenkins Instance URL: https://ci.FreeBSD.org FreeBSD CI artifact archive URL: https://artifact.ci.FreeBSD.org/ FreeBSD Jenkins wiki URL: https://wiki.freebsd.org/Jenkins freebsd-testing Mailing List URL: https://lists.FreeBSD.org/mailman/listinfo/freebsd-testing FreeBSD CI Repository URL: https://github.com/freebsd/freebsd-ci Tickets related to freebsd-testing@ URL: https://preview.tinyurl.com/y9maauwg Hosted CI wiki URL: https://wiki.freebsd.org/HostedCI FreeBSD CI weekly report URL: https://hackmd.io/@FreeBSD-CI Contact: Jenkins Admin Contact: Li-Wen Hsu The FreeBSD CI team maintains continuous integration system and related tasks for the FreeBSD project. The CI system regularly checks the committed changes can be successfully built, then performs various tests and analysis of the results. The results from build jobs are archived in an artifact server, for the further testing and debugging needs. The CI team members examine the failing builds and unstable tests, and work with the experts in that area to fix the code or adjust test infrastructure. The details are of these efforts are available in the weekly CI reports. We had a testing working group at the 201909 DevSummit lwhsu@ has presented the Testing/CI project status and "how to work with the FreeBSD CI system", slides are available at the DevSummit page. Some contents have been migrated to https://wiki.freebsd.org/Jenkins/Debug , extending is welcomed. We continue publishing CI Weekly Report and moved the archive to https://hackmd.io/@FreeBSD-CI Work in progress: * Collecting and sorting CI tasks and ideas at https://hackmd.io/bWCGgdDFTTK_FG0X7J1Vmg * Setup the CI stage environment and put the experimental jobs on it * Extending and publishing the embedded boards testbed * Implementing automatic tests on bare metal hardware * Adding drm ports building test against -CURRENT * Testing and merging pull requests at https://github.com/freebsd/freebsd-ci/pulls * Planning for running ztest and network stack tests * Help more 3rd software get CI on FreeBSD through a hosted CI solution Please see freebsd-testing@ related tickets for more WIP information. This project was sponsored by The FreeBSD Foundation. __________________________________________________________________ FreeBSD Core Team Contact: FreeBSD Core Team The FreeBSD Core Team is the governing body of FreeBSD. * Core has provisionally accepted the BSD+patent license for use in some cases. The Core Team must approve the import of new BSD+Patent licensed components or the change of license of existing components to the BSD+Patent License. https://opensource.org/licenses/BSDplusPatent * Kernel Pseudo Random Number Generator (PRNG) maintainership was updated to reduce the contribution barrier for committers who have demonstrated competence in this part of the tree. * Core approved a source commit bit for Pawel/ Biernacki. Konstantin Belousov will mentor Pawel/ and Mateusz Guzik will be co-mentor. * The Core-initiated Git Transition Working Group met over the last quarter, however a report is still forthcoming. Discussions will continue in the fourth quarter of 2019. There are many issues to resolve including how to deal with contrib/, whether to re-generate hashes in the current Git repository, and how to best implement commit testing. __________________________________________________________________ FreeBSD Foundation Contact: Deb Goodkin The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Funding comes from individual and corporate donations and is used to fund and manage software development projects, conferences and developer summits, and provide travel grants to FreeBSD contributors. The Foundation purchases and supports hardware to improve and maintain FreeBSD infrastructure and provides resources to improve security and quality assurance efforts; publishes marketing material to promote, educate, and advocate for the FreeBSD Project; facilitates collaboration between commercial vendors and FreeBSD developers; and finally, represents the FreeBSD Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity. Here are some highlights of what we did to help FreeBSD last quarter: Partnerships and Commercial User Support We help facilitate collaboration between commercial users and FreeBSD developers. We also meet with companies to discuss their needs and bring that information back to the Project. In Q3, Ed Maste and Deb Goodkin met with a few commercial users in the US. It is not only beneficial for the above, but it also helps us understand some of the applications where FreeBSD is used. We were also able to meet with a good number of commercial users at vBSDCon and EuroBSDCon. These venues provide an excellent opportunity to meet with commercial and individual users and contributors to FreeBSD. Fundraising Efforts Our work is 100% funded by your donations. We are continuing to work hard to get more commercial users to give back to help us continue our work supporting FreeBSD. More importantly, we'd like to thank our individual donors for making $10-$1,000 donations last quarter, for more than $16,000! Please consider making a donation to help us continue and increase our support for FreeBSD! We also have the Partnership Program, to provide more benefits for our larger commercial donors. Find out more information at www.FreeBSDfoundation.org/FreeBSD-foundation-partnership-program/ and share with your companies. OS Improvements The Foundation supports software development projects to improve the FreeBSD operating system through our full time technical staff, contractors, and project grant recipients. They maintain and improve critical kernel subsystems, add new features and functionality, and fix problems. Over the last quarter there were 345 commits to the FreeBSD base system repository sponsored by the FreeBSD Foundation - this represents about one fifth of all commits during this period. Many of these projects have their own entries in this quarterly report (and are not repeated here). Foundation staff member Konstantin Belousov committed many improvements to multiple kernel subsystems, as well as low-level 32-bit and 64-bit x86 infrastructure. These included fixes for robust mutexes, unionfs, the out of memory (OOM) handler, and per-cpu allocators. Additional work included fixes for security issues and introduction and maintenance of vulnerability mitigations, and improving POSIX conformance. Ed Maste committed a number of minor security bug fixes and improvements, as well as the first iteration of a tool for editing the mitigation control ELF note. Additional work included effort on build infrastructure and the tool chain. Clang's integrated assembler (IAS) is now used more widely, as part of the path to retiring the assembler from GNU binutils 2.17.50. The readelf tool now decodes some additional ELF note information. Ed also enabled the Linuxulator (Linux binary support layer) on arm64, and added a trivial implementation of the renameat2 system call (handling common options). Mark Johnston added Capsicum support to a number of ELF Tool Chain utilities, and committed a number of other Capsicum kernel and userland fixes. Mark worked on a number of changes related to security improvements, including integration and support of the Syzkaller automated system call fuzzer, and fixing issues identified by Syzkaller. Other changes included addressing failures caused by refcount wraparound, improvements to the prot_max memory protection. Other work included NUMA, locking, kernel debugging, RISC-V and arm64 kernel improvements. Edward Napierala continued working on Linuxulator improvements over the quarter. The primary focus continued to be tool improvements - strace is now more usable for diagnosing issues with Linux binaries running under the Linuxulator. That said, as with previous work a number of issues have been fixed along the way. These are generally minor issues with a large impact - for example, every binary linked against up-to-date glibc previously segfaulted on startup. This is now fixed. Continuous Integration and Quality Assurance The Foundation provides a full-time staff member who is working on improving our automated testing, continuous integration, and overall quality assurance efforts. During the third quarter of 2019, Foundation staff continued to improve the project's CI infrastructure, worked with contributors to fix the failing build and test cases, and worked with other teams in the Project for their testing needs. We added several new CI jobs and worked on getting the hardware regression testing lab ready. Li-Wen Hsu gave presentations "Testing/CI status update" and "How to work with the FreeBSD CI system" at the 201909 DevSummit. Slides are available at the DevSummit page. We continue publishing the CI weekly report on the freebsd-testing@. mailing list, and an archive is available. See the FreeBSD CI section of this report for completed work items and detailed information. Supporting FreeBSD Infrastructure The Foundation provides hardware and support to improve the FreeBSD infrastructure. Last quarter, we continued supporting FreeBSD hardware located around the world. FreeBSD Advocacy and Education A large part of our efforts are dedicated to advocating for the Project. This includes promoting work being done by others with FreeBSD; producing advocacy literature to teach people about FreeBSD and help make the path to starting using FreeBSD or contributing to the Project easier; and attending and getting other FreeBSD contributors to volunteer to run FreeBSD events, staff FreeBSD tables, and give FreeBSD presentations. The FreeBSD Foundation sponsors many conferences, events, and summits around the globe. These events can be BSD-related, open source, or technology events geared towards underrepresented groups. We support the FreeBSD-focused events to help provide a venue for sharing knowledge, to work together on projects, and to facilitate collaboration between developers and commercial users. This all helps provide a healthy ecosystem. We support the non-FreeBSD events to promote and raise awareness of FreeBSD, to increase the use of FreeBSD in different applications, and to recruit more contributors to the Project. Check out some of the advocacy and education work we did last quarter: * Sponsored USENIX 2019 Annual Technical Conference as an Industry Partner * Represented FreeBSD at OSCON 2019 in Portland, OR * Represented FreeBSD at COSCUP 2019 in Taiwan * Presented at the Open Source Summit, North American in San Diego, CA * Executive Director Deb Goodkin was interviewed by TFiR https://www.freebsdfoundation.org/news-and-events/latest-news/tfir-interview-freebsd-meets-linux-at-the-open-source-summit/ * Sponsored FreeBSD Hackathon at vBSDcon 2019 in Reston, VA * Sponsored the attendee bags and attended vBSDcon 2019 in Reston VA * Represented FreeBSD at APNIC-48 in Chiang Mai, Thailand * Represented FreeBSD at MNNOG-1 in Ulaanbaatar, Mongolia * Served as an administrator for the Project's Google Summer of Code Session. See the Google Summer of Code section of this report for more information. * Sponsored FreeBSD Developers Summit at EuroBSDCon in Lillehammer, Norway * Sponsored and attended EuroBSDcon 2019 in Lillehammer, Norway * Applied and was accepted for a FreeBSD Miniconf at linux.conf.au, in Gold Coast, Australia, Jan 14, 2020 * Our FreeBSD talk was accepted at seaGL, Seattle, WA, November 15 and 16. We continued producing FreeBSD advocacy material to help people promote FreeBSD. Learn more about our recent efforts to advocate for FreeBSD around the world: https://www.freebsdfoundation.org/blog/freebsd-around-the-world/ Our Faces of FreeBSD series is back. Check out the latest post: Roller Angel. Read more about our conference adventures in the conference recaps and trip reports in our monthly newsletters: https://www.freebsdfoundation.org/news-and-events/newsletter/ We help educate the world about FreeBSD by publishing the professionally produced FreeBSD Journal. As we mentioned previously, the FreeBSD Journal is now a free publication. Find out more and access the latest issues at https://www.FreeBSDfoundation.org/journal/. You can find out more about events we attended and upcoming events. We opened our official FreeBSD Swag Store. Get stickers, shirts, mugs and more at ShopFreeBSD. We have continued our work with a new website developer to help us improve our website. Work has begun to make it easier for community members to find information and to make the site more efficient. Legal/FreeBSD IP The Foundation owns the FreeBSD trademarks, and it is our responsibility to protect them. We also provide legal support for the core team to investigate questions that arise. Go to http://www.FreeBSDfoundation.org to find out how we support FreeBSD and how we can help you! __________________________________________________________________ FreeBSD Graphics Team status report Links Project GitHub page URL: https://github.com/FreeBSDDesktop Contact: FreeBSD Graphics Team Contact: Niclas Zeising The FreeBSD X11/Graphics team maintains the lower levels of the FreeBSD graphics stack. This includes graphics drivers, graphics libraries such as the MESA OpenGL implementation, the X.org xserver with related libraries and applications, and Wayland with related libraries and applications. During the last period, several changes have been made, but most of them has been behind the scene. We have also worked on general clean up of old xorg ports that have been deprecated upstream. The ports infrastructure for xorg ports and ports that depend on xorg ports have been updated. We have switched USE_XORG and XORG_CAT to use the USES framework, instead of the old way of including bsd.xorg.mk from bsd.port.mk. This infrastructure work has been fairly substantial, and new ports depending on xorg ports should add USES=xorg to their makefiles. As part of this bsd.xorg.mk was split up, and the XORG_CAT part was split out to USES=xorg-cat. This is used for the xorg ports themselves, and sets up a common environment for building all xorg ports. In addition, framework for pulling xorg ports directly from freedesktop.org gitlab was added, which will make improve development and testing, since it makes it possible to create ports of unreleased versions. Further improvements in this area includes framework for using meson instead of autotools for building xorg ports. This is still a work in progress. We have also worked to clean up and deprecate several old xorg ports and libraries. Some of these ports have already been removed, and some are still waiting on removal after a sufficient deprecation period. Most notably amongst the deprecations are x11/libXp, which required to fix several dependencies. Several other old libraries have also been deprecated, such as x11/Xxf86misc, x11-fonts/libXfontcache and graphics/libGLw. Some applications and drivers have also been deprecated during the period. With the remaining removals in this area, we should be up to speed with deprecations upstream. We are currently investigating if there are new software added upstream that we need to port to FreeBSD. We have also continued our regularly scheduled bi-weekly meetings. People who are interested in helping out can find us on the x11@FreeBSD.org mailing list, or on our gitter chat: https://gitter.im/FreeBSDDesktop/Lobby. We are also available in #freebsd-xorg on EFNet. We also have a team area on GitHub where our work repositories can be found: https://github.com/FreeBSDDesktop __________________________________________________________________ FreeBSD Release Engineering Team Links FreeBSD 11.3-RELEASE announcement URL: https://www.freebsd.org/releases/11.3R/announce.html FreeBSD 12.1-RELEASE schedule URL: https://www.freebsd.org/releases/12.1R/schedule.html FreeBSD 12.1-RELEASE BETA/RC builds URL: https://download.freebsd.org/ftp/releases/ISO-IMAGES/12.1/ FreeBSD development snapshots URL: https://download.freebsd.org/ftp/snapshots/ISO-IMAGES/ Contact: FreeBSD Release Engineering Team The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes and maintaining the respective branches, among other things. During the third quarter of 2019, the FreeBSD Release Engineering team finished the 11.3-RELEASE cycle, with the final release build started on July 5th and the official announcement sent on July 9th. FreeBSD 11.3-RELEASE is the fourth release from the stable/11 branch, building on the stability and reliability of 11.2-RELEASE. The FreeBSD Release Engineering Team also started work on the upcoming 12.1-RELEASE, which started September 6th. This release cycle is the first "freeze-less" release from the Subversion repository, and the test bed for eliminating the requirement of a hard code freeze on development branches. Commits to the releng/12.1 branch still require explicit approval from the Release Engineering Team, however. At present, there have been three BETA builds, and so far, two RC builds, with the final 12.1-RELEASE build scheduled for November 4th. Additionally throughout the quarter, several development snapshots builds were released for the head and stable/11 branches; snapshots for stable/12 were released as well although not during the 12.1-RELEASE cycle. Much of this work was sponsored by Rubicon Communications, LLC (Netgate) and the FreeBSD Foundation. __________________________________________________________________ FreeBSD Security Team Links FreeBSD security information URL: https://www.freebsd.org/security/ Contact: Security Team Several members of the security team met at the Vendor Summit in October to formalize team structure dedicated for architecture and crypto engineering in addition to the existing product security incident response function. Since June we have started having fortnightly conference calls to discuss important issues and to collaborate closely on advisories and errata notices in the pipeline. * Security advisories sent out in 2019-Q3: 7 * Errata Notices sent out in 2019-Q3: 5 __________________________________________________________________ Projects Projects that span multiple categories, from the kernel and userspace to the Ports Collection or external projects. FAT / msdosfs support for makefs(8) Contact: Ed Maste In order to streamline the process of creating install or virtual machine system images we needed FAT filesystem support in makefs(8). Makefs was originally developed in NetBSD, and FAT support was added there not much later, but after the tool was ported to FreeBSD. Siva Mahadevan, one of the FreeBSD Foundation's interns from the University of Waterloo, worked on porting FAT support from NetBSD. I rebased and updated Siva's work, and committed it during this quarter. After a few follow-up fixes we are able to build FAT filesystem images without using md(4) and without requiring root. This project was sponsored by The FreeBSD Foundation. __________________________________________________________________ FUSE Contact: Alan Somers FUSE (File system in USErspace) allows a userspace program to implement a file system. It is widely used to support out-of-tree file systems like NTFS, as well as for exotic pseudo file systems like sshfs. FreeBSD's fuse driver was added as a GSoC project in 2012. Since that time, it has been largely neglected. The FUSE software is buggy and out-of-date. Our implementation is about 11 years behind. I completed this work during Q3. I fixed a few newly-introduced bugs, fixed a long-standing sendfile bug that affects FUSE ([236466](https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=236466)) and merged everything to head and stable/12. Then I fixed the resulting Coverity CIDs. There have been no new FUSE-related bug reports, so I can only assume that everything is working great. Report any problems to asomers@FreeBSD.org. This project was sponsored by The FreeBSD Foundation. __________________________________________________________________ Google Summer of Code 2019 Links 2019 Summer of Code Project Wikis URL: https://wiki.freebsd.org/SummerOfCode2019Projects 2019 Summer of Code Projects URL: https://summerofcode.withgoogle.com/archive/2019/organizations/6504969929228288/ Contact: Summer of Code Admins The FreeBSD Project is pleased to have participated in Google Summer of Code 2019 marking our 14th year of participation. This year we had six successful projects: * Dual-stack ping command by Ján Sucan * Firewall test suite by Ahsan Barkati * Kernel sanitizers by Costin Carabas * MAC policy on IP addresses for FreeBSD Jail by Shivank Garg * Separation of ports build process from local installation by Theron Tarigo * Virtual memory compression by Paavo-Einari Kaipila We thank Google for the opportunity to work with these students and hope they continue to work with FreeBSD in the future. This project was sponsored by Google Summer of Code. __________________________________________________________________ GSoC'19 Project - MAC policy on IP addresses in Jail: mac_ipacl Links FreeBSD's Phabricator Differential Link URL: https://reviews.freebsd.org/D20967 Github Diff Link URL: https://github.com/freebsd/freebsd/compare/master...shivankgarg98:shivank_MACPolicyIPAddressJail Project Wiki Page URL: https://wiki.freebsd.org/SummerOfCode2019Projects/MACPolicyIPAddressJail Contact: Shivank Garg About - With the introduction of VNET(9) in FreeBSD, Jails are free to set their IP addresses. However, this privilege may need to be limited by the host as per its need for multiple security reasons. This project uses mac(9) for an access control framework to impose restrictions on FreeBSD jails according to rules defined by the root of the host using sysctl(8). It involves the development of a dynamically loadable kernel module (mac_ipacl) based on The TrustedBSD MAC Framework to implement a security policy for configuring the network stack. This project allows the root of the host to define the policy rules to limit the root of a jail to a set of IP (v4 or v6) addresses and/or subnets for a set of interfaces. Features this new MAC policy module are: * The host can define one or more lists of IP addresses/subnets for the jail to choose from. * The host can restrict the jail from setting certain IP addresses or prefixes (subnets). * The host can restrict this privilege to a few network interfaces. Implementation - The mac_ipacl module is a loadable kernel module. It implements mac checks in netinet/in.c and netinet6/in6.c to check the IP addresses requested by jail. The idea to implement these checks at these places comes from the fact that SIOCAIFADDR (for IPv4) and SIOCAIFADDR_IN6 (for IPv6) ioctl handlers are defined for adding the IP addresses to an interface. This is used by ifconfig (in userspace) for setting the IP address. The MAC Framework acts as multiplexer between the netinet and the module. The requested IP and the credentials are checked with the rules in mac_ipacl and output is returned accordingly to netinet. The module can be tuned with various sysctl and similarly, policy rules are also be defined with sysctl. TestSuite - Test scripts integrated with kyua and ATF are included with the module. Using the module - I have written a man page for the module. Please refer to the mac_ipacl(4) for using the new MAC module and various examples. Final Deliverables - * A loadable kernel module - mac_ipacl in sys/security/mac_ipacl * ATF tests for the module in tests/sys/mac/ipacl * A man page for this new mac module - mac_ipacl.4 in share/man/man4/mac_ipacl.4 This is a new project, developed as part of Google Summer of Code'19 under the guidance of Bjoern A. Zeeb . The module is reviewed and Revision for this project is accepted and ready to land. It is yet to be merged with FreeBSD HEAD, and waiting to be tested by few more hands in the industry. I'll be very thankful if you can give this module a try and share your valuable experience about it. Please be free to share your ideas and feedback on this module and please do not hesitate to send me an email. __________________________________________________________________ Improving laptop support Contact: Ed Maste The FreeBSD Foundation would like to ensure that running FreeBSD on contemporary hardware, including laptops, remains viable. To that end we plan to purchase the latest generation of one or more of a family of laptops preferred by members of the FreeBSD community, evaluate the existing state of hardware support, and implement missing hardware support where possible. As the first laptop for this project we have selected a 7th Generation Lenovo X1 Carbon. This project was sponsored by The FreeBSD Foundation. __________________________________________________________________ NFS Version 4.2 implementation Contact: Rick Macklem RFC-7862 describes a new minor revision to the NFS Version 4 protocol. This project implements this new minor revision. The NFS Version 4 Minor Version 2 protocol adds several optional features to NFS, such as support for SEEK_DATA/SEEK_HOLE, file copying done on the server that avoids data transfer over the wire and support for posix_fallocate(), posix_fadvise(). Hopefully these features can improve performance for certain applications. The implementation is now nearing completion and recent work has been mostly testing. A cycle of interoperability testing with Linux has just been completed. The main area that still needs testing is use of the pNFS server with NFSv4.2 and that should start soon. Once testing of pNFS is completed, I believe the code is ready to be incorporated into FreeBSD head/current. Testing by others would be appreciated. The modified kernel can be found at https://svn.freebsd.org/base/projects/nfsv42/sys and should run with a recent FreeBSD head/current system. Client mounts need the "minorversion=2" mount option to enable this protocol. __________________________________________________________________ Rockchip RK3399 SoC's eMMC support Contact: Ganbold Tsagaankhuu The followings features have been added to support RK3399 SoC eMMC on FreeBSD: * Extended simple_mfd driver to expose a syscon interface if that node is also compatible with syscon. For instance, Rockchip RK3399's GRF (General Register Files) is compatible with simple-mfd as well as syscon and has devices like usb2-phy, emmc-phy and pcie-phy etc. under it. * Made Rockchip's General Register Files driver the subclass of Simple MFD driver * Added driver for Rockchip RK3399 eMMC PHY. * Added eMMC support codes for Rockchip RK3399 SoC. * All above was tested on NanoPC-T4 board. __________________________________________________________________ syzkaller on FreeBSD Contact: Andrew Turner Contact: Mark Johnston Contact: Michael Tuexen Contact: Samy Al Bahra See the syzkaller entry in the 2019q1 quarterly report for an introduction to syzkaller. Work has continued on fixing bugs uncovered by syzkaller. Over a dozen kernel bugs fixed in the past three months have been directly attributed to syzkaller, and a number of syzkaller reproducers have been incorporated into our test suite. backtrace.io, via Samy, has graciously provided a large server for running a dedicated syzkaller instance to fuzz FreeBSD under bhyve. Though syzbot, the public syzkaller instance run by Google, already fuzzes FreeBSD, it has proven fruitful to run multiple syzkaller instances: different instances find different bugs, and syzkaller.backtrace.io allows us to experiment with FreeBSD-specific extensions. In particular, this instance currently uploads data about each crash to backtrace.io, making it much easier to triage and analyze crashes. We plan to make this service generally available to FreeBSD developers in the near future. Going forward we will continue to extend syzkaller coverage and make it simpler for users and developers to run private instances, and optionally collect kernel crash information for debugging or for submission. This project was sponsored by backtrace.io, and The FreeBSD Foundation. __________________________________________________________________ TPM2 Software Stack (TSS2) Links tpm2-tss Documentation URL: https://tpm2-tss.readthedocs.io/en/latest/index.html tpm2 Source Repository URL: https://github.com/tpm2-software/ tpm2 mailing list URL: https://lists.01.org/postorius/lists/tpm2.lists.01.org/ tpm2 irc channel URL: ircs://chat.freenode.net:6697/tpm2.0-tss Contact: D Scott Phillips Intel has contributed ports of the TPM2 Software Stack to the ports tree, with the new ports securtity/tpm2-tss, security/tpm2-tools, security/tpm2-abrmd. tpm2-tss contains a set of libraries which expose various TPM2 APIs for using a TPM conforming to the TCG TPM 2.0 specification. tpm2-tools provides a set of command line tools which use the tpm2-tss libraries to perform tpm operations. Finally, tpm2-abrmd is a userspace daemon which acts as a TPM Access Broker and Resource Manager, multiplexing many TPM users onto a single TPM device. Sponsored by: Intel Corporation __________________________________________________________________ Kernel Updates to kernel subsystems/features, driver support, filesystems, and more. Casueword(9) livelock Contact: Konstantin Belousov The Compare-And-Swap (CAS) is one of the fundamental building blocks for hardware-assisted atomic read/modify/write operations. Some architectures support it directly, for instance x86 and sparc. Others provide different building blocks, usually the pair of Load Linked/Store Conditional instructions (ll/sc) which can be used to construct CAS or other atomic operations like Fetch-And-Add or any atomic arithmetic ops using plain arithmetic instructions. An example is the LDXR/STXR pair on AArch64. The ll/sc operation is performed by first using the load linked instruction to load a value from memory and simultaneously mark the cache line for exclusive access. The value is then updated by the store conditional instruction, but only if there were not any writes to the marked cache line. Any write by another CPU makes the store instruction fail. So typically atomic primitives on ll/sc architectures retry the whole operation if only store failed, because it just means that this CPU either lost a race, or even the failure was spurious. This is so called strong version of CAS and atomics. If primitive returns failure instead, the CAS variant is called weak by C standard. For the FreeBSD threading library lock implementation, when the fast path mode in userspace was not possible, the kernel often needs to do a CAS operation on user memory location. In addition to all guarantees of normal CAS, it also must ensure the safety and tolerance to paging. In FreeBSD, the casueword32(9) primitive provides CAS on usermode 32bit words for kernel. Casueword32(9) was implemented as strong CAS, similarly to the mode of operation of hardware CAS instructions on x86. Using the strong implementation for casueword may be dangerous, since the same address is potentially accessible to other, potentially malicious, threads in the same or other processes. If such a thread constantly dirties the cache line used by a ll/sc loop, it could practically force the kernel-mode thread to remain stuck in the loop forever. Since the loop is tight, and it does not check for signals, the thread cannot be stopped or killed. The solution is to make the casueword implementation weak, which means that the interface of the primitive must be changed to allow notifying the caller about spurious failures. Also, it is now the caller's responsibility to retry as appropriate. The change to casueword was made for all architectures. Even on x86, the PSL.ZF value after the CMPXCHG instruction was returned directly for the new casueword. All two dozens uses of the primitive, all located in kern_umtx.c, were inspected and retry added as needed. As a somewhat related note, in-kernel atomic_cmpset(9) operations are strong, while atomic_fcmpset(9) should be weak, unless broken by a specific architecture. The general attitude seems to be that retry is the duty of the primitive's caller. This project was sponsored by The FreeBSD Foundation. __________________________________________________________________ Kernel Mapping Protections Contact: Mark Johnston Modern CPU architectures have the ability to flag memory mappings as "no-execute" (NX), which prevents that memory from being executed by a processor. NX mappings are an important security mitigation since they help segregate code and data, blocking unintentional execution of memory whose contents is controlled by an attacker. W^X (write XOR execute) is a security policy which disallows the creation of mappings that are simultaneously writeable and executable. Under this policy, memory whose contents can be modified must be mapped with the NX flag. This policy makes it harder to exploit bugs that permit an attacker to arbitrarily overwrite data. FreeBSD has long made use of the NX flag for userspace mappings whenever possible, but only in the past several years has it been applied to kernel mappings. A recent project has sought to implement a W^X-by-default policy for the amd64 kernel by completing an audit of the remaining executable mappings in the kernel, and making modifications to either apply the NX bit to those mappings or to make them read-only. This work has landed in HEAD and will be available in FreeBSD 13.0 and 12.2. Similar work for other CPU architectures is also planned. To help audit kernel mapping protections, the vm.pmap.kernel_maps sysctl was added; it dumps a snapshot of the kernel's page table entries and their attributes. This way, mappings violating the W^X policy can easily be discovered and investigated, and the sysctl provides information useful to anyone interested in kernel memory layout. With a few rare exceptions, the only kernel mappings which require execute permission are those of the kernel executable itself, and loadable kernel modules. A number of other regions of memory were unnecessarily being mapped without NX, and these were identified and corrected first. To address the kernel code mappings, the amd64 kernel linker script was modified to pad the .text segment to a 2MB boundary. To improve performance, the kernel creates superpage mappings of its .text segment, but this means that any data cohabiting the final 2MB .text mapping is mapped with execute permissions. The padding allows executable code to be segregated from data which follows in the kernel image, avoiding this problem and maintaining the superpage optimization at the expense of some wasted RAM. Enforcing W^X turned out to be somewhat trickier. Unlike other CPU architectures supported by FreeBSD, amd64 kernel modules are linked as relocatable object files, i.e., .o files. (On other architectures, they are dynamically shared objects (DSOs, or .so files), as one might naively expect.) The use of .o files means that amd64 kernel modules contain more efficient code than they would if linked as DSOs, since DSOs inherently make use of certain types of indirection which allow shared libraries to be loaded at arbitrary addresses, and this indirection is useless in the kernel. As part of this project an attempt was made to switch amd64 to using DSOs as well, since the cost of this indirection can largely be mitigated with modern toolchains, but it was found that the use of DSOs would also force a change to the code model used when compiling amd64 kernel code, resulting in a further performance penalty. The main obstacle with the use of .o files is that sections are not page-aligned by default; the segregation of sections with differing mapping protection requirements is done by the static linker when linking a DSO or executable file. Since mapping protections are applied at the granularity of the page size (4KB on amd64), the overlap of sections within a page causes problems since, for example, the end of the read-only .text section may overlap with the beginning of the read-write .data section. One possible solution is to perform any required section reordering and padding at kernel module load time, but this approach breaks debugging tools such as dtrace(1) and kgdb which assume that the kernel linker does not modify the layout of loadable modules. As a result, an amd64 kernel module linker script is now used to insert padding for specific sections. Finally, the kernel linker was modified to restrict mapping protections based on section flags. As a result of all of this, amd64 kernels now boot without any writeable, executable mappings. Though some of the work was architecture-specific, much of it can and will be leveraged to provide the same policy for our other supported architectures. This project was sponsored by Netflix. __________________________________________________________________ Kernel ZLIB Update Contact: Xin Li Contact: Yoshihiro Ota The ZLIB is a compression library is widely used in various software. The FreeBSD system had used an ancient (over 20 year-old) version of zlib (version 1.0.4) and total of 3 versions, one in user-land, one in ZFS, and one in kernel. Xin and Yoshihiro upgraded zlib to the latest and eliminated 2 extra copies. Along the efforts, zlib version was upgraded to 1.2.11, netgraph's ppp is re-implemented to use the standard zlib, and removed unmaintained code. We now only have one and the latest version of zlib in FreeBSD code base, new compress, compress2, and uncompress APIs exposed in the kernel, and importing changes from zlib will be simple. Kernel zlib changes * sys/crc32.h is split to avoid object file name conflict with ZLIB's * contrib/zlib is moved to sys/contrib/zlib * Kernel started switching to sys/contrib/zlib and ZFS copy dropped * Kernel zcalloc is introduced and compress, compress2, and uncompress APIs are exposed to kernel * Removed zlib 1.0.4 from kernel Kernel zlib user updates * kern_ctf.c updated * opencryptodeflate updated * geom_uzip updated * subr_compressor updated * if_mxge updated * bxe updated * ng_deflate updated Legacy code removals * Removed MIPS zlib elf trampoline * Removed kgzip and kgzldr * Removed gzip'ed a.out support * Removed ArmvX elf_trampoline.c __________________________________________________________________ PROT_MAX mmap/mprotect maximum protections API Links PROT_MAX commit URL: https://reviews.freebsd.org/rS349240 Contact: Brooks Davis Unix-like systems provide the mmap(2) system call to allocate memory or map files or devices into memory with specified access protection, and the mprotect(2) system call to change protections on mapped memory. Protection flags specify whether the memory may be read, written, and/or executed (PROT_READ, PROT_WRITE, PROT_EXEC respectively). Traditionally, mprotect(2) can be used to set any desired protections (except that a shared mapping of a file opened read-only cannot have PROT_WRITE set). A new macro PROT_MAX() adds a facility for specifying the maximum possible protection flags for mmap(2) and mprotect(2) calls. The program can then specify whether a mapping may be changed in the future to allow a given access protection. For example, a memory mapping can be set such that it can have read and write protections set, but may never be made executable. Maximum protection values are provided to the PROT_MAX() macro, and are OR'd with regular protection values. This change allows (e.g.) a region that must be writable during run-time linking or JIT code generation to be made permanently read+execute after writes are complete. This complements Write-XOR-Execute (W^X) protections available on other operating systems, allowing more precise control by the programmer. For example, to request memory that cannot be made executable: mmap(NULL, size, PROT_READ | PROT_WRITE | PROT_MAX(PROT_READ | PROT_WRITE), MAP_ANON, -1, 0); and to request memory that may have execute permission enabled later on, but is not currently executable: mmap(NULL, size, PROT_READ | PROT_WRITE | PROT_MAX(PROT_READ | PROT_WRITE | PROT_EXECUTE), MAP_ANON, -1, 0); This change alters mprotect argument checking and returns an error when unhandled protection flags are set. This differs from POSIX (in that POSIX only specifies an error if a valid permission can not be set), but is the documented behavior on Linux and more closely matches historical mmap behavior. In addition to explicit setting of the maximum permissions, an experimental sysctl vm.imply_prot_max causes mmap to assume that the initial permissions requested should be the maximum when the sysctl is set to 1. This behavior is known to break code that uses PROT_NONE reservations before mapping contents into part of the reservation. A later version of this work is expected to provide per-binary and per-process opt-in/out options and this sysctl may be removed in its current form. As such it is undocumented. While these flags are non-portable, they can be used in portable code with simple ifdefs to expand PROT_MAX() to 0. Sponsors: DARPA, AFRL __________________________________________________________________ Randomized Top of Stack pointer Contact: Konstantin Belousov After the ASLR so useful addition, next in the series of the buzzword-compliant checkboxes is the stack addresses randomization. The main userspace thread stack on FreeBSD is traditionally allocated from the top of the user address space and grows down. Besides the initial pointer for the stack on userspace entry, this area also contains structures for program arguments and environment (so called strings), and aux vector data for ELF images. Considering the goal of randomizing the addresses of strings and main thread initial frame, moving the main stack area in the address space is not feasible. It would cause significant ABI breakage, invalidates the knowledge already coded into many introspection tools, and causes unneeded additional fragmentation of the user address space. Instead a typical approach of adding a gap of randomized size between top of stack and a place for strings was used. It is done in a way which preserves the stack alignment requirement. Stack gap is only enabled if ASLR is enabled (not default) and stack gap itself is enabled (default if ASLR is enabled). Stack gap is specified in percentage of the total stack size that can be used for maximum gap. The main drawback of the gap approach was shortly identified. Since gap is cut from the normal stack area, attempts of the programs to reduce stack size using rlimit(RLIMIT_STACK) could cut the active stack region if new limit happens to be smaller than the gap. E.g. on amd64 with its default 512M main thread stack, even one percent of the max gap gives approximately 5M of unused stack, that can blow up the limit of several KBs. Typical reason for using rlimit(2) this way is for programs that wire all of its address space with mlockall(2), trying to reduce potential wired stack size to avoid exceeding RLIMIT_MEMLOCK. First victim of that issue was ntpd, which resets the stack limit after start for a really small value. Currently the wiring was removed from ntpd, because apparently it does not make the timekeeping better by any means, contrary to popular belief. My opinion is that the problem is more in the user interface area than in the gap approach itself. We should make it easy to specify small gap sizes, which cannot be done with integral percentage interface. So far I did not formulated a way to do this which I would like, and since nobody looked for a good solution because after ntpd was fixed, the severity of the issue seems very low. This project was sponsored by The FreeBSD Foundation. __________________________________________________________________ Signals delivered on unhandled Page Faults Contact: Konstantin Belousov By necessity, handling of page faults is separated into two pieces. The first is the architecture-dependent low-level machine exception handler, and the second is the architecture-independent vm_fault() function in sys/vm/vm_fault.c. Machine-dependent code for page faults typically consists of three components: a trampoline written in assembly which creates the C execution environment and gathers hardware-supplied data about page fault reason, a trap() function which is common C-level entry point to dispatch all exceptions processing, and the trap_pfault() C function to specifically handle page faults. trap_pfault() calls vm_fault() when help from VM is needed to resolve the situation. If the fault was handled either by trap()/trap_pfault() or vm_fault(), the faulting instruction is restarted. If fault cannot be handled for any reason, the next action depends on the mode in which the fault occured. If it was in kernel, and the kernel installed a helper, then the helper is called instead of returning to the faulting instruction. Otherwise, a kernel-mode page fault causes a panic. If the unhandled fault occured in usermode, then all Unixes send a signal to the thread whose execution caused the exception. POSIX (or Single Unix Specification) lists several cases where a signal should be sent, and specifies the signal number and si_code from siginfo that must be supplied with the signal. Unfortunately, FreeBSD was rather non-compliant in this regard. A long time ago, to improve compliance, we changed the signal sent on access to a page with permissions incompatible with the access mode. That caused multiple problems with garbage collection (GC)-based runtimes which incorporated knowledge of the FreeBSD quirks, so the machdep.prot_fault_translation sysctl knob was added. More cases of incompatibility were identified recently. Part of the problem is that code to calculate the signal and si_code from the Mach error returned by vm_fault() was located in trap_pfault(). In other words, each architecture did that on its own, with a specific set of bugs and non-compliance. Sometimes code even mis-interpreted returned Mach errors as errno. A new helper function vm_fault_trap() was added, that does several things for trap handlers: it creates ktrace points for faults, calls vm_fault(), and then interprets the result in terms of the user-visible error condition, returning precalculated signal number and si_code to the caller. Now trap_pfault() only needs to provide signal numbers for truly machine-dependent errors. For amd64, an example of such a case is a protection key violation. Besides compliance and bug fixes, we also provided some refinements to userspace about the reason of the delivered signal. For instance, on SIGBUS caused by copy-on-write failure due to exceeding swap reservation limit, we provide specific si_code BUS_OOMERR. This project was sponsored by The FreeBSD Foundation. __________________________________________________________________ Architectures Updating platform-specific features and bringing in support for new hardware platforms. Broadcom ARM64 SoC support Contact: Michal Stanek Contact: Kornel Duleba Contact: Marcin Wojtas The Semihalf team continued working on FreeBSD support for the Broadcom BCM5871X SoC series BCM5871X are quad-core 64-bit ARMv8 Cortex-A57 communication processors targeted for networking applications such as 10G routers, gateways, control plane processing and NAS. Completed since the last update: * iProc PCIe root complex (internal and external buses): fixes and improvements, * Crypto engine acceleration for IPsec offloading. Todo: * Upstreaming of work. This work is expected to be submitted/merged to HEAD in the Q4 of 2019. This project was sponsored by Juniper Networks, Inc. __________________________________________________________________ FreeBSD support for the forthcoming Arm Morello CPU, SoC, and board Contact: Robert Watson Contact: Andrew Turner Contact: Brooks Davis In September 2019, Arm announced Morello, an experimental multicore superscalar ARMv8-A CPU, SoC, and prototype board extended to implement the CHERI protection model. Morello will become available in 2021. More details can be found in Arm's blog, a Light Blue Touchpaper blog, and the main CHERI website. We have updated CheriBSD, a CHERI-adapted version of FreeBSD originally targeted at the MIPS-based SRI/Cambridge CHERI processor prototype, to support the current draft architecture. This includes full userspace spatial and referential memory safety CheriABI, as well as backwards compatibility to support running existing ARMv8-A userspace binaries. We will continue to update CheriBSD/Morello as the ISA is finalised. Will also closely track the public CheriBSD/MIPS trunk, picking up new software features utilizing CHERI as they mature, as well as to pick up changes from the baseline FreeBSD development trunk. We currently anticipate releasing CheriBSD/Morello as open source once the ISA and toolchain are published in 2020. Sponsors: DARPA, AFRL __________________________________________________________________ FreeBSD/powerpc Project Links Status of FreeBSD ports on PowerPC URL: https://wiki.freebsd.org/powerpc/ports Status of FreeBSD ports on PowerPC built using gcc URL: https://wiki.freebsd.org/powerpc/ports/PortsOnGcc Status of FreeBSD ports on PowerPC built using clang URL: https://wiki.freebsd.org/powerpc/ports/PortsOnClang Bringing LLVM to PowerPC64 target, using OpenPower ELF v2 ABI URL: https://wiki.freebsd.org/powerpc/llvm-elfv2 Contact: Mark Linimon (ports) Contact: Justin Hibbits (src) Contact: Piotr Kubaj (ports) The FreeBSD/powerpc project continues to make great strides. However, as we discovered at BSDCan 2019, we have not done a great job of letting people know. Key points: * powerpc64 src on recent hardware has been production-quality for over a year now. * powerpc64 ports has been developer-quality for over 18 months now. The main targeted platforms: * powerpc64 on IBM POWER8 and POWER9 chips (the most recent available). This is the primary focus going forward. FreeBSD 12 is required; FreeBSD 13 is recommended. * powerpc64 on older Apple Power Macs, on a continuing but secondary basis. Any FreeBSD version should work. * powerpc64 on x5000. However, this is still developer-only quality. FreeBSD 13 is recommended. * powerpcspe on Amiga A1222. However, this is still developer-only quality. FreeBSD 13 is recommended. The software status: * powerpc*/12 and earlier still remain on the antiquated gcc4.2.1 in base. * powerpc*/13 will soon be switched to llvm90 in base. A great deal of work has been undertaken to ensure as few regressions as possible. Once that switch has occurred (see llvm-elfv2 link above), powerpc*/13 support on gcc4.2.1 will no longer be a priority. * FreeBSD.org package sets are available for powerpc64/12 (quarterly) and powerpc64/13 (default) through the usual method. * Firefox works on powerpc64 using experimental (not-yet committed) patches, * As of the most recent package build on powerpc64/13 (still gcc4.2.1), the following statistics apply: Queued Built Failed Skipped Ignored 33306 30514 245 1686 861 * More ports fixes are being committed every day. Also, Piotr would like to thank the FreeBSD Foundation for funding his personal Talos, and Raptor (via its IntegriCloud subsidiary) for loaning a server on which talos.anongoth.pl runs. The team would like to thank IBM for the loan of two POWER8 machines, and Oregon State University (OSU) for providing the hosting. As well, we would like to thank the clusteradm team for keeping the Tyan POWER8 machines online that are hosted at NYI. __________________________________________________________________ NXP ARM64 SoC support Contact: Marcin Wojtas Contact: Artur Rojek The Semihalf team initiated working on FreeBSD support for the NXP LS1046A SoC LS1046A are quad-core 64-bit ARMv8 Cortex-A72 processors with integrated packet processing acceleration and high speed peripherals including 10 Gb Ethernet, PCIe 3.0, SATA 3.0 and USB 3.0 for a wide range of networking, storage, security and industrial applications. Completed since the last update: * DPAA Network interface support * SD/MMC * USB3.0 * I2C * GPIO In progress: * QSPI * Network performance improvements Todo: * Upstreaming of developed features. This work is expected to be submitted/merged to HEAD in the Q4 of 2019. This project was sponsored by Alstom Group. __________________________________________________________________ Userland Programs Changes affecting the base system and programs in it. gets(3) retirement Contact: Ed Maste gets is an obsolete C library routine for reading a string from standard input. It was removed from the C standard as of C11 because there was no way to use it safely. Prompted by a comment during Paul Vixie's talk at vBSDCon 2017 I started investigating what it would take to remove gets from libc. The patch was posted to Phabricator and refined several times, and the portmgr team performed several exp-runs to identify ports broken by the removal. Symbol versioning is used to preserve binary compatibility for existing software that uses gets. The change was committed in September, and will be in FreeBSD 13.0. This project was sponsored by The FreeBSD Foundation. __________________________________________________________________ Ports Changes affecting the Ports Collection, whether sweeping changes that touch most of the tree, or individual ports themselves. FreshPorts Links FreshPorts URL: https://www.FreshPorts.org/ git_proc_commit code URL: https://github.com/FreshPorts/git_proc_commit Things you didn't know FreshPorts can do URL: https://news.freshports.org/2019/09/03/things-you-didnt-know-freshports-can-do/ Contact: Dan Langille FreshPorts consolidates commits into an easy-to-follow format so you can track changes to your favorite ports. It also processes src, doc, and www commit. FreshPorts parses incoming emails and refreshes the database with what it finds. In early September I started looking at how FreshPorts could use a git repository for processing commits. The result was an approach for identifying new commits and for iterating through them. The next idea was commit hooks but the most interesting bit of that exercise was commit iteration. At the EuroBSDCon 2019 FreeBSD Developer summit, I wrote up a small requirements section and then received great help from two sources: * Jan-Piet MENS recommended a Python library and it turned out to be great * Sergey Kozlov wrote Python code to create xml using that Python library On the trip home, I was able to get the code to parse a git commit into XML and loaded into a FreshPorts database. Returning home from the conference, I created a new FreshPorts instance for processing git based on the above. The website has needed no changes related to git. The remaining tasks: * automate the script (git pull, etc) * detect new commits (for later iteration) * design a light-weight git hook Event: EuroBSDCon 2019 Hackathon Sponsored by: Intel Corporation (work done by Sergey Kozlov) __________________________________________________________________ Java on FreeBSD Links OpenJDK 11 repository at FreeBSD GitHub URL: https://github.com/freebsd/openjdk-jdk11u Contact: Greg Lewis Over the last few quarters there has been considerable work in improving support for Java 11 and higher, with some work being backported to Java 8. Starting with the initial release in March on amd64, over the intervening months FreeBSD support was added for features such as: * Java Flight Recorder * HotSpot Serviceability Agent * HotSpot Debugger * DTrace * Javac Server * Java Sound * SCTP In addition, support for the aarch64, i386 and powerpc64 architectures was also added. With most features supported, attention turned to compliance, using the internal JDK test suite as a method of measuring this. Most work during the third quarter has focused on this, with test failures dropping from 50+ failures to only 2 tier1 test failures (which don't appear to impact functionality at all). Some significant fixes include: * A stack overflow crash * Errors in external process handling * The unpack200 utility (on little endian systems) * HotSpot Debugger functionality such as thread enumeration * Networking functionality Finally, this work has been forward ported to Java 12 and 13, with FreeBSD gaining support for these versions on or just after the day of release. Note that this work has been a collaboration with many others. While there are too many contributors to list here (please take a look at the commit history of the GitHub repository), I'd like to recognise Kurt Miller of OpenBSD for his tireless work as a co-collaborator on Java for BSD through many years. I'm also grateful to the sponsorship of the FreeBSD Foundation, which has allowed me to focus on this work for Q3. This project was sponsored by FreeBSD Foundation. __________________________________________________________________ KDE on FreeBSD Links KDE FreeBSD URL: https://freebsd.kde.org/ KDE Community FreeBSD URL: https://community.kde.org/FreeBSD Contact: Adriaan de Groot The KDE on FreeBSD project packages the software produced by the KDE Community for FreeBSD. The software includes a full desktop environment, the art application https://kdenlive.org and hundreds of other applications that can be used on any FreeBSD desktop machine. Along with KDE itself, the team maintains the Qt libraries, the CMake build system, and a handful of other C++ libraries used in the KDE stack. The upstream releases KDE Frameworks (libraries) on a monthly cycle, KDE Plasma (the desktop environment) quarterly and a collection of KDE Applications (usable everywhere) twice a year. The KDE on FreeBSD team chased a dozen updates to these components so that FreeBSD remains one of the most up-to-date systems with KDE software (and Qt). A large (and possibly breaking, still needs more investigation) change came with the release to KDE's Digikam 6.3.0, which stopped using its previous plugins system (the "old" plugins are still used by other KDE software). A new entry in the net-im category was added for Ruqola, a Rocket.chat client for rich instant-messaging. CMake was updated twice. This forces the rebuild of several thousand C++-based ports. The KDE on FreeBSD team then fixes those ports, regardless of whether the error is in the CMake update, or in the upstream code. These updates tend to take a large amount of effort, since they touch unfamiliar (and often very-very-legacy) code. The open bugs list grew to 24 this quarter. It tends to hover around 20 items as new things come in and old ones are resolved. We welcome detailed bug reports and patches. KDE packaging updates are prepared in a copy of the ports repository on GitHub and then merged in SVN. We welcome pull requests there as well. __________________________________________________________________ Ports Collection Links About FreeBSD Ports URL: https://www.FreeBSD.org/ports/ Contributing to Ports URL: https://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing/ports-contributing.html FreeBSD Ports Monitoring URL: http://portsmon.freebsd.org/index.html Ports Management Team URL: https://www.freebsd.org/portmgr/index.html Contact: René Ladan Contact: FreeBSD Ports Management Team The FreeBSD Ports Management Team, tasked with overseeing the Ports Tree and its committers, reports that the following events happened during 2019Q3: The number of ports grew to just under 38,000 during the last quarter. We have just over 2,000 open ports PRs, of which 400 are unassigned. In this period, 169 committers made 7,340 commits to HEAD and 561 commits to the quarterly branch. This shows that the trend of last quarter of increased activity is continuing. During the last quarter, we welcomed Santhosh Raju (fox@) and Dmitri Goutnik (dmgk@) to the team, and said goodbye to gabor@. During the last quarter, feld@ decided to step down from the portmgr@ and ports-secteam@ teams. We would like to thank them for their past services. In the last three months, bapt@ put on his engineering hat and released a new version of pkg (1.12), added support for overlays to the Ports Tree, fixed two Make targets (deinstall-depends and reinstall), and cleaned up bsd.sites.mk. On the infrastructure side, USES=pure became obsolete and has therefore been removed, and two new USES, xorg and xorg-cat have been added and replace the old bsd.xorg.mk. Two new keywords, ldconfig and ldconfig-linux, were added to simplify formatting of package lists. A number of default versions changed: Lazarus to 2.0.4, Linux to CentOS 7, LLVM to 9.0, Perl to 5.30, PostgreSQL to 11, and Ruby to 2.6. Of the big user-visible ports, Firefox got updated to 69.0.1, Firefox-esr to 68.1.0, Chromium to 76.0.3809.132, and Xfce to 4.14. During the last quarter, antoine@ ran 48 exp-runs to test package updates, test updating bsd.java.mk, test the new ldconfig and ldconfig-linux keywords, test default version updates, test the new xorg and xorg-cat USES, test base updates, and test various improvements to Go and Ruby. __________________________________________________________________ XFCE 4.14 update Links XFCE 4.14 announce URL: https://xfce.org/about/news/?post=1565568000 Contact: Guido Falsi On September 19th the XFCE desktop environment ports have been updated to the recently released XFCE 4.14 version. These updates include upgrades of all the main XFCE components to the latest versions which have been migrated to GTK3, with few exceptions. Base components still require and link to GTK2 in addition to GTK3 to allow older GTK2 XFCE applications, for example panel plugins, to keep working. Due to this change the gtk-xfce-engine theme is deprecated since it only supports GTK2. The XFCE metaport by default installs the greybird theme, but it is not enabled by default. This new version also includes now it's own xfce4-screensaver program which is installed by default. Finally the default Display Manager on which XFCE depends has been changed to LightDM. Some regressions were reported in bugzilla. In particular the one affecting most users is a regression in the window manager: on specific graphic display hardware the window manager fails to properly draw window decorations, which appear black and blocky on affected systems. This problem has also been reported in the upstream bug tracker and a solution is being sought. If anyone is experiencing this display glitch and is able to test, please contact xfce@freebsd.org to help trying to figure out a solution. __________________________________________________________________ Third-Party Projects Many projects build upon FreeBSD or incorporate components of FreeBSD into their project. As these projects may be of interest to the broader FreeBSD community, we sometimes include brief updates submitted by these projects in our quarterly report. The FreeBSD project makes no representation as to the accuracy or veracity of any claims in these submissions. ClonOS: virtualization platform on top of FreeBSD Operating System Links ClonOS 19.09 URL: https://clonos.tekroutine.com/download.html CBSD URL: https://www.bsdstore.ru/ Contact: Oleg Ginzburg What is ClonOS? ClonOS is a turnkey open-source platform based on FreeBSD and the CBSD framework. ClonOS offers a complete web UI for an easy control, deployment and management of FreeBSD jails containers and bhyve/Xen hypervisor virtual environments. ClonOS is currently the only available platforms which allow both Xen and bhyve hypervisors to coexist on the same host. Since ClonOS is a FreeBSD-based platform, it has the ability to create and manage jails natively, allowing you to run FreeBSD applications without losing performance. ClonOS/CBSD 2019Q3 Status Report * Added support for cloud-init (Linux/BSD VMs) and cloudbase-init (Windows VMs). It gives the ability to use FreeBSD as IaaS platform for instant deployment and usage of virtual machines based on bhyve hypervisor. * Project started to use own cloud images for better stability and resiliency. * New mirrors in France, US and Canada were added for distributing ISO/cloud-init images in addition to Russia, Latvia and Ukraine. * Now we're using Jenkins CI for testing regular ClonOS builds: Update clonos packages (Thanks to Daniel Shafer) * New pkg repository was deployed to support ClonOS installation from packages (at this moment only FreeBSD-12 packages are available) ClonOS package repo (Thanks to Daniel Shafer) Open issues and tasks: * Volunteers, contributors and testers are urgently needed to distribute ClonOS on FreeBSD environments. * We'd like to expand our mirrors number geographically, your help and contribution will be much appriciated. * We're urgently looking for hosting sponsorship for various developing and testing activities. __________________________________________________________________ ENA FreeBSD Driver Update Links ENA README URL: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/ena/README Contact: Michal Krawczyk Contact: Maciej Bielski Contact: Marcin Wojtas ENA (Elastic Network Adapter) is the smart NIC available in the virtualized environment of Amazon Web Services (AWS). The ENA driver supports multiple transmit and receive queues and can handle up to 100 Gb/s of network traffic, depending on the instance type on which it is used. ENAv2 has been under development for FreeBSD, similar to Linux and DPDK. Since the last update internal review and improvements of the patches were done, followed by validation on various AWS instances. Completed since the last update: * Verification and review of the NETMAP support * Mapping of the memory as WC on A1 instances in order to enable LLQ mode Todo: * Upstream of NETMAP support * Upstream of the fix for LLQ mode on A1 instances Amazon.com Inc __________________________________________________________________ Nomad pot driver - Orchestrating jails via nomad Links Nomad pot driver URL: https://github.com/trivago/nomad-pot-driver Pot project URL: https://github.com/pizzamig/pot Contact: Luca Pizzamiglio Contact: Esteban Barrios An experimental project has started to provide jail orchestration based on nomad and the jail framework pot, similarly to how orchestration works with docker. This model allows us to split the jail creation and the jail deployment. Jail images can be created and exported using the pot framework. The images can be deployed and orchestrated using nomad. A driver has been developed to allow nomad to interact with pot. One of the goals of this project is to use non-persistent jails as containers, allowing us to: * define containers similar to Docker (but not identical, because the underlaying OS is different) * identify potential missing features in FreeBSD to support such a computational model In the next quarter, we will launch the first service based on this project. Next steps are: * provide more guides and howtos * improve stability, extending the tests suite * improving tooling to create/manage images This project was sponsored by trivago N.V.. __________________________________________________________________ sysctlinfo Links gitlab.com/alfix/sysctlinfo URL: https://gitlab.com/alfix/sysctlinfo Contact: Alfonso Sabato Siciliano The FreeBSD kernel maintains a Management Information Base (MIB) where a component (object) represents a parameter of the system. The sysctl system call explores the MIB to find an object by its Object Identifier (OID) and calls its handler to get or set the value of the parameter. It is often necessary to find an object not to call its handler but to get its info (description, type, name, next object, etc.), so the kernel provides an undocumented interface implemented in kern_sysctl.c. sysctlinfo is a new interface to explore the sysctl MIB and to pass the info of an object to the userland. The project provides: a README, a manual, helper macros, examples, tests and converted tools. Primarily sysctlinfo provides a new set of sysctl nodes (corresponding to the kernel interface) to handle an object up to CTL_MAXNAME levels: sysctl.entryfakename, sysctl.entrydesc, sysctl.entrylabel, sysctl.entrykind, sysctl.entryfmt and sysctl.entrynextleaf. Moreover new features have been implemented: the support for the capability mode, sysctl.entryname, sysctl.entryidbyname and sysctl.entrynextnode. To get all the info about an object the kernel needs to find it many times, then the new sysctl.entryallinfo* nodes were written, they are 30% more efficient. Finally, *byname nodes were added: they search the object by its name avoiding to call sysctl.name2oid (or similar) to explore the MIB just to find the corresponding OID. sysctlinfo can be installed via sysutils/sysctlinfo-kmod or by applying the sysctlinfo.diff patch (more efficient than the module because uses a shared lock). The interface is used by deskutils/sysctlview 1.5, sysutils/nsysctl 1.2 and the converted version of sysctl(8), sysctlbyname(3), sysctlnametomib(3), they should be used to get the value of an object with 23/24 levels or if some level-name has only the '\0' character. In the future a new byname node will be added to allow sysctlbyname() to manage a CTLTYPE_NODE with a no-NULL handler, example sysctlbyname("kern.proc.pid.\"). __________________________________________________________________ From owner-freebsd-stable@freebsd.org Thu Nov 28 06:50:27 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A53001C69B1 for ; Thu, 28 Nov 2019 06:50:27 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47NpDf3pp5z48hW for ; Thu, 28 Nov 2019 06:50:26 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id xAS6kJwC063327 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 28 Nov 2019 06:46:21 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id xAS6kGPX033204 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT) for ; Thu, 28 Nov 2019 13:46:16 +0700 (+07) (envelope-from eugen@grosbein.net) To: FreeBSD stable From: Eugene Grosbein Subject: Slow zfs destroy Message-ID: <01ca3a41-6cd2-823f-a8c1-30a47a0bbc98@grosbein.net> Date: Thu, 28 Nov 2019 13:46:07 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit X-Spam-Flag: YES X-Spam-Status: Yes, score=6.0 required=5.0 tests=BAYES_00, DATE_IN_FUTURE_96_Q, HELO_MISC_IP,LOCAL_FROM,RDNS_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.7 DATE_IN_FUTURE_96_Q Date: is 4 days to 4 months after Received: * date * -0.0 SPF_PASS SPF: sender matches SPF record * 1.9 RDNS_NONE Delivered to internal network by a host with no rDNS * 2.6 LOCAL_FROM From my domains * 1.1 HELO_MISC_IP Looking for more Dynamic IP Relays X-Spam-Level: ***** X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on eg.sd.rdtc.ru X-Rspamd-Queue-Id: 47NpDf3pp5z48hW X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [1.29 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.959,0]; SPAM_FLAG(5.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.98)[-0.985,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[grosbein.net]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.66)[ip: (-4.39), ipnet: 2a01:4f8::/29(-2.33), asn: 24940(-1.57), country: DE(-0.01)]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Nov 2019 06:50:27 -0000 Hi! Is it normal that "zfs destroy" for one ZVOL with attribute "used" equal to 2112939808 bytes (~2GB) takes over two minutes waiting on "tx_sync_done_cv"? The pool is RAID1 over five SSDs encrypted with GELI having ZIL and Cache on distinct unencrypted SSD. 11.3-STABLE/amd64 r354667. System has 360G RAM and vfs.zfs.arc_max=160g. From owner-freebsd-stable@freebsd.org Thu Nov 28 07:04:03 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6CE901C71DB for ; Thu, 28 Nov 2019 07:04:03 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47NpXL4b27z4B5B for ; Thu, 28 Nov 2019 07:04:02 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id xAS73bJC063457 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 28 Nov 2019 07:03:38 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id xAS73ZRP033586 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT) for ; Thu, 28 Nov 2019 14:03:35 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: Slow zfs destroy To: FreeBSD stable References: <01ca3a41-6cd2-823f-a8c1-30a47a0bbc98@grosbein.net> From: Eugene Grosbein Message-ID: Date: Thu, 28 Nov 2019 14:03:26 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <01ca3a41-6cd2-823f-a8c1-30a47a0bbc98@grosbein.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 47NpXL4b27z4B5B X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.77 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[grosbein.net]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.67)[ip: (-4.41), ipnet: 2a01:4f8::/29(-2.33), asn: 24940(-1.57), country: DE(-0.01)]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Nov 2019 07:04:03 -0000 28.11.2019 13:46, Eugene Grosbein wrote: > Hi! > > Is it normal that "zfs destroy" for one ZVOL with attribute "used" equal to 2112939808 bytes (~2GB) > takes over two minutes waiting on "tx_sync_done_cv"? The pool is RAID1 over five SSDs encrypted with GELI The pool is *RAIDZ1* > having ZIL and Cache on distinct unencrypted SSD. > > 11.3-STABLE/amd64 r354667. System has 360G RAM and vfs.zfs.arc_max=160g. From owner-freebsd-stable@freebsd.org Thu Nov 28 07:26:55 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CE4AC1C7B07 for ; Thu, 28 Nov 2019 07:26:55 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47Nq2l19Plz4C4d for ; Thu, 28 Nov 2019 07:26:54 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: by mail-io1-xd30.google.com with SMTP id u24so26116981iob.5 for ; Wed, 27 Nov 2019 23:26:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=U9iUccDpm4uCN5S/lr27t6uxqf8POF6Mhywu4/aEXSU=; b=CIUvymTvujVgYnWILjlS2k9w4R2z6mXHoybO932OkMsHxOYbKKn5hDXzK9ZCmkT0R5 k47cZvkS7AlIxv85NNhmXaMMybyGsCi7PPz/BKGZOHIi8AftArnMzMtpH+UYiBTZf+za AT1274PzCKGR1+U5NIaBgDjb8046PGaCdmr1D2cAnCn5cs3S9J1j7kekoyDfYmTEeQcN PwRQhsMJ6ObWv1m7REhHUicm0f9/uqI5K0itB+jCfqqQCxL4y3Kbc0727DjdN+j6gMWW I9R4RlcmcFowBNLpZXtn7zrleZLQEE61KhF9u4g99blhuQYJnmqteoaZh4zmXnZaNQ44 x71Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=U9iUccDpm4uCN5S/lr27t6uxqf8POF6Mhywu4/aEXSU=; b=HwzWH2Oovg7ma+4Cg9gV522AfIpj86aS+KKFAOv9k+dXbkDw0TYAtGOXKm/ecax+i+ tPWYjoa5w1EFEUYKFOCnLWU9AD0u2rSioVvLbhLMskLn/rGBSBYHjJ5Amg4abXEdfes3 hiEK27NtiCDNfn2Pvr44V8azZSpWeUlcdcmES1ingtK1rENbetIInU7xkrU66I5hodq5 bnOTCqldPPtU5KOSLyvpmopplY+PIe4k4mkWERn+6e5mvvcL39gZ7OLn6N8ka1VivHxF mSwH3nDumdFT6hQ2Jd9orp7n46m22UfmGA5ecltukUVJaIIQoOzEp5Sa1eTd3nW3Qv/2 geeQ== X-Gm-Message-State: APjAAAWz84D766h2yLv765FFp5GwiGDCfOeb9rkioYHxEzQ1XVSmQ8TI OhAkdsyuZfHY1sb5uVzjeSgSxZKzkWut79iUhRcA88FQ X-Google-Smtp-Source: APXvYqxd3I/bhVb4frL0ubI6dmVjWLQPdh3BdbewkzSdEcXY2i+1S4j+pVLM60DpjJ7nuuDb2addzEFbdsRIizIx6/c= X-Received: by 2002:a05:6602:2246:: with SMTP id o6mr17676808ioo.225.1574926013609; Wed, 27 Nov 2019 23:26:53 -0800 (PST) MIME-Version: 1.0 References: <01ca3a41-6cd2-823f-a8c1-30a47a0bbc98@grosbein.net> In-Reply-To: <01ca3a41-6cd2-823f-a8c1-30a47a0bbc98@grosbein.net> From: Steven Hartland Date: Thu, 28 Nov 2019 07:26:42 +0000 Message-ID: Subject: Re: Slow zfs destroy To: Eugene Grosbein Cc: FreeBSD stable X-Rspamd-Queue-Id: 47Nq2l19Plz4C4d X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=multiplay-co-uk.20150623.gappssmtp.com header.s=20150623 header.b=CIUvymTv; dmarc=pass (policy=none) header.from=multiplay.co.uk; spf=pass (mx1.freebsd.org: domain of steven@multiplay.co.uk designates 2607:f8b0:4864:20::d30 as permitted sender) smtp.mailfrom=steven@multiplay.co.uk X-Spamd-Result: default: False [-3.75 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[multiplay-co-uk.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; URI_COUNT_ODD(1.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[multiplay-co-uk.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[0.3.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[multiplay.co.uk,none]; FORGED_SENDER(0.30)[killing@multiplay.co.uk,steven@multiplay.co.uk]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.05)[ip: (-5.99), ipnet: 2607:f8b0::/32(-2.26), asn: 15169(-1.95), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[killing@multiplay.co.uk,steven@multiplay.co.uk]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Nov 2019 07:26:55 -0000 As you mentioned it=E2=80=99s on SSD you could be suffering from poor TRIM performance from your devices if you run gstat -pd you=E2=80=99ll be able t= o get an indication if this is the case. On Thu, 28 Nov 2019 at 06:50, Eugene Grosbein wrote: > Hi! > > Is it normal that "zfs destroy" for one ZVOL with attribute "used" equal > to 2112939808 bytes (~2GB) > takes over two minutes waiting on "tx_sync_done_cv"? The pool is RAID1 > over five SSDs encrypted with GELI > having ZIL and Cache on distinct unencrypted SSD. > > 11.3-STABLE/amd64 r354667. System has 360G RAM and vfs.zfs.arc_max=3D160g= . > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Thu Nov 28 09:38:43 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E1A921CAAD7 for ; Thu, 28 Nov 2019 09:38:43 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (constantine.ingresso.co.uk [31.24.6.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47Nsyq01pgz4KGH for ; Thu, 28 Nov 2019 09:38:42 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from [2001:470:6cc4:1:a1:86fc:9caa:dac5] (helo=foula.drayhouse.twisted.org.uk) by constantine.ingresso.co.uk with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.92.3 (FreeBSD)) (envelope-from ) id 1iaGFv-000LvJ-Cg for freebsd-stable@freebsd.org; Thu, 28 Nov 2019 09:38:35 +0000 Subject: Re: Slow zfs destroy To: freebsd-stable@freebsd.org References: <01ca3a41-6cd2-823f-a8c1-30a47a0bbc98@grosbein.net> From: Pete French Message-ID: Date: Thu, 28 Nov 2019 09:38:42 +0000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:60.0) Gecko/20100101 Thunderbird/60.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 47Nsyq01pgz4KGH X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=ingresso.co.uk; spf=pass (mx1.freebsd.org: domain of petefrench@ingresso.co.uk designates 31.24.6.74 as permitted sender) smtp.mailfrom=petefrench@ingresso.co.uk X-Spamd-Result: default: False [-6.11 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:31.24.6.74]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-3.31)[ip: (-9.81), ipnet: 31.24.0.0/21(-4.90), asn: 16082(-1.74), country: GB(-0.08)]; DMARC_POLICY_ALLOW(-0.50)[ingresso.co.uk,none]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16082, ipnet:31.24.0.0/21, country:GB]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Nov 2019 09:38:43 -0000 On 28/Nov/2019 07:03, Eugene Grosbein wrote: > 28.11.2019 13:46, Eugene Grosbein wrote: > >> Hi! >> >> Is it normal that "zfs destroy" for one ZVOL with attribute "used" equal to 2112939808 bytes (~2GB) >> takes over two minutes waiting on "tx_sync_done_cv"? The pool is RAID1 over five SSDs encrypted with GELI > > The pool is *RAIDZ1* Not normal, no. Having said that I have seen zfs destory take a very lng time on occasion, but for a 2GB volume then thats surprisng. Is there actual disc activity during this time ? I've been using 12 a while now, and I don't recall seeing this recently, so maybe its soemthing which has been improved on over 11. -pete. From owner-freebsd-stable@freebsd.org Thu Nov 28 09:57:14 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BB6651CB630 for ; Thu, 28 Nov 2019 09:57:14 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47NtN91TTYz4LpT for ; Thu, 28 Nov 2019 09:57:12 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id xAS9ujsM064537 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 28 Nov 2019 09:56:47 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: petefrench@ingresso.co.uk Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id xAS9uZwf035986 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 28 Nov 2019 16:56:35 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: Slow zfs destroy To: Pete French , freebsd-stable@freebsd.org References: <01ca3a41-6cd2-823f-a8c1-30a47a0bbc98@grosbein.net> From: Eugene Grosbein Message-ID: Date: Thu, 28 Nov 2019 16:56:30 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 47NtN91TTYz4LpT X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.77 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-1.67)[ip: (-4.43), ipnet: 2a01:4f8::/29(-2.33), asn: 24940(-1.57), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Nov 2019 09:57:14 -0000 28.11.2019 16:38, Pete French wrote: > On 28/Nov/2019 07:03, Eugene Grosbein wrote: >> 28.11.2019 13:46, Eugene Grosbein wrote: >> >>> Hi! >>> >>> Is it normal that "zfs destroy" for one ZVOL with attribute "used" equal to 2112939808 bytes (~2GB) >>> takes over two minutes waiting on "tx_sync_done_cv"? The pool is RAID1 over five SSDs encrypted with GELI >> >> The pool is *RAIDZ1* > > Not normal, no. Having said that I have seen zfs destory take a very lng time on occasion, > but for a 2GB volume then thats surprisng. Is there actual disc activity during this time ? Yes, a lot. Several rsync instances writing to another ZFS (mounted file systems) in parallel: copying data over ssh from remote hosts (backups). From owner-freebsd-stable@freebsd.org Thu Nov 28 09:59:40 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 585D51CB868 for ; Thu, 28 Nov 2019 09:59:40 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47NtQz5S2hz4M3Z for ; Thu, 28 Nov 2019 09:59:39 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id xAS9xGC3064561 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 28 Nov 2019 09:59:17 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: killing@multiplay.co.uk Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id xAS9x6HU036011 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 28 Nov 2019 16:59:06 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: Slow zfs destroy To: Steven Hartland References: <01ca3a41-6cd2-823f-a8c1-30a47a0bbc98@grosbein.net> Cc: FreeBSD stable From: Eugene Grosbein Message-ID: <49c9c4da-c078-43b9-e7e2-51a1bacb2671@grosbein.net> Date: Thu, 28 Nov 2019 16:59:01 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 47NtQz5S2hz4M3Z X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.78 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-1.68)[ip: (-4.46), ipnet: 2a01:4f8::/29(-2.33), asn: 24940(-1.57), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Nov 2019 09:59:40 -0000 28.11.2019 14:26, Steven Hartland wrote: > As you mentioned it’s on SSD you could be suffering from poor TRIM performance from your devices > if you run gstat -pd you’ll be able to get an indication if this is the case. Yes, this box does have problems with poor TRIM performance. But isn't "zfs destroy" supposed to perform actual removal in background? "feature@async_destroy" is "enabled" for this pool. From owner-freebsd-stable@freebsd.org Thu Nov 28 12:22:46 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D703D1CFEBC for ; Thu, 28 Nov 2019 12:22:46 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from mail.rlwinm.de (mail.rlwinm.de [138.201.35.217]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47Nxc55wNtz4W7y for ; Thu, 28 Nov 2019 12:22:45 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from crest.bultmann.eu (unknown [IPv6:2a00:c380:c0d5:1:f0f5:ee99:97b6:e22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.rlwinm.de (Postfix) with ESMTPSA id 641982F30F for ; Thu, 28 Nov 2019 12:22:38 +0000 (UTC) Subject: Re: No amdtemp sysctls, AMD Ryzen 5 3600X To: freebsd-stable@freebsd.org References: <9368f66ee33b733f4d6a9c69dc8a2537@ijs.si> <157133a0-dd31-d498-c214-d75ccb21a125@FreeBSD.org> From: Jan Bramkamp Message-ID: Date: Thu, 28 Nov 2019 13:22:37 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:60.0) Gecko/20100101 Thunderbird/60.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 47Nxc55wNtz4W7y X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of crest@rlwinm.de designates 138.201.35.217 as permitted sender) smtp.mailfrom=crest@rlwinm.de X-Spamd-Result: default: False [-2.42 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; DMARC_NA(0.00)[rlwinm.de]; IP_SCORE(-0.12)[ipnet: 138.201.0.0/16(0.96), asn: 24940(-1.57), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:138.201.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Nov 2019 12:22:46 -0000 Here is the patch to add support for 3rd gen Ryzen to amdtemp: Index: sys/dev/amdsmn/amdsmn.c =================================================================== --- sys/dev/amdsmn/amdsmn.c    (revision 355171) +++ sys/dev/amdsmn/amdsmn.c    (working copy) @@ -59,6 +59,7 @@  #define    PCI_DEVICE_ID_AMD_15H_M60H_ROOT        0x1576  #define    PCI_DEVICE_ID_AMD_17H_ROOT        0x1450  #define    PCI_DEVICE_ID_AMD_17H_M10H_ROOT        0x15d0 +#define    PCI_DEVICE_ID_AMD_17H_M30H_ROOT        0x1480  struct pciid;  struct amdsmn_softc { @@ -90,6 +91,12 @@          .amdsmn_addr_reg = F17H_SMN_ADDR_REG,          .amdsmn_data_reg = F17H_SMN_DATA_REG,      }, +    { +        .amdsmn_vendorid = CPU_VENDOR_AMD, +        .amdsmn_deviceid = PCI_DEVICE_ID_AMD_17H_M30H_ROOT, +        .amdsmn_addr_reg = F17H_SMN_ADDR_REG, +        .amdsmn_data_reg = F17H_SMN_DATA_REG, +    },  };  /* Index: sys/dev/amdtemp/amdtemp.c =================================================================== --- sys/dev/amdtemp/amdtemp.c    (revision 355171) +++ sys/dev/amdtemp/amdtemp.c    (working copy) @@ -96,6 +96,7 @@  #define    DEVICEID_AMD_MISC16_M30H    0x1583  #define    DEVICEID_AMD_HOSTB17H_ROOT    0x1450  #define    DEVICEID_AMD_HOSTB17H_M10H_ROOT    0x15d0 +#define    DEVICEID_AMD_HOSTB17H_M30H_ROOT    0x1480  static const struct amdtemp_product {      uint16_t    amdtemp_vendorid; @@ -118,6 +119,7 @@      { VENDORID_AMD,    DEVICEID_AMD_MISC16_M30H, true },      { VENDORID_AMD,    DEVICEID_AMD_HOSTB17H_ROOT, false },      { VENDORID_AMD,    DEVICEID_AMD_HOSTB17H_M10H_ROOT, false }, +    { VENDORID_AMD,    DEVICEID_AMD_HOSTB17H_M30H_ROOT, false },  };  /* On 15.11.19 15:07, Mark Martinec wrote: >> On 15/11/2019 3:27 am, Mark Martinec wrote: >>> Running 12.1-RELEASE-p1 on AMD Ryzen 5 3600X cpu, >>> but I don't see any temperatures reported in sysctl, >>> even though amdtemp.ko and amdsmn.ko are loaded >>> and they don't produce any complaints on loading. > > 2019-11-15 03:01, Kubilay Kocak wrote: >> Resolver of original Ryzen 2 temperature support: >>   https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=218264 >> "In Progress" issue for Ryzen 5 support with patch: >>   https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239607 > > I have applied the patch from Bug 239607 and it works now! > Perfect, thanks! > >> Was committed to head (CURRENT), apparently merged to stable/12 >> (cant find the MFC commit). >> I've updated/retriaged the issue, and asked about a merge to >> stable/11, but at this point it looks like it missed the 12.1-RELEASE >> window > > It's unfortunate that it missed the 12.1-RELEASE. > Thanks for a quick response! > >   Mark > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Thu Nov 28 13:34:06 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6B5F41A9D75 for ; Thu, 28 Nov 2019 13:34:06 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47NzBP46Hlz4Znd for ; Thu, 28 Nov 2019 13:34:05 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm1-x333.google.com with SMTP id f129so11692935wmf.2 for ; Thu, 28 Nov 2019 05:34:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding:content-language; bh=cCPYNgqm+Sbuvi8ukDYUZ9PKVZU+bstZ3rmnMeWv/Lg=; b=ZG5M7b0fSre8rhf/svxDWP5We57VwI1TIPN64DCFOS4LkTEqtugvoqnbXXpHVJod35 Vehcdv4O7GtZocn2Vy+92tQC3UOsPb40MYrVjxKtgMG+LjQPJkAdu2uJ8fLlQyjOlaLL XjEBVxF+QIpNRCSZMYYfkpK9GQ+ChelEUgQvCo3Zd84SfRpI/0hP2sxewsnnNYTlaW0Y VAFCJhRhQ0/EL3xh9H/EZ1x2jcnuDRqrPkDd2K67P61+Hjb/iGff/phxuQSUcGKUqMJX Dj/3aHeXFGgQGWmbjCm2EwXEYkyUiWX34YbVqAiIXLn25OLERpB4mH2Piwo4ur/OIEj0 sa8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=cCPYNgqm+Sbuvi8ukDYUZ9PKVZU+bstZ3rmnMeWv/Lg=; b=HdnsUUIn9va+RO9LQjt08srZO8fozcJa8maVD8KyaevB8G9xVbEbVLH6fhXYoWdowK rmN68VSPYb7g0ZPmL7x9IKmQkQyceJI6XtpCQm35mafZgkI1J/ReqSctiSNYk8RET9WL nLfXmFAaijLaTj9wGFDfkpLD2SHGobYrwrDp8Cm+BTUiaQa0jMkvZdcoDudnW3tTRz91 sU9hZ/DH9jgwXiziszBoEFSUauQIBNI/N8Oe7lerPSAM7WFY6yT8xzs9riw5T9CfGPVe lT3fh6ggf5w/D2CHkStV/Cxlk6ab4XURskawwEy8IoMm7hgYt7ybx1vwoQW9Ztfla2i7 pv4g== X-Gm-Message-State: APjAAAUTEqLV0mtVAYwF+R0FtK/BQlhzqSgpql+6yRa/hKJHfdTADCpJ o7XMmtHWMAqF6mnZdb1+LxuwE3pFYTY= X-Google-Smtp-Source: APXvYqyfASaucWZmWeAnZDKpILDq+QCpFZe5JMlPy1eNGYeVdrXapyv0C3GRQ2iDcGxVCKDwv00KEA== X-Received: by 2002:a7b:c38c:: with SMTP id s12mr9841233wmj.84.1574948042870; Thu, 28 Nov 2019 05:34:02 -0800 (PST) Received: from [10.44.128.75] ([193.117.175.106]) by smtp.gmail.com with ESMTPSA id c15sm24018279wrx.78.2019.11.28.05.34.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 28 Nov 2019 05:34:01 -0800 (PST) Subject: Re: Slow zfs destroy To: Eugene Grosbein Cc: FreeBSD stable References: <01ca3a41-6cd2-823f-a8c1-30a47a0bbc98@grosbein.net> <49c9c4da-c078-43b9-e7e2-51a1bacb2671@grosbein.net> From: Steven Hartland Message-ID: Date: Thu, 28 Nov 2019 13:34:01 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.9.1 MIME-Version: 1.0 In-Reply-To: <49c9c4da-c078-43b9-e7e2-51a1bacb2671@grosbein.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-Rspamd-Queue-Id: 47NzBP46Hlz4Znd X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=multiplay-co-uk.20150623.gappssmtp.com header.s=20150623 header.b=ZG5M7b0f; dmarc=pass (policy=none) header.from=multiplay.co.uk; spf=pass (mx1.freebsd.org: domain of killing@multiplay.co.uk designates 2a00:1450:4864:20::333 as permitted sender) smtp.mailfrom=killing@multiplay.co.uk X-Spamd-Result: default: False [-5.67 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[multiplay-co-uk.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[multiplay-co-uk.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[3.3.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[multiplay.co.uk,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-2.67)[ip: (-8.66), ipnet: 2a00:1450::/32(-2.69), asn: 15169(-1.95), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Nov 2019 13:34:06 -0000 It may well depend on the extent of the deletes occurring. Have you tried disabling TRIM to see if it eliminates the delay?     Regards     Steve On 28/11/2019 09:59, Eugene Grosbein wrote: > 28.11.2019 14:26, Steven Hartland wrote: > >> As you mentioned it’s on SSD you could be suffering from poor TRIM performance from your devices >> if you run gstat -pd you’ll be able to get an indication if this is the case. > Yes, this box does have problems with poor TRIM performance. > But isn't "zfs destroy" supposed to perform actual removal in background? > "feature@async_destroy" is "enabled" for this pool. From owner-freebsd-stable@freebsd.org Thu Nov 28 16:19:24 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1A0E31AE6D6 for ; Thu, 28 Nov 2019 16:19:24 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47P2s66L08z3GTJ for ; Thu, 28 Nov 2019 16:19:22 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id xASGIoj4067935 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 28 Nov 2019 16:18:51 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: killing@multiplay.co.uk Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id xASGIgBk039408 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 28 Nov 2019 23:18:42 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: Slow zfs destroy To: Steven Hartland References: <01ca3a41-6cd2-823f-a8c1-30a47a0bbc98@grosbein.net> <49c9c4da-c078-43b9-e7e2-51a1bacb2671@grosbein.net> Cc: FreeBSD stable From: Eugene Grosbein Message-ID: <7b7a8ed0-2acf-2fc4-7c42-00423cdf4812@grosbein.net> Date: Thu, 28 Nov 2019 23:18:37 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 47P2s66L08z3GTJ X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.76 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-1.66)[ip: (-4.40), ipnet: 2a01:4f8::/29(-2.33), asn: 24940(-1.58), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Nov 2019 16:19:24 -0000 28.11.2019 20:34, Steven Hartland wrote: > It may well depend on the extent of the deletes occurring. > > Have you tried disabling TRIM to see if it eliminates the delay? This system used mfi(4) first and mfi(4) does not support TRIM at all. Performance was abysmal. Now it uses mrsas(4) and after switch I ran trim(8) for all SSDs one-by-one then re-added them to RAID1. Disabling TRIM is not an option. Almost a year has passed since then and I suspect SSDs have no or a few spare trimmed cells for some reason. Is there documented way to check this out? Maybe some SMART attribute? From owner-freebsd-stable@freebsd.org Fri Nov 29 15:26:42 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A4ADA1B3796 for ; Fri, 29 Nov 2019 15:26:42 +0000 (UTC) (envelope-from amdmi3@amdmi3.ru) Received: from ihor-5.amdmi3.ru (ihor-5.amdmi3.ru [185.238.136.130]) by mx1.freebsd.org (Postfix) with ESMTP id 47Pddr3fPSz420m for ; Fri, 29 Nov 2019 15:26:39 +0000 (UTC) (envelope-from amdmi3@amdmi3.ru) Received: from amdmi3.ru (localhost [IPv6:::1]) by ihor-5.amdmi3.ru (Postfix) with SMTP id CF1F913B86 for ; Fri, 29 Nov 2019 18:26:30 +0300 (MSK) Received: by amdmi3.ru (nbSMTP-1.00) for uid 1001 amdmi3@amdmi3.ru; Fri, 29 Nov 2019 18:26:30 +0300 (MSK) Date: Fri, 29 Nov 2019 18:16:06 +0300 From: Dmitry Marakasov To: freebsd-stable@freebsd.org Subject: How can kill(-1, 0) return EPERM? Message-ID: <20191129151606.GD4071@hades.panopticon> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: 47Pddr3fPSz420m X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of amdmi3@amdmi3.ru has no SPF policy when checking 185.238.136.130) smtp.mailfrom=amdmi3@amdmi3.ru X-Spamd-Result: default: False [4.41 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_DN_NONE(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_MEDIUM(0.99)[0.988,0]; DMARC_NA(0.00)[amdmi3.ru]; NEURAL_SPAM_LONG(1.00)[0.997,0]; R_SPF_NA(0.00)[]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:48666, ipnet:185.238.136.0/22, country:RU]; MIME_TRACE(0.00)[0:+]; IP_SCORE(0.42)[ip: (-0.18), ipnet: 185.238.136.0/22(0.48), asn: 48666(1.79), country: RU(0.01)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Nov 2019 15:26:42 -0000 Hi! I'm helping to investigate some userspace issue [1], where kill(-1, SIGKILL) fails with EPERM. I've managed to isolate this case in a small program: ``` #include #include #include #include #include #include int main() { if (setuid(66) == -1) // uucp, just for the test err(1, "setuid"); int res = kill(-1, 0); // <- fails with EPERM fprintf(stderr, "kill(-1, 0) result=%d, errno=%s\n", res, strerror(errno)); return 0; } ``` when run from root on 12.1 kill call fails with EPERM. However I cannot comprehend what it is caused by and how it's even possible: kill(2) manpage says that with pid=-1 kill should only send (and in this case of sig=0, /not/ send) signals to the processes belonging to the current uid, so there should be no permission problems. I've also looked into the kernel code (sys_kill, killpg1), and it matches to what manpage says, I see no way for it to return EPERM: sys_kill() should fall through to the switch, call killpg1() with all=1 and killpg1() if(all) branch may only set `ret` to either 0 or ESRCH. Am I missing something, or is there a problem somewhere? [1] https://github.com/NixOS/nix/issues/3250 -- Dmitry Marakasov . 55B5 0596 FF1E 8D84 5F56 9510 D35A 80DD F9D2 F77D amdmi3@amdmi3.ru ..: https://github.com/AMDmi3 From owner-freebsd-stable@freebsd.org Fri Nov 29 16:55:39 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0CAD91B5CCC for ; Fri, 29 Nov 2019 16:55:39 +0000 (UTC) (envelope-from amdmi3@amdmi3.ru) Received: from ihor-5.amdmi3.ru (ihor-5.amdmi3.ru [185.238.136.130]) by mx1.freebsd.org (Postfix) with ESMTP id 47PgcT6scjz45y3 for ; Fri, 29 Nov 2019 16:55:37 +0000 (UTC) (envelope-from amdmi3@amdmi3.ru) Received: from amdmi3.ru (localhost [IPv6:::1]) by ihor-5.amdmi3.ru (Postfix) with SMTP id A05A213BC6 for ; Fri, 29 Nov 2019 19:55:33 +0300 (MSK) Received: by amdmi3.ru (nbSMTP-1.00) for uid 1001 amdmi3@amdmi3.ru; Fri, 29 Nov 2019 19:55:33 +0300 (MSK) Date: Fri, 29 Nov 2019 19:45:09 +0300 From: Dmitry Marakasov To: freebsd-stable@freebsd.org Subject: Re: How can kill(-1, 0) return EPERM? Message-ID: <20191129164509.GE4071@hades.panopticon> References: <20191129151606.GD4071@hades.panopticon> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20191129151606.GD4071@hades.panopticon> User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: 47PgcT6scjz45y3 X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of amdmi3@amdmi3.ru has no SPF policy when checking 185.238.136.130) smtp.mailfrom=amdmi3@amdmi3.ru X-Spamd-Result: default: False [4.41 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_MEDIUM(0.99)[0.989,0]; DMARC_NA(0.00)[amdmi3.ru]; NEURAL_SPAM_LONG(1.00)[0.998,0]; R_SPF_NA(0.00)[]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:48666, ipnet:185.238.136.0/22, country:RU]; MIME_TRACE(0.00)[0:+]; IP_SCORE(0.42)[ip: (-0.18), ipnet: 185.238.136.0/22(0.48), asn: 48666(1.78), country: RU(0.01)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Nov 2019 16:55:39 -0000 * Dmitry Marakasov (amdmi3@amdmi3.ru) wrote: > I'm helping to investigate some userspace issue [1], where kill(-1, SIGKILL) > fails with EPERM. I've managed to isolate this case in a small program: > > > ``` > #include > #include > #include > #include > #include > #include > > int main() { > if (setuid(66) == -1) // uucp, just for the test > err(1, "setuid"); > > int res = kill(-1, 0); // <- fails with EPERM > fprintf(stderr, "kill(-1, 0) result=%d, errno=%s\n", res, strerror(errno)); > > return 0; > } > ``` > > when run from root on 12.1 kill call fails with EPERM. However I cannot > comprehend what it is caused by and how it's even possible: kill(2) manpage > says that with pid=-1 kill should only send (and in this case of sig=0, > /not/ send) signals to the processes belonging to the current uid, so there > should be no permission problems. I've also looked into the kernel code > (sys_kill, killpg1), and it matches to what manpage says, I see no way > for it to return EPERM: sys_kill() should fall through to the switch, call > killpg1() with all=1 and killpg1() if(all) branch may only set `ret` to > either 0 or ESRCH. Am I missing something, or is there a problem somewhere? It looks like I have misread the `else if' path of this core. if (all) { /* * broadcast */ sx_slock(&allproc_lock); FOREACH_PROC_IN_SYSTEM(p) { if (p->p_pid <= 1 || p->p_flag & P_SYSTEM || p == td->td_proc || p->p_state == PRS_NEW) { continue; } PROC_LOCK(p); err = p_cansignal(td, p, sig); if (err == 0) { if (sig) pksignal(p, sig, ksi); ret = err; } else if (ret == ESRCH) ret = err; PROC_UNLOCK(p); } sx_sunlock(&allproc_lock); } ... so it's clear now where EPERM comes from. However it looks like the behavior contradicts the manpage - there are no signs of check that the signalled process has the same uid as the caller. -- Dmitry Marakasov . 55B5 0596 FF1E 8D84 5F56 9510 D35A 80DD F9D2 F77D amdmi3@amdmi3.ru ..: https://github.com/AMDmi3 From owner-freebsd-stable@freebsd.org Fri Nov 29 17:57:05 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4B12B1B73AF for ; Fri, 29 Nov 2019 17:57:05 +0000 (UTC) (envelope-from bennett@sdf.org) Received: from mx.sdf.org (mx.sdf.org [205.166.94.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx.sdf.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47PhzM3MQVz48h7 for ; Fri, 29 Nov 2019 17:57:03 +0000 (UTC) (envelope-from bennett@sdf.org) Received: from sdf.org (IDENT:bennett@otaku.sdf.org [205.166.94.8]) by mx.sdf.org (8.15.2/8.14.5) with ESMTPS id xATHv1bJ024589 (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256 bits) verified NO); Fri, 29 Nov 2019 17:57:01 GMT Received: (from bennett@localhost) by sdf.org (8.15.2/8.12.8/Submit) id xATHv1P1003382; Fri, 29 Nov 2019 11:57:01 -0600 (CST) From: Scott Bennett Message-Id: <201911291757.xATHv1P1003382@sdf.org> Date: Fri, 29 Nov 2019 11:57:00 -0600 To: Eugene Grosbein Subject: Re: Slow zfs destroy Cc: freebsd-stable@freebsd.org User-Agent: Heirloom mailx 12.5 6/20/10 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 47PhzM3MQVz48h7 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of bennett@sdf.org has no SPF policy when checking 205.166.94.20) smtp.mailfrom=bennett@sdf.org X-Spamd-Result: default: False [0.30 / 15.00]; ARC_NA(0.00)[]; FAKE_REPLY(1.00)[]; NEURAL_HAM_MEDIUM(-0.85)[-0.847,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; IP_SCORE(-0.32)[ip: (-1.01), ipnet: 205.166.94.0/24(-0.50), asn: 14361(-0.03), country: US(-0.05)]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sdf.org]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-0.43)[-0.431,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[20.94.166.205.list.dnswl.org : 127.0.10.0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:14361, ipnet:205.166.94.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Nov 2019 17:57:05 -0000 On Thu, 28 Nov 2019 23:18:37 +0700 Eugene Grosbein wrote: >28.11.2019 20:34, Steven Hartland wrote: > >> It may well depend on the extent of the deletes occurring. >> >> Have you tried disabling TRIM to see if it eliminates the delay? > >This system used mfi(4) first and mfi(4) does not support TRIM at all. Performance was abysmal. >Now it uses mrsas(4) and after switch I ran trim(8) for all SSDs one-by-one then re-added them to RAID1. >Disabling TRIM is not an option. > >Almost a year has passed since then and I suspect SSDs have no or a few spare trimmed cells for some reason. >Is there documented way to check this out? Maybe some SMART attribute? > You neglected to state whether you used "zfs destroy datasetname" or "zfs destroy -d datasetname". If you used the former, then ZFS did what you told it to do. If you want the data set destroyed in the background, you will need to include the "-d" option in the command. (See the zfs(1) man page at defer_destroy under "Native Properties".) Scott Bennett, Comm. ASMELG, CFIAG ********************************************************************** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * *--------------------------------------------------------------------* * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * * -- Gov. John Hancock, New York Journal, 28 January 1790 * ********************************************************************** From owner-freebsd-stable@freebsd.org Fri Nov 29 22:58:53 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 586891BEF56 for ; Fri, 29 Nov 2019 22:58:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47Pqgc2N5Gz4Pd3 for ; Fri, 29 Nov 2019 22:58:51 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id xATMwZUQ087155 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 30 Nov 2019 00:58:38 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua xATMwZUQ087155 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id xATMwYxM087154; Sat, 30 Nov 2019 00:58:34 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 30 Nov 2019 00:58:34 +0200 From: Konstantin Belousov To: Dmitry Marakasov Cc: freebsd-stable@freebsd.org Subject: Re: How can kill(-1, 0) return EPERM? Message-ID: <20191129225834.GY10580@kib.kiev.ua> References: <20191129151606.GD4071@hades.panopticon> <20191129164509.GE4071@hades.panopticon> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20191129164509.GE4071@hades.panopticon> User-Agent: Mutt/1.12.2 (2019-09-21) X-Spam-Status: No, score=0.7 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED,URIBL_SBL,URIBL_SBL_A autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-Rspamd-Queue-Id: 47Pqgc2N5Gz4Pd3 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-1.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(0.00)[ip: (-2.72), ipnet: 2001:470::/32(-4.64), asn: 6939(-3.52), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Nov 2019 22:58:53 -0000 On Fri, Nov 29, 2019 at 07:45:09PM +0300, Dmitry Marakasov wrote: > * Dmitry Marakasov (amdmi3@amdmi3.ru) wrote: > > > I'm helping to investigate some userspace issue [1], where kill(-1, SIGKILL) > > fails with EPERM. I've managed to isolate this case in a small program: > > > > > > ``` > > #include > > #include > > #include > > #include > > #include > > #include > > > > int main() { > > if (setuid(66) == -1) // uucp, just for the test > > err(1, "setuid"); > > > > int res = kill(-1, 0); // <- fails with EPERM > > fprintf(stderr, "kill(-1, 0) result=%d, errno=%s\n", res, strerror(errno)); > > > > return 0; > > } > > ``` > > > > when run from root on 12.1 kill call fails with EPERM. However I cannot > > comprehend what it is caused by and how it's even possible: kill(2) manpage > > says that with pid=-1 kill should only send (and in this case of sig=0, > > /not/ send) signals to the processes belonging to the current uid, so there > > should be no permission problems. I've also looked into the kernel code > > (sys_kill, killpg1), and it matches to what manpage says, I see no way > > for it to return EPERM: sys_kill() should fall through to the switch, call > > killpg1() with all=1 and killpg1() if(all) branch may only set `ret` to > > either 0 or ESRCH. Am I missing something, or is there a problem somewhere? > > It looks like I have misread the `else if' path of this core. > > if (all) { > /* > * broadcast > */ > sx_slock(&allproc_lock); > FOREACH_PROC_IN_SYSTEM(p) { > if (p->p_pid <= 1 || p->p_flag & P_SYSTEM || > p == td->td_proc || p->p_state == PRS_NEW) { > continue; > } > PROC_LOCK(p); > err = p_cansignal(td, p, sig); > if (err == 0) { > if (sig) > pksignal(p, sig, ksi); > ret = err; > } > else if (ret == ESRCH) > ret = err; > PROC_UNLOCK(p); > } > sx_sunlock(&allproc_lock); > } ... > > so it's clear now where EPERM comes from. However it looks like the > behavior contradicts the manpage - there are no signs of check that > the signalled process has the same uid as the caller. I am not sure what you mean by 'signs of check'. Look at p_cansignal() and cr_cansignal() implementation. From owner-freebsd-stable@freebsd.org Sat Nov 30 08:53:14 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A55061CCC00 for ; Sat, 30 Nov 2019 08:53:14 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47Q4sP2vHSz3NXs for ; Sat, 30 Nov 2019 08:53:13 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id xAU8n0Jb086327 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sat, 30 Nov 2019 08:49:03 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: Received: from [10.58.0.4] (dadv@[10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id xAU8mvE5065738 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT) for ; Sat, 30 Nov 2019 15:48:57 +0700 (+07) (envelope-from eugen@grosbein.net) To: FreeBSD stable From: Eugene Grosbein Subject: ZFS deadlock Message-ID: <2a1d0c98-03a9-66e4-4602-7f81c28cd34e@grosbein.net> Date: Sat, 30 Nov 2019 15:48:56 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=4.1 required=5.0 tests=BAYES_00, DATE_IN_FUTURE_96_Q, HELO_MISC_IP, LOCAL_FROM, SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.7 DATE_IN_FUTURE_96_Q Date: is 4 days to 4 months after Received: * date * 2.6 LOCAL_FROM From my domains * 1.1 HELO_MISC_IP Looking for more Dynamic IP Relays X-Spam-Level: **** X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on eg.sd.rdtc.ru X-Rspamd-Queue-Id: 47Q4sP2vHSz3NXs X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.78 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[grosbein.net]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.68)[ip: (-4.47), ipnet: 2a01:4f8::/29(-2.35), asn: 24940(-1.58), country: DE(-0.01)]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Nov 2019 08:53:14 -0000 Hi! I have RAIDZ1 with five GELI-encrypted SSDs da[2-6].eli (non-boot pool). I've exported the pool, destroyed da2.eli then successfully imported pool back in degraded state. Then I've mounted some file systems successfully but zfs mount for next one hung on [tx->tx_sync_done_cv] for 4400 seconds and counting. # procstat -kk -L 55464 PID TID COMM TDNAME KSTACK 55464 102422 zfs - mi_switch+0xeb sleepq_wait+0x2c _cv_wait+0x16e txg_wait_synced+0xa5 dmu_tx_assign+0x48 zfs_rmnode+0x122 zfs_freebsd_reclaim+0x4e VOP_RECLAIM_APV+0x80 vgonel+0x213 vrecycle+0x46 zfs_freebsd_inactive+0xd VOP_INACTIVE_APV+0x80 vinactive+0xf0 vputx+0x2c3 zfs_unlinked_drain+0x1b8 zfsvfs_setup+0x5e zfs_mount+0x5f5 vfs_domount+0x573 It looks like deadlock for me. What can I do to resolve this? FreeBSD 11.3-STABLE/amd64 r354667. From owner-freebsd-stable@freebsd.org Sat Nov 30 09:21:35 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DD6461CD8EF for ; Sat, 30 Nov 2019 09:21:35 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47Q5V70YGtz3PnX for ; Sat, 30 Nov 2019 09:21:34 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id xAU9HV2I086497 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sat, 30 Nov 2019 09:17:31 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id xAU9HStX066035 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT) for ; Sat, 30 Nov 2019 16:17:28 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: ZFS deadlock To: FreeBSD stable References: <2a1d0c98-03a9-66e4-4602-7f81c28cd34e@grosbein.net> From: Eugene Grosbein Message-ID: Date: Sat, 30 Nov 2019 16:17:23 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <2a1d0c98-03a9-66e4-4602-7f81c28cd34e@grosbein.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Flag: YES X-Spam-Status: Yes, score=6.0 required=5.0 tests=BAYES_00, DATE_IN_FUTURE_96_Q, HELO_MISC_IP,LOCAL_FROM,RDNS_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.7 DATE_IN_FUTURE_96_Q Date: is 4 days to 4 months after Received: * date * 1.9 RDNS_NONE Delivered to internal network by a host with no rDNS * 2.6 LOCAL_FROM From my domains * 1.1 HELO_MISC_IP Looking for more Dynamic IP Relays X-Spam-Level: ****** X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on eg.sd.rdtc.ru X-Rspamd-Queue-Id: 47Q5V70YGtz3PnX X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [1.31 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.93)[-0.933,0]; SPAM_FLAG(5.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.98)[-0.981,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[grosbein.net]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.68)[ip: (-4.44), ipnet: 2a01:4f8::/29(-2.35), asn: 24940(-1.58), country: DE(-0.01)]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Nov 2019 09:21:35 -0000 30.11.2019 15:48, Eugene Grosbein wrote: > Hi! > > I have RAIDZ1 with five GELI-encrypted SSDs da[2-6].eli (non-boot pool). > > I've exported the pool, destroyed da2.eli then successfully imported pool back in degraded state. > Then I've mounted some file systems successfully but zfs mount for next one hung on [tx->tx_sync_done_cv] > for 4400 seconds and counting. > > # procstat -kk -L 55464 > PID TID COMM TDNAME KSTACK > 55464 102422 zfs - mi_switch+0xeb sleepq_wait+0x2c _cv_wait+0x16e txg_wait_synced+0xa5 dmu_tx_assign+0x48 zfs_rmnode+0x122 zfs_freebsd_reclaim+0x4e VOP_RECLAIM_APV+0x80 vgonel+0x213 vrecycle+0x46 zfs_freebsd_inactive+0xd VOP_INACTIVE_APV+0x80 vinactive+0xf0 vputx+0x2c3 zfs_unlinked_drain+0x1b8 zfsvfs_setup+0x5e zfs_mount+0x5f5 vfs_domount+0x573 > > It looks like deadlock for me. > > What can I do to resolve this? FreeBSD 11.3-STABLE/amd64 r354667. "zfs mount" just completed successfully after trim -f /dev/da2 (3.5TB) finished in 76 minutes.