From owner-freebsd-questions@freebsd.org Fri Sep 3 20:30:51 2021 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0B80567CE29 for ; Fri, 3 Sep 2021 20:30:51 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) Received: from holgerdanske.com (holgerdanske.com [IPv6:2001:470:0:19b::b869:801b]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "xray.he.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H1TvY6JTpz3hbR for ; Fri, 3 Sep 2021 20:30:49 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=holgerdanske.com; s=nov-20210719-112354; t=1630701039; bh=8coMxaF37cudU3wlvB43zBIGzUD3nErOs9n2pFTGyHM=; h=Received:Subject:To:References:From:Message-ID:Date:User-Agent: MIME-Version:In-Reply-To:Content-Type:Content-Language: Content-Transfer-Encoding; b=qLWfFtrJ2rCN0eHagalneNdDhavOrPGYGwJvuHyS7pJi6/SB+qGzaASUV8ScN0dLP SaH0SEKENIIfPMy8mqoSaEBNLA0ajn4KG1hGdlR5gPGQpxaHMErOAWDaBHzw7Rxjx9 uumkWZ8XrgAZbPZ7mR/4YDRzxiPcelIq5X7GxW58q4ePBvloSkwI7C32eEq4W5B4b/ gJ19u4wJ2nlmUzqv1l2TI2DjOpCnqQHXC9fpurFGCa0WReRpoQEVLvTecSED5sZqyn 3hVJ+kEwhujV/1p92CLYa63DF2yl8651Jy6d3blPfew05wNyqM2a4Z4EZnx2Z/r1Ru xzCh4WajlLzZqCyDVTk5ZrqHipbEV5HNMnLOEEpSuj2tHdjOl+o9rsovduKBKcSmZN 9337d8MWKmwmBvOxNtZEZzQQeBUyf9SmBm2eEpLmlh9K6do3gemnC0TKi7hsdr6iD+ Cukj464XqPQ6TTGA5qnU4iTXKvZ7jaPhaXy7LEiMPd0rID1IoHEcojQPML+TeevjOC lUfSLYv4j4170AFTkasGGL/XLWE99KUSZTdai5ceORRIyOPra2YM8u1CGP3TguePX7 k41UXvzlQh/F5jP3nO2tyvZhojAN21//hqxOdHDAne0Oi/zD8Ts3LN25b1dam8ixRA vA2UZDDxOvFDCirL+utEHmOU= Received: from 99.100.19.101 (99-100-19-101.lightspeed.frokca.sbcglobal.net [99.100.19.101]) by holgerdanske.com with ESMTPSA (TLS_AES_128_GCM_SHA256:TLSv1.3:Kx=any:Au=any:Enc=AESGCM(128):Mac=AEAD) (SMTP-AUTH username dpchrist@holgerdanske.com, mechanism PLAIN) for ; Fri, 3 Sep 2021 13:30:39 -0700 Subject: Re: FreeBSD 12.2-RELEASE-p9 trim: open failed: /dev/ada0: Operation not permitted To: freebsd-questions@freebsd.org References: <47d0f250-a47c-bbe0-2875-4f42f4464fcc@chezmarcotte.ca> From: David Christensen Message-ID: <64099455-5526-beb5-8ead-6ce8a9d073e6@holgerdanske.com> Date: Fri, 3 Sep 2021 13:30:38 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: <47d0f250-a47c-bbe0-2875-4f42f4464fcc@chezmarcotte.ca> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4H1TvY6JTpz3hbR X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=holgerdanske.com header.s=nov-20210719-112354 header.b=qLWfFtrJ; dmarc=pass (policy=none) header.from=holgerdanske.com; spf=pass (mx1.freebsd.org: domain of dpchrist@holgerdanske.com designates 2001:470:0:19b::b869:801b as permitted sender) smtp.mailfrom=dpchrist@holgerdanske.com X-Spamd-Result: default: False [-3.57 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[holgerdanske.com:s=nov-20210719-112354]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+a:november.he.net]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DKIM_TRACE(0.00)[holgerdanske.com:+]; DMARC_POLICY_ALLOW(-0.50)[holgerdanske.com,none]; NEURAL_HAM_SHORT(-0.57)[-0.568]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-questions]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Sep 2021 20:30:51 -0000 On 9/3/21 10:09 AM, Derek (freebsd lists) wrote: > On 2021-08-29 10:26 p.m., David Christensen wrote: >> I would like to trim the SSD, but trim(8) is not happy: >> >> 2021-08-29 19:22:36 toor@f1 ~ >> # trim -f /dev/ada0 >> trim /dev/ada0 offset 0 length 60022480896 >> trim: open failed: /dev/ada0: Operation not permitted >> > > I don't know if it's still relevant to the thread, but I feel like what > you are encountering here relates to what you are trying to trim. > > It feels like if you trim the "raw device", the system wouldn't know > what is in use, and what isn't. > > I feel like you should be trying to trim a filesystem on the device, > rather than the device itself, as the utility would be able to determine > what is currently allocated with the filesystem data. > > It might be that a raw device could be trimmed, but perhaps there's some > safety there to stop it from wiping a device that is backing a mounted > filesystem. > > Something to look at? > > Derek That is a good point -- the filesystem knows what blocks are in use, what blocks are not in use, and when blocks are removed from use. Somehow, trim(8) gets this information (from the kernel filesystem stack?). RTFM trim(8): SYNOPSIS trim [-Nfqv] [-[lo] offset[K|k|M|m|G|g|T|t]] [-r rfile] device ... -l offset[K|k|M|m|G|g|T|t] -o offset[K|k|M|m|G|g|T|t] Specify the length -l of the region to trim or its offset -o from the beginning of the device. The whole device is erased by default unless one or both of these options are presented. Note that actual success of the operation depends of underlying device driver such as ada(4), da(4) and others. Refer to corresponding manual pages for detail on possible caveats in low level support for ATA TRIM or SCSI UNMAP commands. Note: 1. The command line argument is *device*. 2. The descriptions for the -l and -o options state trim(8) erases the whole *device*. Here is another console session: 2021-09-03 12:07:27 toor@f2 ~ # freebsd-version ; uname -a 12.2-RELEASE-p10 FreeBSD f2.tracy.holgerdanske.com 12.2-RELEASE-p7 FreeBSD 12.2-RELEASE-p7 GENERIC amd64 2021-09-03 12:10:06 toor@f2 ~ # kldstat | egrep 'Id|zfs' Id Refs Address Size Name 2 1 0xffffffff8247c000 3bad38 zfs.ko 2021-09-03 12:11:54 toor@f2 ~ # find /boot -iname '*zfs*' 2021-09-03 12:12:18 toor@f2 ~ # geom disk list ada0 Geom name: ada0 Providers: 1. Name: ada0 Mediasize: 60022480896 (56G) Sectorsize: 512 Mode: r3w3e9 descr: INTEL SSDSC2CW060A3 lunid: **************** ident: ****************** rotationrate: 0 fwsectors: 63 fwheads: 16 2021-09-03 12:13:14 toor@f2 ~ # geom part show -p => 63 117231345 ada0 MBR (56G) 63 1985 - free - (993K) 2048 29360128 ada0s1 freebsd [active] (14G) 29362176 87867392 ada0s2 freebsd (42G) 117229568 1840 - free - (920K) => 0 29360128 ada0s1 BSD (14G) 0 4194304 ada0s1a freebsd-zfs (2.0G) 4194304 4194304 ada0s1b freebsd-swap (2.0G) 8388608 20971520 ada0s1d freebsd-zfs (10G) 2021-09-03 12:15:28 toor@f2 ~ # geli status Name Status Components ada0s1d.eli ACTIVE ada0s1d mirror/swap.eli ACTIVE mirror/swap 2021-09-03 12:16:41 toor@f2 ~ # zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT bootpool 1.88G 475M 1.41G - - 3% 24% 1.08x ONLINE - f2_zroot 9.50G 5.88G 3.62G - - 23% 61% 1.21x ONLINE - 2021-09-03 12:17:34 toor@f2 ~ # zpool status pool: bootpool state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: scrub repaired 0 in 0 days 00:00:02 with 0 errors on Sat May 29 16:18:50 2021 config: NAME STATE READ WRITE CKSUM bootpool ONLINE 0 0 0 ada0s1a ONLINE 0 0 0 errors: No known data errors pool: f2_zroot state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: scrub repaired 0 in 0 days 00:00:23 with 0 errors on Sat May 29 16:19:12 2021 config: NAME STATE READ WRITE CKSUM f2_zroot ONLINE 0 0 0 ada0s1d.eli ONLINE 0 0 0 errors: No known data errors 2021-09-03 12:18:46 toor@f2 ~ # zfs get mountpoint bootpool f2_zroot NAME PROPERTY VALUE SOURCE bootpool mountpoint /bootpool local f2_zroot mountpoint /f2_zroot local 2021-09-03 12:19:06 toor@f2 ~ # trim -f -v /bootpool trim /bootpool offset 0 length 3 trim: open failed: /bootpool: Is a directory 2021-09-03 12:20:00 toor@f2 ~ # trim -f -v /f2_zroot trim /f2_zroot offset 0 length 2 trim: open failed: /f2_zroot: Is a directory 2021-09-03 12:21:49 toor@f2 ~ # trim -f -v ada0 trim ada0 offset 0 length 60022480896 trim: open failed: ada0: Operation not permitted 2021-09-03 12:22:52 toor@f2 ~ # trim -f -v ada0s1 trim ada0s1 offset 0 length 15032385536 trim: open failed: ada0s1: Operation not permitted 2021-09-03 12:23:55 toor@f2 ~ # trim -f -v ada0s1a trim ada0s1a offset 0 length 2147483648 trim: open failed: ada0s1a: Operation not permitted 2021-09-03 12:24:57 toor@f2 ~ # trim -f -v ada0s1b trim ada0s1b offset 0 length 2147483648 trim: open failed: ada0s1b: Operation not permitted 2021-09-03 12:25:58 toor@f2 ~ # trim -f -v ada0s1d trim ada0s1d offset 0 length 10737418240 trim: open failed: ada0s1d: Operation not permitted 2021-09-03 12:26:57 toor@f2 ~ # man 4 ada | grep -i trim 2021-09-03 12:27:35 toor@f2 ~ # man 4 ada | grep -i unmap 2021-09-03 13:24:51 toor@f2 ~ # man 8 zpool | grep -i trim 2021-09-03 13:24:54 toor@f2 ~ # man 8 zpool | grep -i unmap 2021-09-03 13:24:55 toor@f2 ~ # man 8 zfs | grep -i trim 2021-09-03 13:25:07 toor@f2 ~ # man 8 zfs | grep -i unmap My WAG: 1. trim(8) is not integrated with ZFS. 2. ZFS does not provide a TRIM configuration settings or userland tools. 3. Given the disk was zeroed before installation and given the monotonically increasing sizes of the 'gzip -1' compressed raw images I take monthly, TRIM is not implemented by ZFS. David