Date: Wed, 1 May 2024 15:24:53 -0500 From: Matthew Grooms <mgrooms@shrew.net> To: stable@freebsd.org Subject: Re: how to tell if TRIM is working Message-ID: <67721332-fa1d-4b3c-aa57-64594ad5d77a@shrew.net> In-Reply-To: <5e1b5097-c1c0-4740-a491-63c709d01c25@sentex.net> References: <5e1b5097-c1c0-4740-a491-63c709d01c25@sentex.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 5/1/24 14:38, mike tancsa wrote: > Kind of struggling to check if TRIM is actually working or not with my > SSDs on RELENG_14 in ZFS. > > On a pool that has almost no files on it (capacity at 0% out of 3TB), > should not > > zpool -w trim <pool> be almost instant after a couple of runs ? > Instead it seems to always take about 10min to complete. > > Looking at the stats, > > kstat.zfs.tortank1.misc.iostats.trim_bytes_failed: 0 > kstat.zfs.tortank1.misc.iostats.trim_extents_failed: 0 > kstat.zfs.tortank1.misc.iostats.trim_bytes_skipped: 2743435264 > kstat.zfs.tortank1.misc.iostats.trim_extents_skipped: 253898 > kstat.zfs.tortank1.misc.iostats.trim_bytes_written: 14835526799360 > kstat.zfs.tortank1.misc.iostats.trim_extents_written: 1169158 > > what and why are bytes being skipped ? > > One of the drives for example > > sysctl -a kern.cam.ada.0 > kern.cam.ada.0.trim_ticks: 0 > kern.cam.ada.0.trim_goal: 0 > kern.cam.ada.0.sort_io_queue: 0 > kern.cam.ada.0.rotating: 0 > kern.cam.ada.0.unmapped_io: 1 > kern.cam.ada.0.flags: > 0x1be3bde<CAN_48BIT,CAN_FLUSHCACHE,CAN_NCQ,CAN_DMA,WAS_OTAG,CAN_TRIM,OPEN,SCTX_INIT,CAN_POWERMGT,CAN_DMA48,CAN_LOG,CAN_WCACHE,CAN_RAHEAD,PROBED,ANNOUNCED,DIRTY,PIM_ATA_EXT,UNMAPPEDIO> > kern.cam.ada.0.max_seq_zones: 0 > kern.cam.ada.0.optimal_nonseq_zones: 0 > kern.cam.ada.0.optimal_seq_zones: 0 > kern.cam.ada.0.zone_support: None > kern.cam.ada.0.zone_mode: Not Zoned > kern.cam.ada.0.write_cache: -1 > kern.cam.ada.0.read_ahead: -1 > kern.cam.ada.0.trim_lbas: 7771432624 > kern.cam.ada.0.trim_ranges: 371381 > kern.cam.ada.0.trim_count: 310842 > kern.cam.ada.0.delete_method: DSM_TRIM > > If I take one of the disks out of the pool and replace it with a > spare, and do a manual trim it seems to work > I had a hard time seeing evidence of this at the disk level while fiddling with TRIM recently. It appeared that at least some counters are driver and operation specific. For example, the da driver appears to update counters in some paths but not others. I assume that ada is different. There is a bug report for da, but haven't seen any feedback ... https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277673 You could try to run gstat with the -d flag during the time period when the delete operations are expected to occur. That should give you an idea of what's happening at the disk level in real time but may not offer more info than you're already seeing. > e.g. here was one disk in the pool that was taking a long time for > each zpool trim > > # time trim -f /dev/ada1 > trim /dev/ada1 offset 0 length 1000204886016 > 0.000u 0.057s 1:29.33 0.0% 5+184k 0+0io 0pf+0w > and then if I re-run it > # time trim -f /dev/ada1 > trim /dev/ada1 offset 0 length 1000204886016 > 0.000u 0.052s 0:04.15 1.2% 1+52k 0+0io 0pf+0w > > 90 seconds and then 4 seconds after that. > -Matthew
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?67721332-fa1d-4b3c-aa57-64594ad5d77a>