Date: Wed, 18 Nov 2015 00:28:54 +0000 From: Steven Hartland <killing@multiplay.co.uk> To: freebsd-stable@freebsd.org Cc: Alexander Motin <mav@FreeBSD.org> Subject: Re: Bug 204641 - 10.2 UNMAP/TRIM not available on a zfs zpool that uses iSCSI disks, backed on a zpool file target Message-ID: <564BC646.5070708@multiplay.co.uk> In-Reply-To: <CAB2_NwCtAAKG13ZqY2L1FDyBbfPYscUAGXBWnTLu_oQMM-daYw@mail.gmail.com> References: <CAB2_NwCtAAKG13ZqY2L1FDyBbfPYscUAGXBWnTLu_oQMM-daYw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 17/11/2015 22:08, Christopher Forgeron wrote: > I just submitted this as a bug: > > ( https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204641 ) > > ..but I thought I should bring it to the list's attention for more exposure > - If that's a no-no, let me know, as I have a few others that are related > to this that I'd like to discuss. > > - - - - > > > Consider this scenario: > > Virtual FreeBSD Machine, with a zpool created out of iSCSI disks. > Physical FreeBSD Machine, with a zpool holding a sparse file that is the > target for the iSCSI disk. > > This setup works in an environment with all 10.1 machines, doesn't with all > 10.2 machines. > > - The 10.2 Machines are 10.2-p7 RELEASE, updated via freebsd-update, no > custom. > - The 10.1 Machine are 10.1-p24 RELEASE, updated via freebsd-update, no > custom. > - iSCSI is all CAM iSCSI, not the old istgt platform. > - The iSCSI Target is a sparse file, stored on a zpool (not a vdev Target) > > The target machine is the same physical machine, with the same zpools - I > either boot 10.1 or 10.2 for testing, and use the same zpool/disks > > to ensure nothing is changing. > > If I have a 10.2 iSCSI Initiator (client) connected to a 10.2 iSCSI Target, > TRIM doesn't work (shows as NONE below). > If I have a 10.2 iSCSI Initiator (client) connected to a 10.1 iSCSI Target, > TRIM does work. > > (There is another bug with that last scenario as well, but I will open it > separately) > > ...for clarity, a 10.1 iSCSI Initiator connected to a 10.1 iSCSI Target > also works perfectly. I have ~20 of these in the field. > > On the 10.1 / 10.2 Targets, the ctl.conf file is identical. Zpools are > identical, because they are shared between reboots of the same iSCSI > > target machine. > > > > On the 10.2 initiator machine, connected to a 10.2 Target machine: > > # sysctl -a | grep cam.da > > kern.cam.da.2.minimum_cmd_size: 6 > kern.cam.da.2.delete_max: 131072 > kern.cam.da.2.delete_method: NONE > kern.cam.da.1.error_inject: 0 > kern.cam.da.1.sort_io_queue: 0 > kern.cam.da.1.minimum_cmd_size: 6 > kern.cam.da.1.delete_max: 131072 > kern.cam.da.1.delete_method: NONE > kern.cam.da.0.error_inject: 0 > kern.cam.da.0.sort_io_queue: -1 > kern.cam.da.0.minimum_cmd_size: 6 > kern.cam.da.0.delete_max: 131072 > kern.cam.da.0.delete_method: NONE > > Note the delete_method is NONE > > > # sysctl -a | grep trim > vfs.zfs.trim.max_interval: 1 > vfs.zfs.trim.timeout: 30 > vfs.zfs.trim.txg_delay: 32 > vfs.zfs.trim.enabled: 1 > vfs.zfs.vdev.trim_max_pending: 10000 > vfs.zfs.vdev.trim_max_active: 64 > vfs.zfs.vdev.trim_min_active: 1 > vfs.zfs.vdev.trim_on_init: 1 > kstat.zfs.misc.zio_trim.failed: 0 > kstat.zfs.misc.zio_trim.unsupported: 181 > kstat.zfs.misc.zio_trim.success: 0 > kstat.zfs.misc.zio_trim.bytes: 0 > > Note no trimmed bytes. > > > On the target machine, 10.1 and 10.2 share the same config file: > /etc/ctl.conf > > portal-group pg0 { > discovery-auth-group no-authentication > listen 0.0.0.0 > listen [::] > } > > lun 0 { > path /pool92/iscsi/iscsi.zvol > blocksize 4K > size 5T > option unmap "on" > option scsiname "pool92" > option vendor "pool92" > option insecure_tpc "on" > } > } > > > target iqn.iscsi1.zvol { > auth-group no-authentication > portal-group pg0 > > lun 0 { > path /pool92_1/iscsi/iscsi.zvol > blocksize 4K > size 5T > option unmap "on" > option scsiname "pool92_1" > option vendor "pool92_1" > option insecure_tpc "on" > } > } > > > When I boot a 10.1 Target server, the 10.2 initiator connects, and we do > see proper UNMAP ability: > > > kern.cam.da.2.minimum_cmd_size: 6 > kern.cam.da.2.delete_max: 5497558138880 > kern.cam.da.2.delete_method: UNMAP > kern.cam.da.1.error_inject: 0 > kern.cam.da.1.sort_io_queue: 0 > kern.cam.da.1.minimum_cmd_size: 6 > kern.cam.da.1.delete_max: 5497558138880 > kern.cam.da.1.delete_method: UNMAP > kern.cam.da.0.error_inject: 0 > kern.cam.da.0.sort_io_queue: -1 > kern.cam.da.0.minimum_cmd_size: 6 > kern.cam.da.0.delete_max: 131072 > kern.cam.da.0.delete_method: NONE > > > Please let me know what you'd like to know next. > Having a quick flick through the code it looks like umap is now only supported on dev backed and not file backed. I believe the following commit is the cause: https://svnweb.freebsd.org/base?view=revision&revision=279005 This was an MFC of: https://svnweb.freebsd.org/base?view=revision&revision=278672 I'm guessing this was an unintentional side effect mav? Regards Steve
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?564BC646.5070708>