Date: Tue, 31 Jan 2017 15:40:53 -0600 From: Larry Rosenman <ler@FreeBSD.org> To: Steven Hartland <killing@multiplay.co.uk> Cc: Freebsd fs <freebsd-fs@freebsd.org> Subject: Re: 16.0E ExpandSize? -- New Server Message-ID: <530ac81cbcdf268919a887f3ceff41d8@FreeBSD.org> In-Reply-To: <f6e819d3-3770-fd64-f6d3-addbcdc05e28@multiplay.co.uk> References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> <ce5a1d39612d694077accda33266a3ab@FreeBSD.org> <ad07e84e-f297-362a-1398-c5503bb56a8d@multiplay.co.uk> <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> <76fc9505-f681-0de0-fe0c-5624b29de321@multiplay.co.uk> <22e1bfc5840d972cf93643733682cda1@FreeBSD.org> <f2600a53-0dc1-9f41-1405-ed22d96d30cf@multiplay.co.uk> <8a710dc75c129f58b0372eeaeca575b5@FreeBSD.org> <aef02eb0-0888-6fea-a4b8-4033ca56f4a3@multiplay.co.uk> <d3181bd00c827fb99fbcebe6fe097ef8@FreeBSD.org> <f6e819d3-3770-fd64-f6d3-addbcdc05e28@multiplay.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
perfect: borg-new /home/ler $ zpool list NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT zroot 10.8T 94.3G 10.7T - 0% 0% 1.00x ONLINE - borg-new /home/ler $ zpool get all NAME PROPERTY VALUE SOURCE zroot size 10.8T - zroot capacity 0% - zroot altroot - default zroot health ONLINE - zroot guid 11945658884309024932 default zroot version - default zroot bootfs zroot/ROOT/default local zroot delegation on default zroot autoreplace off default zroot cachefile - default zroot failmode wait default zroot listsnapshots off default zroot autoexpand off default zroot dedupditto 0 default zroot dedupratio 1.00x - zroot free 10.7T - zroot allocated 94.3G - zroot readonly off - zroot comment - default zroot expandsize - - zroot freeing 0 default zroot fragmentation 0% - zroot leaked 0 default zroot feature@async_destroy enabled local zroot feature@empty_bpobj active local zroot feature@lz4_compress active local zroot feature@multi_vdev_crash_dump enabled local zroot feature@spacemap_histogram active local zroot feature@enabled_txg active local zroot feature@hole_birth active local zroot feature@extensible_dataset enabled local zroot feature@embedded_data active local zroot feature@bookmarks enabled local zroot feature@filesystem_limits enabled local zroot feature@large_blocks enabled local zroot feature@sha512 enabled local zroot feature@skein enabled local borg-new /home/ler $ On 01/31/2017 3:17 pm, Steven Hartland wrote: > Ok so that confirms it, try the attached patch (only a new kernel is needed) on a read only import of the pool and see if that fixes it. > > Regards > Steve > > On 31/01/2017 21:00, Larry Rosenman wrote: > > borg-new /home/ler $ sudo ./vdev-stats.d > Password: > vdev_path: n/a, vdev_max_asize: 0, vdev_asize: 0, vdev_min_asize: 0 > vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: 11947478089728, vdev_min_asize: 11888469475328 > vdev_path: /dev/mfid4p4, vdev_max_asize: 1991245299712, vdev_asize: 1991245299712, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid0p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid1p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid3p4, vdev_max_asize: 1991247921152, vdev_asize: 1991247921152, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid2p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid5p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > ^C > > borg-new /home/ler $ > > borg-new /home/ler $ sudo zpool list -v > Password: > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > zroot 10.8T 94.3G 10.7T 16.0E 0% 0% 1.00x ONLINE - > raidz1 10.8T 94.3G 10.7T 16.0E 0% 0% > mfid4p4 - - - - - - > mfid0p4 - - - - - - > mfid1p4 - - - - - - > mfid3p4 - - - - - - > mfid2p4 - - - - - - > mfid5p4 - - - - - - > borg-new /home/ler $ > > On 01/31/2017 2:37 pm, Steven Hartland wrote: In that case based on your zpool history I suspect that the original mfid4p4 was the same size as mfid0p4 (1991246348288) but its been replaced with a drive which is (1991245299712), slightly smaller. > > This smaller size results in a max_asize of 1991245299712 * 6 instead of original 1991246348288* 6. > > Now given the way min_asize (the value used to check if the device size is acceptable) is rounded to the the nearest metaslab I believe that replace would be allowed. > https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c#L4947 > > Now the problem is that on open the calculated asize is only updated if its expanding: > https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c#L1424 > > The updated dtrace file outputs vdev_min_asize which should confirm my suspicion about why the replace was allowed. > > On 31/01/2017 19:05, Larry Rosenman wrote: > > I've replaced some disks due to failure, and some of the pariition sizes are different. > > autoexpand is off: > > borg-new /home/ler $ zpool get all zroot > NAME PROPERTY VALUE SOURCE > zroot size 10.8T - > zroot capacity 0% - > zroot altroot - default > zroot health ONLINE - > zroot guid 11945658884309024932 default > zroot version - default > zroot bootfs zroot/ROOT/default local > zroot delegation on default > zroot autoreplace off default > zroot cachefile - default > zroot failmode wait default > zroot listsnapshots off default > zroot autoexpand off default > zroot dedupditto 0 default > zroot dedupratio 1.00x - > zroot free 10.7T - > zroot allocated 94.3G - > zroot readonly off - > zroot comment - default > zroot expandsize 16.0E - > zroot freeing 0 default > zroot fragmentation 0% - > zroot leaked 0 default > zroot feature@async_destroy enabled local > zroot feature@empty_bpobj active local > zroot feature@lz4_compress active local > zroot feature@multi_vdev_crash_dump enabled local > zroot feature@spacemap_histogram active local > zroot feature@enabled_txg active local > zroot feature@hole_birth active local > zroot feature@extensible_dataset enabled local > zroot feature@embedded_data active local > zroot feature@bookmarks enabled local > zroot feature@filesystem_limits enabled local > zroot feature@large_blocks enabled local > zroot feature@sha512 enabled local > zroot feature@skein enabled local > borg-new /home/ler $ > > borg-new /home/ler $ gpart show > => 40 3905945520 mfid0 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid1 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid2 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid3 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 16777216 3 freebsd-swap (8.0G) > 16779880 3889165680 4 freebsd-zfs (1.8T) > > => 40 3905945520 mfid5 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid4 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889160192 4 freebsd-zfs (1.8T) > 3905941504 4056 - free - (2.0M) > > borg-new /home/ler $ > > this system was built last week, and I **CAN** rebuild it if necessary, but I didn't do anything strange (so I thought :) ) > > On 01/31/2017 12:30 pm, Steven Hartland wrote: Your issue is the reported vdev_max_asize > vdev_asize: > vdev_max_asize: 11947471798272 > vdev_asize: 11947478089728 > > max asize is smaller than asize by 6291456 > > For raidz1 Xsize should be the smallest disk Xsize * disks so: > 1991245299712 * 6 = 11947471798272 > > So your max asize looks right but asize looks too big > > Expand Size is calculated by: > if (vd->vdev_aux == NULL && tvd != NULL && vd->vdev_max_asize != 0) { > vs->vs_esize = P2ALIGN(vd->vdev_max_asize - vd->vdev_asize, > 1ULL << tvd->vdev_ms_shift); > } > > So the question is why is asize too big? > > Given you seem to have some random disk sizes do you have auto expand turned on? > > On 31/01/2017 17:39, Larry Rosenman wrote: vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: 11947478089728 -- Larry Rosenman http://people.freebsd.org/~ler [1] Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 -- Larry Rosenman http://people.freebsd.org/~ler [1] Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 -- Larry Rosenman http://people.freebsd.org/~ler Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 Links: ------ [1] http://people.freebsd.org/%7Eler From owner-freebsd-fs@freebsd.org Tue Jan 31 21:47:16 2017 Return-Path: <owner-freebsd-fs@freebsd.org> Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF579CCA2DD for <freebsd-fs@mailman.ysv.freebsd.org>; Tue, 31 Jan 2017 21:47:16 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x22c.google.com (mail-wm0-x22c.google.com [IPv6:2a00:1450:400c:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4052C1C42 for <freebsd-fs@freebsd.org>; Tue, 31 Jan 2017 21:47:16 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x22c.google.com with SMTP id r141so8851877wmg.1 for <freebsd-fs@freebsd.org>; Tue, 31 Jan 2017 13:47:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=KtjSVuxS/F1T0pgh6geYC6EQ6w0GPJDmu58DLtkB/kQ=; b=0k8FWkX3dwK47uQ1Ce151DXSy2jlCWuk7dxd/+dWFkqwqvOBM0Y4OLTBu1AcyhxbvQ RC3PEgG5/+/QRz10UOnj6JclnRl2ZuxT8v4QPCLnx2raVgKVsYgHyUj3eILOOBSVRj9U HqWAinrP1fZ4CYhzFGfJrEtrRSrVbsdy2uFbAUdkKVyIN92hpsaFrmVjTYQRksyp31Ne yyaM7tqdvGy/DE56jav3P78SxRkGqTYIv5CC6tZ5MDUZrLnrHg+Z9USD3P/BgsNXYZPs 29hAOqcNhPGcvF3ijLcCAj7XK5pJXBtd810uuI6sEhdvVFmUEI3usw5M475PsXT4sqtF TJcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=KtjSVuxS/F1T0pgh6geYC6EQ6w0GPJDmu58DLtkB/kQ=; b=f/NJBxge313Ky60u4UgieU4dRPeKktj+BggOsON8keh7RLEGlxLIB6NvXTL0QOzL0T nzOg2Ky7dK9KSjyPNDEUfCicAZNfPyxcQvY+iikZMQ+AlK+SasLHwM0xuTGEIl0wQN5h ThJZY7UW+SHxSGJhH98ksMk6GmiFLbai9TpbCvGjBGAMoqPluJQQodbnekfPprfqlLFt Cq4EuUXKY/saORLDLeH6fbh3pY2I9dyw9xRX57HSZKooTAZMydz0yC+wVMdARafiA00P QaJXMnhjRbFA4ph2fnxtReYloRggCql/bcAnV1lirtn7VhN//sM5yHi2FLLvGndj6VfW LHDA== X-Gm-Message-State: AIkVDXIUnai42DpNV7bfMDxEDa02u2p499B/RWv6RydqqNvAg8wLxIGgPsrqhwNul/dw+O8H X-Received: by 10.223.171.149 with SMTP id s21mr25145187wrc.64.1485899234139; Tue, 31 Jan 2017 13:47:14 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id x39sm30518733wrb.3.2017.01.31.13.47.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 31 Jan 2017 13:47:13 -0800 (PST) Subject: Re: 16.0E ExpandSize? -- New Server To: Larry Rosenman <ler@FreeBSD.org> References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> <ce5a1d39612d694077accda33266a3ab@FreeBSD.org> <ad07e84e-f297-362a-1398-c5503bb56a8d@multiplay.co.uk> <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> <76fc9505-f681-0de0-fe0c-5624b29de321@multiplay.co.uk> <22e1bfc5840d972cf93643733682cda1@FreeBSD.org> <f2600a53-0dc1-9f41-1405-ed22d96d30cf@multiplay.co.uk> <8a710dc75c129f58b0372eeaeca575b5@FreeBSD.org> <aef02eb0-0888-6fea-a4b8-4033ca56f4a3@multiplay.co.uk> <d3181bd00c827fb99fbcebe6fe097ef8@FreeBSD.org> Cc: Freebsd fs <freebsd-fs@freebsd.org> From: Steven Hartland <killing@multiplay.co.uk> Message-ID: <a3d78923-5046-11c8-daea-713eacf47bd2@multiplay.co.uk> Date: Tue, 31 Jan 2017 21:47:14 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: <d3181bd00c827fb99fbcebe6fe097ef8@FreeBSD.org> Content-Type: multipart/mixed; boundary="------------3F69C0E438D93AD11208EDA7" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems <freebsd-fs.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-fs>, <mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs/>; List-Post: <mailto:freebsd-fs@freebsd.org> List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-fs>, <mailto:freebsd-fs-request@freebsd.org?subject=subscribe> X-List-Received-Date: Tue, 31 Jan 2017 21:47:16 -0000 This is a multi-part message in MIME format. --------------3F69C0E438D93AD11208EDA7 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Hmm, looks like there's also a bug in the way vdev_min_asize is calculated for raidz as it can and has resulted in child min_asize which won't provided enough space for the parent due to the use of unrounded integer division. 1981411579221 * 6 = 11888469475326 < 11888469475328 You should have vdev_min_asize: 1981411579222 for your children. Updated patch attached, however calculation still isn't 100% reversible so may need work, however it does now ensure that the children will provide enough capacity for min_asize even if all of them are shrunk to their individual min_asize, which I believe previously may not have been the case. This isn't related to the incorrect EXPANDSZ output, but would be good if you could confirm it doesn't cause any issues for your pool given its state. On 31/01/2017 21:00, Larry Rosenman wrote: > > borg-new /home/ler $ sudo ./vdev-stats.d > Password: > vdev_path: n/a, vdev_max_asize: 0, vdev_asize: 0, vdev_min_asize: 0 > vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: > 11947478089728, vdev_min_asize: 11888469475328 > vdev_path: /dev/mfid4p4, vdev_max_asize: 1991245299712, vdev_asize: > 1991245299712, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid0p4, vdev_max_asize: 1991246348288, vdev_asize: > 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid1p4, vdev_max_asize: 1991246348288, vdev_asize: > 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid3p4, vdev_max_asize: 1991247921152, vdev_asize: > 1991247921152, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid2p4, vdev_max_asize: 1991246348288, vdev_asize: > 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid5p4, vdev_max_asize: 1991246348288, vdev_asize: > 1991246348288, vdev_min_asize: 1981411579221 > ^C > > borg-new /home/ler $ > > > borg-new /home/ler $ sudo zpool list -v > Password: > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > zroot 10.8T 94.3G 10.7T 16.0E 0% 0% 1.00x ONLINE - > raidz1 10.8T 94.3G 10.7T 16.0E 0% 0% > mfid4p4 - - - - - - > mfid0p4 - - - - - - > mfid1p4 - - - - - - > mfid3p4 - - - - - - > mfid2p4 - - - - - - > mfid5p4 - - - - - - > borg-new /home/ler $ > > > On 01/31/2017 2:37 pm, Steven Hartland wrote: > >> In that case based on your zpool history I suspect that the original >> mfid4p4 was the same size as mfid0p4 (1991246348288) but its been >> replaced with a drive which is (1991245299712), slightly smaller. >> >> This smaller size results in a max_asize of 1991245299712 * 6 instead >> of original 1991246348288* 6. >> >> Now given the way min_asize (the value used to check if the device >> size is acceptable) is rounded to the the nearest metaslab I believe >> that replace would be allowed. >> https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c#L4947 >> >> Now the problem is that on open the calculated asize is only updated >> if its expanding: >> https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c#L1424 >> >> The updated dtrace file outputs vdev_min_asize which should confirm >> my suspicion about why the replace was allowed. >> >> On 31/01/2017 19:05, Larry Rosenman wrote: >>> >>> I've replaced some disks due to failure, and some of the pariition >>> sizes are different. >>> >>> >>> autoexpand is off: >>> >>> borg-new /home/ler $ zpool get all zroot >>> NAME PROPERTY VALUE SOURCE >>> zroot size 10.8T - >>> zroot capacity 0% - >>> zroot altroot - default >>> zroot health ONLINE - >>> zroot guid 11945658884309024932 default >>> zroot version - default >>> zroot bootfs zroot/ROOT/default local >>> zroot delegation on default >>> zroot autoreplace off default >>> zroot cachefile - default >>> zroot failmode wait default >>> zroot listsnapshots off default >>> zroot autoexpand off default >>> zroot dedupditto 0 default >>> zroot dedupratio 1.00x - >>> zroot free 10.7T - >>> zroot allocated 94.3G - >>> zroot readonly off - >>> zroot comment - default >>> zroot expandsize 16.0E - >>> zroot freeing 0 default >>> zroot fragmentation 0% - >>> zroot leaked 0 default >>> zroot feature@async_destroy enabled local >>> zroot feature@empty_bpobj active local >>> zroot feature@lz4_compress active local >>> zroot feature@multi_vdev_crash_dump enabled local >>> zroot feature@spacemap_histogram active local >>> zroot feature@enabled_txg active local >>> zroot feature@hole_birth active local >>> zroot feature@extensible_dataset enabled local >>> zroot feature@embedded_data active local >>> zroot feature@bookmarks enabled local >>> zroot feature@filesystem_limits enabled local >>> zroot feature@large_blocks enabled local >>> zroot feature@sha512 enabled local >>> zroot feature@skein enabled local >>> borg-new /home/ler $ >>> >>> >>> borg-new /home/ler $ gpart show >>> => 40 3905945520 mfid0 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid1 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid2 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid3 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 16777216 3 freebsd-swap (8.0G) >>> 16779880 3889165680 4 freebsd-zfs (1.8T) >>> >>> => 40 3905945520 mfid5 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid4 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889160192 4 freebsd-zfs (1.8T) >>> 3905941504 4056 - free - (2.0M) >>> >>> borg-new /home/ler $ >>> >>> >>> this system was built last week, and I **CAN** rebuild it if >>> necessary, but I didn't do anything strange (so I thought :) ) >>> >>> >>> >>> >>> On 01/31/2017 12:30 pm, Steven Hartland wrote: >>> >>> Your issue is the reported vdev_max_asize > vdev_asize: >>> vdev_max_asize: 11947471798272 >>> vdev_asize: 11947478089728 >>> >>> max asize is smaller than asize by 6291456 >>> >>> For raidz1 Xsize should be the smallest disk Xsize * disks so: >>> 1991245299712 * 6 = 11947471798272 >>> >>> So your max asize looks right but asize looks too big >>> >>> Expand Size is calculated by: >>> if (vd->vdev_aux == NULL && tvd != NULL && vd->vdev_max_asize != >>> 0) { >>> vs->vs_esize = P2ALIGN(vd->vdev_max_asize - vd->vdev_asize, >>> 1ULL << tvd->vdev_ms_shift); >>> } >>> >>> So the question is why is asize too big? >>> >>> Given you seem to have some random disk sizes do you have auto >>> expand turned on? >>> >>> On 31/01/2017 17:39, Larry Rosenman wrote: >>> >>> vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: >>> 11947478089728 >>> >>> >>> -- >>> Larry Rosenman http://people.freebsd.org/~ler >>> <http://people.freebsd.org/%7Eler>; >>> Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org >>> <mailto:ler@FreeBSD.org> >>> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 > > > -- > Larry Rosenman http://people.freebsd.org/~ler > <http://people.freebsd.org/%7Eler>; > Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org > <mailto:ler@FreeBSD.org> > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 --------------3F69C0E438D93AD11208EDA7 Content-Type: text/plain; charset=UTF-8; name="expand-sz-16e.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="expand-sz-16e.patch" SW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMv dmRldi5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMv dXRzL2NvbW1vbi9mcy96ZnMvdmRldi5jCShyZXZpc2lvbiAzMTMwMDMpCisrKyBzeXMvY2Rk bC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3ZkZXYuYwkod29ya2lu ZyBjb3B5KQpAQCAtMjI5LDcgKzIyOSw4IEBACiAJICogc28gZWFjaCBjaGlsZCBtdXN0IHBy b3ZpZGUgYXQgbGVhc3QgMS9OdGggb2YgaXRzIGFzaXplLgogCSAqLwogCWlmIChwdmQtPnZk ZXZfb3BzID09ICZ2ZGV2X3JhaWR6X29wcykKLQkJcmV0dXJuIChwdmQtPnZkZXZfbWluX2Fz aXplIC8gcHZkLT52ZGV2X2NoaWxkcmVuKTsKKwkJcmV0dXJuICgocHZkLT52ZGV2X21pbl9h c2l6ZSArIHB2ZC0+dmRldl9jaGlsZHJlbiAtIDEpIC8KKwkJICAgIHB2ZC0+dmRldl9jaGls ZHJlbik7CiAKIAlyZXR1cm4gKHB2ZC0+dmRldl9taW5fYXNpemUpOwogfQpAQCAtMTM3Nyw3 ICsxMzc4LDcgQEAKIAl2ZC0+dmRldl9wc2l6ZSA9IHBzaXplOwogCiAJLyoKLQkgKiBNYWtl IHN1cmUgdGhlIGFsbG9jYXRhYmxlIHNpemUgaGFzbid0IHNocnVuay4KKwkgKiBNYWtlIHN1 cmUgdGhlIGFsbG9jYXRhYmxlIHNpemUgaGFzbid0IHNocnVuayB0b28gbXVjaC4KIAkgKi8K IAlpZiAoYXNpemUgPCB2ZC0+dmRldl9taW5fYXNpemUpIHsKIAkJdmRldl9zZXRfc3RhdGUo dmQsIEJfVFJVRSwgVkRFVl9TVEFURV9DQU5UX09QRU4sCkBAIC0xNDIwLDEwICsxNDIxLDE5 IEBACiAJICogSWYgYWxsIGNoaWxkcmVuIGFyZSBoZWFsdGh5IGFuZCB0aGUgYXNpemUgaGFz IGluY3JlYXNlZCwKIAkgKiB0aGVuIHdlJ3ZlIGV4cGVyaWVuY2VkIGR5bmFtaWMgTFVOIGdy b3d0aC4gIElmIGF1dG9tYXRpYwogCSAqIGV4cGFuc2lvbiBpcyBlbmFibGVkIHRoZW4gdXNl IHRoZSBhZGRpdGlvbmFsIHNwYWNlLgorCSAqIAorCSAqIE90aGVyd2lzZSBpZiBhc2l6ZSBo YXMgcmVkdWNlZCwgc2hyaW5rIHRvIGVuc3VyZSB0aGF0CisJICogY2FsY3VsYXRpb25zIGJh c2VkIG9mIG1heF9hc2l6ZSBhbmQgYXNpemUgZS5nLiBlc2l6ZSBhcmUKKwkgKiBhbHdheXMg dmFsaWQuIFRoaXMgaXMgc2FmZSBhcyB3ZSd2ZSBhbHJlYWR5IHZhbGlkYXRlZCB0aGF0CisJ ICogYXNpemUgaXMgbm90IGxlc3MgdGhhbiBtaW5fYXNpemUuCiAJICovCi0JaWYgKHZkLT52 ZGV2X3N0YXRlID09IFZERVZfU1RBVEVfSEVBTFRIWSAmJiBhc2l6ZSA+IHZkLT52ZGV2X2Fz aXplICYmCi0JICAgICh2ZC0+dmRldl9leHBhbmRpbmcgfHwgc3BhLT5zcGFfYXV0b2V4cGFu ZCkpCi0JCXZkLT52ZGV2X2FzaXplID0gYXNpemU7CisJaWYgKHZkLT52ZGV2X3N0YXRlID09 IFZERVZfU1RBVEVfSEVBTFRIWSkgeworCQlpZiAoYXNpemUgPiB2ZC0+dmRldl9hc2l6ZSAm JgorCQkgICAgKHZkLT52ZGV2X2V4cGFuZGluZyB8fCBzcGEtPnNwYV9hdXRvZXhwYW5kKSkK KwkJCXZkLT52ZGV2X2FzaXplID0gYXNpemU7CisJCWVsc2UgaWYgKGFzaXplIDwgdmQtPnZk ZXZfYXNpemUpCisJCQl2ZC0+dmRldl9hc2l6ZSA9IGFzaXplOworCX0KIAogCXZkZXZfc2V0 X21pbl9hc2l6ZSh2ZCk7CiAK --------------3F69C0E438D93AD11208EDA7--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?530ac81cbcdf268919a887f3ceff41d8>