Date: Wed, 15 Oct 2014 05:52:08 +0100 From: "Steven Hartland" <killing@multiplay.co.uk> To: "Steven Hartland" <killing@multiplay.co.uk>, "Mark Martinec" <Mark.Martinec+freebsd@ijs.si>, <freebsd-fs@freebsd.org>, <freebsd-stable@freebsd.org> Subject: Re: zpool import hangs when out of space - Was: zfs pool import hangs on [tx->tx_sync_done_cv] Message-ID: <8F4036C658724468B34B20CCBA658E43@multiplay.co.uk> References: <54372173.1010100@ijs.si> <644FA8299BF848E599B82D2C2C298EA7@multiplay.co.uk> <54372EBA.1000908@ijs.si> <DE7DD7A94E9B4F1FBB3AFF57EDB47C67@multiplay.co.uk> <543731F3.8090701@ijs.si> <543AE740.7000808@ijs.si> <A5BA41116A7F4B23A9C9E469C4146B99@multiplay.co.uk> <CAHM0Q_N%2BC=3qgUnyDkEugOFcL=J8gBjbTg8v45Vz3uT=e=Fn2g@mail.gmail.com> <6E01BBEDA9984CCDA14F290D26A8E14D@multiplay.co.uk> <CAHM0Q_OpV2sAQQAH6Cj_=yJWAOt8pTPWQ-m45JSiXDpBwT6WTA@mail.gmail.com> <E2E24A91B8B04C2DBBBC7E029A12BD05@multiplay.co.uk> <CAHM0Q_Oeka25-kdSDRC2evS1R8wuQ0_XgbcdZCjS09aXJ9_WWQ@mail.gmail.com> <14ADE02801754E028D9A0EAB4A16527E@multiplay.co.uk> <543C3C47.4010208@ijs.si> <E3C3C359999140B48943A0E1A04F83A9@multiplay.co.uk> <CAHM0Q_O7LNBiQAEjygANa%2B0rqm9cywjTPbNXabB4TePfEHAZsA@mail.gmail.com> <A7771879317F4194A1D5E4921CD33593@multiplay.co.uk> <138CF459AA0B41EB8CB4E11B3DE932CF@multiplay.co.uk> <543D0953.1070604@ijs.si> <C7A0928FD72346DD95970145BD2ADC8B@multiplay.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
----- Original Message ----- From: "Steven Hartland" <killing@multiplay.co.uk> To: "Mark Martinec" <Mark.Martinec+freebsd@ijs.si>; <freebsd-fs@freebsd.org>; <freebsd-stable@freebsd.org> Sent: Tuesday, October 14, 2014 12:40 PM Subject: Re: zpool import hangs when out of space - Was: zfs pool import hangs on [tx->tx_sync_done_cv] > ----- Original Message ----- > From: "Mark Martinec" <Mark.Martinec+freebsd@ijs.si> > > >> On 10/14/2014 13:19, Steven Hartland wrote: >>> Well interesting issue I left this pool alone this morning literally doing >>> nothing, and its now out of space. >>> zpool list >>> NAME SIZE ALLOC FREE FRAG EXPANDSZ CAP DEDUP HEALTH >>> ALTROOT >>> sys1boot 3.97G 3.97G 190K 0% - 99% 1.00x ONLINE - >>> sys1copy 3.97G 3.97G 8K 0% - 99% 1.00x ONLINE - >>> >>> There's something very wrong here as nothing has been accessing the pool. >>> >>> pool: zfs >>> state: ONLINE >>> status: One or more devices are faulted in response to IO failures. >>> action: Make sure the affected devices are connected, then run 'zpool >>> clear'. >>> see: http://illumos.org/msg/ZFS-8000-HC >>> scan: none requested >>> config: >>> >>> NAME STATE READ WRITE CKSUM >>> zfs ONLINE 0 2 0 >>> md1 ONLINE 0 0 0 >>> >>> I tried destroying the pool and ever that failed, presumably because >>> the pool has suspended IO. >> >> That's exactly how trouble started here. Got the >> "One or more devices are faulted in response to IO failures" >> on all three small cloned boot pools one day, out of the blue. >> There was no activity there, except for periodic snapshoting >> every 10 minutes. > > Yer this isn't fragmentation, this is something else. I've started a > thread on the openzfs list to discuss this as theres something quite > odd going on. After bisecting the kernel versions in stable/10 the problem commit appears to be: https://svnweb.freebsd.org/base?view=revision&revision=268650 Removing it or using a pool without async_destory enabled prevents the leak. More debugging tomorrow. Regards steve
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?8F4036C658724468B34B20CCBA658E43>