Date: Thu, 24 Jul 2014 19:56:30 -0400 From: Allan Jude <allanjude@freebsd.org> To: freebsd-current@freebsd.org Subject: Re: zfs send/recv: STILL invalid Backup Stream Message-ID: <53D19D2E.30908@freebsd.org> In-Reply-To: <6e382c7d7efc8b4ddc3b770985eac1e0@thebighonker.lerctr.org> References: <62315eb454a95db636b7764aad3c0f9b@thebighonker.lerctr.org> <53D1448C.40908@freebsd.org> <d0612ef9399b4e95bd09e5271c91505b@thebighonker.lerctr.org> <eefa10e173e2e23a88c85c38221a1c22@thebighonker.lerctr.org> <53D15438.6040105@freebsd.org> <0a9cd451c3b4304d2b9d899fcb3decc3@thebighonker.lerctr.org> <cc635a377e817cb57d27650ed3e558f4@thebighonker.lerctr.org> <3d2aac84d962a703fbf56a864ba5f19c@mailbox.ijs.si> <c93ef94a8e9e076b86a65f4986d0e30d@thebighonker.lerctr.org> <53D1678C.4000007@freebsd.org> <6e382c7d7efc8b4ddc3b770985eac1e0@thebighonker.lerctr.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --rV4TUiNK3Hrv04cCwBD3XIjxGtjW4e5TW Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 2014-07-24 16:11, Larry Rosenman wrote: > On 2014-07-24 15:07, Allan Jude wrote: >> On 2014-07-24 15:57, Larry Rosenman wrote: >>> On 2014-07-24 14:53, Mark Martinec wrote: >>>> 2014-07-24 21:31, Larry Rosenman wrote: >>>>> borg.lerctr.org /home/ler # zxfer -dFkPvs -g 376 -O >>>>> root@tbh.lerctr.org -R zroot zroot/backups/TBH >>>>> Creating recursive snapshot zroot@zxfer_26699_20140724135840. >>>>> Checking grandfather status of all snapshots marked for deletion...= >>>>> Grandfather check passed. >>>>> Sending zroot@zxfer_26699_20140724135840 to zroot/backups/TBH/zroot= =2E >>>>> Sending zroot/ROOT@zxfer_26699_20140724135840 to >>>>> zroot/backups/TBH/zroot/ROOT. >>>>> Sending zroot/ROOT/default@zxfer_23699_20140724134435 to >>>>> zroot/backups/TBH/zroot/ROOT/default. >>>>> Sending zroot/ROOT/default@zxfer_26699_20140724135840 to >>>>> zroot/backups/TBH/zroot/ROOT/default. >>>>> (incremental to zroot/ROOT/default@zxfer_23699_20140724134435.) >>>>> Sending zroot/home@zxfer_26699_20140724135840 to >>>>> zroot/backups/TBH/zroot/home. >>>> >>>>> Write failed: Cannot allocate memory >>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>>> >>>>> cannot receive new filesystem stream: invalid backup stream >>>>> Error when zfs send/receiving. >>>>> borg.lerctr.org /home/ler # >>>>> >>>>> well that's different....... >>>> >>>> Sounds familiar, check my posting of today and links therein: >>>> >>>> http://lists.freebsd.org/pipermail/freebsd-net/2014-July/039347.ht= ml >>>> >>>> Mark >>> I'm not using netgraph to the best of my knowledge.... >>> and the only fails on the SENDING host are: >>> 8 Bucket: 64, 0, 41, 3555, 257774, 11, = 0 >>> 12 Bucket: 96, 0, 96, 2569, 123653, 0, = 0 >>> 16 Bucket: 128, 0, 17195, 506, 215573, 0, = 0 >>> 32 Bucket: 256, 0, 340, 4670, 900638, 50, = 0 >>> 64 Bucket: 512, 0, 10691, 365,=20 >>> 546888,185232, 0 >>> 128 Bucket: 1024, 0, 3563, 905, 348419, 0, = 0 >>> 256 Bucket: 2048, 0, 2872, 162,=20 >>> 249995,59834, 0 >>> vmem btag: 56, 0, 192811, 51500, 502264,1723, = 0 >>> >>> >> >> I regularly use zxfer to transfer 500+ GiB datasets over the internet.= >> This week I actually replicated a 2.1 TiB dataset with zxfer without >> issue. >> >> I wonder which thing is running out of memory. Is there a delay while = it >> is 'running out of memory', or does it fail immediately? Does running >> top while it is working on running out of memory reveal anything? >> >> I would expect to use up a lot of memory while doing deduplication, bu= t >> not otherwise. >> >> Note: I most often use openssh-portable rather than base ssh for >> replication, as I enable the nonecipher to reduce CPU usage, and adjus= t >> the TcpRcvBuf upwards to actually saturate a gigabit over the internet= =2E >=20 > I wasn't watching exactly what it was doing, but the sending box has 16= G > and 18G Swap and swap > has NOT been touched. >=20 > last pid: 74288; load averages: 4.70, 5.61, 5.91 up 1+03:14:18=20 > 15:10:44 > 115 processes: 3 running, 112 sleeping > CPU: 0.6% user, 33.3% nice, 0.6% system, 0.1% interrupt, 65.4% idle > Mem: 847M Active, 761M Inact, 14G Wired, 4616K Cache, 357M Free > ARC: 12G Total, 6028M MFU, 5281M MRU, 3152K Anon, 120M Header, 688M Oth= er > Swap: 18G Total, 18G Free >=20 > so I have zero idea where to go here. >=20 >=20 Most ZFS memory usage is 'wired' and so cannot be swapped, so lack of swap activity isn't a good indicator. --=20 Allan Jude --rV4TUiNK3Hrv04cCwBD3XIjxGtjW4e5TW Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJT0Z0uAAoJEJrBFpNRJZKflhEP/1ZzsIUWVbBbMf/aKKVcWfKO mnHWGaNt0aVueau/+QUq1L5P2AKyY5/CVXV1mwFP3dGSJZxz0kyUINSPI08ijtig CrZODJcXXo+nwWW7nxG64bwlkzDgag+zmhxwyUCSyMMoo+bUrsDrq8mN64kEaU2G IZcaHSdFeELtHnv8fD2qHFLJdiMGqMGiCEVHc8NlEeq2QvVwBhRc/70HzxhX82hR YZtUZ0AWH/0G8W8EZwAW4LTFc1GMtwwWJGpV432XcVDhYI9E4kYON/qrGxmlSmGN OORrEILQMcfQpxrI/zUtV8+YmcOlywz8K63OjtSKc/GYa5eBLEUsY+5HCpNqIzif nQ7xJUE8bs9Msb8EiwnB0yU1j9MvCxBL33awhWUw4FrMncm+0FCeMrcnWNPTgzys bPQ/gB8itWx4PxAt6/dIfZjDd0HqV7Y1ugqDh7z7b0GtRQ1w4R6i6MspvcARnES2 RA2jfU7kScFpAafXevJmLTeBOa90ieQ+SW6rmw5VRO0qocB0snn8RMqtyxehCN0T 4KfpoigZTJOKN7L9G7bt3ch6ElpFfd1y/5IiAru/5urD1nnU0l78V/doPSiuSM1/ 1JS9RMIIJX9RtU9LGkH1azOxj9fMMYvcc5r8W0vKpzdmETmKxjo0pcYPaJpHG6za f6OAGZmYh4u0FCgOV68W =JL+o -----END PGP SIGNATURE----- --rV4TUiNK3Hrv04cCwBD3XIjxGtjW4e5TW--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?53D19D2E.30908>