Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 29 Jan 2017 22:57:42 +0000
From:      bugzilla-noreply@freebsd.org
To:        freebsd-amd64@FreeBSD.org
Subject:   [Bug 216586] zfs panic: sa.sa_magic == 0x2F505A in zfs_space_delta_cb()
Message-ID:  <bug-216586-6@https.bugs.freebsd.org/bugzilla/>

next in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216586

            Bug ID: 216586
           Summary: zfs panic: sa.sa_magic =3D=3D 0x2F505A in
                    zfs_space_delta_cb()
           Product: Base System
           Version: 10.3-STABLE
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: freebsd-bugs@FreeBSD.org
          Reporter: johannes@jo-t.de
                CC: freebsd-amd64@FreeBSD.org
                CC: freebsd-amd64@FreeBSD.org

While rsync'ing some files, the system eventually panics with:

> panic: solaris assert: sa.sa_magic =3D=3D 0x2F505A (0x51786a88 =3D=3D 0x2=
f505a), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/c=
ommon/fs/zfs/zfs_vfsops.c, line: 610

The backtrace is:

> #0 0xffffffff804f23c0 at kdb_backtrace+0x60
> #1 0xffffffff804b3824 at vpanic+0x124
> #2 0xffffffff804b36f3 at panic+0x43
> #3 0xffffffff8109122f at assfail3+0x2f
> #4 0xffffffff80e4d417 at zfs_space_delta_cb+0x107
> #5 0xffffffff80dc156f at dmu_objset_userquota_get_ids+0x36f
> #6 0xffffffff80dc952f at dnode_sync+0x9f
> #7 0xffffffff80dc0aeb at dmu_objset_sync_dnodes+0x2b
> #8 0xffffffff80dc0920 at dmu_objset_sync+0x220
> #9 0xffffffff80dd241a at dsl_dataset_sync+0x1ca
> #10 0xffffffff80ddabca at dsl_pool_sync+0x7a
> #11 0xffffffff80dfa77d at spa_sync+0x57d
> #12 0xffffffff80e03f56 at txg_sync_thread+0x3a6
> #13 0xffffffff8048037a at fork_exit+0x9a
> #14 0xffffffff806ccfde at fork_trampoline+0xe

This happens on:
> FreeBSD fred 10.3-STABLE FreeBSD 10.3-STABLE #1 r308740: Sat Nov 19 21:15=
:27 GMT 2016 root@fred:/usr/obj/usr/src/sys/FRED amd64

and as well on a newer version:
> FreeBSD fred 10.3-STABLE FreeBSD 10.3-STABLE #0 r312924: Sun Jan 29 01:45=
:50 GMT 2017     root@fred:/usr/obj/usr/src/sys/FRED  amd64

The machine has ECC RAM, and a zpool scrub did not report any errors. The p=
ool
is a 2-way-mirror.

The actual value of sa.sa_magic changes (not everytime though): e.g. 0x5178=
6a88
or 0x5178691e.

The code in question at zfs_vfsops.c, line 610 is:

> static int
> zfs_space_delta_cb(dmu_object_type_t bonustype, void *data,
>     uint64_t *userp, uint64_t *groupp)
> {
> 	/*
> 	 * Is it a valid type of object to track?
> 	 */
> 	if (bonustype !=3D DMU_OT_ZNODE && bonustype !=3D DMU_OT_SA)
> 		return (SET_ERROR(ENOENT));
>=20
> 	/*
> 	 * If we have a NULL data pointer
> 	 * then assume the id's aren't changing and
> 	 * return EEXIST to the dmu to let it know to
> 	 * use the same ids
> 	 */
> 	if (data =3D=3D NULL)
> 		return (SET_ERROR(EEXIST));
>=20
> 	if (bonustype =3D=3D DMU_OT_ZNODE) {
> 		znode_phys_t *znp =3D data;
> 		*userp =3D znp->zp_uid;
> 		*groupp =3D znp->zp_gid;
> 	} else {
> 		int hdrsize;
> 		sa_hdr_phys_t *sap =3D data;
> 		sa_hdr_phys_t sa =3D *sap;
> 		boolean_t swap =3D B_FALSE;
>=20
> 		ASSERT(bonustype =3D=3D DMU_OT_SA);
>=20
> 		if (sa.sa_magic =3D=3D 0) {
> 			/*
> 			 * This should only happen for newly created
> 			 * files that haven't had the znode data filled
> 			 * in yet.
> 			 */
> 			*userp =3D 0;
> 			*groupp =3D 0;
> 			return (0);
> 		}
> 		if (sa.sa_magic =3D=3D BSWAP_32(SA_MAGIC)) {
> 			sa.sa_magic =3D SA_MAGIC;
> 			sa.sa_layout_info =3D BSWAP_16(sa.sa_layout_info);
> 			swap =3D B_TRUE;
> 		} else {
> /*610*/		VERIFY3U(sa.sa_magic, =3D=3D, SA_MAGIC);   // FAILS
> 		}
>=20
> 		hdrsize =3D sa_hdrsize(&sa);
> 		VERIFY3U(hdrsize, >=3D, sizeof (sa_hdr_phys_t));
> 		*userp =3D *((uint64_t *)((uintptr_t)data + hdrsize +
> 		    SA_UID_OFFSET));
> 		*groupp =3D *((uint64_t *)((uintptr_t)data + hdrsize +
> 		    SA_GID_OFFSET));
> 		if (swap) {
> 			*userp =3D BSWAP_64(*userp);
> 			*groupp =3D BSWAP_64(*groupp);
> 		}
> 	}
> 	return (0);
> }

The stacktrace has dmu_objset_userquota_get_ids listed. And the target file
system (into which rsync is writing) has quota enabled:

> NAME                        PROPERTY              VALUE                  =
SOURCE
> freddata/backup/alexs-imac  type                  filesystem             -
> freddata/backup/alexs-imac  creation              Wed Mar  7 18:01 2012  -
> freddata/backup/alexs-imac  used                  521G                   -
> freddata/backup/alexs-imac  available             29.2G                  -
> freddata/backup/alexs-imac  referenced            213G                   -
> freddata/backup/alexs-imac  compressratio         1.11x                  -
> freddata/backup/alexs-imac  mounted               yes                    -
> freddata/backup/alexs-imac  quota                 550G                   =
local
> freddata/backup/alexs-imac  reservation           300G                   =
local
> freddata/backup/alexs-imac  recordsize            128K                   =
default
> freddata/backup/alexs-imac  mountpoint            /backup/alexs-imac     =
inherited from freddata/backup
> freddata/backup/alexs-imac  sharenfs              off                    =
default
> freddata/backup/alexs-imac  checksum              fletcher4              =
inherited from freddata
> freddata/backup/alexs-imac  compression           lz4                    =
local
> freddata/backup/alexs-imac  atime                 off                    =
inherited from freddata/backup
> freddata/backup/alexs-imac  devices               on                     =
default
> freddata/backup/alexs-imac  exec                  on                     =
default
> freddata/backup/alexs-imac  setuid                on                     =
default
> freddata/backup/alexs-imac  readonly              off                    =
default
> freddata/backup/alexs-imac  jailed                off                    =
default
> freddata/backup/alexs-imac  snapdir               hidden                 =
default
> freddata/backup/alexs-imac  aclmode               discard                =
default
> freddata/backup/alexs-imac  aclinherit            restricted             =
default
> freddata/backup/alexs-imac  canmount              on                     =
default
> freddata/backup/alexs-imac  xattr                 off                    =
temporary
> freddata/backup/alexs-imac  copies                1                      =
default
> freddata/backup/alexs-imac  version               5                      -
> freddata/backup/alexs-imac  utf8only              off                    -
> freddata/backup/alexs-imac  normalization         none                   -
> freddata/backup/alexs-imac  casesensitivity       sensitive              -
> freddata/backup/alexs-imac  vscan                 off                    =
default
> freddata/backup/alexs-imac  nbmand                off                    =
default
> freddata/backup/alexs-imac  sharesmb              off                    =
default
> freddata/backup/alexs-imac  refquota              none                   =
default
> freddata/backup/alexs-imac  refreservation        none                   =
default
> freddata/backup/alexs-imac  primarycache          all                    =
default
> freddata/backup/alexs-imac  secondarycache        all                    =
default
> freddata/backup/alexs-imac  usedbysnapshots       308G                   -
> freddata/backup/alexs-imac  usedbydataset         213G                   -
> freddata/backup/alexs-imac  usedbychildren        0                      -
> freddata/backup/alexs-imac  usedbyrefreservation  0                      -
> freddata/backup/alexs-imac  logbias               latency                =
default
> freddata/backup/alexs-imac  dedup                 off                    =
default
> freddata/backup/alexs-imac  mlslabel                                     -
> freddata/backup/alexs-imac  sync                  standard               =
default
> freddata/backup/alexs-imac  refcompressratio      1.06x                  -
> freddata/backup/alexs-imac  written               145G                   -
> freddata/backup/alexs-imac  logicalused           569G                   -
> freddata/backup/alexs-imac  logicalreferenced     221G                   -
> freddata/backup/alexs-imac  volmode               default                =
default
> freddata/backup/alexs-imac  filesystem_limit      none                   =
default
> freddata/backup/alexs-imac  snapshot_limit        none                   =
default
> freddata/backup/alexs-imac  filesystem_count      none                   =
default
> freddata/backup/alexs-imac  snapshot_count        none                   =
default
> freddata/backup/alexs-imac  redundant_metadata    all                    =
default

--=20
You are receiving this mail because:
You are on the CC list for the bug.=



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-216586-6>