Date: Mon, 1 Apr 2013 17:51:30 +0200 From: Fabian Keil <freebsd-listen@fabiankeil.de> To: Andriy Gapon <avg@FreeBSD.org> Cc: FreeBSD Current <freebsd-current@FreeBSD.org> Subject: Re: panic: solaris assert: sa.sa_magic == 0x2F505A (0x4d5ea364 == 0x2f505a), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c, line: 625 Message-ID: <20130401175126.1d28b0f4@fabiankeil.de> In-Reply-To: <5159A517.5070802@FreeBSD.org> References: <20130401161813.6e42e2a1@fabiankeil.de> <5159A517.5070802@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--Sig_/N2bYarDm3IOk.gkHS0qDag6
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable
Andriy Gapon <avg@FreeBSD.org> wrote:
> on 01/04/2013 17:18 Fabian Keil said the following:
> > #10 0xffffffff8130323f in assfail3 (a=3D<value optimized out>, lv=3D<va=
lue optimized out>, op=3D<value optimized out>, rv=3D<value optimized out>,=
f=3D<value optimized out>, l=3D<value optimized out>)
> > at /usr/src/sys/modules/opensolaris/../../cddl/compat/opensolaris/k=
ern/opensolaris_cmn_err.c:89
> > #11 0xffffffff8117924e in zfs_space_delta_cb (bonustype=3D<value optimi=
zed out>, data=3D0xffffff8015eeb8c0, userp=3D0xfffffe004261c640, groupp=3D0=
xfffffe004261c648)
> > at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/comm=
on/fs/zfs/zfs_vfsops.c:625
> > #12 0xffffffff8110003b in dmu_objset_userquota_get_ids (dn=3D0xfffffe00=
4261c358, before=3D0, tx=3D<value optimized out>) at /usr/src/sys/modules/z=
fs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c:1249
> > #13 0xffffffff811071b6 in dnode_sync (dn=3D0xfffffe004261c358, tx=3D0xf=
ffffe00186e1300) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris=
/uts/common/fs/zfs/dnode_sync.c:554
> > #14 0xffffffff810ff98b in dmu_objset_sync_dnodes (list=3D0xfffffe00691a=
5250, newlist=3D<value optimized out>, tx=3D<value optimized out>)
> > at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/comm=
on/fs/zfs/dmu_objset.c:910
> > #15 0xffffffff810ff825 in dmu_objset_sync (os=3D0xfffffe00691a5000, pio=
=3D<value optimized out>, tx=3D0xfffffe00186e1300)
> > at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/comm=
on/fs/zfs/dmu_objset.c:1027
> > #16 0xffffffff8110cb0d in dsl_dataset_sync (ds=3D0xfffffe001f3d0c00, zi=
o=3D0x780, tx=3D0xfffffe00186e1300) at /usr/src/sys/modules/zfs/../../cddl/=
contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c:1411
> > #17 0xffffffff8111399a in dsl_pool_sync (dp=3D0xfffffe0069ec4000, txg=
=3D<value optimized out>) at /usr/src/sys/modules/zfs/../../cddl/contrib/op=
ensolaris/uts/common/fs/zfs/dsl_pool.c:409
> > #18 0xffffffff8112f0ee in spa_sync (spa=3D0xfffffe0050f00000, txg=3D329=
2) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs=
/zfs/spa.c:6328
> > #19 0xffffffff81137c45 in txg_sync_thread (arg=3D0xfffffe0069ec4000) at=
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/=
txg.c:493
> > #20 0xffffffff80569c1a in fork_exit (callout=3D0xffffffff811378d0 <txg_=
sync_thread>, arg=3D0xfffffe0069ec4000, frame=3D0xffffff80dc9d6c00) at /usr=
/src/sys/kern/kern_fork.c:991
> > #21 0xffffffff8086a1ee in fork_trampoline () at exception.S:602
> > #22 0x0000000000000000 in ?? ()
> > Current language: auto; currently minimal
> > (kgdb) f 12
> > #12 0xffffffff8110003b in dmu_objset_userquota_get_ids (dn=3D0xfffffe00=
4261c358, before=3D0, tx=3D<value optimized out>) at /usr/src/sys/modules/z=
fs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c:1249
> > 1249 error =3D used_cbs[os->os_phys->os_type](dn->dn_bonustype, data,
> > (kgdb) p *dn
>=20
> Could you please also provide *dn->dn_phys?
vmcore.12:
(kgdb) p *dn->dn_phys
Cannot access memory at address 0xffffff8019492800
(kgdb) p *dn->dn_dbuf
$1 =3D {db =3D {db_object =3D 0, db_offset =3D 28491776, db_size =3D 16384,=
db_data =3D 0xffffff8019492000}, db_objset =3D 0xfffffe002bc62400, db_dnod=
e_handle =3D 0xfffffe002bc62420, db_parent =3D 0xfffffe005f1071c0,=20
db_hash_next =3D 0x0, db_blkid =3D 1739, db_blkptr =3D 0xffffff801936c580=
, db_level =3D 0 '\0', db_mtx =3D {lock_object =3D {lo_name =3D 0xffffffff8=
11d8696 "db->db_mtx", lo_flags =3D 40960000, lo_data =3D 0,=20
lo_witness =3D 0x0}, sx_lock =3D 1}, db_state =3D DB_CACHED, db_holds=
=3D {rc_count =3D 2}, db_buf =3D 0xfffffe005f34c798, db_changed =3D {cv_de=
scription =3D 0xffffffff811d86a2 "db->db_changed", cv_waiters =3D 0},=20
db_data_pending =3D 0xfffffe004bcc0c00, db_last_dirty =3D 0xfffffe004bcc0=
c00, db_link =3D {list_next =3D 0xfffffe005f3506d0, list_prev =3D 0xfffffe0=
030c392a0}, db_user_ptr =3D 0xfffffe005f269000,=20
db_user_data_ptr_ptr =3D 0x0, db_evict_func =3D 0xffffffff81105770 <dnode=
_buf_pageout>, db_immediate_evict =3D 0 '\0', db_freed_in_flight =3D 0 '\0'=
, db_dirtycnt =3D 1 '\001'}
vmcore.13:
(kgdb) p *dn->dn_phys
Cannot access memory at address 0xffffff8015eeb800
(kgdb) p *dn->dn_dbuf
$1 =3D {db =3D {db_object =3D 0, db_offset =3D 28491776, db_size =3D 16384,=
db_data =3D 0xffffff8015eeb000}, db_objset =3D 0xfffffe00691a5000, db_dnod=
e_handle =3D 0xfffffe00691a5020, db_parent =3D 0xfffffe004254ab60,=20
db_hash_next =3D 0x0, db_blkid =3D 1739, db_blkptr =3D 0xffffff8015d65580=
, db_level =3D 0 '\0', db_mtx =3D {lock_object =3D {lo_name =3D 0xffffffff8=
11d8696 "db->db_mtx", lo_flags =3D 40960000, lo_data =3D 0,=20
lo_witness =3D 0x0}, sx_lock =3D 1}, db_state =3D DB_CACHED, db_holds=
=3D {rc_count =3D 2}, db_buf =3D 0xfffffe0042bedcf0, db_changed =3D {cv_de=
scription =3D 0xffffffff811d86a2 "db->db_changed", cv_waiters =3D 0},=20
db_data_pending =3D 0xfffffe00406d6500, db_last_dirty =3D 0xfffffe00406d6=
500, db_link =3D {list_next =3D 0xfffffe0042758c10, list_prev =3D 0xfffffe0=
069cab5f8}, db_user_ptr =3D 0xfffffe0069f70000,=20
db_user_data_ptr_ptr =3D 0x0, db_evict_func =3D 0xffffffff81105770 <dnode=
_buf_pageout>, db_immediate_evict =3D 0 '\0', db_freed_in_flight =3D 0 '\0'=
, db_dirtycnt =3D 1 '\001'}
vmcore.14:
(kgdb) p *dn->dn_phys
Cannot access memory at address 0xffffff8014d21800
(kgdb) p *dn->dn_dbuf
$1 =3D {db =3D {db_object =3D 0, db_offset =3D 28508160, db_size =3D 16384,=
db_data =3D 0xffffff8014d21000}, db_objset =3D 0xfffffe0073372000, db_dnod=
e_handle =3D 0xfffffe0073372020, db_parent =3D 0xfffffe00369eec40,=20
db_hash_next =3D 0x0, db_blkid =3D 1740, db_blkptr =3D 0xffffff8014ac3600=
, db_level =3D 0 '\0', db_mtx =3D {lock_object =3D {lo_name =3D 0xffffffff8=
11d8696 "db->db_mtx", lo_flags =3D 40960000, lo_data =3D 0,=20
lo_witness =3D 0x0}, sx_lock =3D 1}, db_state =3D DB_CACHED, db_holds=
=3D {rc_count =3D 2}, db_buf =3D 0xfffffe0036b759d8, db_changed =3D {cv_de=
scription =3D 0xffffffff811d86a2 "db->db_changed", cv_waiters =3D 0},=20
db_data_pending =3D 0xfffffe0034467700, db_last_dirty =3D 0xfffffe0034467=
700, db_link =3D {list_next =3D 0xfffffe0036c17970, list_prev =3D 0xfffffe0=
0737bc950}, db_user_ptr =3D 0xfffffe00369fe000,=20
db_user_data_ptr_ptr =3D 0x0, db_evict_func =3D 0xffffffff81105770 <dnode=
_buf_pageout>, db_immediate_evict =3D 0 '\0', db_freed_in_flight =3D 0 '\0'=
, db_dirtycnt =3D 1 '\001'}
vmcore.15:
(kgdb) p *dn->dn_phys
Cannot access memory at address 0xffffff8030e73800
(kgdb) p *dn->dn_dbuf
$1 =3D {db =3D {db_object =3D 0, db_offset =3D 28508160, db_size =3D 16384,=
db_data =3D 0xffffff8030e73000}, db_objset =3D 0xfffffe0028fe2800, db_dnod=
e_handle =3D 0xfffffe0028fe2820, db_parent =3D 0xfffffe005ac4dd20,=20
db_hash_next =3D 0x0, db_blkid =3D 1740, db_blkptr =3D 0xffffff8030d49600=
, db_level =3D 0 '\0', db_mtx =3D {lock_object =3D {lo_name =3D 0xffffffff8=
11d8696 "db->db_mtx", lo_flags =3D 40960000, lo_data =3D 0,=20
lo_witness =3D 0x0}, sx_lock =3D 1}, db_state =3D DB_CACHED, db_holds=
=3D {rc_count =3D 2}, db_buf =3D 0xfffffe0010130a68, db_changed =3D {cv_de=
scription =3D 0xffffffff811d86a2 "db->db_changed", cv_waiters =3D 0},=20
db_data_pending =3D 0xfffffe0057b85200, db_last_dirty =3D 0xfffffe0057b85=
200, db_link =3D {list_next =3D 0xfffffe005ad57890, list_prev =3D 0xfffffe0=
0568765f8}, db_user_ptr =3D 0xfffffe005a7a1000,=20
db_user_data_ptr_ptr =3D 0x0, db_evict_func =3D 0xffffffff81105770 <dnode=
_buf_pageout>, db_immediate_evict =3D 0 '\0', db_freed_in_flight =3D 0 '\0'=
, db_dirtycnt =3D 1 '\001'}
> > The offending sa_magic in the panic message is always the same.
>=20
> Which part, left side or right side?
I meant the left side, but it looks like I only compared the first two pani=
cs:
fk@r500 /usr/crash $grep ^panic: core.txt.1[2-5]
core.txt.12:panic: solaris assert: sa.sa_magic =3D=3D 0x2F505A (0x4d5ea364 =
=3D=3D 0x2f505a), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensol=
aris/uts/common/fs/zfs/zfs_vfsops.c, line: 625
core.txt.13:panic: solaris assert: sa.sa_magic =3D=3D 0x2F505A (0x4d5ea364 =
=3D=3D 0x2f505a), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensol=
aris/uts/common/fs/zfs/zfs_vfsops.c, line: 625
core.txt.14:panic: solaris assert: sa.sa_magic =3D=3D 0x2F505A (0x4be9810a =
=3D=3D 0x2f505a), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensol=
aris/uts/common/fs/zfs/zfs_vfsops.c, line: 625
core.txt.15:panic: solaris assert: sa.sa_magic =3D=3D 0x2F505A (0x4be9810a =
=3D=3D 0x2f505a), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensol=
aris/uts/common/fs/zfs/zfs_vfsops.c, line: 625
The matches seem to correlate with the matching dn->dn_dbuf->db->db_offset
values above.
Fabian
--Sig_/N2bYarDm3IOk.gkHS0qDag6
Content-Type: application/pgp-signature; name=signature.asc
Content-Disposition: attachment; filename=signature.asc
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (FreeBSD)
iEYEARECAAYFAlFZrQEACgkQBYqIVf93VJ3e1QCeODdEVTiD6TksAu/p815aOPmE
40QAnAhaLasLYhXXXRCfMydh3GPIPq2l
=OcRh
-----END PGP SIGNATURE-----
--Sig_/N2bYarDm3IOk.gkHS0qDag6--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20130401175126.1d28b0f4>