Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 28 May 2009 16:38:11 +0200
From:      Henri Hennebert <hlh@restart.be>
To:        Andriy Gapon <avg@icyb.net.ua>
Cc:        freebsd-stable@freebsd.org, Kip Macy <kmacy@freebsd.org>, Artem Belevich <fbsdlist@src.cx>
Subject:   Re: ZFS MFC heads down
Message-ID:  <4A1EA1D3.405@restart.be>
In-Reply-To: <4A1E97D8.4080901@icyb.net.ua>
References:  <3c1674c90905201643m540c8b1v8a8bd88f071c233d@mail.gmail.com>		<4A1D0F2B.4030006@restart.be>		<ed91d4a80905271104g2a824d0fna004d1c4f3126c67@mail.gmail.com>	<3c1674c90905280052q281f6172j2409fe2d64db6914@mail.gmail.com>	<4A1E90F7.2000000@restart.be> <4A1E97D8.4080901@icyb.net.ua>

next in thread | previous in thread | raw e-mail | index | archive | help


Andriy Gapon wrote:
> on 28/05/2009 16:26 Henri Hennebert said the following:
>> (gdb) bt
>> #0  0x00000008012a6f22 in strlen () from /lib/libc.so.7
>> #1  0x00000008012a0feb in open () from /lib/libc.so.7
>> #2  0x000000080129ea59 in open () from /lib/libc.so.7
>> #3  0x00000008012a1f2e in vfprintf () from /lib/libc.so.7
>> #4  0x0000000801291158 in fprintf () from /lib/libc.so.7
>> #5  0x0000000801290fb0 in __assert () from /lib/libc.so.7
> 
> I find the above part interesting.
> Could this be because of the following discrepancy:
> 
> 1)
> cddl/contrib/opensolaris/lib/libzpool/common/sys/zfs_context.h:
> extern void __assert(const char *, const char *, int);
> 2)
> lib/libc/gen/assert.c:
> void
> __assert(func, file, line, failedexpr)
>         const char *func, *file;
>         int line;
>         const char *failedexpr;
> 
>> #6  0x0000000800fef120 in zmutex_destroy () from /lib/libzpool.so.1
>> #7  0x000000080102e1a0 in dsl_dataset_fast_stat () from /lib/libzpool.so.1
>> #8  0x0000000801045ffa in dbuf_find () from /lib/libzpool.so.1
>> #9  0x0000000801047bf3 in dmu_buf_rele () from /lib/libzpool.so.1
>> #10 0x0000000801027546 in dsl_pool_open () from /lib/libzpool.so.1
>> #11 0x000000080101bcec in spa_create () from /lib/libzpool.so.1
>> #12 0x000000080101c820 in spa_tryimport () from /lib/libzpool.so.1
> 
> But back to the problem - without an additional printf we still can not what was
> the value in m_owner. Only that it was not null.
> Probably it's better to build with debugging symbols and examine with gdb.

Firt try:
[root@avoriaz libzpool]# gdb zdb
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain 
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...(no debugging 
symbols found)...
(gdb) r pool1
Starting program: /usr/sbin/zdb pool1
(no debugging symbols found)...(no debugging symbols found)...(no 
debugging symbols found)...(no debugging symbols found)...(no debugging 
symbols found)...[New LWP 100299]
(no debugging symbols found)...(no debugging symbols found)...(no 
debugging symbols found)...(no debugging symbols found)...[New Thread 
0x8018020b0 (LWP 100299)]
[New Thread 0x801802240 (LWP 100354)]
     version=13
     name='pool1'
     state=0
     txg=4
     pool_guid=9156958376606789
     hostid=1133576597
     hostname='unset'
     vdev_tree
         type='root'
         id=0
         guid=9156958376606789
         children[0]
                 type='raidz'
                 id=0
                 guid=8214939615613279020
                 nparity=1
                 metaslab_array=23
                 metaslab_shift=32
                 ashift=9
                 asize=500108886016
                 is_log=0
                 children[0]
                         type='disk'
                         id=0
                         guid=7001907692988243779
                         path='/dev/ad8p2'
                         whole_disk=0
                 children[1]
                         type='disk'
                         id=1
                         guid=1909032920962573263
                         path='/dev/ad10p2'
                         whole_disk=0
[New Thread 0x8018023d0 (LWP 100369)]
[New Thread 0x801802560 (LWP 100370)]
[New Thread 0x8018026f0 (LWP 100371)]
[New Thread 0x801802880 (LWP 100372)]
[New Thread 0x801802a10 (LWP 100376)]
[New Thread 0x801802ba0 (LWP 100382)]
[New Thread 0x801802d30 (LWP 100383)]
[New Thread 0x801802ec0 (LWP 100384)]
[New Thread 0x801803050 (LWP 100385)]
[New Thread 0x8018031e0 (LWP 100386)]
[New Thread 0x801803370 (LWP 100387)]
[New Thread 0x801803500 (LWP 100388)]
[New Thread 0x801803690 (LWP 100389)]
[New Thread 0x801803820 (LWP 100390)]
[New Thread 0x8018039b0 (LWP 100391)]
[New Thread 0x801803b40 (LWP 100392)]
[New Thread 0x801803cd0 (LWP 100393)]
[New Thread 0x801803e60 (LWP 100394)]
[New Thread 0x801803ff0 (LWP 100395)]
[New Thread 0x801804180 (LWP 100396)]
[New Thread 0x801804310 (LWP 100397)]
[New Thread 0x8018044a0 (LWP 100398)]
[New Thread 0x801804630 (LWP 100399)]
[New Thread 0x8018047c0 (LWP 100400)]
[New Thread 0x801804950 (LWP 100401)]
[New Thread 0x801804ae0 (LWP 100402)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x8018020b0 (LWP 100299)]
0x00000008012a6f22 in strlen () from /lib/libc.so.7
(gdb) bt
#0  0x00000008012a6f22 in strlen () from /lib/libc.so.7
#1  0x00000008012a0feb in open () from /lib/libc.so.7
#2  0x000000080129ea59 in open () from /lib/libc.so.7
#3  0x00000008012a1f2e in vfprintf () from /lib/libc.so.7
#4  0x0000000801291158 in fprintf () from /lib/libc.so.7
#5  0x0000000801290fb0 in __assert () from /lib/libc.so.7
#6  0x0000000800fef230 in zmutex_destroy (mp=0x8018b2cc0)
     at 
/usr/src/cddl/lib/libzpool/../../../cddl/contrib/opensolaris/lib/libzpool/common/kernel.c:112
#7  0x000000080102e2b0 in dsl_dataset_evict (db=Variable "db" is not 
available.
)
     at 
/usr/src/cddl/lib/libzpool/../../../sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c:266
#8  0x000000080104610a in dbuf_evict_user (db=0x8018ca960)
     at 
/usr/src/cddl/lib/libzpool/../../../sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:215
#9  0x0000000801047d03 in dbuf_rele (db=0x8018ca960, tag=Variable "tag" 
is not available.
)
     at 
/usr/src/cddl/lib/libzpool/../../../sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:1739
#10 0x0000000801027656 in dsl_pool_open (spa=0x8018a2000, txg=Variable 
"txg" is not available.
)
     at 
/usr/src/cddl/lib/libzpool/../../../sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c:139
#11 0x000000080101bdfc in spa_load (spa=0x8018a2000, config=Variable 
"config" is not available.
)
     at 
/usr/src/cddl/lib/libzpool/../../../sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:1134
#12 0x000000080101c930 in spa_open_common (pool=0x7fffffffee06 "pool1", 
spapp=0x7fffffffebb8, tag=0x40b790, config=0x0)
     at 
/usr/src/cddl/lib/libzpool/../../../sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:1474
#13 0x0000000000408b41 in ?? ()
#14 0x00000000004036de in ?? ()
#15 0x0000000800534000 in ?? ()
#16 0x0000000000000000 in ?? ()
#17 0x0000000000000000 in ?? ()
#18 0x0000000000000002 in ?? ()
#19 0x00007fffffffedf8 in ?? ()
#20 0x00007fffffffee06 in ?? ()
#21 0x0000000000000000 in ?? ()
#22 0x00007fffffffee0c in ?? ()
#23 0x00007fffffffee22 in ?? ()
#24 0x00007fffffffee2d in ?? ()
#25 0x00007fffffffee47 in ?? ()
#26 0x00007fffffffee64 in ?? ()
#27 0x00007fffffffee6e in ?? ()
#28 0x00007fffffffee79 in ?? ()
#29 0x00007fffffffee85 in ?? ()
#30 0x00007fffffffee9a in ?? ()
#31 0x00007fffffffeeae in ?? ()
#32 0x00007fffffffef08 in ?? ()
#33 0x00007fffffffef17 in ?? ()
#34 0x00007fffffffef23 in ?? ()
#35 0x00007fffffffef42 in ?? ()
#36 0x00007fffffffef4f in ?? ()
#37 0x00007fffffffef64 in ?? ()
#38 0x00007fffffffef76 in ?? ()
#39 0x00007fffffffef7f in ?? ()
#40 0x00007fffffffef8f in ?? ()
#41 0x00007fffffffef97 in ?? ()
#42 0x0000000000000000 in ?? ()
#43 0x0000000000000003 in ?? ()
#44 0x0000000000400040 in ?? ()
#45 0x0000000000000004 in ?? ()
#46 0x0000000000000038 in ?? ()
#47 0x0000000000000005 in ?? ()
---Type <return> to continue, or q <return> to quit---q
Quit
(gdb) frame 6
#6  0x0000000800fef230 in zmutex_destroy (mp=0x8018b2cc0)
     at 
/usr/src/cddl/lib/libzpool/../../../cddl/contrib/opensolaris/lib/libzpool/common/kernel.c:112
112		ASSERT(mp->m_owner == NULL);
(gdb) print mp->m_owner
$1 = (void *) 0x8018f2060

As I understand mp->m_owner is a thread ref ...

Feel free to ask for more

Henri


> 



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4A1EA1D3.405>