Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 14 Mar 2016 01:50:52 +0300
From:      DemIS <demis@yandex.ru>
To:        Steven Hartland <killing@multiplay.co.uk>, "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org>
Subject:   Re: Kernel panic zio.c, line: 270 FreeBSD 10.2 (or 10.3)
Message-ID:  <711901457909452@web19j.yandex.ru>
In-Reply-To: <56E5E96B.8040209@multiplay.co.uk>
References:  <1061671455578760@web3g.yandex.ru> <56C2655F.9010809@multiplay.co.uk> <1076701455583595@web2g.yandex.ru> <56C2782E.2010404@multiplay.co.uk> <2311371455610101@web11j.yandex.ru> <56C2E7A6.9090004@multiplay.co.uk> <4332271457902960@web8o.yandex.ru> <56E5E96B.8040209@multiplay.co.uk>

next in thread | previous in thread | raw e-mail | index | archive | help
Hi Stiven,

frame 8
#8  0xffffffff824419bd in arc_read (pio=0xfffff80071def398, spa=0xfffff800719d1000, bp=0xfffffe001589e180, done=0xffffffff82455380 <dbuf_read_done>,
    private=0xfffff803b1e8de00, priority=ZIO_PRIORITY_SYNC_READ, zio_flags=128, arc_flags=0xfffffe0677ec0094, zb=0xfffffe0677ec0098)
    at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:3389
3389                            buf = arc_buf_alloc(spa, size, private, type);
(kgdb) p bp
$1 = (const blkptr_t *) 0xfffffe001589e180
(kgdb) p *bp
Cannot access memory at address 0xfffffe001589e180
(kgdb)


14.03.2016, 01:27, "Steven Hartland" <killing@multiplay.co.uk>:
> Thanks DemIS, size = 0 is definitely bad however I don't see bp listed
> from frame #8.
>
> šFrom the debugger:
> š> frame 8
> š> print bp
>
> On 13/03/2016 21:02, DemIS wrote:
>> šHi Steven ,
>>
>> šFinally I was able to get the data that you have requested.
>> šThe following quote from the dump to
>> šššsetfacl -b -m <someAccount> /hdd/usr/wf/some/long/path
>>
>> škgdb /usr/obj/usr/src/sys/TEO/kernel.debug /usr/crash/vmcore.last
>> šGNU gdb 6.1.1 [FreeBSD]
>> šCopyright 2004 Free Software Foundation, Inc.
>> šGDB is free software, covered by the GNU General Public License, and you are
>> šwelcome to change it and/or distribute copies of it under certain conditions.
>> šType "show copying" to see the conditions.
>> šThere is absolutely no warranty for GDB. Type "show warranty" for details.
>> šThis GDB was configured as "amd64-marcel-freebsd"...
>>
>> šUnread portion of the kernel message buffer:
>> španic: solaris assert: c < (1ULL << 24) >> 9 (0x7fffffffffffff < 0x8000), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 270
>> šcpuid = 2
>> šKDB: stack backtrace:
>> šdb_trace_self_wrapper() at db_trace_self_wrapper+0x2e/frame 0xfffffe0677ebfbe0
>> škdb_backtrace() at kdb_backtrace+0x58/frame 0xfffffe0677ebfc90
>> švpanic() at vpanic+0x1df/frame 0xfffffe0677ebfd00
>> švpanic() at vpanic/frame 0xfffffe0677ebfd60
>> šzfs_kmem_alloc() at zfs_kmem_alloc/frame 0xfffffe0677ebfdb0
>> šzio_buf_alloc() at zio_buf_alloc+0x84/frame 0xfffffe0677ebfdf0
>> šarc_get_data_buf() at arc_get_data_buf+0x7a/frame 0xfffffe0677ebfe60
>> šarc_buf_alloc() at arc_buf_alloc+0xe0/frame 0xfffffe0677ebfeb0
>> šarc_read() at arc_read+0x65d/frame 0xfffffe0677ec0040
>> šdbuf_read_impl() at dbuf_read_impl+0x4af/frame 0xfffffe0677ec00e0
>> šdbuf_read() at dbuf_read+0x33b/frame 0xfffffe0677ec0190
>> šdmu_spill_hold_by_dnode() at dmu_spill_hold_by_dnode+0xf7/frame 0xfffffe0677ec01e0
>> šdmu_spill_hold_existing() at dmu_spill_hold_existing+0x16b/frame 0xfffffe0677ec0240
>> šsa_get_spill() at sa_get_spill+0x3c/frame 0xfffffe0677ec0260
>> šsa_modify_attrs() at sa_modify_attrs+0x118/frame 0xfffffe0677ec0330
>> šsa_attr_op() at sa_attr_op+0x9db/frame 0xfffffe0677ec03a0
>> šsa_bulk_update_impl() at sa_bulk_update_impl+0x74/frame 0xfffffe0677ec03f0
>> šsa_bulk_update() at sa_bulk_update+0x45/frame 0xfffffe0677ec0430
>> šzfs_aclset_common() at zfs_aclset_common+0x95f/frame 0xfffffe0677ec0620
>> šzfs_setacl() at zfs_setacl+0x451/frame 0xfffffe0677ec06c0
>> šzfs_setsecattr() at zfs_setsecattr+0x10d/frame 0xfffffe0677ec0720
>> šzfs_freebsd_setacl() at zfs_freebsd_setacl+0x15c/frame 0xfffffe0677ec0790
>> šVOP_SETACL_APV() at VOP_SETACL_APV+0x174/frame 0xfffffe0677ec07f0
>> šVOP_SETACL() at VOP_SETACL+0x5a/frame 0xfffffe0677ec0860
>> švacl_set_acl() at vacl_set_acl+0xf1/frame 0xfffffe0677ec08c0
>> šsys___acl_set_file() at sys___acl_set_file+0x91/frame 0xfffffe0677ec09d0
>> šsyscallenter() at syscallenter+0x754/frame 0xfffffe0677ec0af0
>> šamd64_syscall() at amd64_syscall+0x1f/frame 0xfffffe0677ec0bf0
>> šXfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0677ec0bf0
>> š--- syscall (348, FreeBSD ELF64, sys___acl_set_file), rip = 0x80092a17a, rsp = 0x7fffffffe2e8, rbp = 0x7fffffffe7c0 ---
>> šUptime: 2d11h24m58s
>> šDumping 5431 out of 24533 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
>>
>> šReading symbols from /boot/kernel/if_lagg.ko.symbols...done.
>> šLoaded symbols for /boot/kernel/if_lagg.ko.symbols
>> šReading symbols from /boot/kernel/aio.ko.symbols...done.
>> šLoaded symbols for /boot/kernel/aio.ko.symbols
>> šReading symbols from /boot/kernel/ichsmb.ko.symbols...done.
>> šLoaded symbols for /boot/kernel/ichsmb.ko.symbols
>> šReading symbols from /boot/kernel/ipmi.ko.symbols...done.
>> šLoaded symbols for /boot/kernel/ipmi.ko.symbols
>> šReading symbols from /boot/kernel/zfs.ko.symbols...done.
>> šLoaded symbols for /boot/kernel/zfs.ko.symbols
>> šReading symbols from /boot/kernel/opensolaris.ko.symbols...done.
>> šLoaded symbols for /boot/kernel/opensolaris.ko.symbols
>> šReading symbols from /boot/kernel/ums.ko.symbols...done.
>> šLoaded symbols for /boot/kernel/ums.ko.symbols
>> šReading symbols from /boot/kernel/smbfs.ko.symbols...done.
>> šLoaded symbols for /boot/kernel/smbfs.ko.symbols
>> šReading symbols from /boot/kernel/libiconv.ko.symbols...done.
>> šLoaded symbols for /boot/kernel/libiconv.ko.symbols
>> šReading symbols from /boot/kernel/libmchain.ko.symbols...done.
>> šLoaded symbols for /boot/kernel/libmchain.ko.symbols
>> š#0 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:263
>> š263 dumptid = curthread->td_tid;
>> š(kgdb) bt full
>> š#0 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:263
>> ššššššššššcoredump = 1
>> š#1 0xffffffff80e21a6a in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:451
>> ššššššššššfirst_buf_printf = 1
>> š#2 0xffffffff80e2240c in vpanic (fmt=0xffffffff8268f154 "solaris assert: %s (0x%jx %s 0x%jx), file: %s, line: %d", ap=0xfffffe0677ebfd40)
>> ššššššat /usr/src/sys/kern/kern_shutdown.c:758
>> ššššššššššbuf = "solaris assert: c < (1ULL << 24) >> 9 (0x7", 'f' <repeats 13 times>, " < 0x8000), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 270", '\0' <repeats 92 times>
>> ššššššššššother_cpus = {__bits = {65531}}
>> šššššššššštd = (struct thread *) 0xfffff800713124a0
>> ššššššššššbootopt = 260
>> ššššššššššnewpanic = 1
>> š#3 0xffffffff80e221d0 in panic (fmt=0xffffffff8268f154 "solaris assert: %s (0x%jx %s 0x%jx), file: %s, line: %d")
>> ššššššat /usr/src/sys/kern/kern_shutdown.c:687
>> ššššššššššap = {{gp_offset = 48, fp_offset = 48, overflow_arg_area = 0xfffffe0677ebfd78, reg_save_area = 0xfffffe0677ebfd10}}
>> š#4 0xffffffff8268e470 in assfail3 (a=0xffffffff825cbcb9 "c < (1ULL << 24) >> 9", lv=36028797018963967, op=0xffffffff825cbccf "<", rv=32768,
>> ššššššf=0xffffffff825cbcd1 "/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c", l=270)
>> ššššššat /usr/src/sys/modules/opensolaris/../../cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:91
>> šNo locals.
>> š#5 0xffffffff8250e414 in zio_buf_alloc (size=0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:270
>> šššššššššš__left = 36028797018963967
>> šššššššššš__right = 32768
>> ššššššššššc = 36028797018963967
>> ššššššššššflags = 0
>> š#6 0xffffffff8243dc1a in arc_get_data_buf (buf=0xfffff804795ddc18)
>> ššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2898
>> ššššššššššstate = (arc_state_t *) 0xffffffff825e8700
>> ššššššššššsize = 0
>> šššššššššštype = ARC_BUFC_METADATA
>> š#7 0xffffffff8243db60 in arc_buf_alloc (spa=0xfffff800719d1000, size=0, tag=0xfffff803b1e8de00, type=ARC_BUFC_METADATA)
>> ššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1551
>> ššššššššššhdr = (arc_buf_hdr_t *) 0xfffff8025cc22288
>> ššššššššššbuf = (arc_buf_t *) 0xfffff804795ddc18
>> š#8 0xffffffff824419bd in arc_read (pio=0xfffff80071def398, spa=0xfffff800719d1000, bp=0xfffffe001589e180, done=0xffffffff82455380 <dbuf_read_done>,
>> ššššššprivate=0xfffff803b1e8de00, priority=ZIO_PRIORITY_SYNC_READ, zio_flags=128, arc_flags=0xfffffe0677ec0094, zb=0xfffffe0677ec0098)
>> ššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:3389
>> ššššššššššexists = (arc_buf_hdr_t *) 0x0
>> šššššššššštype = ARC_BUFC_METADATA
>> ššššššššššvd = (vdev_t *) 0x0
>> ššššššššššdevw = 0
>> ššššššššššb_asize = 0
>> ššššššššššsize = 0
>> ššššššššššacb = (arc_callback_t *) 0xfffff800713124a0
>> ššššššššššaddr = 0
>> ššššššššššb_compress = ZIO_COMPRESS_OFF
>> ššššššššššsdt_sdt___arc__hit = {{version = 1287817688, prov = 0x30c48348fffff834, probe_entry = {tqe_next = 0x2e6666666666c35d, tqe_prev = 0x841f0f},
>> ššššššargtype_list = {tqh_first = 0x20ec8348e5894855, tqh_last = 0xf0758948f87d8948},
>> ššššššmod = 0xf87d8b48e8558948 <Address 0xf87d8b48e8558948 out of bounds>, func = 0xe8558b48f0758b48 <Address 0xe8558b48f0758b48 out of bounds>,
>> ššššššname = 0x12b8c189c031 <Address 0x12b8c189c031 out of bounds>, id = 5531134888965130240, n_args = -387955317, sdtp_lf = 0x9066c35d20c48348}}
>> šššššššššš__set_sdt_probes_set_sym_sdt_sdt___arc__hit = (const void * const) 0x58f0f000000
>> ššššššššššsdta_sdt___arc__hit0 = {{ndx = -443987883, type = 0x4c984d894c000001 <Address 0x4c984d894c000001 out of bounds>,
>> ššššššxtype = 0x48884d8948904589 <Address 0x48884d8948904589 out of bounds>, argtype_entry = {tqe_next = 0xb8d04d8d48805589, tqe_prev = 0xba4100000001},
>> ššššššprobe = 0x7589f87d89480000}}
>> š---Type <return> to continue, or q <return> to quit---
>> šššššššššš__set_sdt_argtypes_set_sym_sdta_sdt___arc__hit0 = (const void * const) 0x1518e0f000000
>> ššššššššššsdt_sdt___arc__miss = {{version = -193623052, prov = 0xfe810000, probe_entry = {tqe_next = 0x41db8445c3950f41, tqe_prev = 0x45c7c44589c2440f},
>> ššššššargtype_list = {tqh_first = 0x958d4800000000c0, tqh_last = 0x10518948ffffff70}, mod = 0x851894810558d48 <Address 0x851894810558d48 out of bounds>,
>> ššššššfunc = 0xc7000000300441c7 <Address 0xc7000000300441c7 out of bounds>, name = 0x8100b00000001001 <Address 0x8100b00000001001 out of bounds>,
>> ššššššid = -8824803469832437635, n_args = -145, sdtp_lf = 0x4d8bd0458d480000}}
>> šššššššššš__set_sdt_probes_set_sym_sdt_sdt___arc__miss = (const void * const) 0x2b800000174e900
>> ššššššššššsdta_sdt___arc__miss0 = {{ndx = 687440336, type = 0x8d89ffffff608589 <Address 0x8d89ffffff608589 out of bounds>,
>> ššššššxtype = 0x2b870fffffff5c <Address 0x2b870fffffff5c out of bounds>, argtype_entry = {tqe_next = 0xffff60858b480000, tqe_prev = 0x5c958b10488b48ff},
>> ššššššprobe = 0x148f26348ffffff}}
>> šššššššššš__set_sdt_argtypes_set_sym_sdta_sdt___arc__miss0 = (const void * const) 0x8be04589e4458b00
>> ššššššššššsdta_sdt___arc__miss1 = {{ndx = 146964977, type = 0xffffff508d894810 <Address 0xffffff508d894810 out of bounds>,
>> ššššššxtype = 0x858b4800000020e9 <Address 0x858b4800000020e9 out of bounds>, argtype_entry = {tqe_next = 0x8488b48ffffff60, tqe_prev = 0x8c18148ca8948},
>> ššššššprobe = 0x8948084889480000}}
>> šššššššššš__set_sdt_argtypes_set_sym_sdta_sdt___arc__miss1 = (const void * const) 0x1f8d0ff4453be045
>> ššššššššššsdta_sdt___arc__miss2 = {{ndx = -44907, type = 0x48008b48ffffff50 <Address 0x48008b48ffffff50 out of bounds>,
>> ššššššxtype = 0x3d48c84589 <Address 0x3d48c84589 out of bounds>, argtype_entry = {tqe_next = 0xff6f8d88c1950f00, tqe_prev = 0xffffff6f858affff},
>> ššššššprobe = 0x5850f01a8}}
>> šššššššššš__set_sdt_argtypes_set_sym_sdta_sdt___arc__miss2 = (const void * const) 0x452be0458b000001
>> ššššššššššsdta_sdt___arc__miss3 = {{ndx = 186601, type = 0x28f981d04d8bd0 <Address 0x28f981d04d8bd0 out of bounds>,
>> ššššššxtype = 0xffff488589480000 <Address 0xffff488589480000 out of bounds>, argtype_entry = {tqe_next = 0xfffffff448d89ff,
>> šššššššštqe_prev = 0x858b480000002b87}, probe = 0x10488b48ffffff48}}
>> šššššššššš__set_sdt_argtypes_set_sym_sdta_sdt___arc__miss3 = (const void * const) 0xdc7d81dc4589e4
>> ššššššššššsdt_sdt___l2arc__hit = {{version = -12282485, prov = 0x8c281f10148f2, probe_entry = {tqe_next = 0x388d894810890000, tqe_prev = 0x20e9ffffff},
>> ššššššargtype_list = {tqh_first = 0x48ffffff48858b48, tqh_last = 0x8148ca894808488b},
>> ššššššmod = 0x48894800000008c1 <Address 0x48894800000008c1 out of bounds>, func = 0xffffff3895894808 <Address 0xffffff3895894808 out of bounds>,
>> ššššššname = 0x8bffffff38858b48 <Address 0x8bffffff38858b48 out of bounds>, id = 5587026407265569032, n_args = -1982953028, sdtp_lf = 0x89ffffff309589ff}}
>> šššššššššš__set_sdt_probes_set_sym_sdt_sdt___l2arc__hit = (const void * const) 0x4589f44d8b000000
>> ššššššššššsdta_sdt___l2arc__hit0 = {{ndx = -54091, type = 0xec03fcb8000002 <Address 0xec03fcb8000002 out of bounds>,
>> ššššššxtype = 0xa30fffffff348d8b <Address 0xa30fffffff348d8b out of bounds>, argtype_entry = {tqe_next = 0xb800000058820fc8,
>> šššššššštqe_prev = 0xff348d8b0713fc00}, probe = 0xdb820fc8a30fffff}}
>> šššššššššš__set_sdt_argtypes_set_sym_sdta_sdt___l2arc__hit0 = (const void * const) 0xf48c0f000000
>> ššššššššššsdt_sdt___l2arc__read = {{version = -1962934272, prov = 0xff28858901e883ff, probe_entry = {tqe_next = 0x5840fffff,
>> šššššššštqe_prev = 0x7d8b48000001d3e9}, argtype_list = {tqh_first = 0xbc558bc8758b48f8, tqh_last = 0x208d8948c189c031},
>> ššššššmod = 0xff20858b4cffffff <Address 0xff20858b4cffffff out of bounds>, func = 0x89fffff5c1e8ffff <Address 0x89fffff5c1e8ffff out of bounds>,
>> ššššššname = 0x48000001b0e9c045 <Address 0x48000001b0e9c045 out of bounds>, id = -467863854793079411, n_args = 40, sdtp_lf = 0xffff148d89ffffff}}
>> šššššššššš__set_sdt_probes_set_sym_sdt_sdt___l2arc__read = (const void * const) 0xf7c04d8b99c889c0
>> ššššššššššsdta_sdt___l2arc__read0 = {{ndx = 730271743, type = 0x8b48ffffff18858b <Address 0x8b48ffffff18858b out of bounds>,
>> ššššššxtype = 0xffffff14958b1048 <Address 0xffffff14958b1048 out of bounds>, argtype_entry = {tqe_next = 0xc281f10148f26348,
>> šššššššštqe_prev = 0x8948108900000008}, probe = 0x20e9ffffff088d}}
>> šššššššššš__set_sdt_argtypes_set_sym_sdta_sdt___l2arc__read0 = (const void * const) 0x4d8b48e8458b4800
>> ššššššššššsdta_sdt___l2arc__read1 = {{ndx = -1958215680, type = 0xca894808488b48ff <Address 0xca894808488b48ff out of bounds>,
>> ššššššxtype = 0x4800000008c18148 <Address 0x4800000008c18148 out of bounds>, argtype_entry = {tqe_next = 0xff08958948084889,
>> šššššššštqe_prev = 0xffff08858b48ffff}, probe = 0xb0458948008b48ff}}
>> šššššššššš__set_sdt_argtypes_set_sym_sdta_sdt___l2arc__read1 = (const void * const) 0xf055af0fdc558bf8
>> ššššššššššsdt_sdt___l2arc__miss = {{version = -125990072, prov = 0x31b0458b4cbc558b, probe_entry = {tqe_next = 0x4589fffff52ae8c9,
>> šššššššštqe_prev = 0x8d4800000119e9c0}, argtype_list = {tqh_first = 0x28f981d04d8bd045, tqh_last = 0xff00858948000000},
>> ššššššmod = 0xfffffefc8d89ffff <Address 0xfffffefc8d89ffff out of bounds>, func = 0x8b480000002b870f <Address 0x8b480000002b870f out of bounds>,
>> ššššššname = 0x488b48ffffff0085 <Address 0x488b48ffffff0085 out of bounds>, id = 5260204360416463632, n_args = 21557859, sdtp_lf = 0x8d89481089000000}}
>> šššššššššš__set_sdt_probes_set_sym_sdt_sdt___l2arc__miss = (const void * const) 0xe47d81e44589f9
>> ššššššššššsdta_sdt___l2arc__miss0 = {{ndx = -272, type = 0xffffff00858b4800 <Address 0xffffff00858b4800 out of bounds>,
>> ššššššxtype = 0x48ca894808488b48 <Address 0x48ca894808488b48 out of bounds>, argtype_entry = {tqe_next = 0x894800000008c181,
>> šššššššštqe_prev = 0xfffef09589480848}, probe = 0xfffffef0858b48ff}}
>> šššššššššš__set_sdt_argtypes_set_sym_sdta_sdt___l2arc__miss0 = (const void * const) 0x8b48f10148f26348
>> š---Type <return> to continue, or q <return> to quit---
>> ššššššššššsdt_sdt___l2arc__miss = {{version = -800223928, prov = 0xfa81d0558bb04589, probe_entry = {tqe_next = 0xe88d894800000028,
>> šššššššštqe_prev = 0xfffee49589fffffe}, argtype_list = {tqh_first = 0x480000002b870fff, tqh_last = 0x8b48fffffee8858b},
>> ššššššmod = 0xfffffee4958b1048 <Address 0xfffffee4958b1048 out of bounds>, func = 0xc281f10148f26348 <Address 0xc281f10148f26348 out of bounds>,
>> ššššššname = 0x8948108900000008 <Address 0x8948108900000008 out of bounds>, id = 9264484975564941, n_args = -1958215680, sdtp_lf = 0xca894808488b48ff}}
>> šššššššššš__set_sdt_probes_set_sym_sdt_sdt___l2arc__miss = (const void * const) 0x1518e0f000000
>> ššššššššššsdta_sdt___l2arc__miss0 = {{ndx = 146899272, type = 0xfed8958948084889 <Address 0xfed8958948084889 out of bounds>,
>> ššššššxtype = 0xfffed8858b48ffff <Address 0xfffed8858b48ffff out of bounds>, argtype_entry = {tqe_next = 0xa8458948008b48ff,
>> šššššššštqe_prev = 0xc8758b48f87d8b48}, probe = 0x4ca84d8b48bc558b}}
>> šššššššššš__set_sdt_argtypes_set_sym_sdta_sdt___l2arc__miss0 = (const void * const) 0xe45503dc558bf875
>> ššššššššššhdr = (arc_buf_hdr_t *) 0x0
>> ššššššššššbuf = (arc_buf_t *) 0x0
>> ššššššššššhash_lock = (kmutex_t *) 0x0
>> ššššššššššrzio = (zio_t *) 0xfffffe0677ebfff0
>> ššššššššššguid = 18385013180861733512
>> š#9 0xffffffff8244f7df in dbuf_read_impl (db=0xfffff803b1e8de00, zio=0xfffff80071def398, flags=0xfffffe0677ec0174)
>> ššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:573
>> ššššššššššdn = (dnode_t *) 0xfffff805c8be5000
>> ššššššššššzb = {zb_objset = 44, zb_object = 7962880, zb_level = 0, zb_blkid = 18446744073709551614}
>> ššššššššššaflags = 100
>> š#10 0xffffffff8244ef8b in dbuf_read (db=0xfffff803b1e8de00, zio=0xfffff80071def398, flags=6)
>> ššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:621
>> ššššššššššspa = (spa_t *) 0xfffff800719d1000
>> ššššššššššsdt_sdt___blocked__read = {{version = 549749576, prov = 0x841f0f, probe_entry = {tqe_next = 0x20ec8348e5894855, tqe_prev = 0xf0758948f87d8948},
>> ššššššargtype_list = {tqh_first = 0xf0758b48f87d8b48, tqh_last = 0x6b8c289c031}, mod = 0x48c689e875894800 <Address 0x48c689e875894800 out of bounds>,
>> ššššššfunc = 0xfffffc60e8e84d8b <Address 0xfffffc60e8e84d8b out of bounds>, name = 0x2e66c35d20c48348 <Address 0x2e66c35d20c48348 out of bounds>,
>> ššššššid = 8658703, n_args = -443987883, sdtp_lf = 0xf0758948f87d8948}}
>> šššššššššš__set_sdt_probes_set_sym_sdt_sdt___blocked__read = (const void * const) 0xd089b44589f0558b
>> ššššššššššsdta_sdt___blocked__read0 = {{ndx = -125990072, type = 0x7b8c289c031 <Address 0x7b8c289c031 out of bounds>,
>> ššššššxtype = 0x48c689e875894800 <Address 0x48c689e875894800 out of bounds>, argtype_entry = {tqe_next = 0xfffffc20e8e84d8b,
>> šššššššštqe_prev = 0x2e66c35d20c48348}, probe = 0x841f0f}}
>> šššššššššš__set_sdt_argtypes_set_sym_sdta_sdt___blocked__read0 = (const void * const) 0x4c28148c28948
>> ššššššššššsdta_sdt___blocked__read1 = {{ndx = -443987883, type = 0xf0758948f87d8948 <Address 0xf0758948f87d8948 out of bounds>,
>> ššššššxtype = 0xf0758b48f87d8b48 <Address 0xf0758b48f87d8b48 out of bounds>, argtype_entry = {tqe_next = 0x8b8c289c031, tqe_prev = 0x48c689e875894800},
>> ššššššprobe = 0xfffffbe0e8e84d8b}}
>> šššššššššš__set_sdt_argtypes_set_sym_sdta_sdt___blocked__read1 = (const void * const) 0x889c85589480000
>> ššššššššššerr = 0
>> ššššššššššhavepzio = 0
>> ššššššššššprefetch = 1
>> ššššššššššdn = (dnode_t *) 0xfffff805c8be5000
>> š#11 0xffffffff8245a777 in dmu_spill_hold_by_dnode (dn=0xfffff805c8be5000, flags=6, tag=0x0, dbp=0xfffff802f48dfa28)
>> ššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:333
>> ššššššššššdb = (dmu_buf_impl_t *) 0xfffff803b1e8de00
>> ššššššššššerr = -512
>> š#12 0xffffffff8245a91b in dmu_spill_hold_existing (bonus=0xfffff80052609620, tag=0x0, dbp=0xfffff802f48dfa28)
>> ššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:359
>> ššššššššššdb = (dmu_buf_impl_t *) 0xfffff80052609620
>> ššššššššššdn = (dnode_t *) 0xfffff805c8be5000
>> ššššššššššerr = 1
>> š#13 0xffffffff824b140c in sa_get_spill (hdl=0xfffff802f48dfa00) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:310
>> ššššššššššrc = 0
>> š#14 0xffffffff824b1658 in sa_modify_attrs (hdl=0xfffff802f48dfa00, newattr=19, action=SA_REPLACE, locator=0xffffffff82520d50 <zfs_acl_data_locator>,
>> ššššššdatastart=0xfffffe0677ec05c8, buflen=152, tx=0xfffff8034ef07800)
>> ššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:1681
>> š---Type <return> to continue, or q <return> to quit---
>> ššššššššššsa = (sa_os_t *) 0xfffff80071e5da00
>> ššššššššššdb = (dmu_buf_impl_t *) 0xfffff80052609620
>> ššššššššššdn = (dnode_t *) 0xfffff805c8be5000
>> ššššššššššattr_desc = (sa_bulk_attr_t *) 0xfffff803c1d2a000
>> ššššššššššold_data = {0xfffff80122383800, 0xfffff8034ef07800}
>> ššššššššššbonus_attr_count = 14
>> ššššššššššbonus_data_size = 320
>> ššššššššššspill_data_size = 0
>> ššššššššššspill_attr_count = 0
>> ššššššššššerror = -2045
>> ššššššššššlength = 49618
>> šššššššššši = 0
>> ššššššššššj = 0
>> ššššššššššk = 2
>> ššššššššššlength_idx = 16691240
>> ššššššššššhdr = (sa_hdr_phys_t *) 0xfffff80052600200
>> ššššššššššidx_tab = (sa_idx_tab_t *) 0xffffffff825bb2ea
>> ššššššššššattr_count = 0
>> ššššššššššcount = -1367343104
>> š#15 0xffffffff824b12fb in sa_attr_op (hdl=0xfffff802f48dfa00, bulk=0xfffffe0677ec04f0, count=5, data_op=SA_UPDATE, tx=0xfffff8034ef07800)
>> ššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:392
>> ššššššššššsa = (sa_os_t *) 0xfffff80071e5da00
>> šššššššššši = 3
>> ššššššššššerror = 0
>> ššššššššššbuftypes = SA_BONUS
>> š#16 0xffffffff824b5a04 in sa_bulk_update_impl (hdl=0xfffff802f48dfa00, bulk=0xfffffe0677ec04f0, count=5, tx=0xfffff8034ef07800)
>> ššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:1790
>> ššššššššššerror = 1888
>> ššššššššššsa = (sa_os_t *) 0xfffff80071e5da00
>> ššššššššššbonustype = DMU_OT_SA
>> š#17 0xffffffff824b5ca5 in sa_bulk_update (hdl=0xfffff802f48dfa00, attrs=0xfffffe0677ec04f0, count=5, tx=0xfffff8034ef07800)
>> ššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:1889
>> ššššššššššerror = 1
>> š#18 0xffffffff82521b5f in zfs_aclset_common (zp=0xfffff803b27e85c0, aclp=0xfffff8001ebcfa00, cr=0xfffff804455e6e00, tx=0xfffff8034ef07800)
>> ššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_acl.c:1329
>> ššššššššššerror = -2045
>> ššššššššššzfsvfs = (zfsvfs_t *) 0xfffff80086ee2000
>> ššššššššššotype = DMU_OT_ACL
>> ššššššššššlocate = {cb_aclp = 0xfffff8001ebcfa00, cb_acl_node = 0x0}
>> ššššššššššmode = 16893
>> ššššššššššbulk = {{sa_data = 0xfffffe0677ec05c0, sa_data_func = 0, sa_length = 8, sa_attr = 5, sa_addr = 0xfffff80052600208, sa_buftype = 1,
>> ššššššsa_size = 8}, {sa_data = 0xfffff803b27e86c8, sa_data_func = 0, sa_length = 8, sa_attr = 11, sa_addr = 0xfffff80052600238, sa_buftype = 1,
>> ššššššsa_size = 8}, {sa_data = 0xfffffe0677ec04e0, sa_data_func = 0, sa_length = 16, sa_attr = 2, sa_addr = 0xfffff80052600260, sa_buftype = 1,
>> ššššššsa_size = 16}, {sa_data = 0xfffffe0677ec05c8, sa_data_func = 0xffffffff82520d50 <zfs_acl_data_locator>, sa_length = 152, sa_attr = 19,
>> ššššššsa_addr = 0xfffff80052600290, sa_buftype = 1, sa_size = 160}, {sa_data = 0xfffff8001ebcfa00, sa_data_func = 0, sa_length = 8, sa_attr = 16,
>> ššššššsa_addr = 0xfffffe0008161250, sa_buftype = 0, sa_size = 48}}
>> ššššššššššctime = {1457872316, 787888506}
>> ššššššššššcount = 5
>> š#19 0xffffffff82524391 in zfs_setacl (zp=0xfffff803b27e85c0, vsecp=0xfffffe0677ec0748, skipaclchk=0, cr=0xfffff804455e6e00)
>> ššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_acl.c:2010
>> ššššššššššzfsvfs = (zfsvfs_t *) 0xfffff80086ee2000
>> ššššššššššzilog = (zilog_t *) 0xfffff80086ebd000
>> ššššššššššmask = 16
>> š---Type <return> to continue, or q <return> to quit---
>> šššššššššštx = (dmu_tx_t *) 0xfffff8034ef07800
>> ššššššššššerror = 0
>> ššššššššššaclp = (zfs_acl_t *) 0xfffff8001ebcfa00
>> ššššššššššfuidp = (zfs_fuid_info_t *) 0x0
>> ššššššššššfuid_dirtied = 0
>> ššššššššššacl_obj = 0
>> š#20 0xffffffff8254e33d in zfs_setsecattr (vp=0xfffff8034dbfa000, vsecp=0xfffffe0677ec0748, flag=0, cr=0xfffff804455e6e00, ct=0x0)
>> ššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:5332
>> ššššššššššzp = (znode_t *) 0xfffff803b27e85c0
>> ššššššššššzfsvfs = (zfsvfs_t *) 0xfffff80086ee2000
>> ššššššššššerror = 2
>> ššššššššššskipaclchk = 0
>> ššššššššššzilog = (zilog_t *) 0xfffff80086ebd000
>> š#21 0xffffffff8254ff5c in zfs_freebsd_setacl (ap=0xfffffe0677ec0808)
>> ššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:7076
>> ššššššššššerror = 0
>> ššššššššššvsecattr = {vsa_mask = 16, vsa_aclcnt = 11, vsa_aclentp = 0xfffff804bd725e00, vsa_dfaclcnt = 2011957208, vsa_dfaclentp = 0xfffffe0677ec07c0,
>> ššššvsa_aclentsz = 132, vsa_aclflags = 2}
>> ššššššššššaclbsize = 132
>> ššššššššššaaclp = (aclent_t *) 0xfffff804bd725e00
>> š#22 0xffffffff816db594 in VOP_SETACL_APV (vop=0xffffffff825de1a0, a=0xfffffe0677ec0808) at vnode_if.c:2993
>> ššššššššššmp__ = (struct mount *) 0xfffff80086df5cc0
>> šššššššššš_enable_stops = 0
>> ššššššššššrc = -1
>> š#23 0xffffffff80f0ba5a in VOP_SETACL (vp=0xfffff8034dbfa000, type=4, aclp=0xfffff800612a1000, cred=0xfffff804455e6e00, td=0xfffff800713124a0)
>> ššššššat vnode_if.h:1253
>> šššššššššša = {a_gen = {a_desc = 0xffffffff81d77eb8}, a_vp = 0xfffff8034dbfa000, a_type = 4, a_aclp = 0xfffff800612a1000, a_cred = 0xfffff804455e6e00,
>> šššša_td = 0xfffff800713124a0}
>> š#24 0xffffffff80f0aea1 in vacl_set_acl (td=0xfffff800713124a0, vp=0xfffff8034dbfa000, type=4, aclp=0x80100a000) at /usr/src/sys/kern/vfs_acl.c:232
>> ššššššššššinkernelacl = (struct acl *) 0xfffff800612a1000
>> ššššššššššmp = (struct mount *) 0xfffff80086df5cc0
>> ššššššššššerror = 0
>> š#25 0xffffffff80f0ad81 in sys___acl_set_file (td=0xfffff800713124a0, uap=0xfffffe0677ec0b98) at /usr/src/sys/kern/vfs_acl.c:375
>> ššššššššššnd = {ni_dirp = 0x7fffffffecd2 <Address 0x7fffffffecd2 out of bounds>, ni_segflg = UIO_USERSPACE, ni_rightsneeded = {cr_rights = {
>> šššššššš144115188075855872, 288230376151711744}}, ni_startdir = 0x0, ni_rootdir = 0xfffff800714cfb10, ni_topdir = 0x0, ni_dirfd = -100,
>> ššššni_strictrelative = 0, ni_filecaps = {fc_rights = {cr_rights = {0, 0}}, fc_ioctls = 0x0, fc_nioctls = -1, fc_fcntls = 0}, ni_vp = 0xfffff8034dbfa000,
>> ššššni_dvp = 0xfffff805c678d000, ni_pathlen = 1, ni_next = 0xfffff80086ddcc65 "", ni_loopcnt = 0, ni_cnd = {cn_nameiop = 0, cn_flags = 49216,
>> ššššššcn_thread = 0xfffff800713124a0, cn_cred = 0xfffff804455e6e00, cn_lkflags = 2097152,
>> ššššššcn_pnbuf = 0xfffff80086ddcc00 "/hdd/usr/wf/_๓-\202-\203ฤษ-\217_-\223-\237/-\234-\236_-\236-\202-\200มฤฮลฮ-\201หฯล -\223-\237/04--\223-\237-\234-\236_-\237-\230-02/07--\224-\200-\203วฯล", cn_nameptr = 0xfffff80086ddcc56 "07--\224-\200-\203วฯล", cn_namelen = 15, cn_consume = 0}}
>> ššššššššššerror = 0
>> š#26 0xffffffff814e32f4 in syscallenter (td=0xfffff800713124a0, sa=0xfffffe0677ec0b88) at subr_syscall.c:134
>> ššššššššššp = (struct proc *) 0xfffff80071317000
>> ššššššššššerror = 0
>> šššššššššštraced = 0
>> š#27 0xffffffff814e2a8f in amd64_syscall (td=0xfffff800713124a0, traced=0) at /usr/src/sys/amd64/amd64/trap.c:952
>> ššššššššššsa = {code = 348, callp = 0xffffffff81d026d0, args = {140737488350418, 4, 34376556544, 2, 140737488345344, 34376515680, 0, 34376896576},
>> ššššnarg = 3}
>> ššššššššššerror = 0
>> ššššššššššksi = {ksi_link = {tqe_next = 0x202, tqe_prev = 0x202}, ksi_info = {si_signo = 2011958080, si_errno = -506, si_code = -2125728811, si_pid = -1,
>> ššššššsi_uid = 4294960432, si_status = 32767, si_addr = 0x202, si_value = {sival_int = 1392864896, sival_ptr = 0xfffffe0653056e80,
>> ššššššššsigval_int = 1392864896, sigval_ptr = 0xfffffe0653056e80}, _reason = {_fault = {_trapno = 1392864896}, _timer = {_timerid = 1392864896,
>> šššššššššš_overrun = -506}, _mesgq = {_mqd = 1392864896}, _poll = {_band = -2171860586880}, __spare__ = {__spare1__ = -2171860586880, __spare2__ = {
>> š---Type <return> to continue, or q <return> to quit---
>> šššššššššššš1392864896, -506, 0, -2048, -723702400, -507, 1899046048}}}}, ksi_flags = 2011958160, ksi_sigq = 0xffffffff80e3028d}
>> š#28 0xffffffff814b9ddb in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:396
>> šNo locals.
>> š#29 0x000000080092a17a in ?? ()
>> šNo symbol table info available.
>> šPrevious frame inner to this frame (corrupt stack?)
>> šCurrent language: auto; currently minimal
>> š(kgdb)q
>>
>> š16.02.2016, 12:10, "Steven Hartland" <killing@multiplay.co.uk>:
>>> šI need the values of the vars specified, so you'll need to:
>>> šprint bp
>>> šIf it reports just an address try:
>>> šprint *bp
>>> šetc.
>>>
>>> šOn 16/02/2016 08:08, DemIS wrote:
>>>> šššframe 8
>>>> ššš#8 0xffffffff81c3669c in dbuf_read (db=0xfffff8013ca46380, zio=0x0, flags=6)
>>>> ššššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:573
>>>> ššš573 (void) arc_read(zio, db->db_objset->os_spa, db->db_blkptr,
>>>> šššCurrent language: auto; currently minimal
>>>>
>>>> šššframe 6
>>>> ššš#6 0xffffffff81c2b9f8 in arc_get_data_buf (buf=<value optimized out>)
>>>> ššššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2898
>>>> ššš2898 buf->b_data = zio_buf_alloc(size);
>>>>
>>>> ššš16.02.2016, 04:15, "Steven Hartland" <killing@multiplay.co.uk>:
>>>>> šššYou don't need the system live just the kernel and the crash dump to get
>>>>> šššthose values.
>>>>>
>>>>> šššOn 16/02/2016 00:46, DemIS wrote:
>>>>>> šššššToday I fell back to version 10.2 . Users work on the server . To reinstall 10.3 I need 4:00 .
>>>>>> šššššTherefore, it is possible only on weekends . But this has the same effect on the version 10.2
>>>>>>
>>>>>> ššššš16.02.2016, 02:55, "Steven Hartland" <killing@multiplay.co.uk>:
>>>>>>> šššššThat sounds like you have some pool data corruption, from the 10.3
>>>>>>> šššššversion dump can you print out the following:
>>>>>>> ššššš1. frame 8: bp and size
>>>>>>> ššššš2. frame 6: buf->b_hdr
>>>>>>>
>>>>>>> šššššOn 15/02/2016 23:26, DemIS wrote:
>>>>>>>> šššššššAny one knows about problem?
>>>>>>>> šššššššServer: SuperMicro Model:SYS-6026T-6RF+, MB:X8DTU-6F+, RAM 24็ย DDR3, two XEON
>>>>>>>> šššššššRAM: KVR1333D3E9S/4G - DDR3, 1333MHz, ECC, CL9, X8, 1.5V, Unbuffered, DIMM
>>>>>>>> šššššššVersion:uname -a
>>>>>>>>
>>>>>>>> šššššššFreeBSD teo.some.loc 10.2-RELEASE-p12 FreeBSD 10.2-RELEASE-p12 #0: Sat Feb 13 18:04:04 MSK 2016 demis@teo.some.loc:/usr/obj/usr/src/sys/TEO amd64
>>>>>>>> ššššššš(on GENERIC or custom kernel config persist too !!!)
>>>>>>>>
>>>>>>>> šššššššMemtest86+ v.4.40 (ECC mode) test - OK.
>>>>>>>> šššššššEvery disk checked too (physically - mhdd, logically - zpool scrub, and additional checkit in external company recovery disk). No errors.
>>>>>>>>
>>>>>>>> šššššššPart of df -H
>>>>>>>> šššššššFilesystem Size Used Avail Capacity Mounted on
>>>>>>>> šššššššhdd/usr/wf 6,6T 4,1T 2,5T 62% /hdd/usr/wf
>>>>>>>>
>>>>>>>> šššššššzpool status hdd
>>>>>>>> ššššššššššpool: hdd
>>>>>>>> šššššššššstate: ONLINE
>>>>>>>> šššššššstatus: Some supported features are not enabled on the pool. The pool can
>>>>>>>> ššššššššššstill be used, but some features are unavailable.
>>>>>>>> šššššššaction: Enable all features using 'zpool upgrade'. Once this is done,
>>>>>>>> ššššššššššthe pool may no longer be accessible by software that does not support
>>>>>>>> ššššššššššthe features. See zpool-features(7) for details.
>>>>>>>> ššššššššššscan: scrub repaired 0 in 14h57m with 0 errors on Thu Feb 11 03:35:43 2016
>>>>>>>> šššššššconfig:
>>>>>>>>
>>>>>>>> ššššššššššNAME STATE READ WRITE CKSUM
>>>>>>>> ššššššššššhdd ONLINE 0 0 0
>>>>>>>> ššššššššššraidz2-0 ONLINE 0 0 0
>>>>>>>> ššššššššššmfid1p1 ONLINE 0 0 0
>>>>>>>> ššššššššššmfid2p1 ONLINE 0 0 0
>>>>>>>> ššššššššššmfid3p1 ONLINE 0 0 0
>>>>>>>> ššššššššššmfid4p1 ONLINE 0 0 0
>>>>>>>> ššššššššššmfid5p1 ONLINE 0 0 0
>>>>>>>>
>>>>>>>> šššššššerrors: No known data errors
>>>>>>>>
>>>>>>>> šššššššhdd - is My zfs volume.
>>>>>>>> šššššššWhen I run command like:
>>>>>>>> šššššššrm /hdd/usr/some/path/to/file
>>>>>>>> šššššššššor
>>>>>>>> šššššššrm /hdd/usr/some/path/to/folder
>>>>>>>> šššššššššor
>>>>>>>> šššššššchown root:wheel /hdd/usr/some/path/to/file
>>>>>>>> šššššššššor
>>>>>>>> šššššššchown root:wheel /hdd/usr/some/path/to/folder
>>>>>>>> šššššššššor
>>>>>>>> šššššššsetfacl ... to /hdd/usr/some/path/to/file
>>>>>>>>
>>>>>>>> šššššššššI'm get kernel panic:
>>>>>>>> šššššššGNU gdb 6.1.1 [FreeBSD]
>>>>>>>> šššššššCopyright 2004 Free Software Foundation, Inc.
>>>>>>>> šššššššGDB is free software, covered by the GNU General Public License, and you are
>>>>>>>> šššššššwelcome to change it and/or distribute copies of it under certain conditions.
>>>>>>>> šššššššType "show copying" to see the conditions.
>>>>>>>> šššššššThere is absolutely no warranty for GDB. Type "show warranty" for details.
>>>>>>>> šššššššThis GDB was configured as "amd64-marcel-freebsd"...
>>>>>>>>
>>>>>>>> šššššššUnread portion of the kernel message buffer:
>>>>>>>> šššššššpanic: solaris assert: c < (1ULL << 24) >> 9 (0x7fffffffffffff < 0x8000), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 270
>>>>>>>> šššššššcpuid = 9
>>>>>>>> šššššššKDB: stack backtrace:
>>>>>>>> ššššššš#0 0xffffffff80984ef0 at kdb_backtrace+0x60
>>>>>>>> ššššššš#1 0xffffffff80948aa6 at vpanic+0x126
>>>>>>>> ššššššš#2 0xffffffff80948973 at panic+0x43
>>>>>>>> ššššššš#3 0xffffffff81c0222f at assfail3+0x2f
>>>>>>>> ššššššš#4 0xffffffff81aa9d40 at zio_buf_alloc+0x50
>>>>>>>> ššššššš#5 0xffffffff81a2b9f8 at arc_get_data_buf+0x358
>>>>>>>> ššššššš#6 0xffffffff81a2e20a at arc_read+0x1ea
>>>>>>>> ššššššš#7 0xffffffff81a3669c at dbuf_read+0x6ac
>>>>>>>> ššššššš#8 0xffffffff81a3d8bf at dmu_spill_hold_existing+0xbf
>>>>>>>> ššššššš#9 0xffffffff81a70dd7 at sa_attr_op+0x167
>>>>>>>> ššššššš#10 0xffffffff81a72ffb at sa_lookup+0x4b
>>>>>>>> ššššššš#11 0xffffffff81abc82a at zfs_rmnode+0x2ba
>>>>>>>> ššššššš#12 0xffffffff81ada58e at zfs_freebsd_reclaim+0x4e
>>>>>>>> ššššššš#13 0xffffffff80e73537 at VOP_RECLAIM_APV+0xa7
>>>>>>>> ššššššš#14 0xffffffff809ec5b4 at vgonel+0x1b4
>>>>>>>> ššššššš#15 0xffffffff809eca49 at vrecycle+0x59
>>>>>>>> ššššššš#16 0xffffffff81ada52d at zfs_freebsd_inactive+0xd
>>>>>>>> ššššššš#17 0xffffffff80e73427 at VOP_INACTIVE_APV+0xa7
>>>>>>>> šššššššUptime: 9m31s
>>>>>>>> šššššššDumping 1286 out of 24543 MB:..2%..12%..22%..32%..42%..51%..61%..71% (CTRL-C to abort) ..81%..91%
>>>>>>>>
>>>>>>>> šššššššReading symbols from /boot/kernel/zfs.ko.symbols...done.
>>>>>>>> šššššššLoaded symbols for /boot/kernel/zfs.ko.symbols
>>>>>>>> šššššššReading symbols from /boot/kernel/opensolaris.ko.symbols...done.
>>>>>>>> šššššššLoaded symbols for /boot/kernel/opensolaris.ko.symbols
>>>>>>>> šššššššReading symbols from /boot/kernel/if_lagg.ko.symbols...done.
>>>>>>>> šššššššLoaded symbols for /boot/kernel/if_lagg.ko.symbols
>>>>>>>> šššššššReading symbols from /boot/kernel/ums.ko.symbols...done.
>>>>>>>> šššššššLoaded symbols for /boot/kernel/ums.ko.symbols
>>>>>>>> šššššššReading symbols from /boot/kernel/ipfw.ko.symbols...done.
>>>>>>>> šššššššLoaded symbols for /boot/kernel/ipfw.ko.symbols
>>>>>>>> ššššššš#0 doadump (textdump=<value optimized out>) at pcpu.h:219
>>>>>>>> ššššššš219 pcpu.h: No such file or directory.
>>>>>>>> ššššššššššin pcpu.h
>>>>>>>> ššššššš(kgdb) bt
>>>>>>>> ššššššš#0 doadump (textdump=<value optimized out>) at pcpu.h:219
>>>>>>>> ššššššš#1 0xffffffff80948702 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:451
>>>>>>>> ššššššš#2 0xffffffff80948ae5 in vpanic (fmt=<value optimized out>, ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:758
>>>>>>>> ššššššš#3 0xffffffff80948973 in panic (fmt=0x0) at /usr/src/sys/kern/kern_shutdown.c:687
>>>>>>>> ššššššš#4 0xffffffff81c0222f in assfail3 (a=<value optimized out>, lv=<value optimized out>, op=<value optimized out>, rv=<value optimized out>,
>>>>>>>> ššššššššššf=<value optimized out>, l=<value optimized out>) at /usr/src/sys/modules/opensolaris/../../cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:91
>>>>>>>> ššššššš#5 0xffffffff81aa9d40 in zio_buf_alloc (size=0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:270
>>>>>>>> ššššššš#6 0xffffffff81a2b9f8 in arc_get_data_buf (buf=<value optimized out>)
>>>>>>>> ššššššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2898
>>>>>>>> ššššššš#7 0xffffffff81a2e20a in arc_read (pio=0xfffff80011791730, spa=0xfffff80011579000, bp=0xfffffe000aee7980, done=0xffffffff81a3a2d0 <dbuf_read_done>,
>>>>>>>> ššššššššššprivate=0xfffff8002244b000, priority=ZIO_PRIORITY_SYNC_READ, zio_flags=-528866606, arc_flags=0xfffffe06727fb3c4, zb=0xffffffff81a3a2d0)
>>>>>>>> ššššššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1551
>>>>>>>> ššššššš#8 0xffffffff81a3669c in dbuf_read (db=0xfffff8002244b000, zio=0x0, flags=6)
>>>>>>>> ššššššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:573
>>>>>>>> ššššššš#9 0xffffffff81a3d8bf in dmu_spill_hold_existing (bonus=0xfffff800223bed20, tag=0x0, dbp=0xfffff800919966b8)
>>>>>>>> ššššššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:333
>>>>>>>> ššššššš#10 0xffffffff81a70dd7 in sa_attr_op (hdl=0xfffff80091996690, bulk=0xfffffe06727fb528, count=1, data_op=SA_LOOKUP, tx=0x0)
>>>>>>>> ššššššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:310
>>>>>>>> ššššššš#11 0xffffffff81a72ffb in sa_lookup (hdl=0xfffff80091996690, attr=<value optimized out>, buf=<value optimized out>, buflen=<value optimized out>)
>>>>>>>> ššššššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:1441
>>>>>>>> ššššššš#12 0xffffffff81abc82a in zfs_rmnode (zp=0xfffff80091993730) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_dir.c:633
>>>>>>>> ššššššš#13 0xffffffff81ada58e in zfs_freebsd_reclaim (ap=<value optimized out>)
>>>>>>>> ššššššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:6569
>>>>>>>> ššššššš#14 0xffffffff80e73537 in VOP_RECLAIM_APV (vop=<value optimized out>, a=<value optimized out>) at vnode_if.c:2019
>>>>>>>> ššššššš#15 0xffffffff809ec5b4 in vgonel (vp=0xfffff800111733b0) at vnode_if.h:830
>>>>>>>> ššššššš#16 0xffffffff809eca49 in vrecycle (vp=0xfffff800111733b0) at /usr/src/sys/kern/vfs_subr.c:2703
>>>>>>>> ššššššš#17 0xffffffff81ada52d in zfs_freebsd_inactive (ap=<value optimized out>)
>>>>>>>> ššššššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:6540
>>>>>>>> ššššššš#18 0xffffffff80e73427 in VOP_INACTIVE_APV (vop=<value optimized out>, a=<value optimized out>) at vnode_if.c:1953
>>>>>>>> ššššššš#19 0xffffffff809eb382 in vinactive (vp=0xfffff800111733b0, td=0xfffff800113d8000) at vnode_if.h:807
>>>>>>>> ššššššš#20 0xffffffff809eb772 in vputx (vp=0xfffff800111733b0, func=2) at /usr/src/sys/kern/vfs_subr.c:2306
>>>>>>>> ššššššš#21 0xffffffff809f401e in kern_rmdirat (td=<value optimized out>, fd=<value optimized out>, path=<value optimized out>, pathseg=<value optimized out>)
>>>>>>>> ššššššššššat /usr/src/sys/kern/vfs_syscalls.c:3842
>>>>>>>> ššššššš#22 0xffffffff80d4b3e7 in amd64_syscall (td=0xfffff800113d8000, traced=0) at subr_syscall.c:134
>>>>>>>> ššššššš#23 0xffffffff80d30acb in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:396
>>>>>>>> ššššššš#24 0x00000008008914ea in ?? ()
>>>>>>>> šššššššPrevious frame inner to this frame (corrupt stack?)
>>>>>>>> šššššššCurrent language: auto; currently minimal
>>>>>>>>
>>>>>>>> šššššššIf setup FreeBSD 10.3 BETA (on GENERIC or custom kernel config):
>>>>>>>> šššššššCopyright 2004 Free Software Foundation, Inc.
>>>>>>>> šššššššGDB is free software, covered by the GNU General Public License, and you are
>>>>>>>> šššššššwelcome to change it and/or distribute copies of it under certain conditions.
>>>>>>>> šššššššType "show copying" to see the conditions.
>>>>>>>> šššššššThere is absolutely no warranty for GDB. Type "show warranty" for details.
>>>>>>>> šššššššThis GDB was configured as "amd64-marcel-freebsd"...
>>>>>>>>
>>>>>>>> šššššššUnread portion of the kernel message buffer:
>>>>>>>> šššššššpanic: solaris assert: c < (1ULL << 24) >> 9 (0x7fffffffffffff < 0x8000), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 273
>>>>>>>> šššššššcpuid = 13
>>>>>>>> šššššššKDB: stack backtrace:
>>>>>>>> ššššššš#0 0xffffffff8098f000 at kdb_backtrace+0x60
>>>>>>>> ššššššš#1 0xffffffff80951d06 at vpanic+0x126
>>>>>>>> ššššššš#2 0xffffffff80951bd3 at panic+0x43
>>>>>>>> ššššššš#3 0xffffffff81e0022f at assfail3+0x2f
>>>>>>>> ššššššš#4 0xffffffff81cacc70 at zio_buf_alloc+0x50
>>>>>>>> ššššššš#5 0xffffffff81c2b8f2 at arc_get_data_buf+0x262
>>>>>>>> ššššššš#6 0xffffffff81c2b657 at arc_buf_alloc+0xc7
>>>>>>>> ššššššš#7 0xffffffff81c2d601 at arc_read+0x1c1
>>>>>>>> ššššššš#8 0xffffffff81c36ce9 at dbuf_read+0x6b9
>>>>>>>> ššššššš#9 0xffffffff81c3e415 at dmu_spill_hold_existing+0xc5
>>>>>>>> ššššššš#10 0xffffffff81c73707 at sa_attr_op+0x167
>>>>>>>> ššššššš#11 0xffffffff81c75972 at sa_lookup+0x52
>>>>>>>> ššššššš#12 0xffffffff81cbf8da at zfs_rmnode+0x2ba
>>>>>>>> ššššššš#13 0xffffffff81cdd75e at zfs_freebsd_reclaim+0x4e
>>>>>>>> ššššššš#14 0xffffffff80e81c27 at VOP_RECLAIM_APV+0xa7
>>>>>>>> ššššššš#15 0xffffffff809f9581 at vgonel+0x221
>>>>>>>> ššššššš#16 0xffffffff809f9a19 at vrecycle+0x59
>>>>>>>> ššššššš#17 0xffffffff81cdd6fd at zfs_freebsd_inactive+0xd
>>>>>>>> šššššššUptime: 11m11s
>>>>>>>> šššššššDumping 1368 out of 24542 MB:..2%..11%..22%..31%..41%..51%..61%..71%..81%..91%
>>>>>>>>
>>>>>>>> šššššššReading symbols from /boot/kernel/if_lagg.ko.symbols...done.
>>>>>>>> šššššššLoaded symbols for /boot/kernel/if_lagg.ko.symbols
>>>>>>>> šššššššReading symbols from /boot/kernel/aio.ko.symbols...done.
>>>>>>>> šššššššLoaded symbols for /boot/kernel/aio.ko.symbols
>>>>>>>> šššššššReading symbols from /boot/kernel/ichsmb.ko.symbols...done.
>>>>>>>> šššššššLoaded symbols for /boot/kernel/ichsmb.ko.symbols
>>>>>>>> šššššššReading symbols from /boot/kernel/smbus.ko.symbols...done.
>>>>>>>> šššššššLoaded symbols for /boot/kernel/smbus.ko.symbols
>>>>>>>> šššššššReading symbols from /boot/kernel/ipmi.ko.symbols...done.
>>>>>>>> šššššššLoaded symbols for /boot/kernel/ipmi.ko.symbols
>>>>>>>> šššššššReading symbols from /boot/kernel/zfs.ko.symbols...done.
>>>>>>>> šššššššLoaded symbols for /boot/kernel/zfs.ko.symbols
>>>>>>>> šššššššReading symbols from /boot/kernel/opensolaris.ko.symbols...done.
>>>>>>>> šššššššLoaded symbols for /boot/kernel/opensolaris.ko.symbols
>>>>>>>> šššššššReading symbols from /boot/kernel/ums.ko.symbols...done.
>>>>>>>> šššššššLoaded symbols for /boot/kernel/ums.ko.symbols
>>>>>>>> šššššššReading symbols from /boot/kernel/ipfw.ko.symbols...done.
>>>>>>>> šššššššLoaded symbols for /boot/kernel/ipfw.ko.symbols
>>>>>>>> ššššššš#0 doadump (textdump=<value optimized out>) at pcpu.h:219
>>>>>>>> ššššššš219 pcpu.h: No such file or directory.
>>>>>>>> ššššššššššin pcpu.h
>>>>>>>> ššššššš(kgdb) backtrace
>>>>>>>> ššššššš#0 doadump (textdump=<value optimized out>) at pcpu.h:219
>>>>>>>> ššššššš#1 0xffffffff80951962 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:486
>>>>>>>> ššššššš#2 0xffffffff80951d45 in vpanic (fmt=<value optimized out>, ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:889
>>>>>>>> ššššššš#3 0xffffffff80951bd3 in panic (fmt=0x0) at /usr/src/sys/kern/kern_shutdown.c:818
>>>>>>>> ššššššš#4 0xffffffff81e0022f in assfail3 (a=<value optimized out>, lv=<value optimized out>, op=<value optimized out>, rv=<value optimized out>, f=<value optimized out>,
>>>>>>>> ššššššššššl=<value optimized out>) at /usr/src/sys/modules/opensolaris/../../cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:91
>>>>>>>> ššššššš#5 0xffffffff81cacc70 in zio_buf_alloc (size=0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:273
>>>>>>>> ššššššš#6 0xffffffff81c2b8f2 in arc_get_data_buf (buf=<value optimized out>) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:3880
>>>>>>>> ššššššš#7 0xffffffff81c2b657 in arc_buf_alloc (spa=<value optimized out>, size=<value optimized out>, tag=0x0, type=<value optimized out>)
>>>>>>>> ššššššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2057
>>>>>>>> ššššššš#8 0xffffffff81c2d601 in arc_read (pio=0xfffff8000fad03b0, spa=0xfffff8000f63d000, bp=0xfffffe000e509980, done=0xffffffff81c3aed0 <dbuf_read_done>, private=0xfffff8000fdd6360,
>>>>>>>> ššššššššššpriority=ZIO_PRIORITY_SYNC_READ, zio_flags=-2117882160, arc_flags=0xfffffe02925483c4, zb=0xfffff8000fdd6360)
>>>>>>>> ššššššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:4397
>>>>>>>> ššššššš#9 0xffffffff81c36ce9 in dbuf_read (db=0xfffff8000fdd6360, zio=0x0, flags=<value optimized out>)
>>>>>>>> ššššššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:682
>>>>>>>> ššššššš#10 0xffffffff81c3e415 in dmu_spill_hold_existing (bonus=0xfffff8001f312438, tag=0x0, dbp=0xfffff80062d4e7d0)
>>>>>>>> ššššššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:333
>>>>>>>> ššššššš#11 0xffffffff81c73707 in sa_attr_op (hdl=0xfffff80062d4e770, bulk=0xfffffe0292548528, count=1, data_op=SA_LOOKUP, tx=0x0)
>>>>>>>> ššššššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:305
>>>>>>>> ššššššš#12 0xffffffff81c75972 in sa_lookup (hdl=0xfffff80062d4e770, attr=<value optimized out>, buf=<value optimized out>, buflen=<value optimized out>)
>>>>>>>> ššššššššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:1443
>>>>>>>> ššššššš#13 0xffffffff81cbf8da in zfs_rmnode (zp=0xfffff80062d4c8a0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_dir.c:633
>>>>>>>> ššššššš#14 0xffffffff81cdd75e in zfs_freebsd_reclaim (ap=<value optimized out>) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:6619
>>>>>>>> ššššššš#15 0xffffffff80e81c27 in VOP_RECLAIM_APV (vop=<value optimized out>, a=<value optimized out>) at vnode_if.c:2019
>>>>>>>> ššššššš#16 0xffffffff809f9581 in vgonel (vp=0xfffff8000f1beb10) at vnode_if.h:830
>>>>>>>> ššššššš#17 0xffffffff809f9a19 in vrecycle (vp=0xfffff8000f1beb10) at /usr/src/sys/kern/vfs_subr.c:2951
>>>>>>>> ššššššš#18 0xffffffff81cdd6fd in zfs_freebsd_inactive (ap=<value optimized out>) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:6590
>>>>>>>> ššššššš#19 0xffffffff80e81b17 in VOP_INACTIVE_APV (vop=<value optimized out>, a=<value optimized out>) at vnode_if.c:1953
>>>>>>>> ššššššš#20 0xffffffff809f8322 in vinactive (vp=0xfffff8000f1beb10, td=0xfffff8000f9f34b0) at vnode_if.h:807
>>>>>>>> ššššššš#21 0xffffffff809f8712 in vputx (vp=0xfffff8000f1beb10, func=2) at /usr/src/sys/kern/vfs_subr.c:2547
>>>>>>>> ššššššš#22 0xffffffff80a0137e in kern_rmdirat (td=<value optimized out>, fd=<value optimized out>, path=<value optimized out>, pathseg=<value optimized out>)
>>>>>>>> ššššššššššat /usr/src/sys/kern/vfs_syscalls.c:3964
>>>>>>>> ššššššš#23 0xffffffff80d574bf in amd64_syscall (td=0xfffff8000f9f34b0, traced=0) at subr_syscall.c:141
>>>>>>>> ššššššš#24 0xffffffff80d3c72b in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:396
>>>>>>>> ššššššš#25 0x000000080089458a in ?? ()
>>>>>>>> šššššššPrevious frame inner to this frame (corrupt stack?)
>>>>>>>> šššššššCurrent language: auto; currently minimal
>>>>>>>>
>>>>>>>> šššššššCrash folder (or file) have strange rights:
>>>>>>>> šššššššd---------+ 3 anna domain users 3 10 ฤลห 10:32 01-Projcts
>>>>>>>> šššššššd---------+ 2 anna domain users 2 8 ฦลื 21:46 02-Text
>>>>>>>>
>>>>>>>> šššššššHow correct kernel panic?
>>>>>>>>
>>>>>>>> ššššššš_______________________________________________
>>>>>>>> šššššššfreebsd-fs@freebsd.org mailing list
>>>>>>>> šššššššhttps://lists.freebsd.org/mailman/listinfo/freebsd-fs
>>>>>>>> šššššššTo unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"
>>>>>>> ššššš_______________________________________________
>>>>>>> šššššfreebsd-fs@freebsd.org mailing list
>>>>>>> šššššhttps://lists.freebsd.org/mailman/listinfo/freebsd-fs
>>>>>>> šššššTo unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?711901457909452>