From owner-freebsd-current@FreeBSD.ORG Tue Jun 16 22:10:27 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B625E106566B for ; Tue, 16 Jun 2009 22:10:27 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from yw-out-2324.google.com (yw-out-2324.google.com [74.125.46.31]) by mx1.freebsd.org (Postfix) with ESMTP id 70B628FC17 for ; Tue, 16 Jun 2009 22:10:27 +0000 (UTC) (envelope-from artemb@gmail.com) Received: by yw-out-2324.google.com with SMTP id 9so2473906ywe.13 for ; Tue, 16 Jun 2009 15:10:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; bh=nK0KCXbUjIzNcx6veAniuVvTqUMtxKcJ7L6mBHHx/C8=; b=Hj64q0hjOxqqJ5AUsbTeIiAA8Y7ChDnOUF8To0TnU5Xm/zdZ7QDckJ5K79WOOQOBtK NiaQ8PD7EVwELXnUMk0vUTrotHU/YKEa9bOtKuSN6cX5rFrGjEyuD1jAhuaoTrMkQGBX OjT8YQZO1R6WSNDvrBUvbLsrSQuKHgDQHOEB4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type:content-transfer-encoding; b=GOeBC/kbfMWoyZnz2Nc9ZcuTYC+VO1X4ZBYMu6vzEHkJ0AHdxQynUIRvSUINNx212M pQTHhZdohIJq4Xo/8L4YJJbMGTkNXqJ0aPApkq44kxUc5m8CVklR+URw0fJjiBtYRRrS O73hVqSid7EOMUX3UssEGoXvZuwRuL2Y9ej4s= MIME-Version: 1.0 Sender: artemb@gmail.com Received: by 10.90.33.15 with SMTP id g15mr2986098agg.115.1245190226674; Tue, 16 Jun 2009 15:10:26 -0700 (PDT) Date: Tue, 16 Jun 2009 15:10:26 -0700 X-Google-Sender-Auth: bd0b6012bac4c95c Message-ID: From: Artem Belevich To: freebsd-current@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Crash in ZFS during vnode sync X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jun 2009 22:10:27 -0000 Got a new crash on fresh (as of yesterday) -current/amd64 while the box was pretty much idle. Haven't seen this one before. Fatal trap 12: page fault while in kernel mode cpuid = 3; apic id = 03 fault virtual address = 0xd8 fault code = supervisor write data, page not present instruction pointer = 0x20:0xffffffff80360a35 stack pointer = 0x28:0xffffff842cfc8900 frame pointer = 0x28:0xffffff842cfc8910 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 48 (syncer) [thread pid 48 tid 100076 ] Stopped at _mtx_lock_flags+0x15: lock cmpxchgq %rsi,0x18(%rdi) db> where Tracing pid 48 tid 100076 td 0xffffff0007b0a390 _mtx_lock_flags() at _mtx_lock_flags+0x15 vn_rele_async() at vn_rele_async+0x31 zfs_get_data() at zfs_get_data+0xd0 zil_commit() at zil_commit+0x532 zfs_sync() at zfs_sync+0xa6 sync_fsync() at sync_fsync+0x184 VOP_FSYNC_APV() at VOP_FSYNC_APV+0x4a sync_vnode() at sync_vnode+0x16b sched_sync() at sched_sync+0x1c9 fork_exit() at fork_exit+0x118 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff842cfc8d40, rbp = 0 --- db> show lockedbufs buf at 0xffffff83c11d8550 b_flags = 0xa00200a0 b_error = 0, b_bufsize = 4096, b_bcount = 4096, b_resid = 0 b_bufobj = (0xffffff0012c7a2f0), b_data = 0xffffff83da5fa000, b_blkno = 19214240, b_dep = 0 b_npages = 1, pages(OBJ, IDX, PA): (0xffffff003d5b47d0, 0x0, 0x3c082000) lock type bufwait: EXCL by thread 0xffffff0010e85720 (pid 886) db> show lockedvnods Locked vnodes 0xffffff0012c7a1d8: tag ufs, type VREG usecount 1, writecount 1, refcount 3 mountedhere 0 flags () v_object 0xffffff003d5b47d0 ref 0 pages 1 lock type ufs: EXCL by thread 0xffffff0010e85720 (pid 886) ino 1201240, on dev ad4s1a 0xffffff0007db9ce8: tag syncer, type VNON usecount 1, writecount 0, refcount 2 mountedhere 0 flags () lock type syncer: EXCL by thread 0xffffff0007b0a390 (pid 48) db> show vnodebufs usage: show vnodebufs db> show vnodebufs 0xffffff0007db9ce8 Clean buffers: Dirty buffers: db> show vnodebufs 0xffffff0012c7a1d8 Clean buffers: Dirty buffers: buf at 0xffffff83c11d8550 b_flags = 0xa00200a0 b_error = 0, b_bufsize = 4096, b_bcount = 4096, b_resid = 0 b_bufobj = (0xffffff0012c7a2f0), b_data = 0xffffff83da5fa000, b_blkno = 19214240, b_dep = 0 b_npages = 1, pages(OBJ, IDX, PA): (0xffffff003d5b47d0, 0x0, 0x3c082000) lock type bufwait: EXCL by thread 0xffffff0010e85720 (pid 886) db> bt 886 Tracing pid 886 tid 100183 td 0xffffff0010e85720 cpustop_handler() at cpustop_handler+0x3b ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x195 nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff803605ca, rsp = 0xffffff800001bfe0, rbp = 0xffffff842f52b7f0 --- _mtx_lock_sleep() at _mtx_lock_sleep+0x11a bdwrite() at bdwrite+0x313 ffs_write() at ffs_write+0x634 VOP_WRITE_APV() at VOP_WRITE_APV+0xc6 vn_write() at vn_write+0x188 dofilewrite() at dofilewrite+0x85 kern_writev() at kern_writev+0x60 writev() at writev+0x41 syscall() at syscall+0x28f Xfast_syscall() at Xfast_syscall+0xd0 --- syscall (121, FreeBSD ELF64, writev), rip = 0x80083501c, rsp = 0x7fffffffcaf8, rbp = 0 --- db> bt 48 Tracing pid 48 tid 100076 td 0xffffff0007b0a390 _mtx_lock_flags() at _mtx_lock_flags+0x15 vn_rele_async() at vn_rele_async+0x31 zfs_get_data() at zfs_get_data+0xd0 zil_commit() at zil_commit+0x532 zfs_sync() at zfs_sync+0xa6 sync_fsync() at sync_fsync+0x184 VOP_FSYNC_APV() at VOP_FSYNC_APV+0x4a sync_vnode() at sync_vnode+0x16b sched_sync() at sched_sync+0x1c9 fork_exit() at fork_exit+0x118 fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff842cfc8d40, rbp = 0 --- --Artem