From owner-freebsd-fs@FreeBSD.ORG Thu Jul 17 09:13:32 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 700CCA7E for ; Thu, 17 Jul 2014 09:13:32 +0000 (UTC) Received: from mail-wg0-x229.google.com (mail-wg0-x229.google.com [IPv6:2a00:1450:400c:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 092AD2076 for ; Thu, 17 Jul 2014 09:13:31 +0000 (UTC) Received: by mail-wg0-f41.google.com with SMTP id z12so2139994wgg.24 for ; Thu, 17 Jul 2014 02:13:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=mILvyCVI+spWTXLeM3kOJf6+jJ39Q1K6ZLLXgPcb038=; b=dzuUW9yC/54KDGa6kvSakHNP57aTAMlnvWeJMuBODJn9o4fM8qfAycylOyii7CgGTx CmombIFvUUMzULq0FVpW6NrEcx63nY87tx7v5NAy68esft0O006LXqcXTOzlayUtvl8z nnhPl5bXZB689tz7bJeOblZcmOLz9pp5vahWHNiQPD+JJtEC6Ud21Oy73fLKRJQBqC2w S912BfuIIb4zcCREPeSUzW/n3Q55HPTUbimsiMljr2S+iUtSEPJjL6UUAo3VkgYpoObD h6FWk3Yuo9Xy654eSO5XOk6GezXXd9bAGU7RYSxkWzwL4GEQ69WPIu8kx6hTOdMn9NGQ kAHQ== X-Received: by 10.180.221.172 with SMTP id qf12mr10842862wic.54.1405588408183; Thu, 17 Jul 2014 02:13:28 -0700 (PDT) Received: from [192.168.1.145] ([193.173.55.180]) by mx.google.com with ESMTPSA id jv9sm4673660wjc.28.2014.07.17.02.13.27 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 17 Jul 2014 02:13:27 -0700 (PDT) Message-ID: <53C793B4.4040700@gmail.com> Date: Thu, 17 Jul 2014 11:13:24 +0200 From: Johan Hendriks User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Re: zpool upgrade does not work with latest 10 STABLE References: <53C54607.2090100@gmail.com> In-Reply-To: <53C54607.2090100@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Jul 2014 09:13:32 -0000 A rebuild did the job, probably I have made some error. It is all working like it is supposed to work. sorry for the noise.. regards op 15-07-14 17:17, Johan Hendriks schreef: > Hello all. > > Two days ago I upgraded my zpool to the latest version using zpool > upgrade. This worked fine. Today there where some new features added > to ZFS and I did rebuild the system. A zpool status shows me that the > pool can be upgraded. > > beasty ~ # zpool list > NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT > storage 1.81T 1.06T 767G 58% 1.00x ONLINE - > > beasty ~ # zpool status > pool: storage > state: ONLINE > status: Some supported features are not enabled on the pool. The pool can > still be used, but some features are unavailable. > action: Enable all features using 'zpool upgrade'. Once this is done, > the pool may no longer be accessible by software that does not > support > the features. See zpool-features(7) for details. > scan: scrub repaired 0 in 2h28m with 0 errors on Tue Jul 15 13:57:56 > 2014 > config: > > NAME STATE READ WRITE CKSUM > storage ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > gpt/disk01 ONLINE 0 0 0 > gpt/disk02 ONLINE 0 0 0 > mirror-1 ONLINE 0 0 0 > gpt/disk03 ONLINE 0 0 0 > gpt/disk04-r ONLINE 0 0 0 > > errors: No known data errors > > beasty ~ # uname -a > FreeBSD beasty.mydomain.local 10.0-STABLE FreeBSD 10.0-STABLE #0 > r268604: Mon Jul 14 11:01:01 CEST 2014 > root@beasty.mydomain.local:/usr/obj/usr/src/sys/KRNL amd64 > > beasty ~ # zpool upgrade > This system supports ZFS pool feature flags. > > All pools are formatted using feature flags. > > Some supported features are not enabled on the following pools. Once a > feature is enabled the pool may become incompatible with software > that does not support the feature. See zpool-features(7) for details. > > POOL FEATURE > --------------- > storage > embedded_data > > > But when i do a zpool upgrade it give me an error. > > beasty ~ # zpool upgrade storage > This system supports ZFS pool feature flags. > > cannot set property for 'storage': invalid argument for this pool > operation > > > All things seem to work, and the scrub did a good job also.... > > Is there something I am missing? > > regards > Johan > > > > > > > From owner-freebsd-fs@FreeBSD.ORG Fri Jul 18 03:21:24 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DDD8DD9A for ; Fri, 18 Jul 2014 03:21:24 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C59DE2F0F for ; Fri, 18 Jul 2014 03:21:24 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6I3LOAp071439 for ; Fri, 18 Jul 2014 03:21:24 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 191573] [zfs] kernel panic when running zpool/add/files.t Date: Fri, 18 Jul 2014 03:21:24 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: yaneurabeya@gmail.com X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jul 2014 03:21:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191573 --- Comment #10 from yaneurabeya@gmail.com --- Crashes on amd64 with 64GB of RAM, so the architecture and memory doesn't appear to matter. The tests are triggering a panic scenario with ZFS. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Fri Jul 18 03:25:08 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BE457E60 for ; Fri, 18 Jul 2014 03:25:08 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8AE682F3D for ; Fri, 18 Jul 2014 03:25:08 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6I3P8TW056921 for ; Fri, 18 Jul 2014 03:25:08 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 191573] [zfs] kernel panic when running zpool/add/files.t Date: Fri, 18 Jul 2014 03:25:08 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: yaneurabeya@gmail.com X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jul 2014 03:25:08 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191573 --- Comment #11 from yaneurabeya@gmail.com --- Slight correction. I meant 256GB of RAM. db> show msgbuf msgbufp = 0xfffff8403fffffb8 magic = 63062, size = 98232, r= 21200, w = 21678, ptr = 0xfffff8403ffe8000, cksum= 1683664 Fatal trap 12: page fault while in kernel mode cpuid = 6; apic id = 20 fault virtual address = 0x0 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80d1c5a6 stack pointer = 0x28:0xfffffe3fcf2090a0 frame pointer = 0x28:0xfffffe3fcf209110 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 1894 (zpool) Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-CURRENT #0 a32b4d9(master)-dirty: Thu Jul 17 03:09:09 PDT 2014 root@mina3020.localdomain:/usr/obj/scratch/freebsd/sys/GENERIC amd64 FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 WARNING: WITNESS option enabled, expect reduced performance. CPU: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz (1995.24-MHz K8-class CPU) Origin="GenuineIntel" Id=0x206d7 Family=0x6 Model=0x2d Stepping=7 Features=0xbfebfbff Features2=0x1fbee3ff AMD Features=0x2c100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 274877906944 (262144 MB) avail memory = 267130585088 (254755 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 2 package(s) x 6 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 2 cpu2 (AP): APIC ID: 4 cpu3 (AP): APIC ID: 6 cpu4 (AP): APIC ID: 8 cpu5 (AP): APIC ID: 10 cpu6 (AP): APIC ID: 32 cpu7 (AP): APIC ID: 34 cpu8 (AP): APIC ID: 36 cpu9 (AP): APIC ID: 38 cpu10 (AP): APIC ID: 40 cpu11 (AP): APIC ID: 42 ACPI BIOS Warning (bug): Invalid length for FADT/Pm1aControlBlock: 32, using default 16 (20130823/tbfadt-682) ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard ioapic2 irqs 48-71 on motherboard kbd1 at kbdmux0 random: initialized acpi0: on motherboard acpi0: Power Button (fixed) acpi0: reservation of 0, 9d000 (3) failed cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 cpu8: on acpi0 cpu9: on acpi0 cpu10: on acpi0 cpu11: on acpi0 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 atrtc0: port 0x70-0x77 irq 8 on acpi0 atrtc0: Warning: Couldn't map I/O. Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 47 at device 1.0 on pci0 pci1: on pcib1 pcib2: irq 47 at device 1.1 on pci0 pci2: on pcib2 igb0: port 0x3060-0x307f mem 0xd0b60000-0xd0b7ffff,0xd0bb0000-0xd0bb3fff irq 27 at device 0.0 on pci2 igb0: Using MSIX interrupts with 9 vectors igb0: Ethernet address: 00:1e:67:6d:7a:0c igb0: Bound queue 0 to cpu 0 igb0: Bound queue 1 to cpu 1 igb0: Bound queue 2 to cpu 2 igb0: Bound queue 3 to cpu 3 igb0: Bound queue 4 to cpu 4 igb0: Bound queue 5 to cpu 5 igb0: Bound queue 6 to cpu 6 igb0: Bound queue 7 to cpu 7 igb1: port 0x3040-0x305f mem 0xd0b40000-0xd0b5ffff,0xd0ba0000-0xd0ba3fff irq 30 at device 0.1 on pci2 igb1: Using MSIX interrupts with 9 vectors igb1: Ethernet address: 00:1e:67:6d:7a:0d igb1: Bound queue 0 to cpu 8 igb1: Bound queue 1 to cpu 9 igb1: Bound queue 2 to cpu 10 igb1: Bound queue 3 to cpu 11 igb1: Bound queue 4 to cpu 0 igb1: Bound queue 5 to cpu 1 igb1: Bound queue 6 to cpu 2 igb1: Bound queue 7 to cpu 3 igb2: port 0x3020-0x303f mem 0xd0b20000-0xd0b3ffff,0xd0b90000-0xd0b93fff irq 28 at device 0.2 on pci2 igb2: Using MSIX interrupts with 9 vectors igb2: Ethernet address: 00:1e:67:6d:7a:0e igb2: Bound queue 0 to cpu 4 igb2: Bound queue 1 to cpu 5 igb2: Bound queue 2 to cpu 6 igb2: Bound queue 3 to cpu 7 igb2: Bound queue 4 to cpu 8 igb2: Bound queue 5 to cpu 9 igb2: Bound queue 6 to cpu 10 igb2: Bound queue 7 to cpu 11 igb3: port 0x3000-0x301f mem 0xd0b00000-0xd0b1ffff,0xd0b80000-0xd0b83fff irq 29 at device 0.3 on pci2 igb3: Using MSIX interrupts with 9 vectors igb3: Ethernet address: 00:1e:67:6d:7a:0f igb3: Bound queue 0 to cpu 0 igb3: Bound queue 1 to cpu 1 igb3: Bound queue 2 to cpu 2 igb3: Bound queue 3 to cpu 3 igb3: Bound queue 4 to cpu 4 igb3: Bound queue 5 to cpu 5 igb3: Bound queue 6 to cpu 6 igb3: Bound queue 7 to cpu 7 pcib3: irq 47 at device 2.0 on pci0 pci4: on pcib3 ix0: port 0x2020-0x203f mem 0xd0e20000-0xd0e3ffff,0xd0e50000-0xd0e53fff irq 32 at device 0.0 on pci4 ix0: Using MSIX interrupts with 9 vectors ix0: Bound queue 0 to cpu 0 ix0: Bound queue 1 to cpu 1 ix0: Bound queue 2 to cpu 2 ix0: Bound queue 3 to cpu 3 ix0: Bound queue 4 to cpu 4 ix0: Bound queue 5 to cpu 5 ix0: Bound queue 6 to cpu 6 ix0: Bound queue 7 to cpu 7 ix0: Ethernet address: 00:1e:67:5b:8b:f4 ix0: PCI Express Bus: Speed 5.0GT/s Width x8 ix1: port 0x2000-0x201f mem 0xd0e00000-0xd0e1ffff,0xd0e40000-0xd0e43fff irq 36 at device 0.1 on pci4 ix1: Using MSIX interrupts with 9 vectors ix1: Bound queue 0 to cpu 0 ix1: Bound queue 1 to cpu 1 ix1: Bound queue 2 to cpu 2 ix1: Bound queue 3 to cpu 3 ix1: Bound queue 4 to cpu 4 ix1: Bound queue 5 to cpu 5 ix1: Bound queue 6 to cpu 6 ix1: Bound queue 7 to cpu 7 ix1: Ethernet address: 00:1e:67:5b:8b:f5 ix1: PCI Express Bus: Speed 5.0GT/s Width x8 pcib4: irq 47 at device 2.2 on pci0 pci6: on pcib4 mps0: port 0x1000-0x10ff mem 0xd0a40000-0xd0a4ffff,0xd0a00000-0xd0a3ffff irq 34 at device 0.0 on pci6 mps0: Firmware: 13.00.57.00, Driver: 16.00.00.00-fbsd mps0: IOCCapabilities: 185c pcib5: irq 16 at device 3.0 on pci0 pci7: on pcib5 pci7: at device 0.0 (no driver attached) pcib6: irq 16 at device 17.0 on pci0 pci8: on pcib6 pci0: at device 22.0 (no driver attached) pci0: at device 22.1 (no driver attached) ehci0: mem 0xd0f20000-0xd0f203ff irq 22 at device 26.0 on pci0 usbus0: EHCI version 1.0 usbus0 on ehci0 pcib7: irq 16 at device 28.0 on pci0 pci9: on pcib7 pcib8: irq 19 at device 28.7 on pci0 pci10: on pcib8 vgapci0: mem 0xea000000-0xeaffffff,0xd0810000-0xd0813fff,0xd0000000-0xd07fffff irq 19 at device 0.0 on pci10 vgapci0: Boot video device ehci1: mem 0xd0f10000-0xd0f103ff irq 20 at device 29.0 on pci0 usbus1: EHCI version 1.0 usbus1 on ehci1 pcib9: at device 30.0 on pci0 pci11: on pcib9 isab0: at device 31.0 on pci0 isa0: on isab0 ahci0: port 0x4070-0x4077,0x4060-0x4063,0x4050-0x4057,0x4040-0x4043,0x4020-0x403f mem 0xd0f00000-0xd0f007ff irq 21 at device 31.2 on pci0 ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahcich4: at channel 4 on ahci0 ahcich5: at channel 5 on ahci0 ahciem0: on ahci0 pcib10: on acpi0 pci128: on pcib10 pcib11: irq 71 at device 1.0 on pci128 pci129: on pcib11 pcib12: irq 71 at device 2.0 on pci128 pci130: on pcib12 pcib13: irq 71 at device 3.0 on pci128 pci131: on pcib13 pcib14: irq 71 at device 3.2 on pci128 pci132: on pcib14 pcib15: on acpi0 pci127: on pcib15 pcib16: on acpi0 pci255: on pcib16 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (115200,n,8,1) uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 orm0: at iomem 0xc0000-0xc7fff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 ppc0: cannot reserve I/O port range est0: on cpu0 est1: on cpu1 est2: on cpu2 est3: on cpu3 est4: on cpu4 est5: on cpu5 est6: on cpu6 est7: on cpu7 est8: on cpu8 est9: on cpu9 est10: on cpu10 est11: on cpu11 Timecounters tick every 1.000 msec random: unblocking device. usbus0: 480Mbps High Speed USB v2.0 usbus1: 480Mbps High Speed USB v2.0 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered ugen0.2: at usbus0 uhub2: on usbus0 ugen1.2: at usbus1 uhub3: on usbus1 uhub2: 6 ports with 6 removable, self powered uhub3: 8 ports with 8 removable, self powered ugen1.3: at usbus1 ukbd0: on usbus1 kbd0 at ukbd0 da0 at mps0 bus 0 scbus0 target 3 lun 0 da0: Fixed Direct Access SCSI-6 device da0: Serial Number PWHKHGPD da0: 600.000MB/s transfers da0: Command Queueing enabled da0: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C) da1 at mps0 bus 0 scbus0 target 5 lun 0 da1: Fixed Direct Access SCSI-6 device da1: Serial Number PWHL87KF da1: 600.000MB/s transfers da1: Command Queueing enabled da1: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C) ses0 at ahciem0 bus 0 scbus7 target 0 lun 0 ses0: SEMB S-E-S 2.00 device ses0: SEMB SES Device SMP: AP CPU #9 Launched! SMP: AP CPU #8 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #11 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #10 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #2 Launched! Timecounter "TSC" frequency 1995238860 Hz quality 1000 SMP: AP CPU #7 Launched! SMP: AP CPU #2 Launched! Timecounter "TSC" frequency 1995238860 Hz quality 1000 WARNING: WITNESS option enabled, expect reduced performance. Trying to mount root from ufs:/dev/da0p2 [rw]... ... lock order reversal: 1st 0xfffffe3e57ffcea8 bufwait (bufwait) @ /scratch/freebsd/sys/kern/vfs_bio.c:3088 2nd 0xfffff8009a9fcc00 dirhash (dirhash) @ /scratch/freebsd/sys/ufs/ufs/ufs_dirhash.c:284 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe3fcf065510 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe3fcf0655c0 witness_checkorder() at witness_checkorder+0xdc2/frame 0xfffffe3fcf065650 _sx_xlock() at _sx_xlock+0x75/frame 0xfffffe3fcf065690 ufsdirhash_remove() at ufsdirhash_remove+0x37/frame 0xfffffe3fcf0656c0 ufs_dirremove() at ufs_dirremove+0x11b/frame 0xfffffe3fcf065720 ufs_remove() at ufs_remove+0x75/frame 0xfffffe3fcf065780 VOP_REMOVE_APV() at VOP_REMOVE_APV+0xf7/frame 0xfffffe3fcf0657b0 kern_unlinkat() at kern_unlinkat+0x209/frame 0xfffffe3fcf0659a0 amd64_syscall() at amd64_syscall+0x25a/frame 0xfffffe3fcf065ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe3fcf065ab0 --- syscall (10, FreeBSD ELF64, sys_unlink), rip = 0x8014dd8aa, rsp = 0x7fffffffce58, rbp = 0x7fffffffcf10 --- ZFS filesystem version: 5 ZFS storage pool version: features support (5000) -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Fri Jul 18 08:18:05 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CF6614E1 for ; Fri, 18 Jul 2014 08:18:05 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B7E9F27FF for ; Fri, 18 Jul 2014 08:18:05 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6I8I5WY004052 for ; Fri, 18 Jul 2014 08:18:05 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 191510] [zfs] ZFS doesn't use all available memory Date: Fri, 18 Jul 2014 08:18:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.3-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: vsjcfm@gmail.com X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jul 2014 08:18:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191510 --- Comment #11 from vsjcfm@gmail.com --- Created attachment 144771 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=144771&action=edit Memory graph #2 A new graph. Server was rebooted because of power problem. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Fri Jul 18 10:53:13 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EE9A79DF for ; Fri, 18 Jul 2014 10:53:13 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D6F062539 for ; Fri, 18 Jul 2014 10:53:13 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6IArDDi039925 for ; Fri, 18 Jul 2014 10:53:13 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 191573] [zfs] kernel panic when running zpool/add/files.t Date: Fri, 18 Jul 2014 10:53:13 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: smh@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jul 2014 10:53:14 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191573 --- Comment #12 from Steven Hartland --- Thats actually a crash in UFS there not in ZFS. Can you try on a ZFS only install to see if it is strictly a UFS issue please? -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Fri Jul 18 11:00:53 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 300F6B20 for ; Fri, 18 Jul 2014 11:00:53 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1872F258C for ; Fri, 18 Jul 2014 11:00:53 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6IB0qpd056190 for ; Fri, 18 Jul 2014 11:00:52 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 191573] [zfs] kernel panic when running zpool/add/files.t Date: Fri, 18 Jul 2014 11:00:53 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: yaneurabeya@gmail.com X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jul 2014 11:00:53 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191573 --- Comment #13 from yaneurabeya@gmail.com --- The witness warning is from UFS (and it's one of the well known ones), but the panic was in the ZFS code. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Sat Jul 19 10:01:03 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A05E57DE for ; Sat, 19 Jul 2014 10:01:03 +0000 (UTC) Received: from smtprelay06.ispgateway.de (smtprelay06.ispgateway.de [80.67.31.102]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0DE7D2C10 for ; Sat, 19 Jul 2014 10:01:02 +0000 (UTC) Received: from [78.35.154.137] (helo=fabiankeil.de) by smtprelay06.ispgateway.de with esmtpsa (SSLv3:AES128-SHA:128) (Exim 4.68) (envelope-from ) id 1X8RRl-00028t-Bg for freebsd-fs@freebsd.org; Sat, 19 Jul 2014 12:00:53 +0200 Date: Sat, 19 Jul 2014 12:00:55 +0200 From: Fabian Keil To: FreeBSD-fs Subject: zfs receive abort(): internal error: File too large Message-ID: <132a24ce.33d69bdc@fabiankeil.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; boundary="Sig_/O9WPSDW0WHp2oOZEwz6yLQd"; protocol="application/pgp-signature" X-Df-Sender: Nzc1MDY3 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jul 2014 10:01:03 -0000 --Sig_/O9WPSDW0WHp2oOZEwz6yLQd Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Receiving a certain differential snapshot reproducible triggers an assertion in zfs(8) on my system (currently 11-CURRENT r268660, but I'll rebuild with HEAD over the weekend): | fk@r500 ~ $zogftw sync | [...] | receiving incremental stream of tank/usr/local@2014-06-23_22:33 into inte= nso1/backup/r500/tank/usr/local@2014-06-23_22:33 | in @ 29.4 MiB/s, out @ 0.0 KiB/s, 4674 MiB total, buffer 100% fullintern= al error: File too large | mbuffer: error: outputThread: error writing to at offset 0x1241c= 0000: Broken pipe | summary: 4674 MiByte in 3 min 45.9 sec - average of 20.7 MiB/s | mbuffer: warning: error during output to : Broken pipe | fk@r500 ~ $gdb-core zfs.core=20 | [...] | Reading symbols from /sbin/zfs...done. | [New process 101713] | [New Thread 803006400 (LWP 101713)] | Core was generated by `zfs'. | Program terminated with signal 6, Aborted. | #0 kill () at kill.S:3 | 3 RSYSCALL(kill) | (gdb) where | #0 kill () at kill.S:3 | #1 0x00000008019fa016 in __raise (s=3D6) at /usr/src/lib/libc/gen/raise.= c:45 | #2 0x00000008019f8849 in abort () at /usr/src/lib/libc/stdlib/abort.c:65 | #3 0x000000080168ba92 in zfs_verror (hdl=3D, error=3D, fmt=3D, ap=3D, hdl=3D,=20 | error=3D, fmt=3D, ap=3D) | at /usr/src/cddl/lib/libzfs/../../../cddl/contrib/opensolaris/lib/lib= zfs/common/libzfs_util.c:277 | #4 zfs_standard_error_fmt (hdl=3D, error=3D, fmt=3D) | at /usr/src/cddl/lib/libzfs/../../../cddl/contrib/opensolaris/lib/lib= zfs/common/libzfs_util.c:403 | #5 0x000000080168b485 in zfs_standard_error (hdl=3D0x3022, warning: Unma= pped DWARF Register #-2 encountered. | error=3D,=20 | msg=3D0x8019fa10a "\017\202\214\031") at /usr/src/cddl/li= b/libzfs/../../../cddl/contrib/opensolaris/lib/libzfs/common/libzfs_util.c:= 341 | #6 0x0000000801681b9a in zfs_receive_one (hdl=3D, infd=3D= , tosnap=3D, flags=3D, drr=3D0= x0,=20 | drr_noswap=3D0x0, sendfs=3D, stream_nv=3D, stream_avl=3D, cleanup_fd=3D,=20 | action_handlep=3D, top_zfs=3D) | at /usr/src/cddl/lib/libzfs/../../../cddl/contrib/opensolaris/lib/lib= zfs/common/libzfs_sendrecv.c:3109 | #7 zfs_receive_impl (hdl=3D0x80301a200, tosnap=3D, flags= =3D0x7fffffffde90, infd=3D, sendfs=3D,=20 | stream_nv=3D, stream_avl=3D, top_zfs=3D= , cleanup_fd=3D, action_handlep=3D) | at /usr/src/cddl/lib/libzfs/../../../cddl/contrib/opensolaris/lib/lib= zfs/common/libzfs_sendrecv.c:3275 | #8 0x0000000801681395 in zfs_receive_package (destname=3D= , flags=3D, drr=3D0x0, zc=3D0x3001777c1, top_zfs=3D,=20 | cleanup_fd=3D, action_handlep=3D, hdl= =3D, fd=3D) | at /usr/src/cddl/lib/libzfs/../../../cddl/contrib/opensolaris/lib/lib= zfs/common/libzfs_sendrecv.c:2501 | #9 zfs_receive_impl (hdl=3D0x80301a200, tosnap=3D0x7fffffffeae5 "intenso= 1/backup/r500/tank/usr/local", flags=3D0x7fffffffde90, infd=3D0,=20 | sendfs=3D, stream_nv=3D, stream_avl=3D<= optimized out>, top_zfs=3D, cleanup_fd=3D,=20 | action_handlep=3D) at /usr/src/cddl/lib/libzfs/../../.= ./cddl/contrib/opensolaris/lib/libzfs/common/libzfs_sendrecv.c:3281 | #10 0x000000080167fe71 in zfs_receive (hdl=3D0x80301a200, tosnap=3D0x7fff= ffffeae5 "intenso1/backup/r500/tank/usr/local", flags=3D0x7fffffffde90, inf= d=3D0,=20 | stream_avl=3D0x0) at /usr/src/cddl/lib/libzfs/../../../cddl/contrib/o= pensolaris/lib/libzfs/common/libzfs_sendrecv.c:3304 | #11 0x00000000004099a1 in zfs_do_receive (argc=3D, argv=3D= 0x7fffffffe7a0) | at /usr/src/cddl/sbin/zfs/../../../cddl/contrib/opensolaris/cmd/zfs/z= fs_main.c:3950 | #12 0x000000000040572f in main (argc=3D, argv=3D) | at /usr/src/cddl/sbin/zfs/../../../cddl/contrib/opensolaris/cmd/zfs/z= fs_main.c:6953 | (gdb) f 3 | #3 0x000000080168ba92 in zfs_verror (hdl=3D, error=3D, fmt=3D, ap=3D, hdl=3D,=20 | error=3D, fmt=3D, ap=3D) | at /usr/src/cddl/lib/libzfs/../../../cddl/contrib/opensolaris/lib/lib= zfs/common/libzfs_util.c:277 | 277 abort(); | (gdb) l | 272=09 | 273 if (hdl->libzfs_printerr) { | 274 if (error =3D=3D EZFS_UNKNOWN) { | 275 (void) fprintf(stderr, dgettext(TEXT_DOMAIN, "internal " | 276 "error: %s\n"), libzfs_error_description(hdl)); | 277 abort(); | 278 } | 279=09 | 280 (void) fprintf(stderr, "%s: %s\n", hdl->libzfs_action, | 281 libzfs_error_description(hdl)); What zogftw does is: | 0 4640 3703 sudo -c - zfs send -I @2013-12-22_11:27 tank/usr/local= @2014-06-23_22:33 | 1001 4641 3703 mbuffer -m 100M | 0 4642 3703 sudo -c - zfs receive -v -u -F intenso1/backup/r500/ta= nk/usr/local | 0 4643 4640 zfs send -I @2013-12-22_11:27 tank/usr/local@2014-06-2= 3_22:33 | 0 4644 4642 zfs receive -v -u -F intenso1/backup/r500/tank/usr/loc= al It happens when receiving to other pools as well, mbuffer always gets to 67.1 MiB. zstreamdump doesn't seem to consider the stream objectionable: | fk@r500 ~ $sudo zfs send -I @2013-12-22_11:27 tank/usr/local@2014-06-23_2= 2:33 | zstreamdump | BEGIN record | hdrtype =3D 2 | features =3D 4 | magic =3D 2f5bacbac | creation_time =3D 0 | type =3D 0 | flags =3D 0x0 | toguid =3D 0 | fromguid =3D 0 | toname =3D tank/usr/local@2014-06-23_22:33 | END checksum =3D 2fb1374c0/c59b43b4b2/1a3f6af4955b/262293c445759 | BEGIN record | hdrtype =3D 1 | features =3D 4 | magic =3D 2f5bacbac | creation_time =3D 53a88f1e | type =3D 2 | flags =3D 0x0 | toguid =3D f2e0cfa41ab80745 | fromguid =3D f1f10991be60bd7d | toname =3D tank/usr/local@2014-06-23_22:33 | END checksum =3D 1bdac72d140ea8d1/9e56d965e806168a/ed215fadf1a47e7f/a99aa= 9cca7be2746 | END checksum =3D 0/0/0/0 | SUMMARY: | Total DRR_BEGIN records =3D 2 | Total DRR_END records =3D 3 | Total DRR_OBJECT records =3D 98837 | Total DRR_FREEOBJECTS records =3D 796 | Total DRR_WRITE records =3D 140583 | Total DRR_WRITE_BYREF records =3D 0 | Total DRR_WRITE_EMBEDDED records =3D 0 | Total DRR_FREE records =3D 103847 | Total DRR_SPILL records =3D 0 | Total records =3D 344068 | Total write size =3D 7722947584 (0x1cc52d400) Receiving tank/usr/local@2014-06-23_22:33 non-incrementally works as expected, receiving additional incremental snapshots works too. The zpools: | fk@r500 ~ $zpool status | pool: intenso1 | state: ONLINE | scan: scrub repaired 0 in 0h8m with 0 errors on Fri Jul 18 17:42:59 2014 | config: |=20 | NAME STATE READ WRITE CKSUM | intenso1 ONLINE 0 0 0 | label/intenso1.eli ONLINE 0 0 0 |=20 | errors: No known data errors |=20 | pool: tank | state: ONLINE | status: Some supported features are not enabled on the pool. The pool can | still be used, but some features are unavailable. | action: Enable all features using 'zpool upgrade'. Once this is done, | the pool may no longer be accessible by software that does not support | the features. See zpool-features(7) for details. | scan: scrub repaired 0 in 4h11m with 0 errors on Sat Mar 22 18:25:01 20= 14 | config: |=20 | NAME STATE READ WRITE CKSUM | tank ONLINE 0 0 0 | ada0s1d.eli ONLINE 0 0 0 |=20 | errors: No known data errors | fk@r500 ~ $zpool upgrade | This system supports ZFS pool feature flags. |=20 | All pools are formatted using feature flags. |=20 |=20 | Some supported features are not enabled on the following pools. Once a | feature is enabled the pool may become incompatible with software | that does not support the feature. See zpool-features(7) for details. |=20 | POOL FEATURE | --------------- | tank | embedded_data =20 | fk@r500 ~ $zpool list | NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT | intenso1 2.72T 13.3G 2.71T 0% 1.00x ONLINE - | tank 228G 193G 35.2G 84% 1.00x ONLINE - According to my logs, I first saw the problem with r267067 (using a different receiving pool), but as the dataset hasn't been zfs-sent for about 6 months, the problem could be quite a bit older. Fabian --Sig_/O9WPSDW0WHp2oOZEwz6yLQd Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlPKQdYACgkQBYqIVf93VJ1LxACgmtDdAjJ4jw6G0ISUeyD5Xadx rl0AnRn3lE+r0h2K+8HF6uQlnb995fLM =Uie7 -----END PGP SIGNATURE----- --Sig_/O9WPSDW0WHp2oOZEwz6yLQd-- From owner-freebsd-fs@FreeBSD.ORG Sat Jul 19 16:41:18 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0AE3662C for ; Sat, 19 Jul 2014 16:41:18 +0000 (UTC) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C3A032A96 for ; Sat, 19 Jul 2014 16:41:17 +0000 (UTC) Received: from vega.codepro.be (unknown [172.16.1.3]) by venus.codepro.be (Postfix) with ESMTP id 7E73EAD04 for ; Sat, 19 Jul 2014 18:41:13 +0200 (CEST) Received: by vega.codepro.be (Postfix, from userid 1001) id 75E1D19658; Sat, 19 Jul 2014 18:41:13 +0200 (CEST) Date: Sat, 19 Jul 2014 18:41:13 +0200 From: Kristof Provost To: freebsd-fs@freebsd.org Subject: Re: ZFS panic on zvol resize Message-ID: <20140719164113.GA2406@vega.codepro.be> References: <20140704194750.GU75721@vega.codepro.be> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20140704194750.GU75721@vega.codepro.be> X-PGP-Fingerprint: E114 D9EA 909E D469 8F57 17A5 7D15 91C6 9EFA F286 X-Checked-By-NSA: Probably User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jul 2014 16:41:18 -0000 I've poked at this a bit more, and I think I understand the problem now. zvol_set_volsize() takes a hold on the file system with dmu_objset_hold() and then verifies that it's not marked as read-only. It does this through dsl_prop_get_integer() which also tries to take a hold on the file system with dmu_objset_hold(). That triggers the assert in dsl_pool_hold(). I don't think it'd be safe to move the read only check so it's done before taking the dmu_objset_hold(). I think it'd open us to races where the file system gets marked as read only after our check, but before the completion of the resize. A possible fix would be to use the unlocked variant of dsl_prop_get_integer() like this: Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c (revision 268871) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c (working copy) @@ -913,8 +913,8 @@ doi.doi_data_block_size)) != 0) goto out; - VERIFY(dsl_prop_get_integer(name, "readonly", &readonly, - NULL) == 0); + VERIFY(dsl_prop_get_ds(dmu_objset_ds(os), "readonly", + 8, 1, &readonly, NULL) == 0); if (readonly) { error = EROFS; goto out; The ZFS on Linux people ran into the same problem: https://github.com/zfsonlinux/zfs/issues/2039 With this patch I no longer see the original panic, but I get a shiny new one in its place: panic: solaris assert: txg_how != TXG_WAIT || !dsl_pool_config_held(tx->tx_pool), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_tx.c, line: 1279 cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe01212e94e0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe01212e9590 vpanic() at vpanic+0x126/frame 0xfffffe01212e95d0 panic() at panic+0x43/frame 0xfffffe01212e9630 assfail() at assfail+0x1d/frame 0xfffffe01212e9640 dmu_tx_assign() at dmu_tx_assign+0xae/frame 0xfffffe01212e96d0 zvol_set_volsize() at zvol_set_volsize+0x1cf/frame 0xfffffe01212e9760 zfs_prop_set_special() at zfs_prop_set_special+0x2e2/frame 0xfffffe01212e97f0 zfs_set_prop_nvlist() at zfs_set_prop_nvlist+0x23f/frame 0xfffffe01212e9880 zfs_ioc_set_prop() at zfs_ioc_set_prop+0x106/frame 0xfffffe01212e98e0 zfsdev_ioctl() at zfsdev_ioctl+0x6ee/frame 0xfffffe01212e9990 devfs_ioctl_f() at devfs_ioctl_f+0xfb/frame 0xfffffe01212e99f0 kern_ioctl() at kern_ioctl+0x22b/frame 0xfffffe01212e9a50 sys_ioctl() at sys_ioctl+0x13c/frame 0xfffffe01212e9aa0 amd64_syscall() at amd64_syscall+0x25a/frame 0xfffffe01212e9bb0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe01212e9bb0 The dmu_tx_assign() function is unhappy about being called with TXG_WAIT while the dsl_pool_config is locked. Regards, Kristof From owner-freebsd-fs@FreeBSD.ORG Sat Jul 19 22:42:26 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EB515B2C for ; Sat, 19 Jul 2014 22:42:25 +0000 (UTC) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7C0692612 for ; Sat, 19 Jul 2014 22:42:24 +0000 (UTC) Received: from vega.codepro.be (unknown [172.16.1.3]) by venus.codepro.be (Postfix) with ESMTP id 715AF1211E for ; Sun, 20 Jul 2014 00:42:22 +0200 (CEST) Received: by vega.codepro.be (Postfix, from userid 1001) id 684E6198C1; Sun, 20 Jul 2014 00:42:22 +0200 (CEST) Date: Sun, 20 Jul 2014 00:42:22 +0200 From: Kristof Provost To: freebsd-fs@freebsd.org Subject: Re: ZFS panic on zvol resize Message-ID: <20140719224222.GB2406@vega.codepro.be> References: <20140704194750.GU75721@vega.codepro.be> <20140719164113.GA2406@vega.codepro.be> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20140719164113.GA2406@vega.codepro.be> X-PGP-Fingerprint: E114 D9EA 909E D469 8F57 17A5 7D15 91C6 9EFA F286 X-Checked-By-NSA: Probably User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jul 2014 22:42:26 -0000 On 2014-07-19 18:41:13 (+0200), Kristof Provost wrote: > With this patch I no longer see the original panic, but I get a shiny > new one in its place: > > panic: solaris assert: txg_how != TXG_WAIT || !dsl_pool_config_held(tx->tx_pool), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_tx.c, line: 1279 Both problems appear to be fixed in Illumos commit 3b2aab18808792cbd248a12f1edf139b89833c13 Essentially they've changed from using dmu_objset_hold() to dmu_objset_own(). See: - https://www.illumos.org/issues/3464 - https://github.com/illumos/illumos-gate/commit/3b2aab18808792cbd248a12f1edf139b89833c13 - ZoL: https://github.com/zfsonlinux/zfs/pull/2048 I included the zvol resize bits of the patch in my tree, and can now resize zvols without panicing the machine. Index: cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zvol.h =================================================================== --- cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zvol.h (revision 268881) +++ cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zvol.h (working copy) @@ -43,7 +43,7 @@ extern int zvol_create_minor(const char *); extern int zvol_remove_minor(const char *); extern void zvol_remove_minors(const char *); -extern int zvol_set_volsize(const char *, major_t, uint64_t); +extern int zvol_set_volsize(const char *, uint64_t); #ifdef sun extern int zvol_open(dev_t *devp, int flag, int otyp, cred_t *cr); Index: cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c =================================================================== --- cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c (revision 268881) +++ cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c (working copy) @@ -2482,8 +2482,7 @@ err = dsl_dataset_set_refreservation(dsname, source, intval); break; case ZFS_PROP_VOLSIZE: - err = zvol_set_volsize(dsname, ddi_driver_major(zfs_dip), - intval); + err = zvol_set_volsize(dsname, intval); break; case ZFS_PROP_VERSION: { Index: cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c =================================================================== --- cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c (revision 268881) +++ cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c (working copy) @@ -203,11 +203,12 @@ static void zvol_geom_worker(void *arg); static void -zvol_size_changed(zvol_state_t *zv) +zvol_size_changed(zvol_state_t *zv, uint64_t volsize) { #ifdef sun dev_t dev = makedevice(maj, min); + zv->zv_volsize = volsize; VERIFY(ddi_prop_update_int64(dev, zfs_dip, "Size", volsize) == DDI_SUCCESS); VERIFY(ddi_prop_update_int64(dev, zfs_dip, @@ -765,9 +766,9 @@ dmu_objset_disown(os, zvol_tag); return (error); } - zv->zv_volsize = volsize; + + zvol_size_changed(zv, volsize); zv->zv_zilog = zil_open(os, zvol_get_data); - zvol_size_changed(zv); VERIFY(dsl_prop_get_integer(zv->zv_name, "readonly", &readonly, NULL) == 0); @@ -891,65 +892,42 @@ PICKUP_GIANT(); } -int -zvol_set_volsize(const char *name, major_t maj, uint64_t volsize) +static int +zvol_update_live_volsize(zvol_state_t *zv, uint64_t volsize) { - zvol_state_t *zv = NULL; - objset_t *os; - int error; - dmu_object_info_t doi; uint64_t old_volsize = 0ULL; - uint64_t readonly; + int error = 0; - mutex_enter(&spa_namespace_lock); - zv = zvol_minor_lookup(name); - if ((error = dmu_objset_hold(name, FTAG, &os)) != 0) { - mutex_exit(&spa_namespace_lock); - return (error); - } + ASSERT(MUTEX_HELD(&spa_namespace_lock)); - if ((error = dmu_object_info(os, ZVOL_OBJ, &doi)) != 0 || - (error = zvol_check_volsize(volsize, - doi.doi_data_block_size)) != 0) - goto out; - - VERIFY(dsl_prop_get_integer(name, "readonly", &readonly, - NULL) == 0); - if (readonly) { - error = EROFS; - goto out; - } - - error = zvol_update_volsize(os, volsize); /* * Reinitialize the dump area to the new size. If we * failed to resize the dump area then restore it back to - * its original size. + * its original size. We must set the new volsize prior + * to calling dumpvp_resize() to ensure that the devices' + * size(9P) is not visible by the dump subsystem. */ - if (zv && error == 0) { + old_volsize = zv->zv_volsize; + zvol_size_changed(zv, volsize); #ifdef ZVOL_DUMP - if (zv->zv_flags & ZVOL_DUMPIFIED) { - old_volsize = zv->zv_volsize; - zv->zv_volsize = volsize; - if ((error = zvol_dumpify(zv)) != 0 || - (error = dumpvp_resize()) != 0) { - (void) zvol_update_volsize(os, old_volsize); - zv->zv_volsize = old_volsize; - error = zvol_dumpify(zv); - } + if (zv->zv_flags & ZVOL_DUMPIFIED) { + if ((error = zvol_dumpify(zv)) != 0 || + (error = dumpvp_resize()) != 0) { + int dumpify_error; + + (void) zvol_update_volsize(zv->zv_objset, old_volsize); + zvol_size_changed(zv, old_volsize); + dumpify_error = zvol_dumpify(zv); + error = dumpify_error ? dumpify_error : error; } -#endif /* ZVOL_DUMP */ - if (error == 0) { - zv->zv_volsize = volsize; - zvol_size_changed(zv); - } } +#endif /* ZVOL_DUMP */ #ifdef sun /* * Generate a LUN expansion event. */ - if (zv && error == 0) { + if (error == 0) { sysevent_id_t eid; nvlist_t *attr; char *physpath = kmem_zalloc(MAXPATHLEN, KM_SLEEP); @@ -967,12 +945,57 @@ kmem_free(physpath, MAXPATHLEN); } #endif /* sun */ + return (error); +} +int +zvol_set_volsize(const char *name, uint64_t volsize) +{ + zvol_state_t *zv = NULL; + objset_t *os; + int error; + dmu_object_info_t doi; + uint64_t readonly; + boolean_t owned = B_FALSE; + + error = dsl_prop_get_integer(name, + zfs_prop_to_name(ZFS_PROP_READONLY), &readonly, NULL); + if (error != 0) + return (error); + if (readonly) + return (SET_ERROR(EROFS)); + + mutex_enter(&spa_namespace_lock); + zv = zvol_minor_lookup(name); + + if (zv == NULL || zv->zv_objset == NULL) { + if ((error = dmu_objset_own(name, DMU_OST_ZVOL, B_FALSE, + FTAG, &os)) != 0) { + mutex_exit(&spa_namespace_lock); + return (error); + } + owned = B_TRUE; + if (zv != NULL) + zv->zv_objset = os; + } else { + os = zv->zv_objset; + } + + if ((error = dmu_object_info(os, ZVOL_OBJ, &doi)) != 0 || + (error = zvol_check_volsize(volsize, doi.doi_data_block_size)) != 0) + goto out; + + error = zvol_update_volsize(os, volsize); + + if (error == 0 && zv != NULL) + error = zvol_update_live_volsize(zv, volsize); out: - dmu_objset_rele(os, FTAG); - + if (owned) { + dmu_objset_disown(os, FTAG); + if (zv != NULL) + zv->zv_objset = NULL; + } mutex_exit(&spa_namespace_lock); - return (error); } Regards, Kristof