From owner-freebsd-hackers@freebsd.org Fri Jan 26 09:03:17 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2EC91ECB201 for ; Fri, 26 Jan 2018 09:03:17 +0000 (UTC) (envelope-from Hartmut.Brandt@dlr.de) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id BB3CD6A5E5 for ; Fri, 26 Jan 2018 09:03:16 +0000 (UTC) (envelope-from Hartmut.Brandt@dlr.de) Received: by mailman.ysv.freebsd.org (Postfix) id 75484ECB200; Fri, 26 Jan 2018 09:03:16 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5085EECB1FF for ; Fri, 26 Jan 2018 09:03:16 +0000 (UTC) (envelope-from Hartmut.Brandt@dlr.de) Received: from mailhost.dlr.de (mailhost.dlr.de [129.247.252.32]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mailhost.dlr.de", Issuer "DLR CA - G02" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C6BAA6A5E4 for ; Fri, 26 Jan 2018 09:03:15 +0000 (UTC) (envelope-from Hartmut.Brandt@dlr.de) Received: from DLREXHUB01.intra.dlr.de (172.21.152.130) by mailhost.dlr.de (172.21.163.100) with Microsoft SMTP Server (TLS) id 14.3.361.1; Fri, 26 Jan 2018 10:03:01 +0100 Received: from KNOP-BEAGLE.kn.op.dlr.de (129.247.178.136) by smtp.dlr.de (172.21.152.151) with Microsoft SMTP Server (TLS) id 14.3.361.1; Fri, 26 Jan 2018 10:03:06 +0100 Date: Fri, 26 Jan 2018 10:05:14 +0100 From: Hartmut Brandt X-X-Sender: brandt_h@KNOP-BEAGLE.kn.op.dlr.de To: Konstantin Belousov CC: Subject: Re: allocating large piece of contiguous kernel memory In-Reply-To: <20180125172116.GS55707@kib.kiev.ua> Message-ID: References: <20180125172116.GS55707@kib.kiev.ua> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" X-TM-AS-Product-Ver: SMEX-11.0.0.4283-8.100.1062-23620.005 X-TM-AS-Result: No--5.400600-5.000000-31 X-TM-AS-MatchedID: 150567-700075-708712-708060-705262-708218-861157-705388-7 09584-703523-707760-702807-186027-121202-188199-702688-106420-105630-704421 -700918-704318-708497-701428-701465-702098-701461-701236-703788-148004-1480 50-29997-42000-42003-63 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jan 2018 09:03:17 -0000 Hi, On Thu, 25 Jan 2018, Konstantin Belousov wrote: KB>On Thu, Jan 25, 2018 at 05:52:57PM +0100, Hartmut Brandt wrote: KB>> Hi all, KB>> KB>> for a device driver communicating with an FPGA I would like to allocate KB>> an 8GByte piece or two 4GByte pieces of contiguous memory in the device KB>> driver. The machine is amd64 and has 256GByte. KB>> KB>> The maximum I can allocate is 512MByte. If I try more (in one piece or in KB>> two pieces) contigfree() crashes in pmap_resident_count_dec() with an KB>> "pmap resident count underflow". KB>> KB>> Is this something which is not supposed to work? Or is there something KB>> tunable or is this a bug? KB> KB>I suspect a bug. Print out the value of KB>kernel_pmap->pm_stats.resident_count before and after contigmalloc. Also KB>interesting are the values of the resident count and decrement at the KB>panic time (but they are already printed by the panic, you did not shown KB>them). This is the panic: knot_attach: attaching kernel_pmap->pm_stats.resident_count=57322 cmem=0xfffffe0101000000 kernel_pmap->pm_stats.resident_count=57712 kernel_pmap->pm_stats.resident_count=57712 panic: pmap 0xffffffff80e40188 resident count underflow 368 512 cpuid = 3 time = 1516956458 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0100d0d1c0 vpanic() at vpanic+0x19c/frame 0xfffffe0100d0d240 kassert_panic() at kassert_panic+0x126/frame 0xfffffe0100d0d2b0 pmap_remove_pde() at pmap_remove_pde+0x645/frame 0xfffffe0100d0d340 pmap_remove() at pmap_remove+0x484/frame 0xfffffe0100d0d3c0 kmem_unback() at kmem_unback+0x3a/frame 0xfffffe0100d0d400 kmem_free() at kmem_free+0x3d/frame 0xfffffe0100d0d430 contigfree() at contigfree+0x27/frame 0xfffffe0100d0d460 knot_attach() at knot_attach+0xec/frame 0xfffffe0100d0d4c0 device_attach() at device_attach+0x3f7/frame 0xfffffe0100d0d510 pci_driver_added() at pci_driver_added+0xe9/frame 0xfffffe0100d0d550 devclass_driver_added() at devclass_driver_added+0x7d/frame 0xfffffe0100d0d590 devclass_add_driver() at devclass_add_driver+0x144/frame 0xfffffe0100d0d5d0 module_register_init() at module_register_init+0xc0/frame 0xfffffe0100d0d600 linker_load_module() at linker_load_module+0xb78/frame 0xfffffe0100d0d910 kern_kldload() at kern_kldload+0xa9/frame 0xfffffe0100d0d950 sys_kldload() at sys_kldload+0x5b/frame 0xfffffe0100d0d980 amd64_syscall() at amd64_syscall+0x271/frame 0xfffffe0100d0dab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0100d0dab0 --- syscall (304, FreeBSD ELF64, sys_kldload), rip = 0x80087228a, rsp = 0x7fffffffd508, rbp = 0x7fffffffda80 --- KDB: enter: panic [ thread pid 1150 tid 100146 ] Stopped at kdb_enter+0x3b: movq $0,kdb_why db> The first printf() is just before contigmalloc(), then it prints the returned pointer. The next to prints() are just before the contigfree(). Doesn't the change of the count by 390 look strange when allocating 8GByte? Regards, harti