From owner-freebsd-bugs@FreeBSD.ORG Sat Oct 18 20:45:20 2014 Return-Path: Delivered-To: freebsd-bugs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 24D2B6F6 for ; Sat, 18 Oct 2014 20:45:20 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0657E9E4 for ; Sat, 18 Oct 2014 20:45:20 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s9IKjJtk099606 for ; Sat, 18 Oct 2014 20:45:19 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-bugs@FreeBSD.org Subject: [Bug 194455] New: [PATCH] Preserve lowmem for devices on large memory machines Date: Sat, 18 Oct 2014 20:45:20 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: conrad.meyer@isilon.com X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter attachments.created Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Oct 2014 20:45:20 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194455 Bug ID: 194455 Summary: [PATCH] Preserve lowmem for devices on large memory machines Product: Base System Version: 11.0-CURRENT Hardware: Any OS: Any Status: Needs Triage Severity: Affects Only Me Priority: --- Component: kern Assignee: freebsd-bugs@FreeBSD.org Reporter: conrad.meyer@isilon.com Created attachment 148440 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=148440&action=edit ( applies with -p1 on CURRENT ~r273263 ) Log: ==================================8<================================== Add an intermediate VM_FREELIST "DMA32" between DEFAULT and ISADMA on AMD64. Physical allocations try freelists ascending, so the effect is to prefer DEFAULT, then DMA32, then ISADMA. This leaves low memory available for things that need it (DMA) on large-memory systems which statically allocate a lot (e.g. bufs). Sponsored by: EMC / Isilon storage division ==================================8<================================== Historical note/context: So, we have 96-256 GB systems, and scale nbufs up with memory. On these systems we tune for much more than 4GB in bufs. It turns out the 'buf' static array is allocated from physmem fairly early (along with some other things that scale with memory), and without this change, devices starved for 32-bit DMA space. Internally, we had just bumped the ISADMA region from 16MB (24-bit) to 4GB. I think the attached patch with a seperate 32-bit DMA region is slightly cleaner (and avoids other devices starving out ISA BUS DMA devices, if those still exist). But, either approach fixes the problem for us. So, if people object to adding another freelist, a compromise might be renaming the ISADMA freelist to "DMA" and bumping the region up to 32 bits. Testing on an AMD64 VM: # sysctl vm.phys_free vm.phys_free: DOMAIN 0: FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 | POOL 2 -- -- -- -- -- -- -- -- 12 ( 16384K) | 257 | 0 | 0 11 ( 8192K) | 0 | 0 | 0 10 ( 4096K) | 2 | 1 | 0 9 ( 2048K) | 2 | 0 | 0 8 ( 1024K) | 1 | 0 | 0 7 ( 512K) | 1 | 0 | 0 6 ( 256K) | 1 | 0 | 0 5 ( 128K) | 0 | 2 | 0 4 ( 64K) | 0 | 1 | 0 3 ( 32K) | 4 | 1 | 0 2 ( 16K) | 3 | 3 | 0 1 ( 8K) | 0 | 9 | 0 0 ( 4K) | 1 | 1 | 0 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 | POOL 2 -- -- -- -- -- -- -- -- 12 ( 16384K) | 221 | 0 | 0 11 ( 8192K) | 1 | 0 | 0 10 ( 4096K) | 0 | 0 | 0 9 ( 2048K) | 1 | 0 | 0 8 ( 1024K) | 0 | 0 | 0 7 ( 512K) | 1 | 0 | 0 6 ( 256K) | 1 | 0 | 0 5 ( 128K) | 1 | 0 | 0 4 ( 64K) | 1 | 0 | 0 3 ( 32K) | 0 | 0 | 0 2 ( 16K) | 0 | 0 | 0 1 ( 8K) | 0 | 0 | 0 0 ( 4K) | 1 | 0 | 0 FREE LIST 2: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 | POOL 2 -- -- -- -- -- -- -- -- 12 ( 16384K) | 0 | 0 | 0 11 ( 8192K) | 0 | 0 | 0 10 ( 4096K) | 0 | 0 | 0 9 ( 2048K) | 0 | 0 | 0 8 ( 1024K) | 1 | 0 | 0 7 ( 512K) | 0 | 0 | 0 6 ( 256K) | 1 | 0 | 0 5 ( 128K) | 1 | 0 | 0 4 ( 64K) | 2 | 0 | 0 3 ( 32K) | 2 | 0 | 0 2 ( 16K) | 1 | 0 | 0 1 ( 8K) | 0 | 0 | 0 0 ( 4K) | 1 | 0 | 0 -- You are receiving this mail because: You are the assignee for the bug.