From owner-svn-src-all@FreeBSD.ORG Sat May 23 09:14:31 2015 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 68FB53FD; Sat, 23 May 2015 09:14:31 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 56A6B16DA; Sat, 23 May 2015 09:14:31 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.9/8.14.9) with ESMTP id t4N9EVfo032549; Sat, 23 May 2015 09:14:31 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by svn.freebsd.org (8.14.9/8.14.9/Submit) id t4N9EUMX032545; Sat, 23 May 2015 09:14:30 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201505230914.t4N9EUMX032545@svn.freebsd.org> X-Authentication-Warning: svn.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Sat, 23 May 2015 09:14:30 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r283310 - in stable/10/sys: kern vm X-SVN-Group: stable-10 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 May 2015 09:14:31 -0000 Author: kib Date: Sat May 23 09:14:29 2015 New Revision: 283310 URL: https://svnweb.freebsd.org/changeset/base/283310 Log: MFC r282690: Call uma_reclaim() from the additional pagedaemon thread to reclaim kmem arena address space. Modified: stable/10/sys/kern/kern_malloc.c stable/10/sys/vm/uma.h stable/10/sys/vm/uma_core.c stable/10/sys/vm/vm_pageout.c Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/kern/kern_malloc.c ============================================================================== --- stable/10/sys/kern/kern_malloc.c Sat May 23 08:02:54 2015 (r283309) +++ stable/10/sys/kern/kern_malloc.c Sat May 23 09:14:29 2015 (r283310) @@ -667,13 +667,15 @@ reallocf(void *addr, unsigned long size, } /* - * Wake the page daemon when we exhaust KVA. It will call the lowmem handler - * and uma_reclaim() callbacks in a context that is safe. + * Wake the uma reclamation pagedaemon thread when we exhaust KVA. It + * will call the lowmem handler and uma_reclaim() callbacks in a + * context that is safe. */ static void kmem_reclaim(vmem_t *vm, int flags) { + uma_reclaim_wakeup(); pagedaemon_wakeup(); } Modified: stable/10/sys/vm/uma.h ============================================================================== --- stable/10/sys/vm/uma.h Sat May 23 08:02:54 2015 (r283309) +++ stable/10/sys/vm/uma.h Sat May 23 09:14:29 2015 (r283310) @@ -689,4 +689,7 @@ struct uma_percpu_stat { uint64_t _ups_reserved[5]; /* Reserved. */ }; +void uma_reclaim_wakeup(void); +void uma_reclaim_worker(void *); + #endif /* _VM_UMA_H_ */ Modified: stable/10/sys/vm/uma_core.c ============================================================================== --- stable/10/sys/vm/uma_core.c Sat May 23 08:02:54 2015 (r283309) +++ stable/10/sys/vm/uma_core.c Sat May 23 09:14:29 2015 (r283310) @@ -3192,16 +3192,17 @@ uma_find_refcnt(uma_zone_t zone, void *i } /* See uma.h */ -void -uma_reclaim(void) +static void +uma_reclaim_locked(bool kmem_danger) { + #ifdef UMA_DEBUG printf("UMA: vm asked us to release pages!\n"); #endif - sx_xlock(&uma_drain_lock); + sx_assert(&uma_drain_lock, SA_XLOCKED); bucket_enable(); zone_foreach(zone_drain); - if (vm_page_count_min()) { + if (vm_page_count_min() || kmem_danger) { cache_drain_safe(NULL); zone_foreach(zone_drain); } @@ -3213,9 +3214,42 @@ uma_reclaim(void) zone_drain(slabzone); zone_drain(slabrefzone); bucket_zone_drain(); +} + +void +uma_reclaim(void) +{ + + sx_xlock(&uma_drain_lock); + uma_reclaim_locked(false); sx_xunlock(&uma_drain_lock); } +static int uma_reclaim_needed; + +void +uma_reclaim_wakeup(void) +{ + + uma_reclaim_needed = 1; + wakeup(&uma_reclaim_needed); +} + +void +uma_reclaim_worker(void *arg __unused) +{ + + sx_xlock(&uma_drain_lock); + for (;;) { + sx_sleep(&uma_reclaim_needed, &uma_drain_lock, PVM, + "umarcl", 0); + if (uma_reclaim_needed) { + uma_reclaim_needed = 0; + uma_reclaim_locked(true); + } + } +} + /* See uma.h */ int uma_zone_exhausted(uma_zone_t zone) Modified: stable/10/sys/vm/vm_pageout.c ============================================================================== --- stable/10/sys/vm/vm_pageout.c Sat May 23 08:02:54 2015 (r283309) +++ stable/10/sys/vm/vm_pageout.c Sat May 23 09:14:29 2015 (r283310) @@ -1726,8 +1726,9 @@ vm_pageout_init(void) static void vm_pageout(void) { + int error; #if MAXMEMDOM > 1 - int error, i; + int i; #endif swap_pager_swap_init(); @@ -1741,6 +1742,10 @@ vm_pageout(void) } } #endif + error = kthread_add(uma_reclaim_worker, NULL, curproc, NULL, + 0, 0, "uma"); + if (error != 0) + panic("starting uma_reclaim helper, error %d\n", error); vm_pageout_worker((void *)(uintptr_t)0); }