From owner-freebsd-net@FreeBSD.ORG Fri Nov 21 11:02:02 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2C26DFD3; Fri, 21 Nov 2014 11:02:02 +0000 (UTC) Received: from mail.fer.hr (mail.fer.hr [161.53.72.233]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.fer.hr", Issuer "TERENA SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 857CEE58; Fri, 21 Nov 2014 11:02:00 +0000 (UTC) Received: from x23 (161.53.63.210) by MAIL.fer.hr (161.53.72.233) with Microsoft SMTP Server (TLS) id 14.2.342.3; Fri, 21 Nov 2014 12:01:59 +0100 Date: Fri, 21 Nov 2014 12:02:01 +0100 From: Marko Zec To: "Bjoern A. Zeeb" Subject: Re: VIMAGE UDP memory leak fix Message-ID: <20141121120201.6c77ea5b@x23> In-Reply-To: <9300CB5F-6140-4C49-B026-EB69B0E8B37E@FreeBSD.org> References: <20141121002937.4f82daea@x23> <9300CB5F-6140-4C49-B026-EB69B0E8B37E@FreeBSD.org> X-Mailer: Claws Mail 3.9.2 (GTK+ 2.24.19; amd64-portbld-freebsd9.1) MIME-Version: 1.0 Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 8bit X-Originating-IP: [161.53.63.210] Cc: Craig Rodrigues , FreeBSD Net , "Robert N. M. Watson" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Nov 2014 11:02:02 -0000 On Fri, 21 Nov 2014 10:37:25 +0000 "Bjoern A. Zeeb" wrote: > > On 21 Nov 2014, at 08:25 , Robert N. M. Watson > wrote: > > > > > On 20 Nov 2014, at 23:29, Marko Zec wrote: > > > >>> Can folks take a look at this? > >>> > >>> https://reviews.freebsd.org/D1201 > >> > >> All UMA zones used in the network stack have been marked as > >> UMA_ZONE_NOFREE for ages, probably for a reason, so perhaps it > >> might not hurt to provide more insight why and how it suddenly > >> became safe to remove that flag? > > > > Historically, this was (if I recall) a property of the way data was > > exported for netstat, which depended on memory stability of various > > data types. We have worked quite hard to remove the causes of those > > sorts of dependencies by introducing stronger reference and memory > > ownership models throughout the stack -- in the case of inpcbs, for > > example, we introduced a true reference model during the > > multiprocessing scalability work (which, it should be pointed out, > > was enormously painful and took years to shake the bugs out of due > > to complexity/subtlety). It may be that it is now safe to remove > > UMA_ZONE_NOFREE for some of the types where it was previously > > required -- but it's definitely something you want to do > > intentionally and in the context of a careful analysis to convince > > yourself that all the causes have been addressed. A fair amount of > > stress testing in low-memory conditions wouldn't hurt either... > > I had convinced myself for UDP many years ago that it was ok to > remove it. People have touched the code however so it’s definitively > worth re-checking again. > > For TCP we clearly cannot do it (yet, and couldn’t back then). But > TCP was the only of the few cases I had left unfixed back then. > Can’t remember what the others were (was like 3 or 4 of them; could > also be some of the fixes were indeed committed back then. Now that we've found ourselves in this discussion, I'm really becoming curious why exactly do we need UMA_ZONE_NOFREE for network stack zones at all? Admittedly, I always thought that the primary purpose of UMA_ZONE_NOFREE was to prevent uma_reclaim() from paging out _used_ zone pages, but reviewing the uma code reveals that this might not be the case, i.e. that NOFREE only prevents _unused_ pages to be freed by uma_reclaim(). Moreover, all uma_zalloc() calls as far as I can see are flagged as M_NOWAIT and are followed by checks for allocation failures, so that part seems to be covered. So, what's really the problem which UMA_ZONE_NOFREE flagging is supposed to solve these days? (you claim that we clearly need it for TCP - why)? Marko