From owner-freebsd-arch@FreeBSD.ORG Wed Dec 3 00:43:13 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5466160C for ; Wed, 3 Dec 2014 00:43:13 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2CE99DA6 for ; Wed, 3 Dec 2014 00:43:12 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id sB30hBmE095318 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 2 Dec 2014 16:43:12 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id sB30hBKO095317; Tue, 2 Dec 2014 16:43:11 -0800 (PST) (envelope-from jmg) Date: Tue, 2 Dec 2014 16:43:11 -0800 From: John-Mark Gurney To: Rui Paulo Subject: Re: Disparity between /etc/services and /var/db/services.db Message-ID: <20141203004311.GD99957@funkthat.com> Mail-Followup-To: Rui Paulo , Garrett Cooper , "freebsd-arch@freebsd.org" , Benjamin Kaduk References: <6F3959BB-3B71-4515-B7BD-C1A640E8327A@gmail.com> <556364B6-EB7C-421F-B2FB-64A170611A56@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Tue, 02 Dec 2014 16:43:12 -0800 (PST) Cc: "freebsd-arch@freebsd.org" , Benjamin Kaduk , Garrett Cooper X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2014 00:43:13 -0000 Rui Paulo wrote this message on Tue, Dec 02, 2014 at 11:51 -0800: > On Dec 2, 2014, at 08:13, Garrett Cooper wrote: > > > > On Dec 1, 2014, at 11:28, Benjamin Kaduk wrote: > > > >> On Mon, 1 Dec 2014, Garrett Cooper wrote: > >> > >>> $ ls -l /scratch/2/etc/services /scratch/2/var/db/services.db > >>> -rw-r--r-- 1 ngie wheel 86802 Nov 27 02:23 /scratch/2/etc/services > >>> -rw-r--r-- 1 ngie wheel 2097920 Nov 27 02:23 /scratch/2/var/db/services.db > >> > >> One's a text file and the other a Berkeley DB file ... I wouldn't expect > >> them to be the same size. > > > > Shoot. I didn?t mean for this message to get sent out without a lot of context. For that I apologize... > > > > Basically what I was going to comment on was the fact that the .db file was so large, and by adjusting the number of entries I was able to reduce the size of the file by 4 (it?s bloated by a couple thousand): > > > > From usr.sbin/services_mkdb/services_mkdb.c: > > > > 70 HASHINFO hinfo = { > > 71 .bsize = 256, > > 72 .ffactor = 4, > > 73 .nelem = 32768, > > 74 .cachesize = 1024, > > 75 .hash = NULL, > > 76 .lorder = 0 > > 77 }; > > I doubt you'll find any history without contacting the original author (ume@). If I had to guess, I think this was a premature optimisation. The database just needs to contain a two level hash up: port number and service number. If you can prove that reducing nelem size doesn't cause a performance regression, then we could change it. 4MB is way too much on an embedded system. I'd say we don't even need the proof... Do you really look up service numbers in tight loops? As long as the size is resonable, it'll be fine.. If anything, maybe services_mkdb.c should warn, or even preprocess the file to get the number of entries before creating it.. Looks like all parameters need to be adjusted... bsize The bsize element defines the hash table bucket size, and is, by default, 4096 bytes. It may be preferable to increase the page size for disk-resident tables and tables with large data items. nelem The nelem element is an estimate of the final size of the hash table. If not set or set too low, hash tables will expand grace- fully as keys are entered, although a slight performance degrada- tion may be noticed. The default value is 1. It looks like the smaller bsize is so that things aren't too large... It looks like the resulting hash size will be nelem / ffactor * bsize... Also, the count is inflated because the database stores two entries per services file plus an additional entry for each alias of a service... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."