Date: Tue, 2 Dec 2014 16:43:11 -0800 From: John-Mark Gurney <jmg@funkthat.com> To: Rui Paulo <rpaulo@me.com> Cc: "freebsd-arch@freebsd.org" <arch@freebsd.org>, Benjamin Kaduk <kaduk@MIT.EDU>, Garrett Cooper <yaneurabeya@gmail.com> Subject: Re: Disparity between /etc/services and /var/db/services.db Message-ID: <20141203004311.GD99957@funkthat.com> In-Reply-To: <D4099F51-E574-47E6-95BB-D4F6298ADBCE@me.com> References: <6F3959BB-3B71-4515-B7BD-C1A640E8327A@gmail.com> <alpine.GSO.1.10.1412011428160.23489@multics.mit.edu> <556364B6-EB7C-421F-B2FB-64A170611A56@gmail.com> <D4099F51-E574-47E6-95BB-D4F6298ADBCE@me.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Rui Paulo wrote this message on Tue, Dec 02, 2014 at 11:51 -0800:
> On Dec 2, 2014, at 08:13, Garrett Cooper <yaneurabeya@gmail.com> wrote:
> >
> > On Dec 1, 2014, at 11:28, Benjamin Kaduk <kaduk@MIT.EDU> wrote:
> >
> >> On Mon, 1 Dec 2014, Garrett Cooper wrote:
> >>
> >>> $ ls -l /scratch/2/etc/services /scratch/2/var/db/services.db
> >>> -rw-r--r-- 1 ngie wheel 86802 Nov 27 02:23 /scratch/2/etc/services
> >>> -rw-r--r-- 1 ngie wheel 2097920 Nov 27 02:23 /scratch/2/var/db/services.db
> >>
> >> One's a text file and the other a Berkeley DB file ... I wouldn't expect
> >> them to be the same size.
> >
> > Shoot. I didn?t mean for this message to get sent out without a lot of context. For that I apologize...
> >
> > Basically what I was going to comment on was the fact that the .db file was so large, and by adjusting the number of entries I was able to reduce the size of the file by 4 (it?s bloated by a couple thousand):
> >
> > From usr.sbin/services_mkdb/services_mkdb.c:
> >
> > 70 HASHINFO hinfo = {
> > 71 .bsize = 256,
> > 72 .ffactor = 4,
> > 73 .nelem = 32768,
> > 74 .cachesize = 1024,
> > 75 .hash = NULL,
> > 76 .lorder = 0
> > 77 };
>
> I doubt you'll find any history without contacting the original author (ume@). If I had to guess, I think this was a premature optimisation. The database just needs to contain a two level hash up: port number and service number. If you can prove that reducing nelem size doesn't cause a performance regression, then we could change it. 4MB is way too much on an embedded system.
I'd say we don't even need the proof... Do you really look up service
numbers in tight loops? As long as the size is resonable, it'll be
fine..
If anything, maybe services_mkdb.c should warn, or even preprocess the
file to get the number of entries before creating it..
Looks like all parameters need to be adjusted...
bsize The bsize element defines the hash table bucket size, and is, by
default, 4096 bytes. It may be preferable to increase the page
size for disk-resident tables and tables with large data items.
nelem The nelem element is an estimate of the final size of the hash
table. If not set or set too low, hash tables will expand grace-
fully as keys are entered, although a slight performance degrada-
tion may be noticed. The default value is 1.
It looks like the smaller bsize is so that things aren't too large...
It looks like the resulting hash size will be nelem / ffactor * bsize...
Also, the count is inflated because the database stores two entries
per services file plus an additional entry for each alias of a service...
--
John-Mark Gurney Voice: +1 415 225 5579
"All that I will do, has been done, All that I have, has not."
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20141203004311.GD99957>