Date: Sun, 23 Jun 1996 22:23:31 -0400 (EDT) From: Brian Tao <taob@io.org> To: FREEBSD-HACKERS-L <freebsd-hackers@FreeBSD.org> Cc: FREEBSD-ISP-L <freebsd-isp@FreeBSD.org> Subject: Increasing openinfo.cachesize in pwd_mkdb (was Re: Incremental [s]pwd.db updates?) Message-ID: <Pine.NEB.3.92.960623213725.3528B-100000@zap.io.org> In-Reply-To: <199606152001.WAA24925@gvr.win.tue.nl>
next in thread | previous in thread | raw e-mail | index | archive | help
I did some simple benchmarking with a modified pwd_mkdb that
allows me to specify the size of the cache. It defaults to 2MB, which
is less than optimal if you have more than a few thousand users in
/etc/master.passwd (like many ISP's).
Server is a lightly loaded P133 with 128MB doing primary DNS and a
bit of SMTP/POP3 service. There is typically 80 to 90MB free after
the OS, named and various user processes have taken their chunk.
The first test is with an actual 14241-line passwd file generated
from one of our customer databases (we have a system that creates
master.passwd and RADIUS files from a billing database).
# time /root/bin/pwd_mkdb -p -d . master.passwd
9.024u 15.201s 4:26.16 9.0% 10+2799k 0+17624io 0pf+0w
# time /root/bin/pwd_mkdb -c 3072 -p -d . master.passwd
7.724u 8.949s 2:36.64 10.6% 10+4022k 3+10081io 0pf+0w
# time /root/bin/pwd_mkdb -c 4096 -p -d . master.passwd
6.957u 3.041s 0:58.69 17.0% 10+4767k 3+3565io 0pf+0w
# time /root/bin/pwd_mkdb -c 8192 -p -d . master.passwd
7.110u 1.645s 0:30.44 28.7% 10+7600k 3+1941io 0pf+0w
# ls -l
total 14600
-rw------- 1 root wheel 1192261 Jun 23 21:50 master.passwd
-rw-r--r-- 1 root wheel 950366 Jun 23 22:04 passwd
-rw-r--r-- 1 root wheel 9220096 Jun 23 22:04 pwd.db
-rw------- 1 root wheel 9666560 Jun 23 22:05 spwd.db
The second test is with an artificially generated 150,000-line
passwd file, and allocating 64MB of core to pwd_mkdb:
# /root/bin/pwd_mkdb -c 65536 -p -d . big-pass
83.962u 16.826s 4:39.83 36.0% 10+60692k 1753+22470io 0pf+0w
# ls -l
total 134576
-rw------- 1 root wheel 9300000 Jun 23 21:06 master.passwd
-rw-r--r-- 1 root wheel 6750000 Jun 23 22:12 passwd
-rw-r--r-- 1 root wheel 70475776 Jun 23 22:12 pwd.db
-rw------- 1 root wheel 73740288 Jun 23 22:14 spwd.db
With enough memory, I can now rehash a passwd file over 10 times
the size in roughly the same amount of time. The trick is to know how
much RAM you can dedicate to one process, without causing it to hit
swap.
The diffs to support the -c switch are quite simple, if anyone is
interested in them (maybe even committing them?). I think it would be
a perfect compliment to gvr's enhancements to passwd/chpass/chfn/chsh.
I still need to edit the man page and patch vipw to pass a cachesize
parameter to pwd_mkdb.
--
Brian Tao (BT300, taob@io.org, taob@ican.net)
Systems and Network Administrator, Internet Canada Corp.
"Though this be madness, yet there is method in't"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.NEB.3.92.960623213725.3528B-100000>