Date: Wed, 20 Dec 2000 11:59:50 +0100 (CET) From: Reddy Crashalott <autoretributionmeowbot@free-N0rp.netscum.dk> To: Matt Dillon <dillon@earth.backplane.com> Cc: stable@freebsd.org, hackers@freebsd.org Subject: Re: New performance patch available for testing on stable Message-ID: <200012201059.eBKAxnt24063@crotchety.newsbastards.org> In-Reply-To: <200012152247.eBFMlg727583@earth.backplane.com> References: <200012152247.eBFMlg727583@earth.backplane.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[disclaimer: this is an invalid sender address; replies should be posted to the list or perhaps better not posted at all. the reply-to header address also will soon become invalid] [further disclaimer: I've crossposted this to -hackers as well, to continue the thread I was carrying on singlehandedly there, in the highly unlikely event that someone there who professed an interest is not already subscribed to -stable. I'm not expecting any response at all to this post, but perhaps if there is, it would be best dumped on -hackers rather than -stable. Sorry.] :: http://apollo.backplane.com/FreeBSD4/ :: This patch reworks the pageout daemon and the buf_daemon. It is based :: on my previous patch but hopefully has the kinks worked out. The patch :: is for -stable only, I will have a -current patch tonight. Thanks, I've been using it on -stable for several days now... :: The current patchset will pageout a little more then 4.2-RELEASE, but :: hopefully to the benefit of the system rather then the detriment. This I'm happy to report that -- if you recall, I had problems with getting one of the two history files to perform as I wanted -- I've figured out what I've been doing wrong, and it was pilot error, since I've had to make a good number of hacks to the recent INN k0deZ, but I had not taken the time to figure out what was intended by the virginal code, why my hacks were needed, and why things were done as they were. The short of it was that I discovered I was only doing madvise/mlock on part of the problem file, so part of it was the cause of the heavy disk activity. I'm going to take some time off to recover and hope to be able to think clearly and find out the reason for using (size_of) in one size calculation and not in another -- mmap() didn't and was fine. But that's neither here nor there. In any case, when I finally managed to accomplish what I had been trying to do all this time, it was immensely gratifying to see the history file read/write access times dropping to practically zero, thanks to mlock() in luserland. Witness the numbers here, where I start from a dead stop and chew on a waiting backlog: Dec 19 23:52:03 crotchety innd: dbz madvise NOSYNC 90000000 ok [time passes] Dec 19 23:52:16 crotchety innd: dbz mlock 90000000 ok Dec 19 23:52:16 crotchety innd: dbz openhashtable /news/db/history.hash Dec 19 23:52:16 crotchety innd: dbz madvise WILLNEED 135000000 ok Dec 19 23:52:16 crotchety innd: dbz madvise RANDOM 135000000 ok Dec 19 23:52:16 crotchety innd: dbz madvise NOSYNC 135000000 ok [more time passes] Dec 19 23:52:40 crotchety innd: dbz mlock 135000000 ok [five mintes pass until the first timer stats are logged] Dec 19 23:57:45 crotchety innd: ME time 300004 idle 28838(286832) artwrite 111263(6811) artlink 0(0) hiswrite 2556(7883) hissync 3(15) ^^^^ sitesend 2683(20443) artctrl 56(242) artcncl 39(235) hishave 398(19812) ^^^ hisgrep 2(94) artclean 32393(7910) perl 23339(7742) overv 0(0) python 0(0) ncread 52681(324760) ncproc 194980(324760) [backlog gets chewed...] Dec 20 00:52:45 crotchety innd: ME time 300000 idle 131122(246534) artwrite 62458(3701) artlink 0(0) hiswrite 1603(4091) hissync 0(8) sitesend 1369(11103) artctrl 9(109) artcncl 9(109) hishave 147(10711) hisgrep 1(80) artclean 18996(4062) perl 13120(4062) overv 0(0) python 0(0) ncread 33002(248568) ncproc 116670(248568) Now, before I got this one started, I had had a few failures in the mlock() call to the 135MB history file, and as you see from the numbers below, performance was *awful* -- even with your patches that are the subject of this thread. The reason the numbers I had been seeing earlier were not quite this bad was that I was only mlock()ing part of the file. Aside from the mlock(), all the madvise() and MAP_NOSYNC tweaks were completed with success, so, well, huge difference. And the other file did successfully mlock()... Dec 19 23:38:53 crotchety innd: ME time 306143 idle 6279(92487) artwrite 66436(4114) artlink 0(0) hiswrite 71770(4727) hissync 1(10) ^^^^^ <spit> sitesend 2082(12343) artctrl 840(116) artcncl 839(114) hishave 62930(10547) ^^^^^ <spit> hisgrep 1(50) artclean 20776(4664) perl 14707(4663) overv 0(0) python 0(0) ncread 31888(119022) ncproc 257889(119022) You had mentioned that you would perhaps want to see the debug numbers from this machine and these two hash files *without* the hacks to mlock() them. Well, I dunno if I can stand the thought of letting the machine run for too long like this, but do you really want this data? And would you want to see the debug activity when the mlock() has succeeded for the happy near-zero history times? I can't say for certain, but it seems that with the current patchset, the history numbers are far worse than they would have been in a similar situation with the previous patchset or at the beginning of this month. On the other hand, most of the apparent unresponsiveness I've noticed has been directly traceable to the on-disk updates of the history files when the mlock() failed, whereas that which is the result of paging activity appears to be far less, to the point where I don't really notice it. These on-disk updates, as you know, block all related accesses, which is just about everything I'm doing (reading and so on) Now, in order for this to work with the near-zero history access times (yeah, it'll be far more impressive when I install this on a transit machine with 450 incoming connections, I promise), I've needed to make some non-trivial changes to both INN and the FreeBSD kernel, so in my Copious Free Time, I hope to figure out exactly what I want to do and merge in the hacks in a non-intrusive and portable way so that anyone could see this performance. I'll hold off on the FreeBSD end of this, though, since you know what you're doing and I don't, particularly if it's possible that coming VM improvements will have the effect that I see here of mlock() in userland, with or without auxiliary helper programs. (urg, sounds like all followups are best directed to -hackers) thanks, happy holidays with good cheer and all that, barry bouwsma, ex-teledanmark news admangler k0dewanker To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200012201059.eBKAxnt24063>