Date: Tue, 11 Aug 1998 05:29:10 +0200 (CEST) From: Barry Bierbauch <fluffy@vszbr.cz> To: freebsd-current@FreeBSD.ORG Subject: update frequency with recent current snaps? Message-ID: <Pine.BSF.3.96.980811041041.1712C-100000@coredump.int.tele.dk>
next in thread | raw e-mail | index | archive | help
I've recently upgraded my suffering news swerver from a -current snapshot of last November this past week to 3.0-19980802-SNAP. One problem I suffered was that sometimes, after rebuilding the kernel and rebooting, /etc/rc.conf would somehow have the line router_enable="YES" # Set to YES to enable a routing daemon. rewritten to "NO", so I had to fix this several times. This didn't always happen, only enough to unnerve me, and it also didn't happen at last reboot, so I can't say anything more about it other than I don't plan any remote kernel upgrades any more. Also, a minor nit, lines in /var/log/messages are sometimes split in the middle of the line: Aug 5 14:02:53 marley /kernel: lnc1 Aug 5 14:02:53 marley /kernel: : <PCNet/PCI Ethernet adapter> rev 0x16 int a irq 10 on pci0.11.0 [...] Aug 5 14:02:54 marley /kernel: changing root Aug 5 14:02:54 marley /kernel: device to wd1s1a [...] Aug 5 15:28:06 marley /kernel-updated-xl0: ccd0-3: Concatenated disk d Aug 5 15:28:06 marley /kernel-updated-xl0: rivers But the big question is about the kern.update sysctl variable, which used to allow one to tune how often one would sync data to disk from the default of 30 seconds: # We only sync once an hour ... we don't believe in crashes :-) sysctl -w kern.update=3600 I presume this has disappeared with the softupdates code. The point of doing something as risky as this, is that when running INN as news server, one has a history database text file which is updated by appending to the end for each incoming article, plus one or two files which are updated at unpredictable points throughout the file for each of some 15 articles/second. The thing is that when one is handling a full feed, the text file runs to around a gigabyte in size, and the index/hash/pag files can be several hundred megabytes in size when one keeps two weeks worth of entries. What happens when the periodic syncs happen is that the system practically freezes as the single-threaded innd gets stuck in ufslk2 state for several seconds up to several minutes under 2.2-RELENG, as the random points in the index/hash/pag files need to be updated for all of the half-thousand or more new entries that have arrived in those 30 seconds, which then happens again 30 seconds after this lengthy process completes, and so on. Barring a major rewrite of the dbz database code to only update a much smaller database (as one has with a much smaller history file), experimenting has shown that by delaying the updates to the disk, the additional data that needs to be written does not significantly increase the time spent blocked, as one is already updating most of the files anyway, so one gains a significant amount of time by postponing all the updates to a much less frequent time. As far as I can see now, the innd process periodically gets blocked in the `inode' state with the latest code on the server that handles far less than a full feed, so if this state could be postponed until `later', like a few minutes later, I'd gain more productive time per hour. At least, that's been my theory. There does seem to be a difference with the latest code, although I have not yet enabled softupdates for the history database, but my swerver hasn't been stable enough yet to know if it is still a significant problem with the database files ten times larger than on this micro-server. Actual file sizes in question (no significant filesystem tuning has been done w/r/t blocksize) for the recently-updated server -- the .pag file will probably stabilize around 2/3 of this size Real Soon Now: -rw-rw-r-- 1 news news 263050014 Aug 11 04:56 history -rw-rw-r-- 1 news news 127 Aug 11 04:56 history.dir -rw-rw-r-- 1 news news 32374604 Aug 11 04:55 history.pag The swerver where the files were significantly large enough to merit the hourly updates: -rw-rw-r-- 1 news news 845530318 Aug 11 04:56 history -rw-rw-r-- 1 news news 120 Aug 11 04:56 history.dir -rw-rw-r-- 1 news news 100214082 Aug 11 03:59 history.hash -rw-rw-r-- 1 news news 66809856 Aug 11 03:59 history.index Barry Bouwsma, <ag786@yfn.ysu.edu> Mendel University of Forestry and Agriculture, Brno, CZ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.3.96.980811041041.1712C-100000>