Date: Thu, 5 Oct 2000 15:39:57 +0400 From: Alex Povolotsky <tarkhil@over.ru> To: freebsd-fs@freebsd.org Subject: Specialised mailstorage system? Message-ID: <20001005153957.E65600@mail.over.ru>
next in thread | raw e-mail | index | archive | help
On Wed, Oct 04, 2000 at 02:30:53PM +0200, Dag-Erling Smorgrav wrote: > If you want something that runs purely in userland, try: Not sure. I'm rather looking for userland prototype with possible kernel release... > a) QMail's Maildir system (which Postfix also supports) Creating and removing messages is QUITE expensive. Also, access to large maildir (thousands of files) is slow. Things based on FFS (usenetfs...) can help with the second issue, but relatively slow metadata ops if deep inside FFS. > b) Something similar to Squid or Diablo's storage systems, i.e. one > file per object, with a hash function that spreads files across > 65536 buckets organized in 256 directories with 256 subdirectories > each, with (optionally) an index kept in a DBM file or something I'd look closely at these. > c) A commercial (or commercial-grade) relational database system > (Oracle, PostgreSQL, Frontbase) I beleive that highly specialised mail storage system will yield sufficently higher performance than 'generic' database. A typical lifecycle of file is: create write/read many times, with resizing delete So metadata performance is not very important A typical lifecycle of article/cached page is create and write read many times delete So metadata operations is not important as well But a typical lifecycle of email message is create and write read several times, most likely once or two sequental reading (HEAD/RETR) delete So metadata operation is AS important as file. I'm thinking of Btree with mailbox names and several variations of chunk-based allocation for messages themselves. I'm thinking either of some log-structured system, or btree of message numbers with explict grouping on mailboxes and in-page storage of small mails (about 14k...) With 128k buffers (I think it is the optimal buffer size for disk I/O nowadays) and about 64 Mbs of buffers for btrees first access to a mailbox will not require more than 10-12 disk operations (6-10 on filled-cache case) and subsequental accesses to this box will require no more than 1 disk op per file, actually about 1 operation per 4-6 messages. Anyone to criticize? Alex. ----- End forwarded message ----- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20001005153957.E65600>