Date: Tue, 11 Mar 97 16:27:23 CST From: Joe Greco <jgreco@solaria.sol.net> To: terry@lambert.org (Terry Lambert) Cc: terry@lambert.org, scrappy@hub.org, marc@bowtie.nl, neal@pernet.net, hackers@freebsd.org Subject: Re: freebsd as a news server? Message-ID: <199703112227.QAA01054@solaria.sol.net> In-Reply-To: <199703112149.OAA26149@phaeton.artisoft.com> from "Terry Lambert" at Mar 11, 97 02:49:28 pm
next in thread | previous in thread | raw e-mail | index | archive | help
> > Terry, what ARE you babbling about?
> >
> > A list that is stored as directory data certainly IS an index, and INN
> > really does not use anything much more complicated than the FS structure,
> > flat files, and a big hash table _anywhere_ in the code. If you wish to
> > insist on calling _those_ indexes and databases, fine, but your original
> > babble still makes no sense even in that context.
>
> I think you are mixing threads here:
>
> Index is Index is Index is Index is
> in file in file in directory in directory
> mount -async !mount -async mount -async !mount -async
> Index safe
> from crash
> corruption? NO NO NO YES
There is no "index in file".
There is only "index in directory".
In the case where you have a directory mounted -async, and a crash prevents
an article from being written, "who cares!"
> I am making an exception for directory data in this discussion so I
> can talk about *only* data that may imply state that *is* subject to
> corruption on crash, regardless of sync vs. async mount.
>
> For two stage commit using sync/fsync/O_WRITESYNC, I can say:
>
> Index safe
> from crash
> corruption? NO YES NO YES
>
> In no -async case is an index safe.
The only case in which -async is not safe is when it toasts your disk,
which I haven't seen to be the case.
It is possible you will lose an article. But, you may lose articles due
to dozens of other reasons, and it certainly is not going to cause some
god awful end of the world catastrophe.
> > What little chance there is for desynchronization is generally worked
> > around in various manners. A great example is the news overview
> > "database"... if it is missing, broken, or doesn't contain the
> > information, the data is generated on the fly.
>
> Unless the information was locally generated, in which case it is not
> possible to recover it. If I post to a host running mounted -async,
> and that host does not guaranteed to put the article on disk before
> it responds to me, and crashes after the response before the article
> has been committed to stable storage, then my article is lost.
Wrong, Terry, it doesn't have to be that way, just because it works
that way for some folks. You're building a case based on assumptions
that do not necessarily _have_ to be valid.
In my case, when offered an article, it's offered to thirty peers
instantly, and sent to twenty eight of them long before the disk
head gets to where it will write it to disk.
That's a pretty small margin for error. Of course, what you're
missing is that the post gets spooled to an area of the disk that
is NOT mounted -o async, first, and gets offered to multiple servers
before getting wiped, so if it doesn't make it out, something really
strange must have happened.
Welcome to '90's news server technology. I don't care what was true
five years ago. I don't care what the clueless newbie newsadmin is
doing, either.
> This is different thatn if it is just a read/mirroring server, since
> all articales which did not locally originate are recoverable, per
> your example above. The special case of locally generated articles
> is not.
The unspecial case of locally generated articles is boring because we're
both smart enough to engineer a way around it.
(I think.)
> > In the commonly distributed version of INN, the current spool IS the final
> > authority as far as which articles exist and which don't. If you choose
> > to crash the system, clri a bunch of articles and then fsck the file
> > system, INN will take the spool's current state as being authoritative.
> >
> > If you wipe out the overview caches, it will generate that data from
> > the actual article files. Neither case is a problem.
>
> The second case is a "delay from crash to service availability" problem.
> Whether it is a real problem or not depends on the type of availability
> guarantees you make to your customers, since the recovery process does
> not happen in zero time.
And your point is...? Go write me a FS that is both fast AND doesn't
require a fsck on startup. Until somebody gives me one, I guess I'll
live with the fsck, and not be too worried because my news servers all
have better availability and more current news than others in the area.
If I really wanted to be picky, I'd take an Uzi to your whole silly
argument by showing you the latest Fritschie CNFS code, which isn't
at all concerned with or vulnerable to this issue, but then we aren't
discussing UFS anymore.
> > That's why I don't really CARE if I lose bunches of files when I mount
> > my disks -o async. It is a system that is relatively impervious to
> > various sorts of {Terry-imagined,real} damage,
>
> Because the data is recoverable from other sources, and because your
> service availability requirements are scoped that downtime following
> a crash permits recovery to occur. Yes.
What ARE you babbling about? I have no downtime after a crash.
> Which all goes to say that, as I said before, it's subjective based on
> your application, how you implement the application, and service
> availability guarantees which are in place. Your setup works for you,
> but your parameters are sufficiently narrowly scoped as to make it a
> non-problem. For you.
Terry, anyone can invent their own problems. My car don't fly, so I don't
get where I'm going fast enough when there's a traffic jam.
But, then, I don't pretend to think that my car should be able to fly.
... Joe
-------------------------------------------------------------------------------
Joe Greco - Systems Administrator jgreco@ns.sol.net
Solaria Public Access UNIX - Milwaukee, WI 414/342-4847
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199703112227.QAA01054>