Date: Sat, 18 Nov 1995 15:46:11 +0500 (GMT+0500) From: "Serge A. Babkin" <babkin@hq.icb.chel.su> To: terry@lambert.org (Terry Lambert) Cc: terry@lambert.org, hackers@freebsd.org Subject: Re: NFS vs. NetWare for DOS file services Message-ID: <199511181046.PAA01710@hq.icb.chel.su> In-Reply-To: <199511171700.KAA05647@phaeton.artisoft.com> from "Terry Lambert" at Nov 17, 95 10:00:02 am
next in thread | previous in thread | raw e-mail | index | archive | help
> > It really looks like a problem. For DOS client the solution may be to > > flush the whole cache before issuing an unlock request. But I think > > any network-aware DOS program must use fsync. > > Here's the rub: most "network-aware DOS programs" don't use NetBIOS. They > use a proprietary API. 70% of all DOS network nodes overall are NetWare. > The majority of small business networks (100 employees or less) are > Lantastic. > > For a proprietary API, you can define caching-friendly interface semantics; > this is exactly what has been done for those systems that cache. I have tried FoxPro 2.0 both on Netware and NFS (I have learned in the university where it was runned on Netware and I did laboratory works at my work where I had NFS installed) and it worked on both. So it looks like it uses DOS lock calls that are then implemented through Netware or NFS. I don't know about other calls. It looks like DOS (and Netware too) has no fsync call or anything like, the full sync call appeared in DOS 6+ olny. > > > Write cache going to disk relies on the fact that some writes are async, > > > and may be cached, and some writes are sync, and must complete before > > > they return, leaving the data on the disk. > > > > > > DOS doesn't have the distinction between async and sync writes, so you > > > can't just say "I'll just cache async writes" -- because there aren't > > > any. > > > > The programs using Netware must know about this difference. Most programs > > that really do record locking are working on Netware, not on a local machine > > (why else you need record locking on a single-tasking OS?) and differ > > async and sync writes. > > OK. I worked on the file system for the NetWare for UNIX 4.x product for > nearly two years using the Native NetWare 4.x source code as a base, so > I know a "little bit" about this. It's interesting ! > NetWare clients do client caching. They use the low 24 bits of the file > handle as a hash index (which incidently screwed me treating the short > file handled as an opaque 32 bit value like you are supposed to be able > to). > > This cache is used mostly when reading .exe and other files which are > opened read-only or which are read-only for the client, as determined > by querying the file attributes. > > This may not seem like a big win, but 70% of all requests to a NetWare > Server are read requests. BTW, which percent of them are reads of .exes ? If it is small all these tricks with 9K will give small effect. > There is a big win on .exe loading to caching. Specifically, the way > DOS loads executables causes it to refer to the first 9k several times > in the load process. Is it because the typical size of .exe relocation table is 9K ? IMHO this can be implemented (may be less effective but enough simple) by reserving 9K cache where to put the data readed from the first 9K of any file. When reading from the first 9K of any file we can lookup first in this buffer. > A DOS client loading a .exe will not trigger read-ahead, because the > first 9k is rereferenced when the file is otherwise sequential I/O. Do other reads trigger read-ahead ? I think that if the common cache is different from the "9K cache" read-aheads will not pain because they will not disturb the "9K cache". > Windows is a bit different here because it pages, but client caching > has a big effect (obviously) on paging. > > The biggest win a UNIX box acting as a server for DOS clients can > implement is file name and open mode specific madvise() operations. > > After reads, the next most frequent operation is writes, which are about > 18% of operations. Typically, writes are not sequential. There is > little you can do to improve write performance, other than to remove the > synchronous requirements. These requirements are removed by: > > 1) Removing POSIX semantics on the file system (dangerous). I'm sorry... Explain me please shortly what is this semantics. > 2) Changing the mechanism which guarantees the POSIX semantics > to use a better scheme than synchronous writes to provide > the ordering guarantees (ala the UnixWare 2.x UFS, and the > U of Michigan EECS "Soft Update" Metadata update mechanism). > > The first isn't a real option; it's bogus. The second reduces > drastically the absolute overhead that you have to eat to maintain the > NFS protocol guarantees consistently. > Of the remaining 12%, 8% are directory operations, and 4% are all other > operations. > > Directory operations accont for a disproportionate amount of the time > for the client waiting for a server response. > > The biggest win, which only recently went into NetWare and NFSv3, is > the ability to return multple entries at the same time. I have read the RFC on NFSv2 and there was said that it has the ability to read multiple directory entries at once. Was it just not implemented in servers ? > The POSIX guarantees once again get in the way of performance. For > directories, however it is possible to cheat. > > POSIX guarantees file times update semantics. In addition, it guarantees > directory time update semantics when getdents() is called. BUT... POSIX > does not require that you consider a directory to be a file: it can't, > since VMS can be POSIX compliant (point of interest: many of the idiotic > semantics with regard to "shall be updated", "shall be marked for update", > and "may be marked for update" that POSIX requires seem to be related to > vendor "special interests" in defining POSIX so they don't have to recode > existing systems). > > So does POSIX require that you use getdents() to access directory > contents? It does not. So you can cheat by "defining" a directory > to be other than a file, and then using an interface other than getdents() > to get at it. Or you can say "to hell with the dancing" and give yourself > a blanket license to relax POSIX guarantees, as long as you don't do > anything that compromises file system integrity. > > Note that this type of crap won't fly with NFS, which pretends to provide > POSIX semantics to its clients. Getdents() reads multiple directory entries at once, why we can't say that reading multiple directory entries through NFS at once is like getdents() ? > So NetWare achieves its speed by not being POSIX and by using interfaces > other than the NetBIOS interface to implement its own cache-friendly > semantics. > > Oh yeah. And by having the entire volume directory structure, including > file metadata, in core at all times and accessed via hashlist. And you > wondered why mounting took a long time. I have throught that the main reason why Netware loads all metadata in core is that its access rights are very "heavy". > > > DOS client applications assume O_SYNC on all opens. > > > > Pure DOS (without any network) assume exclusive access to anything and do > > not use any locks at all. > > No. "SHARE" provides file granularity access locking semantics. Which > means multiple reader OR single writer guarantees are made, even under > NetBIOS. I'm not shure what SHARE does, I have throught that it simply implements DOS FLOCK call on Microsoft networks. Does not it ? > [...] > > > DOS libc fsync() does nothing. Or applications don't call fsync(). > > > Very differnt form assembly programs calling the INT 21 op. > > > > How do they work on the Netware network ? If they want to work on it they > > need to use fsync() too. And I think most dBASE-like databases do these > > calls. If they do not use these calls they would not work on Netware too. > > They use the NetWare TTS API and convert the database file I/O into > transactions that can be backed out. > > The start-transaction/end-transaction pairs indicate to the server > the implied state in the client application that accesses multiple files. > > If I crash and come back up, for every "start-transaction" without an > "end-transaction", the server can decide to throw away the changes that > that transaction wanted to make by identifying them by transaction ID. > > So it works by the "NetWare aware" application using NetWare API's instead > of the standard DOS APIs (and the NFS client typically only hooks the > DOS API's). In my opinion what you describe are "very Netware-aware" applications but most applications work on Lantastic, NFS etc. too. I think that dBASE-like systems must not be so network vendor dependent. Serge Babkin ! (babkin@hq.icb.chel.su) ! Headquarter of Joint Stock Commercial Bank "Chelindbank" ! Chelyabinsk, Russia
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199511181046.PAA01710>