Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 13 Jan 2003 10:29:09 +0100
From:      Andy Sporner <sporner@nentec.de>
To:        "Rouzer, Charles A (Chuck)" <car@vitalit.com>
Cc:        freebsd-cluster@FreeBSD.ORG
Subject:   Re: freebsd cluster target market
Message-ID:  <3E2286E5.20609@nentec.de>
References:  <000901c2b8ef$57e4eb70$0201000a@LAPTOP> <20030110132550.A18143@lava.net> <193699876562.20030112164424@buz.ch> <001701c2ba84$6f307010$0201000a@LAPTOP> <37722441765.20030112230029@buz.ch> <003a01c2ba88$b6f20c70$0201000a@LAPTOP> <31724072031.20030112232740@buz.ch> <004901c2ba8a$cc16ba90$0201000a@LAPTOP>

next in thread | previous in thread | raw e-mail | index | archive | help

This seems to be a good of time as any to post my status on the bproc port.
I have been at this since Dec-23 and I can only say that it is not as 
easy as
I had imagined.    It has been relegated to a lower priority because 
there are
many things that I garnered from it that make sense in my phase 2 plans.  
However I have a fundamental problem with some of the architectural
choices that were made.     If somebody has more cycles, I can make ready
my initial work on porting the BPROC stuff in the form of a tar file.  What
is missing is the actual mechanics of the process extraction and the 
hooks in
the OS.    Where I differ with some of the ideas is that the communications
should be done in a user process, not in the kernel.  Call me a 
traditionalist,
but I don't see that much savings in this particular context to load more
stuff there.  It is more attractive to make a series of IOCTL functions to
insert and remove processes leaving a bare minimum of things in the kernel.

I have searched the web and found a project called "Maui" that is BPROC
API compatible and also exists for FreeBSD.  I have not looked much more
into it.  But it might be a point on where some urgent needs for this kind
of thing can be resolved.

As for filesystems and the like.  I had also time to make a survey of all
of the items in consideration:  
     -  DRBD
     -  IScsi
      - The problems of NFS.

I have come to the conclusion that, while a low layer solution is fine,
there still is a problem coordinating access to filesystems built on top.
Iscsi would make failover of the devices easier, but you still have to
syncronize access.  

As for NFS,  problems are noted with regard to changing things under
the cover--due largely to the opaque nature of the file handles.  I am
still studying this, but I think it is possible to convert the filehandles
into a physical reference (such as filesystem_id:Inode) so that this
reference can float around and be stateless within a cluster.  In this way
this application does not even know which server he got the information
from.  This takes care of the top level syncronization.    The main point
I have not come to a conclusion on yet, is to either patch NFS to handle
things this way, or to just do a new FS from scratch.  Adding NFS
functionality would be great, but there are some new directions I also
want to pursue as well.  Such as a registration service so that a Filesystem
can be found on the network by request.  That way when a failover occurs,
the I/O can continue.   Additionally I would like this to be TCP based 
so that
very large buffers can be transmitted ( > 64K).  This has the additional
benefit of being ready for WAN.

This is really just a small snip from a document I will be publishing
on the web (as soon as Deutsche Telekom get re-establish my DSL!!!).

What is coming in the same time frame is release 215 of my software
suite, which now includes a network NAT to do load balancing as well
as regular failover.   This would allow for highly available virtual 
servers.

The next step after 215 is to focus on the filesystem that I described
above.  Hopefully 2nd week of February this is realizable.

It would be good if I-scsi is available because that just would make
the failover stuff more flexible.  I plan initially to use two servers
with shared SCSI busses.   It would be great to throw this away in
favor of I-SCSI.


Bye for now (more work to do..... ;-)


Andy




To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-cluster" in the body of the message




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3E2286E5.20609>