Date: Wed, 25 Oct 1995 11:35:28 -0700 (MST) From: Terry Lambert <terry@lambert.org> To: julian@ref.tfs.com (Julian Elischer) Cc: terry@lambert.org, bugs@ns1.win.net, hackers@FreeBSD.ORG Subject: Re: process migration Message-ID: <199510251835.LAA19200@phaeton.artisoft.com> In-Reply-To: <199510250434.VAA16988@ref.tfs.com> from "Julian Elischer" at Oct 24, 95 09:34:47 pm
next in thread | previous in thread | raw e-mail | index | archive | help
> > 1) File as swap store. The executable file is acting as its own > > swap store; this means you must reopen the file (which means > > you need its name) and reestablish the flags on the vnode to > > orevent writes to it. > > write the entire process space including non resident pages.. > (implies that shared programs become static ) If we could easily do this, we'd be doing it already for file systems flagged remote and we'd be over the Sun "dataless client hang on server down" bug already. For a remote file system, we'd force clean pages into swap and dissociate the binaries fd. This is also the fix for returning ETXTBSY messages: when you are thinking of returning one,force the image to swap, dissociate the fd, and never return an ETXTBSY. Shared libraries don't necessarily need to be included IF the fixup vectors can be reset so future reference cause a refixup -- the shared library could be swapped out under a running process, assuming the migration waited for the library to not be entered. Since this is a kernel operation in any case, this could be done using the program counter to check. The alternative to delaying is to cause the library to "go static" if it is entered, or reset the vector table if it is not. > > 2) Memory overcommit. There very well may not be enough swap > > to checkpoint the program. > > put it out to a file....... This is unsatisfactory, actually. The migration of a process is tantamount to 1/Nth (N == average number of processes) of the work needed for a full suspend/resume. > > 3) Shared libraries. The shared library mappings must be > > restored, probably seperately. > > static.. quite possibly this might be used in a specialist environment > (such as what russel is working on,) where shared libs might not be required > in any case) That's certainly one possible cop-out. 8-). > > 4) Shared memory, semaphores, etc. > > this gets hard, Anything involving potentitally unrecoverable state is. It means you must shadow the state creation process to allow the state to be recreated. > > 5) Open file descriptors. > > you can only do a single process, file descriptors not open to files > probably are pointed deadfs (as if the other end died) The file descriptors open to files are the problem. This is because of the vnode issue for open-but-deleted files, and for the inode mapping issue. All you have to go on is the inode and device, and on a target system after migration, they won't necessarily be the same (the probably won't be) since files are accessed symbolically (by name). You will need to record the file names at open/creat time, and deal with descriptor cloning in fcntl, dup, dup2, etc. -- in the kernel. This is expensive, and can't be a general facilty because of the memory requirements. > > 6) Network listens (recoverable) and network connections (not > > recoverable if reliable stream delivery, recoverable for > > stateless datagrams). > > listens can probably be re-established if you can track them down.. > certainly a certain amount of info can be stored on each fd, and a listen > sounds feasible to restart.. Yeah. Though without content addressing on the network (something I've been pushing on for literally years), client won't necessarily be able to find the migrated process, since it has to be done by machine address. [ ... ] > > This is a non-trivial task. > I guess that's whey they decided to do it :) The first step is checkpoint/restart on a single system. Worry about actually migrating the things later. This saves you the kernel code for file name and similar per-system uniform symbolic translations. It's quite possible to recover an fd from a dev/inode pair if you are on the same system and it's not a deleted file: NFS does it all the time. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199510251835.LAA19200>