Date: Wed, 01 Jan 2003 12:27:06 -0800 From: Terry Lambert <tlambert2@mindspring.com> To: Brad Knowles <brad.knowles@skynet.be> Cc: Patrick Cable II <freebsd@slaudiovis.org>, chat@freebsd.org Subject: Re: Backup Solutions Message-ID: <3E134F1A.931CA7BF@mindspring.com> References: <3E0DC536.8010001@slaudiovis.org> <3E0EBC49.86AD7E28@mindspring.com> <a05200f09ba3573361365@[10.0.1.5]> <3E0FF119.7792A270@mindspring.com> <a05200f0cba3853b5bcaf@[10.0.1.5]>
next in thread | previous in thread | raw e-mail | index | archive | help
Brad Knowles wrote: > At 11:09 PM -0800 2002/12/29, Terry Lambert wrote: > > For Windows and Macintosh, you are most likely to be using some > > Microsoft based solution, like Access, or Access with a Microsoft > > SQL back end. > > You say I'm going to be using *WHAT* to do a file/filesystem > level backup of my Mac?!? You know, the one running MacOS X, which > is a Mach kernel with a BSD userland? You are going to use software from the database vendor to dump the database contents into a form which flattens out pending transactions so there are none in progress at the time you take your bckup snapshot. If your database vendor is Microsoft, then you are going to use the Microsoft tools for this. > If you're talking databases on Macs, then you're probably talking > Filemaker or something like that. And that's a totally different > kettle of fish than anything related to anything remotely resembling > anything relating to Microsoft. The actual vendor is not relevent; what is relevent is that you are going to be stuck with vendor supplied tools to do the work. > In our case, we're talking about laptops that will probably be > somewhere else in the house, networked via 802.11b with a proper > software VPN solution for real security. There would definitely need > to be some automated process to fire up the backup script to run > overnight, and to shut the machine down when it's done. Client backup solutions suck. The best client backup solution is to store no data on the clients, and back up the server. The next best solution is to constrain all user data to a subhierarchy of the system directory tree, dividing completely machine instance data from user data. This has the side effect of making it very easy to transition data from an old machine to a new machine, simply by copying a folder. A high-latency mechism would be to use the "briefcase" paradigm; the berifcase is then periodically synchronized with a fixed PC system. This approach, uh, "sucks", unless you are willing to seperately dedicate a desktop machine for every laptop machine. An alternative is client/server backup, with a backup client that is installed on the laptop, and backs up to the server. In general, this does not work well because of the transient connection. It also has the effect of providing negative reinforcement on connectivity, by running each time the laptop is "in range of the mother ship", which has the effect of people keeping their machines unconnected while they are "in the office", to avoid being nailed to the floor for the (ever increasing, due to non-connectedness) duration of the backup operation. ...meaning that there is not a satisfying answer to the problem, unless you are willing to replace the file systems on the laptops with "trickle replication" file system implementations, and then dedicate space on a server. As far as I know, the software to do this is all part of university or corporate research projects at this time, and _none_ of it runs on Windows, for lack of source code for Windows. > >> Is there an Amanda PC/Windows client? Or an Amanda MacOS X > >> client? > > > > There is one in beta right now. IT's available from: > > > > http://sourceforge.net/projects/amanda-win32/ > > > > It doesn't seem to have been updated since last June. 8-(. > > > > You probably are not going to find it useful, due to the "open > > file backup" problem. The normal way to handle "open file backup" > > on windows is to install an IFSMgr hook to hook calls into the > > IFS manager. You can then use the existing open instance for the > > file in question in order to back it up. > > Sounds good. I don't suppose anyone knows if anyone is ever > planning on adding features like this to the Amanda PC client? Unlikely. Most Open Source software does not operate on the basis of a design or architecture known in advance -- though it may start that way, if it arises out of corporate or academic research. What happens instead is that people quit adding to the code once it works "well enough", as opposed to working "to spec.". It's also a lot of work, in that it requires you to build a shadow state machine for file operations, and have in depth knowledge of Windows internals in the IFSMgr area. Things like the O'Reilly IFSMgr book don't actually provide enough information, since they fail to document hiddent assumptions and some interfaces (e.g. there is an interface that can obtain the raw block offsets on a disk to use for swapping by Windows, through demand paging to direct device I/O, but that interface is not documented, nor is the fact that an FS must support obtaining Level 3 then Level 1 volume locks, in order to access the raw disk, since that information would help virus writers write viruses that ignored FS and boot block-based protections). > > This is usually not > > very satisfying for database files, since the transactional > > representation of atomicity and/or idempotence to the application > > are done at the database application level, and can't be guaranteed > > at the IFSMgr level (there is no nesting information pushed by the > > application to the IFSMgr, so you can wait for a 1->0 nesting level > > for transactions in progress to complete, before doing your backup). > > While these issues may be important to others, we do not > currently have any call for databases like this in our house. It is not this simple. Microsoft Access will inherently use database methods to store data on the disk. Likewise, and document type based on OLESS (Microsoft OLE Structured Storage) also uses database methods to store data within the container files. This includes, but is not limited to, intermediate modification and automated backup storage of the contents of Microsoft Word documents and Microsoft Excel spreadsheets. The software is not designed to permit snapshot-and-restore of file contents while it is in active use: it depends on the assumption that the system will be quiescent, and files will be closed. > > The most common method is to export the FS as a share, and then > > use Amanda with SAMBA (client) in order to back up the data; this > > has the same problems. See the online book at: > > > > http://www.backupcentral.com/amanda.html > > > > specifically: > > > > http://www.backupcentral.com/amanda-13.html > > There are not currently any Unix filesystems in our house being > exported via Samba, as there are currently not any Unix fileservers > in the house. I think you misunderstand. The SAMBA machine acts as a client of the Windows machine on which the files to be backed up are located. As an example, Fred has a laptop, and you have followed the suggested practice of making him put all files he wants to have backed up into the folder "C:\Fred\" on that laptop. This is then exported via the "File and Printer Sharing for Microsoft Networks" option. The UNIX system on which the backup software runs then, in the process of the backup, mounts this share as a local file system, backs up the local file system, and then unmounts the share. > When I do set up the fileservers, there almost certainly won't > ever be any databases running on the exported filesystems, so I would > be inclined to believe that these filesystems should be backed up > from the client side. This is an invalid assumption. Just because you are not running software which is sold by the vendor as database software does not mean that you do not have database files. As noted above, even "simple" Microsoft Word documents are, in fact, database files. > I'm not worried so much about the server backup issues, at least > not per se. I am more concerned about the client file/filesysystem > backup issues, and how I can get that data onto a server where I can > reliably back it up somewhere else. Moreover, I want to use freely > available software and reasonably inexpensive hardware, which is a > good part of the reason why FreeBSD would also be involved on the > server side. I think you will find that backup is like X windows, in that it will require an inversion of the terms "client" and "server". A "server" is something that provides a "service", in that universe. The Amanda beta client for WIN32, for example, exports a service that the Amanda backup server then exploits in order to backup the data on the Windows side of things. Short of using an alternate FS, and a product built to use special hooks in the FS (e.g. Veritas' Windows VxFS, Volume Manager, and backup products), you are going to have a hard row to hoe, with Windows. > >> What about the handling of tape swapping, archiving, and > >> other things normally done with stackers and libraries? > > > > You use stackers and libraries. 8-). > > IIRC, Amanda doesn't support stackers or libraries. Are there > any other tools that do? Amanda supports both, including the very large capsule-shaped transparent tape-robots you see in some movies (e.g. "Sneakers"). 8-). > > In most cases, you are talking about spending money for commercial > > software, since the professional database backup software is often > > a seperate product add-on. > > At the moment, we're not using any database software, so anything > specific to databases is not currently relevant, although I do want > to keep these things in mind as I try to design a comprehensive > backup solution. It will bite you anyway, since most "flat" data files for Microsoft Office products are in face OLESS-based non-flat-file databases; sorry. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-chat" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3E134F1A.931CA7BF>