Date: Sat, 10 Apr 1999 11:32:22 -0500 From: Dan Nelson <dnelson@emsphone.com> To: flygt@sr.se Cc: FreeBSD Questions <freebsd-questions@FreeBSD.ORG> Subject: Re: Checking if a file transfer is complete Message-ID: <19990410113221.A90660@dan.emsphone.com> In-Reply-To: <19990410124148.B93126@sr.se>; from "Gunnar Flygt" on Sat Apr 10 12:41:48 GMT 1999 References: <19990410124148.B93126@sr.se>
next in thread | previous in thread | raw e-mail | index | archive | help
In the last episode (Apr 10), Gunnar Flygt said: > I'm writing a script that is taking care of files ftp'd to a server. > It is moving the files to the right directory, and then putting data > concerning the file into an Oracle data base. > > I have a small problem though. I'm planning to let cron do a lookup > once every minute to see if there are any new files. But the file > transfers can take several minutes to finish, since the files are > pretty big, and the transfer speed is at max 2Mbit/s with an average > of maybe 400kbit/s > > So the question is, How do I check that the file transfer has > completed? You've got four ways. 1. Check the mtime of the file you're interested in. If (now-mtime) > 30 seconds, assume the file is done. 2. Run fstat on the file you're interested in and see if any process has the file open. If it does, skip it until the next processing run. 3. Force your input files to have a standard "trailer"; i.e. if it's a text file, have "**EOF**" as the last record or something. Then simply check the last line of any files in the directory. 4. Parse the last minute's worth of data from /var/log/xferlog, and process any files you find. This is the most robust way, but requires a little bit more work (it's much easier to stat(2) or check the output of fstat(1)). With this method, though, you can't simply copy a file into the directory and expect your program to process it, since there won't be an xferlog entry for it. I do almost exactly the same thing here at work, and option #1 is my only way, since the file can appear in my directory either by FTP or an NFS copy (an open file over NFS does not show up on fstat, and doesn't get logged anywhere), and I can't control the input format. -Dan Nelson dnelson@emsphone.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?19990410113221.A90660>