Date: Sun, 20 Dec 1998 10:14:09 +1100 From: Sue Blake <sue@welearn.com.au> To: cjclark@home.com Cc: FreeBSD Questions <freebsd-questions@FreeBSD.ORG> Subject: Re: Web Download Message-ID: <19981220101409.14856@welearn.com.au> In-Reply-To: <199812172035.PAA17036@cc942873-a.ewndsr1.nj.home.com>; from Crist J. Clark on Thu, Dec 17, 1998 at 03:35:52PM -0500 References: <199812172035.PAA17036@cc942873-a.ewndsr1.nj.home.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Dec 17, 1998 at 03:35:52PM -0500, Crist J. Clark wrote: > I've got a question that ju-ust borders on a ports question > enough that I feel OK bringing it here. There is a server that > has a single data file that we would like to download daily. > This used to be done via a cron script and ftp. However, the > server recently stopped allowing anonymous logins... but it > is wide open under http. We want to still get this _one_ > file, but the only people here with the relations and the > reasons to get the ftp access to be changed back won't do it. > So, I will need to use http. > > So, my questions are: > > 1) A quick look at lynx has not revealled to me a quick > and dirty way to get it to download it in a script. > Anyone familiar enough with lynx to show me how I > would do it? Lynx can grab files as they are, but most often it's used to translate back to plain text which it does well. It's easy to get confused between all those options. Here's two examples to show the difference. The first example isn't exactly what you need now, but it'll be useful to anyone who wants to turn a web document into text on a regular basis. It's my ~/crontab entry to send the weekly big-stick message to freebsd-newbies: # send Newbies FAK 30 12 * * 6 /usr/local/bin/lynx -dump -nolist /usr/local/www/data/freebsd/newbies/index.html | mail -s "Newbies First Aid Kit" freebsd-newbies@freebsd.org #half-past midday any-date,month saturdays Mail the FAK as plain text You do need to put the full path of the executable so cron can find it. Instead of the second path above you'll want a URL, and instead of -dump (which translates to plain text from HTML) you'll want -source. The -nolist above stops lynx from trying to put a helpful list of all the file's links at the end of the text file. Usually needed with -dump but never with -source. You won't want to mail the file, you'll just want to save it. So I figure you'll want to add something like this to your personal crontab file: # Get daily file by http 30 9 * * 1,2,3,4,5 /usr/local/bin/lynx -source http://www.freebsd.org/gifs/dae_up2.gif > latest.gif # nine-thirty-am mon-to-fri Get today's gif by http Then run $ crontab crontab to make it stick, and wait for 9:30 (or whenever) to check it works. If you can view the gif, I guess it didn't work too badly. Also crontab(5) and crontab(1) might help. > 2) There are a number of mirror type utilities in the ports. > However, I am pressed for space on my hard drive (and > actually would rather do this on a different machine) > and think that an elaborate mirror utility is overkill. > Are there any web utilities that would be reasonable for > downlioading _ONE_ file a day? There's several good small file-getters, probably easier to figure out and less hungry than lynx, but you need to make sure they're doing http protocol for this particular file, don't you. -- Regards, -*Sue*- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?19981220101409.14856>