Date: Mon, 21 Jun 2010 12:38:46 +0100 From: Tom Evans <tevans.uk@googlemail.com> To: gljennjohn@googlemail.com, des@des.no Cc: freebsd-current <freebsd-current@freebsd.org> Subject: Re: Ports doesnt respect fetch environment settings Message-ID: <AANLkTikrXvx_3Y7nkceaKSYqLQemGrIBURt1-5pwl6qr@mail.gmail.com> In-Reply-To: <20100621130743.4df77343@ernst.jennejohn.org> References: <AANLkTinn5bPXDf-tRIlpGCp3iFgtYi20mzRSbqkBcj6b@mail.gmail.com> <20100621101046.GA76036@droso.net> <AANLkTilhgzVDFX9MGQTS_DzJt_5QtlaNF--nQss-5mzj@mail.gmail.com> <20100621130743.4df77343@ernst.jennejohn.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--001485f428486bede1048988bf37 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Mon, Jun 21, 2010 at 12:07 PM, Gary Jennejohn <gljennjohn@googlemail.com> wrote: > Yes. =C2=A0When you ran fetch by hand you didn't have the -ApRr on the CL= . > Could it be that the 'p' flag is causing problems? > > Try running fetch by hand again with those flags and see what happens. > If it fails, try removing the 'p' flag. > > -- > Gary Jennejohn > Yes, I went through the same logic - its not the 'p' flag, its the 'A' flag, which is supposed to prevent it following 302 redirects. In this case, it refuses to retry the request when it receives a 407. > # /usr/bin/fetch -ApRr -v -S 37867 http://googlecl.googlecode.com/files/g= ooglecl-0.9.5.tar.gz looking up proxy connecting to proxy:3128 requesting http://googlecl.googlecode.com/files/googlecl-0.9.5.tar.gz proxy requires authorization fetch: http://googlecl.googlecode.com/files/googlecl-0.9.5.tar.gz: Proxy Authentication Required root@strangepork '12:13:28' '/usr/ports/net/googlecl' > # /usr/bin/fetch -pRr -v -S 37867 http://googlecl.googlecode.com/files/go= oglecl-0.9.5.tar.gz looking up proxy connecting to proxy:3128 requesting http://googlecl.googlecode.com/files/googlecl-0.9.5.tar.gz proxy requires authorization looking up proxy connecting to proxy:3128 requesting http://googlecl.googlecode.com/files/googlecl-0.9.5.tar.gz local size / mtime: 37867 / 1276839258 remote size / mtime: 37867 / 1276839258 That doesn't seem right! Looking in lib/libfetch/http.c it tries to fetch the file in a loop. libfetch first tries without proxy authentication, even if you specify it in your environment. If the request fails due to proxy authentication being required, it sets a flag to add proxy auth details next time through the loop, and continues. If the '-A' flag is set however, it will only go through this loop one time, and so does not attempt to use the supplied proxy authentication. Comments in the source code imply that this is a change in behaviour introduced at the start of the year to support digest authentication; prior to that it would have attempted proxy auth on the first request. The flag for '-A' is documented as: -A Do not automatically follow =E2=80=98=E2=80=98temporary=E2=80= =99=E2=80=99 (302) redirects. Some broken Web sites will return a redirect instead of a not=E2=80=90found error when the requested object does not exist. where as the behaviour is: Do not attempt to download this file more than once, for any reason. Having seen this, the bug is that we wish to go thru the loop one more time to retry with proxy authentication added, but the loop may exit on the next iteration. This diff allows it to go through the loop one more time, and now fetches the file correctly. Incidentally, having fixed fetch to work with '-A' thru a proxy that requires proxy auth, I now dont require anything in FETCH_ENV or FETCH_*_ARGS, it works correctly with the PROXY_* environment variables. Patch attached. Cheers Tom --001485f428486bede1048988bf37 Content-Type: text/plain; charset=US-ASCII; name="lib::libfetch::http.c.diff.txt" Content-Disposition: attachment; filename="lib::libfetch::http.c.diff.txt" Content-Transfer-Encoding: base64 X-Attachment-Id: f_gap8fppp0 SW5kZXg6IC91c3Ivc3JjL2xpYi9saWJmZXRjaC9odHRwLmMKPT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQpSQ1MgZmlsZTog L2hvbWUvbmN2cy9zcmMvbGliL2xpYmZldGNoL2h0dHAuYyx2CnJldHJpZXZpbmcgcmV2aXNpb24g MS43OC4yLjUKZGlmZiAtdSAtcjEuNzguMi41IGh0dHAuYwotLS0gL3Vzci9zcmMvbGliL2xpYmZl dGNoL2h0dHAuYwkyNyBKYW4gMjAxMCAxNDo1NDo0OCAtMDAwMAkxLjc4LjIuNQorKysgL3Vzci9z cmMvbGliL2xpYmZldGNoL2h0dHAuYwkyMSBKdW4gMjAxMCAxMTozMDozMiAtMDAwMApAQCAtMTcx MCw2ICsxNzEwLDcgQEAKIAkJCQlnb3RvIG91Y2g7CiAJCQl9CiAJCQkvKiB0cnkgYWdhaW4sIGJ1 dCBzZW5kIHRoZSBwYXNzd29yZCB0aGlzIHRpbWUgKi8KKwkJCSsrbjsKIAkJCWlmICh2ZXJib3Nl KQogCQkJCWZldGNoX2luZm8oInByb3h5IHJlcXVpcmVzIGF1dGhvcml6YXRpb24iKTsKIAkJCWJy ZWFrOwo= --001485f428486bede1048988bf37--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTikrXvx_3Y7nkceaKSYqLQemGrIBURt1-5pwl6qr>