From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 24 07:46:29 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 641C71065673; Thu, 24 Feb 2011 07:46:29 +0000 (UTC) (envelope-from creddym@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 972738FC14; Thu, 24 Feb 2011 07:46:28 +0000 (UTC) Received: by wyb32 with SMTP id 32so297498wyb.13 for ; Wed, 23 Feb 2011 23:46:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:date:message-id:subject:from:to:cc :content-type; bh=A7RHomDfeEen2UrxdzERCGyKKfgOXuT7iy+bvpdEBa8=; b=lMWKij+Py4aU8bWPGkDkrpmyJ3OvdhHm7XszCUEjpt7zpXQveDvmyeX8YAfSo8/J0S C7WVO14g2ZCo6L3E/jcNtr+HiGYJzGC+AhEpUQvTD8bDSMNzRLv8dlaeM/FYU1LRsreD c5ys4/o1fpjYsc+/NT5yJwPsrZQQt++wujrT8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:cc:content-type; b=xifYm+hEUEgRMbZcf5Xjgkm/gXEnOdORULkMf6IUpD/C55DgTf7UmrYB2Wd9AX9MG4 agjxouFhj9VI5Emr87yPE1qZgHRe1BfGziGyYs71TwRU7fHurbu6RbQCyUnURKuCtDH8 /svMs9IF+sNh0A0bNsa6Q2/C7G4iOXeJXBGVE= MIME-Version: 1.0 Received: by 10.216.142.224 with SMTP id i74mr343926wej.83.1298531946834; Wed, 23 Feb 2011 23:19:06 -0800 (PST) Received: by 10.216.78.147 with HTTP; Wed, 23 Feb 2011 23:19:06 -0800 (PST) Date: Thu, 24 Feb 2011 12:49:06 +0530 Message-ID: From: chandra reddy To: freebsd-questions@freebsd.org X-Mailman-Approved-At: Thu, 24 Feb 2011 12:06:54 +0000 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org, freebsd-i386@freebsd.org Subject: Why FreeBSD fetch does not download a file via a proxy for HTTPS URLS (the same works fine for HTTP urls) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Feb 2011 07:46:29 -0000 Hi All, I am working on a project where i need to download a file via a proxy server using HTTPS protocol. I found that fetch does not work/support HTTPS requests over a proxy. My setup would be like this: Intranet Internet ----------------------------------------------------------------------- | https or http | https | Client m/cs -----------------------------> Porxy Server -------------------------------> Destination Server (or Download server) | | ----------------------------------------------------------------------- I can use https or http protocol between Client and Proxy but only HTTPS is used between proxy and Destination server(or Download server) . I tried to use "squid" proxy as my proxy server and tried to download a file from my download server to Client m/c using FreeBSD "fetch" command. It fails to download a file via proxy for HTTPS requests Please note that Proxy setup is 100% correct and a web server (Apache) running fine. [I have tested it using my Mozilla browser on my PC]. I have done the following: 1. *Download a file using HTTPS over a proxy server* #env HTTP_PROXY=http://:3128/ /usr/sbin/fetch -v -o /tmp/download.out 'https:///index.htm' looking up connecting to:443 connection established fetch: https:///index.htm Authentication error Even I have tried this also and found the same error. #env HTTP_PROXY=https://:3128/ /usr/sbin/fetch -v -o /tmp/download.out 'https:///index.htm' My question is why it is not connected via "Proxy sever". It tries to connect directly. I could see that if I use HTTP protocol then it connects via proxy. Please see the logs here. 2. *Download a file using HTTP over a proxy server* #env HTTP_PROXY=http://:3128/ /usr/sbin/fetch -v -o /tmp/download.out 'http:///index.htm' looking up connecting to :3128 connection established requesting http://destination-server-ip/index.htm Even I have tried this also and found that works fine. #env HTTP_PROXY=https://:3128/ /usr/sbin/fetch -v -o /tmp/download.out 'http:///index.htm' I have debugged "fetch" and found that the following check is stopping HTTPS requests over a proxy. *http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libfetch/http.c .OR. http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libfetch/http.c?annotate=1.78.2.5.4.1 * 1375: 1.58 des 1376: if (purl && strcasecmp(URL->scheme, SCHEME_HTTPS) != 0) { 1.51 des 1377: URL = purl; I could overcome the above problem if I do the following change. 1375: 1.58 des 1376: if (purl) { 1.51 des 1377: URL = purl; I want to know why HTTPS over proxy is not working with "libfetch". I want to make it work how can do it? Thanks -Chandra