From owner-soc-status@FreeBSD.ORG Tue Jul 2 11:00:14 2013 Return-Path: Delivered-To: soc-status@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 99077A56; Tue, 2 Jul 2013 11:00:14 +0000 (UTC) (envelope-from b.ambarisha@gmail.com) Received: from mail-vc0-x232.google.com (mail-vc0-x232.google.com [IPv6:2607:f8b0:400c:c03::232]) by mx1.freebsd.org (Postfix) with ESMTP id 4945E1601; Tue, 2 Jul 2013 11:00:14 +0000 (UTC) Received: by mail-vc0-f178.google.com with SMTP id m17so2641606vca.23 for ; Tue, 02 Jul 2013 04:00:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=vyEDZp+So+3Y0BtafT3LZPTYYhvBAZhjRWJNmFGx7pI=; b=RYVnsIwtf53EnkOxilYQtYo1s2/EsccwlJKbP43oAF4SuadT2HgiycNKSEbcvEXSpU r/gPzdqLe/Jp7wa7AkWJjaf4oMKUV4xLkTES8Rj+OUvaRZnwQF/tOGgYzuZeuKtlNHui onA2AChWTZfKiQ1WkSEm1qJQHfjpEIIRj2svycwoBqB/QiMeVJ4WGEYV4IzrFZKQQeV1 wyKFGHWV6cf1230AnZkIENV8vTgQ5hiw2yKu9VhO4wU4SnQsY1/6mRwNAcdnIsRbHhgk XZqkLkPryn6vAPAL6qLQiWATA+fDrRwZTtpceR68fjrPGmRoewG4qjG8FbTodFO8ISs+ fsOA== MIME-Version: 1.0 X-Received: by 10.58.207.195 with SMTP id ly3mr10996164vec.77.1372762813794; Tue, 02 Jul 2013 04:00:13 -0700 (PDT) Received: by 10.58.45.162 with HTTP; Tue, 2 Jul 2013 04:00:13 -0700 (PDT) In-Reply-To: References: Date: Tue, 2 Jul 2013 07:00:13 -0400 Message-ID: Subject: Re: IDMS : Weekly status report #2 of 14 From: Ambarisha B To: David Chisnall Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: soc-status@freebsd.org X-BeenThere: soc-status@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Summer of Code Status Reports and Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jul 2013 11:00:14 -0000 On Tue, Jul 2, 2013 at 5:23 AM, David Chisnall wrote: > On 1 Jul 2013, at 19:27, Ambarisha B wrote: > > > 3. Refactor the client to give an open fd to the server (instead of > > expecting one, as is the model with libfetch) > > Can you explain a bit what this means? The daemon should be the thing > openning the connection. The forked worker should then pass it to > libfetch. > fd isn't the connection, it is the local file. libfetch doesn't take an open local file descriptor. It returns the remote file stream wrapped as an fd. The client (fectch program) expects this fd for cryptographic hash verification etc and when it is done writes the data in that fd into a local file. This doesn't work for us because that has to be done in the daemon. I was talking about refactoring that into the daemon and the client passes the fd of the local file to the daemon. > struct client_request req; > > int IPC_connection_to_clients = /* whatever you're using here, most likely > a UNIX domain socket so that you can pass in the fd for the output file > from the client. */; > > while (wait_for_request(IPC_connection_to_clients, &req) { > /* URL parsing code goes here. */ > int fd server = connect(...); > /* Error handling goes here */ > > /* You may want to create a pipe here for sending progress > messages from the client back to the server */ > > /* You may want to queue requests and not start downloading > immediately if some bandwidth or number of concurrent downloads quota is > reached */ > > /* Create the worker */ > pid_t child = fork(); > /* Error handling here */ > > if (child == 0) > { > /* Now we're in the worker, close any file descriptors > that we shouldn't have */ > close(IPC_connection_to_clients); > close(stdin); > close(stdout); > /* Then enter sandboxed mode */ > cap_enter(); > /* Now do the real downloading and exit */ > do_the_real_downloading(server, req.output_file, req.url); > exit(0); > } > else > { > /* Close the file descriptors that the worker is using. */ > close(server); > close(req.output_file); > /* Probably add child to a list of things that you'll wait > for exit signals from in the kqueue call in the wait_for_request part */ > } > } > > Yes this is what I'm thinking as well. For this, libfetch has to be modified to accept a connection. Right now, it takes a url and returns the stream containing the remote file wrapped as an fd. Opening the connection is abstracted in libfetch. Cheers Ambarish