From owner-freebsd-current Sun Dec 15 9:28:22 2002 Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 933A737B401 for ; Sun, 15 Dec 2002 09:28:18 -0800 (PST) Received: from mail.lolotte.org (klemm.delta6.net [80.65.226.82]) by mx1.FreeBSD.org (Postfix) with ESMTP id E8C8C43ED1 for ; Sun, 15 Dec 2002 09:28:16 -0800 (PST) (envelope-from clement@klemm.delta6.net) Received: from klemm.delta6.net (shells.mouarf.org. [192.168.2.101]) by mail.lolotte.org (8.12.3/8.12.5) with ESMTP id gBFHSEtv095403 for ; Sun, 15 Dec 2002 18:28:14 +0100 (CET) (envelope-from clement@klemm.delta6.net) Received: from shells.mouarf.org (shell.mouarf.org [192.168.2.101] (may be forged)) by klemm.delta6.net (8.12.3/8.12.5) with ESMTP id gBFHSEH8095400 for ; Sun, 15 Dec 2002 18:28:14 +0100 (CET) (envelope-from clement@shells.mouarf.org) Received: (from clement@localhost) by shells.mouarf.org (8.12.3/8.12.5/Submit) id gBFHSE6r095399 for freebsd-current@freebsd.org; Sun, 15 Dec 2002 18:28:14 +0100 (CET) Date: Sun, 15 Dec 2002 18:28:14 +0100 From: clement To: freebsd-current@freebsd.org Subject: current&stable jail problem Message-ID: <20021215182814.C95269@shells.mouarf.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi, I've got a problem with jail(2), that affects both -STABLE and -CURRENT (Tested on a 4.7-RELEASE and on a CURRENT updated the 12/14/2002, both on i386) I've found a reference to this problem in PR kern/26506, but it's 1,5years old and it's still not fixed, and I think I've also found a problem in the patch provided with the PR (see the explanation of this 2nd problem just before my patch) I) Description of the original problem In a jail, when you open an UDP socket, then use sendto() several times to send UDP packets, only the first sendto() works, the others fail with EINVAL. The problem occurs only when you don't bind() the socket before sendto() (and therefore let the kernel set the source port) Here is a sample program to demonstrate this. It will work fine outside jail(2) but it will fail with err msg "sendto: Invalid argument" inside a jail. -------------------------------------------- #include #include #include #include #include #include int main(void) { struct sockaddr_in pom; int s,r; s = socket(PF_INET, SOCK_DGRAM, 0); pom.sin_family = AF_INET; pom.sin_addr.s_addr = inet_addr("123.123.123.123"); pom.sin_port = htons(4242); sendto(s, "test", 4, 0, (struct sockaddr *) &pom, sizeof(pom)); r = sendto(s, "test", 4, 0, (struct sockaddr *) &pom, sizeof(pom)); if (r == -1) { perror("sendto"); close(s); exit(1); } printf("All is OK\n"); close(s); exit(0); } -------------------------------------------- I've attempted to find out why this problem occurs. (These investigations were done on a 4.7-RELEASE, with the 4.7-RELEASE source, because I didn't have a box with -CURRENT available, but the bug exists also on -CURRENT) : sendto() will eventually call udp_output() (sys/netinet/udp_usrreq.c) udp_output() will call in_pcbconnect() If we are in a jail, and the socket's local IP is INADDR_ANY, in_pcbconnect() will call in_pcbbind() to assign the prison IP address (and a localport between 1024 and 5000) to our not-yet-bound socket. in_pcbconnect() returns, the packet is sent, etc... Then this bit of code in udp_output() resets inp->inp_laddr to its old value (which, in our case, is INADDR_ANY): ---- if (addr) { in_pcbdisconnect(inp); inp->inp_laddr = laddr; /* XXX rehash? */ splx(s); } ---- So when sendto() returns, the local IP of the socket is set to INADDR_ANY, and the local port is set, between 1024 and 5000. When you call the next sendto(), the following thing occurs: Because we are in a jail and the socket's local IP is still to INADDR_ANY, in_pcbconnect() calls in_pcbbind(), and in_pcbbind() fails because the local port IS set: ---- if (inp->inp_lport || inp->inp_laddr.s_addr != INADDR_ANY) return (EINVAL); ---- This does not occur outside jail(2) because when the local port is already set, in_pcbbind() is not called by in_pcbconnect, and the local IP is set by another way, later in in_pcbconnect(): ---- if (inp->inp_laddr.s_addr == INADDR_ANY) { if (inp->inp_lport == 0) { error = in_pcbbind(inp, (struct sockaddr *)0, p); if (error) return (error); } inp->inp_laddr = ifaddr->sin_addr; } ---- I've attempted to fix the problem by modifying in_pcbconnect(), so it will not call in_pcbbind() anymore if we're in a jail and local IP is not set. Instead it will check with ifa_ifwithaddr() if the prison IP address is available, and assigning it to inp->inp_laddr (the same way we do outside a jail) II) Description of the problem that the PR kern/26506 patch introduces: The patch provided in the PR kern/26506 has a problem: If the prison IP is an alias for an interface, the sendto() sends the packets coming from the primary IP of the interface, and not the alias. III) My "attempt" at solving the problem :) I'm not very experienced with FreeBSD kernel programming, so I'm not sure that's the right way to fix it, but anyway here's the patch for sys/netinet/in_pcb.c: (tested on 4.7-RELEASE) --------------------------------------------------- --- in_pcb.c.old Sun Dec 15 00:09:00 2002 +++ in_pcb.c Sun Dec 15 12:56:27 2002 @@ -505,9 +505,10 @@ sa.sin_addr.s_addr = htonl(p->p_prison->pr_ip); sa.sin_len=sizeof (sa); sa.sin_family = AF_INET; - error = in_pcbbind(inp, (struct sockaddr *)&sa, p); - if (error) - return (error); + if (TAILQ_EMPTY(&in_ifaddrhead)) /* XXX same as in_pcbbind() */ + return (EADDRNOTAVAIL); + if (ifa_ifwithaddr((struct sockaddr *) &sa) == 0) + return (EADDRNOTAVAIL); } /* * Call inner routine, to assign local interface address. @@ -526,7 +527,11 @@ if (error) return (error); } - inp->inp_laddr = ifaddr->sin_addr; + if (p->p_prison == NULL) { + inp->inp_laddr = ifaddr->sin_addr; + } else { + inp->inp_laddr = sa.sin_addr; + } } inp->inp_faddr = sin->sin_addr; inp->inp_fport = sin->sin_port; --------------------------------------------------- I was not sure if I needed to post this on stable@ or on current@ but I posted on current@ since -CURRENT is affected as well. regards, Clement Ballabriga To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message