From owner-freebsd-jail@FreeBSD.ORG Sat Jul 12 03:40:13 2014 Return-Path: Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EDB90A6B for ; Sat, 12 Jul 2014 03:40:12 +0000 (UTC) Received: from mail-ig0-x233.google.com (mail-ig0-x233.google.com [IPv6:2607:f8b0:4001:c05::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B7C16274D for ; Sat, 12 Jul 2014 03:40:12 +0000 (UTC) Received: by mail-ig0-f179.google.com with SMTP id h18so122128igc.6 for ; Fri, 11 Jul 2014 20:40:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=N5HpnThpa4VbFPglzoRu5Y2lD8hXwT8UOBGmimJPQX8=; b=AT+Wm5ncX/ReCQTb0/436baR0gnLxRkhl/hcMzTui0n/P8fsO7WfmNVvyIRlr9EP3o 98A78Rjlh9XvKfKjINPHo+EqAbqp98K1sttqE9Dm1jOUEk8pgZBCiHY68TmhccrsdsRz x7L3sknZXph9m/6/7XtQuALr7vSM+siLrddCjBwsdIQxEQhBX9mlqe4iC7zL6nq+mFTy h4bv+41XCp6jYiC/h8EEFy2DXdvjRKjIR1AhqcUnFlVvWSiKHEy8dXvtFzT75nOKyXuH jFLamyz0gkfIqSf2phvzQ5FJTv6cR8zv/5BxbmSOpCsiDAI+EU+oUZFoHn+Qx3BT2Acu A/vg== MIME-Version: 1.0 X-Received: by 10.50.114.226 with SMTP id jj2mr9523174igb.27.1405136410652; Fri, 11 Jul 2014 20:40:10 -0700 (PDT) Received: by 10.43.59.6 with HTTP; Fri, 11 Jul 2014 20:40:10 -0700 (PDT) In-Reply-To: <53C08C74.6000805@a1poweruser.com> References: <53BFE796.7020502@a1poweruser.com> <53C08C74.6000805@a1poweruser.com> Date: Sat, 12 Jul 2014 15:40:10 +1200 Message-ID: Subject: Re: vnet jail and ipfw/nat on host - keep-state problem? From: Peter Toth To: Fbsd8 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Peter Ross , freebsd-jail@freebsd.org X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jul 2014 03:40:13 -0000 Dear Joe Barbish (alias fbsd8@a1poweruser.com), When you going to stop trolling the FreeBSD mailing list and spread disinformation? For anyone interested please check this mail thread on who fbsd8 really is: http://lists.freebsd.org/pipermail/freebsd-jail//2013-March/002147.html Very telling isn't it! People come to this place to learn, share information, help out other folks and most importantly to have a constructive debate! (obviously some would rather divert this effort) The PR number's mentioned are mostly outdated from the 8.x and 9.x series - some of them are completely irrelevant (like ACPI) or for a i386 system. Beyond this I am categorically refusing to waste any energy and time on answering any trolling/diversion attempts by Joe Barbish. Most importantly I encourage anyone avoiding his dubious Qjail project by far - for details please check the link above. I am not going to burn time on dissecting each PR one-by-one but rather share my experience with VNET. Over the last year and a half have deployed numerous production systems based on amd64 10-RELEASE with VNET enabled and PF running on the host. Encountered 0 instability issues! Details on how to do this are here: http://iocage.readthedocs.org/en/latest/real-world.html As I mentioned before IPFW works in a jail and PF only works on the host. Back to the original issue though, Peter could you please share your IPFW config with me (maybe just send it directly to me), would be very interested to get it going in my lab setup and add a howto page to share this with others. Cheers, Peter On Sat, Jul 12, 2014 at 1:16 PM, Fbsd8 wrote: > Peter Toth wrote: > > On Sat, Jul 12, 2014 at 1:33 AM, Fbsd8 > fbsd8@a1poweruser.com>> wrote: >> >> Peter Toth wrote: >> >> Have not used natd with IPFW much as always preferred PF to do >> everything >> on the host. >> >> I have only a wild guess - the "me" keyword in IPFW is >> substituted only to >> the host's IPs known to itself. >> The host's IPFW firewall most likely doesn't know anything about >> IPs >> assigned to vnet interfaces inside the jail. >> >> Vnet jails behave more like separate physical hosts. >> >> Internet ---> [host] ------- (10.0.10.0 LAN) ------> [vnet jail] >> >> The PF issue inside a jail is a separate problem, PF is not fully >> VIMAGE/VNET aware as far as I know. >> >> Can someone comment on these or correct me? >> >> P >> >> >> >> On Fri, Jul 11, 2014 at 7:11 PM, Peter Ross >> > > >> >> wrote: >> >> On Thu, 10 Jul 2014, Peter Toth wrote: >> >> Hi Peter, >> >> Try to make these changes: >> >> net.inet.ip.forwarding=1 # Enable IP forwarding >> between interfaces >> net.link.bridge.pfil_onlyip=0 # Only pass IP packets >> when pfil is enabled >> net.link.bridge.pfil_bridge=0 # Packet filter on the >> bridge interface >> net.link.bridge.pfil_member=0 # Packet filter on the >> member interface >> >> You can find some info >> here >> http://iocage.readthedocs.org/ >> __en/latest/help-no-internet.__html >> >> > internet.html> >> >> I've had these issues before with PF and IPFW, by >> default these will be >> filtering on your bridge and member interfaces. >> >> Thanks. It did not change anything. >> >> Now, inside_ the jail I run "ipfw allow ip from any to any". >> >> This on the host system: >> >> 01000 check-state >> 01100 allow tcp from any to any established >> 01200 allow ip from any to any frag >> 00100 divert 8668 ip4 from any to any via age0 >> 03100 allow udp from any to 10.0.10.1 dst-port 53 keep-state >> 03200 allow udp from any to me dst-port 53 keep-state >> >> (with natd redirecting "redirect_port udp 10.0.10.1:53 >> external.ip:53") >> >> >> If I add >> >> 03300 allow udp from me 53 to any >> >> it works.. >> >> So it makes me think check-state isn't usable - because >> >> 03200 allow udp from any to me dst-port 53 keep-state >> >> should cover the returning packets. >> >> I played with your parameters but it did not help. But >> thanks for the idea. >> >> Here again the setup: >> >> Internet->age0(host interface with natd and external IP) >> ->bridge10(10.0.10.254)->__epair1a >> >> ->epair1b(10.0.10.1 in bind vnet jail) >> >> I wonder what kind of restrictions exist with vnet.. it does >> not seem to >> work _exactly_ as a "real" network stack (the issues with pf >> inside the >> jail let me think of it too) >> >> Did I find a restriction, a bug - or just that I've got it >> wrong? >> >> Regards >> Peter >> >> >> Any firewall function that runs in the kernel will not function >> inside of a vnet/vimage jail. >> >> >> >> This sounds a bit vague, can you please explain in more detail what you >> meant by this? >> >> IPFW works inside a vnet jail - You can manage per jail firewall >> instances without any issues. >> >> The only firewall which cannot function inside a jail (yet) is PF. >> >> P >> >> >> > You are incorrect. > Here is a list of some of the vnet/vimage outstanding PR's > > 143808, 147950, 148155, 152148, 160496, 160541, 161094, 164763, 165252, > 176112, 176929, 178480, 178482, 179264, 182350, 185092, 188010, 191468 > > > > > > >