From owner-freebsd-net@freebsd.org Wed Mar 1 18:06:56 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7BD47CF3883 for ; Wed, 1 Mar 2017 18:06:56 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-yw0-x22d.google.com (mail-yw0-x22d.google.com [IPv6:2607:f8b0:4002:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3B93CAB4; Wed, 1 Mar 2017 18:06:56 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-yw0-x22d.google.com with SMTP id d1so39071194ywd.2; Wed, 01 Mar 2017 10:06:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=k3BnT/guWzDPv8fW3GiqZ0+zt9x/cmhGpUAh053odtw=; b=LKBM0nCfPRlxV0ojfgYAlyfgkdC0sV5XeQhZeneuv3cdE+1R6wQX3/wx//y7tSVflD 8apMNE8hu/YjrwR62cDOhzbl7RnEpeqQoYPrhI6HXVDS+x8rRGahBsY9uKNzgGxO51R0 /bll5+7E5416guLdnoFKfvQdc9KvRrZrSgbOmKKUaHZLSA/bpXdRcucPMYZivggMXVIl 0umMdASivJpq5TZxdd0cSi1/Yrv9djWEJ5dDZKJjcWVPchFya3o5rk73UGJ9xRp4B0aT MbpJW3HIvPCOh6P3n6R6o8GzlNpIdduzWJ6CUVXTWnOU++urxClouzhLBpJZbsR1evJ0 tcWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=k3BnT/guWzDPv8fW3GiqZ0+zt9x/cmhGpUAh053odtw=; b=OGmsLvaTrzPSj20HslFmtE25tfoRny0eWCfet/2xxhP1X7jILD/GnOHV8Uc0yrR2xL qLhmiTcZLFpym1UqLAwQoKfX/2j+nTe7FZkq6aKaYJ7odBaRsxbEJ1sZCNpXc1XD06DH TM3dpevlxj/yQ9zycbDpYyXTGjGFh2oKCC4iyWE1PP9hq6+N1ar91kqIWL8CRrdIv2xe b5Bp1wnn5XUxjsOUjs7fF6x/X5elxxsBKAREmh0H2h2/8c8rPGmeHl3WwgRf523T3Lty R16Ebje2PJNAtR2cGOiVPBE7IuHWBpxR5dro0v/4hfrow2rR2lXFxqn9AKoUNkq1CEsp I2eQ== X-Gm-Message-State: AMke39mbr0HM3aM1d3YNtFRW+g3wgD+wFJFczTBQUAWR+xLIzt0/oZdt8bZYMLRkHpaelgBM5zOGjXQ37aHyFA== X-Received: by 10.129.182.79 with SMTP id h15mr3346265ywk.253.1488391615290; Wed, 01 Mar 2017 10:06:55 -0800 (PST) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.129.38.133 with HTTP; Wed, 1 Mar 2017 10:06:54 -0800 (PST) In-Reply-To: <4dada28e-2130-70f3-e4b1-9d20f4185437@freebsd.org> References: <4dada28e-2130-70f3-e4b1-9d20f4185437@freebsd.org> From: Alan Somers Date: Wed, 1 Mar 2017 11:06:54 -0700 X-Google-Sender-Auth: PWz5UCuj2GF1G1P0UYBypeiu4to Message-ID: Subject: Re: VNET / netgraph jails -- Locking down? To: Julian Elischer Cc: Jeff Kletsky , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Mar 2017 18:06:56 -0000 I do something similar, but I rely entirely on vnet and PF instead of netgraph. My host has two ethernet ports, so I use one for the host and one for all of the jails. That makes the pf setup easier. I use iocage to configure an ordinary vnet jail, bridged to the host's second ethernet port. Then I configure pf.conf like this. This ensures that the only traffic allowed into the jail is that destined for its intended service, and the only traffic allowed out are responses to inbound traffic. The jail is able to change its own IP address, but if it does then PF will block all of its traffic. www_services = "{ http, https, 8080 }" host_iface = "em0" dmz_iface = "em1" www_jail_iface = "vnet0:1" www_ip = "192.168.0.40" set state-policy if-bound scrub in block in all block out all pass in on $host_iface pass out on $host_iface set skip on lo0 # Allow all traffic to the DMZ. Filtering happens on individual vnet # interfaces pass in on $dmz_iface pass out on $dmz_iface # Put the www jail in a DMZ. Don't allow outgoing traffic from it except for # the webserver pass out on $www_jail_iface proto tcp to $www_ip port $www_services keep state -Alan On Wed, Mar 1, 2017 at 10:36 AM, Julian Elischer wrote: > many good questions but looking at what you are doing, maybe we should be > asking you the questions. > Certainly firewalling on the outside of the jail makes sense. I've not used > ng_ipfw but it would make sense to do a quick santity check for every packet > leaving each jail. > > > > On 14/2/17 9:47 am, Jeff Kletsky wrote: >> >> For several years I've been using netgraph to provide connectivity for >> "service hosts" in jails on a "jail server" >> >> Since I'm finally getting the jail server off FreeBSD 9 and solidly onto >> 11, I've got the chance to rewrite the scripting of how I'm handling jail >> connectivity and am hoping that I can lock things down a bit better than >> what I have presently. >> >> >> The approach I use looks similar to that now in the jail examples. >> Basically >> >> /---> ng_eiface_jail1 >> real_interface = ng_ether <---> ng_bridge <---> ng_eiface_jail2 >> \---> ng_eiface_jail3 >> >> While this works well, it concerns me that the real interface has to be in >> promiscuous mode (and have autosrc off). >> >> If one of the service jails is "taken over" then there isn't a way that I >> know of to lock out changing the IP address of the interface it has, or >> potentially gaining access to another VLAN through creation of a cloned >> interface, especially if the bridge is off the parent interface, not off a >> VLAN interface. >> >> >> How do people manage this in practice when the jail has the risk of >> compromise? >> >> >> I prefer approaches where the jail's notion of it's own IP address is the >> same as that of other hosts connecting to it, at least within my own little >> private-address-space world. >> >> >> One thing that I've been considering is: >> * Configure the jail's IP on the real interface (or appropriate VLAN >> interface) as an alias >> * Send packets through ng_ipfw to an ng_eiface that the jail gets, using >> ipfw and a lookup table >> * Tag the packets on return with ng_tag with a unique identifier for that >> jail's interface so ipfw can tell the only acceptable source IP >> * Deny any so-tagged packets that don't have the proper source address >> >> (jail ID by itself is not enough for the outbound packets, as some of the >> jails are dual homed.) >> >> >> Has anyone tried this kind of method? Any other/better suggestions? >> >> >> Would ng_ip_input be the appropriate way to "send" the packets coming from >> the jail? >> >> >> Thanks! >> >> >> Jeff >> >> >> >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"