From owner-freebsd-jail@freebsd.org Mon May 16 12:55:05 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BDAB4B3C9F7 for ; Mon, 16 May 2016 12:55:05 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from msa1.earth.yoonka.com (yoonka.com [185.24.122.233]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "msa1.earth.yoonka.com", Issuer "msa1.earth.yoonka.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6072D17F5 for ; Mon, 16 May 2016 12:55:04 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from crayon2.yoonka.com (crayon2.yoonka.com [192.168.1.20]) (authenticated bits=0) by msa1.earth.yoonka.com (8.15.2/8.15.2) with ESMTPSA id u4GCt2Ti004209 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Mon, 16 May 2016 12:55:02 GMT (envelope-from list1@gjunka.com) To: freebsd-jail@freebsd.org From: Grzegorz Junka Subject: Unresponsive jails issues Message-ID: <6beab349-73bb-7159-cd81-443e115b687a@gjunka.com> Date: Mon, 16 May 2016 12:55:02 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 May 2016 12:55:05 -0000 I have a server running 13 jails for various system services. Recently I added two jails to run simple go applications for testing. They open a network socket and nginx, which is in another jail, and which round robin balances requests to them. I mention that because it may be related, however not necessarily because it was happening earlier. The problem is that every 2-3 days jails in my servers stop responding. "jexec jailname tcsh" hangs forever, "service jail stop jailname" hangs forever as well. "top" doesn't show anything suspicious. I can login through SSH to the main server fine. I don't login to jails through SSH so I can't check but it seems that when that happens they stop responding because the services that are running in them stop too (e.g. web server, imap, ...). I tried to "kill -9" the "jexec" process that hangs but that doesn't work. My first question is what evidence should I gather when that happens so that I can investigate the issue later on after the server is restarted? And the second question, any idea why that might be happening in the first place? I am running FreeBSD 10.3 AMD64 updated from 10.2 a couple of weeks ago. Grzegorz From owner-freebsd-jail@freebsd.org Mon May 16 13:08:44 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1EFD7B3CF5B for ; Mon, 16 May 2016 13:08:44 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D9ACF1239 for ; Mon, 16 May 2016 13:08:43 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id E01E725D37C7; Mon, 16 May 2016 13:08:40 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 3F359D1F8A7; Mon, 16 May 2016 13:08:40 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id RbPs8hP7kxxL; Mon, 16 May 2016 13:08:38 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4410:50f4:9329:a995:3c2a] (unknown [IPv6:fde9:577b:c1a9:4410:50f4:9329:a995:3c2a]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 3627DD1F8A5; Mon, 16 May 2016 13:08:37 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: Unresponsive jails issues From: "Bjoern A. Zeeb" In-Reply-To: <6beab349-73bb-7159-cd81-443e115b687a@gjunka.com> Date: Mon, 16 May 2016 13:08:18 +0000 Cc: freebsd-jail@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <7ACDBC85-5B17-4695-8DAD-BCC48817EEBF@lists.zabbadoz.net> References: <6beab349-73bb-7159-cd81-443e115b687a@gjunka.com> To: Grzegorz Junka X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 May 2016 13:08:44 -0000 > On 16 May 2016, at 12:55 , Grzegorz Junka wrote: >=20 > I have a server running 13 jails for various system services. Recently = I added two jails to run simple go applications for testing. They open a = network socket and nginx, which is in another jail, and which round = robin balances requests to them. I mention that because it may be = related, however not necessarily because it was happening earlier. >=20 > The problem is that every 2-3 days jails in my servers stop = responding. "jexec jailname tcsh" hangs forever, "service jail stop = jailname" hangs forever as well. "top" doesn't show anything suspicious. = I can login through SSH to the main server fine. I don't login to jails = through SSH so I can't check but it seems that when that happens they = stop responding because the services that are running in them stop too = (e.g. web server, imap, ...). I tried to "kill -9" the "jexec" process = that hangs but that doesn't work. >=20 > My first question is what evidence should I gather when that happens = so that I can investigate the issue later on after the server is = restarted? >=20 > And the second question, any idea why that might be happening in the = first place? >=20 > I am running FreeBSD 10.3 AMD64 updated from 10.2 a couple of weeks = ago. If you can log into the base system and issue commands there; try to = see what procstat (-k) thinks about various jailed processes. You could = also check ps axl for the WCHAN and see if anything suspicious shows up. /bz From owner-freebsd-jail@freebsd.org Mon May 16 13:29:41 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7A9F3B3D7A8 for ; Mon, 16 May 2016 13:29:41 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from msa1.earth.yoonka.com (yoonka.com [185.24.122.233]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "msa1.earth.yoonka.com", Issuer "msa1.earth.yoonka.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 17AC7123C for ; Mon, 16 May 2016 13:29:40 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from crayon2.yoonka.com (crayon2.yoonka.com [192.168.1.20]) (authenticated bits=0) by msa1.earth.yoonka.com (8.15.2/8.15.2) with ESMTPSA id u4GDTc88004786 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Mon, 16 May 2016 13:29:38 GMT (envelope-from list1@gjunka.com) To: freebsd-jail@freebsd.org From: Grzegorz Junka Subject: Jails and unionfs Message-ID: <63df6e06-ebac-fe44-ca70-5d2a9e78967b@gjunka.com> Date: Mon, 16 May 2016 13:29:38 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 May 2016 13:29:41 -0000 I have been using unionfs to host jails for quite a while now and in general they work as expected, apart from three issues. The setup is as below (example for one jail dev2): _*jail.conf*_ exec.start = "/bin/sh /etc/rc"; exec.stop = "/bin/sh /etc/rc.shutdown"; exec.clean; mount.devfs; mount.fstab = "/usr/local/etc/fstab/$name"; devfs_ruleset = 4; path = "/j/$name"; host.hostname = "$name.*myhost*.*mydomain*.com"; exec.consolelog = "/var/log/jail/$name"; dev2 { ip4.addr = 192.168.1.71; interface = lagg0; } _*/usr/local/etc/fstab/dev2*_ /j/_ro3 /j/dev2 nullfs ro 0 0 /j/_dev2 /j/dev2 unionfs rw,noatime 0 0 devfs /j/dev2/dev devfs rw,ruleset=4 0 0 _*df gives*_ tank1/j/_dev2 1198584120 131255 1198452864 0% /j/_dev2 /j/_ro3 1198722545 269680 1198452864 0% /j/dev2 :/j/_dev2 2397306665 1198853800 1198452864 50% /j/dev2 devfs 1 1 0 100% /j/dev2/dev devfs 1 1 0 100% /j/dev2/dev _*zfs list | grep dev2*_ tank1/j/_dev2 128M 1.12T 128M /j/_dev2 As can be seen I need to mount devfs twice, once in jail.conf and once in the jail's fstab, otherwise it isn't mounted at all. That's the first (smaller) issue. The second issue is that the disks are not mounted/unmounted automatically when I start/stop the jail. To make sure that all disks are mounted properly after starting a jail I need to: mount -F /usr/local/etc/fstab/dev2 -a When stopping the jail sometimes the disks are unmounted but sometimes I have to: umount -F /usr/local/etc/fstab/dev2 -a But the third, most annoying issue is that if I forget to unmount all disks after stopping a jail and then I start the jail, the unionfs is mounted twice. Once that happens and I need to stop the jail, unmounting disks for that jail causes kernel panic. Does anyone have experience with that setup? Are those issues known and are there any possible fixes or workarounds? Grzegorz From owner-freebsd-jail@freebsd.org Mon May 16 14:58:21 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46C7AB3D2D1 for ; Mon, 16 May 2016 14:58:21 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) by mx1.freebsd.org (Postfix) with ESMTP id 2D1BC1B66 for ; Mon, 16 May 2016 14:58:20 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (unknown [10.1.1.2]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 67E8DD759 for ; Mon, 16 May 2016 14:58:19 +0000 (UTC) Subject: Re: Unresponsive jails issues To: freebsd-jail@freebsd.org References: <6beab349-73bb-7159-cd81-443e115b687a@gjunka.com> From: Allan Jude Message-ID: <5739E00B.7090100@freebsd.org> Date: Mon, 16 May 2016 10:58:19 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: <6beab349-73bb-7159-cd81-443e115b687a@gjunka.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 May 2016 14:58:21 -0000 On 2016-05-16 08:55, Grzegorz Junka wrote: > I have a server running 13 jails for various system services. Recently I > added two jails to run simple go applications for testing. They open a > network socket and nginx, which is in another jail, and which round > robin balances requests to them. I mention that because it may be > related, however not necessarily because it was happening earlier. > > The problem is that every 2-3 days jails in my servers stop responding. > "jexec jailname tcsh" hangs forever, "service jail stop jailname" hangs > forever as well. "top" doesn't show anything suspicious. I can login > through SSH to the main server fine. I don't login to jails through SSH > so I can't check but it seems that when that happens they stop > responding because the services that are running in them stop too (e.g. > web server, imap, ...). I tried to "kill -9" the "jexec" process that > hangs but that doesn't work. > > My first question is what evidence should I gather when that happens so > that I can investigate the issue later on after the server is restarted? > > And the second question, any idea why that might be happening in the > first place? > > I am running FreeBSD 10.3 AMD64 updated from 10.2 a couple of weeks ago. > > Grzegorz > > _______________________________________________ > freebsd-jail@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-jail > To unsubscribe, send any mail to "freebsd-jail-unsubscribe@freebsd.org" When you issue the jexec and it hangs, try pressing 'control+t' to see what the waitchan is. Along with what Bjoern said, using procstat -k to examine other processes etc. -- Allan Jude From owner-freebsd-jail@freebsd.org Wed May 18 14:00:04 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 16416B3F34B for ; Wed, 18 May 2016 14:00:04 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from msa1.earth.yoonka.com (yoonka.com [185.24.122.233]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "msa1.earth.yoonka.com", Issuer "msa1.earth.yoonka.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B65C81260 for ; Wed, 18 May 2016 14:00:03 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from crayon2.yoonka.com (crayon2.yoonka.com [192.168.1.20]) (authenticated bits=0) by msa1.earth.yoonka.com (8.15.2/8.15.2) with ESMTPSA id u4IE01Wb010854 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Wed, 18 May 2016 14:00:01 GMT (envelope-from list1@gjunka.com) To: freebsd-jail@freebsd.org From: Grzegorz Junka Subject: jails in different private subnets on the same host Message-ID: Date: Wed, 18 May 2016 14:00:01 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2016 14:00:04 -0000 Is it possible to have two jails on the same host each one in a different private subnet, e.g. 192.168.1.0 and 10.33.1.0, and have routing between them working without issues? I know it's possible to run jails with IPs in those two subnets but it seems there is no routing and I am not sure if it's because I can't configure my router properly or there is a more fundamental problem. One issue I see is that the jail can't have a different default gateway than the host, and that for now is 192.168.1.1, but I don't see a reason why 10.33.1.0 wouldn't be able to use 192.168.1.1 as it's default gateway provided there is routing between those two subnets. Grzegorz From owner-freebsd-jail@freebsd.org Wed May 18 14:12:18 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0E0DB3F79F for ; Wed, 18 May 2016 14:12:18 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 655781E55 for ; Wed, 18 May 2016 14:12:18 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 5CCF625D389C; Wed, 18 May 2016 14:12:15 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 54B57D1F8BA; Wed, 18 May 2016 14:12:14 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id 4htiZQllDf9n; Wed, 18 May 2016 14:12:12 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4410:10a6:9e53:5e9c:c191] (unknown [IPv6:fde9:577b:c1a9:4410:10a6:9e53:5e9c:c191]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 5D085D1F8A6; Wed, 18 May 2016 14:12:12 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: jails in different private subnets on the same host From: "Bjoern A. Zeeb" In-Reply-To: Date: Wed, 18 May 2016 14:11:51 +0000 Cc: freebsd-jail@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Grzegorz Junka X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2016 14:12:18 -0000 > On 18 May 2016, at 14:00 , Grzegorz Junka wrote: >=20 > Is it possible to have two jails on the same host each one in a = different private subnet, e.g. 192.168.1.0 and 10.33.1.0, and have = routing between them working without issues? >=20 > I know it's possible to run jails with IPs in those two subnets but it = seems there is no routing and I am not sure if it's because I can't = configure my router properly or there is a more fundamental problem. One = issue I see is that the jail can't have a different default gateway than = the host, and that for now is 192.168.1.1, but I don't see a reason why = 10.33.1.0 wouldn't be able to use 192.168.1.1 as it's default gateway = provided there is routing between those two subnets. Given they are both on the same base system host, both addresses are = connected locally and thus the kernel knows where to deliver these = packets. If that doesn=E2=80=99t work, there is a bug somewhere. If you want different default gateways then you may want to look into = using different FIBs for different jails. See route(8) and jail(8) for = parameters to set and tune. /bz From owner-freebsd-jail@freebsd.org Wed May 18 15:08:10 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3BE46B41711 for ; Wed, 18 May 2016 15:08:10 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from msa1.earth.yoonka.com (yoonka.com [185.24.122.233]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "msa1.earth.yoonka.com", Issuer "msa1.earth.yoonka.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E68AA1D8C for ; Wed, 18 May 2016 15:08:09 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from crayon2.yoonka.com (crayon2.yoonka.com [192.168.1.20]) (authenticated bits=0) by msa1.earth.yoonka.com (8.15.2/8.15.2) with ESMTPSA id u4IF87RJ012284 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Wed, 18 May 2016 15:08:07 GMT (envelope-from list1@gjunka.com) Subject: Re: jails in different private subnets on the same host References: From: Grzegorz Junka To: freebsd-jail@freebsd.org Message-ID: <07d67bd5-206c-edd8-7f47-ef2b5c538e01@gjunka.com> Date: Wed, 18 May 2016 15:08:07 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2016 15:08:10 -0000 On 18/05/2016 14:11, Bjoern A. Zeeb wrote: > >> On 18 May 2016, at 14:00 , Grzegorz Junka >> wrote: >> >> Is it possible to have two jails on the same host each one in a >> different private subnet, e.g. 192.168.1.0 and 10.33.1.0, and have >> routing between them working without issues? >> >> I know it's possible to run jails with IPs in those two subnets >> but it seems there is no routing and I am not sure if it's because >> I can't configure my router properly or there is a more >> fundamental problem. One issue I see is that the jail can't have a >> different default gateway than the host, and that for now is >> 192.168.1.1, but I don't see a reason why 10.33.1.0 wouldn't be >> able to use 192.168.1.1 as it's default gateway provided there is >> routing between those two subnets. > > Given they are both on the same base system host, both addresses > are connected locally and thus the kernel knows where to deliver > these packets. If that doesn’t work, there is a bug somewhere. > > If you want different default gateways then you may want to look > into using different FIBs for different jails. See route(8) and > jail(8) for parameters to set and tune. > > /bz > I can ping both jails from the main host, however when in the 10.33.1.0 jail I can't access any jail in the 192.168.1.0 network. This is what netstat -r shows: --------------------------------- root@dns1:/ # ifconfig em0: flags=8843 metric 0 mtu 1500 options=4219b ether 00:25:90:ae:e8:bc media: Ethernet autoselect (1000baseT ) status: active em1: flags=8843 metric 0 mtu 1500 options=4219b ether 00:25:90:ae:e8:bc media: Ethernet autoselect (1000baseT ) status: active lo0: flags=8049 metric 0 mtu 16384 options=600003 lagg0: flags=8843 metric 0 mtu 1500 options=4219b ether 00:25:90:ae:e8:bc inet 192.168.1.60 netmask 0xffffffff broadcast 192.168.1.60 media: Ethernet autoselect status: active laggproto lacp lagghash l2,l3,l4 laggport: em0 flags=1c laggport: em1 flags=1c root@dns1:/ # netstat -r Routing tables Internet: Destination Gateway Flags Netif Expire dns1 link#4 UHS lo0 --------------------------------- root@pjp1:/ # ifconfig em0: flags=8843 metric 0 mtu 1500 options=4219b ether 00:25:90:ae:e8:bc media: Ethernet autoselect (1000baseT ) status: active em1: flags=8843 metric 0 mtu 1500 options=4219b ether 00:25:90:ae:e8:bc media: Ethernet autoselect (1000baseT ) status: active lo0: flags=8049 metric 0 mtu 16384 options=600003 lagg0: flags=8843 metric 0 mtu 1500 options=4219b ether 00:25:90:ae:e8:bc inet 10.33.1.40 netmask 0xffffffff broadcast 10.33.1.40 media: Ethernet autoselect status: active laggproto lacp lagghash l2,l3,l4 laggport: em0 flags=1c laggport: em1 flags=1c root@pjp1:/ # netstat -r netstat: kvm not available: /dev/mem: No such file or directory Routing tables rt_tables: symbol not in namelist --------------------------------- On the main host: root@somehost:~ # netstat -r Routing tables Internet: Destination Gateway Flags Netif Expire default 192.168.1.1 UGS lagg0 pjp1.somehost.somedomain. link#4 UHS lo0 10.33.1.40/32 link#4 U lagg0 localhost link#3 UH lo0 192.168.1.0 link#4 U lagg0 somehost link#4 UHS lo0 web1.somehost.somedomain. link#4 UHS lo0 192.168.1.50/32 link#4 U lagg0 dns1.somehost.somedomain. link#4 UHS lo0 192.168.1.60/32 link#4 U lagg0 (... other jails) Internet6: Destination Gateway Flags Netif Expire :: localhost UGRS lo0 localhost link#3 UH lo0 ::ffff:0.0.0.0 localhost UGRS lo0 fe80:: localhost UGRS lo0 fe80::%lo0 link#3 U lo0 fe80::1%lo0 link#3 UHS lo0 ff01::%lo0 localhost U lo0 ff02:: localhost UGRS lo0 ff02::%lo0 localhost U lo0 --------------------------------- I would rather not set up different FIBs for different jails, unless required. First of all I would like to establish what's wrong. I just tried telnet 192.168.1.50 80 from the main host and from the 10.33.1.40 jail. From the main host it works without issues. From the jail it eventually connected after 15 or so seconds of waiting. Grzegorz From owner-freebsd-jail@freebsd.org Wed May 18 16:37:36 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5EBCFB41AE0 for ; Wed, 18 May 2016 16:37:36 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from msa1.earth.yoonka.com (yoonka.com [185.24.122.233]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "msa1.earth.yoonka.com", Issuer "msa1.earth.yoonka.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E76581D2D for ; Wed, 18 May 2016 16:37:35 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from crayon2.yoonka.com (crayon2.yoonka.com [192.168.1.20]) (authenticated bits=0) by msa1.earth.yoonka.com (8.15.2/8.15.2) with ESMTPSA id u4IGbXG7004038 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Wed, 18 May 2016 16:37:33 GMT (envelope-from list1@gjunka.com) To: freebsd-jail@freebsd.org From: Grzegorz Junka Subject: netstat -rn in jail doesn't work Message-ID: <87302b92-dcae-0ed2-92e2-0c29779c0fd3@gjunka.com> Date: Wed, 18 May 2016 16:37:33 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2016 16:37:36 -0000 What may be the reason that netstat -rn works in one jail and doesn't in another? root@app2:/ # netstat -rn Routing tables Internet: Destination Gateway Flags Netif Expire 192.168.1.76 link#4 UHS lo0 root@pjp1:/ # netstat -rn netstat: kvm not available: /dev/mem: No such file or directory Routing tables rt_tables: symbol not in namelist Grzegorz From owner-freebsd-jail@freebsd.org Wed May 18 16:38:36 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9B54B41B15 for ; Wed, 18 May 2016 16:38:36 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) by mx1.freebsd.org (Postfix) with ESMTP id AFFEA1D6E for ; Wed, 18 May 2016 16:38:36 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (unknown [10.1.1.2]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 69C5CD6EA for ; Wed, 18 May 2016 16:38:30 +0000 (UTC) Subject: Re: netstat -rn in jail doesn't work To: freebsd-jail@freebsd.org References: <87302b92-dcae-0ed2-92e2-0c29779c0fd3@gjunka.com> From: Allan Jude Message-ID: <573C9A85.6060200@freebsd.org> Date: Wed, 18 May 2016 12:38:29 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: <87302b92-dcae-0ed2-92e2-0c29779c0fd3@gjunka.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2016 16:38:36 -0000 On 2016-05-18 12:37, Grzegorz Junka wrote: > What may be the reason that netstat -rn works in one jail and doesn't in > another? > > root@app2:/ # netstat -rn > Routing tables > > Internet: > Destination Gateway Flags Netif Expire > 192.168.1.76 link#4 UHS lo0 > > > root@pjp1:/ # netstat -rn > netstat: kvm not available: /dev/mem: No such file or directory > Routing tables > rt_tables: symbol not in namelist > > Grzegorz > > _______________________________________________ > freebsd-jail@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-jail > To unsubscribe, send any mail to "freebsd-jail-unsubscribe@freebsd.org" Do you have /dev/mem exposed in one of them? -- Allan Jude From owner-freebsd-jail@freebsd.org Wed May 18 16:46:59 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 31631B41CF4 for ; Wed, 18 May 2016 16:46:59 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from msa1.earth.yoonka.com (yoonka.com [185.24.122.233]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "msa1.earth.yoonka.com", Issuer "msa1.earth.yoonka.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D13051255 for ; Wed, 18 May 2016 16:46:58 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from crayon2.yoonka.com (crayon2.yoonka.com [192.168.1.20]) (authenticated bits=0) by msa1.earth.yoonka.com (8.15.2/8.15.2) with ESMTPSA id u4IGku6t096895 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Wed, 18 May 2016 16:46:56 GMT (envelope-from list1@gjunka.com) Subject: Re: netstat -rn in jail doesn't work To: freebsd-jail@freebsd.org References: <87302b92-dcae-0ed2-92e2-0c29779c0fd3@gjunka.com> <573C9A85.6060200@freebsd.org> From: Grzegorz Junka Message-ID: <0b676bf0-2bc1-f1e1-4bb1-202c48396f33@gjunka.com> Date: Wed, 18 May 2016 16:46:56 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 In-Reply-To: <573C9A85.6060200@freebsd.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2016 16:46:59 -0000 On 18/05/2016 16:38, Allan Jude wrote: > On 2016-05-18 12:37, Grzegorz Junka wrote: >> What may be the reason that netstat -rn works in one jail and doesn't in >> another? >> >> root@app2:/ # netstat -rn >> Routing tables >> >> Internet: >> Destination Gateway Flags Netif Expire >> 192.168.1.76 link#4 UHS lo0 >> >> >> root@pjp1:/ # netstat -rn >> netstat: kvm not available: /dev/mem: No such file or directory >> Routing tables >> rt_tables: symbol not in namelist >> >> Grzegorz >> >> _______________________________________________ >> freebsd-jail@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-jail >> To unsubscribe, send any mail to "freebsd-jail-unsubscribe@freebsd.org" > Do you have /dev/mem exposed in one of them? > No. It's just that -rn doesn't show that error in that particular case, but it shows it at another occasion: root@app2:/ # netstat -a netstat: kvm not available: /dev/mem: No such file or directory Active Internet connections (including servers) Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp4 0 0 app2.8484 *.* LISTEN tcp4 0 0 app2.smtp *.* LISTEN tcp4 0 0 app2.ssh *.* LISTEN udp4 0 0 app2.syslog *.* Active UNIX domain sockets Address Type Recv-Q Send-Q Inode Conn Refs Nextref Addr fffff8003068f870 dgram 0 0 0 fffff80030634870 0 fffff80030634780 fffff8003068f960 dgram 0 0 0 fffff80030634960 0 0 fffff80030634780 dgram 0 0 0 fffff80030634870 0 0 fffff80030634870 dgram 0 0 fffff801057be1d8 0 fffff8003068f870 0 /var/run/logpriv fffff80030634960 dgram 0 0 fffff801058eb1d8 0 fffff8003068f960 0 /var/run/log From owner-freebsd-jail@freebsd.org Wed May 18 18:29:09 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BDD39B40421 for ; Wed, 18 May 2016 18:29:09 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5390816BF for ; Wed, 18 May 2016 18:29:08 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 3EA1128416; Wed, 18 May 2016 20:29:00 +0200 (CEST) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id C0D4A28412; Wed, 18 May 2016 20:28:58 +0200 (CEST) Message-ID: <573CB46A.6040308@quip.cz> Date: Wed, 18 May 2016 20:28:58 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:35.0) Gecko/20100101 Firefox/35.0 SeaMonkey/2.32 MIME-Version: 1.0 To: Grzegorz Junka , freebsd-jail@freebsd.org Subject: Re: netstat -rn in jail doesn't work References: <87302b92-dcae-0ed2-92e2-0c29779c0fd3@gjunka.com> In-Reply-To: <87302b92-dcae-0ed2-92e2-0c29779c0fd3@gjunka.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2016 18:29:09 -0000 Grzegorz Junka wrote on 05/18/2016 18:37: > What may be the reason that netstat -rn works in one jail and doesn't in > another? > > root@app2:/ # netstat -rn > Routing tables > > Internet: > Destination Gateway Flags Netif Expire > 192.168.1.76 link#4 UHS lo0 > > > root@pjp1:/ # netstat -rn > netstat: kvm not available: /dev/mem: No such file or directory > Routing tables > rt_tables: symbol not in namelist I don't know the reason but I can confirm this behavior. I know about this for a long time. Netstat complains about /dev/mem for some other params too even if it outputs correct values for example for opened tcp connections: /# netstat -s -p tcp netstat: kvm not available: /dev/mem: No such file or directory tcp: 1517892073 packets sent 1453939900 data packets (2274781047202 bytes) 759536 data packets (929141944 bytes) retransmitted 59175 data packets unnecessarily retransmitted 0 resends initiated by MTU discovery 51907865 ack-only packets (26667901 delayed) 0 URG only packets 267 window probe packets 795506 window update packets 10493883 control packets 1487401217 packets received 1417951529 acks (for 2273802396874 bytes) 7502860 duplicate acks 38600 acks for unsent data 1368386110 packets (2153255668968 bytes) received in-sequence 222423 completely duplicate packets (39239815 bytes) 11980 old duplicate packets 221 packets with some dup. data (94160 bytes duped) 35171 out-of-order packets (15770219 bytes) 21 packets (11 bytes) of data after window 11 window probes 1863690 window update packets 1642030 packets received after close 281 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 87 discarded due to memory problems 2448384 connection requests 7800552 connection accepts 0 bad connection attempts 109 listen queue overflows 339306 ignored RSTs in the windows 10221160 connections established (including accepts) 10554092 connections closed (including 1990441 drops) 5674590 connections updated cached RTT on close 5677848 connections updated cached RTT variance on close 1583021 connections updated cached ssthresh on close 10125 embryonic connections dropped 1405786035 segments updated rtt (of 1374995187 attempts) 404689 retransmit timeouts 1681 connections dropped by rexmit timeout 608 persist timeouts 0 connections dropped by persist timeout 0 Connections (fin_wait_2) dropped because of timeout 12388 keepalive timeouts 11896 keepalive probes sent 492 connections dropped by keepalive 38184853 correct ACK header predictions 46419366 correct data packet header predictions 7826351 syncache entries added 45759 retransmitted 55797 dupsyn 84 dropped 7800552 completed 40 bucket overflow 0 cache overflow 19220 reset 7941 stale 109 aborted 0 badack 230 unreach 0 zone failures 7826435 cookies sent 1784 cookies received 212203 hostcache entries added 28 bucket overflow 104273 SACK recovery episodes 242234 segment rexmits in SACK recovery episodes 303575028 byte rexmits in SACK recovery episodes 1538523 SACK options (SACK blocks) received 12421 SACK options (SACK blocks) sent 114 SACK scoreboard overflow 0 packets with ECN CE bit set 0 packets with ECN ECT(0) bit set 0 packets with ECN ECT(1) bit set 0 successful ECN handshakes 0 times ECN reduced the congestion window 0 packets with valid tcp-md5 signature received 0 packets with invalid tcp-md5 signature received 0 packets with tcp-md5 signature mismatch 0 packets with unexpected tcp-md5 signature received 0 packets without expected tcp-md5 signature received I tried netstat -rn in all 8 jails on our test machine. 4 of them works, the other 4 don't work. netstat -rn doesn't work in those jail which are older than host environment netstat -s -p tcp prints error message even in the newest jails: netstat: kvm not available: /dev/mem: No such file or directory Miroslav Lachman From owner-freebsd-jail@freebsd.org Wed May 18 18:38:50 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C0FB9B40737 for ; Wed, 18 May 2016 18:38:50 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from msa1.earth.yoonka.com (yoonka.com [185.24.122.233]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "msa1.earth.yoonka.com", Issuer "msa1.earth.yoonka.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 777F11CD3 for ; Wed, 18 May 2016 18:38:49 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from crayon2.yoonka.com (crayon2.yoonka.com [192.168.1.20]) (authenticated bits=0) by msa1.earth.yoonka.com (8.15.2/8.15.2) with ESMTPSA id u4IIclvf043323 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Wed, 18 May 2016 18:38:47 GMT (envelope-from list1@gjunka.com) Subject: Re: netstat -rn in jail doesn't work To: freebsd-jail@freebsd.org References: <87302b92-dcae-0ed2-92e2-0c29779c0fd3@gjunka.com> <573CB46A.6040308@quip.cz> From: Grzegorz Junka Message-ID: Date: Wed, 18 May 2016 18:38:47 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 In-Reply-To: <573CB46A.6040308@quip.cz> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2016 18:38:50 -0000 OK, thanks, so it looks like it doesn't prevent the jail from working correctly, it's just the reporting that's broken. Grzegorz On 18/05/2016 18:28, Miroslav Lachman wrote: > Grzegorz Junka wrote on 05/18/2016 18:37: >> What may be the reason that netstat -rn works in one jail and doesn't in >> another? >> >> root@app2:/ # netstat -rn >> Routing tables >> >> Internet: >> Destination Gateway Flags Netif Expire >> 192.168.1.76 link#4 UHS lo0 >> >> >> root@pjp1:/ # netstat -rn >> netstat: kvm not available: /dev/mem: No such file or directory >> Routing tables >> rt_tables: symbol not in namelist > > I don't know the reason but I can confirm this behavior. I know about > this for a long time. Netstat complains about /dev/mem for some other > params too even if it outputs correct values for example for opened > tcp connections: > > /# netstat -s -p tcp > netstat: kvm not available: /dev/mem: No such file or directory > tcp: > 1517892073 packets sent > 1453939900 data packets (2274781047202 bytes) > 759536 data packets (929141944 bytes) retransmitted > 59175 data packets unnecessarily retransmitted > 0 resends initiated by MTU discovery > 51907865 ack-only packets (26667901 delayed) > 0 URG only packets > 267 window probe packets > 795506 window update packets > 10493883 control packets > 1487401217 packets received > 1417951529 acks (for 2273802396874 bytes) > 7502860 duplicate acks > 38600 acks for unsent data > 1368386110 packets (2153255668968 bytes) received > in-sequence > 222423 completely duplicate packets (39239815 bytes) > 11980 old duplicate packets > 221 packets with some dup. data (94160 bytes duped) > 35171 out-of-order packets (15770219 bytes) > 21 packets (11 bytes) of data after window > 11 window probes > 1863690 window update packets > 1642030 packets received after close > 281 discarded for bad checksums > 0 discarded for bad header offset fields > 0 discarded because packet too short > 87 discarded due to memory problems > 2448384 connection requests > 7800552 connection accepts > 0 bad connection attempts > 109 listen queue overflows > 339306 ignored RSTs in the windows > 10221160 connections established (including accepts) > 10554092 connections closed (including 1990441 drops) > 5674590 connections updated cached RTT on close > 5677848 connections updated cached RTT variance on close > 1583021 connections updated cached ssthresh on close > 10125 embryonic connections dropped > 1405786035 segments updated rtt (of 1374995187 attempts) > 404689 retransmit timeouts > 1681 connections dropped by rexmit timeout > 608 persist timeouts > 0 connections dropped by persist timeout > 0 Connections (fin_wait_2) dropped because of timeout > 12388 keepalive timeouts > 11896 keepalive probes sent > 492 connections dropped by keepalive > 38184853 correct ACK header predictions > 46419366 correct data packet header predictions > 7826351 syncache entries added > 45759 retransmitted > 55797 dupsyn > 84 dropped > 7800552 completed > 40 bucket overflow > 0 cache overflow > 19220 reset > 7941 stale > 109 aborted > 0 badack > 230 unreach > 0 zone failures > 7826435 cookies sent > 1784 cookies received > 212203 hostcache entries added > 28 bucket overflow > 104273 SACK recovery episodes > 242234 segment rexmits in SACK recovery episodes > 303575028 byte rexmits in SACK recovery episodes > 1538523 SACK options (SACK blocks) received > 12421 SACK options (SACK blocks) sent > 114 SACK scoreboard overflow > 0 packets with ECN CE bit set > 0 packets with ECN ECT(0) bit set > 0 packets with ECN ECT(1) bit set > 0 successful ECN handshakes > 0 times ECN reduced the congestion window > 0 packets with valid tcp-md5 signature received > 0 packets with invalid tcp-md5 signature received > 0 packets with tcp-md5 signature mismatch > 0 packets with unexpected tcp-md5 signature received > 0 packets without expected tcp-md5 signature received > > > I tried netstat -rn in all 8 jails on our test machine. 4 of them > works, the other 4 don't work. > > netstat -rn doesn't work in those jail which are older than host > environment > > netstat -s -p tcp prints error message even in the newest jails: > netstat: kvm not available: /dev/mem: No such file or directory > > > Miroslav Lachman > From owner-freebsd-jail@freebsd.org Thu May 19 14:51:40 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D00D5B42AB7 for ; Thu, 19 May 2016 14:51:40 +0000 (UTC) (envelope-from jamie@freebsd.org) Received: from gritton.org (gritton.org [162.220.209.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "www.gritton.org", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id AA44A1FC3 for ; Thu, 19 May 2016 14:51:40 +0000 (UTC) (envelope-from jamie@freebsd.org) Received: from gritton.org (gritton.org [162.220.209.3]) by gritton.org (8.15.2/8.15.2) with ESMTPS id u4JEoCnV091728 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 19 May 2016 08:50:13 -0600 (MDT) (envelope-from jamie@freebsd.org) Received: (from www@localhost) by gritton.org (8.15.2/8.15.2/Submit) id u4JEoCMB091727; Thu, 19 May 2016 08:50:12 -0600 (MDT) (envelope-from jamie@freebsd.org) X-Authentication-Warning: gritton.org: www set sender to jamie@freebsd.org using -f To: freebsd-jail@freebsd.org Subject: Re: jails in different private subnets on the same host X-PHP-Originating-Script: 0:rcube.php MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 19 May 2016 08:50:12 -0600 From: James Gritton In-Reply-To: <07d67bd5-206c-edd8-7f47-ef2b5c538e01@gjunka.com> References: <07d67bd5-206c-edd8-7f47-ef2b5c538e01@gjunka.com> Message-ID: X-Sender: jamie@freebsd.org User-Agent: Roundcube Webmail/1.1.2 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 May 2016 14:51:40 -0000 On 2016-05-18 09:08, Grzegorz Junka wrote: > I just tried telnet 192.168.1.50 80 from the main host and from the > 10.33.1.40 jail. From the main host it works without issues. From the > jail it eventually connected after 15 or so seconds of waiting. That sounds like about the kind of timeout I'd expect from DNS resolution not working. If you're adding a new subnet when the jail is created, you'll need to do something to get a nameserver to listen to it. - Jamie From owner-freebsd-jail@freebsd.org Thu May 19 14:57:07 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9895EB42C67 for ; Thu, 19 May 2016 14:57:07 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from msa1.earth.yoonka.com (yoonka.com [185.24.122.233]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "msa1.earth.yoonka.com", Issuer "msa1.earth.yoonka.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 430791398 for ; Thu, 19 May 2016 14:57:06 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from crayon2.yoonka.com (crayon2.yoonka.com [192.168.1.20]) (authenticated bits=0) by msa1.earth.yoonka.com (8.15.2/8.15.2) with ESMTPSA id u4JEv4vp009091 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Thu, 19 May 2016 14:57:04 GMT (envelope-from list1@gjunka.com) Subject: Re: jails in different private subnets on the same host To: freebsd-jail@freebsd.org References: <07d67bd5-206c-edd8-7f47-ef2b5c538e01@gjunka.com> From: Grzegorz Junka Message-ID: <3aaa36dc-c658-5760-c4bb-d0f991834194@gjunka.com> Date: Thu, 19 May 2016 14:57:04 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 May 2016 14:57:07 -0000 On 19/05/2016 14:50, James Gritton wrote: > On 2016-05-18 09:08, Grzegorz Junka wrote: >> I just tried telnet 192.168.1.50 80 from the main host and from the >> 10.33.1.40 jail. From the main host it works without issues. From the >> jail it eventually connected after 15 or so seconds of waiting. > > That sounds like about the kind of timeout I'd expect from DNS > resolution not working. If you're adding a new subnet when the jail > is created, you'll need to do something to get a nameserver to listen > to it. > > - Jamie Why would it need to use the nameserver if I am telneting through IP? My nameserver is running in 192.168.1.60 but drill @192.168.1.60 from inside the 10.33.1.40 jail doesn't see it. I am using telnet with the IP specifically to avoid using the nameserver because I know the jail can't use the nameserver at this moment (until this is solved). Grzegorz From owner-freebsd-jail@freebsd.org Thu May 19 15:19:14 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6C2FAB4114D for ; Thu, 19 May 2016 15:19:14 +0000 (UTC) (envelope-from lists@opsec.eu) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 325661F4C for ; Thu, 19 May 2016 15:19:14 +0000 (UTC) (envelope-from lists@opsec.eu) Received: from pi by home.opsec.eu with local (Exim 4.87 (FreeBSD)) (envelope-from ) id 1b3PjG-0005YP-Mq; Thu, 19 May 2016 17:19:14 +0200 Date: Thu, 19 May 2016 17:19:14 +0200 From: Kurt Jaeger To: Grzegorz Junka Cc: freebsd-jail@freebsd.org Subject: Re: jails in different private subnets on the same host Message-ID: <20160519151914.GL15034@home.opsec.eu> References: <07d67bd5-206c-edd8-7f47-ef2b5c538e01@gjunka.com> <3aaa36dc-c658-5760-c4bb-d0f991834194@gjunka.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3aaa36dc-c658-5760-c4bb-d0f991834194@gjunka.com> X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 May 2016 15:19:14 -0000 Hi! > Why would it need to use the nameserver if I am telneting through IP? Use telnet -N to avoid DNS lookups. -- pi@opsec.eu +49 171 3101372 4 years to go ! From owner-freebsd-jail@freebsd.org Thu May 19 15:39:55 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9550B416D1 for ; Thu, 19 May 2016 15:39:55 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: from mail-ig0-x235.google.com (mail-ig0-x235.google.com [IPv6:2607:f8b0:4001:c05::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 71DC41B73 for ; Thu, 19 May 2016 15:39:55 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: by mail-ig0-x235.google.com with SMTP id vs11so7909388igb.1 for ; Thu, 19 May 2016 08:39:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-transfer-encoding; bh=s3r835t1LWPZaQ4KdpWPP34ysZtpeqBwKJkUfbe0agE=; b=POhut20REUb6XCCzkT/wJ0OAVqLOX5ItdcukB2tQhykDsHBt7tEcTFZCiDixmPrpRj Mcf844aYMpfkBfxJBb/bTbBXEsmUAM+T7fqfKigwyeU9Anl4EdaGa10extHhNXTvmHsp uCTVndq+hGrdQUkax7FV+TLvDPw/uuUQ9wM8sOQE4QjGoRIKaITtuvrqeEcw+NBGQiL1 mzQcx3DQE00PnzaXrm4VZX38CFRZr8VYac9yF/QmsuuPNYRcpS6dy3B1gZkzIVqsVw/B aMAFryYoao5DMUhvwpkDXWTvIWul5eMTG0MGX9kTjNuZAlUIimknniLUurf6GWWMLzuu BLJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-transfer-encoding; bh=s3r835t1LWPZaQ4KdpWPP34ysZtpeqBwKJkUfbe0agE=; b=WawoR0PaIUoHwo2p3JU0TSkh5PXLUBpyacfrJ7Lr+BsI0RPjsE83Q23XWQ//uibXUz xNeq4lOFywMVdCsBjwDK7iFZ+nACHy0JMINeEjKvOq/4nTxIqVnCXFMWGZoYFjW8IZaj PM+bnn+g17feX6M1ZrFAHPyoKg0c/kIO4+xLREMOwp7JbBcMIH0tYVWBNQeLKwqGLzuC qUfry/v9It7pCfyYaGquG4Ujrik9TKkd5IoY2+V0fkoncwO3hzga+r1/6o33W5oXIVJG 2t4U7EsR+l8hrUXnHm0xBLgI2oWyADqzS5vjJopdkSQVn8BOiM8TQHRa3z/61RBMMsH8 KEyA== X-Gm-Message-State: AOPr4FUqcjUnr7ovo5Xovg/QyKzxlxTqSkXPviavXfq4YbNiFwx4+hrTf97AT+S7nYfi+A== X-Received: by 10.50.72.237 with SMTP id g13mr3691396igv.4.1463672394711; Thu, 19 May 2016 08:39:54 -0700 (PDT) Received: from [10.0.10.3] (cpe-184-56-210-236.neo.res.rr.com. [184.56.210.236]) by smtp.googlemail.com with ESMTPSA id c184sm10573811itc.18.2016.05.19.08.39.53 for (version=TLSv1/SSLv3 cipher=OTHER); Thu, 19 May 2016 08:39:54 -0700 (PDT) Message-ID: <573DDE59.8000300@gmail.com> Date: Thu, 19 May 2016 11:40:09 -0400 From: Ernie Luzar User-Agent: Thunderbird 2.0.0.24 (Windows/20100228) MIME-Version: 1.0 To: freebsd-jail@freebsd.org Subject: Re: jails in different private subnets on the same host References: <07d67bd5-206c-edd8-7f47-ef2b5c538e01@gjunka.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 May 2016 15:39:55 -0000 James Gritton wrote: > On 2016-05-18 09:08, Grzegorz Junka wrote: >> I just tried telnet 192.168.1.50 80 from the main host and from the >> 10.33.1.40 jail. From the main host it works without issues. From the >> jail it eventually connected after 15 or so seconds of waiting. > > That sounds like about the kind of timeout I'd expect from DNS > resolution not working. If you're adding a new subnet when the jail is > created, you'll need to do something to get a nameserver to listen to it. > > - Jamie You have not copied the hosts /etc/resolv.conf to the jail in question. From owner-freebsd-jail@freebsd.org Thu May 19 15:53:26 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9B8C5B41B64 for ; Thu, 19 May 2016 15:53:26 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from msa1.earth.yoonka.com (yoonka.com [185.24.122.233]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "msa1.earth.yoonka.com", Issuer "msa1.earth.yoonka.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3B44512C4 for ; Thu, 19 May 2016 15:53:25 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from crayon2.yoonka.com (crayon2.yoonka.com [192.168.1.20]) (authenticated bits=0) by msa1.earth.yoonka.com (8.15.2/8.15.2) with ESMTPSA id u4JFrN6Z009962 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Thu, 19 May 2016 15:53:24 GMT (envelope-from list1@gjunka.com) Subject: Re: jails in different private subnets on the same host To: freebsd-jail@freebsd.org References: <07d67bd5-206c-edd8-7f47-ef2b5c538e01@gjunka.com> <573DDE59.8000300@gmail.com> From: Grzegorz Junka Message-ID: <62a0bf1c-88f3-a815-1187-a26e51c9b8bb@gjunka.com> Date: Thu, 19 May 2016 15:53:23 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 In-Reply-To: <573DDE59.8000300@gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 May 2016 15:53:26 -0000 On 19/05/2016 15:40, Ernie Luzar wrote: > James Gritton wrote: >> On 2016-05-18 09:08, Grzegorz Junka wrote: >>> I just tried telnet 192.168.1.50 80 from the main host and from the >>> 10.33.1.40 jail. From the main host it works without issues. From the >>> jail it eventually connected after 15 or so seconds of waiting. >> >> That sounds like about the kind of timeout I'd expect from DNS >> resolution not working. If you're adding a new subnet when the jail >> is created, you'll need to do something to get a nameserver to listen >> to it. >> >> - Jamie > > You have not copied the hosts /etc/resolv.conf to the jail in question. > Of course I did. root@somehost:/# cat /etc/resolv.conf search somehost.somedomain.com nameserver 192.168.1.60 nameserver 8.8.8.8 I installed the jail using bsdinstall and it copies that automatically. Grzegorz From owner-freebsd-jail@freebsd.org Fri May 20 11:07:58 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1EA2B43006 for ; Fri, 20 May 2016 11:07:58 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from msa1.earth.yoonka.com (yoonka.com [185.24.122.233]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "msa1.earth.yoonka.com", Issuer "msa1.earth.yoonka.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5EB4A16D1 for ; Fri, 20 May 2016 11:07:57 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from crayon2.yoonka.com (crayon2.yoonka.com [192.168.1.20]) (authenticated bits=0) by msa1.earth.yoonka.com (8.15.2/8.15.2) with ESMTPSA id u4KB7sPN036320 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Fri, 20 May 2016 11:07:55 GMT (envelope-from list1@gjunka.com) Subject: Re: jails in different private subnets on the same host References: <07d67bd5-206c-edd8-7f47-ef2b5c538e01@gjunka.com> <3aaa36dc-c658-5760-c4bb-d0f991834194@gjunka.com> <20160519151914.GL15034@home.opsec.eu> From: Grzegorz Junka To: freebsd-jail@freebsd.org Message-ID: <573349b9-b10d-e976-6d41-1118f5de4b2c@gjunka.com> Date: Fri, 20 May 2016 11:07:54 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 In-Reply-To: <20160519151914.GL15034@home.opsec.eu> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 May 2016 11:07:58 -0000 On 19/05/2016 15:19, Kurt Jaeger wrote: > Hi! > >> Why would it need to use the nameserver if I am telneting through IP? > Use telnet -N to avoid DNS lookups. Oh, great! That worked. It could connect to the web server jail immediately. So it looks like the problem is with connecting to the DNS jail, but why? This is inside the DNS jail: *root@dns1:/ # netstat -an* netstat: kvm not available: /dev/mem: No such file or directory Active Internet connections (including servers) Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp4 0 0 192.168.1.60.53 *.* LISTEN tcp4 0 0 192.168.1.60.25 *.* LISTEN udp4 0 0 192.168.1.60.53 *.* udp4 0 0 192.168.1.60.514 *.* (... IPv6 entries) On the problematic jail: *root@pjp1:/ # cat /etc/resolv.conf * search myserver.mydomain.com nameserver 192.168.1.60 options edns0 *root@pjp1:/ # netstat -an* netstat: kvm not available: /dev/mem: No such file or directory Active Internet connections (including servers) Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp4 0 0 10.33.1.40.25 *.* LISTEN tcp4 0 0 10.33.1.40.3306 *.* LISTEN tcp4 0 0 10.33.1.40.80 *.* LISTEN udp4 0 0 10.33.1.40.514 *.* *root@pjp1:/ # netstat -rn* Routing tables Internet: Destination Gateway Flags Netif Expire 10.33.1.40 link#4 UHS lo0 This works immediately: *root@pjp1:/ # telnet -N 192.168.1.60 53* Trying 192.168.1.60... Connected to 192.168.1.60. Escape character is '^]'. But this connects after exactly 15 seconds: *root@pjp1:/ # telnet 192.168.1.60 53* Trying 192.168.1.60... Connected to 192.168.1.60. Escape character is '^]'. Grzegorz From owner-freebsd-jail@freebsd.org Fri May 20 11:21:06 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 24CF1B4387D for ; Fri, 20 May 2016 11:21:06 +0000 (UTC) (envelope-from lists@opsec.eu) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D67471041 for ; Fri, 20 May 2016 11:21:05 +0000 (UTC) (envelope-from lists@opsec.eu) Received: from pi by home.opsec.eu with local (Exim 4.87 (FreeBSD)) (envelope-from ) id 1b3iUK-0007tt-OZ; Fri, 20 May 2016 13:21:04 +0200 Date: Fri, 20 May 2016 13:21:04 +0200 From: Kurt Jaeger To: Grzegorz Junka Cc: freebsd-jail@freebsd.org Subject: Re: jails in different private subnets on the same host Message-ID: <20160520112104.GM15034@home.opsec.eu> References: <07d67bd5-206c-edd8-7f47-ef2b5c538e01@gjunka.com> <3aaa36dc-c658-5760-c4bb-d0f991834194@gjunka.com> <20160519151914.GL15034@home.opsec.eu> <573349b9-b10d-e976-6d41-1118f5de4b2c@gjunka.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <573349b9-b10d-e976-6d41-1118f5de4b2c@gjunka.com> X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 May 2016 11:21:06 -0000 Hi! > >> Why would it need to use the nameserver if I am telneting through IP? > > Use telnet -N to avoid DNS lookups. > Oh, great! That worked. It could connect to the web server jail > immediately. So it looks like the problem is with connecting to the DNS > jail, but why? It's not the problem connecting, it's getting an answer. Does your DNS have an answer for 60.1.168.192.in-addr.arpa ? -- pi@opsec.eu +49 171 3101372 4 years to go ! From owner-freebsd-jail@freebsd.org Sat May 21 14:53:41 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 76F1AB44740 for ; Sat, 21 May 2016 14:53:41 +0000 (UTC) (envelope-from freekai@outlook.com) Received: from BLU004-OMC3S12.hotmail.com (blu004-omc3s12.hotmail.com [65.55.116.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 38BD31FC7 for ; Sat, 21 May 2016 14:53:40 +0000 (UTC) (envelope-from freekai@outlook.com) Received: from BLU403-EAS386 ([65.55.116.74]) by BLU004-OMC3S12.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.23008); Sat, 21 May 2016 07:52:33 -0700 X-TMN: [2EiiGhL2YI2YyVf7hSD9x0MNZ9AYNCoL9uy1QJ517C8=] X-Originating-Email: [freekai@outlook.com] Message-ID: From: =?UTF-8?B?5qKF5Yex?= To: Subject: cannot freebsd jail by c Date: Sat, 21 May 2016 22:52:30 +0800 MIME-Version: 1.0 X-Mailer: Microsoft Outlook 16.0 Content-Language: zh-cn Thread-Index: AdGzb2ogor8UkAw8S6+GQ3gxLe0POQ== X-OriginalArrivalTime: 21 May 2016 14:52:33.0908 (UTC) FILETIME=[685CDB40:01D1B370] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 May 2016 14:53:41 -0000 This is my part c code: =20 11 int main() 12 { 13 in_addr_t addr_t=3Dinet_addr("192.168.2.1"); 14=20 15 struct in_addr in_addr=3Dinet_makeaddr(addr_t,AF_INET); 16=20 17=20 18 struct jail j=3D{ 19 .version=3DJAIL_API_VERSION, 20 .path=3D"./jail_test", 21 .hostname=3D"myjail", 22 .ip4s=3Daddr_t, 23 .ip6s=3D0, 24 .ip4=3D&in_addr, 25 .ip6=3DNULL 26 }; 27=20 28 errno=3D0; 29 int rs=3Djail(&j); 30 if(0=3D=3Drs){ 31 printf("create jail ok!!!\r\n"); 32 return 0; 33 } 34=20 35 switch(errno){ 36 case EPERM: 37 printf("eperm\r\n"); 38 break; 39 case EFAULT: 40 printf("efault\r\n"); 41 break; 42 case EINVAL: 43 printf("einval\r\n"); 44 break; 45 case EAGAIN: 46 printf("eagain\r\n"); 47 break; 48 default: 49 printf("---------------\r\n"); 50 break; 51 } 52 return 0; 53 } Unfortunately,the errno return EINVAL,it means =E2=80=9CThe version = number of the argument is not correct.=E2=80=9D,why? From owner-freebsd-jail@freebsd.org Sat May 21 21:49:15 2016 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A335EB4499B for ; Sat, 21 May 2016 21:49:15 +0000 (UTC) (envelope-from jamie@freebsd.org) Received: from gritton.org (gritton.org [162.220.209.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "www.gritton.org", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5BCB81946 for ; Sat, 21 May 2016 21:49:14 +0000 (UTC) (envelope-from jamie@freebsd.org) Received: from gritton.org (gritton.org [162.220.209.3]) by gritton.org (8.15.2/8.15.2) with ESMTPS id u4LLn7hR037498 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 21 May 2016 15:49:07 -0600 (MDT) (envelope-from jamie@freebsd.org) Received: (from www@localhost) by gritton.org (8.15.2/8.15.2/Submit) id u4LLn6GS037497; Sat, 21 May 2016 15:49:06 -0600 (MDT) (envelope-from jamie@freebsd.org) X-Authentication-Warning: gritton.org: www set sender to jamie@freebsd.org using -f To: freebsd-jail@freebsd.org Subject: Re: cannot freebsd jail by c X-PHP-Originating-Script: 0:rcube.php MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Sat, 21 May 2016 15:49:06 -0600 From: James Gritton Cc: =?UTF-8?Q?=E6=A2=85=E5=87=B1?= In-Reply-To: References: Message-ID: <9898baf72b32a2feada28638e09a3c2f@gritton.org> X-Sender: jamie@freebsd.org User-Agent: Roundcube Webmail/1.1.2 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 May 2016 21:49:15 -0000 On 2016-05-21 08:52, 梅凱 wrote: > This is my part c code: > > > 11 int main() > 12 { > 13 in_addr_t addr_t=inet_addr("192.168.2.1"); > 14 > 15 struct in_addr in_addr=inet_makeaddr(addr_t,AF_INET); > 16 > 17 > 18 struct jail j={ > 19 .version=JAIL_API_VERSION, > 20 .path="./jail_test", > 21 .hostname="myjail", > 22 .ip4s=addr_t, > 23 .ip6s=0, > 24 .ip4=&in_addr, > 25 .ip6=NULL > 26 }; > 27 > 28 errno=0; > 29 int rs=jail(&j); > 30 if(0==rs){ > 31 printf("create jail ok!!!\r\n"); > 32 return 0; > 33 } > 34 > 35 switch(errno){ > 36 case EPERM: > 37 printf("eperm\r\n"); > 38 break; > 39 case EFAULT: > 40 printf("efault\r\n"); > 41 break; > 42 case EINVAL: > 43 printf("einval\r\n"); > 44 break; > 45 case EAGAIN: > 46 printf("eagain\r\n"); > 47 break; > 48 default: > 49 printf("---------------\r\n"); > 50 break; > 51 } > 52 return 0; > 53 } > > Unfortunately,the errno return EINVAL,it means “The version number of > the argument is not correct.”,why? Actually, jail(2) can give EINVAL not only for the reason listed, but also for some of the reasons mentioned under jail_set. Really, it means just some value was wrong. In this case there were two errors. You passed addr_t in .ip4s, but that's supposed to be the number of addresses and not the address itself - pass 1 instead. Also, the path of "./jail_test" won't work; it needs to be a full pathname instead. Fix those two and the jail will create correctly. - Jamie