From owner-freebsd-jail@FreeBSD.ORG Thu Jun 28 10:16:06 2007 Return-Path: X-Original-To: freebsd-jail@freebsd.org Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0B05D16A468 for ; Thu, 28 Jun 2007 10:16:06 +0000 (UTC) (envelope-from roger@gujord.com) Received: from red.newsgrip.com (red.newsgrip.com [193.69.188.102]) by mx1.freebsd.org (Postfix) with ESMTP id 9A99713C45D for ; Thu, 28 Jun 2007 10:16:05 +0000 (UTC) (envelope-from roger@gujord.com) Received: from [192.168.1.2] ([192.168.1.2]) by red.newsgrip.com (8.12.8/8.12.5) with ESMTP id l5SA06qV010078 for ; Thu, 28 Jun 2007 12:00:07 +0200 Message-ID: <468386A5.9040900@gujord.com> Date: Thu, 28 Jun 2007 12:00:05 +0200 From: Roger Gujord Organization: Gujord.com User-Agent: Thunderbird 1.5.0.12 (X11/20070530) MIME-Version: 1.0 To: freebsd-jail@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-MailScanner-Information: Please contact the ISP for more information X-MailScanner: Found to be clean Subject: Different Netmasks for jails X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jun 2007 10:16:06 -0000 Hi, I'm about to set up jails for a production environment and I stumbled across a problem with different netmasks for my jails. How can I have a netmask like 255.255.255.128 for one jail and a netmask like 255.255.255.240 for another jail without creating static aliases at hosts boot-time in /etc/rc.conf ? I would like the /etc/rc.d/jail to create the network aliases according to the jails respective network mask. I see that /etc/rc.d/jail sets the netmask to 255.255.255.255 for all network aliases. Thanx! :-) --Roger This is my rc.conf: ------------------------------------------------------------------------------- defaultrouter="192.168.1.49" hostname="myhost.mydomain.com" ifconfig_bge0="inet 192.168.1.1 netmask 255.255.255.128" # FreeBSD jail: jail_enable="YES" jail_set_hostname_allow="YES" jail_interface="bge0" jail_devfs_enable="YES" jail_devfs_ruleset="devfsrules_jail" jail_procfs_enable="YES" jail_socket_unixiproute_only="YES" # Space separated list of names of jails: jail_list="jailhosta jailhostb" # Jail: jailhosta.mydomain.com (the netmask for this jail should be 255.255.255.128) jail_jailhosta_hostname="jailhosta.mydomain.com" jail_jailhosta_ip="192.168.1.2" jail_jailhosta_rootdir="/usr/jails/jailhosta.mydomain.com" # Jail: jailhostb.mydomain.com (the netmask for this jail should be 255.255.255.240) jail_jailhostb_hostname="jailhostb.mydomain.com" jail_jailhostb_ip="192.168.1.54" jail_jailhostb_rootdir="/usr/jails/jailhostb.mydomain.com" From owner-freebsd-jail@FreeBSD.ORG Thu Jun 28 12:02:07 2007 Return-Path: X-Original-To: freebsd-jail@freebsd.org Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 71F1716A46B for ; Thu, 28 Jun 2007 12:02:07 +0000 (UTC) (envelope-from beto@octantis.com.au) Received: from sigma.octantis.com.au (ns2.octantis.com.au [207.44.189.124]) by mx1.freebsd.org (Postfix) with ESMTP id 2626013C4C5 for ; Thu, 28 Jun 2007 12:02:06 +0000 (UTC) (envelope-from beto@octantis.com.au) Received: (qmail 9218 invoked from network); 28 Jun 2007 06:35:25 -0500 Received: from unknown (HELO localhost) (210.84.50.231) by sigma.octantis.com.au with (DHE-RSA-AES256-SHA encrypted) SMTP; 28 Jun 2007 06:35:24 -0500 Date: Thu, 28 Jun 2007 21:35:20 +1000 From: Norberto Meijome To: Roger Gujord Message-ID: <20070628213520.6e561b42@localhost> In-Reply-To: <468386A5.9040900@gujord.com> References: <468386A5.9040900@gujord.com> Organization: Octantis Pty Ltd X-Mailer: Claws Mail 2.9.2 (GTK+ 2.10.13; i386-portbld-freebsd6.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-jail@freebsd.org Subject: Re: Different Netmasks for jails X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jun 2007 12:02:07 -0000 On Thu, 28 Jun 2007 12:00:05 +0200 Roger Gujord wrote: > How can I have a netmask like 255.255.255.128 for one jail and a netmask > like 255.255.255.240 for another jail without creating static aliases at > hosts boot-time in /etc/rc.conf ? > I would like the /etc/rc.d/jail to create the network aliases according > to the jails respective network mask. I see that /etc/rc.d/jail sets the > netmask to 255.255.255.255 for all network aliases. Hi :) I'm pretty sure this is the same FAQ as seen in questions@ / net@ wrt netmasks of aliases. In short, if your aliased interfaces are in the *same* subnet as your main IP (eg, the main IP's subnet includes all the aliased subnets), they will be /32. _________________________ Norberto Meijome Octantis Pty Ltd "Unix is very simple, but it takes a genius to understand the simplicity." Dennis Ritchie NOTICE: The contents of this email and its attachments are confidential and intended only for the individuals or entities named above. If you have received this message in error, please advise the sender by reply email and immediately delete the message and any attachments without using, copying or disclosing the contents. Thank you. From owner-freebsd-jail@FreeBSD.ORG Fri Jun 29 07:29:18 2007 Return-Path: X-Original-To: freebsd-jail@freebsd.org Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D85C816A474 for ; Fri, 29 Jun 2007 07:29:18 +0000 (UTC) (envelope-from roger@gujord.com) Received: from red.newsgrip.com (red.newsgrip.com [193.69.188.102]) by mx1.freebsd.org (Postfix) with ESMTP id 71DBD13C480 for ; Fri, 29 Jun 2007 07:29:18 +0000 (UTC) (envelope-from roger@gujord.com) Received: from [192.168.1.2] ([192.168.1.2]) by red.newsgrip.com (8.12.8/8.12.5) with ESMTP id l5T7TEqV027263; Fri, 29 Jun 2007 09:29:14 +0200 Message-ID: <4684B4C9.90101@gujord.com> Date: Fri, 29 Jun 2007 09:29:13 +0200 From: Roger Gujord Organization: Gujord.com User-Agent: Thunderbird 1.5.0.12 (X11/20070530) MIME-Version: 1.0 To: freebsd-jail@freebsd.org References: <468386A5.9040900@gujord.com> <20070628213520.6e561b42@localhost> In-Reply-To: <20070628213520.6e561b42@localhost> Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-MailScanner-Information: Please contact the ISP for more information X-MailScanner: Found to be clean Cc: beto@octantis.com.au Subject: Re: Different Netmasks for jails X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jun 2007 07:29:18 -0000 Norberto Meijome wrote: > On Thu, 28 Jun 2007 12:00:05 +0200 > Roger Gujord wrote: > > >> How can I have a netmask like 255.255.255.128 for one jail and a netmask >> like 255.255.255.240 for another jail without creating static aliases at >> hosts boot-time in /etc/rc.conf ? >> I would like the /etc/rc.d/jail to create the network aliases according >> to the jails respective network mask. I see that /etc/rc.d/jail sets the >> netmask to 255.255.255.255 for all network aliases. >> > > Hi :) > > I'm pretty sure this is the same FAQ as seen in questions@ / net@ wrt netmasks > of aliases. In short, if your aliased interfaces are in the *same* subnet as > your main IP (eg, the main IP's subnet includes all the aliased subnets), they > will be /32. > Thanks for your answer Norberto :-) If I understand you correctly, you are saying that the same netmask as the one for the hosts main IP will apply to all jails and therefore there is no need to specify a netmask other than /32 for each jail in the same subnet. But this is not my case. I have two different subnets in the same IP-range with different netmasks (of course) and also another IP-range. ------------------------------------------------------- Subnet ID Broadcast Address Subnet Mask ------------------------------------------------------- 192.168.1.0 /25 192.168.1.127 255.255.255.128 192.168.1.48 /28 192.168.1.62 255.255.255.240 192.168.2.96 /27 192.168.2.127 255.255.255.224 I'm trying to configure jails to run in all three subnets on the same host: mainHost 192.168.1.1/25 jailHostA 192.168.1.2/25 jailHostB 192.168.1.54/28 jailHostC 192.168.2.100/27 I guess the jailHostA is OK since it's running in the same subnet as the mainHost. But how do I configure jailHostB and C without creating a static aliased interface (in /etc/rc.conf) when all jails are started up with netmask /32 (from /etc/rc.d/jail) ? Thanks! :-) --Roger From owner-freebsd-jail@FreeBSD.ORG Fri Jun 29 08:25:56 2007 Return-Path: X-Original-To: freebsd-jail@freebsd.org Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BB3E416A400 for ; Fri, 29 Jun 2007 08:25:56 +0000 (UTC) (envelope-from roger@gujord.com) Received: from red.newsgrip.com (red.newsgrip.com [193.69.188.102]) by mx1.freebsd.org (Postfix) with ESMTP id 3C00113C43E for ; Fri, 29 Jun 2007 08:25:55 +0000 (UTC) (envelope-from roger@gujord.com) Received: from [192.168.1.2] ([192.168.1.2]) by red.newsgrip.com (8.12.8/8.12.5) with ESMTP id l5T8PsqV027963 for ; Fri, 29 Jun 2007 10:25:54 +0200 Message-ID: <4684C212.1070005@gujord.com> Date: Fri, 29 Jun 2007 10:25:54 +0200 From: Roger Gujord Organization: Gujord.com User-Agent: Thunderbird 1.5.0.12 (X11/20070530) MIME-Version: 1.0 To: freebsd-jail@freebsd.org References: <468386A5.9040900@gujord.com> <20070628213520.6e561b42@localhost> <4684B4C9.90101@gujord.com> In-Reply-To: <4684B4C9.90101@gujord.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-MailScanner-Information: Please contact the ISP for more information X-MailScanner: Found to be clean Subject: Re: Different Netmasks for jails X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jun 2007 08:25:56 -0000 UPDATE: ------- I managed to do what I wanted by adding the following to /etc/rc.d/jail: # init_variables _j eval _netmask=\"\$jail_${_j}_netmask\" # Debugging aid debug "$_j netmask: $_netmask" ... if [ -z "${_netmask}" ]; then err 3 "$name: No netmask has been defined for ${_j}" fi # jail_start() ifconfig ${_interface} alias ${_ip} netmask ${_netmask} and the following for each jail in /etc/rc.conf: jail_jailname_netmask="xxx.xxx.xxx.xxx" I'm not sure if this is the proper way to do it, but it seems to work for me. --Roger From owner-freebsd-jail@FreeBSD.ORG Fri Jun 29 15:39:08 2007 Return-Path: X-Original-To: freebsd-jail@freebsd.org Delivered-To: freebsd-jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CA98816A400 for ; Fri, 29 Jun 2007 15:39:08 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [82.208.36.70]) by mx1.freebsd.org (Postfix) with ESMTP id 4758A13C45B for ; Fri, 29 Jun 2007 15:39:07 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from localhost (localhost.codelab.cz [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 6C91219E02A for ; Fri, 29 Jun 2007 17:39:06 +0200 (CEST) Received: from [192.168.1.3] (klient92-110.sxg.cz [62.77.92.110]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTP id 82CD119E027 for ; Fri, 29 Jun 2007 17:38:58 +0200 (CEST) Message-ID: <46852798.3070509@quip.cz> Date: Fri, 29 Jun 2007 17:39:04 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.4) Gecko/20070509 SeaMonkey/1.1.2 MIME-Version: 1.0 To: freebsd-jail@freebsd.org Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit Subject: jtune reports 0M Memory Used X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jun 2007 15:39:08 -0000 Hi, I am testing cdjones_jail_soc2006.patch on FreeBSD 6.2-RELEASE, patch applied and GENERIC kernel, libc + jls, jail, jtune binaries recompiled / reinstalled. I have some problems / questions. My testing setup is bellow (FreeBSD running in Qemu on Windows XP) notes: host system hase name qemuspeed jails are alpha, beta, gamma !!! next content is very long !!! root@qemuspeed ~/# jls JID IP Address Hostname Path 3 172.16.16.4 gamma.xps.quip.test /usr/jail/gamma 2 172.16.16.3 beta.xps.quip.test /usr/jail/beta 1 172.16.16.2 alpha.xps.quip.test /usr/jail/alpha root@qemuspeed ~/# ./jtune_all.sh JID Hostname Memory Used / Limit CPU Shares 3 gamma.xps.quip.test 0 M / None 0 JID Hostname Memory Used / Limit CPU Shares 2 beta.xps.quip.test 0 M / None 0 JID Hostname Memory Used / Limit CPU Shares 1 alpha.xps.quip.test 0 M / None 0 ## from top (load 3) PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 1498 root 1 123 0 1680K 1172K RUN 0:54 32.21% sh 1499 root 1 122 0 1680K 1172K RUN 0:52 32.04% sh 1497 root 1 122 0 1680K 1172K RUN 0:57 32.03% sh root@qemuspeed ~/# ./jtune_sysctls.sh kern.sched.limit_jail_cpu: 0 -> 1 security.jail.limit_jail_memory: 0 -> 1 kern.sched.system_cpu_shares: 0 -> 10 security.jail.jail_pager_interval: 5 -> 5 ## top is still same, endless loops in Jails are still running root@qemuspeed ~/# ./jtune_tunes.sh JID Hostname Memory Used / Limit CPU Shares 1 alpha.xps.quip.test 0 M / 5 M 5 JID Hostname Memory Used / Limit CPU Shares 2 beta.xps.quip.test 0 M / 15 M 15 JID Hostname Memory Used / Limit CPU Shares 3 gamma.xps.quip.test 0 M / 20 M 20 ## few minutes after aplying jtune settings - WCPU is still the same [it seems wrong] PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 1497 root 1 170 0 1680K 844K RUN 5:17 32.62% sh 1498 root 1 170 0 1680K 1172K RUN 5:16 31.54% sh 1499 root 1 170 0 1680K 1172K RUN 5:17 30.71% sh ## retest with tcsh while (1) endless loop [it seems OK according to jtune settings] PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 1368 root 1 129 0 3924K 2692K RUN 3:29 58.25% tcsh 1359 root 1 170 0 3924K 2692K RUN 1:55 24.27% tcsh 1350 root 1 170 0 3928K 1068K RUN 0:55 12.06% tcsh root@qemuspeed ~/# ./jtune_all.sh JID Hostname Memory Used / Limit CPU Shares 3 gamma.xps.quip.test 0 M / 20 M 20 JID Hostname Memory Used / Limit CPU Shares 2 beta.xps.quip.test 0 M / 15 M 15 JID Hostname Memory Used / Limit CPU Shares 1 alpha.xps.quip.test 0 M / 5 M 5 ## Why jtune -j JID -i shows Memory Used 0M if the process in Jail uses 2M5 (reported by top) root@qemuspeed ~/# /etc/rc.d/jail stop alpha Stopping jails: alpha.xps.quip.test. ## ^^^ takes about 1 minute to stop busy jail alpha root@qemuspeed ~/# jls JID IP Address Hostname Path 3 172.16.16.4 gamma.xps.quip.test /usr/jail/gamma 2 172.16.16.3 beta.xps.quip.test /usr/jail/beta root@qemuspeed ~/# ./jtune_all.sh JID Hostname Memory Used / Limit CPU Shares 3 gamma.xps.quip.test 0 M / 20 M 20 JID Hostname Memory Used / Limit CPU Shares 2 beta.xps.quip.test 0 M / 15 M 15 ## try to start Jail again with new jtune parameters: jail -M 5 -S 5 /usr/jail/alpha alpha.xps.quip.test 172.16.16.2 /bin/sh /etc/rc ## it takes more than 2 minutes to start new jail [host system is too busy by other 2 jails running endless loop] root@qemuspeed ~/# ./jtune_all.sh JID Hostname Memory Used / Limit CPU Shares 4 alpha.xps.quip.test 0 M / 5 M 5 JID Hostname Memory Used / Limit CPU Shares 3 gamma.xps.quip.test 0 M / 20 M 20 JID Hostname Memory Used / Limit CPU Shares 2 beta.xps.quip.test 0 M / 15 M 15 ## top after few minutes runnig new jail [seems OK] PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 1368 root 1 170 0 3924K 2692K RUN 19:59 51.76% tcsh 1359 root 1 129 0 3924K 2456K RUN 11:04 34.03% tcsh 2194 root 1 170 0 3788K 1936K RUN 0:18 9.26% tcsh ## now some Mem limit testing ## run dd command in each jail dd if=/dev/zero of=/dev/null bs=16m ## top after few minutes PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 2607 root 1 170 0 34052K 16688K RUN 5:42 43.64% dd 2606 root 1 132 0 34052K 16572K RUN 4:10 36.35% dd 2605 root 1 170 0 34052K 16736K RUN 0:56 12.85% dd ## ^^^ SIZE is 34MB and RES is 16MB even if jtune setting is 5MB for jail aplha and jtune is still reporting 0MB usage: root@qemuspeed ~/# ./jtune_all.sh JID Hostname Memory Used / Limit CPU Shares 5 alpha.xps.quip.test 0 M / 5 M 5 JID Hostname Memory Used / Limit CPU Shares 3 gamma.xps.quip.test 0 M / 20 M 20 JID Hostname Memory Used / Limit CPU Shares 2 beta.xps.quip.test 0 M / 15 M 15 So am I doing something wrong? Is there any useful tutorial / howto / example of jtune usage and useful values? Why sh loops shows same WCPU for each jail (not corresponding to jtune values) but tcsh loops are correct? Are there somebody with some success stories with this patch / good knowlendge to explain some details about my examples? content of my "shortcut scripts": root@qemuspeed ~/# cat jtune_all.sh #!/bin/sh jails=`jls | awk '$1 ~ /[0-9]+/ { print $1 }'` for jid in $jails do jtune -j $jid -i done root@qemuspeed ~/# cat jtune_sysctls.sh #!/bin/sh sysctl kern.sched.limit_jail_cpu=1 sysctl security.jail.limit_jail_memory=1 sysctl kern.sched.system_cpu_shares=10 sysctl security.jail.jail_pager_interval=5 root@qemuspeed ~/# cat jtune_tunes.sh #!/bin/sh jtune -j 1 -m 5 -s 5 jtune -j 2 -m 15 -s 15 jtune -j 3 -m 20 -s 20 jtune -j 1 -i jtune -j 2 -i jtune -j 3 -i root@alpha ~/# cat while.sh #!/bin/sh while [ 1 ] do # nothing done ## or same loop in tcsh shell as while (1) end I know that this loop is not ideal for CPU usage testing but... Miroslav Lachman