From owner-freebsd-hackers@FreeBSD.ORG Sun Feb 6 00:52:12 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5744B106564A for ; Sun, 6 Feb 2011 00:52:12 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id DC6F68FC0C for ; Sun, 6 Feb 2011 00:52:11 +0000 (UTC) Received: by wyf19 with SMTP id 19so3591720wyf.13 for ; Sat, 05 Feb 2011 16:52:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=o488qfcQ4ezAuu2+POnUDzQ2qZTGubw0/fhdywumTcc=; b=rs2Jx4Lqw00ZSjr+qHGTAyoK5XCBNP6HCcRI9jp78F2uY1LNnEQBwjsV3vOtNo8eS4 x8TpYRY6rcB0ysAywgYjo5SfFdghHT64+ji9vSMILDN1bWDFCWXWj7hkHfwthenWHJdV eO0OeGit2BA667OVlvMiFqBf6yLeYid+dzPq8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=dH6t7K1/Es+5i/cOLnzw1GMvkYE0pB8M7a3wMx63sHEdgA6EE3I9zo1TnEyHjvQU4C 1fncb82mQrMx1hzGOCXCo+ane2kSjmF374WDTtES96cKwO403vAbLDCYfB7lcLnY9wgP N6LnzaVN3pDKPHXSNhR/PiToiFkJAxcMKQ5n4= MIME-Version: 1.0 Received: by 10.216.165.85 with SMTP id d63mr12905525wel.12.1296953529662; Sat, 05 Feb 2011 16:52:09 -0800 (PST) Sender: yanegomi@gmail.com Received: by 10.216.71.200 with HTTP; Sat, 5 Feb 2011 16:52:09 -0800 (PST) In-Reply-To: <87fws2rqui.fsf@gmail.com> References: <87vd0ys2no.fsf@gmail.com> <87fws2rqui.fsf@gmail.com> Date: Sat, 5 Feb 2011 16:52:09 -0800 X-Google-Sender-Auth: w_5800DCnfZMwDZst-RJupWtt8c Message-ID: From: Garrett Cooper To: Raphael Kubo da Costa Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Missing stdint.h includes? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Feb 2011 00:52:12 -0000 On Sat, Feb 5, 2011 at 3:42 PM, Raphael Kubo da Costa wr= ote: > Garrett Cooper writes: > >> On Sat, Feb 5, 2011 at 11:27 AM, Raphael Kubo da Costa wrote: >>> Hey there, >>> >>> I was working on some code that used devinfo(3). According to devinfo's >>> man page, #including devinfo.h should be enough to use it. However, a >>> program which only #includes devinfo.h fails due to stdint.h (or >>> sys/types.h) not being included, thus giving me some errors about >>> uint32_t and other types not being defined. >>> >>> A few headers in sys/, such as sys/rman.h, have the same problem. >>> >>> Is it a bug in the headers themselves or are the man pages just >>> incorrect? >> >> =A0 =A0 I'd say it's the manpages probably because a lot of the types we= re >> changed to POSIX integral types, and the manpages weren't updated. >> Thanks! > > Hmm. In the case of sys/rman.h, I see that commits using uint32_t and > other such types are almost 10 years old. Older versions of those types were u_int, u_long, etc. Those are the ones that I was referring to in my last reply. > As for devinfo.h, shouldn't it just include stdint.h, sys/types.h or > sys/param.h? sys/types.h should suffice, even though POSIX says it should be in inttypes.h: http://pubs.opengroup.org/onlinepubs/009695399/basedefs/inttype= s.h.html#tag_13_20 Kind of odd why they're redefined there (in sys/types.h) to be honest, esp because POSIX doesn't say that they should be defined there. Probably some pollution introduced that would widely break compiles because sys/types.h is used everywhere, not inttypes.h. > What's the best way for me to help with this, PR-wise? I would honestly not go create more PR fodder. Just find a (doc?) developer with a commit bit who's interested in cleaning up incorrectness. Thanks! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Sun Feb 6 03:40:01 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 73BE41065670 for ; Sun, 6 Feb 2011 03:40:01 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id F2E4B8FC0A for ; Sun, 6 Feb 2011 03:40:00 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1PlvTj-0002ci-76 for freebsd-hackers@freebsd.org; Sun, 06 Feb 2011 04:39:59 +0100 Received: from 189.61.208.111 ([189.61.208.111]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Feb 2011 04:39:59 +0100 Received: from kubito by 189.61.208.111 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 06 Feb 2011 04:39:59 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Raphael Kubo da Costa Date: Sun, 06 Feb 2011 01:39:43 -0200 Lines: 79 Message-ID: <87sjw1rfuo.fsf@gmail.com> References: <87vd0ys2no.fsf@gmail.com> <87fws2rqui.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 189.61.208.111 User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/23.2 (berkeley-unix) Cancel-Lock: sha1:wXgO0UxjDio+ejcu5b0NBL0xE0E= Subject: Re: Missing stdint.h includes? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Feb 2011 03:40:01 -0000 Garrett Cooper writes: > On Sat, Feb 5, 2011 at 3:42 PM, Raphael Kubo da Costa wrote: >> As for devinfo.h, shouldn't it just include stdint.h, sys/types.h or >> sys/param.h? > > sys/types.h should suffice, even though POSIX says it should be in > inttypes.h: http://pubs.opengroup.org/onlinepubs/009695399/basedefs/inttypes.h.html#tag_13_20 > > Kind of odd why they're redefined there (in sys/types.h) to be honest, > esp because POSIX doesn't say that they should be defined there. > Probably some pollution introduced that would widely break compiles > because sys/types.h is used everywhere, not inttypes.h. > >> What's the best way for me to help with this, PR-wise? > > I would honestly not go create more PR fodder. Just find a (doc?) > developer with a commit bit who's interested in cleaning up > incorrectness. I've inlined two patches below: * The first one should correct the documentation for rman(9). I'm sure there are problems with other headers, but I haven't checked yet. It adds sys/types.h to the man page -- I'm not sure if inttypes.h would be better (DragonFly just adds sys/param.h, FWIW). * The second one makes devinfo.h include sys/types.h instead of sys/_types.h -- I still think devinfo.h users should not need to explicitly include whateverdefinesuint32_t.h, as this, IMHO, is an implementation detail. It also adds the include to devinfo_var.h, which, although not installed, should be self-contained. Feedback very welcome. As a sidenote, what is the purpose of having both types.h and _types.h? Thanks! Patch 1: Index: stable/8/share/man/man9/rman.9 =================================================================== --- stable/8/share/man/man9/rman.9 (revision 218362) +++ stable/8/share/man/man9/rman.9 (working copy) @@ -55,6 +55,7 @@ .Nm rman_get_rid .Nd resource management functions .Sh SYNOPSIS +.In sys/types.h .In sys/rman.h .Ft int .Fn rman_activate_resource "struct resource *r" Patch 2: Index: stable/8/lib/libdevinfo/devinfo.h =================================================================== --- stable/8/lib/libdevinfo/devinfo.h (revision 218362) +++ stable/8/lib/libdevinfo/devinfo.h (working copy) @@ -31,7 +31,7 @@ #define _DEVINFO_H_INCLUDED #include -#include +#include typedef __uintptr_t devinfo_handle_t; #define DEVINFO_ROOT_DEVICE ((devinfo_handle_t)0) Index: stable/8/lib/libdevinfo/devinfo_var.h =================================================================== --- stable/8/lib/libdevinfo/devinfo_var.h (revision 218362) +++ stable/8/lib/libdevinfo/devinfo_var.h (working copy) @@ -27,6 +27,7 @@ * $FreeBSD$ */ +#include #include #include From owner-freebsd-hackers@FreeBSD.ORG Sun Feb 6 03:46:42 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2499A10656C9 for ; Sun, 6 Feb 2011 03:46:42 +0000 (UTC) (envelope-from dieterbsd@engineer.com) Received: from imr-mb01.mx.aol.com (imr-mb01.mx.aol.com [64.12.207.164]) by mx1.freebsd.org (Postfix) with ESMTP id D88888FC1B for ; Sun, 6 Feb 2011 03:46:41 +0000 (UTC) Received: from imo-ma04.mx.aol.com (imo-ma04.mx.aol.com [64.12.78.139]) by imr-mb01.mx.aol.com (8.14.1/8.14.1) with ESMTP id p163aPNs012241; Sat, 5 Feb 2011 22:36:25 -0500 Received: from dieterbsd@engineer.com by imo-ma04.mx.aol.com (mail_out_v42.9.) id n.ebf.681ee34 (34919); Sat, 5 Feb 2011 22:36:24 -0500 (EST) Received: from smtprly-mb01.mx.aol.com (smtprly-mb01.mx.aol.com [64.12.207.148]) by cia-da03.mx.aol.com (v129.8) with ESMTP id MAILCIADA034-5c624d4e17312e3; Sat, 05 Feb 2011 22:36:23 -0500 Received: from web-mmc-m06 (web-mmc-m06.sim.aol.com [64.12.224.139]) by smtprly-mb01.mx.aol.com (v129.8) with ESMTP id MAILSMTPRLYMB018-5c624d4e17312e3; Sat, 05 Feb 2011 22:36:17 -0500 To: freebsd-hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Date: Sat, 05 Feb 2011 22:36:16 -0500 X-AOL-IP: 67.206.161.198 X-MB-Message-Source: WebUI Received: from 67.206.161.198 by web-mmc-m06.sysops.aol.com (64.12.224.139) with HTTP (WebMailUI); Sat, 05 Feb 2011 22:36:16 -0500 MIME-Version: 1.0 From: dieterbsd@engineer.com X-MB-Message-Type: User Content-Type: text/plain; charset="us-ascii" X-Mailer: Mail.com Webmail 33189-STANDARD Message-Id: <8CD93C6255B8920-660-30BBA@web-mmc-m06.sysops.aol.com> X-Spam-Flag: NO X-AOL-SENDER: dieterbsd@engineer.com Cc: freebsd@sopwith.solgatos.com Subject: Why does printf(9) hang network? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Feb 2011 03:46:42 -0000 Why would doing a printf(9) in a device driver (usb, firewire, probably others) cause an obscenely long lockout on /usr/src/sys/kern/uipc_sockbuf.c:148 (sx:so_rcv_sx) ? Printf(9) alone isn't the problem, adding printfs to chown(2) does not cause the problem, but printfs from device drivers do. Grep says that uipc_sockbuf.c is the only file that locks/unlocks sb_sx. The device drivers and printf don't even know that sb_sx exists. 135 int 136 sblock(struct sockbuf *sb, int flags) 137 { 138 139 KASSERT((flags & SBL_VALID) =3D=3D flags, 140 ("sblock: flags invalid (0x%x)", flags)); 141 142 if (flags & SBL_WAIT) { 143 if ((sb->sb_flags & SB_NOINTR) || 144 (flags & SBL_NOINTR)) { 145 sx_xlock(&sb->sb_sx); 146 return (0); 147 } 148 return (sx_xlock_sig(&sb->sb_sx)); 149 } else { 150 if (sx_try_xlock(&sb->sb_sx) =3D=3D 0) 151 return (EWOULDBLOCK); 152 return (0); 153 } 154 } More info at: http://www.freebsd.org/cgi/query-pr.cgi?pr=3D118093 From owner-freebsd-hackers@FreeBSD.ORG Sun Feb 6 06:02:11 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0CB961065670 for ; Sun, 6 Feb 2011 06:02:11 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id DB9128FC0A for ; Sun, 6 Feb 2011 06:02:10 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 3EAE246B17; Sun, 6 Feb 2011 01:02:10 -0500 (EST) Date: Sun, 6 Feb 2011 06:02:10 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: dieterbsd@engineer.com In-Reply-To: <8CD93C6255B8920-660-30BBA@web-mmc-m06.sysops.aol.com> Message-ID: References: <8CD93C6255B8920-660-30BBA@web-mmc-m06.sysops.aol.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-hackers@freebsd.org, freebsd@sopwith.solgatos.com Subject: Re: Why does printf(9) hang network? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Feb 2011 06:02:11 -0000 On Sat, 5 Feb 2011, dieterbsd@engineer.com wrote: > Why would doing a printf(9) in a device driver (usb, firewire, probably > others) cause an obscenely long lockout on > /usr/src/sys/kern/uipc_sockbuf.c:148 (sx:so_rcv_sx) ? > > Printf(9) alone isn't the problem, adding printfs to chown(2) does not cause > the problem, but printfs from device drivers do. > > Grep says that uipc_sockbuf.c is the only file that locks/unlocks sb_sx. The > device drivers and printf don't even know that sb_sx exists. I can't speak to the details of your situation, but one possible explanation might be: printf runs at the speed of the console, which for serious consoles can be extremely slowly. Device driver interrupt threads can preempt other threads, possibly while those threads hold locks. That causes them to hold the locks for much longer, as the threads may not get rescheduled for some period (for example, until the device driver is done doing a printf), leading other threads waiting for that lock to wait significantly longer. Especially the case if the other thread was spinning adaptively, in which case it will then yield since the holder of the lock effectively yielded. You might try forcing all the various threads to run on different CPUs using cpuset and see if the variance goes down. You can also use KTR + schedgraph to explore the specific scheduling going on, although be aware that KTR can also noticeably perturb schediling itself. In general, things shouldn't call kernel printf in steady state operation; if they need to log something, they should use log(9) or similar. printf is primarily a tool for printing out device probe information, and for debugging purposes: it is not intended to be fast. Robert > > 135 int > 136 sblock(struct sockbuf *sb, int flags) > 137 { > 138 > 139 KASSERT((flags & SBL_VALID) == flags, > 140 ("sblock: flags invalid (0x%x)", flags)); > 141 > 142 if (flags & SBL_WAIT) { > 143 if ((sb->sb_flags & SB_NOINTR) || > 144 (flags & SBL_NOINTR)) { > 145 sx_xlock(&sb->sb_sx); > 146 return (0); > 147 } > 148 return (sx_xlock_sig(&sb->sb_sx)); > 149 } else { > 150 if (sx_try_xlock(&sb->sb_sx) == 0) > 151 return (EWOULDBLOCK); > 152 return (0); > 153 } > 154 } > > More info at: http://www.freebsd.org/cgi/query-pr.cgi?pr=118093 > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 7 01:41:25 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9D2781065679; Mon, 7 Feb 2011 01:41:25 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id 205298FC0A; Mon, 7 Feb 2011 01:41:24 +0000 (UTC) Received: from ur.gsoft.com.au (Ur.gsoft.com.au [203.31.81.44]) (authenticated bits=0) by cain.gsoft.com.au (8.14.4/8.14.3) with ESMTP id p171fLwm021331 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 7 Feb 2011 12:11:22 +1030 (CST) (envelope-from doconnor@gsoft.com.au) Mime-Version: 1.0 (Apple Message framework v1082) Content-Type: text/plain; charset=us-ascii From: "Daniel O'Connor" In-Reply-To: <7108E013-77D1-47F8-892E-5027DB7D432B@gsoft.com.au> Date: Mon, 7 Feb 2011 12:11:21 +1030 Content-Transfer-Encoding: quoted-printable Message-Id: <990005CD-39BD-45F6-BD07-ACEE79DF5A03@gsoft.com.au> References: <53A394ED-7C2E-4E4B-A9A7-CB5F1B27DBE3@gsoft.com.au> <7108E013-77D1-47F8-892E-5027DB7D432B@gsoft.com.au> To: "Daniel O'Connor" X-Mailer: Apple Mail (2.1082) X-Spam-Score: -2.51 () ALL_TRUSTED,BAYES_00,T_RP_MATCHES_RCVD X-Scanned-By: MIMEDefang 2.67 on 203.31.81.10 Cc: freebsd-hackers@freebsd.org, Ivan Voras Subject: Re: Scheduler question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Feb 2011 01:41:25 -0000 On 05/02/2011, at 12:43, Daniel O'Connor wrote: > On 05/02/2011, at 11:09, Ivan Voras wrote: >>> It doesn't allocate memory once it's going, everything is = preallocated before the data transfer starts. >>>=20 >>> I'll have a go with mlock() and see what happens. >>=20 >> Did you find anything interesting? >=20 > I'll be looking at it on Monday, I will let you know :) No luck with mlock() so it wouldn't appear to be paging is the issue :( -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 7 02:32:54 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 94FC3106566C for ; Mon, 7 Feb 2011 02:32:54 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 43BE68FC12 for ; Mon, 7 Feb 2011 02:32:54 +0000 (UTC) Received: by qwj9 with SMTP id 9so3155358qwj.13 for ; Sun, 06 Feb 2011 18:32:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:from :date:x-google-sender-auth:message-id:subject:to:cc:content-type; bh=eD6WSZvpyVrVrZze9k/25mDiwx4qGqEGLO+K3eMzCMs=; b=x0ZdnyQPPUabEvxNMxT6WcKDHYqhq8m1dECjGN+4omSfCqIPP5ROjyoA4gcT4MPefA RLeDOInEfFRGutqGcgobVZV4znKaJxxnqJyfzD8yZqHQ155e1VNVSda2Y+tm0btteU1Y mVNT/m6IDM/wTHENEisr7yz2Bd4nO3E6vKSZI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; b=UnzATV9BXp5oAJWe1JSYgOwBYnNaV0RXe71pxANsLjcj53CC9HRylt3a8X+Z9qdYmd /b5/B/kHGwOdK/SH11MJCgCktRTam6baLvPrJR6uk0CMitxkii6m/bgwdphcNLrlrq4I miWNOvSLbegC9s6cgfIEjVhZSehmVbVbm6RtE= Received: by 10.229.97.134 with SMTP id l6mr5117314qcn.253.1297045973468; Sun, 06 Feb 2011 18:32:53 -0800 (PST) MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.229.67.2 with HTTP; Sun, 6 Feb 2011 18:32:13 -0800 (PST) In-Reply-To: <990005CD-39BD-45F6-BD07-ACEE79DF5A03@gsoft.com.au> References: <53A394ED-7C2E-4E4B-A9A7-CB5F1B27DBE3@gsoft.com.au> <7108E013-77D1-47F8-892E-5027DB7D432B@gsoft.com.au> <990005CD-39BD-45F6-BD07-ACEE79DF5A03@gsoft.com.au> From: Ivan Voras Date: Mon, 7 Feb 2011 03:32:13 +0100 X-Google-Sender-Auth: CZ1pLjwbWIN8StuviUdkC1pIyJo Message-ID: To: "Daniel O'Connor" Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org Subject: Re: Scheduler question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Feb 2011 02:32:54 -0000 On 7 February 2011 02:41, Daniel O'Connor wrote: > > On 05/02/2011, at 12:43, Daniel O'Connor wrote: >> On 05/02/2011, at 11:09, Ivan Voras wrote: >>>> It doesn't allocate memory once it's going, everything is preallocated before the data transfer starts. >>>> >>>> I'll have a go with mlock() and see what happens. >>> >>> Did you find anything interesting? >> >> I'll be looking at it on Monday, I will let you know :) > > No luck with mlock() so it wouldn't appear to be paging is the issue :( I'm also interested in raw device vs file system access! From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 7 03:12:47 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7AC2E1065670; Mon, 7 Feb 2011 03:12:47 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id F0BC78FC08; Mon, 7 Feb 2011 03:12:46 +0000 (UTC) Received: from ur.gsoft.com.au (Ur.gsoft.com.au [203.31.81.44]) (authenticated bits=0) by cain.gsoft.com.au (8.14.4/8.14.3) with ESMTP id p173CiOG028558 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 7 Feb 2011 13:42:44 +1030 (CST) (envelope-from doconnor@gsoft.com.au) Mime-Version: 1.0 (Apple Message framework v1082) Content-Type: text/plain; charset=us-ascii From: "Daniel O'Connor" In-Reply-To: Date: Mon, 7 Feb 2011 13:42:43 +1030 Content-Transfer-Encoding: quoted-printable Message-Id: <772B352C-7241-4326-8B49-3FB675896609@gsoft.com.au> References: <53A394ED-7C2E-4E4B-A9A7-CB5F1B27DBE3@gsoft.com.au> <7108E013-77D1-47F8-892E-5027DB7D432B@gsoft.com.au> <990005CD-39BD-45F6-BD07-ACEE79DF5A03@gsoft.com.au> To: Ivan Voras X-Mailer: Apple Mail (2.1082) X-Spam-Score: -2.51 () ALL_TRUSTED,BAYES_00,T_RP_MATCHES_RCVD X-Scanned-By: MIMEDefang 2.67 on 203.31.81.10 Cc: freebsd-hackers@freebsd.org Subject: Re: Scheduler question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Feb 2011 03:12:47 -0000 On 07/02/2011, at 13:02, Ivan Voras wrote: >>> I'll be looking at it on Monday, I will let you know :) >>=20 >> No luck with mlock() so it wouldn't appear to be paging is the issue = :( >=20 > I'm also interested in raw device vs file system access! Oops, sorry.. I just tried that now but it doesn't improve things :( I am writing directly to /dev/ad10 but stressing /dev/ad14 (sudo tar -cf = /dev/null /local0) It is interesting also that if I have md5's soaking up CPU then it's = much less likely to start streaming properly and generally bombs out = straight away. If I start it streaming and then start md5 it stays = running... (even if it's rtprio'd) -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 7 10:38:24 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 79999106567A for ; Mon, 7 Feb 2011 10:38:24 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 304128FC14 for ; Mon, 7 Feb 2011 10:38:23 +0000 (UTC) Received: by gyf3 with SMTP id 3so1722638gyf.13 for ; Mon, 07 Feb 2011 02:38:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:from :date:x-google-sender-auth:message-id:subject:to:cc:content-type; bh=8y/JR3Sf21zpSgHgBGhdmGZxm/xbc6vRYk+c/7GlvcA=; b=j08LzjbmU9LDfI02d1eE6zOIVXO9oQAsHyZ5Zg6cwT4rxH8aP7bDtAm44Azig6jMKf sJuJcQ61fEpulkm9UYODj5x5zQ7RKmsgeGPPsyfHeJ+kVBFEr5m5k3YpBya2axdSPtrt HSefLhfEBCMfizUGoC8mzTUFFv/9Z8XR9JgY0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; b=Yh6EnrTpzBxJAinHQQzfVE69pq4/9ZBZwILbxjD/RRVAFrf60IDnnaDkrpJy3XwlZo 5mSKp20uBj0Em3rh6AdVedveaWJKZKRdoLRUi/NiC+IAkTV3yJtOd2xUTtMwOYb9w1Qt 5K/POw7AXQk/i/ceTao5GTjm5rzLumBdKRtPc= Received: by 10.236.111.19 with SMTP id v19mr4669739yhg.86.1297075103282; Mon, 07 Feb 2011 02:38:23 -0800 (PST) MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.151.11.12 with HTTP; Mon, 7 Feb 2011 02:37:43 -0800 (PST) In-Reply-To: <772B352C-7241-4326-8B49-3FB675896609@gsoft.com.au> References: <53A394ED-7C2E-4E4B-A9A7-CB5F1B27DBE3@gsoft.com.au> <7108E013-77D1-47F8-892E-5027DB7D432B@gsoft.com.au> <990005CD-39BD-45F6-BD07-ACEE79DF5A03@gsoft.com.au> <772B352C-7241-4326-8B49-3FB675896609@gsoft.com.au> From: Ivan Voras Date: Mon, 7 Feb 2011 11:37:43 +0100 X-Google-Sender-Auth: r9lJ6ru3t5laPYHGQt-IN1suZyo Message-ID: To: "Daniel O'Connor" Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org Subject: Re: Scheduler question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Feb 2011 10:38:24 -0000 On 07/02/2011 04:12, Daniel O'Connor wrote: > > On 07/02/2011, at 13:02, Ivan Voras wrote: >>>> I'll be looking at it on Monday, I will let you know :) >>> >>> No luck with mlock() so it wouldn't appear to be paging is the issue :( >> >> I'm also interested in raw device vs file system access! > > Oops, sorry.. I just tried that now but it doesn't improve things :( Meaning: you still get jitter? > I am writing directly to /dev/ad10 but stressing /dev/ad14 (sudo tar -cf /dev/null /local0) Can you do only one of those things? I.e. leave all the file systems alone and just do something like 'diskinfo -vt /dev/ad14'? From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 7 12:38:26 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AA0D71065694; Mon, 7 Feb 2011 12:38:26 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id 2930D8FC19; Mon, 7 Feb 2011 12:38:25 +0000 (UTC) Received: from ur.dons.net.au (ppp203-122-198-229.lns6.adl6.internode.on.net [203.122.198.229]) (authenticated bits=0) by cain.gsoft.com.au (8.14.4/8.14.3) with ESMTP id p17CcMTT080140 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 7 Feb 2011 23:08:23 +1030 (CST) (envelope-from doconnor@gsoft.com.au) Mime-Version: 1.0 (Apple Message framework v1082) Content-Type: text/plain; charset=us-ascii From: "Daniel O'Connor" In-Reply-To: Date: Mon, 7 Feb 2011 23:08:21 +1030 Content-Transfer-Encoding: quoted-printable Message-Id: <71549325-5FD1-4516-B49E-098AFBEFE8B7@gsoft.com.au> References: <53A394ED-7C2E-4E4B-A9A7-CB5F1B27DBE3@gsoft.com.au> <7108E013-77D1-47F8-892E-5027DB7D432B@gsoft.com.au> <990005CD-39BD-45F6-BD07-ACEE79DF5A03@gsoft.com.au> <772B352C-7241-4326-8B49-3FB675896609@gsoft.com.au> To: Ivan Voras X-Mailer: Apple Mail (2.1082) X-Spam-Score: 0.163 () BAYES_00,RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.67 on 203.31.81.10 Cc: freebsd-hackers@freebsd.org Subject: Re: Scheduler question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Feb 2011 12:38:26 -0000 On 07/02/2011, at 21:07, Ivan Voras wrote: >>> I'm also interested in raw device vs file system access! >>=20 >> Oops, sorry.. I just tried that now but it doesn't improve things :( >=20 > Meaning: you still get jitter? Yes, well I didn't measure the read frequency but it dropped out = (stopped streaming due to a full FIFO) no less often. >> I am writing directly to /dev/ad10 but stressing /dev/ad14 (sudo tar = -cf /dev/null /local0) >=20 > Can you do only one of those things? I.e. leave all the file systems > alone and just do something like 'diskinfo -vt /dev/ad14'? OK, I wrote the data to /dev/null from USB and ran diskutil in a loop = and it doesn't drop out. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 7 13:07:03 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56DDA1065670 for ; Mon, 7 Feb 2011 13:07:03 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 0AF5E8FC08 for ; Mon, 7 Feb 2011 13:07:02 +0000 (UTC) Received: by ywl2 with SMTP id 2so88400ywl.13 for ; Mon, 07 Feb 2011 05:07:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:from :date:x-google-sender-auth:message-id:subject:to:cc:content-type; bh=atOqE0HIKNQUilowWQkoxLfxMXqDOrUbJM0l82QEInk=; b=fi91E7b4g1ZTjZnG5WUbo6CYia4Bvoz1k9VI5jNn4VKcWmuGUvmO5SUePoFajex8sb iBDGUwJnk7WCFA41TYcyQUTkT1ih5EPFwHBdefo9J/qswNRd4s/ENWa+RVnyIEnKyKRA irVc6vYiso0W7Vfla+ounA8aWQb0Dn4llBAmg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; b=d+a+gXSSskACA/8PaEP+hEfr5lMdarzGszDSIASFrT/nz/JSzrEvZ8ypbz11QzkeLO We6WWJ2EkbhII3i4d+GkLG0zy/dyt1eCHmQQ2q3rVIETmIz3Jg7g2PSlY7dwCGzcKyOe O4E9E4H/TO8OYKqyBW0QS5GxxPnr42RQy24T4= Received: by 10.90.94.7 with SMTP id r7mr19569442agb.45.1297084022304; Mon, 07 Feb 2011 05:07:02 -0800 (PST) MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.151.11.12 with HTTP; Mon, 7 Feb 2011 05:06:22 -0800 (PST) In-Reply-To: <71549325-5FD1-4516-B49E-098AFBEFE8B7@gsoft.com.au> References: <53A394ED-7C2E-4E4B-A9A7-CB5F1B27DBE3@gsoft.com.au> <7108E013-77D1-47F8-892E-5027DB7D432B@gsoft.com.au> <990005CD-39BD-45F6-BD07-ACEE79DF5A03@gsoft.com.au> <772B352C-7241-4326-8B49-3FB675896609@gsoft.com.au> <71549325-5FD1-4516-B49E-098AFBEFE8B7@gsoft.com.au> From: Ivan Voras Date: Mon, 7 Feb 2011 14:06:22 +0100 X-Google-Sender-Auth: o__Cfm2Shgl9fghPD4hkLGZI-ak Message-ID: To: "Daniel O'Connor" Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org Subject: Re: Scheduler question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Feb 2011 13:07:03 -0000 On 7 February 2011 13:38, Daniel O'Connor wrote: >>> I am writing directly to /dev/ad10 but stressing /dev/ad14 (sudo tar -cf /dev/null /local0) >> >> Can you do only one of those things? I.e. leave all the file systems >> alone and just do something like 'diskinfo -vt /dev/ad14'? > > OK, I wrote the data to /dev/null from USB and ran diskutil in a loop and it doesn't drop out. Maybe I misunderstood you and it's a different problem than what I was experiencing; is this a better description of your problem: 1) you have a program communicating with a USB device 2) it reads from the device and writes to a file 3) you experience stalls when you write the data recived from the USB device to the file but only if the file system you're writing on is also loaded by something else - heavy reads? ? From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 7 13:44:30 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD78B1065673; Mon, 7 Feb 2011 13:44:30 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id 276868FC21; Mon, 7 Feb 2011 13:44:29 +0000 (UTC) Received: from ur.dons.net.au (ppp203-122-198-229.lns6.adl6.internode.on.net [203.122.198.229]) (authenticated bits=0) by cain.gsoft.com.au (8.14.4/8.14.3) with ESMTP id p17DiN1D082826 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 8 Feb 2011 00:14:24 +1030 (CST) (envelope-from doconnor@gsoft.com.au) Mime-Version: 1.0 (Apple Message framework v1082) Content-Type: text/plain; charset=us-ascii From: "Daniel O'Connor" In-Reply-To: Date: Tue, 8 Feb 2011 00:14:23 +1030 Content-Transfer-Encoding: quoted-printable Message-Id: <88A9D148-7E3C-4ADB-90FC-B95C4D3BBD2E@gsoft.com.au> References: <53A394ED-7C2E-4E4B-A9A7-CB5F1B27DBE3@gsoft.com.au> <7108E013-77D1-47F8-892E-5027DB7D432B@gsoft.com.au> <990005CD-39BD-45F6-BD07-ACEE79DF5A03@gsoft.com.au> <772B352C-7241-4326-8B49-3FB675896609@gsoft.com.au> <71549325-5FD1-4516-B49E-098AFBEFE8B7@gsoft.com.au> To: Ivan Voras X-Mailer: Apple Mail (2.1082) X-Spam-Score: 0.163 () BAYES_00,RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.67 on 203.31.81.10 Cc: freebsd-hackers@freebsd.org Subject: Re: Scheduler question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Feb 2011 13:44:30 -0000 On 07/02/2011, at 23:36, Ivan Voras wrote: >> OK, I wrote the data to /dev/null from USB and ran diskutil in a loop = and it doesn't drop out. >=20 > Maybe I misunderstood you and it's a different problem than what I was > experiencing; is this a better description of your problem: >=20 > 1) you have a program communicating with a USB device > 2) it reads from the device and writes to a file > 3) you experience stalls when you write the data recived from the USB > device to the file but only if the file system you're writing on is > also loaded by something else - heavy reads? >=20 > ? Yes, however CPU loading also seems to affect it. Unfortunately I don't have a useful measurement to show the problem - ie = I don't have a metric which correlates with the hardware FIFO filling = up. This makes the testing rather annoying :) -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 7 15:11:05 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D6538106564A; Mon, 7 Feb 2011 15:11:05 +0000 (UTC) (envelope-from pluknet@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 526698FC19; Mon, 7 Feb 2011 15:11:05 +0000 (UTC) Received: by qwj9 with SMTP id 9so3475223qwj.13 for ; Mon, 07 Feb 2011 07:11:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=XryZCvgso1IzruhvkTCma8LWRtTaGTsguDVPkpyTxhY=; b=pmWbrmZIQFqbbrt+wryRjUZCUIDhsAWxPGVqOHSt9/92nOymAqcjkvZ8mQNlYO9ZBM l0YR9k62XxLM0xICAxvVYEmoYQJ8LUDGeC6RaPqLFAvRedRaYZ6hEkE1rlhEgfFtbnxw 1R7YVqvIUuHYj0MO4OIEani9FOUPRT37K4oQ8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=nJIaozY8Ub2c6PtofQ/QCxI93LA6MLWwgdDXJAIA0gbAsppQz8uGKX4oZSDe6VgrTz PU68jsWRJC3C4DAcrabxvxtX3DLNehMWTXPGURWjr3DuIBv2GbBi1OcTBxv0CTxkj5MU Lx6TsdbjR4DskRrvI7OXhYDK8DH7bHDjJdhTY= MIME-Version: 1.0 Received: by 10.229.240.66 with SMTP id kz2mr11185202qcb.233.1297091463554; Mon, 07 Feb 2011 07:11:03 -0800 (PST) Received: by 10.229.102.87 with HTTP; Mon, 7 Feb 2011 07:11:03 -0800 (PST) In-Reply-To: References: <201101211244.13830.jhb@freebsd.org> Date: Mon, 7 Feb 2011 18:11:03 +0300 Message-ID: From: Sergey Kandaurov To: alc@freebsd.org Content-Type: multipart/mixed; boundary=0016363b8848ed2be3049bb2a3d8 Cc: freebsd-hackers@freebsd.org, Konstantin Belousov Subject: Re: [rfc] allow to boot with >= 256GB physmem X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Feb 2011 15:11:05 -0000 --0016363b8848ed2be3049bb2a3d8 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 22 January 2011 00:43, Alan Cox wrote: > On Fri, Jan 21, 2011 at 2:58 PM, Alan Cox wrote: >> >> On Fri, Jan 21, 2011 at 11:44 AM, John Baldwin wrote: >>> >>> On Friday, January 21, 2011 11:09:10 am Sergey Kandaurov wrote: >>> > Hello. >>> > >>> > Some time ago I faced with a problem booting with 400GB physmem. >>> > The problem is that vm.max_proc_mmap type overflows with >>> > such high value, and that results in a broken mmap() syscall. >>> > The max_proc_mmap value is a signed int and roughly calculated >>> > at vmmapentry_rsrc_init() as u_long vm_kmem_size quotient: >>> > vm_kmem_size / sizeof(struct vm_map_entry) / 100. >>> > >>> > Although at the time it was introduced at svn r57263 the value >>> > was quite low (f.e. the related commit log stands: >>> > "The value defaults to around 9000 for a 128MB machine."), >>> > the problem is observed on amd64 where KVA space after >>> > r212784 is factually bound to the only physical memory size. >>> > >>> > With INT_MAX here is 0x7fffffff, and sizeof(struct vm_map_entry) >>> > is 120, it's enough to have sligthly less than 256GB to be able >>> > to reproduce the problem. >>> > >>> > I rewrote vmmapentry_rsrc_init() to set large enough limit for >>> > max_proc_mmap just to protect from integer type overflow. >>> > As it's also possible to live tune this value, I also added a >>> > simple anti-shoot constraint to its sysctl handler. >>> > I'm not sure though if it's worth to commit the second part. >>> > >>> > As this patch may cause some bikeshedding, >>> > I'd like to hear your comments before I will commit it. >>> > >>> > http://plukky.net/~pluknet/patches/max_proc_mmap.diff >>> >>> Is there any reason we can't just make this variable and sysctl a long? >>> >> >> Or just delete it. >> >> 1. Contrary to what the commit message says, this sysctl does not >> effectively limit the number of vm map entries.=A0 It only limits the nu= mber >> that are created by one system call, mmap().=A0 Other system calls creat= e vm >> map entries just as easily, for example, mprotect(), madvise(), mlock(),= and >> minherit().=A0 Basically, anything that alters the properties of a mappi= ng. >> Thus, in 2000, after this sysctl was added, the same resource exhaustion >> induced crash could have been reproduced by trivially changing the progr= am >> in PR/16573 to do an mprotect() or two. >> >> In a nutshell, if you want to really limit the number of vm map entries >> that a process can allocate, the implementation is a bit more involved t= han >> what was done for this sysctl. >> >> 2. UMA implements M_WAITOK, whereas the old zone allocator in 2000 did >> not.=A0 Moreover, vm map entries for user maps are allocated with M_WAIT= OK. >> So, the exact crash reported in PR/16573 couldn't happen any longer. >> > > Actually, I take back part of what I said here.=A0 The old zone allocator= did > implement something like M_WAITOK, and that appears to have been used for > user maps.=A0 However, the crash described in PR/16573 was actually on th= e > allocation of a vm map entry within the *kernel* address space for a proc= ess > U area.=A0 This type of allocation did not use the old zone allocator's > equivalent to M_WAITOK.=A0 However, we no longer have U areas, so the exa= ct > crash scenario is clearly no longer possible.=A0 Interestingly, the sysct= l in > question has no direct effect on the allocation of kernel vm map entries. > > So, I remain skeptical that this sysctl is preventing any resource > exhaustion based panics in the current kernel.=A0 Again, I would be thril= led > to see one or more people do some testing, such as rerunning the program > from PR/16573. > > >> 3. We now have the "vmemoryuse" resource limit.=A0 When this sysctl was >> defined, we didn't.=A0 Limiting the virtual memory indirectly but effect= ively >> limits the number of vm map entries that a process can allocate. >> >> In summary, I would do a little due diligence, for example, run the >> program from PR/16573 with the limit disabled.=A0 If you can't reproduce= the >> crash, in other words, nothing contradicts point #2 above, then I would = just >> delete this sysctl. >> I tried the test from PR/16573 running as root. If unmodified it just quick= ly bounds on kern.maxproc limit. So, I added signal(SIGCHLD, SIG_IGN); to not create zombie processes at all to give it more workload. With this change i= t also survived. Submitter reported that it crashes with 10000 iterations. After increasing the limit up to 1000000 I still couldn't get it to crash. * The testing was done with commented out max_proc_mmap part. The change effectively reverts r57263. --=20 wbr, pluknet --0016363b8848ed2be3049bb2a3d8 Content-Type: application/octet-stream; name="vm_mmap_maxprocmmap.diff" Content-Disposition: attachment; filename="vm_mmap_maxprocmmap.diff" Content-Transfer-Encoding: base64 X-Attachment-Id: f_gjviptnp0 SW5kZXg6IC9zeXMvdm0vdm1fbW1hcC5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIC9zeXMvdm0vdm1fbW1hcC5j CShyZXZpc2lvbiAyMTgwMjYpCisrKyAvc3lzL3ZtL3ZtX21tYXAuYwkod29ya2luZyBjb3B5KQpA QCAtNDgsNyArNDgsNiBAQAogCiAjaW5jbHVkZSA8c3lzL3BhcmFtLmg+CiAjaW5jbHVkZSA8c3lz L3N5c3RtLmg+Ci0jaW5jbHVkZSA8c3lzL2tlcm5lbC5oPgogI2luY2x1ZGUgPHN5cy9sb2NrLmg+ CiAjaW5jbHVkZSA8c3lzL211dGV4Lmg+CiAjaW5jbHVkZSA8c3lzL3N5c3Byb3RvLmg+CkBAIC02 Niw3ICs2NSw2IEBACiAjaW5jbHVkZSA8c3lzL3N0YXQuaD4KICNpbmNsdWRlIDxzeXMvc3lzZW50 Lmg+CiAjaW5jbHVkZSA8c3lzL3ZtbWV0ZXIuaD4KLSNpbmNsdWRlIDxzeXMvc3lzY3RsLmg+CiAK ICNpbmNsdWRlIDxzZWN1cml0eS9tYWMvbWFjX2ZyYW1ld29yay5oPgogCkBAIC04MCw3ICs3OCw2 IEBACiAjaW5jbHVkZSA8dm0vdm1fcGFnZW91dC5oPgogI2luY2x1ZGUgPHZtL3ZtX2V4dGVybi5o PgogI2luY2x1ZGUgPHZtL3ZtX3BhZ2UuaD4KLSNpbmNsdWRlIDx2bS92bV9rZXJuLmg+CiAKICNp ZmRlZiBIV1BNQ19IT09LUwogI2luY2x1ZGUgPHN5cy9wbWNrZXJuLmg+CkBAIC05MiwzMCArODks NiBAQAogfTsKICNlbmRpZgogCi1zdGF0aWMgaW50IG1heF9wcm9jX21tYXA7Ci1TWVNDVExfSU5U KF92bSwgT0lEX0FVVE8sIG1heF9wcm9jX21tYXAsIENUTEZMQUdfUlcsICZtYXhfcHJvY19tbWFw LCAwLAotICAgICJNYXhpbXVtIG51bWJlciBvZiBtZW1vcnktbWFwcGVkIGZpbGVzIHBlciBwcm9j ZXNzIik7Ci0KLS8qCi0gKiBTZXQgdGhlIG1heGltdW0gbnVtYmVyIG9mIHZtX21hcF9lbnRyeSBz dHJ1Y3R1cmVzIHBlciBwcm9jZXNzLiAgUm91Z2hseQotICogc3BlYWtpbmcgdm1fbWFwX2VudHJ5 IHN0cnVjdHVyZXMgYXJlIHRpbnksIHNvIGFsbG93aW5nIHRoZW0gdG8gZWF0IDEvMTAwCi0gKiBv ZiBvdXIgS1ZNIG1hbGxvYyBzcGFjZSBzdGlsbCByZXN1bHRzIGluIGdlbmVyb3VzIGxpbWl0cy4g IFdlIHdhbnQgYQotICogZGVmYXVsdCB0aGF0IGlzIGdvb2QgZW5vdWdoIHRvIHByZXZlbnQgdGhl IGtlcm5lbCBydW5uaW5nIG91dCBvZiByZXNvdXJjZXMKLSAqIGlmIGF0dGFja2VkIGZyb20gY29t cHJvbWlzZWQgdXNlciBhY2NvdW50IGJ1dCBnZW5lcm91cyBlbm91Z2ggc3VjaCB0aGF0Ci0gKiBt dWx0aS10aHJlYWRlZCBwcm9jZXNzZXMgYXJlIG5vdCB1bmR1bHkgaW5jb252ZW5pZW5jZWQuCi0g Ki8KLXN0YXRpYyB2b2lkIHZtbWFwZW50cnlfcnNyY19pbml0KHZvaWQgKik7Ci1TWVNJTklUKHZt bWVyc3JjLCBTSV9TVUJfS1ZNX1JTUkMsIFNJX09SREVSX0ZJUlNULCB2bW1hcGVudHJ5X3JzcmNf aW5pdCwKLSAgICBOVUxMKTsKLQotc3RhdGljIHZvaWQKLXZtbWFwZW50cnlfcnNyY19pbml0KGR1 bW15KQotICAgICAgICB2b2lkICpkdW1teTsKLXsKLSAgICBtYXhfcHJvY19tbWFwID0gdm1fa21l bV9zaXplIC8gc2l6ZW9mKHN0cnVjdCB2bV9tYXBfZW50cnkpOwotICAgIG1heF9wcm9jX21tYXAg Lz0gMTAwOwotfQotCiBzdGF0aWMgaW50IHZtX21tYXBfdm5vZGUoc3RydWN0IHRocmVhZCAqLCB2 bV9zaXplX3QsIHZtX3Byb3RfdCwgdm1fcHJvdF90ICosCiAgICAgaW50ICosIHN0cnVjdCB2bm9k ZSAqLCB2bV9vb2Zmc2V0X3QgKiwgdm1fb2JqZWN0X3QgKik7CiBzdGF0aWMgaW50IHZtX21tYXBf Y2RldihzdHJ1Y3QgdGhyZWFkICosIHZtX3NpemVfdCwgdm1fcHJvdF90LCB2bV9wcm90X3QgKiwK QEAgLTM3NywxOCArMzUwLDYgQEAKIAkJaGFuZGxlX3R5cGUgPSBPQkpUX1ZOT0RFOwogCX0KIG1h cDoKLQotCS8qCi0JICogRG8gbm90IGFsbG93IG1vcmUgdGhlbiBhIGNlcnRhaW4gbnVtYmVyIG9m IHZtX21hcF9lbnRyeSBzdHJ1Y3R1cmVzCi0JICogcGVyIHByb2Nlc3MuICBTY2FsZSB3aXRoIHRo ZSBudW1iZXIgb2YgcmZvcmtzIHNoYXJpbmcgdGhlIG1hcAotCSAqIHRvIG1ha2UgdGhlIGxpbWl0 IHJlYXNvbmFibGUgZm9yIHRocmVhZHMuCi0JICovCi0JaWYgKG1heF9wcm9jX21tYXAgJiYKLQkg ICAgdm1zLT52bV9tYXAubmVudHJpZXMgPj0gbWF4X3Byb2NfbW1hcCAqIHZtcy0+dm1fcmVmY250 KSB7Ci0JCWVycm9yID0gRU5PTUVNOwotCQlnb3RvIGRvbmU7Ci0JfQotCiAJdGQtPnRkX2Zwb3Ag PSBmcDsKIAllcnJvciA9IHZtX21tYXAoJnZtcy0+dm1fbWFwLCAmYWRkciwgc2l6ZSwgcHJvdCwg bWF4cHJvdCwKIAkgICAgZmxhZ3MsIGhhbmRsZV90eXBlLCBoYW5kbGUsIHBvcyk7Cg== --0016363b8848ed2be3049bb2a3d8-- From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 7 20:27:53 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 706AC106564A for ; Mon, 7 Feb 2011 20:27:53 +0000 (UTC) (envelope-from anti_spamsys@yahoo.com) Received: from nm7.bullet.mail.sp2.yahoo.com (nm7.bullet.mail.sp2.yahoo.com [98.139.91.77]) by mx1.freebsd.org (Postfix) with SMTP id 4F4A78FC16 for ; Mon, 7 Feb 2011 20:27:53 +0000 (UTC) Received: from [98.139.91.67] by nm7.bullet.mail.sp2.yahoo.com with NNFMP; 07 Feb 2011 20:15:14 -0000 Received: from [98.139.91.13] by tm7.bullet.mail.sp2.yahoo.com with NNFMP; 07 Feb 2011 20:15:14 -0000 Received: from [127.0.0.1] by omp1013.mail.sp2.yahoo.com with NNFMP; 07 Feb 2011 20:15:14 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 61922.74855.bm@omp1013.mail.sp2.yahoo.com Received: (qmail 68645 invoked by uid 60001); 7 Feb 2011 20:15:13 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1297109713; bh=BbSHT+j6zutImzoMEkRd2H1lbwCfpWpw8PGIbKyL7cQ=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type; b=LeyYTMHwAOlWfxyBUAp8Fk/WoiPuE7jLK8QErWanPgIJgh6ik8/9wNyVjELvKMhVEk4xi2KEyyOslqNejTmYqLXvDlhVIAwnVXqWkuAisHWhmmtmChT026nP/TKnhvCH3ZkHii5ydzQERbQGYCElOGuByRnHQIIOHZQFGz8x1+8= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type; b=R9VfAroozWTCjkq1F9aZGPIGgt5TB6dM7cQCHNALWazQRZSTlbi4vdQBjEIsvHKqsT+cg8PvO3n5Eua8014bHBSM3VAqgw+feT5VW14ymMEF4+aE1S4pP9GaPQGTsmUX4dUyIAWEHGhBoxUP2EXYys/QMbVuHntLJxbFQ9Ev+LQ=; Message-ID: <681598.66709.qm@web113216.mail.gq1.yahoo.com> X-YMail-OSG: 2ZmS0fIVM1kh2l9G4zzypCbnhypWwJsEXi7P0J7tOOQXtZS XIMJ8lYZAV8anCIgrq2p_LovMP6wbgICBvvq1kJE7GGgrOCPM4QHIG4qihHH ASQOJTAufPmHMF0jRIdk3inTC9KOHMkbixmXuDJyO9dcPzEl0Soal.1gkko1 9xOOJjUaH1Bds1CJuAZR68wO20LfaWZJZY1V85Ewj2HlbqpRcexI20yNYsCH GnT84CXuX14iP86C27vejXylYZnTt7cNoActJgM0k6qZVaVHZY_rF0MKCVVG fu7l6lwiprMUb432NlTcxUzi0iRVncisxak15hSiEEXQIGBTj6I9DFFtHkTB bzgXFD6vC3KxBFuxqhQiDDWulPo1o0qfxQxGBc8FCENxlhBffOkUbfw-- Received: from [128.55.19.49] by web113216.mail.gq1.yahoo.com via HTTP; Mon, 07 Feb 2011 12:15:13 PST X-Mailer: YahooMailClassic/11.4.20 YahooMailWebService/0.8.108.291010 Date: Mon, 7 Feb 2011 12:15:13 -0800 (PST) From: Trever To: freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Subject: boot0sio waits a minute X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Feb 2011 20:27:53 -0000 Does anyone know why boot0sio would wait about 1 minute before proceeding? If this is a known issue I can't find anything about it. I know there is a bug with boot0cfg not changing the partition to boot to. Don't know if this is related. FreeBSD 8.2RC2 i386. Boot blocks were compiled with speed 115200. Installed via nanobsd to a compact flash card on an Habey 6620II Z530, but there doesn't seem to be anything wrong other than with boot0sio, or something with how nanobsd installs boot0sio (NOT boot0- that works). But I have checked what nanobsd is doing, that seems ruled out, seems fine. BIOS seems ruled out. Seems like boot0sio. Everything works in all manner of configurations, just this very annoying long "hang" before boot0sio runs. Again, boot0 does not do this. I'm not talking about the pause after it displays the boot partition selection options. It's before anything happens. I get a blinking underscore in the upper left hand corner of my screen (and nothing on the serial port) for a very long minute before the system boots. Doesn't matter if I have serial attached or not, etc.. Thanks! Trever From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 7 22:21:51 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EA4A5106564A for ; Mon, 7 Feb 2011 22:21:51 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 97BAF8FC18 for ; Mon, 7 Feb 2011 22:21:51 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id p17MHKcw063469 for ; Mon, 7 Feb 2011 15:17:20 -0700 (MST) (envelope-from imp@bsdimp.com) Message-ID: <4D506F70.8090405@bsdimp.com> Date: Mon, 07 Feb 2011 15:17:20 -0700 From: Warner Losh User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20101211 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <20110113202848.GI2518@deviant.kiev.zoral.com.ua> <4D2F8BFE.9070607@bsdimp.com> <20110114082840.GM2518@deviant.kiev.zoral.com.ua> <4D307D0F.7000008@bsdimp.com> In-Reply-To: <4D307D0F.7000008@bsdimp.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: What does the FreeBSD/i386 ABI say about stack alignment? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Feb 2011 22:21:52 -0000 On 01/14/2011 09:42, Warner Losh wrote: > On 01/14/2011 01:28, Kostik Belousov wrote: >> On Thu, Jan 13, 2011 at 04:34:22PM -0700, Warner Losh wrote: >>> On 01/13/2011 13:28, Kostik Belousov wrote: >>>> On Thu, Jan 13, 2011 at 12:19:00PM -0500, Ryan Stone wrote: >>>>> I've been trying to get an application compiled with gcc 4.5.1 >>>>> running >>>>> on FreeBSD 8.1, but it's been crashing during startup with a SIGBUS. >>>>> It turns out that the problem is that gcc is issuing SSE >>>>> instructions(in my case, a movdqa) that assume that the stack will be >>>>> aligned to a 16-byte boundary. It seems that Linux/i386 guarantees >>>>> this, and I worry that gcc has extended this assumption to all i386 >>>>> architectures. I'm assuming that FreeBSD doesn't make any such >>>>> promises based on the fact that I'm getting crashes. >>>>> >>>>> There does seem to be a flag (-mstackrealign) that you can set to >>>>> force gcc to align the stack to what it wants, but that pessimizes >>>>> the >>>>> generated code a bit. Some googling would seem to indicate that >>>>> -mpreferred-stack-boundary won't always handle this problem >>>>> correctly. >>>>> >>>>> Any ideas? My inclination, at least for our local source tree >>>>> here at >>>>> $WORK, would be to accommodate gcc and guarantee the stack alignment >>>>> that it wants rather than pessimize our application. It seems we >>>>> have >>>>> an old local patch/hack in our FreeBSD 6.1 tree(apparently based on >>>>> this: >>>>> http://www.freebsd.org/cgi/getmsg.cgi?fetch=438552+0+/usr/local/www/db/text/2000/freebsd-current/20000507.freebsd-current). >>>>> >>>>> I believe that this patch is the reason why we haven't seen the >>>>> problem when running on 6.1, but the patch doesn't seem to work >>>>> anymore on 8.1. >>>> Look at lib/csu/i386-elf/crt1_s.S, we align stack on startup. >>>> My understanding is that the requirement is (%esp& 0xf) == 0 just >>>> before >>>> the call to the function. And we are off by 4 (this is my fault). >>>> >>>> Please give this a try. >>>> >>>> diff --git a/lib/csu/i386-elf/crt1_s.S b/lib/csu/i386-elf/crt1_s.S >>>> index d7ed0a2..17ac0e3 100644 >>>> --- a/lib/csu/i386-elf/crt1_s.S >>>> +++ b/lib/csu/i386-elf/crt1_s.S >>>> @@ -42,6 +42,7 @@ _start: >>>> .cfi_def_cfa_register %ebp >>>> andl $0xfffffff0,%esp # align stack >>>> leal 8(%ebp),%eax >>>> + subl $4,%esp >>>> pushl %eax # argv >>>> pushl 4(%ebp) # argc >>>> pushl %edx # rtld cleanup >>> I'm seeing weird core dumps for ssh and friends on i386 on stable/8 >>> from >>> a few days ago. Could that be related? >> Few days ago ? It was in the tree for probably one year. >> I very much doubt it, but cannot say anything until you show the >> backtrace. >> >> Our in-tree gcc masks this by typically doing stack realignment on the >> entry into the main(). > > I tend to think you are right... The backtrace doesn't have an > aligned instruction to worry about. I'll rebuild to make sure it is > all sane... Finally got a chance to test the rebuild. Once I rebuilt, everything worked fine. I rebuild exactly the same sources I started with, so this wasn't my sshd dumping core problem. I believe I narrowed the problem down to a badly installed libc over NFS... Warner From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 7 23:49:36 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CCBEA106564A for ; Mon, 7 Feb 2011 23:49:36 +0000 (UTC) (envelope-from dieterbsd@engineer.com) Received: from imr-ma03.mx.aol.com (imr-ma03.mx.aol.com [64.12.206.41]) by mx1.freebsd.org (Postfix) with ESMTP id 8A6818FC0A for ; Mon, 7 Feb 2011 23:49:36 +0000 (UTC) Received: from imo-da02.mx.aol.com (imo-da02.mx.aol.com [205.188.169.200]) by imr-ma03.mx.aol.com (8.14.1/8.14.1) with ESMTP id p17NnPwt013533; Mon, 7 Feb 2011 18:49:25 -0500 Received: from dieterbsd@engineer.com by imo-da02.mx.aol.com (mail_out_v42.9.) id n.f85.aa90127 (37516); Mon, 7 Feb 2011 18:49:17 -0500 (EST) Received: from smtprly-da01.mx.aol.com (smtprly-da01.mx.aol.com [205.188.249.144]) by cia-ma08.mx.aol.com (v129.8) with ESMTP id MAILCIAMA081-5baf4d5084f888; Mon, 07 Feb 2011 18:49:17 -0500 Received: from web-mmc-m02 (web-mmc-m02.sim.aol.com [64.12.224.135]) by smtprly-da01.mx.aol.com (v129.8) with ESMTP id MAILSMTPRLYDA016-5baf4d5084f888; Mon, 07 Feb 2011 18:49:12 -0500 To: freebsd-hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Date: Mon, 07 Feb 2011 18:49:12 -0500 X-AOL-IP: 67.206.164.141 X-MB-Message-Source: WebUI Received: from 67.206.164.141 by web-mmc-m02.sysops.aol.com (64.12.224.135) with HTTP (WebMailUI); Mon, 07 Feb 2011 18:49:12 -0500 MIME-Version: 1.0 From: dieterbsd@engineer.com X-MB-Message-Type: User Content-Type: text/plain; charset="us-ascii"; format=flowed X-Mailer: Mail.com Webmail 33189-STANDARD Message-Id: <8CD9538C148865F-1B10-A4C8@web-mmc-m02.sysops.aol.com> X-Spam-Flag: NO X-AOL-SENDER: dieterbsd@engineer.com Cc: Subject: Re: Why does printf(9) hang network? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Feb 2011 23:49:37 -0000 Robert writes: >> Why would doing a printf(9) in a device driver (usb, firewire,=20 probably >> others) cause an obscenely long lockout on >> /usr/src/sys/kern/uipc_sockbuf.c:148 (sx:so_rcv_sx) ? >> >> Printf(9) alone isn't the problem, adding printfs to chown(2) does=20 not >> cause the problem, but printfs from device drivers do. >> >> Grep says that uipc_sockbuf.c is the only file that locks/unlocks=20 sb_sx. >> The device drivers and printf don't even know that sb_sx exists. > > I can't speak to the details of your situation, but one possible > explanation might be: printf runs at the speed of the console, which=20 for > serious consoles can be extremely slowly. But shouldn't the RS-232 driver just fill up the UART's FIFO and then=20 sleep? Why can't the network code run while the RS-232 driver sleeps? Actually this must be happening in the call-printf-from-chown case. There must=20 be something different when printf is called from a device driver. > Device driver interrupt threads can preempt other threads, > possibly while those threads hold locks. That causes them to hold > the locks for much longer, as the threads may not get rescheduled for=20 some > period (for example, until the device driver is done doing a printf), If the CPU is mostly idle, shouldn't the network thread get scheduled=20 right away? > leading > other threads waiting for that lock to wait significantly longer. > Especially the case if the other thread was spinning adaptively, in=20 which > case it will then yield since the holder of the lock effectively=20 yielded. My head is spinning attempting to understand this... > You might try forcing all the various threads to run on different CPUs > using cpuset and see if the variance goes down. Uniprocessor > You can also use KTR + schedgraph > to explore the specific scheduling going on, although be aware that=20 KTR > can also noticeably perturb schediling itself. Scheduling? The CPU can be mostly idle and the problem still happens. > In general, things shouldn't call kernel printf in steady state=20 operation; > if they need to log something, they should use log(9) or similar. =20 printf > is primarily a tool for printing out device probe information, and for > debugging purposes: it is not intended to be fast. Sounds fine to me. Is there a consensus on this? If so, does this=20 need to go into some developer's handbook? How do we get developers to fix the existing code? From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 8 00:12:49 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC65A106564A for ; Tue, 8 Feb 2011 00:12:49 +0000 (UTC) (envelope-from dieterbsd@engineer.com) Received: from imr-ma03.mx.aol.com (imr-ma03.mx.aol.com [64.12.206.41]) by mx1.freebsd.org (Postfix) with ESMTP id AAA0B8FC1A for ; Tue, 8 Feb 2011 00:12:49 +0000 (UTC) Received: from imo-da02.mx.aol.com (imo-da02.mx.aol.com [205.188.169.200]) by imr-ma03.mx.aol.com (8.14.1/8.14.1) with ESMTP id p180CZPt008917; Mon, 7 Feb 2011 19:12:35 -0500 Received: from dieterbsd@engineer.com by imo-da02.mx.aol.com (mail_out_v42.9.) id n.f24.5b79d0c (45072); Mon, 7 Feb 2011 19:12:29 -0500 (EST) Received: from smtprly-mc03.mx.aol.com (smtprly-mc03.mx.aol.com [64.12.95.99]) by cia-mc02.mx.aol.com (v129.8) with ESMTP id MAILCIAMC024-d3de4d508a6621c; Mon, 07 Feb 2011 19:12:29 -0500 Received: from web-mmc-m02 (web-mmc-m02.sim.aol.com [64.12.224.135]) by smtprly-mc03.mx.aol.com (v129.8) with ESMTP id MAILSMTPRLYMC038-d3de4d508a6621c; Mon, 07 Feb 2011 19:12:22 -0500 To: freebsd-hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Date: Mon, 07 Feb 2011 19:12:22 -0500 X-AOL-IP: 67.206.164.141 X-MB-Message-Source: WebUI Received: from 67.206.164.141 by web-mmc-m02.sysops.aol.com (64.12.224.135) with HTTP (WebMailUI); Mon, 07 Feb 2011 19:12:22 -0500 MIME-Version: 1.0 From: dieterbsd@engineer.com X-MB-Message-Type: User Content-Type: text/plain; charset="us-ascii"; format=flowed X-Mailer: Mail.com Webmail 33189-STANDARD Message-Id: <8CD953BFE2324E5-1B10-A8CE@web-mmc-m02.sysops.aol.com> X-Spam-Flag: NO X-AOL-SENDER: dieterbsd@engineer.com Cc: Subject: witness Re: Why does printf(9) hang network? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Feb 2011 00:12:50 -0000 I received a suggestion to try witness, so I build a kernel with WITNESS, WITNESS_KDB, KDB, DDB, KDB_TRACE, and DDB_NUMSYM. This is my first attempt to use witness, so if I got something wrong let me know. Didn't quite make it all the way up to a multiuser prompt: Starting syslogd. Starting rpcbind. lock order reversal: 1st 0xffffff8029549320 bufwait (bufwait) @=20 /usr/src/sys/kern/vfs_bio.c:2559 2nd 0xffffff000498b000 dirhash (dirhash) @=20 /usr/src/sys/ufs/ufs/ufs_dirhash.c:2 85 KDB: stack backtrace: db_trace_self_wrapper() at 0xffffffff801dab0a =3D=20 db_trace_self_wrapper+0x2a _witness_debugger() at 0xffffffff805a144c =3D _witness_debugger+0x2c witness_checkorder() at 0xffffffff805a24af =3D witness_checkorder+0x66f _sx_xlock() at 0xffffffff8056b6d4 =3D _sx_xlock+0x34 ufsdirhash_acquire() at 0xffffffff8076f833 =3D ufsdirhash_acquire+0x33 ufsdirhash_add() at 0xffffffff8076fd99 =3D ufsdirhash_add+0x19 ufs_direnter() at 0xffffffff80772498 =3D ufs_direnter+0x848 ufs_mkdir() at 0xffffffff807783d6 =3D ufs_mkdir+0x5e6 VOP_MKDIR_APV() at 0xffffffff808650d4 =3D VOP_MKDIR_APV+0x34 kern_mkdirat() at 0xffffffff805eb740 =3D kern_mkdirat+0x270 syscall() at 0xffffffff8081ec5e =3D syscall+0x19e Xfast_syscall() at 0xffffffff80806ab1 =3D Xfast_syscall+0xe1 --- syscall (136, FreeBSD ELF64, mkdir), rip =3D 0x80072c53c, rsp =3D=20 0x7fffffffec88 , rbp =3D 0x7fffffffef66 --- KDB: enter: witness_checkorder [thread pid 1255 tid 100076 ] Stopped at 0xffffffff8059083d =3D kdb_enter+0x3d: movq =20 $0,0x6508a0(%rip ) db> Managed to reboot to single user mode, changed /boot/kernel back to my production kernel, and got another lock order reversal rebooting: # sync # reboot Waiting (max 60 seconds) for system process `vnlru' to stop...done Waiting (max 60 seconds) for system process `bufdaemon' to stop...done Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining...0 done All buffers synced. lock order reversal: 1st 0xffffff0004831448 ufs (ufs) @ /usr/src/sys/kern/vfs_mount.c:1200 2nd 0xffffff0004831d80 devfs (devfs) @=20 /usr/src/sys/kern/vfs_subr.c:2083 KDB: stack backtrace: db_trace_self_wrapper() at 0xffffffff801dab0a =3D=20 db_trace_self_wrapper+0x2a _witness_debugger() at 0xffffffff805a144c =3D _witness_debugger+0x2c witness_checkorder() at 0xffffffff805a24af =3D witness_checkorder+0x66f __lockmgr_args() at 0xffffffff80552054 =3D __lockmgr_args+0xd04 vop_stdlock() at 0xffffffff805d9239 =3D vop_stdlock+0x39 VOP_LOCK1_APV() at 0xffffffff80864f56 =3D VOP_LOCK1_APV+0x46 _vn_lock() at 0xffffffff805f3cc7 =3D _vn_lock+0x47 vget() at 0xffffffff805e8856 =3D vget+0x56 devfs_allocv() at 0xffffffff804fa993 =3D devfs_allocv+0x103 devfs_root() at 0xffffffff804f9268 =3D devfs_root+0x48 dounmount() at 0xffffffff805e3369 =3D dounmount+0x419 vfs_unmountall() at 0xffffffff805e82a2 =3D vfs_unmountall+0x42 boot() at 0xffffffff80564bd3 =3D boot+0x683 reboot() at 0xffffffff80564ef8 =3D reboot+0x68 syscall() at 0xffffffff8081ec5e =3D syscall+0x19e Xfast_syscall() at 0xffffffff80806ab1 =3D Xfast_syscall+0xe1 --- syscall (55, FreeBSD ELF64, reboot), rip =3D 0x80078f83c, rsp =3D=20 0x7fffffffece8 , rbp =3D 0 --- KDB: enter: witness_checkorder [thread pid 35 tid 100073 ] Stopped at 0xffffffff8059083d =3D kdb_enter+0x3d: movq =20 $0,0x6508a0(%rip ) db> lock order reversal: 1st 0xffffff0004831da8 vnode interlock (vnode interlock) @=20 /usr/src/sys/fs/devf s/devfs_vnops.c:349 2nd 0xffffff8000248858 firewire (firewire) @=20 /usr/src/sys/dev/firewire/fwohci.c :2227 KDB: stack backtrace: db_trace_self_wrapper() at 0xffffffff801dab0a =3D=20 db_trace_self_wrapper+0x2a _witness_debugger() at 0xffffffff805a144c =3D _witness_debugger+0x2c witness_checkorder() at 0xffffffff805a24af =3D witness_checkorder+0x66f _mtx_lock_flags() at 0xffffffff80557b52 =3D _mtx_lock_flags+0x32 fwohci_poll() at 0xffffffff8035feb1 =3D fwohci_poll+0x31 dcons_cngetc() at 0xffffffff80303d69 =3D dcons_cngetc+0x59 cncheckc() at 0xffffffff8052d425 =3D cncheckc+0x65 cngetc() at 0xffffffff8052d44c =3D cngetc+0x1c db_readline() at 0xffffffff801d9ef7 =3D db_readline+0x77 db_read_line() at 0xffffffff801da975 =3D db_read_line+0x15 db_command_loop() at 0xffffffff801d8ad8 =3D db_command_loop+0x38 db_trap() at 0xffffffff801daa49 =3D db_trap+0x89 kdb_trap() at 0xffffffff80590665 =3D kdb_trap+0x95 trap() at 0xffffffff8081f200 =3D trap+0x170 calltrap() at 0xffffffff808067d3 =3D calltrap+0x8 --- trap 0x3, rip =3D 0xffffffff8059083d, rsp =3D 0xffffff80405d9710, rbp= =3D=20 0xffffff 80405d9730 --- kdb_enter() at 0xffffffff8059083d =3D kdb_enter+0x3d witness_checkorder() at 0xffffffff805a24af =3D witness_checkorder+0x66f __lockmgr_args() at 0xffffffff80552054 =3D __lockmgr_args+0xd04 vop_stdlock() at 0xffffffff805d9239 =3D vop_stdlock+0x39 VOP_LOCK1_APV() at 0xffffffff80864f56 =3D VOP_LOCK1_APV+0x46 _vn_lock() at 0xffffffff805f3cc7 =3D _vn_lock+0x47 vget() at 0xffffffff805e8856 =3D vget+0x56 devfs_allocv() at 0xffffffff804fa993 =3D devfs_allocv+0x103 devfs_root() at 0xffffffff804f9268 =3D devfs_root+0x48 dounmount() at 0xffffffff805e3369 =3D dounmount+0x419 vfs_unmountall() at 0xffffffff805e82a2 =3D vfs_unmountall+0x42 boot() at 0xffffffff80564bd3 =3D boot+0x683 reboot() at 0xffffffff80564ef8 =3D reboot+0x68 syscall() at 0xffffffff8081ec5e =3D syscall+0x19e Xfast_syscall() at 0xffffffff80806ab1 =3D Xfast_syscall+0xe1 --- syscall (55, FreeBSD ELF64, reboot), rip =3D 0x80078f83c, rsp =3D=20 0x7fffffffece8 , rbp =3D 0 --- From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 8 12:20:32 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C1B4C1065679 for ; Tue, 8 Feb 2011 12:20:32 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 7F7668FC21 for ; Tue, 8 Feb 2011 12:20:32 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1PmmYZ-0004M6-3J for freebsd-hackers@freebsd.org; Tue, 08 Feb 2011 13:20:31 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 08 Feb 2011 13:20:31 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 08 Feb 2011 13:20:31 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Date: Tue, 08 Feb 2011 13:20:20 +0100 Lines: 9 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.12) Gecko/20101102 Thunderbird/3.1.6 X-Enigmail-Version: 1.1.2 Subject: Analyzing wired memory? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Feb 2011 12:20:32 -0000 Is it possible to track by some way what kernel system, process or thread has wired memory? (including "data exists but needs code to extract it") I'd like to analyze a system where there is a lot of memory wired but not accounted for in the output of vmstat -m and vmstat -z. There are no user processes which would lock memory themselves. Any pointers? From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 8 09:36:47 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A4D071065670 for ; Tue, 8 Feb 2011 09:36:47 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [62.231.161.221]) by mx1.freebsd.org (Postfix) with ESMTP id A2C118FC15 for ; Tue, 8 Feb 2011 09:36:45 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.4/8.14.4) with ESMTP id p1890TRB043382; Tue, 8 Feb 2011 15:00:29 +0600 (NOVT) (envelope-from egrosbein@rdtc.ru) Message-ID: <4D510628.7060100@rdtc.ru> Date: Tue, 08 Feb 2011 15:00:24 +0600 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: Trever References: <681598.66709.qm@web113216.mail.gq1.yahoo.com> In-Reply-To: <681598.66709.qm@web113216.mail.gq1.yahoo.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Tue, 08 Feb 2011 12:39:28 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: boot0sio waits a minute X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Feb 2011 09:36:47 -0000 On 08.02.2011 02:15, Trever wrote: > Does anyone know why boot0sio would wait about 1 minute before proceeding? > > If this is a known issue I can't find anything about it. I know there is a bug with boot0cfg not changing the partition to boot to. Don't know if this is related. > > FreeBSD 8.2RC2 i386. Boot blocks were compiled with speed 115200. Installed via nanobsd to a compact flash card on an Habey 6620II Z530, but there doesn't seem to be anything wrong other than with boot0sio, or something with how nanobsd installs boot0sio (NOT boot0- that works). But I have checked what nanobsd is doing, that seems ruled out, seems fine. BIOS seems ruled out. Seems like boot0sio. > > Everything works in all manner of configurations, just this very annoying long "hang" before boot0sio runs. Again, boot0 does not do this. > > I'm not talking about the pause after it displays the boot partition selection options. It's before anything happens. I get a blinking underscore in the upper left hand corner of my screen (and nothing on the serial port) for a very long minute before the system boots. Doesn't matter if I have serial attached or not, etc.. I've seen this too. I do not known a reason. And since I do not use physical serial console I just replace boot0sio with boot0 while building NanoBSD and skip the problem :-) Eugene Grosbein From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 8 15:04:50 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42B4E106564A for ; Tue, 8 Feb 2011 15:04:50 +0000 (UTC) (envelope-from alan.l.cox@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id A50B18FC19 for ; Tue, 8 Feb 2011 15:04:49 +0000 (UTC) Received: by fxm16 with SMTP id 16so6469018fxm.13 for ; Tue, 08 Feb 2011 07:04:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:reply-to:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=rBArosaNQAd5g4+NZ0kAu/CDKzDSUuUc83Z3tq5R+7w=; b=gN8hiv4mTFWl4peYN75rEj+oWf/SS5o61KhiTVVlokkuo1UjBUM2mclhrfhyC1s+Go X8/g0NHmRaPHFev7gJHf60Xt8Jd4x4g6dn5bZ6uloCn6JnyVm4S08Mi4Z/G8eZz+qSO6 2Wlygk5wCCGewuFPiPomlbTIMkrOz2vSQoXfg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; b=aaTbDH802azyB4qERZi+YPEI9M9o+1qsU8b2VRi5fI4134WRoXsLNgTGFXPHfz4R0v QgjqVXyIHhbxWNmJy7dZnfyd9pdYygnmFtIc02sia8+AKFkyqW1tjj50o83qYv4mNe+c bTe56ayiXmWake1x5baL+wUkVLWTEy1DT+v0A= MIME-Version: 1.0 Received: by 10.223.72.206 with SMTP id n14mr7587887faj.2.1297177488537; Tue, 08 Feb 2011 07:04:48 -0800 (PST) Received: by 10.223.120.205 with HTTP; Tue, 8 Feb 2011 07:04:48 -0800 (PST) In-Reply-To: References: Date: Tue, 8 Feb 2011 09:04:48 -0600 Message-ID: From: Alan Cox To: Ivan Voras Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: Re: Analyzing wired memory? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: alc@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Feb 2011 15:04:50 -0000 On Tue, Feb 8, 2011 at 6:20 AM, Ivan Voras wrote: > Is it possible to track by some way what kernel system, process or thread > has wired memory? (including "data exists but needs code to extract it") > > No. > I'd like to analyze a system where there is a lot of memory wired but not > accounted for in the output of vmstat -m and vmstat -z. There are no user > processes which would lock memory themselves. > > Any pointers? > > Have you accounted for the buffer cache? Alan From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 8 18:27:23 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 10201106566B; Tue, 8 Feb 2011 18:27:23 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id DC7A78FC0C; Tue, 8 Feb 2011 18:27:22 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 71DC446B1A; Tue, 8 Feb 2011 13:27:22 -0500 (EST) Date: Tue, 8 Feb 2011 18:27:22 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: alc@freebsd.org In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-hackers@freebsd.org, Ivan Voras Subject: Re: Analyzing wired memory? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Feb 2011 18:27:23 -0000 On Tue, 8 Feb 2011, Alan Cox wrote: > On Tue, Feb 8, 2011 at 6:20 AM, Ivan Voras wrote: > >> Is it possible to track by some way what kernel system, process or thread >> has wired memory? (including "data exists but needs code to extract it") >> > No. > > >> I'd like to analyze a system where there is a lot of memory wired but not >> accounted for in the output of vmstat -m and vmstat -z. There are no user >> processes which would lock memory themselves. >> >> Any pointers? >> > Have you accounted for the buffer cache? John and I have occasionally talked about making procstat -v work on the kernel; conceivably it could also export a wired page count for mappings where it makes sense. Ideally procstat would drill in a bit and allow you to see things at least at the granularty of "this page range was allocated to UMA". Robert From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 8 18:53:45 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EA177106566B; Tue, 8 Feb 2011 18:53:45 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id C2FEE8FC14; Tue, 8 Feb 2011 18:53:45 +0000 (UTC) Received: from [10.255.241.24] (192-5-67-11.sri.com [192.5.67.11]) by cyrus.watson.org (Postfix) with ESMTPSA id 1BFD046B09; Tue, 8 Feb 2011 13:53:45 -0500 (EST) Mime-Version: 1.0 (Apple Message framework v1082) Content-Type: text/plain; charset=us-ascii From: "Robert N. M. Watson" In-Reply-To: <4D518D82.4090102@rice.edu> Date: Tue, 8 Feb 2011 10:53:44 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <709DC06E-11E8-4B92-8623-0EB8C835B28B@FreeBSD.org> References: <4D518D82.4090102@rice.edu> To: Alan Cox X-Mailer: Apple Mail (2.1082) Cc: alc@freebsd.org, freebsd-hackers@freebsd.org, Ivan Voras Subject: Re: Analyzing wired memory? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Feb 2011 18:53:46 -0000 On 8 Feb 2011, at 10:37, Alan Cox wrote: >> John and I have occasionally talked about making procstat -v work on = the kernel; conceivably it could also export a wired page count for = mappings where it makes sense. Ideally procstat would drill in a bit = and allow you to see things at least at the granularty of "this page = range was allocated to UMA". >=20 > I would certainly have found this useful on a few occasions, and would = gladly help out with implementing it. For example, it would help us in = understanding the kmem_map fragmentation caused by ZFS. That said, I'm = not sure how you will represent the case where UMA allocates physical = memory directly and uses the direct map to access it. I agree -- in some sense, my proposal is related to, but somewhat = orthogonal to, the problem Ivan is running into. We could also have a more detailed but UMA-specific inspection tool; I = have a uma dumping tool in src/tools somewhere that dumps zone = information, buckets, etc, for UMA types. I've used this in the past to = explore mbuf alloc/free behaviour, multi-CPU imbalances in allocation, = over-caching, etc. It may be time to revisit that, decide how to make it = a bit more user/developer-friendly, and make it a formally supported = tool as part of vmstat (which involves adding some new sysctls as well). Robert From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 8 18:57:04 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 583CC1065672 for ; Tue, 8 Feb 2011 18:57:04 +0000 (UTC) (envelope-from alc@rice.edu) Received: from mh5.mail.rice.edu (mh5.mail.rice.edu [128.42.199.32]) by mx1.freebsd.org (Postfix) with ESMTP id 2F3E88FC18 for ; Tue, 8 Feb 2011 18:57:03 +0000 (UTC) Received: from mh5.mail.rice.edu (localhost.localdomain [127.0.0.1]) by mh5.mail.rice.edu (Postfix) with ESMTP id 4600228F8C5; Tue, 8 Feb 2011 12:38:28 -0600 (CST) X-Virus-Scanned: by amavis-2.6.4 at mh5.mail.rice.edu, auth channel Received: from mh5.mail.rice.edu ([127.0.0.1]) by mh5.mail.rice.edu (mh5.mail.rice.edu [127.0.0.1]) (amavis, port 10026) with ESMTP id 09lqL90dJ1+V; Tue, 8 Feb 2011 12:38:28 -0600 (CST) Received: from [10.209.194.7] (unknown [10.209.194.7]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) (Authenticated sender: alc) by mh5.mail.rice.edu (Postfix) with ESMTPSA id 0ABA128F89B; Tue, 8 Feb 2011 12:38:28 -0600 (CST) Message-ID: <4D518D82.4090102@rice.edu> Date: Tue, 08 Feb 2011 12:37:54 -0600 From: Alan Cox User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Robert Watson References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Tue, 08 Feb 2011 19:07:43 +0000 Cc: alc@freebsd.org, freebsd-hackers@freebsd.org, Ivan Voras Subject: Re: Analyzing wired memory? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Feb 2011 18:57:04 -0000 On 2/8/2011 12:27 PM, Robert Watson wrote: > > On Tue, 8 Feb 2011, Alan Cox wrote: > >> On Tue, Feb 8, 2011 at 6:20 AM, Ivan Voras wrote: >> >>> Is it possible to track by some way what kernel system, process or >>> thread has wired memory? (including "data exists but needs code to >>> extract it") >>> >> No. >> >> >>> I'd like to analyze a system where there is a lot of memory wired >>> but not accounted for in the output of vmstat -m and vmstat -z. >>> There are no user processes which would lock memory themselves. >>> >>> Any pointers? >>> >> Have you accounted for the buffer cache? > > John and I have occasionally talked about making procstat -v work on > the kernel; conceivably it could also export a wired page count for > mappings where it makes sense. Ideally procstat would drill in a bit > and allow you to see things at least at the granularty of "this page > range was allocated to UMA". I would certainly have found this useful on a few occasions, and would gladly help out with implementing it. For example, it would help us in understanding the kmem_map fragmentation caused by ZFS. That said, I'm not sure how you will represent the case where UMA allocates physical memory directly and uses the direct map to access it. Alan From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 8 20:47:43 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35C851065672; Tue, 8 Feb 2011 20:47:43 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 02E188FC12; Tue, 8 Feb 2011 20:47:43 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id A12E746B06; Tue, 8 Feb 2011 15:47:42 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.10]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 84CC08A027; Tue, 8 Feb 2011 15:47:38 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Tue, 8 Feb 2011 15:47:37 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.4-CBSD-20110107; KDE/4.4.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201102081547.37650.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Tue, 08 Feb 2011 15:47:38 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=0.5 required=4.2 tests=BAYES_00,MAY_BE_FORGED, RDNS_DYNAMIC autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Cc: alc@freebsd.org, Robert Watson , Ivan Voras Subject: Re: Analyzing wired memory? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Feb 2011 20:47:43 -0000 On Tuesday, February 08, 2011 1:27:22 pm Robert Watson wrote: > > On Tue, 8 Feb 2011, Alan Cox wrote: > > > On Tue, Feb 8, 2011 at 6:20 AM, Ivan Voras wrote: > > > >> Is it possible to track by some way what kernel system, process or thread > >> has wired memory? (including "data exists but needs code to extract it") > >> > > No. > > > > > >> I'd like to analyze a system where there is a lot of memory wired but not > >> accounted for in the output of vmstat -m and vmstat -z. There are no user > >> processes which would lock memory themselves. > >> > >> Any pointers? > >> > > Have you accounted for the buffer cache? > > John and I have occasionally talked about making procstat -v work on the > kernel; conceivably it could also export a wired page count for mappings where > it makes sense. Ideally procstat would drill in a bit and allow you to see > things at least at the granularty of "this page range was allocated to UMA". What I have used is a 'kvm' command in my kgdb scripts, but it is not nearly that granular. It is also a view of the virtual address space (similar to procstat -v), not a view of the physical address space. I have found it useful though (sample output from my desktop running 7-ish): (kgdb) kvm ffffff8000000000 - ffffff800000f000 kmem_alloc() / contigmalloc() ffffff800000f000 - ffffff8000021000 AP stacks ffffff8000021000 - ffffff80000a3000 kmem_alloc_nofault() (kstack/mapdev) ffffff80000a3000 - ffffff80000c3000 kmem_alloc() / contigmalloc() ffffff80000c3000 - ffffff80000c8000 kmem_alloc_nofault() (kstack/mapdev) ffffff80000c8000 - ffffff80000e8000 kmem_alloc() / contigmalloc() ffffff80000e8000 - ffffff80000f2000 kmem_alloc_nofault() (kstack/mapdev) ffffff80000f2000 - ffffff8000200000 kmem_alloc() / contigmalloc() ffffff8000200000 - ffffff8051da4000 kmem_map ffffff8051da4000 - ffffff80523787e0 callouts ffffff80523787e0 - ffffff80523a07e0 swbuf ffffff80523a07e0 - ffffff80533f2000 buf ffffff80533f2000 - ffffff806f5aa000 buffer_map + pager_map ffffff806f5aa000 - ffffff806f9da000 exec_map ffffff806f9da000 - ffffff8073b3b000 pipe_map ffffff8073b3b000 - ffffff807534f000 kmem_alloc_nofault() (kstack/mapdev) ffffff807534f000 - ffffff8076261000 kmem_alloc() / contigmalloc() ffffff8076261000 - ffffff807626b000 kmem_alloc_nofault() (kstack/mapdev) ffffff807626b000 - ffffff8077281000 kmem_alloc() / contigmalloc() ffffff8077281000 - ffffff8077295000 kmem_alloc_nofault() (kstack/mapdev) ffffff8077295000 - ffffff807729c000 kmem_alloc() / contigmalloc() ffffff807729c000 - ffffff80772a6000 kmem_alloc_nofault() (kstack/mapdev) ffffff80772a6000 - ffffff80772ad000 kmem_alloc() / contigmalloc() ffffff80772ad000 - ffffff80772b2000 kmem_alloc_nofault() (kstack/mapdev) ffffff80772b2000 - ffffff80772b9000 kmem_alloc() / contigmalloc() ffffff80772b9000 - ffffff80772be000 kmem_alloc_nofault() (kstack/mapdev) ffffff80772be000 - ffffff80772d0000 kmem_alloc() / contigmalloc() ffffff80772d0000 - ffffff80772d5000 kmem_alloc_nofault() (kstack/mapdev) ffffff80772d5000 - ffffff80772d9000 kmem_alloc() / contigmalloc() ffffff80772d9000 - ffffff80772de000 kmem_alloc_nofault() (kstack/mapdev) ffffff80772de000 - ffffff80772f6000 kmem_alloc() / contigmalloc() ffffff80772f6000 - ffffff8077300000 kmem_alloc_nofault() (kstack/mapdev) ffffff8077300000 - ffffff807730a000 kmem_alloc() / contigmalloc() ffffff807730a000 - ffffff807730f000 kmem_alloc_nofault() (kstack/mapdev) ffffff807730f000 - ffffff8077310000 kmem_alloc() / contigmalloc() ffffff8077310000 - ffffff8077315000 kmem_alloc_nofault() (kstack/mapdev) ffffff8077315000 - ffffff8077316000 kmem_alloc() / contigmalloc() ffffff8077316000 - ffffff807733e000 kmem_alloc_nofault() (kstack/mapdev) ffffff807733e000 - ffffff8079641000 swap zone ffffff8079641000 - ffffff807988f000 kmem_alloc_nofault() (kstack/mapdev) ffffff807988f000 - ffffff807989a000 kmem_alloc() / contigmalloc() ffffff807989a000 - ffffff80798a5000 object 0xffffff000fbd20d8 ffffff80798a5000 - ffffff80798a6000 kmem_alloc() / contigmalloc() ffffff80798a6000 - ffffff80798a7000 object 0xffffff000fa8c798 ffffff80798a7000 - ffffff80798b8000 kmem_alloc() / contigmalloc() ffffff80798b8000 - ffffff80798b9000 object 0xffffff000fa8c798 ffffff80798b9000 - ffffff80798ba000 object 0xffffff000fa8c798 ffffff80798ba000 - ffffff80798bb000 object 0xffffff000fa8c798 ffffff80798bb000 - ffffff80798bc000 object 0xffffff000fa8c798 ffffff80798bc000 - ffffff80798cc000 kmem_alloc() / contigmalloc() ffffff80798cc000 - ffffff80798cd000 object 0xffffff000fa8c798 ffffff80798cd000 - ffffff80798ce000 object 0xffffff000fa8c798 ffffff80798ce000 - ffffff80798e3000 kmem_alloc() / contigmalloc() ffffff80798e3000 - ffffff80798e4000 object 0xffffff000f6965e8 ffffff80798e4000 - ffffff80798e5000 object 0xffffff000f6965e8 ffffff80798e5000 - ffffff80798e6000 object 0xffffff000f6965e8 ffffff80798e6000 - ffffff80798e7000 object 0xffffff000f6965e8 ffffff80798e7000 - ffffff80798e8000 object 0xffffff000f6965e8 ffffff80798e8000 - ffffff80798ea000 kmem_alloc() / contigmalloc() ffffff80798ea000 - ffffff8079949000 kmem_alloc_nofault() (kstack/mapdev) ffffff8079949000 - ffffff807994a000 kmem_alloc() / contigmalloc() ffffff807994a000 - ffffff807994b000 object 0xffffff0060568af8 ffffff807994b000 - ffffff8079969000 kmem_alloc_nofault() (kstack/mapdev) ffffff8079969000 - ffffff807996b000 ---- ffffff807996b000 - ffffff80799b0000 kmem_alloc() / contigmalloc() ffffff80799b0000 - ffffff80799b1000 object 0xffffff00606caca8 ffffff80799b1000 - ffffff80799b2000 object 0xffffff00606caca8 ffffff80799b2000 - ffffff80799b6000 kmem_alloc() / contigmalloc() ffffff80799b6000 - ffffff80799b7000 object 0xffffff0060568af8 ffffff80799b7000 - ffffff80799b8000 object 0xffffff0060568af8 ffffff80799b8000 - ffffff8079cbc000 kmem_alloc() / contigmalloc() ffffff8079cbc000 - ffffff807aa0e000 kmem_alloc_nofault() (kstack/mapdev) ffffff807aa0e000 - ffffffff80000000 ---- ffffffff80000000 - ffffffff808164e8 text/data/bss ffffffff808164e8 - ffffffff81822000 bootstrap data (The various objects inserted directly into the kernel_map are likely from the nvidia driver.) The 'kvm' command in my gdb script is mostly MI, but some bits are MD such as the code to handle the 'AP stacks' region. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 8 21:05:21 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 621C510656A4; Tue, 8 Feb 2011 21:05:21 +0000 (UTC) (envelope-from rwatson@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 39E248FC17; Tue, 8 Feb 2011 21:05:21 +0000 (UTC) Received: from [10.255.241.24] (192-5-67-11.sri.com [192.5.67.11]) by cyrus.watson.org (Postfix) with ESMTPSA id 6DAB846B23; Tue, 8 Feb 2011 16:05:20 -0500 (EST) Mime-Version: 1.0 (Apple Message framework v1082) Content-Type: text/plain; charset=us-ascii From: "Robert N. M. Watson" In-Reply-To: <201102081547.37650.jhb@freebsd.org> Date: Tue, 8 Feb 2011 13:05:19 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <92A3111D-AE7A-46DA-BEB7-A2D315097F63@freebsd.org> References: <201102081547.37650.jhb@freebsd.org> To: John Baldwin X-Mailer: Apple Mail (2.1082) Cc: alc@freebsd.org, freebsd-hackers@freebsd.org, Ivan Voras Subject: Re: Analyzing wired memory? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Feb 2011 21:05:21 -0000 On 8 Feb 2011, at 12:47, John Baldwin wrote: > What I have used is a 'kvm' command in my kgdb scripts, but it is not = nearly > that granular. It is also a view of the virtual address space = (similar to > procstat -v), not a view of the physical address space. I have found = it > useful though (sample output from my desktop running 7-ish): >=20 > (kgdb) kvm > ffffff8000000000 - ffffff800000f000 kmem_alloc() / contigmalloc() > ffffff800000f000 - ffffff8000021000 AP stacks > ffffff8000021000 - ffffff80000a3000 kmem_alloc_nofault() = (kstack/mapdev) > ffffff80000a3000 - ffffff80000c3000 kmem_alloc() / contigmalloc() ... > (The various objects inserted directly into the kernel_map are likely = from > the nvidia driver.) >=20 > The 'kvm' command in my gdb script is mostly MI, but some bits are MD = such as > the code to handle the 'AP stacks' region. This is the sort of thing I have in mind, although passing a bit more = information down through the allocators to VM so that it can expose that = information more readily to userspace. Really it's just an information = management problem -- how to get the information in the right place, and = also not have it be too overwhelming. My temptation would be to simply = export this data via the same procstat VM sysctls we already have, but = for process 0. We have storage for a path there already, although my = temptation would be to add a new mapping type to the procstat = enumeration to capture the idea of kernel memory, so that the names = aren't "paths" from an API perspective, but strings. Robert= From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 8 22:57:10 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A67A7106566B for ; Tue, 8 Feb 2011 22:57:10 +0000 (UTC) (envelope-from alip@exherbo.org) Received: from bach.exherbo.org (bach.exherbo.org [78.47.197.147]) by mx1.freebsd.org (Postfix) with ESMTP id 0B1808FC08 for ; Tue, 8 Feb 2011 22:57:09 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=karatren.karatren.ev ident=alip) by bach.exherbo.org with esmtpa (Exim 4.71) (envelope-from ) id 1PmwFx-0002IH-3I for freebsd-hackers@freebsd.org; Tue, 08 Feb 2011 22:41:57 +0000 Received: by karatren.karatren.ev (Postfix, from userid 1000) id 7B14A2588B; Wed, 9 Feb 2011 00:42:18 +0200 (EET) From: Ali Polatel To: FreeBSD Hackers User-Agent: Notmuch/0.5 (http://notmuchmail.org) Emacs/23.2.1 (x86_64-pc-linux-gnu) Date: Wed, 09 Feb 2011 00:42:15 +0200 Message-ID: <87fwrydu7s.fsf@karatren.ev> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Subject: ptrace weirdness with 9.0-CURRENT X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Feb 2011 22:57:10 -0000 --=-=-= Content-Transfer-Encoding: quoted-printable Hello everyone, I'm the developer of pinktrace - http://dev.exherbo.org/~alip/pinktrace/ =2D a simple ptrace() wrapper library for FreeBSD and Linux. I have set up a FreeBSD-9.0-CURRENT VM today to test various new features recently added to ptrace(). This is about a behaviour difference between 8.1-RELEASE and 9.0-CURRENT which I've noticed through a unit test of pinktrace. I don't want to bother you with the internals of this library so I'll briefly explain the problem. I've inserted the testcase I've used below. The aim is to trace a open(NULL, 0) call which should fail with EFAULT. Running this on two different VMs I get: % uname -a FreeBSD 9.0-CURRENT FreeBSD 9.0-CURRENT #0: Wed Feb 9 05:02:31 EET 2011 = root@:/usr/obj/usr/src/sys/GENERIC amd64 % sudo cat /root/world.txt =2D------------------------------------------------------------- >>> World build completed on Wed Feb 9 00:23:30 EET 2011 =2D------------------------------------------------------------- % gcc -Wall ptrace-amd64-fbsd-return.c % ./a.out retval:0 error:0 $ uname -a FreeBSD 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:36:49 UTC 2010 = root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 $ gcc -Wall ptrace-amd64-fbsd-return.c $ ./a.out retval:14 error:1 $=20 Important note: I couldn't notice a problem with truss tracing a open(NULL, 0) call so I think this is a problem with my testcase. I'll be happy if you can shed some light on what I'm doing wrong here: #include #include #include #include #include #include #include #include #include #include #include #include #undef NDEBUG #include int main(void) { int status; pid_t pid; if ((pid =3D fork()) < 0) { perror("fork"); abort(); } else if (!pid) { /* child */ assert(!(ptrace(PT_TRACE_ME, 0, NULL, 0) < 0)); kill(getpid(), SIGSTOP); open(NULL, 0); fprintf(stderr, "open: (errno:%d %s)\n", errno, strerror(errno)); _exit(0); } else { assert(!(waitpid(pid, &status, 0) < 0)); assert(WIFSTOPPED(status)); assert(WSTOPSIG(status) =3D=3D SIGSTOP); assert(!(ptrace(PT_TO_SCX, pid, (caddr_t)1, 0) < 0)); assert(!(waitpid(pid, &status, 0) < 0)); assert(WIFSTOPPED(status)); assert(WSTOPSIG(status) =3D=3D SIGTRAP); #if defined(PT_LWPINFO) && defined(PL_FLAG_SCX) struct ptrace_lwpinfo info; assert(!(ptrace(PT_LWPINFO, pid, (caddr_t)&info, sizeof(struct ptrace_lwp= info)) < 0)); assert(info.pl_flags & PL_FLAG_SCX); #endif struct reg r; assert(!(ptrace(PT_GETREGS, pid, (caddr_t)&r, 0) < 0)); printf("retval:%ld error:%d\n", r.r_rax, !!(r.r_rflags & PSL_C)); ptrace(PT_CONTINUE, pid, (caddr_t)1, 0); waitpid(pid, &status, 0); return 0; } } =2D-=20 Regards, Ali Polatel --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) iEYEARECAAYFAk1RxskACgkQQU4yORhF8iBBxgCguEk/Js8WyUlDQaXIJ6PwygdD AqIAn3F6cdQeqiODzStm1UKxiRSUYa05 =rnOI -----END PGP SIGNATURE----- --=-=-=-- From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 8 23:49:58 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 22D8D1065670 for ; Tue, 8 Feb 2011 23:49:58 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id AE2AD8FC13 for ; Tue, 8 Feb 2011 23:49:57 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p18NnqrD091904 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 9 Feb 2011 01:49:52 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p18Nnq4j086788; Wed, 9 Feb 2011 01:49:52 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p18NnqEe086787; Wed, 9 Feb 2011 01:49:52 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 9 Feb 2011 01:49:52 +0200 From: Kostik Belousov To: Ali Polatel Message-ID: <20110208234952.GG78089@deviant.kiev.zoral.com.ua> References: <87fwrydu7s.fsf@karatren.ev> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="h6w+13shfCQ8v2Yw" Content-Disposition: inline In-Reply-To: <87fwrydu7s.fsf@karatren.ev> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: FreeBSD Hackers Subject: Re: ptrace weirdness with 9.0-CURRENT X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Feb 2011 23:49:58 -0000 --h6w+13shfCQ8v2Yw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Feb 09, 2011 at 12:42:15AM +0200, Ali Polatel wrote: > Hello everyone, >=20 > I'm the developer of pinktrace - http://dev.exherbo.org/~alip/pinktrace/ > - a simple ptrace() wrapper library for FreeBSD and Linux. I have set up > a FreeBSD-9.0-CURRENT VM today to test various new features recently > added to ptrace(). This is about a behaviour difference between > 8.1-RELEASE and 9.0-CURRENT which I've noticed through a unit test of > pinktrace. I don't want to bother you with the internals of this library > so I'll briefly explain the problem. >=20 > I've inserted the testcase I've used below. The aim is to trace a > open(NULL, 0) call which should fail with EFAULT. Running this on two > different VMs I get: >=20 > % uname -a > FreeBSD 9.0-CURRENT FreeBSD 9.0-CURRENT #0: Wed Feb 9 05:02:31 EET 2011= root@:/usr/obj/usr/src/sys/GENERIC amd64 > % sudo cat /root/world.txt > -------------------------------------------------------------- > >>> World build completed on Wed Feb 9 00:23:30 EET 2011 > -------------------------------------------------------------- > % gcc -Wall ptrace-amd64-fbsd-return.c > % ./a.out > retval:0 error:0 >=20 > $ uname -a > FreeBSD 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:36:49 UTC 2010= root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 > $ gcc -Wall ptrace-amd64-fbsd-return.c > $ ./a.out > retval:14 error:1 > $=20 >=20 > Important note: I couldn't notice a problem with truss tracing a > open(NULL, 0) call so I think this is a problem with my testcase. > I'll be happy if you can shed some light on what I'm doing wrong here: There is no issue with ptrace(2). Your test fails because, apparently, rtld in HEAD calls setjmp(3) when resolving symbols, and setjmp(3) calls sigprocmask(2). The end result is that you get SCX event for sigprocmask, and not for your open(2). The issue with sigprocmask call from setjmp shall be fixed, but this is not an issue with ptrace(2). >=20 > #include > #include > #include >=20 > #include > #include >=20 > #include > #include > #include > #include > #include > #include > #include >=20 > #undef NDEBUG > #include >=20 > int > main(void) > { > int status; > pid_t pid; >=20 > if ((pid =3D fork()) < 0) { > perror("fork"); > abort(); > } > else if (!pid) { /* child */ > assert(!(ptrace(PT_TRACE_ME, 0, NULL, 0) < 0)); > kill(getpid(), SIGSTOP); > open(NULL, 0); > fprintf(stderr, "open: (errno:%d %s)\n", errno, strerror(errno)); > _exit(0); > } > else { > assert(!(waitpid(pid, &status, 0) < 0)); > assert(WIFSTOPPED(status)); > assert(WSTOPSIG(status) =3D=3D SIGSTOP); >=20 > assert(!(ptrace(PT_TO_SCX, pid, (caddr_t)1, 0) < 0)); > assert(!(waitpid(pid, &status, 0) < 0)); > assert(WIFSTOPPED(status)); > assert(WSTOPSIG(status) =3D=3D SIGTRAP); >=20 > #if defined(PT_LWPINFO) && defined(PL_FLAG_SCX) > struct ptrace_lwpinfo info; > assert(!(ptrace(PT_LWPINFO, pid, (caddr_t)&info, sizeof(struct ptrace_l= wpinfo)) < 0)); > assert(info.pl_flags & PL_FLAG_SCX); > #endif >=20 > struct reg r; > assert(!(ptrace(PT_GETREGS, pid, (caddr_t)&r, 0) < 0)); >=20 > printf("retval:%ld error:%d\n", r.r_rax, !!(r.r_rflags & PSL_C)); >=20 > ptrace(PT_CONTINUE, pid, (caddr_t)1, 0); > waitpid(pid, &status, 0); >=20 > return 0; > } > } >=20 > --=20 > Regards, > Ali Polatel --h6w+13shfCQ8v2Yw Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk1R1p8ACgkQC3+MBN1Mb4hDMgCg6MXFbqChftKh9M55mW81nZ2T 9bUAnjVudJXmMtJfDZHJxj8tUDs9QTX9 =9c0P -----END PGP SIGNATURE----- --h6w+13shfCQ8v2Yw-- From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 8 23:55:47 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BAE5106566C for ; Tue, 8 Feb 2011 23:55:47 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 1FE498FC0C for ; Tue, 8 Feb 2011 23:55:46 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p18NtgKX092308 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 9 Feb 2011 01:55:42 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p18Ntg8Q086836; Wed, 9 Feb 2011 01:55:42 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p18NtgsG086835; Wed, 9 Feb 2011 01:55:42 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 9 Feb 2011 01:55:42 +0200 From: Kostik Belousov To: Ali Polatel Message-ID: <20110208235542.GH78089@deviant.kiev.zoral.com.ua> References: <87fwrydu7s.fsf@karatren.ev> <20110208234952.GG78089@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="CwciIYoKw+3ryeRd" Content-Disposition: inline In-Reply-To: <20110208234952.GG78089@deviant.kiev.zoral.com.ua> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: FreeBSD Hackers Subject: Re: ptrace weirdness with 9.0-CURRENT X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Feb 2011 23:55:47 -0000 --CwciIYoKw+3ryeRd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Feb 09, 2011 at 01:49:52AM +0200, Kostik Belousov wrote: > On Wed, Feb 09, 2011 at 12:42:15AM +0200, Ali Polatel wrote: > > Hello everyone, > >=20 > > I'm the developer of pinktrace - http://dev.exherbo.org/~alip/pinktrace/ > > - a simple ptrace() wrapper library for FreeBSD and Linux. I have set up > > a FreeBSD-9.0-CURRENT VM today to test various new features recently > > added to ptrace(). This is about a behaviour difference between > > 8.1-RELEASE and 9.0-CURRENT which I've noticed through a unit test of > > pinktrace. I don't want to bother you with the internals of this library > > so I'll briefly explain the problem. > >=20 > > I've inserted the testcase I've used below. The aim is to trace a > > open(NULL, 0) call which should fail with EFAULT. Running this on two > > different VMs I get: > >=20 > > % uname -a > > FreeBSD 9.0-CURRENT FreeBSD 9.0-CURRENT #0: Wed Feb 9 05:02:31 EET 20= 11 root@:/usr/obj/usr/src/sys/GENERIC amd64 > > % sudo cat /root/world.txt > > -------------------------------------------------------------- > > >>> World build completed on Wed Feb 9 00:23:30 EET 2011 > > -------------------------------------------------------------- > > % gcc -Wall ptrace-amd64-fbsd-return.c > > % ./a.out > > retval:0 error:0 > >=20 > > $ uname -a > > FreeBSD 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:36:49 UTC 20= 10 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 > > $ gcc -Wall ptrace-amd64-fbsd-return.c > > $ ./a.out > > retval:14 error:1 > > $=20 > >=20 > > Important note: I couldn't notice a problem with truss tracing a > > open(NULL, 0) call so I think this is a problem with my testcase. > > I'll be happy if you can shed some light on what I'm doing wrong here: > There is no issue with ptrace(2). Your test fails because, apparently, > rtld in HEAD calls setjmp(3) when resolving symbols, and setjmp(3) > calls sigprocmask(2). The end result is that you get SCX event for > sigprocmask, and not for your open(2). >=20 > The issue with sigprocmask call from setjmp shall be fixed, but this > is not an issue with ptrace(2). The following should fix the problem. diff --git a/libexec/rtld-elf/rtld.c b/libexec/rtld-elf/rtld.c index 50ab393..948cf49 100644 --- a/libexec/rtld-elf/rtld.c +++ b/libexec/rtld-elf/rtld.c @@ -560,7 +560,7 @@ _rtld_bind(Obj_Entry *obj, Elf_Size reloff) RtldLockState lockstate; =20 rlock_acquire(rtld_bind_lock, &lockstate); - if (setjmp(lockstate.env) !=3D 0) + if (sigsetjmp(lockstate.env, 0) !=3D 0) lock_upgrade(rtld_bind_lock, &lockstate); if (obj->pltrel) rel =3D (const Elf_Rel *) ((caddr_t) obj->pltrel + reloff); @@ -2142,7 +2142,7 @@ dlopen(const char *name, int mode) ld_tracing =3D (mode & RTLD_TRACE) =3D=3D 0 ? NULL : "1"; if (ld_tracing !=3D NULL) { rlock_acquire(rtld_bind_lock, &lockstate); - if (setjmp(lockstate.env) !=3D 0) + if (sigsetjmp(lockstate.env, 0) !=3D 0) lock_upgrade(rtld_bind_lock, &lockstate); environ =3D (char **)*get_program_var_addr("environ", &lockstate); lock_release(rtld_bind_lock, &lockstate); @@ -2264,7 +2264,7 @@ do_dlsym(void *handle, const char *name, void *retadd= r, const Ver_Entry *ve, req.lockstate =3D &lockstate; =20 rlock_acquire(rtld_bind_lock, &lockstate); - if (setjmp(lockstate.env) !=3D 0) + if (sigsetjmp(lockstate.env, 0) !=3D 0) lock_upgrade(rtld_bind_lock, &lockstate); if (handle =3D=3D NULL || handle =3D=3D RTLD_NEXT || handle =3D=3D RTLD_DEFAULT || handle =3D=3D RTLD_SELF) { diff --git a/libexec/rtld-elf/rtld.h b/libexec/rtld-elf/rtld.h index 8941d29..bb365a7 100644 --- a/libexec/rtld-elf/rtld.h +++ b/libexec/rtld-elf/rtld.h @@ -276,7 +276,7 @@ typedef struct Struct_DoneList { =20 struct Struct_RtldLockState { int lockstate; - jmp_buf env; + sigjmp_buf env; }; =20 /* diff --git a/libexec/rtld-elf/rtld_lock.c b/libexec/rtld-elf/rtld_lock.c index e76a4da..024e1e2 100644 --- a/libexec/rtld-elf/rtld_lock.c +++ b/libexec/rtld-elf/rtld_lock.c @@ -259,7 +259,7 @@ lock_restart_for_upgrade(RtldLockState *lockstate) case RTLD_LOCK_WLOCKED: break; case RTLD_LOCK_RLOCKED: - longjmp(lockstate->env, 1); + siglongjmp(lockstate->env, 1); break; default: assert(0); --CwciIYoKw+3ryeRd Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk1R1/4ACgkQC3+MBN1Mb4hcdwCgmzmKJ/ETTwsOX7LYBuWnUG5z uyMAoJcsD4id/vK7s7voxMFOFknTaxhs =TIlA -----END PGP SIGNATURE----- --CwciIYoKw+3ryeRd-- From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 9 02:31:01 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 12C38106567A; Wed, 9 Feb 2011 02:31:01 +0000 (UTC) (envelope-from davidxu@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D76C78FC08; Wed, 9 Feb 2011 02:31:00 +0000 (UTC) Received: from xyf.my.dom (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p192Uv7l096254; Wed, 9 Feb 2011 02:30:58 GMT (envelope-from davidxu@freebsd.org) Message-ID: <4D51FC60.8010005@freebsd.org> Date: Wed, 09 Feb 2011 10:30:56 +0800 From: David Xu User-Agent: Thunderbird 2.0.0.24 (X11/20100630) MIME-Version: 1.0 To: Ivan Voras References: <4D3CB2AF.9050003@yandex.ru> <4D4D9A4D.2070508@yandex.ru> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Ruslan Mahmatkhanov Subject: Re: Tracking down a problem with php on FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Feb 2011 02:31:01 -0000 Ivan Voras wrote: > On 5 February 2011 19:43, Ruslan Mahmatkhanov wrote: >> Hi, Ivan! >> >> Thank you much for response and sorry for late answer. We was able to >> collect some data about the issue to make discussion more objective. See >> below. > >>>> Simple php-fpm restart solves the problem, but i need to track it down >>>> to the cause of this situation and ask for your assistance and >>>> instructions on how to debug it. Some facts about this: >>> On one hand, FPM is said to be very experimental... >>> >>> Personally, I've been using apache22-worker or apache22-event + >>> mod_fcgid for years without trouble. >> We prefer to avoid using apache at all, because in this it's just adds yet >> another unneeded link and complexity. > > I guess it's about tradeoffs beween complexity and stability :) > >>>> - `top -mio` shows very high (80000-90000 for VCSW) VCSW/IVCSW values >>>> for php-fpm processes and LA is more than 120 > > I think this is significant, especially with this: > >> When attaching to any hanging php-fpm proccess with truss, than i see a lot >> of this calls: >> sched_yield(0x80516c000,0x1,0x4d4828b6,0x8012ef45c,0xffffffff808bfd80,0x7fffffffa078) >> = 0 (0x0) >> sched_yield(0x80516c000,0x1,0x4d4828b6,0x8012ef45c,0xffffffff808bfd80,0x7fffffffa078) >> = 0 (0x0) >> sched_yield(0x80516c000,0x1,0x4d4828b6,0x8012ef45c,0xffffffff808bfd80,0x7fffffffa078) >> = 0 (0x0) >> sched_yield(0x80516c000,0x1,0x4d4828b6,0x8012ef45c,0xffffffff808bfd80,0x7fffffffa078) >> = 0 (0x0) > > "Normal" processes of the type PHP is have no need to call > sched_yield() arbitrarily, unless they are implementing something they > shouldn't - like a synchronization primitive (semaphore/lock). If "a > lot" means "of the same order of magnitude as your VCSW rate", this is > the reason for it. > > I've analyzed my php-cgi binary and modules and they don't use sched_yield. > > And yes, grepping for it in the source finds it only in FPM: > > sapi/fpm/fpm/fpm_atomic.h:140: sched_yield(); > > It seems they are trying to implement a spinlock by hand, instead of > using what the OS provides. (on the other hand, the implementation > might be correct but they may be using it wrong). > > In any case, this points to bugs in FPM. if so, unfortunately I can't > help you further. > > If you really want to continue using FPM, I guess you should probably > replace this hand-made lock implementation by sem(4) or see if > "robust" pthreads mutexes can be committed and MFCed (maybe with David > Xu). > Yes, there is a p4 branch implemented pthread robust mutex, it requires ABI change. Document for the POSIX robust mutex is here: http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_mutexattr_getrobust.html From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 9 23:07:54 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3940106566B for ; Wed, 9 Feb 2011 23:07:54 +0000 (UTC) (envelope-from alip@exherbo.org) Received: from bach.exherbo.org (bach.exherbo.org [78.47.197.147]) by mx1.freebsd.org (Postfix) with ESMTP id 384D08FC18 for ; Wed, 9 Feb 2011 23:07:53 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=karatren.karatren.ev ident=alip) by bach.exherbo.org with esmtpa (Exim 4.71) (envelope-from ) id 1PnJ8S-0005qL-CV; Wed, 09 Feb 2011 23:07:44 +0000 Received: by karatren.karatren.ev (Postfix, from userid 1000) id 5675D219E9; Thu, 10 Feb 2011 01:08:05 +0200 (EET) From: Ali Polatel To: Kostik Belousov In-Reply-To: <20110208235542.GH78089@deviant.kiev.zoral.com.ua> References: <87fwrydu7s.fsf@karatren.ev> <20110208234952.GG78089@deviant.kiev.zoral.com.ua> <20110208235542.GH78089@deviant.kiev.zoral.com.ua> User-Agent: Notmuch/0.5 (http://notmuchmail.org) Emacs/23.2.1 (x86_64-pc-linux-gnu) Date: Thu, 10 Feb 2011 01:07:58 +0200 Message-ID: <87fwrw9581.fsf@karatren.ev> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" Cc: FreeBSD Hackers Subject: Re: ptrace weirdness with 9.0-CURRENT X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Feb 2011 23:07:54 -0000 --=-=-= Content-Transfer-Encoding: quoted-printable On Wed, 9 Feb 2011 01:55:42 +0200, Kostik Belousov wr= ote: > On Wed, Feb 09, 2011 at 01:49:52AM +0200, Kostik Belousov wrote: > > On Wed, Feb 09, 2011 at 12:42:15AM +0200, Ali Polatel wrote: > > > Hello everyone, > > >=20 > > > I'm the developer of pinktrace - http://dev.exherbo.org/~alip/pinktra= ce/ > > > - a simple ptrace() wrapper library for FreeBSD and Linux. I have set= up > > > a FreeBSD-9.0-CURRENT VM today to test various new features recently > > > added to ptrace(). This is about a behaviour difference between > > > 8.1-RELEASE and 9.0-CURRENT which I've noticed through a unit test of > > > pinktrace. I don't want to bother you with the internals of this libr= ary > > > so I'll briefly explain the problem. > > >=20 > > > I've inserted the testcase I've used below. The aim is to trace a > > > open(NULL, 0) call which should fail with EFAULT. Running this on two > > > different VMs I get: > > >=20 > > > % uname -a > > > FreeBSD 9.0-CURRENT FreeBSD 9.0-CURRENT #0: Wed Feb 9 05:02:31 EET = 2011 root@:/usr/obj/usr/src/sys/GENERIC amd64 > > > % sudo cat /root/world.txt > > > -------------------------------------------------------------- > > > >>> World build completed on Wed Feb 9 00:23:30 EET 2011 > > > -------------------------------------------------------------- > > > % gcc -Wall ptrace-amd64-fbsd-return.c > > > % ./a.out > > > retval:0 error:0 > > >=20 > > > $ uname -a > > > FreeBSD 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:36:49 UTC = 2010 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 > > > $ gcc -Wall ptrace-amd64-fbsd-return.c > > > $ ./a.out > > > retval:14 error:1 > > > $=20 > > >=20 > > > Important note: I couldn't notice a problem with truss tracing a > > > open(NULL, 0) call so I think this is a problem with my testcase. > > > I'll be happy if you can shed some light on what I'm doing wrong here: > > There is no issue with ptrace(2). Your test fails because, apparently, > > rtld in HEAD calls setjmp(3) when resolving symbols, and setjmp(3) > > calls sigprocmask(2). The end result is that you get SCX event for > > sigprocmask, and not for your open(2). Ah interesting. > > The issue with sigprocmask call from setjmp shall be fixed, but this > > is not an issue with ptrace(2). >=20 > The following should fix the problem. I confirm that this patch fixes the problem, thanks! =20 > diff --git a/libexec/rtld-elf/rtld.c b/libexec/rtld-elf/rtld.c > index 50ab393..948cf49 100644 > --- a/libexec/rtld-elf/rtld.c > +++ b/libexec/rtld-elf/rtld.c > @@ -560,7 +560,7 @@ _rtld_bind(Obj_Entry *obj, Elf_Size reloff) > RtldLockState lockstate; >=20=20 > rlock_acquire(rtld_bind_lock, &lockstate); > - if (setjmp(lockstate.env) !=3D 0) > + if (sigsetjmp(lockstate.env, 0) !=3D 0) > lock_upgrade(rtld_bind_lock, &lockstate); > if (obj->pltrel) > rel =3D (const Elf_Rel *) ((caddr_t) obj->pltrel + reloff); > @@ -2142,7 +2142,7 @@ dlopen(const char *name, int mode) > ld_tracing =3D (mode & RTLD_TRACE) =3D=3D 0 ? NULL : "1"; > if (ld_tracing !=3D NULL) { > rlock_acquire(rtld_bind_lock, &lockstate); > - if (setjmp(lockstate.env) !=3D 0) > + if (sigsetjmp(lockstate.env, 0) !=3D 0) > lock_upgrade(rtld_bind_lock, &lockstate); > environ =3D (char **)*get_program_var_addr("environ", &lockstate); > lock_release(rtld_bind_lock, &lockstate); > @@ -2264,7 +2264,7 @@ do_dlsym(void *handle, const char *name, void *reta= ddr, const Ver_Entry *ve, > req.lockstate =3D &lockstate; >=20=20 > rlock_acquire(rtld_bind_lock, &lockstate); > - if (setjmp(lockstate.env) !=3D 0) > + if (sigsetjmp(lockstate.env, 0) !=3D 0) > lock_upgrade(rtld_bind_lock, &lockstate); > if (handle =3D=3D NULL || handle =3D=3D RTLD_NEXT || > handle =3D=3D RTLD_DEFAULT || handle =3D=3D RTLD_SELF) { > diff --git a/libexec/rtld-elf/rtld.h b/libexec/rtld-elf/rtld.h > index 8941d29..bb365a7 100644 > --- a/libexec/rtld-elf/rtld.h > +++ b/libexec/rtld-elf/rtld.h > @@ -276,7 +276,7 @@ typedef struct Struct_DoneList { >=20=20 > struct Struct_RtldLockState { > int lockstate; > - jmp_buf env; > + sigjmp_buf env; > }; >=20=20 > /* > diff --git a/libexec/rtld-elf/rtld_lock.c b/libexec/rtld-elf/rtld_lock.c > index e76a4da..024e1e2 100644 > --- a/libexec/rtld-elf/rtld_lock.c > +++ b/libexec/rtld-elf/rtld_lock.c > @@ -259,7 +259,7 @@ lock_restart_for_upgrade(RtldLockState *lockstate) > case RTLD_LOCK_WLOCKED: > break; > case RTLD_LOCK_RLOCKED: > - longjmp(lockstate->env, 1); > + siglongjmp(lockstate->env, 1); > break; > default: > assert(0); =2D-=20 Regards, Ali Polatel --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) iEYEARECAAYFAk1THlQACgkQQU4yORhF8iAxRQCeIJiSlayl8G+85db+KvnbMz3p dGsAn3JV0xCN8ZD9bLlxYsRz9ug9I3ic =UIXh -----END PGP SIGNATURE----- --=-=-=-- From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 10 04:56:23 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 968761065670 for ; Thu, 10 Feb 2011 04:56:23 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 4617A8FC14 for ; Thu, 10 Feb 2011 04:56:23 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id p1A4spVt096725 for ; Wed, 9 Feb 2011 21:54:52 -0700 (MST) (envelope-from imp@bsdimp.com) Message-ID: <4D536F9B.6070809@bsdimp.com> Date: Wed, 09 Feb 2011 21:54:51 -0700 From: Warner Losh User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20101211 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <87vd0ys2no.fsf@gmail.com> <87fws2rqui.fsf@gmail.com> <87sjw1rfuo.fsf@gmail.com> In-Reply-To: <87sjw1rfuo.fsf@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Missing stdint.h includes? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Feb 2011 04:56:23 -0000 On 02/05/2011 20:39, Raphael Kubo da Costa wrote: > Garrett Cooper writes: > >> On Sat, Feb 5, 2011 at 3:42 PM, Raphael Kubo da Costa wrote: >>> As for devinfo.h, shouldn't it just include stdint.h, sys/types.h or >>> sys/param.h? >> sys/types.h should suffice, even though POSIX says it should be in >> inttypes.h: http://pubs.opengroup.org/onlinepubs/009695399/basedefs/inttypes.h.html#tag_13_20 >> >> Kind of odd why they're redefined there (in sys/types.h) to be honest, >> esp because POSIX doesn't say that they should be defined there. >> Probably some pollution introduced that would widely break compiles >> because sys/types.h is used everywhere, not inttypes.h. >> >>> What's the best way for me to help with this, PR-wise? >> I would honestly not go create more PR fodder. Just find a (doc?) >> developer with a commit bit who's interested in cleaning up >> incorrectness. > I've inlined two patches below: > > * The first one should correct the documentation for rman(9). I'm sure > there are problems with other headers, but I haven't checked yet. It > adds sys/types.h to the man page -- I'm not sure if inttypes.h would > be better (DragonFly just adds sys/param.h, FWIW). > > * The second one makes devinfo.h include sys/types.h instead of > sys/_types.h -- I still think devinfo.h users should not need to > explicitly include whateverdefinesuint32_t.h, as this, IMHO, is an > implementation detail. It also adds the include to devinfo_var.h, > which, although not installed, should be self-contained. Feedback > very welcome. > > As a sidenote, what is the purpose of having both types.h and _types.h? > > Thanks! > > Patch 1: > > Index: stable/8/share/man/man9/rman.9 > =================================================================== > --- stable/8/share/man/man9/rman.9 (revision 218362) > +++ stable/8/share/man/man9/rman.9 (working copy) > @@ -55,6 +55,7 @@ > .Nm rman_get_rid > .Nd resource management functions > .Sh SYNOPSIS > +.In sys/types.h > .In sys/rman.h > .Ft int > .Fn rman_activate_resource "struct resource *r" > > Patch 2: > > Index: stable/8/lib/libdevinfo/devinfo.h > =================================================================== > --- stable/8/lib/libdevinfo/devinfo.h (revision 218362) > +++ stable/8/lib/libdevinfo/devinfo.h (working copy) > @@ -31,7 +31,7 @@ > #define _DEVINFO_H_INCLUDED > > #include > -#include > +#include > > typedef __uintptr_t devinfo_handle_t; > #define DEVINFO_ROOT_DEVICE ((devinfo_handle_t)0) > Index: stable/8/lib/libdevinfo/devinfo_var.h > =================================================================== > --- stable/8/lib/libdevinfo/devinfo_var.h (revision 218362) > +++ stable/8/lib/libdevinfo/devinfo_var.h (working copy) > @@ -27,6 +27,7 @@ > * $FreeBSD$ > */ > > +#include > #include > #include > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > > committed. Warner From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 10 20:28:52 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A9FA31065693 for ; Thu, 10 Feb 2011 20:28:52 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id 686208FC20 for ; Thu, 10 Feb 2011 20:28:52 +0000 (UTC) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.4/8.14.1) with ESMTP id p1AKSpwl067326; Thu, 10 Feb 2011 12:28:51 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.14.4/8.13.4/Submit) id p1AKSnAl067323; Thu, 10 Feb 2011 12:28:49 -0800 (PST) Date: Thu, 10 Feb 2011 12:28:49 -0800 (PST) From: Matthew Dillon Message-Id: <201102102028.p1AKSnAl067323@apollo.backplane.com> To: "Daniel O'Connor" References: <53A394ED-7C2E-4E4B-A9A7-CB5F1B27DBE3@gsoft.com.au> <7108E013-77D1-47F8-892E-5027DB7D432B@gsoft.com.au> <990005CD-39BD-45F6-BD07-ACEE79DF5A03@gsoft.com.au> <772B352C-7241-4326-8B49-3FB675896609@gsoft.com.au> <71549325-5FD1-4516-B49E-098AFBEFE8B7@gsoft.com.au> <88A9D148-7E3C-4ADB-90FC-B95C4D3BBD2E@gsoft.com.au> Cc: freebsd-hackers@freebsd.org, Ivan Voras Subject: Re: Scheduler question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Feb 2011 20:28:52 -0000 It sounds like there are at least two issues involved. The first could be a buffer cache starvation issue due to the load on the filesystem from the tar. If the usb program is doing any filesystem operation at all, even at low bandwidths, it could be hitting blockages due to the disk intensive tar eating up available buffer cache buffers (e.g. causing an excessive ratio of dirty buffers vs clean buffers). This would NOT be a scheduler problem per-say, but instead a kernel resource management problem. The way to test this is to double-buffer or tripple-buffer the output via shared memory. A pipe might not do the job if it gets stuck doing direct transfers (I eventually gave up trying to optimize pipes in DFly due to a similar problem and just run everything through a kernel buffer now). Still, it may be possible to test against this particular problem by having the program write to a pipe and another program or fork handle the actual writing to the disk or filesystem. Another way to test this is to comment out the writing in the usb program entirely and see if things improve. -- The second issue sounds more scheduler-related. Try running the usb program at nice -20? You could even run it at a pseudo-realtime priority using rtprio but nice -20 had better work properly against a md5 or there is something seriously broken in the scheduler. Dynamic priority handling is supposed to deal with this sort of thing automatically, particularly if the usb program is not using a lot of cpu, but sometimes it can't tell whether a newly-exec'd program is going to be interactive or batch until after it has run for a while. Tuning initial conditions after an exec for the scheduler is not an easy task. Simply giving a program a more batch/bulk-run priority on exec and letting the dynamic priority shift it more to interactive operation tends to mess up interactive shells in the face of cpu-intensive system operation, for example. Theoretically dynamic priority handling should bump up the priority for the usb program well beyond any initial conditions for exec once it has been running a while, assuming it doesn't use tons of cpu. -- An md5, or any single-file reading operation, would not overload the buffer cache. File writing and/or multi-file operations (such as a tar extraction or a tar-up) can create blockages in the buffer cache. It takes a considerable amount of VM/buffer-cache tuning to get those subsystems to pipeline properly and sometimes things can go stale and stop pipelining properly for months without anyone realizing it. -Matt From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 00:28:10 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A0EE6106566B for ; Fri, 11 Feb 2011 00:28:10 +0000 (UTC) (envelope-from lgj@usenix.org) Received: from lonestar.usenix.org (lonestar.usenix.org [131.106.3.102]) by mx1.freebsd.org (Postfix) with ESMTP id 80B468FC12 for ; Fri, 11 Feb 2011 00:28:10 +0000 (UTC) Received: from negroni.usenix.org (negroni.usenix.org [131.106.3.145]) (authenticated bits=0) by lonestar.usenix.org (8.14.2/8.14.2) with ESMTP id p1B0Q2UL000596 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO) for ; Thu, 10 Feb 2011 16:28:10 -0800 (PST) From: Lionel Garth Jones Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Thu, 10 Feb 2011 16:28:10 -0800 Message-Id: <93D2848D-7E47-434D-97F5-4ADA4A49FB3F@usenix.org> To: freebsd-hackers@freebsd.org Mime-Version: 1.0 (Apple Message framework v1082) X-Mailer: Apple Mail (2.1082) X-DCC-USENIX-Metrics: lonestar; whitelist X-Spam-Status: No, score=0.7 required=6.0 tests=ALL_TRUSTED, FH_DATE_PAST_20XX autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on lonestar X-Mailman-Approved-At: Fri, 11 Feb 2011 00:53:44 +0000 Subject: WIOV '11 Call for Papers Now Available X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 00:28:10 -0000 The Program Committee for the 3rd Workshop on I/O Virtualization (WIOV '11) invites you to contribute your work. Virtualization technology has grown beyond server systems to include desktops, notebooks, and even mobile devices. The rich settings in which virtualized systems are used range from cloud computing infrastructures requiring access to high-end I/O devices like GPUs and networks to the many devices present in embedded and mobile systems. Consequent requirements imposed on I/O include the need for extreme-scale performance in high performance or low latency systems; fairness in device sharing in cloud and server systems, which includes online methods for understanding the I/O behavior of applications; trusted I/O both for cloud and for mobile systems; and low overhead and reliable device access in the mobile or embedded domains. While I/O systems have a long history of using abstraction to shield applications from the intricacies of devices and storage systems, this workshop's focus is on the interplay of I/O with virtualization and cloud technologies. Our goal is to provide a forum to discuss the impact and challenges of I/O virtualization along multiple dimensions, including: the application domains in which virtualized solutions are used and the subsystems involved; platform or hardware support for I/O virtualization; the virtual machine monitor; the guest operating system; the processor, GPU, memory, and I/O subsystems. Also of interest are higher-level virtualization solutions, such as those concerning API virtualization, new virtual I/O APIs like those used for cloud storage, and innovative approaches to device virtualization in cloud computing and in the mobile domain. Topics of interest include but are not limited to: * Hardware support for I/O virtualization * Novel I/O device architectures for virtualization * New software approaches to I/O virtualization * Hardware and software methods for I/O device emulation * Para-virtualized I/O device driver design for virtualization * Virtual machine monitors' support for I/O * Security and trust considerations for virtualized I/O, including = hardware support * Energy-efficient I/O virtualization * Virtualized I/O in the cloud: API virtualization, challenges of large-scale I/O (performance, availability) * Virtualization mechanisms for datacenter networking * I/O monitoring and black box methods or system support for such = functions Paper submissions are due Monday, March 21, 2011, at 11:59 p.m. PST. Submissions guidelines and more information can be a found at http://www.usenix.org/wiov11/cfpa WIOV '11 will be part of USENIX Federated Conferences Week, which will take place June 12-17, 2011, in Portland, OR. We look forward to receiving your submissions! On behalf of the WIOV '11 Program Committee, Sanjay Kumar, Intel Labs Himanshu Raj, Microsoft Research Karsten Schwan, Georgia Institute of Technology WIOV '11 Program Co-Chairs wiov11chairs@usenix.org ----------------------------------------------------------------------- WIOV '11 Call for Papers 3rd Workshop on I/O Virtualization (WIOV '11)=20 June 14, 2011, Portland, OR Part of USENIX Federated Conferences Week, June 12-17, 2011 http://www.usenix.org/wiov11/cfpa Submission deadline: March 21, 2011, 11:59 p.m. PST=20 ------------------------------------------------------------------------ From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 03:53:45 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D23251065693; Fri, 11 Feb 2011 03:53:45 +0000 (UTC) (envelope-from dteske@vicor.com) Received: from postoffice.vicor.com (postoffice.vicor.com [69.26.56.53]) by mx1.freebsd.org (Postfix) with ESMTP id B80858FC18; Fri, 11 Feb 2011 03:53:45 +0000 (UTC) Received: from [192.82.228.105] (port=59572) by postoffice.vicor.com with esmtpsa (SSLv3:AES256-SHA:256) (Exim 4.71) (envelope-from ) id 1Pnk4l-0005Sa-Nq; Thu, 10 Feb 2011 19:53:45 -0800 From: Devin Teske To: freebsd-questions@freebsd.org Content-Type: multipart/mixed; boundary="=-+A3QXkrxgonKAGyxTIQq" Organization: VICOR, Inc. Date: Thu, 10 Feb 2011 19:53:53 -0800 Message-ID: <1297396433.7232.9.camel@dt.vicor.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 X-Scan-Signature: 4266861497c25e0c814ad9f3c38b9441 X-Scan-Host: postoffice.vicor.com X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: [RELEASE] host-setup(1): a dialog(1)-based utility for configuring FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 03:53:45 -0000 --=-+A3QXkrxgonKAGyxTIQq Content-Type: text/plain; charset="cp1252" Content-Transfer-Encoding: 7bit Hi All, I'd like to announce the release of a new script. A script that I've developed for our field engineers that I'd like to share with the rest of the world. http://druidbsd.sourceforge.net/download/host-setup.txt host-setup(1) is a dialog(1)-based utility (written in sh(1)) designed to make configuring FreeBSD more efficient. We have this script configured to be run as root's initial login immediately after "first-boot". The field engineer uses our custom installer to install RELENG_8, and after the machine presents a login prompt, they login with the pre-configured root password. After which they are presented with this dialog (image attached: host-setup.pub.png). The dialogs should all be intuitive, and I hope that you like my work. I've worked very hard to make this utility smooth, robust, fault-tolerant and even enjoyable to use. Not only that, but it helps prevents mistakes that could arise in doing these steps by-hand (like forgetting to unmount active NFS-mounts before executing "route flush"). Please give it a try and let me know what you think. -- Devin P.S. This is not a trivial script. ``More nuclear reactor than bike shed'' ^_^ P.P.S. Should be backward compatible to RELENG_4. P.P.S. I know the screenshot says "host-setup.pub" -- that's only because our in-house-only version has more functionality than the one I'm releasing to the general public today (no functionality that anyone in the public audience would ever care about though). --=-+A3QXkrxgonKAGyxTIQq-- From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 07:42:13 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B132F106566B for ; Fri, 11 Feb 2011 07:42:13 +0000 (UTC) (envelope-from freebsd-hackers@herveybayaustralia.com.au) Received: from mail.unitedinsong.com.au (mail.unitedinsong.com.au [150.101.178.33]) by mx1.freebsd.org (Postfix) with ESMTP id 5EC458FC18 for ; Fri, 11 Feb 2011 07:42:13 +0000 (UTC) Received: from laptop1.herveybayaustralia.com.au (laptop1.herveybayaustralia.com.au [192.168.0.186]) by mail.unitedinsong.com.au (Postfix) with ESMTP id EB0C35C44 for ; Fri, 11 Feb 2011 17:31:50 +1000 (EST) Message-ID: <4D54E39D.1000505@herveybayaustralia.com.au> Date: Fri, 11 Feb 2011 17:22:05 +1000 From: Da Rock User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.16) Gecko/20110204 Thunderbird/3.0.11 ThunderBrowse/3.3.4 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: linux PF_PACKET compatibility X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 07:42:13 -0000 "In recent versions of the Linux kernel (post-2.0 releases) a new protocol family has been introduced, named PF_PACKET. This family allows an application to send and receive packets dealing directly with the network card driver, thus avoiding the usual protocol stack-handling (e.g., IP/TCP or IP/UDP processing). That is, any packet sent through the socket will be directly passed to the Ethernet interface, and any packet received through the interface will be directly passed to the application." I've been chasing the answer to a FreeBSD version of this (approx. anyway), but I needed to find out what exactly PF_PACKET was first. Finally found this answer here: http://www.linuxjournal.com/article/4659 I looked up man socket and I can see possibilities (in my mind anyway), but I thought I'd be best to check if the gurus here might have a better idea. My reason for this is I'm attempting to build l2tpns (which supposedly builds on 7.3?! with no trouble), and I'm chasing the errors which appear to be linuxisms mostly. So in man socket simply looking at the list of protocol families I'd say network driver level would be similar to PF_LINK link layer interface? Is there another man page I should be looking at as well? FWIW my gmake output is this: gcc -Wall -Wformat-security -Wno-format-zero-length -g -O3 -I. -I/usr/include -I/usr/local/include -DLIBDIR='"/lib/l2tpns"' -DETCDIR='"/etc/l2tpns"' -DSTATISTICS -DSTAT_CALLS -DRINGBUFFER -DHAVE_EPOLL -DBGP -c -o arp.o arp.c arp.c: In function 'sendarp': arp.c:34: error: storage size of 'sll' isn't known arp.c:59: error: 'PF_PACKET' undeclared (first use in this function) arp.c:59: error: (Each undeclared identifier is reported only once arp.c:59: error: for each function it appears in.) arp.c:62: error: 'AF_PACKET' undeclared (first use in this function) arp.c:34: warning: unused variable 'sll' gmake: *** [arp.o] Error 1 I posted this over at -questions@ but it was suggested to try here instead (or -net@). I figured this would be the best place to start... :) Cheers From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 08:17:44 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 39EDA106564A for ; Fri, 11 Feb 2011 08:17:44 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id E5E8C8FC12 for ; Fri, 11 Feb 2011 08:17:43 +0000 (UTC) Received: from julian-mac.elischer.org (home-nat.elischer.org [67.100.89.137]) (authenticated bits=0) by vps1.elischer.org (8.14.4/8.14.4) with ESMTP id p1B8HebD067020 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 11 Feb 2011 00:17:41 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <4D54F0B0.7010503@freebsd.org> Date: Fri, 11 Feb 2011 00:17:52 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Da Rock References: <4D54E39D.1000505@herveybayaustralia.com.au> In-Reply-To: <4D54E39D.1000505@herveybayaustralia.com.au> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: linux PF_PACKET compatibility X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 08:17:44 -0000 On 2/10/11 11:22 PM, Da Rock wrote: > "In recent versions of the Linux kernel (post-2.0 releases) a new > protocol family has been introduced, named PF_PACKET. This family > allows an application to send and receive packets dealing directly > with the network card driver, thus avoiding the usual protocol > stack-handling (e.g., IP/TCP or IP/UDP processing). That is, any > packet sent through the socket will be directly passed to the > Ethernet interface, and any packet received through the interface > will be directly passed to the application." > > I've been chasing the answer to a FreeBSD version of this (approx. > anyway), but I needed to find out what exactly PF_PACKET was first. > Finally found this answer here: > http://www.linuxjournal.com/article/4659 > > I looked up man socket and I can see possibilities (in my mind > anyway), but I thought I'd be best to check if the gurus here might > have a better idea. My reason for this is I'm attempting to build > l2tpns (which supposedly builds on 7.3?! with no trouble), and I'm > chasing the errors which appear to be linuxisms mostly. > > So in man socket simply looking at the list of protocol families I'd > say network driver level would be similar to PF_LINK link layer > interface? Is there another man page I should be looking at as well? We don't have an exact equivalent.. but we have ways of doing the same thing. one way that is suggested is to use pcap and bpf which I am pretty certain has been enhanced to allow sending as well as receiving. you can also hook directly to the interface using netgraph(4) there are other ways too but those are the two that came to mind immediately. > > FWIW my gmake output is this: > > gcc -Wall -Wformat-security -Wno-format-zero-length -g -O3 -I. > -I/usr/include -I/usr/local/include -DLIBDIR='"/lib/l2tpns"' > -DETCDIR='"/etc/l2tpns"' -DSTATISTICS -DSTAT_CALLS -DRINGBUFFER > -DHAVE_EPOLL -DBGP -c -o arp.o arp.c > arp.c: In function 'sendarp': > arp.c:34: error: storage size of 'sll' isn't known > arp.c:59: error: 'PF_PACKET' undeclared (first use in this function) > arp.c:59: error: (Each undeclared identifier is reported only once > arp.c:59: error: for each function it appears in.) > arp.c:62: error: 'AF_PACKET' undeclared (first use in this function) > arp.c:34: warning: unused variable 'sll' > gmake: *** [arp.o] Error 1 > > I posted this over at -questions@ but it was suggested to try here > instead (or -net@). I figured this would be the best place to > start... :) > > Cheers > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 08:35:34 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A57951065670 for ; Fri, 11 Feb 2011 08:35:34 +0000 (UTC) (envelope-from dudu@dudu.ro) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 31D378FC17 for ; Fri, 11 Feb 2011 08:35:33 +0000 (UTC) Received: by wyf19 with SMTP id 19so2235770wyf.13 for ; Fri, 11 Feb 2011 00:35:33 -0800 (PST) Received: by 10.216.18.194 with SMTP id l44mr166399wel.87.1297411466112; Fri, 11 Feb 2011 00:04:26 -0800 (PST) MIME-Version: 1.0 Received: by 10.216.185.9 with HTTP; Fri, 11 Feb 2011 00:03:46 -0800 (PST) In-Reply-To: <4D54E39D.1000505@herveybayaustralia.com.au> References: <4D54E39D.1000505@herveybayaustralia.com.au> From: Vlad Galu Date: Fri, 11 Feb 2011 10:03:46 +0200 Message-ID: To: Da Rock Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: Re: linux PF_PACKET compatibility X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 08:35:34 -0000 On Fri, Feb 11, 2011 at 9:22 AM, Da Rock < freebsd-hackers@herveybayaustralia.com.au> wrote: > "In recent versions of the Linux kernel (post-2.0 releases) a new protocol > family has been introduced, named PF_PACKET. This family allows an > application to send and receive packets dealing directly with the network > card driver, thus avoiding the usual protocol stack-handling (e.g., IP/TCP > or IP/UDP processing). That is, any packet sent through the socket will be > directly passed to the Ethernet interface, and any packet received through > the interface will be directly passed to the application." > > I've been chasing the answer to a FreeBSD version of this (approx. anyway), > but I needed to find out what exactly PF_PACKET was first. Finally found > this answer here: http://www.linuxjournal.com/article/4659 > > I looked up man socket and I can see possibilities (in my mind anyway), but > I thought I'd be best to check if the gurus here might have a better idea. > My reason for this is I'm attempting to build l2tpns (which supposedly > builds on 7.3?! with no trouble), and I'm chasing the errors which appear to > be linuxisms mostly. > > So in man socket simply looking at the list of protocol families I'd say > network driver level would be similar to PF_LINK link layer interface? Is > there another man page I should be looking at as well? > > FWIW my gmake output is this: > > gcc -Wall -Wformat-security -Wno-format-zero-length -g -O3 -I. > -I/usr/include -I/usr/local/include -DLIBDIR='"/lib/l2tpns"' > -DETCDIR='"/etc/l2tpns"' -DSTATISTICS -DSTAT_CALLS -DRINGBUFFER -DHAVE_EPOLL > -DBGP -c -o arp.o arp.c > arp.c: In function 'sendarp': > arp.c:34: error: storage size of 'sll' isn't known > arp.c:59: error: 'PF_PACKET' undeclared (first use in this function) > arp.c:59: error: (Each undeclared identifier is reported only once > arp.c:59: error: for each function it appears in.) > arp.c:62: error: 'AF_PACKET' undeclared (first use in this function) > arp.c:34: warning: unused variable 'sll' > gmake: *** [arp.o] Error 1 > > I posted this over at -questions@ but it was suggested to try here instead > (or -net@). I figured this would be the best place to start... :) > > Cheers > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > Take a look at bpf(4). I believe you could bypass the TCP/IP stack with netgraph(4), as well, although with possibly more overhead. -- Good, fast & cheap. Pick any two. From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 09:38:36 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F8A0106566B for ; Fri, 11 Feb 2011 09:38:36 +0000 (UTC) (envelope-from freebsd-hackers@herveybayaustralia.com.au) Received: from mail.unitedinsong.com.au (mail.unitedinsong.com.au [150.101.178.33]) by mx1.freebsd.org (Postfix) with ESMTP id B50878FC1C for ; Fri, 11 Feb 2011 09:38:35 +0000 (UTC) Received: from laptop1.herveybayaustralia.com.au (laptop1.herveybayaustralia.com.au [192.168.0.186]) by mail.unitedinsong.com.au (Postfix) with ESMTP id 1490C5C44; Fri, 11 Feb 2011 19:45:44 +1000 (EST) Message-ID: <4D550300.5090000@herveybayaustralia.com.au> Date: Fri, 11 Feb 2011 19:36:00 +1000 From: Da Rock User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.16) Gecko/20110204 Thunderbird/3.0.11 ThunderBrowse/3.3.4 MIME-Version: 1.0 To: Julian Elischer References: <4D54E39D.1000505@herveybayaustralia.com.au> <4D54F0B0.7010503@freebsd.org> In-Reply-To: <4D54F0B0.7010503@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, dudu@dudu.ro Subject: Re: linux PF_PACKET compatibility X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 09:38:36 -0000 On 02/11/11 18:17, Julian Elischer wrote: > On 2/10/11 11:22 PM, Da Rock wrote: >> "In recent versions of the Linux kernel (post-2.0 releases) a new >> protocol family has been introduced, named PF_PACKET. This family >> allows an application to send and receive packets dealing directly >> with the network card driver, thus avoiding the usual protocol >> stack-handling (e.g., IP/TCP or IP/UDP processing). That is, any >> packet sent through the socket will be directly passed to the >> Ethernet interface, and any packet received through the interface >> will be directly passed to the application." >> >> I've been chasing the answer to a FreeBSD version of this (approx. >> anyway), but I needed to find out what exactly PF_PACKET was first. >> Finally found this answer here: http://www.linuxjournal.com/article/4659 >> >> I looked up man socket and I can see possibilities (in my mind >> anyway), but I thought I'd be best to check if the gurus here might >> have a better idea. My reason for this is I'm attempting to build >> l2tpns (which supposedly builds on 7.3?! with no trouble), and I'm >> chasing the errors which appear to be linuxisms mostly. >> >> So in man socket simply looking at the list of protocol families I'd >> say network driver level would be similar to PF_LINK link layer >> interface? Is there another man page I should be looking at as well? > > We don't have an exact equivalent.. but we have ways of doing the > same thing. > one way that is suggested is to use pcap and bpf which I am pretty > certain has been enhanced to allow sending as > well as receiving. > you can also hook directly to the interface using netgraph(4) > there are other ways too but those are the two that came to mind > immediately. So I'm going to have to rewrite that interface entirely? Bugger! I just can't fathom how this howto could even exist for l2tpns on FreeBSD if it isn't even close to buildable... weird! http://kuapp.com/2010/07/14/how-to-setup-l2tpipsec-vpn-on-freebsd.html Thanks guys. I'll probably come back with more problems as I slowly crack this one... :) > > >> >> FWIW my gmake output is this: >> >> gcc -Wall -Wformat-security -Wno-format-zero-length -g -O3 -I. >> -I/usr/include -I/usr/local/include -DLIBDIR='"/lib/l2tpns"' >> -DETCDIR='"/etc/l2tpns"' -DSTATISTICS -DSTAT_CALLS -DRINGBUFFER >> -DHAVE_EPOLL -DBGP -c -o arp.o arp.c >> arp.c: In function 'sendarp': >> arp.c:34: error: storage size of 'sll' isn't known >> arp.c:59: error: 'PF_PACKET' undeclared (first use in this function) >> arp.c:59: error: (Each undeclared identifier is reported only once >> arp.c:59: error: for each function it appears in.) >> arp.c:62: error: 'AF_PACKET' undeclared (first use in this function) >> arp.c:34: warning: unused variable 'sll' >> gmake: *** [arp.o] Error 1 >> >> I posted this over at -questions@ but it was suggested to try here >> instead (or -net@). I figured this would be the best place to >> start... :) >> >> Cheers >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to >> "freebsd-hackers-unsubscribe@freebsd.org" >> > From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 09:55:00 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E8DF106564A for ; Fri, 11 Feb 2011 09:55:00 +0000 (UTC) (envelope-from dudu@dudu.ro) Received: from mail-ww0-f42.google.com (mail-ww0-f42.google.com [74.125.82.42]) by mx1.freebsd.org (Postfix) with ESMTP id 992EF8FC1A for ; Fri, 11 Feb 2011 09:54:59 +0000 (UTC) Received: by wwi17 with SMTP id 17so4017561wwi.1 for ; Fri, 11 Feb 2011 01:54:58 -0800 (PST) Received: by 10.216.165.204 with SMTP id e54mr272446wel.48.1297418096460; Fri, 11 Feb 2011 01:54:56 -0800 (PST) MIME-Version: 1.0 Received: by 10.216.185.9 with HTTP; Fri, 11 Feb 2011 01:54:16 -0800 (PST) In-Reply-To: <4D550300.5090000@herveybayaustralia.com.au> References: <4D54E39D.1000505@herveybayaustralia.com.au> <4D54F0B0.7010503@freebsd.org> <4D550300.5090000@herveybayaustralia.com.au> From: Vlad Galu Date: Fri, 11 Feb 2011 11:54:16 +0200 Message-ID: To: Da Rock Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: Re: linux PF_PACKET compatibility X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 09:55:00 -0000 On Fri, Feb 11, 2011 at 11:36 AM, Da Rock < freebsd-hackers@herveybayaustralia.com.au> wrote: > On 02/11/11 18:17, Julian Elischer wrote: > >> On 2/10/11 11:22 PM, Da Rock wrote: >> >>> "In recent versions of the Linux kernel (post-2.0 releases) a new >>> protocol family has been introduced, named PF_PACKET. This family allows an >>> application to send and receive packets dealing directly with the network >>> card driver, thus avoiding the usual protocol stack-handling (e.g., IP/TCP >>> or IP/UDP processing). That is, any packet sent through the socket will be >>> directly passed to the Ethernet interface, and any packet received through >>> the interface will be directly passed to the application." >>> >>> I've been chasing the answer to a FreeBSD version of this (approx. >>> anyway), but I needed to find out what exactly PF_PACKET was first. Finally >>> found this answer here: http://www.linuxjournal.com/article/4659 >>> >>> I looked up man socket and I can see possibilities (in my mind anyway), >>> but I thought I'd be best to check if the gurus here might have a better >>> idea. My reason for this is I'm attempting to build l2tpns (which supposedly >>> builds on 7.3?! with no trouble), and I'm chasing the errors which appear to >>> be linuxisms mostly. >>> >>> So in man socket simply looking at the list of protocol families I'd say >>> network driver level would be similar to PF_LINK link layer interface? Is >>> there another man page I should be looking at as well? >>> >> >> We don't have an exact equivalent.. but we have ways of doing the same >> thing. >> one way that is suggested is to use pcap and bpf which I am pretty certain >> has been enhanced to allow sending as >> well as receiving. >> you can also hook directly to the interface using netgraph(4) >> there are other ways too but those are the two that came to mind >> immediately. >> > So I'm going to have to rewrite that interface entirely? Bugger! I just > can't fathom how this howto could even exist for l2tpns on FreeBSD if it > isn't even close to buildable... weird! > > http://kuapp.com/2010/07/14/how-to-setup-l2tpipsec-vpn-on-freebsd.html > > Thanks guys. I'll probably come back with more problems as I slowly crack > this one... :) > > I suppose you could just use mpd :) -- Good, fast & cheap. Pick any two. From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 09:48:00 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7EC901065670 for ; Fri, 11 Feb 2011 09:48:00 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id F270E8FC15 for ; Fri, 11 Feb 2011 09:47:59 +0000 (UTC) Received: from outgoing.leidinger.net (p57B3B9CC.dip.t-dialin.net [87.179.185.204]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 37149844012; Fri, 11 Feb 2011 10:30:32 +0100 (CET) Received: from webmail.leidinger.net (unknown [IPv6:fd73:10c7:2053:1::2:102]) by outgoing.leidinger.net (Postfix) with ESMTP id 2274B2F48; Fri, 11 Feb 2011 10:30:29 +0100 (CET) Received: (from www@localhost) by webmail.leidinger.net (8.14.4/8.13.8/Submit) id p1B9USQ2072052; Fri, 11 Feb 2011 10:30:28 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.ec.europa.eu (pslux.ec.europa.eu [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Fri, 11 Feb 2011 10:30:28 +0100 Message-ID: <20110211103028.12684f54yrw8tgqo@webmail.leidinger.net> Date: Fri, 11 Feb 2011 10:30:28 +0100 From: Alexander Leidinger To: hackers@freebsd.org MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=_3y2cjioe7e80" Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 37149844012.A7A5E X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=1.351, required 6, autolearn=disabled, RDNS_NONE 1.27, TW_KM 0.08) X-EBL-MailScanner-SpamScore: s X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1298021433.35377@C3NLCtrvSL2/Pe4y3HmBfw X-EBL-Spam-Status: No X-Mailman-Approved-At: Fri, 11 Feb 2011 12:15:57 +0000 Cc: kibab@freebsd.org Subject: CFR: FEATURE macros for AUDIT/CAM/IPC/KTR/MAC/NFS/NTP/PMC/SYSV/... X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 09:48:00 -0000 This message is in MIME format. --=_3y2cjioe7e80 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit Hi, during the last GSoC various FEATURE macros where added to the system. Before committing them, I would like to get some review (like if macro is in the correct file, and for those FEATURES where the description was not taken from NOTES if the description is OK). If nobody complains, I would like to commit this in 1-2 weeks. If you need more time to review, just tell me. Here is the list of affected files (for those impatient ones which do not want to look at the attached patch before noticing that they are not interested to look at it): ---snip--- cam/cam.c fs/nfsclient/nfs_clvfsops.c fs/nfsserver/nfs_nfsdport.c kern/kern_dtrace.c kern/kern_ktr.c kern/kern_ktrace.c kern/kern_lock.c kern/kern_lockstat.c kern/kern_ntptime.c kern/kern_pmc.c kern/kern_prot.c kern/ksched.c kern/subr_mchain.c kern/subr_stack.c kern/sysv_msg.c kern/sysv_sem.c kern/sysv_shm.c kern/uipc_cow.c kern/uipc_mqueue.c kern/uipc_sem.c nfsclient/nfs_vfsops.c nfsserver/nfs_serv.c security/audit/audit.c security/mac/mac_syscalls.c ---snip--- Thanks, Alexander. -- Most people in this society who aren't actively mad are, at best, reformed or potential lunatics. -- Susan Sontag http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 --=_3y2cjioe7e80 Content-Type: text/x-patch; charset=UTF-8; name="features_rest.diff" Content-Disposition: attachment; filename="features_rest.diff" Content-Transfer-Encoding: 7bit Index: cam/cam.c =================================================================== --- cam/cam.c (Revision 218482) +++ cam/cam.c (Arbeitskopie) @@ -51,6 +51,9 @@ #include #include #include + +FEATURE(scbus, "SCSI devices support"); + #endif static int camstatusentrycomp(const void *key, const void *member); Index: fs/nfsclient/nfs_clvfsops.c =================================================================== --- fs/nfsclient/nfs_clvfsops.c (Revision 218482) +++ fs/nfsclient/nfs_clvfsops.c (Arbeitskopie) @@ -73,6 +73,8 @@ #include #include +FEATURE(nfscl, "NFSv4 client"); + extern int nfscl_ticks; extern struct timeval nfsboottime; extern struct nfsstats newnfsstats; Index: fs/nfsserver/nfs_nfsdport.c =================================================================== --- fs/nfsserver/nfs_nfsdport.c (Revision 218482) +++ fs/nfsserver/nfs_nfsdport.c (Arbeitskopie) @@ -46,6 +46,8 @@ #include #include +FEATURE(nfsd, "NFSv4 server"); + extern u_int32_t newnfs_true, newnfs_false, newnfs_xdrneg1; extern int nfsrv_useacl; extern int newnfs_numnfsd; Index: kern/kern_dtrace.c =================================================================== --- kern/kern_dtrace.c (Revision 218482) +++ kern/kern_dtrace.c (Arbeitskopie) @@ -37,10 +37,14 @@ #include #include #include +#include #define KDTRACE_PROC_SIZE 64 #define KDTRACE_THREAD_SIZE 256 +FEATURE(kdtrace_hooks, + "Kernel DTrace hooks which are required to load DTrace kernel modules"); + MALLOC_DEFINE(M_KDTRACE, "kdtrace", "DTrace hooks"); /* Return the DTrace process data size compiled in the kernel hooks. */ Index: kern/kern_ktr.c =================================================================== --- kern/kern_ktr.c (Revision 218482) +++ kern/kern_ktr.c (Arbeitskopie) @@ -80,6 +80,8 @@ #define KTR_CPU PCPU_GET(cpuid) #endif +FEATURE(ktr, "Kernel support for KTR kernel tracing facility"); + SYSCTL_NODE(_debug, OID_AUTO, ktr, CTLFLAG_RD, 0, "KTR options"); int ktr_cpumask = KTR_CPUMASK; Index: kern/kern_ktrace.c =================================================================== --- kern/kern_ktrace.c (Revision 218482) +++ kern/kern_ktrace.c (Arbeitskopie) @@ -83,6 +83,8 @@ #ifdef KTRACE +FEATURE(ktrace, "Kernel support for system-call tracing"); + #ifndef KTRACE_REQUEST_POOL #define KTRACE_REQUEST_POOL 100 #endif Index: kern/kern_lock.c =================================================================== --- kern/kern_lock.c (Revision 218482) +++ kern/kern_lock.c (Arbeitskopie) @@ -1299,6 +1299,10 @@ } #ifdef INVARIANT_SUPPORT + +FEATURE(invariant_support, + "Support for modules compiled with INVARIANTS option"); + #ifndef INVARIANTS #undef _lockmgr_assert #endif Index: kern/kern_lockstat.c =================================================================== --- kern/kern_lockstat.c (Revision 218482) +++ kern/kern_lockstat.c (Arbeitskopie) @@ -39,7 +39,10 @@ #include #include #include - +#include +#include +#include +FEATURE(kdtrace_hooks, "Kernel DTRACE hooks"); /* * The following must match the type definition of dtrace_probe. It is * defined this way to avoid having to rely on CDDL code. Index: kern/kern_ntptime.c =================================================================== --- kern/kern_ntptime.c (Revision 218482) +++ kern/kern_ntptime.c (Arbeitskopie) @@ -51,6 +51,10 @@ #include #include +#ifdef PPS_SYNC +FEATURE(pps_sync, "Support usage of external PPS signal by kernel PLL"); +#endif + /* * Single-precision macros for 64-bit machines */ Index: kern/kern_pmc.c =================================================================== --- kern/kern_pmc.c (Revision 218482) +++ kern/kern_pmc.c (Arbeitskopie) @@ -37,8 +37,10 @@ #include #include #include +#include #ifdef HWPMC_HOOKS +FEATURE(hwpmc_hooks, "Kernel support for HW PMC"); #define PMC_KERNEL_VERSION PMC_VERSION #else #define PMC_KERNEL_VERSION 0 Index: kern/kern_prot.c =================================================================== --- kern/kern_prot.c (Revision 218482) +++ kern/kern_prot.c (Arbeitskopie) @@ -69,6 +69,11 @@ #include #include +#ifdef REGRESSION +FEATURE(regression, + "Kernel support for interfaces nessesary for regression testing (SECURITY RISK!)"); +#endif + #if defined(INET) || defined(INET6) #include #include Index: kern/ksched.c =================================================================== --- kern/ksched.c (Revision 218482) +++ kern/ksched.c (Arbeitskopie) @@ -41,12 +41,16 @@ #include #include #include +#include +#include #include #include #include #include #include +FEATURE(kposix_priority_scheduling, "POSIX P1003.1B realtime extensions"); + /* ksched: Real-time extension to support POSIX priority scheduling. */ Index: kern/subr_mchain.c =================================================================== --- kern/subr_mchain.c (Revision 218482) +++ kern/subr_mchain.c (Arbeitskopie) @@ -32,6 +32,7 @@ #include #include +#include #include #include #include @@ -40,6 +41,8 @@ #include +FEATURE(libmchain, "mchain library"); + MODULE_VERSION(libmchain, 1); #define MBERROR(format, ...) printf("%s(%d): "format, __func__ , \ Index: kern/subr_stack.c =================================================================== --- kern/subr_stack.c (Revision 218482) +++ kern/subr_stack.c (Arbeitskopie) @@ -39,7 +39,10 @@ #include #include #include +#include +FEATURE(stack, "Support for capturing kernel stack"); + static MALLOC_DEFINE(M_STACK, "stack", "Stack Traces"); static int stack_symbol(vm_offset_t pc, char *namebuf, u_int buflen, Index: kern/sysv_msg.c =================================================================== --- kern/sysv_msg.c (Revision 218482) +++ kern/sysv_msg.c (Arbeitskopie) @@ -72,6 +72,8 @@ #include +FEATURE(sysv_msg, "System V message queues support"); + static MALLOC_DEFINE(M_MSG, "msg", "SVID compatible message queues"); static int msginit(void); Index: kern/sysv_sem.c =================================================================== --- kern/sysv_sem.c (Revision 218482) +++ kern/sysv_sem.c (Arbeitskopie) @@ -62,6 +62,8 @@ #include +FEATURE(sysv_sem, "System V semaphores support"); + static MALLOC_DEFINE(M_SEM, "sem", "SVID compatible semaphores"); #ifdef SEM_DEBUG Index: kern/sysv_shm.c =================================================================== --- kern/sysv_shm.c (Revision 218482) +++ kern/sysv_shm.c (Arbeitskopie) @@ -95,6 +95,8 @@ #include #include +FEATURE(sysv_shm, "System V shared memory segments support"); + static MALLOC_DEFINE(M_SHM, "shm", "SVID compatible shared memory segments"); static int shmget_allocate_segment(struct thread *td, Index: kern/uipc_cow.c =================================================================== --- kern/uipc_cow.c (Revision 218482) +++ kern/uipc_cow.c (Arbeitskopie) @@ -40,6 +40,7 @@ #include #include +#include #include #include #include @@ -57,6 +58,7 @@ #include #include +FEATURE(zero_copy_sockets, "Zero copy sockets support"); struct netsend_cow_stats { int attempted; Index: kern/uipc_mqueue.c =================================================================== --- kern/uipc_mqueue.c (Revision 218482) +++ kern/uipc_mqueue.c (Arbeitskopie) @@ -82,6 +82,8 @@ #include #include +FEATURE(p1003_1b_mqueue, "POSIX P1003.1B message queues support"); + /* * Limits and constants */ Index: kern/uipc_sem.c =================================================================== --- kern/uipc_sem.c (Revision 218482) +++ kern/uipc_sem.c (Arbeitskopie) @@ -65,6 +65,7 @@ #include +FEATURE(p1003_1b_semaphores, "POSIX1003.1B semaphores support"); /* * TODO * Index: nfsclient/nfs_vfsops.c =================================================================== --- nfsclient/nfs_vfsops.c (Revision 218482) +++ nfsclient/nfs_vfsops.c (Arbeitskopie) @@ -78,6 +78,8 @@ #include #include +FEATURE(nfsclient, "NFS client"); + MALLOC_DEFINE(M_NFSREQ, "nfsclient_req", "NFS request header"); MALLOC_DEFINE(M_NFSBIGFH, "nfsclient_bigfh", "NFS version 3 file handle"); MALLOC_DEFINE(M_NFSDIROFF, "nfsclient_diroff", "NFS directory offset data"); Index: nfsserver/nfs_serv.c =================================================================== --- nfsserver/nfs_serv.c (Revision 218482) +++ nfsserver/nfs_serv.c (Arbeitskopie) @@ -97,6 +97,8 @@ #include #include +FEATURE(nfsserver, "NFS server"); + #ifdef NFSRV_DEBUG #define nfsdbprintf(info) printf info #else Index: security/audit/audit.c =================================================================== --- security/audit/audit.c (Revision 218482) +++ security/audit/audit.c (Arbeitskopie) @@ -72,6 +72,8 @@ #include +FEATURE(audit, "BSM audit support"); + static uma_zone_t audit_record_zone; static MALLOC_DEFINE(M_AUDITCRED, "audit_cred", "Audit cred storage"); MALLOC_DEFINE(M_AUDITDATA, "audit_data", "Audit data storage"); Index: security/mac/mac_syscalls.c =================================================================== --- security/mac/mac_syscalls.c (Revision 218482) +++ security/mac/mac_syscalls.c (Arbeitskopie) @@ -56,6 +56,7 @@ #include #include #include +#include #include #include #include @@ -72,6 +73,8 @@ #ifdef MAC +FEATURE(mac, "Mandatory Access Control support"); + int __mac_get_pid(struct thread *td, struct __mac_get_pid_args *uap) { --=_3y2cjioe7e80-- From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 12:56:55 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AE72C106564A; Fri, 11 Feb 2011 12:56:55 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 7D8958FC13; Fri, 11 Feb 2011 12:56:55 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 28DC546B06; Fri, 11 Feb 2011 07:56:55 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.10]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 4D33F8A01D; Fri, 11 Feb 2011 07:56:54 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Fri, 11 Feb 2011 07:51:52 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.4-CBSD-20110107; KDE/4.4.5; amd64; ; ) References: <20110211103028.12684f54yrw8tgqo@webmail.leidinger.net> In-Reply-To: <20110211103028.12684f54yrw8tgqo@webmail.leidinger.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201102110751.52863.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Fri, 11 Feb 2011 07:56:54 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=0.5 required=4.2 tests=BAYES_00,MAY_BE_FORGED, RDNS_DYNAMIC autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Cc: Alexander Leidinger , hackers@freebsd.org, kibab@freebsd.org Subject: Re: CFR: FEATURE macros for AUDIT/CAM/IPC/KTR/MAC/NFS/NTP/PMC/SYSV/... X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 12:56:55 -0000 On Friday, February 11, 2011 4:30:28 am Alexander Leidinger wrote: > Hi, > > during the last GSoC various FEATURE macros where added to the system. > Before committing them, I would like to get some review (like if macro > is in the correct file, and for those FEATURES where the description > was not taken from NOTES if the description is OK). > > If nobody complains, I would like to commit this in 1-2 weeks. If you > need more time to review, just tell me. > > Here is the list of affected files (for those impatient ones which do > not want to look at the attached patch before noticing that they are > not interested to look at it): Hmm, so what is the rationale for adding FEATURE() macros? Do we just want to add them for everything or do we want to add them on-demand as use cases for each knob arrive? Some features can already be inferred (e.g. if KTR is compiled in, then the debug.ktr.mask sysctl will exist). Also, in the case of KTR, I'm not sure that any userland programs need to alter their behavior based on whether or not that feature was present. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 12:56:55 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AE72C106564A; Fri, 11 Feb 2011 12:56:55 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 7D8958FC13; Fri, 11 Feb 2011 12:56:55 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 28DC546B06; Fri, 11 Feb 2011 07:56:55 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.10]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 4D33F8A01D; Fri, 11 Feb 2011 07:56:54 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Fri, 11 Feb 2011 07:51:52 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.4-CBSD-20110107; KDE/4.4.5; amd64; ; ) References: <20110211103028.12684f54yrw8tgqo@webmail.leidinger.net> In-Reply-To: <20110211103028.12684f54yrw8tgqo@webmail.leidinger.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201102110751.52863.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Fri, 11 Feb 2011 07:56:54 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=0.5 required=4.2 tests=BAYES_00,MAY_BE_FORGED, RDNS_DYNAMIC autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Cc: Alexander Leidinger , hackers@freebsd.org, kibab@freebsd.org Subject: Re: CFR: FEATURE macros for AUDIT/CAM/IPC/KTR/MAC/NFS/NTP/PMC/SYSV/... X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 12:56:55 -0000 On Friday, February 11, 2011 4:30:28 am Alexander Leidinger wrote: > Hi, > > during the last GSoC various FEATURE macros where added to the system. > Before committing them, I would like to get some review (like if macro > is in the correct file, and for those FEATURES where the description > was not taken from NOTES if the description is OK). > > If nobody complains, I would like to commit this in 1-2 weeks. If you > need more time to review, just tell me. > > Here is the list of affected files (for those impatient ones which do > not want to look at the attached patch before noticing that they are > not interested to look at it): Hmm, so what is the rationale for adding FEATURE() macros? Do we just want to add them for everything or do we want to add them on-demand as use cases for each knob arrive? Some features can already be inferred (e.g. if KTR is compiled in, then the debug.ktr.mask sysctl will exist). Also, in the case of KTR, I'm not sure that any userland programs need to alter their behavior based on whether or not that feature was present. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 13:43:13 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 898F010656AA; Fri, 11 Feb 2011 13:43:13 +0000 (UTC) (envelope-from freebsd-hackers@herveybayaustralia.com.au) Received: from mail.unitedinsong.com.au (mail.unitedinsong.com.au [150.101.178.33]) by mx1.freebsd.org (Postfix) with ESMTP id BB4D58FC1B; Fri, 11 Feb 2011 13:43:12 +0000 (UTC) Received: from laptop1.herveybayaustralia.com.au (laptop1.herveybayaustralia.com.au [192.168.0.186]) by mail.unitedinsong.com.au (Postfix) with ESMTP id 1CCF65C45; Fri, 11 Feb 2011 23:50:21 +1000 (EST) Message-ID: <4D553C4B.5000101@herveybayaustralia.com.au> Date: Fri, 11 Feb 2011 23:40:27 +1000 From: Da Rock User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.16) Gecko/20110204 Thunderbird/3.0.11 ThunderBrowse/3.3.4 MIME-Version: 1.0 To: Vlad Galu References: <4D54E39D.1000505@herveybayaustralia.com.au> <4D54F0B0.7010503@freebsd.org> <4D550300.5090000@herveybayaustralia.com.au> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: Re: linux PF_PACKET compatibility X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 13:43:13 -0000 On 02/11/11 19:54, Vlad Galu wrote: > > > On Fri, Feb 11, 2011 at 11:36 AM, Da Rock > > wrote: > > On 02/11/11 18:17, Julian Elischer wrote: > > On 2/10/11 11:22 PM, Da Rock wrote: > > "In recent versions of the Linux kernel (post-2.0 > releases) a new protocol family has been introduced, named > PF_PACKET. This family allows an application to send and > receive packets dealing directly with the network card > driver, thus avoiding the usual protocol stack-handling > (e.g., IP/TCP or IP/UDP processing). That is, any packet > sent through the socket will be directly passed to the > Ethernet interface, and any packet received through the > interface will be directly passed to the application." > > I've been chasing the answer to a FreeBSD version of this > (approx. anyway), but I needed to find out what exactly > PF_PACKET was first. Finally found this answer here: > http://www.linuxjournal.com/article/4659 > > I looked up man socket and I can see possibilities (in my > mind anyway), but I thought I'd be best to check if the > gurus here might have a better idea. My reason for this is > I'm attempting to build l2tpns (which supposedly builds on > 7.3?! with no trouble), and I'm chasing the errors which > appear to be linuxisms mostly. > > So in man socket simply looking at the list of protocol > families I'd say network driver level would be similar to > PF_LINK link layer interface? Is there another man page I > should be looking at as well? > > > We don't have an exact equivalent.. but we have ways of doing > the same thing. > one way that is suggested is to use pcap and bpf which I am > pretty certain has been enhanced to allow sending as > well as receiving. > you can also hook directly to the interface using netgraph(4) > there are other ways too but those are the two that came to > mind immediately. > > So I'm going to have to rewrite that interface entirely? Bugger! I > just can't fathom how this howto could even exist for l2tpns on > FreeBSD if it isn't even close to buildable... weird! > > http://kuapp.com/2010/07/14/how-to-setup-l2tpipsec-vpn-on-freebsd.html > > Thanks guys. I'll probably come back with more problems as I > slowly crack this one... :) > > > I suppose you could just use mpd :) I could, I guess. But where's the fun in that? :) Seriously, though, mpd didn't quite cut it (I thought) for me. I need a l2tp vpn server with the capability to handle multiple clients with only one interface. The server is behind a firewall, and I'm trying for a "walled garden" variety I guess. So far my research has brought me here, but I'm open to suggestions. One other that has my attention is l2tpd (in ports). I want radius auth, so IF I can use pppd in base and radius (which as I understand- so far anyway- it needs), and only uses a single interface, then maybe. I'm still hunting and playing- learning on the fly. From what I read mpd uses an ng interface for every single client. L2tpns doesn't, and from what I've read so far neither does l2tpd (I was actually looking at another fork of that xl2tpd). I could use some advice from someone with experience with this, but my feelers on -questions didn't get much response. I may try on -net if this fails... Aside from that I also wanted to get a bit more of a hands on feel for the FreeBSD core. I can't sit on the sidelines yelling at the players any more :) I'm not much for spectator sport either... From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 13:54:54 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 793D91065670; Fri, 11 Feb 2011 13:54:54 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 60CDE8FC1B; Fri, 11 Feb 2011 13:54:52 +0000 (UTC) Received: from outgoing.leidinger.net (p57B3B9CC.dip.t-dialin.net [87.179.185.204]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 2A523844012; Fri, 11 Feb 2011 14:54:49 +0100 (CET) Received: from webmail.leidinger.net (unknown [IPv6:fd73:10c7:2053:1::2:102]) by outgoing.leidinger.net (Postfix) with ESMTP id E4CAA2F64; Fri, 11 Feb 2011 14:54:45 +0100 (CET) Received: (from www@localhost) by webmail.leidinger.net (8.14.4/8.13.8/Submit) id p1BDsjWr032115; Fri, 11 Feb 2011 14:54:45 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.ec.europa.eu (pslux.ec.europa.eu [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Fri, 11 Feb 2011 14:54:45 +0100 Message-ID: <20110211145445.12722r7902rz8vc4@webmail.leidinger.net> Date: Fri, 11 Feb 2011 14:54:45 +0100 From: Alexander Leidinger To: John Baldwin References: <20110211103028.12684f54yrw8tgqo@webmail.leidinger.net> <201102110751.52863.jhb@freebsd.org> In-Reply-To: <201102110751.52863.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 2A523844012.A7430 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=1.274, required 6, autolearn=disabled, RDNS_NONE 1.27) X-EBL-MailScanner-SpamScore: s X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1298037289.71466@LQBnulvZjSdFiOavtlmv+Q X-EBL-Spam-Status: No X-Mailman-Approved-At: Fri, 11 Feb 2011 14:48:48 +0000 Cc: freebsd-hackers@freebsd.org, kibab@freebsd.org Subject: Re: CFR: FEATURE macros for AUDIT/CAM/IPC/KTR/MAC/NFS/NTP/PMC/SYSV/... X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 13:54:54 -0000 Quoting John Baldwin (from Fri, 11 Feb 2011 07:51:52 -0500): > On Friday, February 11, 2011 4:30:28 am Alexander Leidinger wrote: >> Hi, >> >> during the last GSoC various FEATURE macros where added to the system. >> Before committing them, I would like to get some review (like if macro >> is in the correct file, and for those FEATURES where the description >> was not taken from NOTES if the description is OK). >> >> If nobody complains, I would like to commit this in 1-2 weeks. If you >> need more time to review, just tell me. >> >> Here is the list of affected files (for those impatient ones which do >> not want to look at the attached patch before noticing that they are >> not interested to look at it): > > Hmm, so what is the rationale for adding FEATURE() macros? Do we > just want to > add them for everything or do we want to add them on-demand as use cases for > each knob arrive? Some features can already be inferred (e.g. if KTR is The non-answer is: IMO we want to add as much as needed. See below for an explanation. > compiled in, then the debug.ktr.mask sysctl will exist). Also, in > the case of > KTR, I'm not sure that any userland programs need to alter their behavior > based on whether or not that feature was present. The main goal was to have an easy way (e.g. not a lot of parsing) without sideeffects (e.g. autoloading of kernel modules) to query if a feature is available. Regarding this, there is no need to have one for KTR. The 2nd goal is (as you know, as we discussed it in the beginning of the GSoC) to be able to hide a feature administratively (I plan to commit the userland part regarding this last). You can not do this with sysctl, so for me adding a FEATURE for KTR provides more possibilities. I see a use case for this in KTR, an app may see if it is there and start some tracing based upon it (and it could be adminsitratively prohibited by hiding the presence), or there could be a sanity check in a script which prevents some tracing-setup to happen if it is not there (or administratively hidden). In general I do not want to decide if something makes sense for an app or not, as I do not think I can come up with all possible use cases (possibilities instead of policies). As such it makes sense to add more FEATURE macros to the tree than what we have ATM. If used correctly (via the to be committed userland part), this may make the life of some people more easy. As a 3rd point (attention bikeshed ahead), having everything as a FEATURE would give an uniform way of listing what is available or not (with the benefit to administratively hide parts of it). Needless to say that this would reduce the amount of knowledge and code to determine if something is available (as a person who works in a production environement with mixed knowledge levels of the people and who has to analyse problems which are sometimes caused by lack of knowledge of the people which implemented something, I welcome everything which lowers the learning curve and complexity). Bye, Alexander. -- It wasn't exactly a divorce -- I was traded. -- Tim Conway http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 15:01:20 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7DA8D1065670 for ; Fri, 11 Feb 2011 15:01:20 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 19B038FC0C for ; Fri, 11 Feb 2011 15:01:19 +0000 (UTC) Received: by eyf6 with SMTP id 6so1374372eyf.13 for ; Fri, 11 Feb 2011 07:01:18 -0800 (PST) Received: by 10.223.79.74 with SMTP id o10mr197136fak.63.1297411561511; Fri, 11 Feb 2011 00:06:01 -0800 (PST) Received: from [10.202.163.74] ([92.90.20.56]) by mx.google.com with ESMTPS id z1sm188705fau.45.2011.02.11.00.05.57 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 11 Feb 2011 00:06:00 -0800 (PST) References: <1297396433.7232.9.camel@dt.vicor.com> From: Damien Fleuriot Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii In-Reply-To: <1297396433.7232.9.camel@dt.vicor.com> Message-Id: <5AF1B11E-19FF-4FF5-9058-BE8F0AB85E1B@my.gd> Date: Fri, 11 Feb 2011 08:56:42 +0100 To: Devin Teske Mime-Version: 1.0 (iPhone Mail 8A293) X-Mailer: iPhone Mail (8A293) Cc: "freebsd-hackers@freebsd.org" , "freebsd-questions@freebsd.org" Subject: Re: [RELEASE] host-setup(1): a dialog(1)-based utility for configuring FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 15:01:20 -0000 Hi Devin, Thanks for sharing your work. The list strips non-text attachments so there isn't much to see at the momen= t though... --- Fleuriot Damien On 11 Feb 2011, at 04:53, Devin Teske wrote: > Hi All, >=20 > I'd like to announce the release of a new script. A script that I've > developed for our field engineers that I'd like to share with the rest > of the world. >=20 > http://druidbsd.sourceforge.net/download/host-setup.txt >=20 > host-setup(1) is a dialog(1)-based utility (written in sh(1)) designed > to make configuring FreeBSD more efficient. >=20 > We have this script configured to be run as root's initial login > immediately after "first-boot". The field engineer uses our custom > installer to install RELENG_8, and after the machine presents a login > prompt, they login with the pre-configured root password. After which > they are presented with this dialog (image attached: > host-setup.pub.png). >=20 > The dialogs should all be intuitive, and I hope that you like my work. > I've worked very hard to make this utility smooth, robust, > fault-tolerant and even enjoyable to use. Not only that, but it helps > prevents mistakes that could arise in doing these steps by-hand (like > forgetting to unmount active NFS-mounts before executing "route flush"). >=20 > Please give it a try and let me know what you think. > -- > Devin >=20 > P.S. This is not a trivial script. ``More nuclear reactor than bike > shed'' ^_^ >=20 > P.P.S. Should be backward compatible to RELENG_4. >=20 > P.P.S. I know the screenshot says "host-setup.pub" -- that's only > because our in-house-only version has more functionality than the one > I'm releasing to the general public today (no functionality that anyone > in the public audience would ever care about though). > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"= From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 15:07:23 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E97661065675; Fri, 11 Feb 2011 15:07:23 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (unknown [IPv6:2a01:348:0:15:5d59:5c40:0:1]) by mx1.freebsd.org (Postfix) with ESMTP id 1A06F8FC1C; Fri, 11 Feb 2011 15:07:23 +0000 (UTC) Received: from muon.cran.org.uk (localhost [127.0.0.1]) by muon.cran.org.uk (Postfix) with ESMTP id CC7F0E7831; Fri, 11 Feb 2011 15:07:10 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=cran.org.uk; h=date:from :to:cc:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; s=mail; bh=1l9WxvGhwiPh wgMGPRWSjRc+4dM=; b=qSgoH954QvuKwdwPB45UVq0aO1jmyU2oHJH/5UL6fyt9 j+CEdFX7TJdR4sLpaM7BodIXbM929Uhd9KO9GV4gk9uZjvO5m/Yckbb8qLgTXGGC 1e09r+J6p0Q9cZHAIfenfygmALB2iGnTj0Uj39ZzmvHpz8CJGJU3MZzsaLdeKPM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=cran.org.uk; h=date:from:to :cc:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; q=dns; s=mail; b=j/3HGf ElcWt20/1bw5lYdmC3YL7mne2JKWIjpk1DisVkmZCdhmNI4KNtENU1OmhigkDJDl 7UuTmMuo61sfHvHBfTusov/qFOHtxBANZouhPE94a1qxWLlV95YxtJjdtdRWhc3C lfHPt5YGc/4FaxloYoeVsPmaFNETkUgW3dd5o= Received: from unknown (client-86-23-69-78.brhm.adsl.virginmedia.com [86.23.69.78]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA id 7B59FE76F0; Fri, 11 Feb 2011 15:07:10 +0000 (GMT) Date: Fri, 11 Feb 2011 15:06:55 +0000 From: Bruce Cran To: Damien Fleuriot Message-ID: <20110211150655.0000576d@unknown> In-Reply-To: <5AF1B11E-19FF-4FF5-9058-BE8F0AB85E1B@my.gd> References: <1297396433.7232.9.camel@dt.vicor.com> <5AF1B11E-19FF-4FF5-9058-BE8F0AB85E1B@my.gd> X-Mailer: Claws Mail 3.7.8cvs9 (GTK+ 2.16.6; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" , Devin Teske , "freebsd-questions@freebsd.org" Subject: Re: [RELEASE] host-setup(1): a dialog(1)-based utility for configuring FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 15:07:24 -0000 On Fri, 11 Feb 2011 08:56:42 +0100 Damien Fleuriot wrote: > The list strips non-text attachments so there isn't much to see at > the moment though... It wasn't supposed to be attached - try http://druidbsd.sourceforge.net/download/host-setup.txt :) -- Bruce Cran From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 15:45:17 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 68A41106564A for ; Fri, 11 Feb 2011 15:45:17 +0000 (UTC) (envelope-from samflanker@gmail.com) Received: from mail-ew0-f54.google.com (mail-ew0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id EDF978FC15 for ; Fri, 11 Feb 2011 15:45:16 +0000 (UTC) Received: by ewy24 with SMTP id 24so1416681ewy.13 for ; Fri, 11 Feb 2011 07:45:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:message-id:date:from:user-agent:mime-version:to :subject:content-type:content-transfer-encoding; bh=WPhNEuX469O2UQaHnwQ0JaMRucxiQMkMplo9n0tqXVk=; b=E1DVw5rUqz2bHq4qYvQlhQOsMcq34BUrrxmQiytFplRf7Ycwvu9+DYXWZKbVrqv1wA 5chjFZuANHWZmSzoi6dw+/bSNixkMI6mI85LN80iYuuT8rw7K73beKvijnmbqZTFX11M d4IqoAulz9YpipTJETTAXwFBHMrkO9Nfvj5/M= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; b=MoLdX0Y48kNQ8heryyzXmkkizaWI2DYtmnFz6OydT1CqNoCZ3VmKZTEuOcHVO0ls7P XSAUcjVezDP9Pt/6U+ynZVyCmyyQsze3nDMstdNx/VBkbBakWKOTA9sbQL9ptSp3uBMU 2tKDsAc5NC0rxDwdNJ73rcNw6pNerUNTakRPM= Received: by 10.223.83.208 with SMTP id g16mr450326fal.52.1297428470802; Fri, 11 Feb 2011 04:47:50 -0800 (PST) Received: from localhost.localdomain ([213.152.137.45]) by mx.google.com with ESMTPS id 21sm328738fav.41.2011.02.11.04.47.49 (version=SSLv3 cipher=OTHER); Fri, 11 Feb 2011 04:47:49 -0800 (PST) Message-ID: <4D553027.5040606@gmail.com> Date: Fri, 11 Feb 2011 15:48:39 +0300 From: venom User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: problem with build mcelog X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 15:45:17 -0000 Hello. i am trying build mcelog FreeBSD XXXX 8.1-RELEASE-p2 FreeBSD 8.1-RELEASE-p2 #0: Fri Jan 14 04:15:56 UTC 2011 root@freebsd:/usr/obj/usr/src/sys/GENERIC amd64 # fetch http://ftp2.pl.freebsd.org/pub/FreeBSD/distfiles/mcelog-1.0pre2.tar.gz # tar -xf mcelog-1.0pre2.tar.gz # cd mcelog-1.0pre2 # fetch http://people.freebsd.org/~jhb/mcelog/mcelog.patch # fetch http://people.freebsd.org/~jhb/mcelog/memstream.c # patch< mcelog.patch # gmake FREEBSD=yes Makefile:92: .depend: No such file or directory cc -MM -I. p4.c k8.c mcelog.c dmi.c tsc.c core2.c bitfield.c intel.c nehalem.c dunnington.c tulsa.c config.c memutil.c msg.c eventloop.c leaky-bucket.c memdb.c server.c client.c cache.c rbtree.c memstream.c> .depend.X&& mv .depend.X .depend cc -c -g -Os -Wall -Wextra -Wno-missing-field-initializers -Wno-unused-parameter -Wstrict-prototypes -Wformat-security -Wmissing-declarations -Wdeclaration-after-statement -o mcelog.o mcelog.c mcelog.c: In function 'process_kvm': mcelog.c:1342: error: 'X_SNAPDATE' undeclared (first use in this function) mcelog.c:1342: error: (Each undeclared identifier is reported only once mcelog.c:1342: error: for each function it appears in.) mcelog.c:1342: error: 'snapdate' undeclared (first use in this function) gmake: *** [mcelog.o] Error 1 please help me -- Vladimir Laskov From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 16:25:47 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BB7AE106566B for ; Fri, 11 Feb 2011 16:25:47 +0000 (UTC) (envelope-from samflanker@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4CA188FC14 for ; Fri, 11 Feb 2011 16:25:46 +0000 (UTC) Received: by eyf6 with SMTP id 6so1446632eyf.13 for ; Fri, 11 Feb 2011 08:25:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:message-id:date:from:user-agent:mime-version:to :subject:content-type:content-transfer-encoding; bh=TiqLTs+v86mF4oUdEjrPiOTJeCizQHK25nGzmvN/edk=; b=S2imLYf//+5jSHtvdZ0hpFtGguPiRuK7pvlmu3WvsqPzqOLqMMOvY3yA5722yTJ6Qu ofp9AGmE2ivSKVZYYq0beWv4+K6otT/qYE45+RpbojDw5vIK+b2VsJVgbjAOkQn0Tp9o ocMrNPtT2/cVgAjVJbZkt4IASOzsGAuTYMiqU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; b=ODiysJXpb+bNhIOegAeAdOabzwY+Kq+LpAa1hArc184j6mACpn/RYTRS+9ZbUYeGN6 tBkM/oY2TLF+rVX+6bHfoCV2QbljS26ZE+v88oUL3aLW6LVD8mCGtzpveg6ltEgYO3W+ mCTIRcbSKFV+r2Kw+VIcUVp91mgqrbHt6aUeY= Received: by 10.223.103.16 with SMTP id i16mr329341fao.40.1297421237212; Fri, 11 Feb 2011 02:47:17 -0800 (PST) Received: from localhost.localdomain ([213.152.137.45]) by mx.google.com with ESMTPS id j12sm269211fax.33.2011.02.11.02.47.15 (version=SSLv3 cipher=OTHER); Fri, 11 Feb 2011 02:47:16 -0800 (PST) Message-ID: <4D5513E6.30706@gmail.com> Date: Fri, 11 Feb 2011 13:48:06 +0300 From: venom User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: problem with build mcelog X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 16:25:47 -0000 Hello. i am trying build mcelog FreeBSD XXXX 8.1-RELEASE-p2 FreeBSD 8.1-RELEASE-p2 #0: Fri Jan 14 04:15:56 UTC 2011 root@freebsd:/usr/obj/usr/src/sys/GENERIC amd64 # fetch http://ftp2.pl.freebsd.org/pub/FreeBSD/distfiles/mcelog-1.0pre2.tar.gz # tar -xf mcelog-1.0pre2.tar.gz # cd mcelog-1.0pre2 # fetch http://people.freebsd.org/~jhb/mcelog/mcelog.patch # fetch http://people.freebsd.org/~jhb/mcelog/memstream.c # patch< mcelog.patch # gmake FREEBSD=yes Makefile:92: .depend: No such file or directory cc -MM -I. p4.c k8.c mcelog.c dmi.c tsc.c core2.c bitfield.c intel.c nehalem.c dunnington.c tulsa.c config.c memutil.c msg.c eventloop.c leaky-bucket.c memdb.c server.c client.c cache.c rbtree.c memstream.c> .depend.X&& mv .depend.X .depend cc -c -g -Os -Wall -Wextra -Wno-missing-field-initializers -Wno-unused-parameter -Wstrict-prototypes -Wformat-security -Wmissing-declarations -Wdeclaration-after-statement -o mcelog.o mcelog.c mcelog.c: In function 'process_kvm': mcelog.c:1342: error: 'X_SNAPDATE' undeclared (first use in this function) mcelog.c:1342: error: (Each undeclared identifier is reported only once mcelog.c:1342: error: for each function it appears in.) mcelog.c:1342: error: 'snapdate' undeclared (first use in this function) gmake: *** [mcelog.o] Error 1 please help me -- Vladimir Laskov From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 16:37:23 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8797E106564A for ; Fri, 11 Feb 2011 16:37:23 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id 52FB48FC16 for ; Fri, 11 Feb 2011 16:37:23 +0000 (UTC) Received: from julian-mac.elischer.org (home-nat.elischer.org [67.100.89.137]) (authenticated bits=0) by vps1.elischer.org (8.14.4/8.14.4) with ESMTP id p1BGbFce070203 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 11 Feb 2011 08:37:18 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <4D5565C7.1010809@freebsd.org> Date: Fri, 11 Feb 2011 08:37:27 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Da Rock References: <4D54E39D.1000505@herveybayaustralia.com.au> <4D54F0B0.7010503@freebsd.org> <4D550300.5090000@herveybayaustralia.com.au> In-Reply-To: <4D550300.5090000@herveybayaustralia.com.au> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, dudu@dudu.ro Subject: Re: linux PF_PACKET compatibility X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 16:37:23 -0000 On 2/11/11 1:36 AM, Da Rock wrote: > On 02/11/11 18:17, Julian Elischer wrote: >> On 2/10/11 11:22 PM, Da Rock wrote: >>> "In recent versions of the Linux kernel (post-2.0 releases) a new >>> protocol family has been introduced, named PF_PACKET. This family >>> allows an application to send and receive packets dealing directly >>> with the network card driver, thus avoiding the usual protocol >>> stack-handling (e.g., IP/TCP or IP/UDP processing). That is, any >>> packet sent through the socket will be directly passed to the >>> Ethernet interface, and any packet received through the interface >>> will be directly passed to the application." >>> >>> I've been chasing the answer to a FreeBSD version of this (approx. >>> anyway), but I needed to find out what exactly PF_PACKET was >>> first. Finally found this answer here: >>> http://www.linuxjournal.com/article/4659 >>> >>> I looked up man socket and I can see possibilities (in my mind >>> anyway), but I thought I'd be best to check if the gurus here >>> might have a better idea. My reason for this is I'm attempting to >>> build l2tpns (which supposedly builds on 7.3?! with no trouble), >>> and I'm chasing the errors which appear to be linuxisms mostly. >>> >>> So in man socket simply looking at the list of protocol families >>> I'd say network driver level would be similar to PF_LINK link >>> layer interface? Is there another man page I should be looking at >>> as well? >> >> We don't have an exact equivalent.. but we have ways of doing the >> same thing. >> one way that is suggested is to use pcap and bpf which I am pretty >> certain has been enhanced to allow sending as >> well as receiving. >> you can also hook directly to the interface using netgraph(4) >> there are other ways too but those are the two that came to mind >> immediately. > So I'm going to have to rewrite that interface entirely? Bugger! I > just can't fathom how this howto could even exist for l2tpns on > FreeBSD if it isn't even close to buildable... weird! > > http://kuapp.com/2010/07/14/how-to-setup-l2tpipsec-vpn-on-freebsd.html > > Thanks guys. I'll probably come back with more problems as I slowly > crack this one... :) nothing in that page needs to talk to the network interface directly... what do you think does? >> >> >>> >>> FWIW my gmake output is this: >>> >>> gcc -Wall -Wformat-security -Wno-format-zero-length -g -O3 -I. >>> -I/usr/include -I/usr/local/include -DLIBDIR='"/lib/l2tpns"' >>> -DETCDIR='"/etc/l2tpns"' -DSTATISTICS -DSTAT_CALLS -DRINGBUFFER >>> -DHAVE_EPOLL -DBGP -c -o arp.o arp.c >>> arp.c: In function 'sendarp': >>> arp.c:34: error: storage size of 'sll' isn't known >>> arp.c:59: error: 'PF_PACKET' undeclared (first use in this function) >>> arp.c:59: error: (Each undeclared identifier is reported only once >>> arp.c:59: error: for each function it appears in.) >>> arp.c:62: error: 'AF_PACKET' undeclared (first use in this function) >>> arp.c:34: warning: unused variable 'sll' >>> gmake: *** [arp.o] Error 1 >>> >>> I posted this over at -questions@ but it was suggested to try here >>> instead (or -net@). I figured this would be the best place to >>> start... :) >>> >>> Cheers >>> _______________________________________________ >>> freebsd-hackers@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>> To unsubscribe, send any mail to >>> "freebsd-hackers-unsubscribe@freebsd.org" >>> >> > > From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 16:38:15 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 81B66106567A for ; Fri, 11 Feb 2011 16:38:15 +0000 (UTC) (envelope-from swell.k@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id DD35B8FC25 for ; Fri, 11 Feb 2011 16:38:14 +0000 (UTC) Received: by eyf6 with SMTP id 6so1458928eyf.13 for ; Fri, 11 Feb 2011 08:38:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-type; bh=O2MwpsCZbg+Rmu9iFAknhn/y4NtTy0BtKnAPAGul1Qo=; b=rVr8IdpBpgxucz42zhb7FFpH0tSEM75JA1HGKPtenBB73zxIKoCjukRVW8oiI6ACTS yo2FB2Zc8r/PR4soOlFKqtn5/hll2m07kce/JMzHY0pOH8KZpAjuSJzU8KyMhuqeBizN PLORregiA8ZEi5VQ/wg4i3U9SuZNCgHnA5lok= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-type; b=GKiPvBaRYjtvokDmzLIS2j/LOlHzNA+hl5wiFHGqsjMmWB9Q9m0CuWuZunMsjwNE4k FWESnvegd/+q4mS4NaP6yy57Q6aXscgyiMSHe9iho7+t/kVXBe5AvVcq7zQbFA1qtSRH 639SMA6TaZplObLHnKzUw0XULuR1rPGZvxrCY= Received: by 10.14.47.193 with SMTP id t41mr655978eeb.21.1297442293460; Fri, 11 Feb 2011 08:38:13 -0800 (PST) Received: from localhost (tor-exit-readme.hands.com [83.142.228.14]) by mx.google.com with ESMTPS id x54sm852043eeh.17.2011.02.11.08.38.10 (version=SSLv3 cipher=OTHER); Fri, 11 Feb 2011 08:38:11 -0800 (PST) From: Anonymous To: Alexander Leidinger References: <20110211103028.12684f54yrw8tgqo@webmail.leidinger.net> Date: Fri, 11 Feb 2011 19:38:06 +0300 In-Reply-To: <20110211103028.12684f54yrw8tgqo@webmail.leidinger.net> (Alexander Leidinger's message of "Fri, 11 Feb 2011 10:30:28 +0100") Message-ID: <86zkq27ci9.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.50 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain Cc: hackers@freebsd.org, kibab@freebsd.org Subject: Re: CFR: FEATURE macros for AUDIT/CAM/IPC/KTR/MAC/NFS/NTP/PMC/SYSV/... X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 16:38:15 -0000 Alexander Leidinger writes: > Hi, > > during the last GSoC various FEATURE macros where added to the system. > Before committing them, I would like to get some review (like if macro > is in the correct file, and for those FEATURES where the description > was not taken from NOTES if the description is OK). > > If nobody complains, I would like to commit this in 1-2 weeks. If you > need more time to review, just tell me. [...] > Index: kern/kern_dtrace.c [...] > +FEATURE(kdtrace_hooks, > + "Kernel DTrace hooks which are required to load DTrace kernel modules"); > + [...] > Index: kern/kern_lockstat.c [...] > +FEATURE(kdtrace_hooks, "Kernel DTRACE hooks"); Why description differs? From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 16:48:36 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 16F03106566C for ; Fri, 11 Feb 2011 16:48:36 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id E2C2B8FC08 for ; Fri, 11 Feb 2011 16:48:35 +0000 (UTC) Received: from julian-mac.elischer.org (home-nat.elischer.org [67.100.89.137]) (authenticated bits=0) by vps1.elischer.org (8.14.4/8.14.4) with ESMTP id p1BGmW9i070248 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 11 Feb 2011 08:48:33 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <4D55686B.5050202@freebsd.org> Date: Fri, 11 Feb 2011 08:48:43 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Da Rock References: <4D54E39D.1000505@herveybayaustralia.com.au> <4D54F0B0.7010503@freebsd.org> <4D550300.5090000@herveybayaustralia.com.au> <4D553C4B.5000101@herveybayaustralia.com.au> In-Reply-To: <4D553C4B.5000101@herveybayaustralia.com.au> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org, Vlad Galu Subject: Re: linux PF_PACKET compatibility X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 16:48:36 -0000 On 2/11/11 5:40 AM, Da Rock wrote: > On 02/11/11 19:54, Vlad Galu wrote: >> >> >> On Fri, Feb 11, 2011 at 11:36 AM, Da Rock >> > > wrote: >> >> On 02/11/11 18:17, Julian Elischer wrote: >> >> On 2/10/11 11:22 PM, Da Rock wrote: >> >> "In recent versions of the Linux kernel (post-2.0 >> releases) a new protocol family has been introduced, >> named PF_PACKET. This family allows an application to >> send and receive packets dealing directly with the >> network card driver, thus avoiding the usual protocol >> stack-handling (e.g., IP/TCP or IP/UDP processing). >> That is, any packet sent through the socket will be >> directly passed to the Ethernet interface, and any >> packet received through the interface will be directly >> passed to the application." >> >> I've been chasing the answer to a FreeBSD version of >> this (approx. anyway), but I needed to find out what >> exactly PF_PACKET was first. Finally found this answer >> here: http://www.linuxjournal.com/article/4659 >> >> I looked up man socket and I can see possibilities (in >> my mind anyway), but I thought I'd be best to check if >> the gurus here might have a better idea. My reason for >> this is I'm attempting to build l2tpns (which >> supposedly builds on 7.3?! with no trouble), and I'm >> chasing the errors which appear to be linuxisms mostly. >> >> So in man socket simply looking at the list of protocol >> families I'd say network driver level would be similar >> to PF_LINK link layer interface? Is there another man >> page I should be looking at as well? >> >> >> We don't have an exact equivalent.. but we have ways of >> doing the same thing. >> one way that is suggested is to use pcap and bpf which I am >> pretty certain has been enhanced to allow sending as >> well as receiving. >> you can also hook directly to the interface using netgraph(4) >> there are other ways too but those are the two that came to >> mind immediately. >> >> So I'm going to have to rewrite that interface entirely? >> Bugger! I just can't fathom how this howto could even exist for >> l2tpns on FreeBSD if it isn't even close to buildable... weird! >> >> http://kuapp.com/2010/07/14/how-to-setup-l2tpipsec-vpn-on-freebsd.html >> >> Thanks guys. I'll probably come back with more problems as I >> slowly crack this one... :) >> >> >> I suppose you could just use mpd :) > I could, I guess. But where's the fun in that? :) > > Seriously, though, mpd didn't quite cut it (I thought) for me. I > need a l2tp vpn server with the capability to handle multiple > clients with only one interface. The server is behind a firewall, > and I'm trying for a "walled garden" variety I guess. So far my > research has brought me here, but I'm open to suggestions. why do you think you need only one interface? > > One other that has my attention is l2tpd (in ports). I want radius > auth, so IF I can use pppd in base and radius (which as I > understand- so far anyway- it needs), and only uses a single > interface, then maybe. pppd in base will I think give you multiple interfaces.. > > I'm still hunting and playing- learning on the fly. From what I read > mpd uses an ng interface for every single client. L2tpns doesn't, > and from what I've read so far neither does l2tpd (I was actually > looking at another fork of that xl2tpd). I could use some advice > from someone with experience with this, but my feelers on -questions > didn't get much response. I may try on -net if this fails... again, whats' with the single interface? > > Aside from that I also wanted to get a bit more of a hands on feel > for the FreeBSD core. I can't sit on the sidelines yelling at the > players any more :) I'm not much for spectator sport either... From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 17:01:57 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 05A841065670; Fri, 11 Feb 2011 17:01:57 +0000 (UTC) (envelope-from dteske@vicor.com) Received: from postoffice.vicor.com (postoffice.vicor.com [69.26.56.53]) by mx1.freebsd.org (Postfix) with ESMTP id DDEF58FC0A; Fri, 11 Feb 2011 17:01:56 +0000 (UTC) Received: from [192.82.228.105] (port=60679) by postoffice.vicor.com with esmtpsa (SSLv3:AES256-SHA:256) (Exim 4.71) (envelope-from ) id 1PnwNW-00014I-6O; Fri, 11 Feb 2011 09:01:56 -0800 From: Devin Teske To: Damien Fleuriot In-Reply-To: <5AF1B11E-19FF-4FF5-9058-BE8F0AB85E1B@my.gd> References: <1297396433.7232.9.camel@dt.vicor.com> <5AF1B11E-19FF-4FF5-9058-BE8F0AB85E1B@my.gd> Organization: VICOR, Inc. Date: Fri, 11 Feb 2011 09:02:05 -0800 Message-ID: <1297443725.9144.1.camel@dt.vicor.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 X-Scan-Signature: b57c4500898f7f78298605e8a43c9cb5 X-Scan-Host: postoffice.vicor.com Content-Type: text/plain; charset="cp1252" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "freebsd-hackers@freebsd.org" , "freebsd-questions@freebsd.org" Subject: Re: [RELEASE] host-setup(1): a dialog(1)-based utility for configuring FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 17:01:57 -0000 On Fri, 2011-02-11 at 08:56 +0100, Damien Fleuriot wrote: > Hi Devin, > > > Thanks for sharing your work. > > The list strips non-text attachments so there isn't much to see at the moment though... Thanks Damien. Here's a link to the pic I posted online in-tandem with the post to the list(s). http://www.twitpic.com/3yhye7 -- Devin > > > --- > Fleuriot Damien > > On 11 Feb 2011, at 04:53, Devin Teske wrote: > > > Hi All, > > > > I'd like to announce the release of a new script. A script that I've > > developed for our field engineers that I'd like to share with the rest > > of the world. > > > > http://druidbsd.sourceforge.net/download/host-setup.txt > > > > host-setup(1) is a dialog(1)-based utility (written in sh(1)) designed > > to make configuring FreeBSD more efficient. > > > > We have this script configured to be run as root's initial login > > immediately after "first-boot". The field engineer uses our custom > > installer to install RELENG_8, and after the machine presents a login > > prompt, they login with the pre-configured root password. After which > > they are presented with this dialog (image attached: > > host-setup.pub.png). > > > > The dialogs should all be intuitive, and I hope that you like my work. > > I've worked very hard to make this utility smooth, robust, > > fault-tolerant and even enjoyable to use. Not only that, but it helps > > prevents mistakes that could arise in doing these steps by-hand (like > > forgetting to unmount active NFS-mounts before executing "route flush"). > > > > Please give it a try and let me know what you think. > > -- > > Devin > > > > P.S. This is not a trivial script. ``More nuclear reactor than bike > > shed'' ^_^ > > > > P.P.S. Should be backward compatible to RELENG_4. > > > > P.P.S. I know the screenshot says "host-setup.pub" -- that's only > > because our in-house-only version has more functionality than the one > > I'm releasing to the general public today (no functionality that anyone > > in the public audience would ever care about though). > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 17:03:49 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EEA71106566C; Fri, 11 Feb 2011 17:03:49 +0000 (UTC) (envelope-from webmaster@kibab.com) Received: from mx0.deglitch.com (cl-414.sto-01.se.sixxs.net [IPv6:2001:16d8:ff00:19d::2]) by mx1.freebsd.org (Postfix) with ESMTP id 9673C8FC16; Fri, 11 Feb 2011 17:03:49 +0000 (UTC) Received: from kibab-nb.kibab.com (p5099b4c2.dip0.t-ipconnect.de [80.153.180.194]) by mx0.deglitch.com (Postfix) with ESMTPSA id 22FE28FC2B; Fri, 11 Feb 2011 20:03:44 +0300 (MSK) Message-ID: <4D556C1C.6030002@kibab.com> Date: Fri, 11 Feb 2011 18:04:28 +0100 From: Ilya Bakulin User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.9) Gecko/20101007 Thunderbird/3.1.4 MIME-Version: 1.0 To: Alexander Leidinger References: <20110211103028.12684f54yrw8tgqo@webmail.leidinger.net> <201102110751.52863.jhb@freebsd.org> <20110211145445.12722r7902rz8vc4@webmail.leidinger.net> In-Reply-To: <20110211145445.12722r7902rz8vc4@webmail.leidinger.net> X-Enigmail-Version: 1.1.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig9EFCC5F89145175FE988CEA6" Cc: freebsd-hackers@freebsd.org, kibab@freebsd.org Subject: Re: CFR: FEATURE macros for AUDIT/CAM/IPC/KTR/MAC/NFS/NTP/PMC/SYSV/... X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 17:03:50 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig9EFCC5F89145175FE988CEA6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 11.02.2011 14:54, Alexander Leidinger wrote: > As a 3rd point (attention bikeshed ahead), having everything as a > FEATURE would give an uniform way of listing what is available or not > (with the benefit to administratively hide parts of it). Needless to > say that this would reduce the amount of knowledge and code to > determine if something is available (as a person who works in a > production environement with mixed knowledge levels of the people and > who has to analyse problems which are sometimes caused by lack of > knowledge of the people which implemented something, I welcome > everything which lowers the learning curve and complexity). When I was beginning this GSoC work, I primarily thought about unifying the way to determine if particular feature exists in the kernel. Of course there should be at least one way to check if the feature is available or not (by definition: if I may use some functionality, than feature is present, otherwise... Oh, no, may be I have no permissions to use it? or something is terribly wrong with system confuguration? Or?...), but it is better to have a sort of unified way to get this information without looking for files in /dev, parsing `kldstat -v`, etc.= --=20 Regards, Ilya Bakulin http://kibab.com xmpp://kibab612@jabber.ru --------------enig9EFCC5F89145175FE988CEA6 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.15 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk1VbCQACgkQo9vlj1oadwjvegCfbNAoG7/yteKo/0TLXbuKFgIV tFcAoNzLbD+bzlUprV0OMFTk4maSlfDL =YEQ4 -----END PGP SIGNATURE----- --------------enig9EFCC5F89145175FE988CEA6-- From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 17:06:09 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E5E161065697; Fri, 11 Feb 2011 17:06:09 +0000 (UTC) (envelope-from webmaster@kibab.com) Received: from mx0.deglitch.com (cl-414.sto-01.se.sixxs.net [IPv6:2001:16d8:ff00:19d::2]) by mx1.freebsd.org (Postfix) with ESMTP id 9020E8FC22; Fri, 11 Feb 2011 17:06:09 +0000 (UTC) Received: from kibab-nb.kibab.com (p5099b4c2.dip0.t-ipconnect.de [80.153.180.194]) by mx0.deglitch.com (Postfix) with ESMTPSA id D347A8FC2E; Fri, 11 Feb 2011 20:06:01 +0300 (MSK) Message-ID: <4D556CAB.5020703@kibab.com> Date: Fri, 11 Feb 2011 18:06:51 +0100 From: Ilya Bakulin User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.9) Gecko/20101007 Thunderbird/3.1.4 MIME-Version: 1.0 To: Anonymous References: <20110211103028.12684f54yrw8tgqo@webmail.leidinger.net> <86zkq27ci9.fsf@gmail.com> In-Reply-To: <86zkq27ci9.fsf@gmail.com> X-Enigmail-Version: 1.1.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigF15FBABD9442F0ACC540C8F9" Cc: Alexander Leidinger , hackers@freebsd.org, kibab@freebsd.org Subject: Re: CFR: FEATURE macros for AUDIT/CAM/IPC/KTR/MAC/NFS/NTP/PMC/SYSV/... X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 17:06:10 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigF15FBABD9442F0ACC540C8F9 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 11.02.2011 17:38, Anonymous wrote: > Alexander Leidinger writes: > >> Hi, >> >> during the last GSoC various FEATURE macros where added to the system.= >> Before committing them, I would like to get some review (like if macro= >> is in the correct file, and for those FEATURES where the description >> was not taken from NOTES if the description is OK). >> >> If nobody complains, I would like to commit this in 1-2 weeks. If you >> need more time to review, just tell me. > [...] >> Index: kern/kern_dtrace.c > [...] >> +FEATURE(kdtrace_hooks, >> + "Kernel DTrace hooks which are required to load DTrace kernel mod= ules"); >> + > [...] >> Index: kern/kern_lockstat.c > [...] >> +FEATURE(kdtrace_hooks, "Kernel DTRACE hooks"); > Why description differs? > > !DSPAM:4d556bc9311932017213797! > > It was a mistake on my side, I've already removed duplicate definition in Perforce. Thank you for pointing this out though! --=20 Regards, Ilya Bakulin http://kibab.com xmpp://kibab612@jabber.ru --------------enigF15FBABD9442F0ACC540C8F9 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.15 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk1VbK0ACgkQo9vlj1oadwj/UwCg5PUpPqj/Kob8020co+XiZAvt 2JYAoIe737c1U0vLzcxoOA9MD80jO901 =hoQD -----END PGP SIGNATURE----- --------------enigF15FBABD9442F0ACC540C8F9-- From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 18:07:59 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 228C8106566B for ; Fri, 11 Feb 2011 18:07:59 +0000 (UTC) (envelope-from dteske@vicor.com) Received: from postoffice.vicor.com (postoffice.vicor.com [69.26.56.53]) by mx1.freebsd.org (Postfix) with ESMTP id 0937A8FC08 for ; Fri, 11 Feb 2011 18:07:58 +0000 (UTC) Received: from [192.82.228.105] (port=61137) by postoffice.vicor.com with esmtpsa (SSLv3:AES256-SHA:256) (Exim 4.71) (envelope-from ) id 1PnxPQ-0005Gg-Gk; Fri, 11 Feb 2011 10:07:58 -0800 From: Devin Teske To: Ian Smith In-Reply-To: <20110212015155.I96449@sola.nimnet.asn.au> References: <20110211120031.9D37510656E5@hub.freebsd.org> <20110212015155.I96449@sola.nimnet.asn.au> Organization: VICOR, Inc. Date: Fri, 11 Feb 2011 10:08:07 -0800 Message-ID: <1297447687.9144.6.camel@dt.vicor.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 X-Scan-Signature: 417bba55d4dc493adb4d94730825773c X-Scan-Host: postoffice.vicor.com Content-Type: text/plain; charset="cp1252" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: Re: [RELEASE] host-setup(1): a dialog(1)-based utility for configuring FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 18:07:59 -0000 On Sat, 2011-02-12 at 02:30 +1100, Ian Smith wrote: > In freebsd-questions Digest, Vol 349, Issue 8, Message: 15 > On Thu, 10 Feb 2011 19:53:53 -0800 Devin Teske wrote: > > Hi All, > > > > I'd like to announce the release of a new script. A script that I've > > developed for our field engineers that I'd like to share with the rest > > of the world. > > > > http://druidbsd.sourceforge.net/download/host-setup.txt > > > > host-setup(1) is a dialog(1)-based utility (written in sh(1)) designed > > to make configuring FreeBSD more efficient. > > Nice, if only as great bedtime reading so far; I've already learned some > new techniques. I'm particularly proud of this little diddy (see lines 1600-1607 of host-setup; rewritten to be a functional example program that takes a number like "26" and produces "255.255.255.192"): ============================================================ #!/bin/sh blen2netmask() { local nbits="$1" netmask="" n=0 while [ $n -lt 4 ]; do netmask="$netmask${netmask:+.}$(( (65280 >> ($nbits - 8 * $n) & 255) * ((8*$n) < $nbits & $nbits <= (8*($n+1))) + 255 * ($nbits > (8*($n+1))) ))" n=$(( $n + 1 )) done echo "$netmask" } blen2netmask "$@" ============================================================ I wrote _several_ other implementations and above is the one that I finally settled on as being the most elegant. I was after a solution that (a) didn't use anything but built-in internals of sh(1), and (b) got the job done as fast as computationally possible. $ time blen2netmask 26 255.255.255.192 real 0m0.004s user 0m0.001s sys 0m0.004s That's pretty fast, I'd say ^_^ (faster than the other implementations -- especially considering that it doesn't have to fork anything). -- Devin P.S. Maybe I ought to expand it to IPv6 considering that the IPv4 address space has [reportedly] finally ran out (is that true?). > I expect to steal lots of it wholesale (acknowledged :) > > cheers, Ian > _______________________________________________ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 19:09:19 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 276CB106566B for ; Fri, 11 Feb 2011 19:09:19 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id 8D4478FC0C for ; Fri, 11 Feb 2011 19:09:17 +0000 (UTC) Received: from park.js.berklix.net (p5B22F3C7.dip.t-dialin.net [91.34.243.199]) (authenticated bits=0) by tower.berklix.org (8.14.2/8.14.2) with ESMTP id p1BJ9EL8019440 for ; Fri, 11 Feb 2011 19:09:16 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by park.js.berklix.net (8.13.8/8.13.8) with ESMTP id p1BJ9ZCH011451 for ; Fri, 11 Feb 2011 20:09:35 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.3/8.14.3) with ESMTP id p1BJ9UAE097045 for ; Fri, 11 Feb 2011 20:09:35 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <201102111909.p1BJ9UAE097045@fire.js.berklix.net> To: hackers@freebsd.org From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Linux Unix Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com/~jhs/cv/ Date: Fri, 11 Feb 2011 20:09:30 +0100 Sender: jhs@berklix.com Cc: Subject: memstick.img is bloated with 7% 2K blocks of nulls X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 19:09:19 -0000 memstick.img wastes 7% with 2K blocks of nulls. shown by: 8f -b 0 -n 2048 -l -f Fr* http://berklix.com/~jhs/src/bsd/jhs/bin/public/8f/ 8f.c 8f.1 FreeBSD-8.1-RELEASE-amd64-bootonly.iso: 543 FreeBSD-8.1-RELEASE-amd64-disc1.iso: 543 FreeBSD-8.1-RELEASE-amd64-dvd1.iso: 1125 FreeBSD-8.1-RELEASE-amd64-livefs.iso: 1109 FreeBSD-8.1-RELEASE-amd64-memstick.img: 38722 FreeBSD-8.1-RELEASE-i386-bootonly.iso: 511 FreeBSD-8.1-RELEASE-i386-disc1.iso: 511 FreeBSD-8.1-RELEASE-i386-dvd1.iso: 865 FreeBSD-8.1-RELEASE-i386-dvd1.iso: 893 FreeBSD-8.1-RELEASE-i386-livefs.iso: 850 FreeBSD-8.1-RELEASE-i386-memstick.img: 34066 FreeBSD-8.2-RC3-amd64-bootonly.iso: 594 FreeBSD-8.2-RC3-amd64-disc1.iso: 594 FreeBSD-8.2-RC3-amd64-dvd1.iso: 1167 FreeBSD-8.2-RC3-amd64-livefs.iso: 1166 FreeBSD-8.2-RC3-amd64-memstick.img: 39216 FreeBSD-8.2-RC3-i386-bootonly.iso: 565 FreeBSD-8.2-RC3-i386-disc1.iso: 565 FreeBSD-8.2-RC3-i386-dvd1.iso: 906 FreeBSD-8.2-RC3-i386-livefs.iso: 905 FreeBSD-8.2-RC3-i386-memstick.img: 34521 It's not just one big block of trailing nulls. od -c FreeBSD-8.2-RC3-i386-memstick.img > tmp.od ; tail tmp.od 7114017760 @ 244 377 344 Q \b * 325 \t 333 y ` & ' \n W 7114020000 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 * 7114050000 050000 - 020000 = 030000 = 0x3000 = 12288 It's a lot of bursts of null blocks, seen by either: grep -n '^*' tmp.od 8f -f -c -b 0 -n 2048 FreeBSD-8.2-RC3-i386-memstick.img The CD & DVD images are not nearly so wasteful, see above. As near 1G ( 959467520 FreeBSD-8.2-RC3-i386-memstick.img ) it will soon not fit on 1G sticks. If its not easy for someone to find & trim, xz compression would at least save 27% of transmission bandwidth, ( dc 702447528 100 * 959467520 / p 73 ) a much higher % than DVD that also uses compression. PS I sent this to hackers@ rather than re@ as I imagine re@ are busy with more urgent matters at present :-) Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Mail plain text; Not quoted-printable, Not HTML, Not base 64. Reply below text sections not at top, to avoid breaking cumulative context. From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 19:22:50 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9CA92106566B for ; Fri, 11 Feb 2011 19:22:50 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (unknown [IPv6:2a01:348:0:15:5d59:5c40:0:1]) by mx1.freebsd.org (Postfix) with ESMTP id C5F508FC18 for ; Fri, 11 Feb 2011 19:22:49 +0000 (UTC) Received: from muon.cran.org.uk (localhost [127.0.0.1]) by muon.cran.org.uk (Postfix) with ESMTP id B4143E80B2; Fri, 11 Feb 2011 19:22:43 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=cran.org.uk; h=date:from :to:cc:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; s=mail; bh=O7ABAD7AxyDA aGDlbtLKSqXHbQM=; b=F5/79TIXEIYkWdAoCy+nhfPZYKLWJWd1h3roJYRIbgaN U53d3gxKuOVXhm36G63zhVCAHsodEV+5xi0i36JHReDIDE9DQW84KULjcH9wLteZ lV/N6ikTUqYCKdT+9SKEkJMK+bJCZitT/BP2EzxmhLMLWC0mvpDb0D1VjR/h+NI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=cran.org.uk; h=date:from:to :cc:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; q=dns; s=mail; b=vYZx3Z o5VsUOT18GmLas6GIlB/cY0J7+JS/TJC7bQ2XiVvtyjsDwUcnP5tKgFppbrljtzF H0a7myV1MFRilqvi+v1JsUm8aBq2uS7Oq5Kb92HgQXu1bwdcsGe6eziRNZEnQLfm xhiq91l5LdmCXRVlcJ/NIExDYV5XEvXSW97do= Received: from unknown (client-86-23-108-125.brhm.adsl.virginmedia.com [86.23.108.125]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA id 5E65DE7844; Fri, 11 Feb 2011 19:22:43 +0000 (GMT) Date: Fri, 11 Feb 2011 19:22:27 +0000 From: Bruce Cran To: "Julian H. Stacey" Message-ID: <20110211192227.00002188@unknown> In-Reply-To: <201102111909.p1BJ9UAE097045@fire.js.berklix.net> References: <201102111909.p1BJ9UAE097045@fire.js.berklix.net> X-Mailer: Claws Mail 3.7.8cvs9 (GTK+ 2.16.6; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: hackers@freebsd.org Subject: Re: memstick.img is bloated with 7% 2K blocks of nulls X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 19:22:50 -0000 On Fri, 11 Feb 2011 20:09:30 +0100 "Julian H. Stacey" wrote: > memstick.img wastes 7% with 2K blocks of nulls. Could this be due to using UFS1 instead of UFS2? On a related note, at some point the release scripts should be updated to use gpart instead of fdisk/bsdlabel. -- Bruce Cran From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 20:32:13 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CDE8C106566C for ; Fri, 11 Feb 2011 20:32:12 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id A2AE38FC1D for ; Fri, 11 Feb 2011 20:32:12 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 4630446B2E; Fri, 11 Feb 2011 15:32:12 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.10]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 6A1B68A009; Fri, 11 Feb 2011 15:32:11 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Fri, 11 Feb 2011 15:31:00 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.4-CBSD-20110107; KDE/4.4.5; amd64; ; ) References: <4D553027.5040606@gmail.com> In-Reply-To: <4D553027.5040606@gmail.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201102111531.00915.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Fri, 11 Feb 2011 15:32:11 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=0.5 required=4.2 tests=BAYES_00,MAY_BE_FORGED, RDNS_DYNAMIC autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Cc: venom Subject: Re: problem with build mcelog X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 20:32:13 -0000 On Friday, February 11, 2011 7:48:39 am venom wrote: > Hello. > > i am trying build mcelog > > > FreeBSD XXXX 8.1-RELEASE-p2 FreeBSD 8.1-RELEASE-p2 #0: Fri Jan 14 04:15:56 > UTC 2011 root@freebsd:/usr/obj/usr/src/sys/GENERIC amd64 > > > # fetch > http://ftp2.pl.freebsd.org/pub/FreeBSD/distfiles/mcelog-1.0pre2.tar.gz > # tar -xf mcelog-1.0pre2.tar.gz > # cd mcelog-1.0pre2 > # fetch http://people.freebsd.org/~jhb/mcelog/mcelog.patch > # fetch http://people.freebsd.org/~jhb/mcelog/memstream.c Oops, I just updated mcelog.patch and it should work fine now. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 21:23:04 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6ED031065672 for ; Fri, 11 Feb 2011 21:23:04 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3597B8FC0C for ; Fri, 11 Feb 2011 21:23:03 +0000 (UTC) Received: by iyb26 with SMTP id 26so2857963iyb.13 for ; Fri, 11 Feb 2011 13:23:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=domainkey-signature:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=u1ECQ2l87ijN0eKt6Aoy1FS9FnV9fFs6nyxmp+wOc74=; b=giyKqxengjg3EddhuOFSIAP+H5scOOcfn/lVQJx0OvXctBTY16nDWnbd/ezpjQy56z oWF2otKDs1al+RS45LPrW3Mo/XgKz4ZAgageo5E7ZApxWH9VDmNWJISkmhX/a5IjmKg2 vRnsP6E8yw25WjJ+eFitBQxYedFUfkgZ7rq00= DomainKey-Signature: a=rsa-sha1; c=nofws; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; b=udLGWgzto0G5ohmofqFZLdFzHFnqQ8H8btYhDytyhpSNJJlFGXpvwY+Dg5UAmHUXEv 4qtmhx63g5IF8VApet7pGP2cFoGFgaTwRL7InghlC+KKlMMNH1l7af3YhP8EQ3gl1R7p lFC+ZX4xEq11fu4qAeemWCPo7AcHfFRn8uDj0= Received: by 10.42.230.9 with SMTP id jk9mr1195472icb.519.1297457981405; Fri, 11 Feb 2011 12:59:41 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.183.134 with HTTP; Fri, 11 Feb 2011 12:59:21 -0800 (PST) In-Reply-To: <1297447687.9144.6.camel@dt.vicor.com> References: <20110211120031.9D37510656E5@hub.freebsd.org> <20110212015155.I96449@sola.nimnet.asn.au> <1297447687.9144.6.camel@dt.vicor.com> From: Eitan Adler Date: Fri, 11 Feb 2011 15:59:21 -0500 Message-ID: To: Devin Teske Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org, Ian Smith Subject: Re: [RELEASE] host-setup(1): a dialog(1)-based utility for configuring FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 21:23:04 -0000 Nice Script! I intend to steal parts of it for my own use. > > P.S. Maybe I ought to expand it to IPv6 considering that the IPv4 > address space has [reportedly] finally ran out (is that true?). > All the available IPs were allocated to the RIRs. AFIK the RIRs have not had to deny anyone for insufficiency yet - but it will happen soon. -- Eitan Adler From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 21:32:36 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 10AD3106564A for ; Fri, 11 Feb 2011 21:32:36 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id 96B768FC18 for ; Fri, 11 Feb 2011 21:32:35 +0000 (UTC) Received: from park.js.berklix.net (p5B22F3C7.dip.t-dialin.net [91.34.243.199]) (authenticated bits=0) by tower.berklix.org (8.14.2/8.14.2) with ESMTP id p1BLWXlW020783 for ; Fri, 11 Feb 2011 21:32:34 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by park.js.berklix.net (8.13.8/8.13.8) with ESMTP id p1BLWpSE012201 for ; Fri, 11 Feb 2011 22:32:51 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.3/8.14.3) with ESMTP id p1BLWkiP003000 for ; Fri, 11 Feb 2011 22:32:51 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <201102112132.p1BLWkiP003000@fire.js.berklix.net> To: hackers@freebsd.org From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Linux Unix Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com/~jhs/cv/ Date: Fri, 11 Feb 2011 22:32:46 +0100 Sender: jhs@berklix.com Cc: Subject: reverse of getchar() read() open() fopen() ? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 21:32:36 -0000 Hi hackers@, Do we have C libraries with reverse of getchar() [ & maybe read() ] & fopen() [ & maybe open() ] etc, to read from end of file toward beginning ? I dont see anything in the See Also sections. I'm not looking to write, just read. I'm looking for something that returns last char in file as first etc, I'm not interested in wchars etc, I could write some C functions, with seek etc & probably will, if none exist, but no point if they already exist ? Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Mail plain text; Not quoted-printable, Not HTML, Not base 64. Reply below text sections not at top, to avoid breaking cumulative context. From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 21:48:12 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E71F106567A for ; Fri, 11 Feb 2011 21:48:12 +0000 (UTC) (envelope-from aduane@juniper.net) Received: from exprod7og113.obsmtp.com (exprod7og113.obsmtp.com [64.18.2.179]) by mx1.freebsd.org (Postfix) with ESMTP id CEDEA8FC1A for ; Fri, 11 Feb 2011 21:48:11 +0000 (UTC) Received: from source ([66.129.224.36]) (using TLSv1) by exprod7ob113.postini.com ([64.18.6.12]) with SMTP ID DSNKTVWumypeN/6J/t+c1YIfL85ijyKAm9Z2@postini.com; Fri, 11 Feb 2011 13:48:11 PST Received: from p-emfe01-wf.jnpr.net (172.28.145.24) by P-EMHUB01-HQ.jnpr.net (172.24.192.35) with Microsoft SMTP Server (TLS) id 8.2.254.0; Fri, 11 Feb 2011 13:35:41 -0800 Received: from EMBX01-WF.jnpr.net ([fe80::1914:3299:33d9:e43b]) by p-emfe01-wf.jnpr.net ([fe80::d0d1:653d:5b91:a123%11]) with mapi; Fri, 11 Feb 2011 16:35:53 -0500 From: Andrew Duane To: "Julian H. Stacey" , "hackers@freebsd.org" Date: Fri, 11 Feb 2011 16:35:06 -0500 Thread-Topic: reverse of getchar() read() open() fopen() ? Thread-Index: AcvKMzwOVURekqG/S1u3F+r6xZNY8wAAFClQ Message-ID: References: <201102112132.p1BLWkiP003000@fire.js.berklix.net> In-Reply-To: <201102112132.p1BLWkiP003000@fire.js.berklix.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Subject: RE: reverse of getchar() read() open() fopen() ? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 21:48:12 -0000 I've never seen any such thing, but I've done similar things a lot. I'd say= malloc/read the whole file in and use a decrementing pointer to return the= "previous" character. -- Andrew Duane Juniper Networks 978-589-0551 10 Technology Park Dr aduane@juniper.net Westford, MA 01886-3418 ________________________________________ From: owner-freebsd-hackers@freebsd.org [owner-freebsd-hackers@freebsd.org]= On Behalf Of Julian H. Stacey [jhs@berklix.com] Sent: Friday, February 11, 2011 4:32 PM To: hackers@freebsd.org Subject: reverse of getchar() read() open() fopen() ? Hi hackers@, Do we have C libraries with reverse of getchar() [ & maybe read() ] & fopen() [ & maybe open() ] etc, to read from end of file toward beginning ? I dont see anything in the See Also sections. I'm not looking to write, just read. I'm looking for something that returns last char in file as first etc, I'm not interested in wchars etc, I could write some C functions, with seek etc & probably will, if none exist, but no point if they already exist ? Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.c= om Mail plain text; Not quoted-printable, Not HTML, Not base 64. Reply below text sections not at top, to avoid breaking cumulative context= . _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 22:04:37 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CB76E106564A for ; Fri, 11 Feb 2011 22:04:37 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: from email2.allantgroup.com (email2.emsphone.com [199.67.51.116]) by mx1.freebsd.org (Postfix) with ESMTP id 7793D8FC17 for ; Fri, 11 Feb 2011 22:04:37 +0000 (UTC) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by email2.allantgroup.com (8.14.4/8.14.4) with ESMTP id p1BLjse2081684 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 11 Feb 2011 15:45:55 -0600 (CST) (envelope-from dan@dan.emsphone.com) Received: from dan.emsphone.com (smmsp@localhost [127.0.0.1]) by dan.emsphone.com (8.14.4/8.14.4) with ESMTP id p1BLjsfx037944 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 11 Feb 2011 15:45:54 -0600 (CST) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.14.4/8.14.4/Submit) id p1BLjrXJ037943; Fri, 11 Feb 2011 15:45:54 -0600 (CST) (envelope-from dan) Date: Fri, 11 Feb 2011 15:45:53 -0600 From: Dan Nelson To: "Julian H. Stacey" Message-ID: <20110211214553.GH66849@dan.emsphone.com> References: <201102112132.p1BLWkiP003000@fire.js.berklix.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201102112132.p1BLWkiP003000@fire.js.berklix.net> X-OS: FreeBSD 8.2-PRERELEASE User-Agent: Mutt/1.5.21 (2010-09-15) X-Virus-Scanned: clamav-milter 0.96.4 at email2.allantgroup.com X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.6 (email2.allantgroup.com [199.67.51.78]); Fri, 11 Feb 2011 15:45:55 -0600 (CST) X-Scanned-By: MIMEDefang 2.68 on 199.67.51.78 Cc: hackers@freebsd.org Subject: Re: reverse of getchar() read() open() fopen() ? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 22:04:37 -0000 In the last episode (Feb 11), Julian H. Stacey said: > Hi hackers@, > Do we have C libraries with reverse of getchar() [ & maybe read() ] & > fopen() [ & maybe open() ] etc, to read from end of file toward beginning > ? I dont see anything in the See Also sections. I'm not looking to > write, just read. I'm looking for something that returns last char in > file as first etc, I'm not interested in wchars etc, I could write some C > functions, with seek etc & probably will, if none exist, but no point if > they already exist ? tail -r does this on a line-by-line basis. Tail does it for regular files by mmaping and then reading backwards, but you could also read the block of data at (filesize MOD buffersize) and return its bytes in reverse order, then seek back buffersize bytes, read that block and print it reversed, etc. You might even be able to write functions that could be passed to funopen(). Then you'd have a regular FILE* that you could call with regular stdio functions. Getting the buffering right for good performance might get tricky, though. -- Dan Nelson dnelson@allantgroup.com From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 22:04:39 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5E9CC1065672 for ; Fri, 11 Feb 2011 22:04:39 +0000 (UTC) (envelope-from dteske@vicor.com) Received: from postoffice.vicor.com (postoffice.vicor.com [69.26.56.53]) by mx1.freebsd.org (Postfix) with ESMTP id CA46F8FC19 for ; Fri, 11 Feb 2011 22:04:38 +0000 (UTC) Received: from [192.82.228.105] (port=63599) by postoffice.vicor.com with esmtpsa (SSLv3:AES256-SHA:256) (Exim 4.71) (envelope-from ) id 1Po0id-0000es-2o; Fri, 11 Feb 2011 13:40:00 -0800 From: Devin Teske To: "Julian H. Stacey" In-Reply-To: <201102112132.p1BLWkiP003000@fire.js.berklix.net> References: <201102112132.p1BLWkiP003000@fire.js.berklix.net> Organization: VICOR, Inc. Date: Fri, 11 Feb 2011 13:40:10 -0800 Message-ID: <1297460410.9144.10.camel@dt.vicor.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 X-Scan-Signature: 83be3b23014bddc0a2b81e211cffcda6 X-Scan-Host: postoffice.vicor.com Content-Type: text/plain; charset="cp1252" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: hackers@freebsd.org Subject: Re: reverse of getchar() read() open() fopen() ? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 22:04:39 -0000 On Fri, 2011-02-11 at 22:32 +0100, Julian H. Stacey wrote: > Hi hackers@, > Do we have C libraries with reverse of getchar() [ & maybe read() > ] & fopen() [ & maybe open() ] etc, to read from end of file toward > beginning ? `tail -r' will spit out lines of a file in reverse-order. Maybe the source to tail(1) can offer some insights: http://www.freebsd.org/cgi/cvsweb.cgi/src/usr.bin/tail/ -- Devin P.S. Sorry for the non-answer. > I dont see anything in the See Also sections. I'm not > looking to write, just read. I'm looking for something that returns > last char in file as first etc, I'm not interested in wchars etc, > I could write some C functions, with seek etc & probably will, if > none exist, but no point if they already exist ? > > Cheers, > Julian From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 22:05:43 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 06A18106566B for ; Fri, 11 Feb 2011 22:05:43 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id 777548FC12 for ; Fri, 11 Feb 2011 22:05:42 +0000 (UTC) Received: from park.js.berklix.net (p5B22F7C3.dip.t-dialin.net [91.34.247.195]) (authenticated bits=0) by tower.berklix.org (8.14.2/8.14.2) with ESMTP id p1BM5ePi021049 for ; Fri, 11 Feb 2011 22:05:41 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by park.js.berklix.net (8.13.8/8.13.8) with ESMTP id p1BM61lI012436 for ; Fri, 11 Feb 2011 23:06:01 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.3/8.14.3) with ESMTP id p1BM5ti6071257 for ; Fri, 11 Feb 2011 23:06:00 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <201102112206.p1BM5ti6071257@fire.js.berklix.net> to: hackers@freebsd.org From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Unix Linux Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Fri, 11 Feb 2011 22:32:46 +0100." <201102112132.p1BLWkiP003000@fire.js.berklix.net> Date: Fri, 11 Feb 2011 23:05:55 +0100 Sender: jhs@berklix.com Cc: Subject: Re: reverse of getchar() read() open() fopen() ? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 22:05:43 -0000 > Hi hackers@, > Do we have C libraries with reverse of getchar() [ & maybe read() > ] & fopen() [ & maybe open() ] etc, to read from end of file toward > beginning ? I dont see anything in the See Also sections. I'm not > looking to write, just read. I'm looking for something that returns > last char in file as first etc, I'm not interested in wchars etc, > I could write some C functions, with seek etc & probably will, if > none exist, but no point if they already exist ? PS I'm not looking for a program that reverses chars in lines ( eg like my http://berklix.com/~jhs/src/bsd/jhs/bin/local/rev/ ), instead I'm wondering if we have functions that read binary backwards, \0 & \n not special. Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Mail plain text; Not quoted-printable, Not HTML, Not base 64. Reply below text sections not at top, to avoid breaking cumulative context. From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 22:14:40 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE5CA106566B for ; Fri, 11 Feb 2011 22:14:40 +0000 (UTC) (envelope-from reichert@numachi.com) Received: from meisai.numachi.com (meisai.numachi.com [198.175.254.6]) by mx1.freebsd.org (Postfix) with SMTP id 27BAC8FC13 for ; Fri, 11 Feb 2011 22:14:39 +0000 (UTC) Received: (qmail 7484 invoked by uid 1001); 11 Feb 2011 21:47:58 -0000 Date: Fri, 11 Feb 2011 16:47:58 -0500 From: Brian Reichert To: "Julian H. Stacey" Message-ID: <20110211214758.GI36025@numachi.com> References: <201102112132.p1BLWkiP003000@fire.js.berklix.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201102112132.p1BLWkiP003000@fire.js.berklix.net> User-Agent: Mutt/1.5.9i Cc: hackers@freebsd.org Subject: Re: reverse of getchar() read() open() fopen() ? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 22:14:41 -0000 On Fri, Feb 11, 2011 at 10:32:46PM +0100, Julian H. Stacey wrote: > Hi hackers@, > Do we have C libraries with reverse of getchar() [ & maybe read() > ] & fopen() [ & maybe open() ] etc, to read from end of file toward > beginning ? I dont see anything in the See Also sections. I'm not > looking to write, just read. I'm looking for something that returns > last char in file as first etc, I'm not interested in wchars etc, > I could write some C functions, with seek etc & probably will, if > none exist, but no point if they already exist ? Use lseek() to position yourself, iterate over the file, copying into a small buffer, then iterate over your buffer in reverse? Maybe I misunderstood you... > Cheers, > Julian -- Brian Reichert 55 Crystal Ave. #286 Derry NH 03038-1725 USA BSD admin/developer at large From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 11 23:55:10 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7EC25106564A for ; Fri, 11 Feb 2011 23:55:10 +0000 (UTC) (envelope-from samflanker@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 064628FC14 for ; Fri, 11 Feb 2011 23:55:09 +0000 (UTC) Received: by fxm16 with SMTP id 16so3536243fxm.13 for ; Fri, 11 Feb 2011 15:55:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:message-id:date:from:user-agent:mime-version:to :subject:content-type:content-transfer-encoding; bh=8XEoNmBe0yXGX/VVLIzw6Fd3Ya7WQVu5gHs9H4awtw8=; b=cGPQ60Eu81+D8+IgQ6sfb8fAHh93Lg6t/UhQHzaH1ikYaLapXJI8o6cIIwSqD+Wprm 36+bVh34jZ1qfdIOYt+X1zE/pWyreUXQPndOJhyffSSAtP+SZ6KrPc5GpQrktFwEF6BO zaGngJ+BXQM3MZLnuNzZjwhjFbP94x58QdBGg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; b=sfDdYNPpImp/AmkBoOju3wzd/fkinhXHlUpdrJi9lbN3SC/rKmsPAknIpHb4OQdbHv ZKzgqfAKu8BDrlJDK6WhA77jFhg0lTwd0lxJLf78FE46xEisX+KzMH3Y6mlXVV0wxZ09 AGkXGRab7xtLIKeeknn//wzXPq6dmzws1z8SI= Received: by 10.223.103.198 with SMTP id l6mr195282fao.99.1297410723015; Thu, 10 Feb 2011 23:52:03 -0800 (PST) Received: from localhost.localdomain ([213.152.137.45]) by mx.google.com with ESMTPS id o12sm192168fav.6.2011.02.10.23.52.01 (version=SSLv3 cipher=OTHER); Thu, 10 Feb 2011 23:52:02 -0800 (PST) Message-ID: <4D54EAD3.9000005@gmail.com> Date: Fri, 11 Feb 2011 10:52:51 +0300 From: venom User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: problem with build mcelog X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Feb 2011 23:55:10 -0000 Hello. i am trying build mcelog # fetch http://ftp2.pl.freebsd.org/pub/FreeBSD/distfiles/mcelog-1.0pre2.tar.gz # tar -xf mcelog-1.0pre2.tar.gz # cd mcelog-1.0pre2 # fetch http://people.freebsd.org/~jhb/mcelog/mcelog.patch # fetch http://people.freebsd.org/~jhb/mcelog/memstream.c # patch < mcelog.patch # gmake FREEBSD=yes Makefile:92: .depend: No such file or directory cc -MM -I. p4.c k8.c mcelog.c dmi.c tsc.c core2.c bitfield.c intel.c nehalem.c dunnington.c tulsa.c config.c memutil.c msg.c eventloop.c leaky-bucket.c memdb.c server.c client.c cache.c rbtree.c memstream.c > .depend.X && mv .depend.X .depend cc -c -g -Os -Wall -Wextra -Wno-missing-field-initializers -Wno-unused-parameter -Wstrict-prototypes -Wformat-security -Wmissing-declarations -Wdeclaration-after-statement -o mcelog.o mcelog.c mcelog.c: In function 'process_kvm': mcelog.c:1342: error: 'X_SNAPDATE' undeclared (first use in this function) mcelog.c:1342: error: (Each undeclared identifier is reported only once mcelog.c:1342: error: for each function it appears in.) mcelog.c:1342: error: 'snapdate' undeclared (first use in this function) gmake: *** [mcelog.o] Error 1 please help me -- Vladimir Laskov From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 00:06:12 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 923901065670; Sat, 12 Feb 2011 00:06:12 +0000 (UTC) (envelope-from freebsd-hackers@herveybayaustralia.com.au) Received: from mail.unitedinsong.com.au (mail.unitedinsong.com.au [150.101.178.33]) by mx1.freebsd.org (Postfix) with ESMTP id EC11F8FC22; Sat, 12 Feb 2011 00:06:11 +0000 (UTC) Received: from laptop1.herveybayaustralia.com.au (laptop1.herveybayaustralia.com.au [192.168.0.186]) by mail.unitedinsong.com.au (Postfix) with ESMTP id 147D85C44; Sat, 12 Feb 2011 10:13:19 +1000 (EST) Message-ID: <4D55CE5A.8040902@herveybayaustralia.com.au> Date: Sat, 12 Feb 2011 10:03:38 +1000 From: Da Rock User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.16) Gecko/20110204 Thunderbird/3.0.11 ThunderBrowse/3.3.4 MIME-Version: 1.0 To: Julian Elischer References: <4D54E39D.1000505@herveybayaustralia.com.au> <4D54F0B0.7010503@freebsd.org> <4D550300.5090000@herveybayaustralia.com.au> <4D5565C7.1010809@freebsd.org> In-Reply-To: <4D5565C7.1010809@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, dudu@dudu.ro Subject: Re: linux PF_PACKET compatibility X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 00:06:12 -0000 On 02/12/11 02:37, Julian Elischer wrote: > On 2/11/11 1:36 AM, Da Rock wrote: >> On 02/11/11 18:17, Julian Elischer wrote: >>> On 2/10/11 11:22 PM, Da Rock wrote: >>>> "In recent versions of the Linux kernel (post-2.0 releases) a new >>>> protocol family has been introduced, named PF_PACKET. This family >>>> allows an application to send and receive packets dealing directly >>>> with the network card driver, thus avoiding the usual protocol >>>> stack-handling (e.g., IP/TCP or IP/UDP processing). That is, any >>>> packet sent through the socket will be directly passed to the >>>> Ethernet interface, and any packet received through the interface >>>> will be directly passed to the application." >>>> >>>> I've been chasing the answer to a FreeBSD version of this (approx. >>>> anyway), but I needed to find out what exactly PF_PACKET was first. >>>> Finally found this answer here: >>>> http://www.linuxjournal.com/article/4659 >>>> >>>> I looked up man socket and I can see possibilities (in my mind >>>> anyway), but I thought I'd be best to check if the gurus here might >>>> have a better idea. My reason for this is I'm attempting to build >>>> l2tpns (which supposedly builds on 7.3?! with no trouble), and I'm >>>> chasing the errors which appear to be linuxisms mostly. >>>> >>>> So in man socket simply looking at the list of protocol families >>>> I'd say network driver level would be similar to PF_LINK link layer >>>> interface? Is there another man page I should be looking at as well? >>> >>> We don't have an exact equivalent.. but we have ways of doing the >>> same thing. >>> one way that is suggested is to use pcap and bpf which I am pretty >>> certain has been enhanced to allow sending as >>> well as receiving. >>> you can also hook directly to the interface using netgraph(4) >>> there are other ways too but those are the two that came to mind >>> immediately. >> So I'm going to have to rewrite that interface entirely? Bugger! I >> just can't fathom how this howto could even exist for l2tpns on >> FreeBSD if it isn't even close to buildable... weird! >> >> http://kuapp.com/2010/07/14/how-to-setup-l2tpipsec-vpn-on-freebsd.html >> >> Thanks guys. I'll probably come back with more problems as I slowly >> crack this one... :) > > > nothing in that page needs to talk to the network interface > directly... what do you think does? I'm afraid you have me a little stumped. The PF_PACKET family talks directly with the network driver sending data to and from the app. Unfortunately this software uses this family instead of pcap or bpf. So when built it errors. I guess if I am to use this app I will have to rewrite the way it uses the network stack. > > > >>> >>> >>>> >>>> FWIW my gmake output is this: >>>> >>>> gcc -Wall -Wformat-security -Wno-format-zero-length -g -O3 -I. >>>> -I/usr/include -I/usr/local/include -DLIBDIR='"/lib/l2tpns"' >>>> -DETCDIR='"/etc/l2tpns"' -DSTATISTICS -DSTAT_CALLS -DRINGBUFFER >>>> -DHAVE_EPOLL -DBGP -c -o arp.o arp.c >>>> arp.c: In function 'sendarp': >>>> arp.c:34: error: storage size of 'sll' isn't known >>>> arp.c:59: error: 'PF_PACKET' undeclared (first use in this function) >>>> arp.c:59: error: (Each undeclared identifier is reported only once >>>> arp.c:59: error: for each function it appears in.) >>>> arp.c:62: error: 'AF_PACKET' undeclared (first use in this function) >>>> arp.c:34: warning: unused variable 'sll' >>>> gmake: *** [arp.o] Error 1 >>>> >>>> I posted this over at -questions@ but it was suggested to try here >>>> instead (or -net@). I figured this would be the best place to >>>> start... :) >>>> >>>> Cheers >>>> _______________________________________________ >>>> freebsd-hackers@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>>> To unsubscribe, send any mail to >>>> "freebsd-hackers-unsubscribe@freebsd.org" >>>> >>> >> >> > From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 00:11:23 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 620631065670 for ; Sat, 12 Feb 2011 00:11:23 +0000 (UTC) (envelope-from freebsd-hackers@herveybayaustralia.com.au) Received: from mail.unitedinsong.com.au (mail.unitedinsong.com.au [150.101.178.33]) by mx1.freebsd.org (Postfix) with ESMTP id D1CF58FC19 for ; Sat, 12 Feb 2011 00:11:22 +0000 (UTC) Received: from laptop1.herveybayaustralia.com.au (laptop1.herveybayaustralia.com.au [192.168.0.186]) by mail.unitedinsong.com.au (Postfix) with ESMTP id 4697F5C45 for ; Sat, 12 Feb 2011 10:18:31 +1000 (EST) Message-ID: <4D55CF92.4090608@herveybayaustralia.com.au> Date: Sat, 12 Feb 2011 10:08:50 +1000 From: Da Rock User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.16) Gecko/20110204 Thunderbird/3.0.11 ThunderBrowse/3.3.4 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <4D54E39D.1000505@herveybayaustralia.com.au> <4D54F0B0.7010503@freebsd.org> <4D550300.5090000@herveybayaustralia.com.au> <4D553C4B.5000101@herveybayaustralia.com.au> <4D55686B.5050202@freebsd.org> In-Reply-To: <4D55686B.5050202@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: linux PF_PACKET compatibility X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 00:11:23 -0000 On 02/12/11 02:48, Julian Elischer wrote: > On 2/11/11 5:40 AM, Da Rock wrote: >> On 02/11/11 19:54, Vlad Galu wrote: >>> >>> >>> On Fri, Feb 11, 2011 at 11:36 AM, Da Rock >>> >> > wrote: >>> >>> On 02/11/11 18:17, Julian Elischer wrote: >>> >>> On 2/10/11 11:22 PM, Da Rock wrote: >>> >>> "In recent versions of the Linux kernel (post-2.0 >>> releases) a new protocol family has been introduced, >>> named PF_PACKET. This family allows an application to >>> send and receive packets dealing directly with the >>> network card driver, thus avoiding the usual protocol >>> stack-handling (e.g., IP/TCP or IP/UDP processing). >>> That is, any packet sent through the socket will be >>> directly passed to the Ethernet interface, and any >>> packet received through the interface will be directly >>> passed to the application." >>> >>> I've been chasing the answer to a FreeBSD version of >>> this (approx. anyway), but I needed to find out what >>> exactly PF_PACKET was first. Finally found this answer >>> here: http://www.linuxjournal.com/article/4659 >>> >>> I looked up man socket and I can see possibilities (in >>> my mind anyway), but I thought I'd be best to check if >>> the gurus here might have a better idea. My reason for >>> this is I'm attempting to build l2tpns (which >>> supposedly builds on 7.3?! with no trouble), and I'm >>> chasing the errors which appear to be linuxisms mostly. >>> >>> So in man socket simply looking at the list of protocol >>> families I'd say network driver level would be similar >>> to PF_LINK link layer interface? Is there another man >>> page I should be looking at as well? >>> >>> >>> We don't have an exact equivalent.. but we have ways of >>> doing the same thing. >>> one way that is suggested is to use pcap and bpf which I am >>> pretty certain has been enhanced to allow sending as >>> well as receiving. >>> you can also hook directly to the interface using netgraph(4) >>> there are other ways too but those are the two that came to >>> mind immediately. >>> >>> So I'm going to have to rewrite that interface entirely? >>> Bugger! I just can't fathom how this howto could even exist for >>> l2tpns on FreeBSD if it isn't even close to buildable... weird! >>> >>> >>> http://kuapp.com/2010/07/14/how-to-setup-l2tpipsec-vpn-on-freebsd.html >>> >>> Thanks guys. I'll probably come back with more problems as I >>> slowly crack this one... :) >>> >>> >>> I suppose you could just use mpd :) >> I could, I guess. But where's the fun in that? :) >> >> Seriously, though, mpd didn't quite cut it (I thought) for me. I need >> a l2tp vpn server with the capability to handle multiple clients with >> only one interface. The server is behind a firewall, and I'm trying >> for a "walled garden" variety I guess. So far my research has brought >> me here, but I'm open to suggestions. > > why do you think you need only one interface? > >> >> One other that has my attention is l2tpd (in ports). I want radius >> auth, so IF I can use pppd in base and radius (which as I understand- >> so far anyway- it needs), and only uses a single interface, then maybe. > > pppd in base will I think give you multiple interfaces.. >> >> I'm still hunting and playing- learning on the fly. From what I read >> mpd uses an ng interface for every single client. L2tpns doesn't, and >> from what I've read so far neither does l2tpd (I was actually looking >> at another fork of that xl2tpd). I could use some advice from someone >> with experience with this, but my feelers on -questions didn't get >> much response. I may try on -net if this fails... > > again, whats' with the single interface? To be honest I don't know. But from what I've read up on it so far (including mpd - use and ng interface) I will have an net interface for every client. Apparently l2tpns doesn't do that, and one of the arguments for its use is that feature. If one has say 100 clients, each of those needs to be managed- 1 sounds better to me :) I am only working on theory here so far though, so please let me know if I'm on the wrong track. Cheers >> >> Aside from that I also wanted to get a bit more of a hands on feel >> for the FreeBSD core. I can't sit on the sidelines yelling at the >> players any more :) I'm not much for spectator sport either... > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 00:26:02 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E08D51065670 for ; Sat, 12 Feb 2011 00:26:02 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id 4FEC38FC15 for ; Sat, 12 Feb 2011 00:26:01 +0000 (UTC) Received: from park.js.berklix.net (p5B22E7EC.dip.t-dialin.net [91.34.231.236]) (authenticated bits=0) by tower.berklix.org (8.14.2/8.14.2) with ESMTP id p1C0PxBa021869 for ; Sat, 12 Feb 2011 00:26:00 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by park.js.berklix.net (8.13.8/8.13.8) with ESMTP id p1C0QEZI012845 for ; Sat, 12 Feb 2011 01:26:14 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.3/8.14.3) with ESMTP id p1C0Q9BW031963 for ; Sat, 12 Feb 2011 01:26:14 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <201102120026.p1C0Q9BW031963@fire.js.berklix.net> To: hackers@freebsd.org From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Unix Linux Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Fri, 11 Feb 2011 16:35:06 EST." Date: Sat, 12 Feb 2011 01:26:09 +0100 Sender: jhs@berklix.com Cc: Subject: Re: reverse of getchar() read() open() fopen() ? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 00:26:03 -0000 Andrew Duane wrote: > I've never seen any such thing, but I've done similar things a > lot. I'd say malloc/read the whole file in and use a decrementing > pointer to return the "previous" character. Thanks, but I'll need a loop too, as malloc(filesize()) would be too big, as I omitted to say I'll be running it from find, reading all files on system, including DVD images @ 4.7G. Some of my machines dont have that much swap, let alone RAM :-) Aside: I was doing inefficient things here with getchar() (while searching forward, for a one off run, (CPU cycles were cheaper than brain cycles, so it was easier to hack a pre-existing prog that used getchar() ). I was doing a search for (trailing) nulls blocks, from BSD tar silently zeroing data from bad media, per http://lists.freebsd.org/pipermail/freebsd-hackers/2011-January/034254.html http://www.freebsd.org/cgi/query-pr.cgi?pr=154407 http://berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/usr.bin/tar/ But my search logs from find with http://berklix.com/~jhs/src/bsd/jhs/bin/public/8f were too big, so I'll reduce by searching backward & only indexing on trailing nulls. Dan Nelson wrote: > You might even be able to write functions that could be passed to funopen(). > Then you'd have a regular FILE* that you could call with regular stdio > functions. Getting the buffering right for good performance might get > tricky, though. Thanks, I didnt know funopen, I'll read it again tomorrow morning :-) Thanks to Devin Teske re src/usr.bin/tail/, sorry in my first post I forgot to say binary. Brian Reichert wrote: > Use lseek() to position yourself, iterate over the file, copying into a > small buffer, then iterate over your buffer in reverse? Yes, thanks, better for efficiency, being lazy I had first wondered if there was some fopen_starting_from_tail() & getchar_starting_from_tail() I could just call for aone off run :-) Thanks all. Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Mail plain text; Not quoted-printable, Not HTML, Not base 64. Reply below text sections not at top, to avoid breaking cumulative context. From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 00:52:50 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 19B1C106564A; Sat, 12 Feb 2011 00:52:50 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id EB14F8FC08; Sat, 12 Feb 2011 00:52:49 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 792F746B06; Fri, 11 Feb 2011 19:52:49 -0500 (EST) Date: Sat, 12 Feb 2011 00:52:48 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Alexander Leidinger In-Reply-To: <20110211103028.12684f54yrw8tgqo@webmail.leidinger.net> Message-ID: References: <20110211103028.12684f54yrw8tgqo@webmail.leidinger.net> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: hackers@freebsd.org, kibab@freebsd.org Subject: Re: CFR: FEATURE macros for AUDIT/CAM/IPC/KTR/MAC/NFS/NTP/PMC/SYSV/... X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 00:52:50 -0000 On Fri, 11 Feb 2011, Alexander Leidinger wrote: > during the last GSoC various FEATURE macros where added to the system. > Before committing them, I would like to get some review (like if macro is in > the correct file, and for those FEATURES where the description was not taken > from NOTES if the description is OK). > > If nobody complains, I would like to commit this in 1-2 weeks. If you need > more time to review, just tell me. > > Here is the list of affected files (for those impatient ones which do not > want to look at the attached patch before noticing that they are not > interested to look at it): The additions for security/audit and security/mac both seem reasonable; I've been meaning to add them myself for quite a bit. There's then some code in libc that can learn to use this as well, at least for MAC. The one comment I'd make is that the MAC case should indicate that "The MAC Framework" is supported, rather than mandatory access controls being present -- the presence of the framework doesn't imply the presence of mandatory access control policies. Robert From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 00:54:53 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4DA00106566B for ; Sat, 12 Feb 2011 00:54:53 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id D09408FC14 for ; Sat, 12 Feb 2011 00:54:52 +0000 (UTC) Received: from park.js.berklix.net (p5B22FD12.dip.t-dialin.net [91.34.253.18]) (authenticated bits=0) by tower.berklix.org (8.14.2/8.14.2) with ESMTP id p1C0sne9022080; Sat, 12 Feb 2011 00:54:50 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by park.js.berklix.net (8.13.8/8.13.8) with ESMTP id p1C0t8Rj013012; Sat, 12 Feb 2011 01:55:08 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.3/8.14.3) with ESMTP id p1C0swnC069033; Sat, 12 Feb 2011 01:55:04 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <201102120055.p1C0swnC069033@fire.js.berklix.net> To: Bruce Cran From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Unix Linux Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Fri, 11 Feb 2011 19:22:27 GMT." <20110211192227.00002188@unknown> Date: Sat, 12 Feb 2011 01:54:58 +0100 Sender: jhs@berklix.com Cc: hackers@freebsd.org Subject: Re: memstick.img is bloated with 7% 2K blocks of nulls X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 00:54:53 -0000 Bruce Cran wrote: > On Fri, 11 Feb 2011 20:09:30 +0100 > "Julian H. Stacey" wrote: > > > memstick.img wastes 7% with 2K blocks of nulls. > > Could this be due to using UFS1 instead of UFS2? Don't know, Looking: 8.1 /usr/src/release/scripts/make-memstick.sh man makefs -t fs-type ... The following file system types are supported: ffs BSD fast file system (default). .... FFS-specific options .... version UFS version. 1 for FFS (default), 2 for UFS2 /usr/src/release/Makefile --> scripts/doFS.sh: newfs -i ${FSINODE} -o space -m 0 /dev/r${VNDEVICE}c ... newfs -O1 -b 4096 -f 512 -i ${FSINODE} -o space -m 0 /dev/${MDDEVICE} man newfs: -O filesystem-type Use 1 to specify that a UFS1 format file system be built; use 2 to specify that a UFS2 format file system be built. The default format is UFS2. If anyone fancies looking deeper, please do :-) Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Mail plain text; Not quoted-printable, Not HTML, Not base 64. Reply below text sections not at top, to avoid breaking cumulative context. From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 01:01:15 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BF6BC1065670; Sat, 12 Feb 2011 01:01:15 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 99E348FC17; Sat, 12 Feb 2011 01:01:15 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 522C446B06; Fri, 11 Feb 2011 20:01:15 -0500 (EST) Date: Sat, 12 Feb 2011 01:01:15 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Ilya Bakulin In-Reply-To: <4D556C1C.6030002@kibab.com> Message-ID: References: <20110211103028.12684f54yrw8tgqo@webmail.leidinger.net> <201102110751.52863.jhb@freebsd.org> <20110211145445.12722r7902rz8vc4@webmail.leidinger.net> <4D556C1C.6030002@kibab.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Alexander Leidinger , kibab@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: CFR: FEATURE macros for AUDIT/CAM/IPC/KTR/MAC/NFS/NTP/PMC/SYSV/... X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 01:01:15 -0000 On Fri, 11 Feb 2011, Ilya Bakulin wrote: > When I was beginning this GSoC work, I primarily thought about unifying the > way to determine if particular feature exists in the kernel. Of course there > should be at least one way to check if the feature is available or not (by > definition: if I may use some functionality, than feature is present, > otherwise... Oh, no, may be I have no permissions to use it? or something is > terribly wrong with system confuguration? Or?...), but it is better to have > a sort of unified way to get this information without looking for files in > /dev, parsing `kldstat -v`, etc. One of the nice things about this is that when a conditionally compiled feature introduces a new system call, there can be forward (rather than backward) compatibility benefits. If login(1) had checked for the Audit feature before trying audit system calls when we introduced it in 6.x, it would have avoided a few people shooting their feet off in the (officially unsupported) case where following a kernel and userspace roll-forward, a kernel roll-back was required to restore stability. While we don't support it (you shouldn't run a new userspace with an old kernel), the failure mode would have been improved. More abstractly: for a feature like MAC, testing for the presence of the framework is functionally fairly different from exercise the feature, as most instances of exercising it work only based on modules loaded by the framework, which is a different goal. Right now, libc offers a mac_present API, which back-ends into manually testing a system call. I'd rather it backended into a common feature test framework. In many cases, it is of course desirable to test for a feature by using it -- a much more pragmatic approach, and generally one preferred in the world of autoconf, etc... Robert From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 01:01:36 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 183FA10656A3 for ; Sat, 12 Feb 2011 01:01:36 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (unknown [IPv6:2a01:348:0:15:5d59:5c40:0:1]) by mx1.freebsd.org (Postfix) with ESMTP id 403808FC2C for ; Sat, 12 Feb 2011 01:01:35 +0000 (UTC) Received: from muon.cran.org.uk (localhost [127.0.0.1]) by muon.cran.org.uk (Postfix) with ESMTP id 1772DE8189; Sat, 12 Feb 2011 01:01:31 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=cran.org.uk; h=date:from :to:cc:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; s=mail; bh=8XeA7NUojXUf Tq299YkkFOZ0gEM=; b=cMjIfIKZ3XqgUzAqAbs6P40SDSMyUvY6KiPjmm+lW+nq RzO/+v6l6hkyeIJwWlWHY3FKW6HRBRNKlRyjlDz85dZfzmEuCaDs3Ys2HEbI/L6O ZERBNjRB6M/PHpT5PxkH9Sf8sVRxoJPipMU1A17d0wlKcuBDiAHmW60MpJY85Rk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=cran.org.uk; h=date:from:to :cc:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; q=dns; s=mail; b=ggo3aX CCc/d0Bj+LoBymkCs1TW5XXUZNNLP/Za3oyRcewvt9Z/Bmx3nvC8J+lWLE1DrGhq lj7bM3mfcEvUVnlMqcCwOnaMecXAVkT/MPQJvRY2O/NUrNq9UL0VNlirTjmSadtZ tIB1WrE77QiVHH/kUF0FxCX85MuZU17wvNwCA= Received: from unknown (client-86-23-108-125.brhm.adsl.virginmedia.com [86.23.108.125]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA id DDAADE7707; Sat, 12 Feb 2011 01:01:30 +0000 (GMT) Date: Sat, 12 Feb 2011 01:01:15 +0000 From: Bruce Cran To: "Julian H. Stacey" Message-ID: <20110212010115.000066db@unknown> In-Reply-To: <201102120055.p1C0swnC069033@fire.js.berklix.net> References: <20110211192227.00002188@unknown> <201102120055.p1C0swnC069033@fire.js.berklix.net> X-Mailer: Claws Mail 3.7.8cvs9 (GTK+ 2.16.6; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: hackers@freebsd.org Subject: Re: memstick.img is bloated with 7% 2K blocks of nulls X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 01:01:36 -0000 On Sat, 12 Feb 2011 01:54:58 +0100 "Julian H. Stacey" wrote: > -O filesystem-type > Use 1 to specify that a UFS1 format file system be > built; use 2 to specify that a UFS2 format file system be built. The > default format is UFS2. > If anyone fancies looking deeper, please do :-) I checked with dumpfs that memstick.img is UFS1. Also, mounting /dev/md0 confuses the kernel into ultimately panic'ing, since /dev/md0a is the proper slice. For the mfsroot.gz file from the CD ISOs: # mdconfig -a -f mfsroot md0 # mount /dev/md0a /mnt # ls /mnt ls: /mnt: Bad file descriptor # cd /mnt cd: /mnt: Not a directory # vim /mnt panic: ffs_read: type 0 kdb_enter() panic() ffs_read() vn_read dofileread() kern_readv() read() syscallenter() syscall() Xfast_syscall() -- Bruce Cran From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 01:16:33 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BE611106566C for ; Sat, 12 Feb 2011 01:16:33 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id 7DE7F8FC0A for ; Sat, 12 Feb 2011 01:16:33 +0000 (UTC) Received: from julian-mac.elischer.org (home-nat.elischer.org [67.100.89.137]) (authenticated bits=0) by vps1.elischer.org (8.14.4/8.14.4) with ESMTP id p1C1GPWM071937 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 11 Feb 2011 17:16:30 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <4D55DF75.4060703@freebsd.org> Date: Fri, 11 Feb 2011 17:16:37 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Da Rock References: <4D54E39D.1000505@herveybayaustralia.com.au> <4D54F0B0.7010503@freebsd.org> <4D550300.5090000@herveybayaustralia.com.au> <4D553C4B.5000101@herveybayaustralia.com.au> <4D55686B.5050202@freebsd.org> <4D55CF92.4090608@herveybayaustralia.com.au> In-Reply-To: <4D55CF92.4090608@herveybayaustralia.com.au> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: linux PF_PACKET compatibility X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 01:16:33 -0000 On 2/11/11 4:08 PM, Da Rock wrote: > On 02/12/11 02:48, Julian Elischer wrote: >> On 2/11/11 5:40 AM, Da Rock wrote: >>> On 02/11/11 19:54, Vlad Galu wrote: >>>> >>>> >>>> On Fri, Feb 11, 2011 at 11:36 AM, Da Rock >>>> >>> > wrote: >>>> >>>> On 02/11/11 18:17, Julian Elischer wrote: >>>> >>>> On 2/10/11 11:22 PM, Da Rock wrote: >>>> >>>> "In recent versions of the Linux kernel (post-2.0 >>>> releases) a new protocol family has been introduced, >>>> named PF_PACKET. This family allows an application to >>>> send and receive packets dealing directly with the >>>> network card driver, thus avoiding the usual protocol >>>> stack-handling (e.g., IP/TCP or IP/UDP processing). >>>> That is, any packet sent through the socket will be >>>> directly passed to the Ethernet interface, and any >>>> packet received through the interface will be directly >>>> passed to the application." >>>> >>>> I've been chasing the answer to a FreeBSD version of >>>> this (approx. anyway), but I needed to find out what >>>> exactly PF_PACKET was first. Finally found this answer >>>> here: http://www.linuxjournal.com/article/4659 >>>> >>>> I looked up man socket and I can see possibilities (in >>>> my mind anyway), but I thought I'd be best to check if >>>> the gurus here might have a better idea. My reason for >>>> this is I'm attempting to build l2tpns (which >>>> supposedly builds on 7.3?! with no trouble), and I'm >>>> chasing the errors which appear to be linuxisms mostly. >>>> >>>> So in man socket simply looking at the list of protocol >>>> families I'd say network driver level would be similar >>>> to PF_LINK link layer interface? Is there another man >>>> page I should be looking at as well? >>>> >>>> >>>> We don't have an exact equivalent.. but we have ways of >>>> doing the same thing. >>>> one way that is suggested is to use pcap and bpf which I am >>>> pretty certain has been enhanced to allow sending as >>>> well as receiving. >>>> you can also hook directly to the interface using >>>> netgraph(4) >>>> there are other ways too but those are the two that came to >>>> mind immediately. >>>> >>>> So I'm going to have to rewrite that interface entirely? >>>> Bugger! I just can't fathom how this howto could even exist for >>>> l2tpns on FreeBSD if it isn't even close to buildable... weird! >>>> >>>> >>>> http://kuapp.com/2010/07/14/how-to-setup-l2tpipsec-vpn-on-freebsd.html >>>> >>>> >>>> Thanks guys. I'll probably come back with more problems as I >>>> slowly crack this one... :) >>>> >>>> >>>> I suppose you could just use mpd :) >>> I could, I guess. But where's the fun in that? :) >>> >>> Seriously, though, mpd didn't quite cut it (I thought) for me. I >>> need a l2tp vpn server with the capability to handle multiple >>> clients with only one interface. The server is behind a firewall, >>> and I'm trying for a "walled garden" variety I guess. So far my >>> research has brought me here, but I'm open to suggestions. >> >> why do you think you need only one interface? >> >>> >>> One other that has my attention is l2tpd (in ports). I want radius >>> auth, so IF I can use pppd in base and radius (which as I >>> understand- so far anyway- it needs), and only uses a single >>> interface, then maybe. >> >> pppd in base will I think give you multiple interfaces.. >>> >>> I'm still hunting and playing- learning on the fly. From what I >>> read mpd uses an ng interface for every single client. L2tpns >>> doesn't, and from what I've read so far neither does l2tpd (I was >>> actually looking at another fork of that xl2tpd). I could use some >>> advice from someone with experience with this, but my feelers on >>> -questions didn't get much response. I may try on -net if this >>> fails... >> >> again, whats' with the single interface? > To be honest I don't know. But from what I've read up on it so far > (including mpd - use and ng interface) I will have an net interface > for every client. Apparently l2tpns doesn't do that, and one of the > arguments for its use is that feature. If one has say 100 clients, > each of those needs to be managed- 1 sounds better to me :) > > I am only working on theory here so far though, so please let me > know if I'm on the wrong track. if you have multiple interfaces you can set differnt mtus for them etc. and routing is more straight forward. you can do tcpdump on a particular interface or filter on just one interface. there have been people with > 100 interfaces.. who didn't seem to have any problems. there are advantages and disadvantages. > > Cheers >>> >>> Aside from that I also wanted to get a bit more of a hands on feel >>> for the FreeBSD core. I can't sit on the sidelines yelling at the >>> players any more :) I'm not much for spectator sport either... >> >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to >> "freebsd-hackers-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 01:16:35 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EA7A01065670 for ; Sat, 12 Feb 2011 01:16:35 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 895058FC12 for ; Sat, 12 Feb 2011 01:16:33 +0000 (UTC) Received: by fxm16 with SMTP id 16so3598358fxm.13 for ; Fri, 11 Feb 2011 17:16:32 -0800 (PST) Received: by 10.223.81.76 with SMTP id w12mr219956fak.26.1297414383930; Fri, 11 Feb 2011 00:53:03 -0800 (PST) Received: from dfleuriot.local ([83.167.62.196]) by mx.google.com with ESMTPS id n1sm211088fam.40.2011.02.11.00.53.03 (version=SSLv3 cipher=OTHER); Fri, 11 Feb 2011 00:53:03 -0800 (PST) Message-ID: <4D54F8EE.5020409@my.gd> Date: Fri, 11 Feb 2011 09:53:02 +0100 From: Damien Fleuriot User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <1297396433.7232.9.camel@dt.vicor.com> In-Reply-To: <1297396433.7232.9.camel@dt.vicor.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Subject: Re: [RELEASE] host-setup(1): a dialog(1)-based utility for configuring FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 01:16:36 -0000 On 2/11/11 4:53 AM, Devin Teske wrote: > Hi All, > > I'd like to announce the release of a new script. A script that I've > developed for our field engineers that I'd like to share with the rest > of the world. > > http://druidbsd.sourceforge.net/download/host-setup.txt > > host-setup(1) is a dialog(1)-based utility (written in sh(1)) designed > to make configuring FreeBSD more efficient. > > We have this script configured to be run as root's initial login > immediately after "first-boot". The field engineer uses our custom > installer to install RELENG_8, and after the machine presents a login > prompt, they login with the pre-configured root password. After which > they are presented with this dialog (image attached: > host-setup.pub.png). > > The dialogs should all be intuitive, and I hope that you like my work. > I've worked very hard to make this utility smooth, robust, > fault-tolerant and even enjoyable to use. Not only that, but it helps > prevents mistakes that could arise in doing these steps by-hand (like > forgetting to unmount active NFS-mounts before executing "route flush"). > > Please give it a try and let me know what you think. > -- > Devin > > P.S. This is not a trivial script. ``More nuclear reactor than bike > shed'' ^_^ > > P.P.S. Should be backward compatible to RELENG_4. > > P.P.S. I know the screenshot says "host-setup.pub" -- that's only > because our in-house-only version has more functionality than the one > I'm releasing to the general public today (no functionality that anyone > in the public audience would ever care about though). > Hi Devin, Thanks for sharing, but I'm afraid the list automagically strips away non-text attachments, you'll have to upload your screenshots somewhere :) From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 01:19:11 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 70896106564A for ; Sat, 12 Feb 2011 01:19:11 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id 4D7678FC17 for ; Sat, 12 Feb 2011 01:19:10 +0000 (UTC) Received: from julian-mac.elischer.org (home-nat.elischer.org [67.100.89.137]) (authenticated bits=0) by vps1.elischer.org (8.14.4/8.14.4) with ESMTP id p1C1J5b5071948 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 11 Feb 2011 17:19:07 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <4D55E015.3010709@freebsd.org> Date: Fri, 11 Feb 2011 17:19:17 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Da Rock References: <4D54E39D.1000505@herveybayaustralia.com.au> <4D54F0B0.7010503@freebsd.org> <4D550300.5090000@herveybayaustralia.com.au> <4D5565C7.1010809@freebsd.org> <4D55CE5A.8040902@herveybayaustralia.com.au> In-Reply-To: <4D55CE5A.8040902@herveybayaustralia.com.au> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, dudu@dudu.ro Subject: Re: [maybe spam] Re: linux PF_PACKET compatibility X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 01:19:11 -0000 On 2/11/11 4:03 PM, Da Rock wrote: > On 02/12/11 02:37, Julian Elischer wrote: >> On 2/11/11 1:36 AM, Da Rock wrote: >>> On 02/11/11 18:17, Julian Elischer wrote: >>>> On 2/10/11 11:22 PM, Da Rock wrote: >>>>> "In recent versions of the Linux kernel (post-2.0 releases) a >>>>> new protocol family has been introduced, named PF_PACKET. This >>>>> family allows an application to send and receive packets dealing >>>>> directly with the network card driver, thus avoiding the usual >>>>> protocol stack-handling (e.g., IP/TCP or IP/UDP processing). >>>>> That is, any packet sent through the socket will be directly >>>>> passed to the Ethernet interface, and any packet received >>>>> through the interface will be directly passed to the application." >>>>> >>>>> I've been chasing the answer to a FreeBSD version of this >>>>> (approx. anyway), but I needed to find out what exactly >>>>> PF_PACKET was first. Finally found this answer here: >>>>> http://www.linuxjournal.com/article/4659 >>>>> >>>>> I looked up man socket and I can see possibilities (in my mind >>>>> anyway), but I thought I'd be best to check if the gurus here >>>>> might have a better idea. My reason for this is I'm attempting >>>>> to build l2tpns (which supposedly builds on 7.3?! with no >>>>> trouble), and I'm chasing the errors which appear to be >>>>> linuxisms mostly. >>>>> >>>>> So in man socket simply looking at the list of protocol families >>>>> I'd say network driver level would be similar to PF_LINK link >>>>> layer interface? Is there another man page I should be looking >>>>> at as well? >>>> >>>> We don't have an exact equivalent.. but we have ways of doing the >>>> same thing. >>>> one way that is suggested is to use pcap and bpf which I am >>>> pretty certain has been enhanced to allow sending as >>>> well as receiving. >>>> you can also hook directly to the interface using netgraph(4) >>>> there are other ways too but those are the two that came to mind >>>> immediately. >>> So I'm going to have to rewrite that interface entirely? Bugger! I >>> just can't fathom how this howto could even exist for l2tpns on >>> FreeBSD if it isn't even close to buildable... weird! >>> >>> http://kuapp.com/2010/07/14/how-to-setup-l2tpipsec-vpn-on-freebsd.html >>> >>> >>> Thanks guys. I'll probably come back with more problems as I >>> slowly crack this one... :) >> >> >> nothing in that page needs to talk to the network interface >> directly... what do you think does? > I'm afraid you have me a little stumped. The PF_PACKET family talks > directly with the network driver sending data to and from the app. > > Unfortunately this software uses this family instead of pcap or bpf. > So when built it errors. > > I guess if I am to use this app I will have to rewrite the way it > uses the network stack. l2tp runs over UDP packets (port 1701 (like the starship enterprise)) I have no idea why they want raw packets. talk to the writer of the web page you indicated.. maybe he can help.. >> >> >> >>>> >>>> >>>>> >>>>> FWIW my gmake output is this: >>>>> >>>>> gcc -Wall -Wformat-security -Wno-format-zero-length -g -O3 -I. >>>>> -I/usr/include -I/usr/local/include -DLIBDIR='"/lib/l2tpns"' >>>>> -DETCDIR='"/etc/l2tpns"' -DSTATISTICS -DSTAT_CALLS -DRINGBUFFER >>>>> -DHAVE_EPOLL -DBGP -c -o arp.o arp.c >>>>> arp.c: In function 'sendarp': >>>>> arp.c:34: error: storage size of 'sll' isn't known >>>>> arp.c:59: error: 'PF_PACKET' undeclared (first use in this >>>>> function) >>>>> arp.c:59: error: (Each undeclared identifier is reported only once >>>>> arp.c:59: error: for each function it appears in.) >>>>> arp.c:62: error: 'AF_PACKET' undeclared (first use in this >>>>> function) >>>>> arp.c:34: warning: unused variable 'sll' >>>>> gmake: *** [arp.o] Error 1 >>>>> >>>>> I posted this over at -questions@ but it was suggested to try >>>>> here instead (or -net@). I figured this would be the best place >>>>> to start... :) >>>>> >>>>> Cheers >>>>> _______________________________________________ >>>>> freebsd-hackers@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>>>> To unsubscribe, send any mail to >>>>> "freebsd-hackers-unsubscribe@freebsd.org" >>>>> >>>> >>> >>> >> > > From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 02:02:24 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0A8C2106566C for ; Sat, 12 Feb 2011 02:02:24 +0000 (UTC) (envelope-from santosh@fastsoft.com) Received: from hq-es.fastsoft.com (hq-es.fastsoft.com [38.102.243.86]) by mx1.freebsd.org (Postfix) with ESMTP id DEF188FC08 for ; Sat, 12 Feb 2011 02:02:22 +0000 (UTC) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Date: Fri, 11 Feb 2011 17:50:16 -0800 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: ixgbe DMA question Thread-Index: AcvKVzI4leMo3PwkQl6kXdO3C585bg== From: "Santosh Rao Gururajan" To: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: ixgbe DMA question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 02:02:24 -0000 I have a host machine with 2 ixgbe NICs. I am trying to pass the frames from one NIC to the other with the lowest possible overhead to the host (high speed bridge). I am wondering if I can do a rx-ring to tx-ring DMA copy without creating a mbuf on the host. Is that possible? What are the risks? Thanks, -santosh From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 02:05:53 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 43F951065672 for ; Sat, 12 Feb 2011 02:05:53 +0000 (UTC) (envelope-from samflanker@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id C6D918FC13 for ; Sat, 12 Feb 2011 02:05:52 +0000 (UTC) Received: by fxm16 with SMTP id 16so3628098fxm.13 for ; Fri, 11 Feb 2011 18:05:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:message-id:date:from:user-agent:mime-version:to :subject:content-type:content-transfer-encoding; bh=tbx9jTAwnsOghkU0ex5PTPSCI8UGS4RvwnM2oqNkWXc=; b=HX6/7+Lrbyk350kqSu7kOgoJ2wq+B1y3ps7UJMGABXKo9NaE0XYNXLwxcpds/qPRGf k43YBZOPuP7GZGu1iORvSh4Z2nLnSyZ/JGnBmXqCb+ggQXtZNg9pa3uonQB4piUwsJ3G BkzhmqdNgmTMCxS7qXDMvDs/1IziE39ibkZ4s= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; b=AkOhiCvFY1ejw8zGWMufWRKb7hVvUr4iA2CUjRBQbww7HRWG1+zlEFXK94ZIuhF5sf 5WEhLVjpd6hMECYTI6tsBkWdogKXDnjhu7DCJC3oyvjaOZs9CHViZHVJf84KX6xm28qm DD63SK7l7rTjA4PcdtUe/QavC7Dr+mAxhZ4bY= Received: by 10.223.53.68 with SMTP id l4mr276139fag.44.1297417564367; Fri, 11 Feb 2011 01:46:04 -0800 (PST) Received: from localhost.localdomain ([213.152.137.45]) by mx.google.com with ESMTPS id n3sm238532faa.29.2011.02.11.01.46.02 (version=SSLv3 cipher=OTHER); Fri, 11 Feb 2011 01:46:03 -0800 (PST) Message-ID: <4D55058D.30807@gmail.com> Date: Fri, 11 Feb 2011 12:46:53 +0300 From: venom User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc14 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: problem with build mcelog X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 02:05:53 -0000 Hello. i am trying build mcelog FreeBSD XXXX 8.1-RELEASE-p2 FreeBSD 8.1-RELEASE-p2 #0: Fri Jan 14 04:15:56 UTC 2011 root@freebsd:/usr/obj/usr/src/sys/GENERIC amd64 # fetch http://ftp2.pl.freebsd.org/pub/FreeBSD/distfiles/mcelog-1.0pre2.tar.gz # tar -xf mcelog-1.0pre2.tar.gz # cd mcelog-1.0pre2 # fetch http://people.freebsd.org/~jhb/mcelog/mcelog.patch # fetch http://people.freebsd.org/~jhb/mcelog/memstream.c # patch< mcelog.patch # gmake FREEBSD=yes Makefile:92: .depend: No such file or directory cc -MM -I. p4.c k8.c mcelog.c dmi.c tsc.c core2.c bitfield.c intel.c nehalem.c dunnington.c tulsa.c config.c memutil.c msg.c eventloop.c leaky-bucket.c memdb.c server.c client.c cache.c rbtree.c memstream.c> .depend.X&& mv .depend.X .depend cc -c -g -Os -Wall -Wextra -Wno-missing-field-initializers -Wno-unused-parameter -Wstrict-prototypes -Wformat-security -Wmissing-declarations -Wdeclaration-after-statement -o mcelog.o mcelog.c mcelog.c: In function 'process_kvm': mcelog.c:1342: error: 'X_SNAPDATE' undeclared (first use in this function) mcelog.c:1342: error: (Each undeclared identifier is reported only once mcelog.c:1342: error: for each function it appears in.) mcelog.c:1342: error: 'snapdate' undeclared (first use in this function) gmake: *** [mcelog.o] Error 1 please help me -- Vladimir Laskov From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 02:07:47 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DD628106564A; Sat, 12 Feb 2011 02:07:46 +0000 (UTC) (envelope-from freebsd-hackers@herveybayaustralia.com.au) Received: from mail.unitedinsong.com.au (mail.unitedinsong.com.au [150.101.178.33]) by mx1.freebsd.org (Postfix) with ESMTP id CCB6A8FC15; Sat, 12 Feb 2011 02:07:45 +0000 (UTC) Received: from laptop1.herveybayaustralia.com.au (laptop1.herveybayaustralia.com.au [192.168.0.186]) by mail.unitedinsong.com.au (Postfix) with ESMTP id 54A015C44; Sat, 12 Feb 2011 12:14:54 +1000 (EST) Message-ID: <4D55EAD9.2070602@herveybayaustralia.com.au> Date: Sat, 12 Feb 2011 12:05:13 +1000 From: Da Rock User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.16) Gecko/20110204 Thunderbird/3.0.11 ThunderBrowse/3.3.4 MIME-Version: 1.0 To: Julian Elischer References: <4D54E39D.1000505@herveybayaustralia.com.au> <4D54F0B0.7010503@freebsd.org> <4D550300.5090000@herveybayaustralia.com.au> <4D5565C7.1010809@freebsd.org> <4D55CE5A.8040902@herveybayaustralia.com.au> <4D55E015.3010709@freebsd.org> In-Reply-To: <4D55E015.3010709@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org, dudu@dudu.ro Subject: Re: [maybe spam] Re: linux PF_PACKET compatibility X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 02:07:47 -0000 On 02/12/11 11:19, Julian Elischer wrote: > On 2/11/11 4:03 PM, Da Rock wrote: >> On 02/12/11 02:37, Julian Elischer wrote: >>> On 2/11/11 1:36 AM, Da Rock wrote: >>>> On 02/11/11 18:17, Julian Elischer wrote: >>>>> On 2/10/11 11:22 PM, Da Rock wrote: >>>>>> "In recent versions of the Linux kernel (post-2.0 releases) a new >>>>>> protocol family has been introduced, named PF_PACKET. This family >>>>>> allows an application to send and receive packets dealing >>>>>> directly with the network card driver, thus avoiding the usual >>>>>> protocol stack-handling (e.g., IP/TCP or IP/UDP processing). That >>>>>> is, any packet sent through the socket will be directly passed to >>>>>> the Ethernet interface, and any packet received through the >>>>>> interface will be directly passed to the application." >>>>>> >>>>>> I've been chasing the answer to a FreeBSD version of this >>>>>> (approx. anyway), but I needed to find out what exactly PF_PACKET >>>>>> was first. Finally found this answer here: >>>>>> http://www.linuxjournal.com/article/4659 >>>>>> >>>>>> I looked up man socket and I can see possibilities (in my mind >>>>>> anyway), but I thought I'd be best to check if the gurus here >>>>>> might have a better idea. My reason for this is I'm attempting to >>>>>> build l2tpns (which supposedly builds on 7.3?! with no trouble), >>>>>> and I'm chasing the errors which appear to be linuxisms mostly. >>>>>> >>>>>> So in man socket simply looking at the list of protocol families >>>>>> I'd say network driver level would be similar to PF_LINK link >>>>>> layer interface? Is there another man page I should be looking at >>>>>> as well? >>>>> >>>>> We don't have an exact equivalent.. but we have ways of doing the >>>>> same thing. >>>>> one way that is suggested is to use pcap and bpf which I am pretty >>>>> certain has been enhanced to allow sending as >>>>> well as receiving. >>>>> you can also hook directly to the interface using netgraph(4) >>>>> there are other ways too but those are the two that came to mind >>>>> immediately. >>>> So I'm going to have to rewrite that interface entirely? Bugger! I >>>> just can't fathom how this howto could even exist for l2tpns on >>>> FreeBSD if it isn't even close to buildable... weird! >>>> >>>> http://kuapp.com/2010/07/14/how-to-setup-l2tpipsec-vpn-on-freebsd.html >>>> >>>> Thanks guys. I'll probably come back with more problems as I slowly >>>> crack this one... :) >>> >>> >>> nothing in that page needs to talk to the network interface >>> directly... what do you think does? >> I'm afraid you have me a little stumped. The PF_PACKET family talks >> directly with the network driver sending data to and from the app. >> >> Unfortunately this software uses this family instead of pcap or bpf. >> So when built it errors. >> >> I guess if I am to use this app I will have to rewrite the way it >> uses the network stack. > l2tp runs over UDP packets (port 1701 (like the starship enterprise)) > I have no idea why they want raw packets. Neither do I. > > talk to the writer of the web page you indicated.. maybe he can help.. I did. No response so far... It was for 7.3, but I can't see the difference. (I'll merge the 2 threads of this to make it easier and less consuming) >>>again, whats' with the single interface? >>To be honest I don't know. But from what I've read up on it so far (including mpd - use and ng interface) I will have an net interface for every client. Apparently l2tpns doesn't do that, and one of the arguments for its use is that feature. If one has say 100 clients, each of those needs to be managed- 1 sounds better to me :) I am only working on theory here so far though, so please let me know if I'm on the wrong track. >if you have multiple interfaces you can set differnt mtus for them etc. and routing is more straight forward. you can do tcpdump on a particular interface or filter on just one interface. there have been people with > 100 interfaces.. who didn't seem to have any problems. there are advantages and disadvantages. I'd say you're right. But if you have all those interfaces (and this what my first thought was when I started this) then it requires more holes in the firewall- right? So for every ng interface I'd need to accept 1701, which gets messy in PF because I'd have to dictate where to rdr every incoming 1701. OTOH, if I have just a tun device for every 1701 to connect to then I only have to rdr all 1701 to the l2tp server. Routing can be done just on that server (mostly to reroute back out again) either through ppp static routing or otherwise. At least this is my impression so far. By all means let me know if I'm idiot with my head on backwards- I'm trying to get beyond theory before I start chatting on the -net list... :) > >>> >>> >>> >>>>> >>>>> >>>>>> >>>>>> FWIW my gmake output is this: >>>>>> >>>>>> gcc -Wall -Wformat-security -Wno-format-zero-length -g -O3 -I. >>>>>> -I/usr/include -I/usr/local/include -DLIBDIR='"/lib/l2tpns"' >>>>>> -DETCDIR='"/etc/l2tpns"' -DSTATISTICS -DSTAT_CALLS -DRINGBUFFER >>>>>> -DHAVE_EPOLL -DBGP -c -o arp.o arp.c >>>>>> arp.c: In function 'sendarp': >>>>>> arp.c:34: error: storage size of 'sll' isn't known >>>>>> arp.c:59: error: 'PF_PACKET' undeclared (first use in this function) >>>>>> arp.c:59: error: (Each undeclared identifier is reported only once >>>>>> arp.c:59: error: for each function it appears in.) >>>>>> arp.c:62: error: 'AF_PACKET' undeclared (first use in this function) >>>>>> arp.c:34: warning: unused variable 'sll' >>>>>> gmake: *** [arp.o] Error 1 >>>>>> >>>>>> I posted this over at -questions@ but it was suggested to try >>>>>> here instead (or -net@). I figured this would be the best place >>>>>> to start... :) >>>>>> >>>>>> Cheers >>>>>> _______________________________________________ >>>>>> freebsd-hackers@freebsd.org mailing list >>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>>>>> To unsubscribe, send any mail to >>>>>> "freebsd-hackers-unsubscribe@freebsd.org" >>>>>> >>>>> >>>> >>>> >>> >> >> > From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 03:58:48 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6E438106566C for ; Sat, 12 Feb 2011 03:58:48 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) by mx1.freebsd.org (Postfix) with ESMTP id 810D28FC08 for ; Sat, 12 Feb 2011 03:58:47 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id p1C3TuEG041406; Sat, 12 Feb 2011 14:29:57 +1100 (EST) (envelope-from smithi@nimnet.asn.au) Date: Sat, 12 Feb 2011 14:29:56 +1100 (EST) From: Ian Smith To: Eitan Adler In-Reply-To: Message-ID: <20110212140035.H96449@sola.nimnet.asn.au> References: <20110211120031.9D37510656E5@hub.freebsd.org> <20110212015155.I96449@sola.nimnet.asn.au> <1297447687.9144.6.camel@dt.vicor.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Mailman-Approved-At: Sat, 12 Feb 2011 06:15:08 +0000 Cc: freebsd-hackers@freebsd.org, Devin Teske , freebsd-questions@freebsd.org Subject: Re: [RELEASE] host-setup(1): a dialog(1)-based utility for configuring FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 03:58:48 -0000 On Fri, 11 Feb 2011, Eitan Adler wrote: > Nice Script! > I intend to steal parts of it for my own use. It's great when you can plunder without robbing anyone :) > > P.S. Maybe I ought to expand it to IPv6 considering that the IPv4 > > address space has [reportedly] finally ran out (is that true?). > > > > All the available IPs were allocated to the RIRs. AFIK the RIRs have > not had to deny anyone for insufficiency yet - but it will happen > soon. Yes Devin, best not leave it till August! For those wanting a near-obsessively detailed analysis of IPv4 depletion stats and predictions over many years, hard to go past Geoff Huston's: http://www.potaroo.net - blog http://www.potaroo.net/ispcol/2010-10/when.html - explanatory column Oct '10 http://www.potaroo.net/tools/ipv4/index.html - the modelling as of today cheers, Ian (Sorry, missed the cc to hackers@, adding questions@ back in the loop) From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 07:56:51 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2BF3A106566C for ; Sat, 12 Feb 2011 07:56:51 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 082AF8FC12 for ; Sat, 12 Feb 2011 07:56:51 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 8D6C346B09; Sat, 12 Feb 2011 02:56:50 -0500 (EST) Date: Sat, 12 Feb 2011 07:56:50 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Santosh Rao Gururajan In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-hackers@freebsd.org Subject: Re: ixgbe DMA question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 07:56:51 -0000 On Fri, 11 Feb 2011, Santosh Rao Gururajan wrote: > I have a host machine with 2 ixgbe NICs. I am trying to pass the frames from > one NIC to the other with the lowest possible overhead to the host (high > speed bridge). I am wondering if I can do a rx-ring to tx-ring DMA copy > without creating a mbuf on the host. Is that possible? What are the risks? The only real risk is the simple matter of programming, I think. There's no reason not to it except that it involves modifying device drivers, memory models, etc. If you do what you describe, and you decide you do want to pass some frames up the stack, you can always hook up mbufs and use the external storage free routine to return the memory to the ring. Jeff Roberson has been circulating some patches that eliminate the mbuf<->cluster relationship in its current form, instead preferring variable size mbufs, and I can't help but wonder if with such a patch, that wouldn't be simpler than what you propose, offering many of the same performance benefits while making the device driver changes smaller and still allowing you to direct some packets up the stack if desired. Robert From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 09:39:27 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 81C86106566C for ; Sat, 12 Feb 2011 09:39:27 +0000 (UTC) (envelope-from gljennjohn@googlemail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 0C0FC8FC18 for ; Sat, 12 Feb 2011 09:39:26 +0000 (UTC) Received: by fxm16 with SMTP id 16so3820755fxm.13 for ; Sat, 12 Feb 2011 01:39:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:x-mailer:mime-version:content-type :content-transfer-encoding; bh=P+iKEjFEczCmIP7q8AOnRREHf+BbywkvAciEGR/Ltq0=; b=aQr3ViiGEbTkiPiqp66oU4WvdfF22ebgQZ98+t+hjm9KqLrgqSm9qdJXqiC7il8qO0 S8Cx/Jxg7tLQB+55nLDo9OGoNjf2nj6K0NwxRKmPDDgakxYovb2mwPpvDyYho1ATj9RA ei88uKlVpCzm6UqauWozgs7b+12SKzk6Tpjr8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :x-mailer:mime-version:content-type:content-transfer-encoding; b=ZAa+NY6S7vatMlLZ3tOEw88Ra3MQbeaXNRDPRStQuTchXyuutVevzsSQA2g5VP7Zze MlVHLbu16gO8uz1PbmsXAVqU1PpAnSlhbzoFMZu30WeKEFaTZ04xXvZxv1FDJut7ueWg XriJYOBbQsS4hBNOuyPQZTupAKJWdPMek1vEE= Received: by 10.223.96.206 with SMTP id i14mr1712886fan.67.1297503565702; Sat, 12 Feb 2011 01:39:25 -0800 (PST) Received: from ernst.jennejohn.org (p578E2B35.dip.t-dialin.net [87.142.43.53]) by mx.google.com with ESMTPS id n26sm93314fam.13.2011.02.12.01.39.24 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 12 Feb 2011 01:39:25 -0800 (PST) Date: Sat, 12 Feb 2011 10:39:23 +0100 From: Gary Jennejohn To: Julian Elischer Message-ID: <20110212103923.3098f6b3@ernst.jennejohn.org> In-Reply-To: <4D55E015.3010709@freebsd.org> References: <4D54E39D.1000505@herveybayaustralia.com.au> <4D54F0B0.7010503@freebsd.org> <4D550300.5090000@herveybayaustralia.com.au> <4D5565C7.1010809@freebsd.org> <4D55CE5A.8040902@herveybayaustralia.com.au> <4D55E015.3010709@freebsd.org> X-Mailer: Claws Mail 3.7.8 (GTK+ 2.18.7; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Da Rock , dudu@dudu.ro Subject: Re: [maybe spam] Re: linux PF_PACKET compatibility X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gljennjohn@googlemail.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 09:39:27 -0000 On Fri, 11 Feb 2011 17:19:17 -0800 Julian Elischer wrote: > On 2/11/11 4:03 PM, Da Rock wrote: > > Unfortunately this software uses this family instead of pcap or bpf. > > So when built it errors. > > > > I guess if I am to use this app I will have to rewrite the way it > > uses the network stack. > l2tp runs over UDP packets (port 1701 (like the starship enterprise)) > I have no idea why they want raw packets. > Ther's a sendarp() routine which uses PF_PACKET to directly access the network driver and bypass the stack. Lazy Linuxers who have no idea or don't care that other operating systems exist. -- Gary Jennejohn From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 09:46:16 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 18E9F106566B for ; Sat, 12 Feb 2011 09:46:16 +0000 (UTC) (envelope-from gujjenaveen@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id CA84D8FC16 for ; Sat, 12 Feb 2011 09:46:15 +0000 (UTC) Received: by gyc15 with SMTP id 15so1442048gyc.13 for ; Sat, 12 Feb 2011 01:46:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:date:message-id:subject:from:to :content-type; bh=tVKczv7vCqtnyWbjs4uZ9lrhuiYDeeTx00Y2BItcmOg=; b=eLMy0BlDLvAT0NS7m8hcu+SnQ+JRGApx7gb1e/yvY0RSwb4kZ6cddePgmrjqgrcSq4 JglhEPyjqOULyNbMt6vcn8JTUIuFjah91+b1snQm34gFMnLpO8pJzHEmpi7OzwVoj70n IOTafeosXwocNSI3qrGIlEd+20qGBkyYgBpEo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=rN6kQUMhHp6ipkJcET7RJzmsjvOMJTGxJmup2BpnZHJwYkKzgHIXDRMr3iS+T5xoRy Z+lAePurcydRfQ8pAALk+DZxFNqBdrBPdg+B5noDWcdKJ6WAB8C1ThUa8IzRUlNQJI6T w784unzygHUQ5om5FehF7yV6vX0XVaPhVWQ24= MIME-Version: 1.0 Received: by 10.236.110.5 with SMTP id t5mr2726530yhg.8.1297502274810; Sat, 12 Feb 2011 01:17:54 -0800 (PST) Received: by 10.236.110.129 with HTTP; Sat, 12 Feb 2011 01:17:54 -0800 (PST) Date: Sat, 12 Feb 2011 14:47:54 +0530 Message-ID: From: Naveen Gujje To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: SO_SETFIB socket option X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 09:46:16 -0000 Hi All, On my FreeBSD 7.2 box, I've two routing tables (FIBs). Fib 0 and Fib 1 (net.fibs = 2). I have a simple echo client which is the counterpart of an echo server running somewhere. If I run this echo client against fib 0 as 'setfib 0 ./echo-client', it properly uses Fib 0. But, if I run this echo client against Fib 0 by using setsockopt & SO_SETFIB option, setsockopt fails with EINVAL. setsockopt & SO_SETFIB for Fib 1 succeeds. But it fails for Fib 0. By looking at the man page and /sys/kern/uipc_socket.c Excerpt from setsockopt(2) man page: "SO_SETFIB can be used to over-ride the default FIB (routing table) for the given socket. The value must be from 0 to one less than the number returned from the sysctl net.fibs." and this contrasts with the code in /sys/kern/uipc_socket.c case SO_SETFIB : error = sooptcopyin (sopt, &optval, sizeof optval, sizeof optval); if (optval < 1 || optval > rt_numfibs ) { error = EINVAL ; goto bad; } Where as both Fib 0 and Fib 1 work fine if I use setfib() call. So, am confused if the code is incorrect or man page. I want to run the echo-client process against Fib 1, but selectively change it to Fib 0 for few connections. So, is this possible with the current freebsd multiple routing table (MRT)? Thanks, Naveen G. From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 10:31:17 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1432D1065679; Sat, 12 Feb 2011 10:31:17 +0000 (UTC) (envelope-from freebsd-hackers@herveybayaustralia.com.au) Received: from mail.unitedinsong.com.au (mail.unitedinsong.com.au [150.101.178.33]) by mx1.freebsd.org (Postfix) with ESMTP id B151F8FC17; Sat, 12 Feb 2011 10:31:16 +0000 (UTC) Received: from laptop1.herveybayaustralia.com.au (laptop1.herveybayaustralia.com.au [192.168.0.186]) by mail.unitedinsong.com.au (Postfix) with ESMTP id 698D55C44; Sat, 12 Feb 2011 20:38:22 +1000 (EST) Message-ID: <4D5660D9.2040002@herveybayaustralia.com.au> Date: Sat, 12 Feb 2011 20:28:41 +1000 From: Da Rock User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.16) Gecko/20110204 Thunderbird/3.0.11 ThunderBrowse/3.3.4 MIME-Version: 1.0 To: gljennjohn@googlemail.com References: <4D54E39D.1000505@herveybayaustralia.com.au> <4D54F0B0.7010503@freebsd.org> <4D550300.5090000@herveybayaustralia.com.au> <4D5565C7.1010809@freebsd.org> <4D55CE5A.8040902@herveybayaustralia.com.au> <4D55E015.3010709@freebsd.org> <20110212103923.3098f6b3@ernst.jennejohn.org> In-Reply-To: <20110212103923.3098f6b3@ernst.jennejohn.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, dudu@dudu.ro Subject: Re: [maybe spam] Re: linux PF_PACKET compatibility X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 10:31:17 -0000 On 02/12/11 19:39, Gary Jennejohn wrote: > On Fri, 11 Feb 2011 17:19:17 -0800 > Julian Elischer wrote: > > >> On 2/11/11 4:03 PM, Da Rock wrote: >> >>> Unfortunately this software uses this family instead of pcap or bpf. >>> So when built it errors. >>> >>> I guess if I am to use this app I will have to rewrite the way it >>> uses the network stack. >>> >> l2tp runs over UDP packets (port 1701 (like the starship enterprise)) >> I have no idea why they want raw packets. >> >> > Ther's a sendarp() routine which uses PF_PACKET to directly access the > network driver and bypass the stack. Lazy Linuxers who have no idea > or don't care that other operating systems exist. > > Indeed. Is it possible to leverage another compatible routine? I haven't had a look yet as I just read the message, but can I (after checking return values and arguments) just drop in another arp routine? Or are they simply incompatible across the board? From what I understand they should all be essentially doing the same thing, but I could be wrong on this. Alternatively would I have to basically rewrite the arp.c to be posix compatible (for portability)? Cheers From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 10:48:58 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DAB041065673 for ; Sat, 12 Feb 2011 10:48:58 +0000 (UTC) (envelope-from m.seaman@infracaninophile.co.uk) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3fd3:cd67:fafa:3d78]) by mx1.freebsd.org (Postfix) with ESMTP id 4875F8FC16 for ; Sat, 12 Feb 2011 10:48:58 +0000 (UTC) Received: from seedling.black-earth.co.uk (seedling.black-earth.co.uk [81.187.76.163]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.4/8.14.4) with ESMTP id p1CAmrTv090669 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 12 Feb 2011 10:48:54 GMT (envelope-from m.seaman@infracaninophile.co.uk) X-DKIM: Sendmail DKIM Filter v2.8.3 smtp.infracaninophile.co.uk p1CAmrTv090669 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=infracaninophile.co.uk; s=201001-infracaninophile; t=1297507734; bh=2XXnhw+rxSUooe3X3XQ2j3ppsk0KoA6efitYyD+sMzw=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Cc:Content-Type:Date:From:In-Reply-To: Message-ID:Mime-Version:References:To; z=Message-ID:=20<4D566585.7020802@infracaninophile.co.uk>|Date:=20S at,=2012=20Feb=202011=2010:48:37=20+0000|From:=20Matthew=20Seaman= 20|User-Agent:=20Mozilla/5.0=20(M acintosh=3B=20U=3B=20Intel=20Mac=20OS=20X=2010.6=3B=20en-US=3B=20r v:1.9.2.13)=20Gecko/20101207=20Thunderbird/3.1.7|MIME-Version:=201 .0|To:=20Devin=20Teske=20|CC:=20Ian=20Smith=20,=20freebsd-hackers@freebsd.org|Subject:=20Re: =20[RELEASE]=20host-setup(1):=20a=20dialog(1)-based=20utility=20fo r=20configuring=0D=0A=20FreeBSD|References:=20<20110211120031.9D37 510656E5@hub.freebsd.org>=09<20110212015155.I96449@sola.nimnet.asn .au>=20<1297447687.9144.6.camel@dt.vicor.com>|In-Reply-To:=20<1297 447687.9144.6.camel@dt.vicor.com>|X-Enigmail-Version:=201.1.1|Open PGP:=20id=3D60AE908C|Content-Type:=20multipart/signed=3B=20micalg= 3Dpgp-sha1=3B=0D=0A=20protocol=3D"application/pgp-signature"=3B=0D =0A=20boundary=3D"------------enig77090BAEC66020CB072AD5EF"; b=LwrsmrmtR0HFCXWjsREZzyIW7YPv3EtPVD3PLuF3YbGKE0xlhiOiAO+dmEXaN2jaK WRXq0SB9Gar9GKLjpWK5M4dRctOq6YpK67itPl02uTpW4Jw7NIcaMhq2dIrInYSbAH APitTYw2zJDnptPyNXwpYeBnAFLn0roGpLTvSVfQ= Message-ID: <4D566585.7020802@infracaninophile.co.uk> Date: Sat, 12 Feb 2011 10:48:37 +0000 From: Matthew Seaman User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Devin Teske References: <20110211120031.9D37510656E5@hub.freebsd.org> <20110212015155.I96449@sola.nimnet.asn.au> <1297447687.9144.6.camel@dt.vicor.com> In-Reply-To: <1297447687.9144.6.camel@dt.vicor.com> X-Enigmail-Version: 1.1.1 OpenPGP: id=60AE908C Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig77090BAEC66020CB072AD5EF" X-Virus-Scanned: clamav-milter 0.96.5 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,SPF_FAIL autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on lucid-nonsense.infracaninophile.co.uk Cc: freebsd-hackers@freebsd.org, Ian Smith Subject: Re: [RELEASE] host-setup(1): a dialog(1)-based utility for configuring FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 10:48:58 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig77090BAEC66020CB072AD5EF Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 11/02/2011 18:08, Devin Teske wrote: > $ time blen2netmask 26 > 255.255.255.192 >=20 > real 0m0.004s > user 0m0.001s > sys 0m0.004s >=20 > That's pretty fast, I'd say ^_^ (faster than the other implementations > -- especially considering that it doesn't have to fork anything). There are only 33 possible netmasks -- did you evaluate simply enumerating them all and simply looking up the result? Hmmm... blen2netmask() { local nbits=3D$1 case $nbits in 0) echo '0.0.0.0' ;; 1) echo '128.0.0.0' ;; 2) echo '192.0.0.0' ;; 3) echo '224.0.0.0' ;; 4) echo '240.0.0.0' ;; 5) echo '248.0.0.0' ;; 6) echo '252.0.0.0' ;; 7) echo '254.0.0.0' ;; 8) echo '255.0.0.0' ;; 9) echo '255.128.0.0' ;; 10) echo '255.192.0.0' ;; 11) echo '255.224.0.0' ;; 12) echo '255.240.0.0' ;; 13) echo '255.248.0.0' ;; 14) echo '255.252.0.0' ;; 15) echo '255.254.0.0' ;; 16) echo '255.255.0.0' ;; 17) echo '255.255.128.0' ;; 18) echo '255.255.192.0' ;; 19) echo '255.255.224.0' ;; 20) echo '255.255.240.0' ;; 21) echo '255.255.248.0' ;; 22) echo '255.255.252.0' ;; 23) echo '255.255.254.0' ;; 24) echo '255.255.255.0' ;; 25) echo '255.255.255.128' ;; 26) echo '255.255.255.192' ;; 27) echo '255.255.255.224' ;; 28) echo '255.255.255.240' ;; 29) echo '255.255.255.248' ;; 30) echo '255.255.255.252' ;; 31) echo '255.255.255.254' ;; 32) echo '255.255.255.255' ;; *) echo "$nbits -- not a valid IPv4 netmask length" return -1 ;; esac return 0 } Cheers, Matthew --=20 Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate JID: matthew@infracaninophile.co.uk Kent, CT11 9PW --------------enig77090BAEC66020CB072AD5EF Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.16 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk1WZZUACgkQ8Mjk52CukIygCgCggPxCCvtv757bdavBR0dZy8fW xo8AnRV9d+ChTfQ90G4v6ycvSTtCRQRE =vvKn -----END PGP SIGNATURE----- --------------enig77090BAEC66020CB072AD5EF-- From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 16:52:53 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 830C5106566C; Sat, 12 Feb 2011 16:52:53 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 210DA8FC17; Sat, 12 Feb 2011 16:52:52 +0000 (UTC) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.4/8.14.4/NETPLEX) with ESMTP id p1CGeQHY001098; Sat, 12 Feb 2011 11:40:26 -0500 X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.2.6 (mail.netplex.net [204.213.176.10]); Sat, 12 Feb 2011 11:40:26 -0500 (EST) Date: Sat, 12 Feb 2011 11:40:26 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Naveen Gujje In-Reply-To: Message-ID: References: MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="-559023410-1804928587-1297528826=:29788" Cc: freebsd-hackers@freebsd.org Subject: Re: SO_SETFIB socket option X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 16:52:53 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. ---559023410-1804928587-1297528826=:29788 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed On Sat, 12 Feb 2011, Naveen Gujje wrote: > Hi All, > > On my FreeBSD 7.2 box, I've two routing tables (FIBs). Fib 0 and Fib 1 > (net.fibs = 2). > > I have a simple echo client which is the counterpart of an echo server > running somewhere. > If I run this echo client against fib 0 as 'setfib 0 ./echo-client', it > properly uses Fib 0. > But, if I run this echo client against Fib 0 by using setsockopt & SO_SETFIB > option, setsockopt fails with EINVAL. > > setsockopt & SO_SETFIB for Fib 1 succeeds. But it fails for Fib 0. > > By looking at the man page and /sys/kern/uipc_socket.c [ snip ] > Where as both Fib 0 and Fib 1 work fine if I use setfib() call. Looks like the code is wrong. Have you tried patching the source to see if it works for you? Looks like you already know the fix, but here is a patch if you'd like to rebuild your kernel to see if it works. -- DE ---559023410-1804928587-1297528826=:29788 Content-Type: TEXT/PLAIN; charset=US-ASCII; name=uipc_socket.c.diffs Content-Transfer-Encoding: BASE64 Content-ID: Content-Description: Content-Disposition: attachment; filename=uipc_socket.c.diffs SW5kZXg6IHVpcGNfc29ja2V0LmMNCj09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0N Ci0tLSB1aXBjX3NvY2tldC5jCShyZXZpc2lvbiAyMTg0OTMpDQorKysgdWlw Y19zb2NrZXQuYwkod29ya2luZyBjb3B5KQ0KQEAgLTI0NDgsMTUgKzI0NDgs MTYgQEANCiAJCWNhc2UgU09fU0VURklCOg0KIAkJCWVycm9yID0gc29vcHRj b3B5aW4oc29wdCwgJm9wdHZhbCwgc2l6ZW9mIG9wdHZhbCwNCiAJCQkJCSAg ICBzaXplb2Ygb3B0dmFsKTsNCi0JCQlpZiAob3B0dmFsIDwgMSB8fCBvcHR2 YWwgPiBydF9udW1maWJzKSB7DQorCQkJaWYgKG9wdHZhbCA8IDAgfHwgb3B0 dmFsID4gcnRfbnVtZmlicykgew0KIAkJCQllcnJvciA9IEVJTlZBTDsNCiAJ CQkJZ290byBiYWQ7DQogCQkJfQ0KLQkJCWlmICgoc28tPnNvX3Byb3RvLT5w cl9kb21haW4tPmRvbV9mYW1pbHkgPT0gUEZfSU5FVCkgfHwNCi0JCQkgICAg KHNvLT5zb19wcm90by0+cHJfZG9tYWluLT5kb21fZmFtaWx5ID09IFBGX1JP VVRFKSkgew0KKwkJCWlmIChzby0+c29fcHJvdG8gIT0gTlVMTCAmJg0KKwkJ CSAgICgoc28tPnNvX3Byb3RvLT5wcl9kb21haW4tPmRvbV9mYW1pbHkgPT0g UEZfSU5FVCkgfHwNCisJCQkgICAoc28tPnNvX3Byb3RvLT5wcl9kb21haW4t PmRvbV9mYW1pbHkgPT0gUEZfUk9VVEUpKSkgew0KIAkJCQlzby0+c29fZmli bnVtID0gb3B0dmFsOw0KIAkJCQkvKiBOb3RlOiBpZ25vcmUgZXJyb3IgKi8N Ci0JCQkJaWYgKHNvLT5zb19wcm90byAmJiBzby0+c29fcHJvdG8tPnByX2N0 bG91dHB1dCkNCisJCQkJaWYgKHNvLT5zb19wcm90by0+cHJfY3Rsb3V0cHV0 KQ0KIAkJCQkJKCpzby0+c29fcHJvdG8tPnByX2N0bG91dHB1dCkoc28sIHNv cHQpOw0KIAkJCX0gZWxzZSB7DQogCQkJCXNvLT5zb19maWJudW0gPSAwOw0K ---559023410-1804928587-1297528826=:29788-- From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 16:56:03 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D7008106566B for ; Sat, 12 Feb 2011 16:56:03 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id 5EE5D8FC19 for ; Sat, 12 Feb 2011 16:56:02 +0000 (UTC) Received: from park.js.berklix.net (p5B22D64C.dip.t-dialin.net [91.34.214.76]) (authenticated bits=0) by tower.berklix.org (8.14.2/8.14.2) with ESMTP id p1CGttE1038618; Sat, 12 Feb 2011 16:55:56 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by park.js.berklix.net (8.13.8/8.13.8) with ESMTP id p1CGtn4C016594; Sat, 12 Feb 2011 17:55:49 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.3/8.14.3) with ESMTP id p1CGu8kC024460; Sat, 12 Feb 2011 17:56:13 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <201102121656.p1CGu8kC024460@fire.js.berklix.net> To: Bruce Cran From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Unix Linux Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Sat, 12 Feb 2011 01:01:15 GMT." <20110212010115.000066db@unknown> Date: Sat, 12 Feb 2011 17:56:08 +0100 Sender: jhs@berklix.com Cc: hackers@freebsd.org Subject: Re: memstick.img is bloated with 7% 2K blocks of nulls X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 16:56:03 -0000 Bruce Cran wrote: > On Sat, 12 Feb 2011 01:54:58 +0100 > "Julian H. Stacey" wrote: > > > -O filesystem-type > > Use 1 to specify that a UFS1 format file system be > > built; use 2 to specify that a UFS2 format file system be built. The > > default format is UFS2. > > If anyone fancies looking deeper, please do :-) > > I checked with dumpfs that memstick.img is UFS1. > > Also, mounting /dev/md0 confuses the kernel into ultimately panic'ing, > since /dev/md0a is the proper slice. I'm suprised it mounts. Not suprised it crashes. > For the mfsroot.gz file from the > CD ISOs: > > # mdconfig -a -f mfsroot > md0 > # mount /dev/md0a /mnt > # ls /mnt > ls: /mnt: Bad file descriptor > # cd /mnt > cd: /mnt: Not a directory > # vim /mnt > > panic: ffs_read: type 0 It's not so obvious but in man mdconfig, there's no "[]" around "-t type", I read that as "-t something" is mandatory, (though it starts without for you ... & I suspect -t default is malloc, though manual doesnt say that, but look what manual says re. malloc ... panic ). I use "-t vnode", which doesn't crash here on 8.1-RELEASE i686: mdconfig -a -t vnode -f FreeBSD-8.2-RC3-amd64-memstick.img mount /dev/md0a /mnt cd /mnt tar cf - . | ( cd /pri/FreeBSD/releases/amd64/ISO-IMAGES/8.2\ /FreeBSD-8.2-RC3-amd64-memstick && tar xf - ) dumpfs /dev/md0a # UFS1 mdconfig -a -t vnode -f FreeBSD-8.2-RC3-amd64-livefs.iso mount -t cd9660 /dev/md1 /mnt1 cd /mnt1 tar cf - . | ( cd /pri/FreeBSD/releases/amd64/ISO-IMAGES/8.2\ /FreeBSD-8.2-RC3-amd64-livefs && tar xf - ) mdconfig -a -t vnode -f FreeBSD-8.2-RC3-amd64-disc1.iso xs mount -t cd9660 /dev/md2 /mnt2 cd /mnt2 tar cf - . | ( cd /pri/FreeBSD/releases/amd64/ISO-IMAGES/8.2\ /FreeBSD-8.2-RC3-amd64-disc1 && tar xf - ) cd / ; umount /mnt2 ; umount /mnt1 ; umount /mnt mdconfig -d -u 2 ; mdconfig -d -u 1 ; mdconfig -d -u 0 Analysing sizes: ls -l 723834880 FreeBSD-8.2-RC3-amd64-disc1.iso 348796928 FreeBSD-8.2-RC3-amd64-livefs.iso 1087467520 FreeBSD-8.2-RC3-amd64-memstick.img cd /pri/FreeBSD/releases/amd64/ISO-IMAGES/8.2 ; du -s -k /mnt* * 988708 /mnt # memstick.img 985807 /mnt1 # livefs.iso 705444 /mnt2 # disc1.iso 999340 FreeBSD-8.2-RC3-amd64-memstick 1023032 FreeBSD-8.2-RC3-amd64-livefs 711640 FreeBSD-8.2-RC3-amd64-disc1 df /mnt Filesystem Size Used Avail Capacity Mounted on /dev/md0a 1.1G 1.0G -14M 101% /mnt umount /mnt ; tunefs -m 0 /dev/md0a tunefs: minimum percentage of free space changes from 8% to 0% tunefs: should optimize for space with minfree < 8% df /dev/md0a Filesystem Size Used Avail Capacity Mounted on /dev/md0a 1.1G 1.0G 73M 93% cd /pri/FreeBSD/releases/amd64/ISO-IMAGES/8.2 ; /bin/pwd mount | grep /usra /dev/ad4s4e on /usra (ufs, NFS exported, local, soft-updates) dumpfs /dev/ad4s4e | head -1 # UFS2 Note du on UFS1 stick (988708) is near the same as tree copied by tar to UFS2 hard disc (999340), so its not the differerence between UFS1 & UFS2 that wastes the space (though good guess & thanks for suggesting it pushed me to investigate more :-) We are distributing 7% null space: Mainly because the file system is too big. We probably don't need 7% free space on stick just to install or repair, though that might be useful (if slow) for demos ) + Also each file will have unused bytes at end, (but so will livefs.iso have padding per file & not near so bloated, albeit probably some other fragment size. Per my first post: > > 2K blocks of nulls. > > FreeBSD-8.2-RC3-i386-disc1.iso: 565 > > FreeBSD-8.2-RC3-i386-livefs.iso: 905 > > FreeBSD-8.2-RC3-i386-memstick.img: 34521 As a sample test of what makefs produces: makefs /tmp/bla /usr/share Extent size set to 8192 block size 8192, fragment size 1024 du -s -k /usr/share # 54 108 ls -l /tmp/bla # 54 927 360 df /tmp/bla Filesystem Size Used Avail Capacity Mounted on /dev/ad0s2e 520M 55M 423M 12% /tmp # PS Note df lies 520M - some bug as not a node ? mdconfig -a -t vnode -f /tmp/bla # md2 df /dev/md2* Filesystem 1K-blocks Used Avail Capacity /dev/md2 51175 48723 -1642 103% tunefs -m 2 /dev/md2 minimum percentage of free space changes from 8% to 2% df /dev/md2* Filesystem 1K-blocks Used Avail Capacity /dev/md2 51175 48723 1429 97% Conclusion: makefs leaves 2+3=5% here. memstick.img is created by /usr/src/release/scripts/make-memstick.sh which (as above test) at present calls makefs without any tuning parameters Maybe t should have some parameters ? eg: makefs -m 1030000000 -b 1% ${tempfile} ${1} Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Mail plain text; Not quoted-printable, Not HTML, Not base 64. Reply below text sections not at top, to avoid breaking cumulative context. From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 17:05:54 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 16AC81065672 for ; Sat, 12 Feb 2011 17:05:54 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (unknown [IPv6:2a01:348:0:15:5d59:5c40:0:1]) by mx1.freebsd.org (Postfix) with ESMTP id 94BD78FC1D for ; Sat, 12 Feb 2011 17:05:53 +0000 (UTC) Received: from muon.cran.org.uk (localhost [127.0.0.1]) by muon.cran.org.uk (Postfix) with ESMTP id 46707E8269; Sat, 12 Feb 2011 17:05:48 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=cran.org.uk; h=date:from :to:cc:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; s=mail; bh=Dmu2GqM76A+z SyAucWpSKKoxAPI=; b=Hy3+NMQAx0yNFB2ZosyNJR9/o3GaBiclk24ZhMIexktY qPHpn1+aGbxbxjuAAyEERpmdEtuhBVm6UYoTE4UCo3TOFPGhtbxcdkfUpOHnpoWo 2lfIwmwVfKRCoWOmQ1FJJduPYvL/7PXLHe+Ax7KdMYsPsgaJaFPf2z8KPlywwoo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=cran.org.uk; h=date:from:to :cc:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; q=dns; s=mail; b=yzsuo6 RFTo575u+pU7Jb+/9mjVmNoWinatgIa6O1iF3Kww0Zl9z0eCiWO13NcTqjqp/G3I JECpcKuHdNDOJ+O0XW/TLClel5lHLlgynhow/VKgB0G33IDNHuVM5pDi6aScW3uL PMVg1mXaGdeXdewYq4kbrdGmW44eNatKjk7W8= Received: from unknown (client-86-31-165-87.oxfd.adsl.virginmedia.com [86.31.165.87]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA id EB99EE8208; Sat, 12 Feb 2011 17:05:47 +0000 (GMT) Date: Sat, 12 Feb 2011 17:05:26 +0000 From: Bruce Cran To: "Julian H. Stacey" Message-ID: <20110212170526.00004cab@unknown> In-Reply-To: <201102121656.p1CGu8kC024460@fire.js.berklix.net> References: <20110212010115.000066db@unknown> <201102121656.p1CGu8kC024460@fire.js.berklix.net> X-Mailer: Claws Mail 3.7.8cvs9 (GTK+ 2.16.6; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: hackers@freebsd.org Subject: Re: memstick.img is bloated with 7% 2K blocks of nulls X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 17:05:54 -0000 On Sat, 12 Feb 2011 17:56:08 +0100 "Julian H. Stacey" wrote: > It's not so obvious but in man mdconfig, there's no "[]" around "-t > type", I read that as "-t something" is mandatory, (though it starts > without for you ... & I suspect -t default is malloc, though manual > doesnt say that, but look what manual says re. malloc ... panic ). But from the manual page: -f file Filename to use for the vnode type memory disk. Options -a and -t vnode are implied if not specified. So if you specify -f then you get -t vnode automatically. -- Bruce Cran From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 14:14:53 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7F6E6106564A; Sat, 12 Feb 2011 14:14:53 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 296598FC15; Sat, 12 Feb 2011 14:14:52 +0000 (UTC) Received: from outgoing.leidinger.net (p57B3AEEE.dip.t-dialin.net [87.179.174.238]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 7CE34844012; Sat, 12 Feb 2011 15:14:47 +0100 (CET) Received: from unknown (IO.Leidinger.net [192.168.2.110]) by outgoing.leidinger.net (Postfix) with ESMTP id C85272084; Sat, 12 Feb 2011 15:14:43 +0100 (CET) Date: Sat, 12 Feb 2011 15:14:42 +0100 From: Alexander Leidinger To: Robert Watson Message-ID: <20110212151442.000016bb@unknown> In-Reply-To: References: <20110211103028.12684f54yrw8tgqo@webmail.leidinger.net> X-Mailer: Claws Mail 3.7.6 (GTK+ 2.16.0; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 7CE34844012.A5BC2 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-0.4, required 6, autolearn=disabled, ALL_TRUSTED -1.00, J_CHICKENPOX_73 0.60) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1298124889.73498@AIz5j75uNge7x9S3g7G5pQ X-EBL-Spam-Status: No X-Mailman-Approved-At: Sat, 12 Feb 2011 18:08:15 +0000 Cc: hackers@freebsd.org, kibab@freebsd.org Subject: Re: CFR: FEATURE macros for AUDIT/CAM/IPC/KTR/MAC/NFS/NTP/PMC/SYSV/... X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 14:14:53 -0000 On Sat, 12 Feb 2011 00:52:48 +0000 (GMT) Robert Watson wrote: > The one comment I'd make is that the MAC case should indicate that > "The MAC Framework" is supported, rather than mandatory access > controls being present -- the presence of the framework doesn't imply > the presence of mandatory access control policies. Does FEATURE(mac, "Mandatory Access Control Framework support"); look better? Alternatively/additionally we could use mac_framework as the name of the feature. Bye, Alexander. From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 14:19:27 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0869E106564A; Sat, 12 Feb 2011 14:19:27 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id B74268FC0C; Sat, 12 Feb 2011 14:19:26 +0000 (UTC) Received: from outgoing.leidinger.net (p57B3AEEE.dip.t-dialin.net [87.179.174.238]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 691B9844012; Sat, 12 Feb 2011 15:19:22 +0100 (CET) Received: from unknown (IO.Leidinger.net [192.168.2.110]) by outgoing.leidinger.net (Postfix) with ESMTP id 9C1622085; Sat, 12 Feb 2011 15:19:19 +0100 (CET) Date: Sat, 12 Feb 2011 15:19:18 +0100 From: Alexander Leidinger To: Anonymous Message-ID: <20110212151918.00007489@unknown> In-Reply-To: <86zkq27ci9.fsf@gmail.com> References: <20110211103028.12684f54yrw8tgqo@webmail.leidinger.net> <86zkq27ci9.fsf@gmail.com> X-Mailer: Claws Mail 3.7.6 (GTK+ 2.16.0; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 691B9844012.A5A79 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1, required 6, autolearn=disabled, ALL_TRUSTED -1.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1298125163.89778@n9RUCpzgYdseYWLf1isOKQ X-EBL-Spam-Status: No X-Mailman-Approved-At: Sat, 12 Feb 2011 18:08:25 +0000 Cc: hackers@freebsd.org, kibab@freebsd.org Subject: Re: CFR: FEATURE macros for AUDIT/CAM/IPC/KTR/MAC/NFS/NTP/PMC/SYSV/... X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 14:19:27 -0000 On Fri, 11 Feb 2011 19:38:06 +0300 Anonymous wrote: > Alexander Leidinger writes: > > > Hi, > > > > during the last GSoC various FEATURE macros where added to the > > system. Before committing them, I would like to get some review > > (like if macro is in the correct file, and for those FEATURES where > > the description was not taken from NOTES if the description is OK). > > > > If nobody complains, I would like to commit this in 1-2 weeks. If > > you need more time to review, just tell me. > [...] > > Index: kern/kern_dtrace.c > [...] > > +FEATURE(kdtrace_hooks, > > + "Kernel DTrace hooks which are required to load DTrace kernel > > modules"); + > [...] > > Index: kern/kern_lockstat.c > [...] > > +FEATURE(kdtrace_hooks, "Kernel DTRACE hooks"); > > Why description differs? Good catch. I removed the second one completely in my working copy. Bye, Alexander. From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 19:09:00 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F40191065672; Sat, 12 Feb 2011 19:08:59 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id CEBC18FC0A; Sat, 12 Feb 2011 19:08:59 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 66E4246B23; Sat, 12 Feb 2011 14:08:59 -0500 (EST) Date: Sat, 12 Feb 2011 19:08:59 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Alexander Leidinger In-Reply-To: <20110212151442.000016bb@unknown> Message-ID: References: <20110211103028.12684f54yrw8tgqo@webmail.leidinger.net> <20110212151442.000016bb@unknown> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: hackers@freebsd.org, kibab@freebsd.org Subject: Re: CFR: FEATURE macros for AUDIT/CAM/IPC/KTR/MAC/NFS/NTP/PMC/SYSV/... X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 19:09:00 -0000 On Sat, 12 Feb 2011, Alexander Leidinger wrote: > On Sat, 12 Feb 2011 00:52:48 +0000 (GMT) Robert Watson > wrote: > >> The one comment I'd make is that the MAC case should indicate that "The MAC >> Framework" is supported, rather than mandatory access controls being >> present -- the presence of the framework doesn't imply the presence of >> mandatory access control policies. > > Does > FEATURE(mac, "Mandatory Access Control Framework support"); > look better? > > Alternatively/additionally we could use mac_framework as the name of the > feature. The above seems fine -- while I've been moving to names like mac_framework.h, it's still "options MAC" and "security/mac", etc, and think that "mac" is the most consistent options. Robert From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 19:24:24 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7A3C8106564A for ; Sat, 12 Feb 2011 19:24:24 +0000 (UTC) (envelope-from tim@kientzle.com) Received: from mail-pv0-f182.google.com (mail-pv0-f182.google.com [74.125.83.182]) by mx1.freebsd.org (Postfix) with ESMTP id 59A488FC14 for ; Sat, 12 Feb 2011 19:24:24 +0000 (UTC) Received: by pvc22 with SMTP id 22so671297pvc.13 for ; Sat, 12 Feb 2011 11:24:23 -0800 (PST) Received: by 10.142.210.8 with SMTP id i8mr1511714wfg.438.1297537048334; Sat, 12 Feb 2011 10:57:28 -0800 (PST) Received: from [10.123.2.181] (99-74-169-43.lightspeed.sntcca.sbcglobal.net [99.74.169.43]) by mx.google.com with ESMTPS id w42sm1189724wfh.15.2011.02.12.10.57.26 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 12 Feb 2011 10:57:27 -0800 (PST) Mime-Version: 1.0 (Apple Message framework v1082) Content-Type: text/plain; charset=us-ascii From: Tim Kientzle In-Reply-To: <201102111909.p1BJ9UAE097045@fire.js.berklix.net> Date: Sat, 12 Feb 2011 10:57:23 -0800 Content-Transfer-Encoding: 7bit Message-Id: <66758C9D-DCE2-4381-A4B1-956A48423CDD@kientzle.com> References: <201102111909.p1BJ9UAE097045@fire.js.berklix.net> To: "Julian H. Stacey" X-Mailer: Apple Mail (2.1082) Cc: hackers@freebsd.org Subject: Re: memstick.img is bloated with 7% 2K blocks of nulls X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 19:24:24 -0000 On Feb 11, 2011, at 11:09 AM, Julian H. Stacey wrote: > memstick.img wastes 7% with 2K blocks of nulls. > shown by: > 8f -b 0 -n 2048 -l -f Fr* > http://berklix.com/~jhs/src/bsd/jhs/bin/public/8f/ 8f.c 8f.1 > > ... > The CD & DVD images are not nearly so wasteful, see above. > As near 1G ( 959467520 FreeBSD-8.2-RC3-i386-memstick.img ) it will > soon not fit on 1G sticks. There is a big difference between laying out a read-only filesystem and laying out a read-write disk image. CD/DVD writers do pack file contents one after the other on the image. The current UFS code is designed to leave enough "slack space" to support future file writes. The strategy used by libarchive's recent ISO writer is to concatenate the file bodies into a temp file (with minimal padding between entries to meet alignment requirements) while storing directory information in memory. The final output then consists of the directory information followed by the concatenated file bodies. I suspect a similar strategy could be used to lay out and write a read-only optimized UFS image. A few folks have asked about a UFS writer for libarchive; I think it's probably feasible but I doubt very much of the existing UFS code can be recycled for such a project. Alternatively, of course, is there any way to use isofs instead of ufs for memstick.img? Tim From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 20:16:03 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A6465106566C; Sat, 12 Feb 2011 20:16:03 +0000 (UTC) (envelope-from dudu@dudu.ro) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1AE358FC08; Sat, 12 Feb 2011 20:16:02 +0000 (UTC) Received: by wyf19 with SMTP id 19so3597497wyf.13 for ; Sat, 12 Feb 2011 12:16:02 -0800 (PST) Received: by 10.216.173.147 with SMTP id v19mr1964308wel.102.1297541761529; Sat, 12 Feb 2011 12:16:01 -0800 (PST) MIME-Version: 1.0 Received: by 10.216.185.9 with HTTP; Sat, 12 Feb 2011 12:15:21 -0800 (PST) In-Reply-To: <4D5660D9.2040002@herveybayaustralia.com.au> References: <4D54E39D.1000505@herveybayaustralia.com.au> <4D54F0B0.7010503@freebsd.org> <4D550300.5090000@herveybayaustralia.com.au> <4D5565C7.1010809@freebsd.org> <4D55CE5A.8040902@herveybayaustralia.com.au> <4D55E015.3010709@freebsd.org> <20110212103923.3098f6b3@ernst.jennejohn.org> <4D5660D9.2040002@herveybayaustralia.com.au> From: Vlad Galu Date: Sat, 12 Feb 2011 22:15:21 +0200 Message-ID: To: Da Rock Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: Re: [maybe spam] Re: linux PF_PACKET compatibility X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 20:16:03 -0000 On Sat, Feb 12, 2011 at 12:28 PM, Da Rock < freebsd-hackers@herveybayaustralia.com.au> wrote: > On 02/12/11 19:39, Gary Jennejohn wrote: > >> On Fri, 11 Feb 2011 17:19:17 -0800 >> Julian Elischer wrote: >> >> >> >>> On 2/11/11 4:03 PM, Da Rock wrote: >>> >>> >>>> Unfortunately this software uses this family instead of pcap or bpf. >>>> So when built it errors. >>>> >>>> I guess if I am to use this app I will have to rewrite the way it >>>> uses the network stack. >>>> >>>> >>> l2tp runs over UDP packets (port 1701 (like the starship enterprise)) >>> I have no idea why they want raw packets. >>> >>> >>> >> Ther's a sendarp() routine which uses PF_PACKET to directly access the >> network driver and bypass the stack. Lazy Linuxers who have no idea >> or don't care that other operating systems exist. >> >> >> > Indeed. Is it possible to leverage another compatible routine? I haven't > had a look yet as I just read the message, but can I (after checking return > values and arguments) just drop in another arp routine? Or are they simply > incompatible across the board? > > From what I understand they should all be essentially doing the same thing, > but I could be wrong on this. Alternatively would I have to basically > rewrite the arp.c to be posix compatible (for portability)? > > Cheers > You only need to rewrite the sendarp() routine, using a BPF device descriptor instead of the PF_PACKET socket descriptor. Take a look at libdnet[1] and rbootd[2]'s source. [1] http://libdnet.sf.net/ [2] http://ftp.fr.openbsd.org/pub/OpenBSD/src/usr.sbin/rbootd/ You will need, however, to fill the source with #ifdefs to compensate the fact that Linux has assigned different names and sizes to the members of struct ether_header and arphdr (and has a _BSD_SOURCE knob to accomodate compiling BSD-based software).* * -- Good, fast & cheap. Pick any two. From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 20:49:50 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6742106566B; Sat, 12 Feb 2011 20:49:50 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id 982E38FC1C; Sat, 12 Feb 2011 20:49:50 +0000 (UTC) Received: from julian-mac.elischer.org (home-nat.elischer.org [67.100.89.137]) (authenticated bits=0) by vps1.elischer.org (8.14.4/8.14.4) with ESMTP id p1CKnmLo078500 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 12 Feb 2011 12:49:48 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <4D56F278.1060801@freebsd.org> Date: Sat, 12 Feb 2011 12:50:00 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Daniel Eischen References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Naveen Gujje Subject: Re: SO_SETFIB socket option X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 20:49:50 -0000 On 2/12/11 8:40 AM, Daniel Eischen wrote: > On Sat, 12 Feb 2011, Naveen Gujje wrote: > >> Hi All, >> >> On my FreeBSD 7.2 box, I've two routing tables (FIBs). Fib 0 and Fib 1 >> (net.fibs = 2). >> >> I have a simple echo client which is the counterpart of an echo server >> running somewhere. >> If I run this echo client against fib 0 as 'setfib 0 >> ./echo-client', it >> properly uses Fib 0. >> But, if I run this echo client against Fib 0 by using setsockopt & >> SO_SETFIB >> option, setsockopt fails with EINVAL. >> >> setsockopt & SO_SETFIB for Fib 1 succeeds. But it fails for Fib 0. >> >> By looking at the man page and /sys/kern/uipc_socket.c > > [ snip ] > >> Where as both Fib 0 and Fib 1 work fine if I use setfib() call. > > Looks like the code is wrong. Have you tried patching the source > to see if it works for you? Looks like you already know the fix, > but here is a patch if you'd like to rebuild your kernel to see > if it works. > yeah looks like a braino on my part.. I probably only tested by going UP from fib0 to fib 1 and not teh other way around. From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 12 22:50:57 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 32BB6106566C for ; Sat, 12 Feb 2011 22:50:57 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) by mx1.freebsd.org (Postfix) with ESMTP id E68518FC14 for ; Sat, 12 Feb 2011 22:50:56 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.4/8.14.4) with ESMTP id p1CMFGHs003928; Sat, 12 Feb 2011 15:15:16 -0700 (MST) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.4/8.14.4/Submit) with ESMTP id p1CMFFxI003925; Sat, 12 Feb 2011 15:15:16 -0700 (MST) (envelope-from wblock@wonkity.com) Date: Sat, 12 Feb 2011 15:15:15 -0700 (MST) From: Warren Block To: Tim Kientzle In-Reply-To: <66758C9D-DCE2-4381-A4B1-956A48423CDD@kientzle.com> Message-ID: References: <201102111909.p1BJ9UAE097045@fire.js.berklix.net> <66758C9D-DCE2-4381-A4B1-956A48423CDD@kientzle.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.6 (wonkity.com [127.0.0.1]); Sat, 12 Feb 2011 15:15:16 -0700 (MST) Cc: "Julian H. Stacey" , hackers@freebsd.org Subject: Re: memstick.img is bloated with 7% 2K blocks of nulls X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 22:50:57 -0000 On Sat, 12 Feb 2011, Tim Kientzle wrote: > Alternatively, of course, is there any way to use > isofs instead of ufs for memstick.img? Devin Teske's DruidBSD (http://druidbsd.sourceforge.net/) uses the same image for CD, hard disk, or memstick. I don't know the technical details. (VirtualBox didn't like it, but maybe it's a trivial fix.)