From owner-freebsd-hackers@FreeBSD.ORG Sun Jul 27 23:01:16 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5DB85BD4; Sun, 27 Jul 2014 23:01:16 +0000 (UTC) Received: from mail-wi0-x231.google.com (mail-wi0-x231.google.com [IPv6:2a00:1450:400c:c05::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C832321D0; Sun, 27 Jul 2014 23:01:15 +0000 (UTC) Received: by mail-wi0-f177.google.com with SMTP id ho1so3460097wib.4 for ; Sun, 27 Jul 2014 16:01:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=uZ7PsdhBTP40iUYalIV3uytvS1S9dSjU1eBF1Y0IK80=; b=RM7TR8Lxldggbp3zX2BONQKLKZZ6f+mJitvzVq1AIjMkDhPZr+G2eisB73VB/T/xGe j/tI5Bq8ytyoSWrWIDjOE/mDikgTAwsf/Tvpi8tQYKYwmRKS1K9acR8/eZkS3Ricza85 TnAokkSrWvDAuWI0D38m056VR1rx6Bhby1UtVEbJP6zYDCxlzT48dPqCE7htWLSgfuSw zf7faYhaNTYKmyGU26tdlNprU1Q5yxuGKPxXLlHEhoXaHtfr3uPYF2G02o5/PdcAMoE2 gA3Yphfb/l11bSzSm/ZN0VPz366DKvu1b+MKTj+i68fw8wJbTa6ktMYj+seMbitBF1JN RE6w== X-Received: by 10.180.39.34 with SMTP id m2mr24056526wik.80.1406502073980; Sun, 27 Jul 2014 16:01:13 -0700 (PDT) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by mx.google.com with ESMTPSA id ex2sm24211194wic.24.2014.07.27.16.01.12 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jul 2014 16:01:13 -0700 (PDT) Sender: Baptiste Daroussin Date: Mon, 28 Jul 2014 01:01:10 +0200 From: Baptiste Daroussin To: arch@FreeBSD.org, hackers@FreeBSD.org Subject: Add ohash (OpenBSD hash) into libutil Message-ID: <20140727230110.GI50802@ivaldir.etoilebsd.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZY5CS28jBCfb727c" Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Jul 2014 23:01:16 -0000 --ZY5CS28jBCfb727c Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, If noone objects I would like to bring ohash from OpenBSD into our libutil. Ohash is already in base (usr.bin/m4/ohash) having it into libutil will increase compatibility with OpenBSD as well as allowing the rest of the base system to be able to benefit from ohash implementation. regards, Bapt --ZY5CS28jBCfb727c Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlPVhLYACgkQ8kTtMUmk6ExTPwCdE4dr6/QSrXummgrQPNDYwYWd 3jEAoL19AKYlgDDe1bKupWA3T3DpmuSO =fK29 -----END PGP SIGNATURE----- --ZY5CS28jBCfb727c-- From owner-freebsd-hackers@FreeBSD.ORG Mon Jul 28 17:40:12 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F107AC7A for ; Mon, 28 Jul 2014 17:40:11 +0000 (UTC) Received: from mail-vc0-x22d.google.com (mail-vc0-x22d.google.com [IPv6:2607:f8b0:400c:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B32F82901 for ; Mon, 28 Jul 2014 17:40:11 +0000 (UTC) Received: by mail-vc0-f173.google.com with SMTP id hy10so11659215vcb.4 for ; Mon, 28 Jul 2014 10:40:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=ImHtG9xVk6PnzNm3hwEDAR7FwU3g5rj3vqmFAAssy/k=; b=GArCPEvMG8yj8is+CQwWzpWHgef2nHv+KYz90JpGcETHfWhO4n6L8u8WxTz9rR02NK P9lIc7Vl9CFlTzZh2SBEPZkNH9Ab07APqMrOznsafs2nfaQAqGthbfeF17pPkxMSeTiF rTkPuy8kPiuTvcY0bmbUj7r5uHIoF2wZ50DYyGQM+aFeTUWdoVxLNcwQukVDUPADrrk+ XVuvo3rzvjmLMY2ExH/g5ZDInDeLpD/QTo4h/9LZ9Mtfve0NeHvSCYQhNZgMdzJBpUqw 04gElgTz5YUYMINZMnT49GJCgVIB0yEElt2/3wG+UB1ULcep8EwfTbCgbKEIAiAhMJKd s64g== MIME-Version: 1.0 X-Received: by 10.52.147.15 with SMTP id tg15mr8742488vdb.53.1406569210737; Mon, 28 Jul 2014 10:40:10 -0700 (PDT) Received: by 10.220.192.68 with HTTP; Mon, 28 Jul 2014 10:40:10 -0700 (PDT) Date: Mon, 28 Jul 2014 13:40:10 -0400 Message-ID: Subject: Allocation/free history From: suresh gumpula To: freebsd-hackers@freebsd.org X-Mailman-Approved-At: Mon, 28 Jul 2014 19:10:34 +0000 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jul 2014 17:40:12 -0000 Hi, Knowing the PC of an allocation is very usefull in debugging. Having the PC hash table and storing the pc hash either with an object itself( at the end) or allocate an exra structure to hold the hash index help us find out who/where an object was allocated. We already have something like this in our own operating system and has been a useful thing in debugging. BSD allocator uses power of 2, so storing at the end of an object might be wasting lot of memory with large objects. It appears we don=E2=80=99t have something like this in current FBSD codeli= ne and would like to work on this ? Any comments please? It would be something like below. 8 bytes at the end of each object has fecepost which is usefull in finding overwrites and 2 hash indices to the PC table to track allocation history. (kgdb-amd64-7.4-87) x/40w 0xffffff153728b038 0xffffff153728b038: 0xf6970a05 0x06cb7e0c 0x305a134a 0x831c18bb 0xc0000bed 0x134a2115 0xffffff153728b050: 0x85687ef8 0xffffffff 0x00000001 0x83192201 0xc0000bed 0x1741143b Thanks Suresh From owner-freebsd-hackers@FreeBSD.ORG Mon Jul 28 19:20:02 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1217A381 for ; Mon, 28 Jul 2014 19:20:02 +0000 (UTC) Received: from mail-wg0-x22a.google.com (mail-wg0-x22a.google.com [IPv6:2a00:1450:400c:c00::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9BE782653 for ; Mon, 28 Jul 2014 19:20:01 +0000 (UTC) Received: by mail-wg0-f42.google.com with SMTP id l18so7746652wgh.25 for ; Mon, 28 Jul 2014 12:19:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-type:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=U6mC3Bjqx2uCqAmWvbeHB7Cof8k57z9WsIHQpEAdDwY=; b=b+NfJSbTZSO1PXX0gk+nDTXom4QEWex/3QP3o2Xj9IHJdQwkU+oKkjP2S2l27ZVnnC 14pHcyv89LWL7c6LAfPYpajHZIt3YxbIE+rUh9tbwfoC9YI9liqDcDPbQzQ6fmCy5C8U 6xAEyIWfPWj0tObDS4vl3/55f/6FcdwCrY9p2kILmg38N4OvLmfH6qRG7BuPLZBmE/No AEnWrXElf6fSyDyghhxyamMSYC7ksym1t5nCUzIj+sfkXsGWLQJ4QjYpB4pACLfk+qBo 2ech1tpmap+uz+T5WFU6jB0Yop4fq1IN7fH/Eje+zCbekXjCJG2vLUYhymzsxVlFFZVc zoVQ== X-Received: by 10.180.95.66 with SMTP id di2mr28698051wib.60.1406575196254; Mon, 28 Jul 2014 12:19:56 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id sa4sm51580317wjb.45.2014.07.28.12.19.54 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Mon, 28 Jul 2014 12:19:55 -0700 (PDT) Date: Mon, 28 Jul 2014 21:19:51 +0200 From: Mateusz Guzik To: suresh gumpula Subject: Re: Allocation/free history Message-ID: <20140728191950.GA9781@dft-labs.eu> Mail-Followup-To: Mateusz Guzik , suresh gumpula , freebsd-hackers@freebsd.org References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jul 2014 19:20:02 -0000 On Mon, Jul 28, 2014 at 01:40:10PM -0400, suresh gumpula wrote: > Hi, > Knowing the PC of an allocation is very usefull in debugging. Having the > PC hash table and storing the pc hash either with an object itself( at the > end) or allocate an exra structure to hold the > hash index help us find out who/where an object was allocated. We > already have something like this in our own operating system and has been a > useful thing in debugging. > BSD allocator uses power of 2, so storing at the end of an object might be > wasting lot of memory with large objects. > > It appears we don’t have something like this in current FBSD codeline and > would like to work on this ? Any comments please? > > > It would be something like below. 8 bytes at the end of each object has > fecepost which is usefull in finding overwrites and 2 hash indices to the > PC table to track allocation history. > (kgdb-amd64-7.4-87) x/40w 0xffffff153728b038 > 0xffffff153728b038: 0xf6970a05 0x06cb7e0c 0x305a134a > 0x831c18bb > 0xc0000bed 0x134a2115 > > 0xffffff153728b050: 0x85687ef8 0xffffffff 0x00000001 > 0x83192201 > 0xc0000bed 0x1741143b > See redzone(9). -- Mateusz Guzik From owner-freebsd-hackers@FreeBSD.ORG Mon Jul 28 19:20:25 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A1E764EC for ; Mon, 28 Jul 2014 19:20:25 +0000 (UTC) Received: from mail-qa0-x232.google.com (mail-qa0-x232.google.com [IPv6:2607:f8b0:400d:c00::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 62DAC265E for ; Mon, 28 Jul 2014 19:20:25 +0000 (UTC) Received: by mail-qa0-f50.google.com with SMTP id s7so8343475qap.37 for ; Mon, 28 Jul 2014 12:20:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=MXRftp7KiG7FZIbshw+B84EBbgvQbUV3jp7YIAu73Pk=; b=E/fHS7Sx9j63vFmXJOZTo/pVoQU9lneTgdJ0uHpMHV2hjswlf69DmHXI/QOdmljeWT H/SXTIntAxFkyUVvwonQKUE2oIN0PJMnuWBpODsLtqpEb/QDAgPH0ZDQTmCXJVVbOHk1 xHIlGmYgVVejnn5/KqoBUdBK4osGY3Fesn5pvO/NI8uABR+p8XrerrOMdVhZpVsG7dBb oBWCUI67c28ArmNWbI0zsiSNp0bLlHhhghCgEq4fs+20FhG5Avbik0pe+G929SDTqA24 gfHLE2UWUZ4Q2k03YRlcOk/u5yayc9ZXpmDlhT7PqR/d+B5BCZq5fNEe0dqoPkNPLOPp ZNug== MIME-Version: 1.0 X-Received: by 10.224.71.198 with SMTP id i6mr63568561qaj.76.1406575224489; Mon, 28 Jul 2014 12:20:24 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.1.6 with HTTP; Mon, 28 Jul 2014 12:20:24 -0700 (PDT) In-Reply-To: References: Date: Mon, 28 Jul 2014 12:20:24 -0700 X-Google-Sender-Auth: D-WwL3xhjUncZw2VLrajKMQV2mc Message-ID: Subject: Re: Allocation/free history From: Adrian Chadd To: suresh gumpula Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jul 2014 19:20:25 -0000 Hi! Are you talking about userland? kernel? both? -a On 28 July 2014 10:40, suresh gumpula wrote: > Hi, > Knowing the PC of an allocation is very usefull in debugging. Having t= he > PC hash table and storing the pc hash either with an object itself( at t= he > end) or allocate an exra structure to hold the > hash index help us find out who/where an object was allocated. We > already have something like this in our own operating system and has been= a > useful thing in debugging. > BSD allocator uses power of 2, so storing at the end of an object might b= e > wasting lot of memory with large objects. > > It appears we don=E2=80=99t have something like this in current FBSD code= line and > would like to work on this ? Any comments please? > > > It would be something like below. 8 bytes at the end of each object has > fecepost which is usefull in finding overwrites and 2 hash indices to the > PC table to track allocation history. > (kgdb-amd64-7.4-87) x/40w 0xffffff153728b038 > 0xffffff153728b038: 0xf6970a05 0x06cb7e0c 0x305a134a > 0x831c18bb > 0xc0000bed 0x134a2115 > > 0xffffff153728b050: 0x85687ef8 0xffffffff 0x00000001 > 0x83192201 > 0xc0000bed 0x1741143b > > Thanks > Suresh > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " From owner-freebsd-hackers@FreeBSD.ORG Mon Jul 28 19:35:44 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CEAA7BCF; Mon, 28 Jul 2014 19:35:44 +0000 (UTC) Received: from mail-vc0-x22b.google.com (mail-vc0-x22b.google.com [IPv6:2607:f8b0:400c:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7A760280F; Mon, 28 Jul 2014 19:35:44 +0000 (UTC) Received: by mail-vc0-f171.google.com with SMTP id hq11so11968605vcb.16 for ; Mon, 28 Jul 2014 12:35:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=nHhcB/8wcRvcD/GylWH7544IlcbdQv3UjuCiD2Kuv5M=; b=LJL/QPv3FGAKIJyQli/sfAI3V7JwOAH2G9JEAo74Hv+aOmmBkIPm64guCLZv6TSvN2 5q+aRufKiYX4XZFmyXtV+JS9wfSXr+NjxWqz+Go8Gc+tX8/WkSawPurNZN9ufbjTJI+d Vq/1V+q3/32v0sF7q4O903OqPVJKNMGVDcMDdDca56ztUfBYE91yPsdpAWaaKqPuIZLi 0/y4gEoOblU7pNeyuoIxgYWH4mi1FlbWVDYml2jQKjEbZBdvCcfNeLu2i0BEg+gY1PaJ edbhIDEB9PIu2VZxK9gLla4+JoAZKQhSPMrZrNt3nrUhQknrUsJEJIvCqEm50aoZ3BF0 NdWg== MIME-Version: 1.0 X-Received: by 10.52.101.168 with SMTP id fh8mr40002588vdb.34.1406576143453; Mon, 28 Jul 2014 12:35:43 -0700 (PDT) Received: by 10.220.192.68 with HTTP; Mon, 28 Jul 2014 12:35:43 -0700 (PDT) In-Reply-To: References: Date: Mon, 28 Jul 2014 15:35:43 -0400 Message-ID: Subject: Re: Allocation/free history From: suresh gumpula To: Adrian Chadd X-Mailman-Approved-At: Mon, 28 Jul 2014 19:58:44 +0000 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jul 2014 19:35:44 -0000 I am talking about kernel space Chadd. Thanks Suresh On Mon, Jul 28, 2014 at 3:20 PM, Adrian Chadd wrote: > Hi! > > Are you talking about userland? kernel? both? > > > -a > > > On 28 July 2014 10:40, suresh gumpula wrote: > > Hi, > > Knowing the PC of an allocation is very usefull in debugging. Having > the > > PC hash table and storing the pc hash either with an object itself( at > the > > end) or allocate an exra structure to hold the > > hash index help us find out who/where an object was allocated. We > > already have something like this in our own operating system and has > been a > > useful thing in debugging. > > BSD allocator uses power of 2, so storing at the end of an object might > be > > wasting lot of memory with large objects. > > > > It appears we don=E2=80=99t have something like this in current FBSD co= deline and > > would like to work on this ? Any comments please? > > > > > > It would be something like below. 8 bytes at the end of each object has > > fecepost which is usefull in finding overwrites and 2 hash indices to t= he > > PC table to track allocation history. > > (kgdb-amd64-7.4-87) x/40w 0xffffff153728b038 > > 0xffffff153728b038: 0xf6970a05 0x06cb7e0c 0x305a134a > > 0x831c18bb > > 0xc0000bed 0x134a2115 > > > > 0xffffff153728b050: 0x85687ef8 0xffffffff 0x00000001 > > 0x83192201 > > 0xc0000bed 0x1741143b > > > > Thanks > > Suresh > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to " > freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Mon Jul 28 19:37:29 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6EADDC18 for ; Mon, 28 Jul 2014 19:37:29 +0000 (UTC) Received: from mail-vc0-x234.google.com (mail-vc0-x234.google.com [IPv6:2607:f8b0:400c:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2C57A282D for ; Mon, 28 Jul 2014 19:37:29 +0000 (UTC) Received: by mail-vc0-f180.google.com with SMTP id ij19so11968739vcb.39 for ; Mon, 28 Jul 2014 12:37:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=xp8DqLT8afbpNpdJz/Lif95buajJ+dt1BpwhQzzc0f8=; b=P68nEk2ZcouScR1uQFaw9286JsXJQ78hWQEhFl9DeFgJ5blXfyEw1+pwax9lF1yUI0 gl606SgxKp3ASgg0IykiOS77lRi6Ew3bcma6H/TR45kz9zk+8hUPUC1fQxVx3CVNC1P0 ofefkJn33tINKIIfQ5/C7MBbO+M108ML9UmxDaP5flKDRJUKuwdBoLE6DIBrc94Uhj59 YzD3oTm61Cc5PYwQax1Lfy0sKspF9BWseD4xazIYwYPpE8WS6j8vrT/giWW3K+cftSTf B3IV9799dr5FZm1OlD1+QK+/LEnDletoRXh1cgjQGOb5m9g7CIr6Zi6gPy/7+dx9FO7P d+lQ== MIME-Version: 1.0 X-Received: by 10.52.1.39 with SMTP id 7mr3842837vdj.17.1406576248211; Mon, 28 Jul 2014 12:37:28 -0700 (PDT) Received: by 10.220.192.68 with HTTP; Mon, 28 Jul 2014 12:37:28 -0700 (PDT) In-Reply-To: <20140728191950.GA9781@dft-labs.eu> References: <20140728191950.GA9781@dft-labs.eu> Date: Mon, 28 Jul 2014 15:37:28 -0400 Message-ID: Subject: Re: Allocation/free history From: suresh gumpula To: Mateusz Guzik , suresh gumpula , "freebsd-hackers@freebsd.org" X-Mailman-Approved-At: Mon, 28 Jul 2014 20:04:54 +0000 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jul 2014 19:37:29 -0000 Hi, I am aware of redone(9). I am talking about PC allocation history of objects. Do we have something like that ? Thanks Suresh On Mon, Jul 28, 2014 at 3:19 PM, Mateusz Guzik wrote: > On Mon, Jul 28, 2014 at 01:40:10PM -0400, suresh gumpula wrote: > > Hi, > > Knowing the PC of an allocation is very usefull in debugging. Having > the > > PC hash table and storing the pc hash either with an object itself( at > the > > end) or allocate an exra structure to hold the > > hash index help us find out who/where an object was allocated. We > > already have something like this in our own operating system and has > been a > > useful thing in debugging. > > BSD allocator uses power of 2, so storing at the end of an object might > be > > wasting lot of memory with large objects. > > > > It appears we don=E2=80=99t have something like this in current FBSD co= deline and > > would like to work on this ? Any comments please? > > > > > > It would be something like below. 8 bytes at the end of each object has > > fecepost which is usefull in finding overwrites and 2 hash indices to t= he > > PC table to track allocation history. > > (kgdb-amd64-7.4-87) x/40w 0xffffff153728b038 > > 0xffffff153728b038: 0xf6970a05 0x06cb7e0c 0x305a134a > > 0x831c18bb > > 0xc0000bed 0x134a2115 > > > > 0xffffff153728b050: 0x85687ef8 0xffffffff 0x00000001 > > 0x83192201 > > 0xc0000bed 0x1741143b > > > > See redzone(9). > > -- > Mateusz Guzik > From owner-freebsd-hackers@FreeBSD.ORG Mon Jul 28 20:22:02 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D9771137 for ; Mon, 28 Jul 2014 20:22:02 +0000 (UTC) Received: from mail-wg0-x233.google.com (mail-wg0-x233.google.com [IPv6:2a00:1450:400c:c00::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6CA882D77 for ; Mon, 28 Jul 2014 20:22:02 +0000 (UTC) Received: by mail-wg0-f51.google.com with SMTP id b13so7811028wgh.34 for ; Mon, 28 Jul 2014 13:22:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-type:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=keFzr7viVrH1MSUgnXMp3pIRk3T6MCgdldlvANIGHhM=; b=ZCDQiuPn9rVt3lpxtRS4vRRE2FdkfLK0TyNKd8ujp+2QMUiSGvtqQ1CMsx+aoTNbI6 C9CTbDjuE3sfW8t5pZlf/OSHyxi7Ug7GLCOuw6cJqV9n7AsnIP/fMP/mthgiRLJsTz5E 7HCMhpk2TMFBUTCyF8vOjzn73F+TSQEWbIQqqyI0m79vvhpdRxPDGlAqBTxPcxPtYCKW PUp+TMzbKQj3ke2S6J9HfU5ERCXk5d3kS7iNuIVgbK9r+4J2E41J6dDZakkxw+yiL1Eu ZjtSvD12PmD3zViRza3cyWzVzLN+4BMghD2k5Sr2/HpkzwfmEP+n3RW8SnyTJfIjMesD 6Usg== X-Received: by 10.194.158.101 with SMTP id wt5mr15818144wjb.136.1406578919417; Mon, 28 Jul 2014 13:21:59 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id n2sm51935175wjf.40.2014.07.28.13.21.58 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Mon, 28 Jul 2014 13:21:58 -0700 (PDT) Date: Mon, 28 Jul 2014 22:21:56 +0200 From: Mateusz Guzik To: suresh gumpula Subject: Re: Allocation/free history Message-ID: <20140728202156.GB9781@dft-labs.eu> Mail-Followup-To: Mateusz Guzik , suresh gumpula , "freebsd-hackers@freebsd.org" References: <20140728191950.GA9781@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jul 2014 20:22:03 -0000 On Mon, Jul 28, 2014 at 03:37:28PM -0400, suresh gumpula wrote: > Hi, > I am aware of redone(9). I am talking about PC allocation history of > objects. Do we have something like that ? > Well, redzone "remembers" allocation and free backtrace, but does not keep any further history. It is unclear for me if tracking entire history would be of much benefit (note that mere allocation instruction pointer is typically not really helpful and keeping bts for everything would be quite memory consumimg). > Thanks > Suresh > > > On Mon, Jul 28, 2014 at 3:19 PM, Mateusz Guzik wrote: > > > On Mon, Jul 28, 2014 at 01:40:10PM -0400, suresh gumpula wrote: > > > Hi, > > > Knowing the PC of an allocation is very usefull in debugging. Having > > the > > > PC hash table and storing the pc hash either with an object itself( at > > the > > > end) or allocate an exra structure to hold the > > > hash index help us find out who/where an object was allocated. We > > > already have something like this in our own operating system and has > > been a > > > useful thing in debugging. > > > BSD allocator uses power of 2, so storing at the end of an object might > > be > > > wasting lot of memory with large objects. > > > > > > It appears we don’t have something like this in current FBSD codeline and > > > would like to work on this ? Any comments please? > > > > > > > > > It would be something like below. 8 bytes at the end of each object has > > > fecepost which is usefull in finding overwrites and 2 hash indices to the > > > PC table to track allocation history. > > > (kgdb-amd64-7.4-87) x/40w 0xffffff153728b038 > > > 0xffffff153728b038: 0xf6970a05 0x06cb7e0c 0x305a134a > > > 0x831c18bb > > > 0xc0000bed 0x134a2115 > > > > > > 0xffffff153728b050: 0x85687ef8 0xffffffff 0x00000001 > > > 0x83192201 > > > 0xc0000bed 0x1741143b > > > > > > > See redzone(9). > > > > -- > > Mateusz Guzik > > -- Mateusz Guzik From owner-freebsd-hackers@FreeBSD.ORG Mon Jul 28 20:46:26 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E79FEFE4 for ; Mon, 28 Jul 2014 20:46:26 +0000 (UTC) Received: from mail-qa0-x236.google.com (mail-qa0-x236.google.com [IPv6:2607:f8b0:400d:c00::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A829020E3 for ; Mon, 28 Jul 2014 20:46:26 +0000 (UTC) Received: by mail-qa0-f54.google.com with SMTP id k15so8430914qaq.13 for ; Mon, 28 Jul 2014 13:46:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=05V8lhptNK4JB9W9g2psvws+89KfrhFYX3r5cAbjgxs=; b=Qp+6/hm6wBATUrS3Anb+8nMx75lCn8fneOUm0ZMR0lTR645Ra4L74QiLbNcUsMBJHi x1dxqXD1kWRFseZ8y3Z7LsEH8SgrgOgdvLzkhKFC4KFNIDu7w6dcKkBNf5w9+xURKlvr 8NdFct2ogAHjLH4iiwQ4ZtLv9dExQEDuCboLNXtt9AFwBXuTjbsjyojWwiDB/KrJ/+zs CNfMbrHWhw1p/SA64voEfUZ7lpVe+S2hibYkFOgLZPRxR1R//wWgZVyJSn1krw7tcEVh R7v/b0KKa9+xS+Il4yJifmR9zCDRTxIIVnFbcNKxH9/+DgHuPUH42V91U4pV22EYucNl LRGw== MIME-Version: 1.0 X-Received: by 10.224.161.83 with SMTP id q19mr8473087qax.26.1406580385884; Mon, 28 Jul 2014 13:46:25 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.1.6 with HTTP; Mon, 28 Jul 2014 13:46:25 -0700 (PDT) In-Reply-To: References: Date: Mon, 28 Jul 2014 13:46:25 -0700 X-Google-Sender-Auth: E5wUyJ_QUOYx79NX8J43seAPUHw Message-ID: Subject: Re: Allocation/free history From: Adrian Chadd To: suresh gumpula Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jul 2014 20:46:27 -0000 On 28 July 2014 12:35, suresh gumpula wrote: > I am talking about kernel space Chadd. Sweet! Yes. Post a diff, let's get it tidied up. -a From owner-freebsd-hackers@FreeBSD.ORG Mon Jul 28 21:08:26 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AEA93AD2; Mon, 28 Jul 2014 21:08:26 +0000 (UTC) Received: from mail-vc0-x22e.google.com (mail-vc0-x22e.google.com [IPv6:2607:f8b0:400c:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 59CBB22C1; Mon, 28 Jul 2014 21:08:26 +0000 (UTC) Received: by mail-vc0-f174.google.com with SMTP id la4so12333853vcb.19 for ; Mon, 28 Jul 2014 14:08:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=2j2K5L2+FU70k6awch16CnGaVSIuMIJJT1YOYRFGph4=; b=wSE1w+0om/VY91mZS8QaiMUbd5wkmDbH5rAmGtTZV2AAAKkEnjyIWiECJCW6exipGv cxCzg24Z21Qu1BccK7lBVmmz/NJv9InUrSiqoX+mQFS8WzC5cu+bGwh+V0RxGnM9KTw7 UKeMDEfbvupDLrOBYvTu9W4E0vHido0N+79+NeiusyGRK3+/I7td8GPISgOrytWAPeb+ VRhu4dtLuye+nug7ncPJmVOnSCbQzJPdnFlDRJVi404FviqXAdcdBSDgWmi97pNpjqQ2 7N7x/lTXF6oc+2ugf1Etual8Sgxn2LbNuneYiZ3rD5kspYTqyuIRW3mflt+BdzA1tFGC Gavw== MIME-Version: 1.0 X-Received: by 10.220.144.10 with SMTP id x10mr3570956vcu.42.1406581705395; Mon, 28 Jul 2014 14:08:25 -0700 (PDT) Received: by 10.220.192.68 with HTTP; Mon, 28 Jul 2014 14:08:25 -0700 (PDT) In-Reply-To: References: Date: Mon, 28 Jul 2014 17:08:25 -0400 Message-ID: Subject: Re: Allocation/free history From: suresh gumpula To: Adrian Chadd X-Mailman-Approved-At: Mon, 28 Jul 2014 21:39:26 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jul 2014 21:08:26 -0000 Thanks Chadd ! I will come up with the code . Thanks Suresh On Mon, Jul 28, 2014 at 4:46 PM, Adrian Chadd wrote: > On 28 July 2014 12:35, suresh gumpula wrote: > > I am talking about kernel space Chadd. > > Sweet! Yes. Post a diff, let's get it tidied up. > > > > -a > From owner-freebsd-hackers@FreeBSD.ORG Tue Jul 29 21:51:20 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7688D4FB; Tue, 29 Jul 2014 21:51:20 +0000 (UTC) Received: from mail-qa0-x22f.google.com (mail-qa0-x22f.google.com [IPv6:2607:f8b0:400d:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 106632A31; Tue, 29 Jul 2014 21:51:19 +0000 (UTC) Received: by mail-qa0-f47.google.com with SMTP id i13so368129qae.34 for ; Tue, 29 Jul 2014 14:51:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=FG/kshe92B2xwlDeapIm0f13D80apjRna8e2Hhagcsc=; b=DD4CX96Ey69B+RKNgNueYx4XctprC/OyEXe9USQf0sJQiuskVYF9teCJTAgsRt6ksP z/Xcu0dDVsdwOgdrvSisaLrVJlsT6kSbDrabq6q+MJEmCg2FAi5WdLsDNFGeYlsgouEs vsRs5Qcupr0vz3Zl2M8l0lSwFn3FkxfpXnvOpdqQyBdVJ2gi1zzCAj9VH5URlG5DK2o/ l5iHVMgWXt7YM0lhTwC8iEciVlkRRF4Lqf4S8B0snU9qAXixOgpS42YP/18QzpHbOPxJ kE30P/VUpd/6DqdgDK34pekUXbWNTA74UHwOT+IezaLg+gueem59R7rxwxA6QuEqmWe6 /7kw== MIME-Version: 1.0 X-Received: by 10.224.42.196 with SMTP id t4mr7988740qae.48.1406670679199; Tue, 29 Jul 2014 14:51:19 -0700 (PDT) Received: by 10.140.27.172 with HTTP; Tue, 29 Jul 2014 14:51:19 -0700 (PDT) In-Reply-To: References: <00E55D89-BDD1-41AD-BBF6-6752B90E8324@ccsys.com> Date: Tue, 29 Jul 2014 14:51:19 -0700 Message-ID: Subject: Re: Working on NUMA support From: Andrew Bates To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-hackers@freebsd.org" , Jeff Roberson X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Jul 2014 21:51:20 -0000 Hey Adrian, Yes, there has been progress on this - although admittedly not as much as we'd like at this point. I believe to what you're talking about, we have the layout for CPU affinity/locality. I need to go through and cleanup a good half-dozen branches of code. Myself a mere mortal standing on the shoulders of giants in a room of titans, I have to merge in my changes with Jeff's pertinent branch to get this closer to useable. >From my experience and research, in terms of access/response time: 1. localized DMA < all remote 2. (localized DMA + spillover remote) >= all remote As ugly as it may be, I think I said that right.. There have been a few changes since that original email, but yes what we're working to address is the userland <---> kernelspace. On Sat, Jul 26, 2014 at 1:11 PM, Adrian Chadd wrote: > Hi all! > > Has there been any further progress on this? > > I've been working on making the receive side scaling support usable by > mere mortals and I've reached a point where I'm going to need this > awareness in the 10ge/40ge drivers for the hardware I have access to. > > I'm right now more interested in the kernel driver/allocator side of > things, so: > > * when bringing up a NIC, figure out what are the "most local" CPUs to run > on; > * for each NIC queue, figure out what the "most local" bus resources > are for NIC resources like descriptors and packet memory (eg mbufs); > * for each NIC queue, figure out what the "most local" resources are > for local driver structures that the NIC doesn't touch (eg per-queue > state); > * for each RSS bucket, figure out what the "most local" resources are > for things like packet memory (mbufs), tcp/udp/inp control structures, > etc. > > I had a chat with jhb yesterday and he reminded me that y'all at > isilon have been looking into this. > > He described a few interesting cases from the kernel side to me. > > * On architectures with external IO controllers, the path cost from an > IO device to multiple CPUs may be (almost) equivalent, so there's not > a huge penalty to allocate things on the wrong CPU. I think it'll be > nice to get CPU local affinity where possible so we can parallelise > DRAM access fully, but we can play with this and see. > * On architectures with CPU-integrated IO controllers, there's a large > penalty for doing inter-CPU IO, > * .. but there's not such a huge penalty for doing inter-CPU memory access. > > Given that, we may find that we should always put the IO resources > local to the CPU it's attached to, even if we decide to run some / all > of the IO for the device on another CPU. Ie, any RAM that the IO > device is doing data or descriptor DMA into should be local to that > device. John said that in his experience it seemed the penalty for a > non-local CPU touching memory was much less than device DMA crossing > QPI. > > So the tricky bit is figuring that out and expressing it all in a way > that allows us to do memory allocation and CPU binding in a more aware > way. The other half of this tricky thing is to allow it to be easily > overridden by a curious developer or system administrator that wants > to experiment with different policies. > > Now, I'm very specifically only addressing the low level kernel IO / > memory allocation requirements here. There's other things to worry > about up in userland; I think you're trying to address that in your > KPI descriptions. > > Thoughts? > > > -a > -- V/Respectfully, Andrew M Bates From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 30 00:36:29 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B75FB62F; Wed, 30 Jul 2014 00:36:29 +0000 (UTC) Received: from mail-qg0-x231.google.com (mail-qg0-x231.google.com [IPv6:2607:f8b0:400d:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 652EE2B88; Wed, 30 Jul 2014 00:36:29 +0000 (UTC) Received: by mail-qg0-f49.google.com with SMTP id j107so586106qga.22 for ; Tue, 29 Jul 2014 17:36:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=k9jmrvpQNU55ZUJwMCBCZB1FnRCiimgAR1VrMllSclo=; b=nPcf2yDf3czA+jpsErO88nOCOb+XtFSHgZfhpHTNaOKp4jamPllf/S/zLBdgDHeFIf f16cR2OT959GfKGfPb5qSY3s2HMH+BLxxVLF7e1x8OffkE6HCcG/lTpuHCztlJ2Ci0Fa LNYJ+KzaWltbBbe9qWcGklLtTAgF615t7gBHO/4QqzKiacLt1y0XhpscAtvEfjx3YI52 svcgEUmWuH20iB8SS1E+uip7qXKvI+RZi2492jW9rxufxkgG0Gxg6ofSM1eylgF2lbX4 AvcdlDInJYt56ss2UQ4G3qEW4DnkXli0JyFl2DRvgTQU0EjyVk27bOXHMLNWCj4dNWob kFmQ== MIME-Version: 1.0 X-Received: by 10.224.161.83 with SMTP id q19mr1042855qax.26.1406680588414; Tue, 29 Jul 2014 17:36:28 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.1.6 with HTTP; Tue, 29 Jul 2014 17:36:28 -0700 (PDT) In-Reply-To: References: <00E55D89-BDD1-41AD-BBF6-6752B90E8324@ccsys.com> Date: Tue, 29 Jul 2014 17:36:28 -0700 X-Google-Sender-Auth: oXWu8v2arg7xEpHd0poYyx3RIGc Message-ID: Subject: Re: Working on NUMA support From: Adrian Chadd To: Andrew Bates Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" , Jeff Roberson X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2014 00:36:29 -0000 On 29 July 2014 14:51, Andrew Bates wrote: > Hey Adrian, > > Yes, there has been progress on this - although admittedly not as much as > we'd like at this point. I believe to what you're talking about, we have > the layout for CPU affinity/locality. I need to go through and cleanup a > good half-dozen branches of code. > > Myself a mere mortal standing on the shoulders of giants in a room of > titans, I have to merge in my changes with Jeff's pertinent branch to get > this closer to useable. Cool. What / where is jeff's branch? Is it public? > From my experience and research, in terms of access/response time: > 1. localized DMA < all remote > 2. (localized DMA + spillover remote) >= all remote > > As ugly as it may be, I think I said that right.. Hm, what do you mean by spillover remote? > There have been a few changes since that original email, but yes what we're > working to address is the userland <---> kernelspace. Cool. I'd like to get the kernel side worked out. :-) -a From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 30 14:04:25 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 658A1130 for ; Wed, 30 Jul 2014 14:04:25 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 398132268 for ; Wed, 30 Jul 2014 14:04:25 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-236-203.lns20.per1.internode.on.net [121.45.236.203]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id s6UE4Ju5022495 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 30 Jul 2014 07:04:22 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53D8FB5D.2060509@freebsd.org> Date: Wed, 30 Jul 2014 22:04:13 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: suresh gumpula , freebsd-hackers@freebsd.org Subject: Re: Allocation/free history References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2014 14:04:25 -0000 On 7/29/14, 1:40 AM, suresh gumpula wrote: > Hi, > Knowing the PC of an allocation is very usefull in debugging. Having the > PC hash table and storing the pc hash either with an object itself( at the > end) or allocate an exra structure to hold the > hash index help us find out who/where an object was allocated. We > already have something like this in our own operating system and has been a > useful thing in debugging. what OS is that? I assume you are talking about in the kernel? From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 30 16:05:27 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0B24F463; Wed, 30 Jul 2014 16:05:27 +0000 (UTC) Received: from mail-vc0-x22d.google.com (mail-vc0-x22d.google.com [IPv6:2607:f8b0:400c:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AD071226D; Wed, 30 Jul 2014 16:05:26 +0000 (UTC) Received: by mail-vc0-f173.google.com with SMTP id hy10so2150366vcb.4 for ; Wed, 30 Jul 2014 09:05:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=gAneBD/T3sN1Fs1rnBETZXF7eYf3ID55zC7Pa5KK1Dg=; b=fsm4gZNn+bMqzqgPUQWG8xi6mrWcDIrYP/9+zRtz6DkXe9XM7rzKrk5o8pHIffD7c4 sSGSwhFMxhR1HvalYBO0VjSn5Ezs/ZaPC+vk7Q3Gxtv8tBqxKuICKhiluCpJrzSgxLJm H+I+x8reFCOJ70BTcjg3UmyItwdOebrxOeY5rIDgtWRBjXgBjJzBurhCNpBftoaft2/F lklN10Fl2QS5UYDp5IE4czrZlfGKtg30WbRkA+Q8hxY1p7Ui7R7QPOY1nC/92ZACfADH ptjVMHiNWkIj/pmlwAvKBvM/EbFPtEzwL59tob2RqUTK55P5XucPCGh3k8FrzdiEEvJU wEHw== MIME-Version: 1.0 X-Received: by 10.220.196.207 with SMTP id eh15mr2528072vcb.78.1406736325697; Wed, 30 Jul 2014 09:05:25 -0700 (PDT) Received: by 10.220.192.68 with HTTP; Wed, 30 Jul 2014 09:05:25 -0700 (PDT) In-Reply-To: <53D8FB5D.2060509@freebsd.org> References: <53D8FB5D.2060509@freebsd.org> Date: Wed, 30 Jul 2014 12:05:25 -0400 Message-ID: Subject: Re: Allocation/free history From: suresh gumpula To: Julian Elischer X-Mailman-Approved-At: Wed, 30 Jul 2014 17:13:36 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2014 16:05:27 -0000 Hi Julian, Its our proprietary OS called Simple kernel and yes its in the kernel space allocator. Thanks Suresh On Wed, Jul 30, 2014 at 10:04 AM, Julian Elischer wrote: > On 7/29/14, 1:40 AM, suresh gumpula wrote: > >> Hi, >> Knowing the PC of an allocation is very usefull in debugging. Having >> the >> PC hash table and storing the pc hash either with an object itself( at >> the >> end) or allocate an exra structure to hold the >> hash index help us find out who/where an object was allocated. We >> already have something like this in our own operating system and has been >> a >> useful thing in debugging. >> > > what OS is that? > > I assume you are talking about in the kernel? > > From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 30 17:57:52 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 19ACBB0C for ; Wed, 30 Jul 2014 17:57:52 +0000 (UTC) Received: from dmz-mailsec-scanner-2.mit.edu (dmz-mailsec-scanner-2.mit.edu [18.9.25.13]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 992352048 for ; Wed, 30 Jul 2014 17:57:50 +0000 (UTC) X-AuditID: 1209190d-f79c06d000002f07-d5-53d932179a85 Received: from mailhub-auth-2.mit.edu ( [18.7.62.36]) (using TLS with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by dmz-mailsec-scanner-2.mit.edu (Symantec Messaging Gateway) with SMTP id DF.69.12039.71239D35; Wed, 30 Jul 2014 13:57:43 -0400 (EDT) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-2.mit.edu (8.13.8/8.9.2) with ESMTP id s6UHvgei028448; Wed, 30 Jul 2014 13:57:43 -0400 Received: from multics.mit.edu (system-low-sipb.mit.edu [18.187.2.37]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id s6UHvee7010442 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 30 Jul 2014 13:57:42 -0400 Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id s6UHveKn015480; Wed, 30 Jul 2014 13:57:40 -0400 (EDT) Date: Wed, 30 Jul 2014 13:57:40 -0400 (EDT) From: Benjamin Kaduk X-X-Sender: kaduk@multics.mit.edu To: Nidal Khalil Subject: Re: freebsd-hackers Digest, Vol 588, Issue 1 In-Reply-To: Message-ID: References: User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrJIsWRmVeSWpSXmKPExsUixG6noitudDPY4MhfKYvtm/8xWjx6foLR gcljxqf5LB47Z91lD2CK4rJJSc3JLEst0rdL4MqYOL+FtWCdYMWre18ZGxjv83YxcnJICJhI HNq6mQ3CFpO4cG89kM3FISQwm0niUEMDK4SzkVFiybH1TBDOISaJ9Z+uMEM4DYwSvzc+BOtn EdCWuNowgRXEZhNQk3i8t5kVYq6ixOZTk4AaODhEBFQk3k7PBwkzC8hLXNh8iBEkLCxgKrHr lCtImFMgUGLJ32PsIDavgKPEhmctjBCrDrNIPD00kwUkISqgI7F6/xQWiCJBiZMzn7BAzLSU OPfnOtsERqFZSFKzkKQWMDKtYpRNya3SzU3MzClOTdYtTk7My0st0jXSy80s0UtNKd3ECAph TkneHYzvDiodYhTgYFTi4Z3x/3qwEGtiWXFl7iFGSQ4mJVFeOb2bwUJ8SfkplRmJxRnxRaU5 qcWHGCU4mJVEgFYD5XhTEiurUovyYVLSHCxK4rxvra2ChQTSE0tSs1NTC1KLYLIyHBxKErz+ hkCNgkWp6akVaZk5JQhpJg5OkOE8QMPVDUCGFxck5hZnpkPkTzEqSonzmoA0C4AkMkrz4Hph KeYVozjQK8K8BSBVPMD0BNf9CmgwE9Dg57eugwwuSURISTUwHp7+Xezqm2Wn50vxpXxn0Ag9 VMWhHLXxs3OrsL07OyvHR5NVQqUPS5J/mARn/bZ58e5KY6/ti9u5VU76Sk+/XF/L/n5ejrP7 sSDW27/5Nq/rXd64pO78WXeFftGnG6L5OneV3vZQ37yHSfj7y6gbKyfdTayc4rrGdkv8XVPj 1AtmRwNbZpzSVGIpzkg01GIuKk4EAHPbGYQMAwAA Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2014 17:57:52 -0000 On Wed, 30 Jul 2014, Nidal Khalil wrote: > Yeah. This is the doc. > Kgdb works for bt only, info locals results in no symbols found. kgdb works great on crash dumps for me. My kernels are compiled using "makeoptions DEBUG=-g", and sometimes with -O0. I would have to check, but suspect that the kernels for some releases do not have the debug symbols enabled. > In regards to gdb the -k option doesn't work. It results in unknown option. Our kgdb is customized to work with FreeBSD kernel dumps; stock gdb isn't really going to work. > With out it it doesn't understand the vmcore file format. > Debugging tools are broken for release 9.2 and release 9.3 with the > exception of ddb. > The documentation is incorrect and doesn't match the new releases. I think you will need to provide a more detailed pointer to what precise documentation steps you think are incorrect, and (copy/paste of) the actual behavior you are seeing, in order for us to try to reproduce it. As I say, kgdb works fine in my usage. > Nidal > On Jul 29, 2014 5:23 PM, "Benjamin Kaduk" wrote: > >> You did not heed my note about public lists. >> You did not link to what documentation you believe is incorrect. >> You might consult http://www.freebsd.org/doc/en_ >> US.ISO8859-1/books/developers-handbook/kerneldebug-gdb.html >> >> On Tue, 29 Jul 2014, Nidal Khalil wrote: >> >> The documentation for gdb is not correct? >>> >>> I am trying to use gdb to analyze a kernel crash. However the -k is >>> recognized. This is version 6.1.1 that came with the installation. >>> Using it without -k it will not understand the vmcore file format >>> do I need a different gdb. Thanks >>> >>> Nidal >>> >>> >>> >>> On Wed, Jul 23, 2014 at 7:56 AM, Benjamin Kaduk wrote: >>> >>> On Wed, 23 Jul 2014, Nidal Khalil wrote: >>>> >>>> I have no interest in firewire. How can I do this over Ethernet. The >>>> >>>>> documentation is a little bit vague in this area. Thanks I don't believe that remote debugging over ethernet has been implemented yet, though I do remember some talk about interest and/or the beginning of a project, years ago. -Ben From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 30 19:20:21 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02157B5D for ; Wed, 30 Jul 2014 19:20:21 +0000 (UTC) Received: from mail-vc0-f175.google.com (mail-vc0-f175.google.com [209.85.220.175]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AFF732931 for ; Wed, 30 Jul 2014 19:20:20 +0000 (UTC) Received: by mail-vc0-f175.google.com with SMTP id ik5so2597636vcb.34 for ; Wed, 30 Jul 2014 12:20:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=3Jy5DcualqOcwSb0HiWeBhF9aodT1/CD5ni4yCBUwPM=; b=EQwNRmOuQJKNpnXuGoMv8XFCYYBo8jQ5GwbNaeVAozJSaXesAOBz2cfs0VCVL9+vPr DY+UAHcO5y8/mVKOitIdxtES23itCw4OGckt0eq2iNtcHYY0ii2gB7jaZgPOJVejX7Lu mz3J8Sj/ip60mTMvHyVp5mK7gRdXZ7SE+UfTD2uHVtq5iZV1p2Ir2ShnP0WWO2KhQ40P 7gJ99QrSQq0zzs/bB+B3TDWuZJ5OTGy3CVU/tfAjVVybgaHi3HdCKwUfQDIrJRWxFxNY tq/LjFtmfHGBhp8QMDb7fMU4VPl06FK2NjIxQrdjCNuu1ubsr8pu7RMP1oRBQzYAJSyB cS9A== X-Gm-Message-State: ALoCoQlcOqPa3Wh5hsQQSAPHUisZC2r6cZPesmJPwYkzTfSEXgUcw8zmEqwm+b3J59HQcGlXh0k5 X-Received: by 10.52.38.67 with SMTP id e3mr10459659vdk.56.1406748013486; Wed, 30 Jul 2014 12:20:13 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.220.66 with HTTP; Wed, 30 Jul 2014 12:19:53 -0700 (PDT) X-Originating-IP: [216.240.30.23] In-Reply-To: <20140728202156.GB9781@dft-labs.eu> References: <20140728191950.GA9781@dft-labs.eu> <20140728202156.GB9781@dft-labs.eu> From: Bryan Venteicher Date: Wed, 30 Jul 2014 14:19:53 -0500 Message-ID: Subject: Re: Allocation/free history To: Mateusz Guzik , suresh gumpula , "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2014 19:20:21 -0000 On Mon, Jul 28, 2014 at 3:21 PM, Mateusz Guzik wrote: > On Mon, Jul 28, 2014 at 03:37:28PM -0400, suresh gumpula wrote: > > Hi, > > I am aware of redone(9). I am talking about PC allocation history of > > objects. Do we have something like that ? > > > > Well, redzone "remembers" allocation and free backtrace, but does not > keep any further history. It is unclear for me if tracking entire > history would be of much benefit (note that mere allocation instruction > pointer is typically not really helpful and keeping bts for everything > would be quite memory consumimg). > > =E2=80=8BThe AIX kernel debugger can provide a history when and who allocat= ed and freed a piece of memory. In a previous life, this was very useful at times to track down memory corruption issues. Stealing a random example from [1]: (0)> xm -h 0x09410200 *Display debug xmalloc records associated with addr* 0B78DAB0: addr......... 09410200 req_size..... 128 freed unpinned 0B78DAB0: pid.......... 00043158 comm......... bcross Trace during xmalloc() Trace during xmfree() 002329E4(.xmalloc+0000A8) 002328F0(.xmfree+0000FC) 00235CD4(.dlistadd+000040) 00234F04(.setbitmaps+0001BC) 00235520(.newblk+00006C) 00236894(.finicom+0001A4) 0B645120: addr......... 09410200 req_size..... 128 freed unpinned 0B645120: pid.......... 0007DCAC comm......... bcross Trace during xmalloc() Trace during xmfree() 002329E4(.xmalloc+0000A8) 002328F0(.xmfree+0000FC) 00235CD4(.dlistadd+000040) 00236614(.logdfree+0001E8) 00236574(.logdfree+000148) 00236720(.finicom+000030) [1] - http://www.mouritzen.dk/aix-doc/en_US/a_doc_lib/aixprggd/kernextc/kdb_cmd17= .htm > > Thanks > > Suresh > > > > > > On Mon, Jul 28, 2014 at 3:19 PM, Mateusz Guzik > wrote: > > > > > On Mon, Jul 28, 2014 at 01:40:10PM -0400, suresh gumpula wrote: > > > > Hi, > > > > Knowing the PC of an allocation is very usefull in debugging. > Having > > > the > > > > PC hash table and storing the pc hash either with an object itself= ( > at > > > the > > > > end) or allocate an exra structure to hold the > > > > hash index help us find out who/where an object was allocated. W= e > > > > already have something like this in our own operating system and ha= s > > > been a > > > > useful thing in debugging. > > > > BSD allocator uses power of 2, so storing at the end of an object > might > > > be > > > > wasting lot of memory with large objects. > > > > > > > > It appears we don=E2=80=99t have something like this in current FBS= D > codeline and > > > > would like to work on this ? Any comments please? > > > > > > > > > > > > It would be something like below. 8 bytes at the end of each object > has > > > > fecepost which is usefull in finding overwrites and 2 hash indices > to the > > > > PC table to track allocation history. > > > > (kgdb-amd64-7.4-87) x/40w 0xffffff153728b038 > > > > 0xffffff153728b038: 0xf6970a05 0x06cb7e0c 0x305a134a > > > > 0x831c18bb > > > > 0xc0000bed 0x134a2115 > > > > > > > > 0xffffff153728b050: 0x85687ef8 0xffffffff 0x00000001 > > > > 0x83192201 > > > > 0xc0000bed 0x1741143b > > > > > > > > > > See redzone(9). > > > > > > -- > > > Mateusz Guzik > > > > > -- > Mateusz Guzik > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 30 21:26:22 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BAA5D52A for ; Wed, 30 Jul 2014 21:26:22 +0000 (UTC) Received: from mail-wi0-x22a.google.com (mail-wi0-x22a.google.com [IPv6:2a00:1450:400c:c05::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 562422729 for ; Wed, 30 Jul 2014 21:26:22 +0000 (UTC) Received: by mail-wi0-f170.google.com with SMTP id f8so317043wiw.1 for ; Wed, 30 Jul 2014 14:26:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=/7NgxqTjuM9vkj8XC13Yn9u7SI6ni+f3tq2MSRWFg00=; b=QbIisfX9GUbaM8iWXqY9XUemv04NA/+K8SnS/DT1ZrnjoHdO5hI0XHjv7w6J1XCE5X XkcyKXBhaxNOavVeL/xjHerAP7b+RUevohhtQ3XiW2RUOk5m8Z5BrOi8THBCoTdDI/Wx p+bBA2/uV2okDozgMCHa9UycOteKJrm6MeZh5fzWt00AO591gEpoWLtdNvBwXljGANpk zXo/mqJiaGILqQI73xo6syTBvdKwQL5t/qGk6Yijwi4xGFK/6XBxyfvTGJj5E4iLTJwI 7vRrf+2Oq/pC2MSvJLCEJtoZx+dtJ0/BYFlM6Face0IiFaWuRyFeFZUR+x5mlNNuaO8e IS1g== X-Received: by 10.194.92.196 with SMTP id co4mr10444402wjb.4.1406755580350; Wed, 30 Jul 2014 14:26:20 -0700 (PDT) Received: from [192.168.2.100] ([46.187.149.223]) by mx.google.com with ESMTPSA id fb12sm8360818wjc.43.2014.07.30.14.26.12 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 30 Jul 2014 14:26:15 -0700 (PDT) Message-ID: <53D962A7.5090105@gmail.com> Date: Wed, 30 Jul 2014 23:24:55 +0200 From: Adi User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: FreeBSD 10 swapping Idle processes :/ Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2014 21:26:22 -0000 Hello i7-4770 , 32 GB RAM, FreeBSD 10.0-RELEASE-p7, AMD64, UFS2 Problem: Many idle processes is swapping :( Similar system configurations and deamons settings on my old server on FBSD 9.1 with 16 GB was working many years. Maybe some login.conf changed. Probably I made mistakes in several settings. Details info (long) is available at http://pastebin.com/Y9mJUD6F OR RAW (if numbered listing not working well) http://pastebin.com/raw.php?i=Y9mJUD6F Can You help me fix settings? Thanks for Your help and time. Best Regards. From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 30 21:37:47 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A0031A66; Wed, 30 Jul 2014 21:37:47 +0000 (UTC) Received: from mail-vc0-x22e.google.com (mail-vc0-x22e.google.com [IPv6:2607:f8b0:400c:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4438A2835; Wed, 30 Jul 2014 21:37:47 +0000 (UTC) Received: by mail-vc0-f174.google.com with SMTP id la4so2913450vcb.19 for ; Wed, 30 Jul 2014 14:37:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=G9AOJEPuLIybqjWjsm9cxMZENLQhqCt83zFe1sTScq8=; b=s8uSF14QOfm8TtjGU3rJsxtdenGHLF/7pi7Iv/+SyqPGMysJ/8VhulLz91xS1MQCYM oCGnC2H8JK0bJ62LkwfbrqzJMOSZcPBhULiMQ4jl2Ftgd2rRJfWiB2I7xhVSeMJ4IpZw glYGuwVs3RKQsyL3JNR/+aI+AnXsg9+hC4uLCUIPqdzXal6Jz4cjIGfG1iX4572aAUR/ 9BMntHta0HvwgHHqauAjOqTokOqAO1hGtC3guIwab2ggfdCQPzYl0ZIVKs3lbPXFiZg/ 9V4f67Amzq4jlqXasqCmLC1SwlR/X533pXKY3tP9CGw5Wv8IMXdwCnnfea4e+SNaR4hP t7mA== MIME-Version: 1.0 X-Received: by 10.220.119.8 with SMTP id x8mr6466363vcq.62.1406756266082; Wed, 30 Jul 2014 14:37:46 -0700 (PDT) Received: by 10.220.20.74 with HTTP; Wed, 30 Jul 2014 14:37:45 -0700 (PDT) In-Reply-To: References: Date: Wed, 30 Jul 2014 14:37:45 -0700 Message-ID: Subject: Re: freebsd-hackers Digest, Vol 588, Issue 1 From: Nidal Khalil To: Benjamin Kaduk Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2014 21:37:47 -0000 My kernel is compiled with -g as well. However I am loading a driver module. That is compiled with -g but the modules will not load if it is not stripped. it is using the objcopy found in bsd.kmod.mk. This is the reason that bt or where shows the backtrace but not the local frame symbols. On the gdb front The gdb that came with the distro is 6.1.1 has two problems: 1- does not understand -k option 2- without -k it does not understand the vmcore file format. Is there a utility that will convert a vmcore to a format that gdb understands. Thanks Nidal On Wed, Jul 30, 2014 at 10:57 AM, Benjamin Kaduk wrote: > On Wed, 30 Jul 2014, Nidal Khalil wrote: > > Yeah. This is the doc. >> Kgdb works for bt only, info locals results in no symbols found. >> > > kgdb works great on crash dumps for me. > My kernels are compiled using "makeoptions DEBUG=-g", and sometimes with > -O0. I would have to check, but suspect that the kernels for some releases > do not have the debug symbols enabled. > > > In regards to gdb the -k option doesn't work. It results in unknown >> option. >> > > Our kgdb is customized to work with FreeBSD kernel dumps; stock gdb isn't > really going to work. > > > With out it it doesn't understand the vmcore file format. >> Debugging tools are broken for release 9.2 and release 9.3 with the >> exception of ddb. >> The documentation is incorrect and doesn't match the new releases. >> > > I think you will need to provide a more detailed pointer to what precise > documentation steps you think are incorrect, and (copy/paste of) the actual > behavior you are seeing, in order for us to try to reproduce it. As I say, > kgdb works fine in my usage. > > > Nidal >> On Jul 29, 2014 5:23 PM, "Benjamin Kaduk" wrote: >> >> You did not heed my note about public lists. >>> You did not link to what documentation you believe is incorrect. >>> You might consult http://www.freebsd.org/doc/en_ >>> US.ISO8859-1/books/developers-handbook/kerneldebug-gdb.html >>> >>> On Tue, 29 Jul 2014, Nidal Khalil wrote: >>> >>> The documentation for gdb is not correct? >>> >>>> >>>> I am trying to use gdb to analyze a kernel crash. However the -k is >>>> recognized. This is version 6.1.1 that came with the installation. >>>> Using it without -k it will not understand the vmcore file format >>>> do I need a different gdb. Thanks >>>> >>>> Nidal >>>> >>>> >>>> >>>> On Wed, Jul 23, 2014 at 7:56 AM, Benjamin Kaduk >>>> wrote: >>>> >>>> On Wed, 23 Jul 2014, Nidal Khalil wrote: >>>> >>>>> >>>>> I have no interest in firewire. How can I do this over Ethernet. The >>>>> >>>>> documentation is a little bit vague in this area. Thanks >>>>>> >>>>> > I don't believe that remote debugging over ethernet has been implemented > yet, though I do remember some talk about interest and/or the beginning of > a project, years ago. > > -Ben > From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 30 21:42:15 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3C66CB8C for ; Wed, 30 Jul 2014 21:42:15 +0000 (UTC) Received: from dmz-mailsec-scanner-2.mit.edu (dmz-mailsec-scanner-2.mit.edu [18.9.25.13]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C785428DF for ; Wed, 30 Jul 2014 21:42:14 +0000 (UTC) X-AuditID: 1209190d-f79c06d000002f07-21-53d966b4bc4a Received: from mailhub-auth-4.mit.edu ( [18.7.62.39]) (using TLS with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by dmz-mailsec-scanner-2.mit.edu (Symantec Messaging Gateway) with SMTP id EB.97.12039.4B669D35; Wed, 30 Jul 2014 17:42:12 -0400 (EDT) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id s6ULgC2Q024691; Wed, 30 Jul 2014 17:42:12 -0400 Received: from multics.mit.edu (system-low-sipb.mit.edu [18.187.2.37]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id s6ULgAW2002392 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 30 Jul 2014 17:42:11 -0400 Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id s6ULg9xF013790; Wed, 30 Jul 2014 17:42:09 -0400 (EDT) Date: Wed, 30 Jul 2014 17:42:09 -0400 (EDT) From: Benjamin Kaduk X-X-Sender: kaduk@multics.mit.edu To: Nidal Khalil Subject: Re: freebsd-hackers Digest, Vol 588, Issue 1 In-Reply-To: Message-ID: References: User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrJIsWRmVeSWpSXmKPExsUixG6nrrsl7WawwaHtOhbbN/9jtHj0/ASj A5PHjE/zWTx2zrrLHsAUxWWTkpqTWZZapG+XwJVx6MEC1oLdHBWTTp9laWA8z9bFyMkhIWAi sf3lFBYIW0ziwr31QHEuDiGB2UwSB2+uYYRwNjJKvNnfzgLhHGKSePjzN1SmgVHi0eQpYLNY BLQlZmy6xAhiswmoSTze28wKMVdRYvOpScxdjBwcIgIqEm+n54OEmQXkJS5sPsQIEhYWMJXY dcoVJMwpECjx9ew0JhCbV8BRYkXjLVaIVftYJWasfMMOkhAV0JFYvR/ibF4BQYmTM5+wQMy0 lDj35zrbBEahWUhSs5CkFjAyrWKUTcmt0s1NzMwpTk3WLU5OzMtLLdI10svNLNFLTSndxAgK YU5J3h2M7w4qHWIU4GBU4uH9YXIzWIg1say4MvcQoyQHk5Ior2kwUIgvKT+lMiOxOCO+qDQn tfgQowQHs5IIb5cnUI43JbGyKrUoHyYlzcGiJM771toqWEggPbEkNTs1tSC1CCYrw8GhJMFb mQrUKFiUmp5akZaZU4KQZuLgBBnOAzT8AkgNb3FBYm5xZjpE/hSjopQ475sUoIQASCKjNA+u F5ZiXjGKA70izDsLpJ0HmJ7gul8BDWYCGvz81nWQwSWJCCmpBkZGi7RoJ71Ihq4F+Wyl6q0W mhWL5nq9ubY0pOnP2jj5DYf6VF4kbuI/IrHX4OyOlN2WE+fnZfyv5FhUUD9rZ1j7wul35uy6 ymjSbDv3dKEgq/Wtx7997T0+SepJP9Vl+z3pdHuCoeNkV8bXz67NnW/WdX/yyZXmEcIMmzNq nUzqn8+Z/DXUskeJpTgj0VCLuag4EQBtJG8cDAMAAA== Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2014 21:42:15 -0000 On Wed, 30 Jul 2014, Nidal Khalil wrote: > My kernel is compiled with -g as well. However I am loading a driver > module. That is compiled with -g but the modules will not load if it is not > stripped. it is using the objcopy found in bsd.kmod.mk. This is the reason > that bt or where shows the backtrace but not the local frame symbols. You can use the objcopy invocation three lines later in kmod.mk to produce a mykmod.ko.symbols file to put in /boot/modules/. Or you could have your driver module use bsd.kmod.mk for its build system, so everything will be taken care of automatically. > On the gdb front > The gdb that came with the distro is 6.1.1 has two problems: > 1- does not understand -k option > 2- without -k it does not understand the vmcore file format. Is there a > utility that will convert a vmcore to a format that gdb understands. Thanks I am not sure how many times I need to say it: regular gdb will not work with vmcores. You must use kgdb. Concentrate your time on getting kgdb working for you; ignore regular gdb. -Ben From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 30 23:09:28 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C183D6B0 for ; Wed, 30 Jul 2014 23:09:28 +0000 (UTC) Received: from mout.gmx.com (mout.gmx.com [74.208.4.200]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 74E452239 for ; Wed, 30 Jul 2014 23:09:28 +0000 (UTC) Received: from [192.168.1.81] ([172.15.184.248]) by mail.gmx.com (mrgmxus001) with ESMTPSA (Nemesis) id 0LyF75-1WR4fO1MuL-015Yb9 for ; Thu, 31 Jul 2014 01:04:11 +0200 Message-ID: <53D97A50.8090006@gmx.us> Date: Wed, 30 Jul 2014 19:05:52 -0400 From: Dutch Ingraham User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.7.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Audio CDs Not Playing Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:OGXtTLh1i3ZsRvCkyx1BInut41qIjz9IOBn7ofeErMDaoaSTAem +df8zRoQ2n1mkzcjtBAscrenb59HDKJlM1+ZFcc96RdZdEJVNbsZBIhA3QS9Ivy+Nk+ttbD mxkWnqq0/gBSzeIY1kqR00nHC+BuLD3+iY6R5/1yHDDZEfFQ09eAK0oVh3uAggQDL2DNxS/ yrEKbfCH/RZ57C3ALs5sA== X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2014 23:09:28 -0000 Greetings: I asked this question on freebsd-questions a couple of days ago and have received zero responses, so I'm posting here and asking the hackers for their help. Here's the problem: I'm having trouble with playing audio CDs; specifically, they won't play at all. Secondarily, it appears as though if they would play, I would need to be root to do so. My system stats are as follows: : FreeBSD dutch.freebsd.net 10.0-RELEASE-p7 FreeBSD 10.0-RELEASE-p7 #0: Tue Jul 8 06:37:44 UTC 2014 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 I am issuing the command and the CD simply will not play (it does not physically spin). However, if I issue , the CD will eject; if I issue , I will receive the tracks information, etc. So it appears as though communication is happening, but the CD will not play. If I issue these commands as a regular user, I receive a "permission denied" message; if run as root, simply nothing happens - no error, nothing. In fact, issuing returns 0. I have read the handbook sections on multimedia and setting up the sound card, and believe I've done everything by the book. For example, returns: hdac0@pci0:0:27:0: class=0x040300 card=0x04201028 chip=0x3a6e8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801JD/DO (ICH10 Family) HD Audio Controller' class = multimedia subclass = HDA and I have loaded, individually, both the snd_hda and snd_ich drivers with the same result (the hardware notes are ambiguous on the exact driver needed). The only somewhat useful information is returned by which returns: "cdcontrol: Inappropriate ioctl for device" However, a google search has not informed me as to what this means or how to correct. If someone could point me in the correct direction, I'd be appreciative. PS - This issue is not limited to CLI commands; VLC will issue the error "VLC is unable to open the MRL 'cdda:///dev/cd0'. Check the log for details." There is no log info that I can find. In addition, I can burn a cd with xfburn, but only as root. My fstab has the standard entry for CDs. Sound does work, as I can generate white noise. From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 30 23:14:47 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 336067D1 for ; Wed, 30 Jul 2014 23:14:47 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id 0E6C223EE for ; Wed, 30 Jul 2014 23:14:46 +0000 (UTC) Received: from [192.168.1.2] (senat1-01.HML3.ScaleEngine.net [209.51.186.5]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 2F6BD31471 for ; Wed, 30 Jul 2014 23:14:40 +0000 (UTC) Message-ID: <53D97C71.3050806@freebsd.org> Date: Wed, 30 Jul 2014 19:14:57 -0400 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: FreeBSD 10 swapping Idle processes :/ References: <53D962A7.5090105@gmail.com> In-Reply-To: <53D962A7.5090105@gmail.com> X-Enigmail-Version: 1.6 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="fXHQlUTcHrT0bSCAsTRcq5JPgEtd7I193" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2014 23:14:47 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --fXHQlUTcHrT0bSCAsTRcq5JPgEtd7I193 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 2014-07-30 17:24, Adi wrote: > Hello >=20 > i7-4770 , 32 GB RAM, FreeBSD 10.0-RELEASE-p7, AMD64, UFS2 >=20 > Problem: Many idle processes is swapping :( >=20 > Similar system configurations and deamons settings on my old server on > FBSD 9.1 with 16 GB was working many years. Maybe some login.conf chang= ed. >=20 > Probably I made mistakes in several settings. >=20 > Details info (long) is available at >=20 > http://pastebin.com/Y9mJUD6F >=20 > OR RAW (if numbered listing not working well) > http://pastebin.com/raw.php?i=3DY9mJUD6F >=20 >=20 > Can You help me fix settings? >=20 > Thanks for Your help and time. >=20 > Best Regards. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o= rg" >=20 That looks odd, the output of top shows 0 swap usage, but also 0 resident memory for most of the processes... --=20 Allan Jude --fXHQlUTcHrT0bSCAsTRcq5JPgEtd7I193 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJT2Xx0AAoJEJrBFpNRJZKfXd4P+gMhqnEx3q/i6pdnxgtKyopE dqLGQGZSBn5GVDjlDEuPa5S/qP/89lgtoD8uKiXB1XCXjJ0UQdgIcQpDVpv8+9sc ghDpIm+vcbwXOkvg5mXWhZ6O0JYcLGX/k1Y+OM7uZxcSr1clz0914/GOFf57ynmk xkI06O+NuDqXngh3BE6cbqUCRD7MLYnPggmLqQK/gDix1U8Egy1lLNTs4ciBB+AP Pfc3NBYg/GCKbEk/NENG5td2JewyoZv30Mc13bogo87NRGLFrh39ry4THbm9JuoE SiyZce23PQo6Ha/wRAAccl3Ro/TFPHr1wU1+/t8y2yZcicC3AVtT8hmXrI5PpUCJ V/+CJjrXCBSF5Kt/0VCs7MClqJlyfGpPd50MS3RbVzU1e+sIt9Kzf8fmzpwy40DC dPsyikwShLKdH2JHAfJ9WX7+xXnzPsjGSOjG97VC7pWwGl5bsane2ckMn6XzuFvK r2j211eNR9noGNNPr1lXpC3FoPRf0ulqm8b8xi20ZVGADN/kACrrI7OOHGspni/T xOeZEDM9AVx2Fi0PdtRU+bHAozXcCDWrJ4ySkeiYB9erWXnAeuEnXDAO46MF2tNj LyQoaleKynkVYUydTyAB+FKzMTKFWiwTtTd4WrocKBnWlQl52GWWj8uk/AX7NsTn T2D282WjQQGc6PuqGKlg =n0PW -----END PGP SIGNATURE----- --fXHQlUTcHrT0bSCAsTRcq5JPgEtd7I193-- From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 31 00:15:26 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1B1C53E9 for ; Thu, 31 Jul 2014 00:15:26 +0000 (UTC) Received: from mail-wi0-x22a.google.com (mail-wi0-x22a.google.com [IPv6:2a00:1450:400c:c05::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A73B9289C for ; Thu, 31 Jul 2014 00:15:25 +0000 (UTC) Received: by mail-wi0-f170.google.com with SMTP id f8so455978wiw.5 for ; Wed, 30 Jul 2014 17:15:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=date:from:to:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; bh=nzCMT3NDHtmn99XHotv/dZvV0gP/efnl89yuZlMcSYU=; b=xeh/Jn0hkjqDgDt4Gs7ahRuggNFIX4j3DThkL+NA65nxX8q5rbtsQ6ngCFDQDrmcIL D2IPOWXOFTfUF0yl3ZQo+Da7OMt3iOiEVXaHzl0IoJhhGcQW2EPqdmdj9W5oQKIqw4CI NnJiBWMO3MRngKTc5uGubn+W4RESQV7y1sDHo1HD/aZDO236CnvvclL7nrxemS6+M73z ZSRlIHMAx4IacuU2SN/xBVrmpKVO+iDaWJJ2ZspiUR+2HS7lHHv9aPuKhVg/Edzrj8wj +mRYKWvf6Eolk4r8gpt0UgCjuZwPe3Y/2cUgsSJBLRgY18GF6A5iAHDd0pPpRFgfLEYp V+kA== X-Received: by 10.180.183.165 with SMTP id en5mr7501302wic.41.1406765723633; Wed, 30 Jul 2014 17:15:23 -0700 (PDT) Received: from gumby.homeunix.com ([94.195.197.120]) by mx.google.com with ESMTPSA id au7sm9302064wjc.41.2014.07.30.17.15.21 for (version=SSLv3 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 30 Jul 2014 17:15:22 -0700 (PDT) Date: Thu, 31 Jul 2014 01:15:16 +0100 From: RW To: freebsd-hackers@freebsd.org Subject: Re: FreeBSD 10 swapping Idle processes :/ Message-ID: <20140731011516.21b1261d@gumby.homeunix.com> In-Reply-To: <53D97C71.3050806@freebsd.org> References: <53D962A7.5090105@gmail.com> <53D97C71.3050806@freebsd.org> X-Mailer: Claws Mail 3.10.1 (GTK+ 2.24.22; amd64-portbld-freebsd10.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 00:15:26 -0000 On Wed, 30 Jul 2014 19:14:57 -0400 Allan Jude wrote: > On 2014-07-30 17:24, Adi wrote: > > Hello > > > > i7-4770 , 32 GB RAM, FreeBSD 10.0-RELEASE-p7, AMD64, UFS2 > > > > Problem: Many idle processes is swapping :( > > > > That looks odd, the output of top shows 0 swap usage, but also 0 > resident memory for most of the processes... > My guess would be that these processes have had all their pages deactivated to accelerate their paging-out. Linux uses the terms swapping and paging interchangeably to mean paging. In FreeBSD swapping refers to the management of paging at the process level - in some cases it's more efficient to page-out all the memory from some idle or less active processes. I did think swapping was only used under abnormal memory shortage, but I just reproduced this by dd'ing a 16 GB file to tmpfs (with 16GB RAM, and 16GB swap). Some processes were marked as swapped before any actual paging occurred. From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 31 01:32:52 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6A72CCB8 for ; Thu, 31 Jul 2014 01:32:52 +0000 (UTC) Received: from dmz-mailsec-scanner-8.mit.edu (dmz-mailsec-scanner-8.mit.edu [18.7.68.37]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0CF3A206E for ; Thu, 31 Jul 2014 01:32:51 +0000 (UTC) X-AuditID: 12074425-f79766d000006da8-51-53d99b8e475c Received: from mailhub-auth-1.mit.edu ( [18.9.21.35]) (using TLS with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by dmz-mailsec-scanner-8.mit.edu (Symantec Messaging Gateway) with SMTP id 1B.12.28072.E8B99D35; Wed, 30 Jul 2014 21:27:42 -0400 (EDT) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id s6V1RfHi012476; Wed, 30 Jul 2014 21:27:42 -0400 Received: from multics.mit.edu (system-low-sipb.mit.edu [18.187.2.37]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id s6V1Rds7002524 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 30 Jul 2014 21:27:41 -0400 Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id s6V1RdKl012632; Wed, 30 Jul 2014 21:27:39 -0400 (EDT) Date: Wed, 30 Jul 2014 21:27:38 -0400 (EDT) From: Benjamin Kaduk To: Dutch Ingraham Subject: Re: Audio CDs Not Playing In-Reply-To: <53D97A50.8090006@gmx.us> Message-ID: References: <53D97A50.8090006@gmx.us> User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrBIsWRmVeSWpSXmKPExsUixCmqrNs3+2awwYv9JhbbN/9jtNj/UcmB yWPGp/ksHs92r2MOYIrisklJzcksSy3St0vgyuh8vJ254IpARfOHpywNjB28XYycHBICJhIr Nu5ig7DFJC7cWw9mCwnMZpLYOUMAwt7IKPGmhxfCPsQkcWZnVBcjF5DdwChxsm03WAOLgLbE tRMnmUBsNgEViZlvNgLFOThEBBQkGk5FgYSZBeQlLmw+xAhiCwsoSey71sECUsIpoCZx+50h SJhXwFHiaNMxqBNUJU7O/As2UVRAR2L1/iksEDWCQPEnLBAjLSXO/bnONoFRcBaS1CwkqQWM TKsYZVNyq3RzEzNzilOTdYuTE/PyUot0LfRyM0v0UlNKNzGCApTdRXUH44RDSocYBTgYlXh4 A81vBguxJpYVV+YeYpTkYFIS5S3pBgrxJeWnVGYkFmfEF5XmpBYfYpTgYFYS4e3yBMrxpiRW VqUW5cOkpDlYlMR531pbBQsJpCeWpGanphakFsFkZTg4lCR4780CahQsSk1PrUjLzClBSDNx cIIM5wEa/gakhre4IDG3ODMdIn+KUVFKnDcMJCEAksgozYPrhSWQV4ziQK8I824BqeIBJh+4 7ldAg5mABj+/dR1kcEkiQkqqgfGYbnqhvd7MBYc4QhcZdEy7w7pe9Msy6+KDDuotsz4kRF96 smTxvuOLSvld7FINuAsO8O+eMeHp3c4+/g/cW975TlqoeOOHzVmWPtOg419D+GIiWjd+i5S0 km4X1kq7t++YJ+fFD/HTM51sdqw8ZnqW19pw54eNq3+HFMjw/jy3dP7+tPwroh+VWIozEg21 mIuKEwFxZKuS+wIAAA== Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 01:32:52 -0000 I am not sure how helpful this will be, but On Wed, 30 Jul 2014, Dutch Ingraham wrote: > Greetings: > > I asked this question on freebsd-questions a couple of days ago and have > received zero responses, so I'm posting here and asking the hackers for their > help. Here's the problem: > > I'm having trouble with playing audio CDs; specifically, they won't > play at all. Secondarily, it appears as though if they would play, I > would need to be root to do so. > > I am issuing the command and the CD I believe that for this command to actually produce audio playback requires a hardwired connection between the optical drive and the sound card (or motherboard, if it's an integrated sound card as is the norm these days); this is a dedicated 4-pin cable (or so; it's been a while) that's distinct from the power and (P|S)ATA data cable for the drive. That said, I would mostly expect this cdcontrol command to still spin the drive up even if that connection is not in place... > simply will not play (it does not physically spin). However, if I issue > , the CD will eject; if I issue > , I will receive the tracks information, etc. > So it appears as though communication is happening, but the CD will not > play. If I issue these commands as a regular user, I receive a > "permission denied" message; if run as root, simply nothing happens - > no error, nothing. In fact, issuing returns 0. > > PS - This issue is not limited to CLI commands; VLC will issue the > error "VLC is unable to open the MRL 'cdda:///dev/cd0'. Check the > log for details." There is no log info that I can find. In addition, I > can burn a cd with xfburn, but only as root. My fstab has the standard entry > for CDs. Sound does work, as I can generate white noise. It does not seem very surprising to me that root privilege is required to (e.g.) burn a CD. This is related to at least the permissions on on the /dev/cd0 device node, which I don't think were in the context I trimmed... -Ben From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 31 02:12:35 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CC4FBE33 for ; Thu, 31 Jul 2014 02:12:35 +0000 (UTC) Received: from mout.gmx.com (mout.gmx.com [74.208.4.200]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A693D249D for ; Thu, 31 Jul 2014 02:12:35 +0000 (UTC) Received: from [192.168.1.81] ([172.15.184.248]) by mail.gmx.com (mrgmxus001) with ESMTPSA (Nemesis) id 0LmKKS-1Wdt4K0Tw2-00Zvgt for ; Thu, 31 Jul 2014 04:12:34 +0200 Message-ID: <53D9A676.6050303@gmx.us> Date: Wed, 30 Jul 2014 22:14:14 -0400 From: Dutch Ingraham User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.7.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: Audio CDs Not Playing References: <53D97A50.8090006@gmx.us> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:spWmcW1aUw0XIm3kIh3mBuJLeU+XQR16DemMfx632aNjA7VyW7d jirXxQYEz6ISPfnpWJcKsU+K2ZStYdB5Cseis7KWAnXthCt8dQ4PyfllbCR+PX6vEC+pfv/ 1Cpaa2AaHsskxY2NQE6iQf6lnD+aIAB4vJFvBJnXYfZG2phHNAt8icX8AQoGq5nyqXeIpQM bwGMHtT5z+1A8N6C+WVNA== X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 02:12:36 -0000 On 07/30/2014 09:27 PM, Benjamin Kaduk wrote: > I am not sure how helpful this will be, but > > On Wed, 30 Jul 2014, Dutch Ingraham wrote: > >> Greetings: >> >> I asked this question on freebsd-questions a couple of days ago and >> have received zero responses, so I'm posting here and asking the >> hackers for their help. Here's the problem: >> >> I'm having trouble with playing audio CDs; specifically, they won't >> play at all. Secondarily, it appears as though if they would play, I >> would need to be root to do so. >> >> I am issuing the command and the CD > > I believe that for this command to actually produce audio playback > requires a hardwired connection between the optical drive and the sound > card (or motherboard, if it's an integrated sound card as is the norm > these days); this is a dedicated 4-pin cable (or so; it's been a while) > that's distinct from the power and (P|S)ATA data cable for the drive. > That said, I would mostly expect this cdcontrol command to still spin > the drive up even if that connection is not in place... > >> simply will not play (it does not physically spin). However, if I issue >> , the CD will eject; if I issue >> , I will receive the tracks information, etc. >> So it appears as though communication is happening, but the CD will not >> play. If I issue these commands as a regular user, I receive a >> "permission denied" message; if run as root, simply nothing happens - >> no error, nothing. In fact, issuing returns 0. >> >> PS - This issue is not limited to CLI commands; VLC will issue the >> error "VLC is unable to open the MRL 'cdda:///dev/cd0'. Check the >> log for details." There is no log info that I can find. In addition, I >> can burn a cd with xfburn, but only as root. My fstab has the >> standard entry for CDs. Sound does work, as I can generate white noise. > > It does not seem very surprising to me that root privilege is required > to (e.g.) burn a CD. This is related to at least the permissions on on > the /dev/cd0 device node, which I don't think were in the context I > trimmed... > > -Ben > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > Thanks for the info, Ben. I had actually seen something on that issue in an old (May 2008) daemonforums post while researching this problem. There, the poster suggested using amarok or xmms - programs capable of "digital audio extraction" ( I don't know what that is ) and in which it was implied that cdcontrol was not capable of. xmms is deprecated and I don't want the hundreds of files that come with amarok. I suppose I could try something like audacious, but as noted, vlc doesn't work either, so I'd prefer to not get into the cycle of downloading a bunch of similar programs just to find out there was a simple setting I was missing. Also, the current man page for and the handbook don't adress such a restriction that I could find. Does "digital audio extraction" mean anything to you or is it helpful in ay way? From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 31 02:16:25 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A19C017C for ; Thu, 31 Jul 2014 02:16:25 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7BFAC24C8 for ; Thu, 31 Jul 2014 02:16:25 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-250-191.lns20.per2.internode.on.net [121.45.250.191]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id s6V2GLkk024153 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 30 Jul 2014 19:16:23 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53D9A6F0.3030303@freebsd.org> Date: Thu, 31 Jul 2014 10:16:16 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: suresh gumpula Subject: Re: Allocation/free history References: <53D8FB5D.2060509@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 02:16:25 -0000 On 7/31/14, 12:05 AM, suresh gumpula wrote: > Hi Julian, > Its our proprietary OS called Simple kernel and yes its in the > kernel space allocator. who is "our"? :-) > > Thanks > Suresh > > > On Wed, Jul 30, 2014 at 10:04 AM, Julian Elischer > > wrote: > > On 7/29/14, 1:40 AM, suresh gumpula wrote: > > Hi, > Knowing the PC of an allocation is very usefull in > debugging. Having the > PC hash table and storing the pc hash either with an object > itself( at the > end) or allocate an exra structure to hold the > hash index help us find out who/where an object was > allocated. We > already have something like this in our own operating system > and has been a > useful thing in debugging. > > > what OS is that? > > I assume you are talking about in the kernel? > > From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 31 02:42:52 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2956E453; Thu, 31 Jul 2014 02:42:52 +0000 (UTC) Received: from mail-vc0-x22d.google.com (mail-vc0-x22d.google.com [IPv6:2607:f8b0:400c:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C8F1026DC; Thu, 31 Jul 2014 02:42:51 +0000 (UTC) Received: by mail-vc0-f173.google.com with SMTP id hy10so3273432vcb.4 for ; Wed, 30 Jul 2014 19:42:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=si0dBY5juOJ1tPF1DGvuI3q+fK9qKou+YvX6BbcV1ww=; b=YnHQst2j4arX7rDu8eWaOObgtdAF7k7A0exBq7mg1FVTTA35W9+pS/FHkAsj2moTrE Oh/CjsOx0FKbSmJCV/JByTOXMBRKTcPCIv6wTEPMSKIdNY1AHvyPSsyjufHwfk98zxTo VbgEdkJhgREq8n9/81n2UZpGCuHW43/jT1uhdPq555/fb/DNO8SWf62gPwjfA91ogXQ4 zpMZCYRZM7ZIC95wT0tvUPezNuH9dhcV+ror1gs2wsvrbhX729Mz5dOuRxC351WQEMNr fQDWRIxnaoAPsBAjA1gOT7d5JfJr3J8gh3tl0ok5N+yjN3iFnVACcfmrqy3OBpiM42er 5Hmg== MIME-Version: 1.0 X-Received: by 10.52.61.164 with SMTP id q4mr5473823vdr.89.1406774570830; Wed, 30 Jul 2014 19:42:50 -0700 (PDT) Received: by 10.220.192.68 with HTTP; Wed, 30 Jul 2014 19:42:50 -0700 (PDT) In-Reply-To: <53D9A6F0.3030303@freebsd.org> References: <53D8FB5D.2060509@freebsd.org> <53D9A6F0.3030303@freebsd.org> Date: Wed, 30 Jul 2014 22:42:50 -0400 Message-ID: Subject: Re: Allocation/free history From: suresh gumpula To: Julian Elischer X-Mailman-Approved-At: Thu, 31 Jul 2014 04:24:36 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 02:42:52 -0000 Its NETAPP :-) On Wed, Jul 30, 2014 at 10:16 PM, Julian Elischer wrote: > On 7/31/14, 12:05 AM, suresh gumpula wrote: > > Hi Julian, > Its our proprietary OS called Simple kernel and yes its in the kernel > space allocator. > > > who is "our"? > :-) > > > > Thanks > Suresh > > > On Wed, Jul 30, 2014 at 10:04 AM, Julian Elischer > wrote: > >> On 7/29/14, 1:40 AM, suresh gumpula wrote: >> >>> Hi, >>> Knowing the PC of an allocation is very usefull in debugging. Having >>> the >>> PC hash table and storing the pc hash either with an object itself( at >>> the >>> end) or allocate an exra structure to hold the >>> hash index help us find out who/where an object was allocated. We >>> already have something like this in our own operating system and has >>> been a >>> useful thing in debugging. >>> >> >> what OS is that? >> >> I assume you are talking about in the kernel? >> >> > > From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 31 07:09:19 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B024C85C for ; Thu, 31 Jul 2014 07:09:19 +0000 (UTC) Received: from mail-we0-x230.google.com (mail-we0-x230.google.com [IPv6:2a00:1450:400c:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B37621A1 for ; Thu, 31 Jul 2014 07:09:19 +0000 (UTC) Received: by mail-we0-f176.google.com with SMTP id q58so2308425wes.21 for ; Thu, 31 Jul 2014 00:09:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=whj7KzywBIDWgheVysJjqcPYhs1pk+z9rgVwy+VZW3w=; b=b9Y8XVRkKaT5xRPs0g+LS/07oiTCYSvO7cmh+nFuEITfcUgsRmRa4Sp3mgRgRpzVLN brS54WZMONyMFJdEKq9HAraRUC6FQb8m8ajvnX7sDVq/Rw+B6unsiee3dBSxiD8GhNvJ gU0kjoKrK2GvnXqDeTHyu24ih29u2rSKBKYLqONCMLOjDjPnj4xPlinNv6s5cKYKi/ya QjKvZSl08gkki/0GV2w3OOYq3Fm7Ji34MzUsBa+tzgfR8pr19gt7CcKLdq7qYsJ7EeVp nddvkWR5jRHbuRGoSCsAYUznkOBnzsrutJcQKiYZvMDVry5I9bIcYYKc+68lbCNoDVKi JAMA== X-Received: by 10.194.222.5 with SMTP id qi5mr13526822wjc.62.1406790557163; Thu, 31 Jul 2014 00:09:17 -0700 (PDT) Received: from [192.168.2.100] (dhcp46-187-149-223.eaw.com.pl. [46.187.149.223]) by mx.google.com with ESMTPSA id 10sm11270436wjr.22.2014.07.31.00.09.16 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 31 Jul 2014 00:09:16 -0700 (PDT) Message-ID: <53D9EBA0.5030600@gmail.com> Date: Thu, 31 Jul 2014 09:09:20 +0200 From: Adi User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: FreeBSD 10 swapping Idle processes :/ References: <53D962A7.5090105@gmail.com> <53D97C71.3050806@freebsd.org> <20140731011516.21b1261d@gumby.homeunix.com> In-Reply-To: <20140731011516.21b1261d@gumby.homeunix.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 07:09:19 -0000 Hello > swapping refers to the management of paging at the process level - in > some cases it's more efficient to page-out all the memory from some > idle or less active processes. But this is probably new behavior in FreebSD 10 ? I don't see it before on 9.1 in similar server load. > I just reproduced this by dd'ing a 16 GB file to tmpfs (with 16GB RAM, > and 16GB swap). Some processes were marked as swapped before any > actual paging occurred. \ I forgot write, have small tmpfs too: tmpfs 2,0M 252K 1,8M 12% /var/pgsql/data/pg_stat_tmp Best Regards. From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 31 07:13:16 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 61CE6A2D for ; Thu, 31 Jul 2014 07:13:16 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id 3B2BD2241 for ; Thu, 31 Jul 2014 07:13:15 +0000 (UTC) Received: from [192.168.1.2] (senat1-01.HML3.ScaleEngine.net [209.51.186.5]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id B77E131B08 for ; Thu, 31 Jul 2014 07:13:14 +0000 (UTC) Message-ID: <53D9EC9E.3020207@freebsd.org> Date: Thu, 31 Jul 2014 03:13:34 -0400 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: FreeBSD 10 swapping Idle processes :/ References: <53D962A7.5090105@gmail.com> <53D97C71.3050806@freebsd.org> <20140731011516.21b1261d@gumby.homeunix.com> <53D9EBA0.5030600@gmail.com> In-Reply-To: <53D9EBA0.5030600@gmail.com> X-Enigmail-Version: 1.6 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="jPmRdmE0cXMEdEngCpNcnTvFaP1UKKtO1" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 07:13:16 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --jPmRdmE0cXMEdEngCpNcnTvFaP1UKKtO1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 2014-07-31 03:09, Adi wrote: > Hello >=20 >> swapping refers to the management of paging at the process level - in >> some cases it's more efficient to page-out all the memory from some >> idle or less active processes.=20 >=20 > But this is probably new behavior in FreebSD 10 ? > I don't see it before on 9.1 in similar server load. >=20 >=20 >=20 >> I just reproduced this by dd'ing a 16 GB file to tmpfs (with 16GB RAM,= >> and 16GB swap). Some processes were marked as swapped before any >> actual paging occurred. \ >=20 > I forgot write, have small tmpfs too: >=20 > tmpfs 2,0M 252K 1,8M 12% > /var/pgsql/data/pg_stat_tmp >=20 > Best Regards. >=20 >=20 >=20 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o= rg" >=20 If you reboot and don't use the tmpfs, do the symptoms go away? --=20 Allan Jude --jPmRdmE0cXMEdEngCpNcnTvFaP1UKKtO1 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJT2eyhAAoJEJrBFpNRJZKfkK0P/1fx9dOrIgua3EbD+3y5pAaR aB+Epjd+AhQCcZ7/IF3CP7iE/uu7jTrdkzOc4RRfhWQl5g8CrkEMMIWGBbliOWqL 3dTJNWNNwRwOt5UJ4+pdfJ3tS0WJTRnw3BpJLSNk2MyvIvMYx+WbnmfWxY5wHF8I fPSPuveQHlupdYvZIQFtpljbMPPpw+7Ae8b+OAoV23JG86qusQnhkBafSUz7i0RJ wMjLpMJFscv7eW/kyaaj46qdyCch7y7M5WJDVo2GO+VMiKDY2rABy3aN0shd/d/x YBamJm2uwPcHc2CUuW41SvGtAD6tOhp1pTx4JmtsRbxzde18g9bBgd1/KLhIgRKD MJTZCZ/S5i76FNQ+1/UBi1WG2Op3idYn2M8qIxU3sALZYEPM3qtVUlpox5nef2Mx V4gcfvCrBBt4Wp3ca7zi8CDADCMp3qFnItl/jzbgLg92Lq9DPrsX7aULB20QRFnZ drtKKigYkD510g/Xf248mz2wmvN/ypC2PAarUtrXlApXqV8/I6hV8WexvWQX2Hkz tIEguFHKmxIn5fjRrTLYc9OGBNCrp9QisWntVMzls9oNFD8v5akuOgiAn6yQkkD7 E9eEupu6Epd6CLO+N4j1pnBbU0algXvr/7P2llZifuUvfVmEPE39hZFBLbqqqpaX pPCMvtVFhCKDvU9Fn32e =hl6X -----END PGP SIGNATURE----- --jPmRdmE0cXMEdEngCpNcnTvFaP1UKKtO1-- From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 31 07:29:12 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8044D154 for ; Thu, 31 Jul 2014 07:29:12 +0000 (UTC) Received: from mail-wg0-x232.google.com (mail-wg0-x232.google.com [IPv6:2a00:1450:400c:c00::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 197042442 for ; Thu, 31 Jul 2014 07:29:11 +0000 (UTC) Received: by mail-wg0-f50.google.com with SMTP id n12so2340676wgh.9 for ; Thu, 31 Jul 2014 00:29:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=MUL5b8UMRcHRsHxj4ghNIiSG5qQdDGFhqg3vMEVhusw=; b=RhEjxx4ymX5s4IaBj50bW00auVvXAxOEeo0c6qOk8hoDiEBSeUkp67/hfsCeA0Elxz X9oXwxbzM/DTXZnb+1ycYXMNGFw3H2xtF+3XsR+zfV9s4yIXMjlFlYQq5HvnjQCJKYPJ yObdPCTpSw0xGr2fij54CCloNjZ8jQubcM1VkOrfiVu3q/n/gQH55yuifl7w9rlOr07M 11QpGP9bPLGHp5jE8CNXZhNVwIAQRB6Vcf0mWDJ2uzoO1AUwRs8nDzJ13zhfPNu7Ig+L m0gyyA4BEIE9HA8k1tjpcIT0Rqq8awNoFE6MqnO74WYHOzJ4thrHVa/jnbuvAE3goVz5 AL5Q== X-Received: by 10.194.62.140 with SMTP id y12mr14161641wjr.27.1406791750305; Thu, 31 Jul 2014 00:29:10 -0700 (PDT) Received: from [192.168.2.100] (dhcp46-187-149-223.eaw.com.pl. [46.187.149.223]) by mx.google.com with ESMTPSA id fb12sm11351401wjc.43.2014.07.31.00.29.09 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 31 Jul 2014 00:29:09 -0700 (PDT) Message-ID: <53D9F04C.6080607@gmail.com> Date: Thu, 31 Jul 2014 09:29:16 +0200 From: Adi User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: FreeBSD 10 swapping Idle processes :/ References: <53D962A7.5090105@gmail.com> <53D97C71.3050806@freebsd.org> In-Reply-To: <53D97C71.3050806@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 07:29:12 -0000 Hello > That looks odd, the output of top shows 0 swap usage, but also 0 > resident memory for most of the processes... Yes very odd. After server restart or after restart all deamons processes working normally (in memory not in SWAP) for few (1-3) hours. Then start odd swapping. And this is not temporary (for the possible disappearance of the "swapping trigger") only lasting behavior. Process is moved from SWAP to memory if is not idle. And back to SWAP after idle few seconds. Now after 10 h from write this thread still 84 processes is in swap. procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad0 ad1 in sy cs us sy id 0 0 84 14863M 682M 315 0 0 0 331 546 0 0 89 1769 722 0 0 100 After restart all processes procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad0 ad1 in sy cs us sy id 0 0 0 13908M 1213M 318 0 0 0 335 547 0 0 89 1775 722 0 0 100 SWAP partition is gmirrored - it has no significance ;) # Device Mountpoint FStype Options Dump Pass# /dev/mirror/swap none swap sw 0 0 /dev/mirror/root / ufs rw 1 1 /dev/mirror/tmp /tmp ufs rw 2 2 /dev/mirror/var /var ufs rw 2 2 /dev/mirror/usr /usr ufs rw 2 2 /dev/mirror/home /home ufs rw 2 2 tmpfs /var/pgsql/data/pg_stat_tmp tmpfs rw,size=2m,uid=pgsql,gid=pgsql,nosuid 0 0 Similar problem is in thread: http://lists.freebsd.org/pipermail/freebsd-hackers/2014-January/044040.html Best Regards From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 31 07:57:22 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 64BADBB6 for ; Thu, 31 Jul 2014 07:57:22 +0000 (UTC) Received: from mail-we0-x235.google.com (mail-we0-x235.google.com [IPv6:2a00:1450:400c:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F026B26C8 for ; Thu, 31 Jul 2014 07:57:21 +0000 (UTC) Received: by mail-we0-f181.google.com with SMTP id k48so2382043wev.12 for ; Thu, 31 Jul 2014 00:57:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=4zlVVbFGgvr2vrLA79t2D0DVaDOd+bU1/xjcABfQXr4=; b=elPqf+SlyCz3HVUC85Y9shoODP+EamtLM49fw/VQREfdx4iJDXFp5H31SiIgET76f+ DzsdM3zkkRVFxKChxTrvyspFHib+O6L1NQPDtJlDVia8dQMQ+NN8mk+aD1ipmdGWbi7b Dw2sX7zPV4cpxWGhK2Oq/ZrUdsvV9pMj6UwZxjA3jsWj4W5k62p4BYpGA/MggI3Su/QQ sxQ/Cv+kK2bmv7BHK6cDPHlVxWINFX5hCuT7ax9ximwHpYHBlSSIOyfvl9DwwANkvRUQ +b0ETgTrWZHnY1yutdJ2DYbMio1s9LOOGH2lyAbrFSYIBvk1NXRC+RXwZq3QeaGqiEVc x9HA== X-Received: by 10.180.183.167 with SMTP id en7mr14894009wic.6.1406793439940; Thu, 31 Jul 2014 00:57:19 -0700 (PDT) Received: from [192.168.2.100] (dhcp46-187-149-223.eaw.com.pl. [46.187.149.223]) by mx.google.com with ESMTPSA id w10sm17950049wie.22.2014.07.31.00.57.18 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 31 Jul 2014 00:57:19 -0700 (PDT) Message-ID: <53D9F6EB.4000303@gmail.com> Date: Thu, 31 Jul 2014 09:57:31 +0200 From: Adi User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: FreeBSD 10 swapping Idle processes :/ References: <53D962A7.5090105@gmail.com> <53D97C71.3050806@freebsd.org> <20140731011516.21b1261d@gumby.homeunix.com> <53D9EBA0.5030600@gmail.com> <53D9EC9E.3020207@freebsd.org> In-Reply-To: <53D9EC9E.3020207@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 07:57:22 -0000 Hello > If you reboot and don't use the tmpfs, do the symptoms go away? I can try but this is only 2 MB tmpfs. And only 2xx KB used. I think about change kern.maxusers: 2373 to, for example: 500 ? Maybe this will reduce limits for open files and proportional other settings and will stop swapping? Next question vm.swap_total: 34359734272 vm.swap_reserved: 8904851456 What is swap_reserved? I looked on old server: kern.maxusers: 384 vm.swap_reserved: 108970188800 vm.swap_total: 17179869184 reserved is > total (sic!) Maybe this reseved is from login.conf klient:\ :datasize=128M:\ :vmemoryuse=180M:\ Mayby I should reduce this settings ? But old server (16 GB) is not swapping (even reserved is very high) ;/ Best Regards From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 31 08:08:42 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 687F3F05 for ; Thu, 31 Jul 2014 08:08:42 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 2E73127B4 for ; Thu, 31 Jul 2014 08:08:42 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id E26A120E7088C; Thu, 31 Jul 2014 08:08:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.2 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id 9D9B020E70886; Thu, 31 Jul 2014 08:08:35 +0000 (UTC) Message-ID: <3526D2C767B54D3597CC1380AAC5E07E@multiplay.co.uk> From: "Steven Hartland" To: "Adi" , References: <53D962A7.5090105@gmail.com> Subject: Re: FreeBSD 10 swapping Idle processes :/ Date: Thu, 31 Jul 2014 09:08:40 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 08:08:42 -0000 Try patching your sources with: MFC r265418 - vm_pageout stuck pages MFC r260567 - vm_pageout stuck pages http://svnweb.freebsd.org/base?view=revision&revision=265944 http://svnweb.freebsd.org/base?view=revision&revision=265945 And see if that makes any difference. Regards Steve ----- Original Message ----- From: "Adi" To: Sent: Wednesday, July 30, 2014 10:24 PM Subject: FreeBSD 10 swapping Idle processes :/ > Hello > > i7-4770 , 32 GB RAM, FreeBSD 10.0-RELEASE-p7, AMD64, UFS2 > > Problem: Many idle processes is swapping :( > > Similar system configurations and deamons settings on my old server on > FBSD 9.1 with 16 GB was working many years. Maybe some login.conf changed. > > Probably I made mistakes in several settings. > > Details info (long) is available at > > http://pastebin.com/Y9mJUD6F > > OR RAW (if numbered listing not working well) > http://pastebin.com/raw.php?i=Y9mJUD6F > > > Can You help me fix settings? > > Thanks for Your help and time. > > Best Regards. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 31 09:25:17 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 338CED30 for ; Thu, 31 Jul 2014 09:25:17 +0000 (UTC) Received: from nm7-vm2.bullet.mail.ir2.yahoo.com (nm7-vm2.bullet.mail.ir2.yahoo.com [212.82.96.127]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 446492092 for ; Thu, 31 Jul 2014 09:25:15 +0000 (UTC) Received: from [212.82.98.51] by nm7.bullet.mail.ir2.yahoo.com with NNFMP; 31 Jul 2014 09:22:40 -0000 Received: from [46.228.39.85] by tm4.bullet.mail.ir2.yahoo.com with NNFMP; 31 Jul 2014 09:22:40 -0000 Received: from [127.0.0.1] by smtp122.mail.ir2.yahoo.com with NNFMP; 31 Jul 2014 09:22:40 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.es; s=s1024; t=1406798560; bh=uMRbvj4kL7skvzblRKtQmH4jtgrnSWS0Ir26bUIt+EU=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:Date:From:To:Subject:Message-Id:In-Reply-To:References:X-Mailer:Disposition-Notification-To:Mime-Version:Content-Type:Content-Transfer-Encoding; b=yt++lIY57la2NjaVaSiAGy+BasT1WTsjGYsNbBcm5FR+0a8Qxg7Itm269SZNCPH10OnxBzJWM+revckd4f8LfkcwYEogBwCyB75hK/vWl+vpu+AY0gEViYKSw6Ndg1p4sgz2GQogJ4ybPdtq+fTnATgl/kW24TVXleaf4ux35jQ= X-Yahoo-Newman-Id: 283595.59705.bm@smtp122.mail.ir2.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: LgtEJtQVM1mV6nU862IZFsDoZIYjvUhGafbpkt4pY1IF2AO dj8um4moDKpNbBBeZ60tZSWbQ8phZRaZ3GlOAIv4CvQcNqzuTG4p8F8c8P58 4yoetwXYRCGZnYH3QTKrF9tayOM2ddDY7zxjIVzk9l6T8Wet2rTUJF8jStyg K4fuVtb7kaM5jW4rHjtn5RqOPAh0sv9Z_C6pzNDe5clTlq6GafXC1gOKN.WK 3nHmn0r0WOWk60hKMe.Nbx4rdKERmZNvU0NPv1C8n06fjocVqQCE0FQBq37x g0qhJDbKOCX4anvrXNDOVHSoCGjpXzCtp9yYklyrbEOXI.dVoZqBRLYOZS36 K8wfktbDXsBfE5X4o6ZqOviB2l12BSzBbWPsY1.VgAPXo3Tj2CtW6dZs.dYZ nYo8ixbqCtxIOgkG_8i4MKF8bOmruLbsH3labrTKsBHQTLfHY3tLA.bdxxEp S406hEdDluOiHDVyvpwWQWGFTpcTC.NN4ZkuyV9tBDXAArwoMi8.565vlneh S9aPhmfTUC2XCiiRNLG3F2PgO2SqEuZpYryDvdkyDy0TU_rzpjViKW9TroJ0 - X-Yahoo-SMTP: mX392iiswBAeJNdO_s.EW62LZDJR Date: Thu, 31 Jul 2014 11:22:42 +0200 From: Eduardo Morras To: freebsd-hackers@freebsd.org Subject: Re: Audio CDs Not Playing Message-Id: <20140731112242.2ddeb8c8a37f4077f1a03260@yahoo.es> In-Reply-To: <53D97A50.8090006@gmx.us> References: <53D97A50.8090006@gmx.us> X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd9.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 09:25:17 -0000 On Wed, 30 Jul 2014 19:05:52 -0400 Dutch Ingraham wrote: > Greetings: > > I asked this question on freebsd-questions a couple of days ago and > have received zero responses, so I'm posting here and asking the > hackers for their help. Here's the problem: > > I'm having trouble with playing audio CDs; specifically, they won't > play at all. Secondarily, it appears as though if they would play, I > would need to be root to do so. > > My system stats are as follows: : > FreeBSD dutch.freebsd.net 10.0-RELEASE-p7 FreeBSD 10.0-RELEASE-p7 #0: > Tue Jul 8 06:37:44 UTC 2014 > root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 > > > I am issuing the command and the CD > simply will not play (it does not physically spin). However, if I > issue , the CD will eject; if I issue > , I will receive the tracks information, etc. > So it appears as though communication is happening, but the CD will > not play. If I issue these commands as a regular user, I receive a > "permission denied" message; if run as root, simply nothing happens - > no error, nothing. In fact, issuing returns 0. Is your user in operator group? Check with id and register your user in operator group My /dev/cd0: crw-r----- 1 root operator 0x4b Jul 31 09:43 cd0 Perhaps creating a link in /dev to cd0 with root:wheel u:g may work too --- --- Eduardo Morras From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 31 12:35:28 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 88BDFB0 for ; Thu, 31 Jul 2014 12:35:28 +0000 (UTC) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "wonkity.com", Issuer "wonkity.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 233F42622 for ; Thu, 31 Jul 2014 12:35:27 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.9/8.14.9) with ESMTP id s6VCZK5I003707 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 31 Jul 2014 06:35:20 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.9/8.14.9/Submit) with ESMTP id s6VCZJ5U003704; Thu, 31 Jul 2014 06:35:19 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Thu, 31 Jul 2014 06:35:19 -0600 (MDT) From: Warren Block To: Benjamin Kaduk Subject: Re: Audio CDs Not Playing In-Reply-To: Message-ID: References: <53D97A50.8090006@gmx.us> User-Agent: Alpine 2.11 (BSF 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Thu, 31 Jul 2014 06:35:20 -0600 (MDT) Cc: freebsd-hackers@freebsd.org, Dutch Ingraham X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 12:35:28 -0000 On Wed, 30 Jul 2014, Benjamin Kaduk wrote: > I am not sure how helpful this will be, but > > On Wed, 30 Jul 2014, Dutch Ingraham wrote: > >> Greetings: >> >> I asked this question on freebsd-questions a couple of days ago and have >> received zero responses, so I'm posting here and asking the hackers for >> their help. Here's the problem: >> >> I'm having trouble with playing audio CDs; specifically, they won't >> play at all. Secondarily, it appears as though if they would play, I >> would need to be root to do so. >> >> I am issuing the command and the CD > > I believe that for this command to actually produce audio playback requires a > hardwired connection between the optical drive and the sound card (or > motherboard, if it's an integrated sound card as is the norm these days); > this is a dedicated 4-pin cable (or so; it's been a while) that's distinct > from the power and (P|S)ATA data cable for the drive. That said, I would > mostly expect this cdcontrol command to still spin the drive up even if that > connection is not in place... Yes, 'cdcontrol play' is effectively the same as pressing the play button on a standalone CD player. It plays the CD, sending output to the hardware audio outputs, which must be connected to the motherboard or sound card audio inputs. Most IDE CD players had those audio connectors, although there was at least one LG player where they were present but not connected.. Most SATA CD drives do not have audio outputs at all, and audio must be extracted from the digital data connection to the computer. There used to be an example in the Handbook, although I can't find it now. From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 31 12:48:01 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E066BA1A for ; Thu, 31 Jul 2014 12:48:01 +0000 (UTC) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 791BE27B3 for ; Thu, 31 Jul 2014 12:48:01 +0000 (UTC) Received: by mail-wi0-f181.google.com with SMTP id bs8so4070535wib.14 for ; Thu, 31 Jul 2014 05:47:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=qsWUMvoH9SPAXl99CkY4p2tD3PsDxJYGanpUdy/HhH8=; b=G0k6jHLstSxsG1ookE3OSIESxXSpKcExAGLCFG2CfFopZ66h0dOlUtmlWXIfmBMwqQ my4n2szI51PjxlX5OSopQ6XwLFJPFZ4qtt6oXI2lse77CRHRK2LnDRDpRw0jDwloek52 kSPz/CKmS0ydT8zMuJA7Kg+WeQj1rsPt3EjWoFlbU20oLAYLn61LAzbImjC1cOCBG5DK g7ukpU5WDhdSK85OWhEesCNRWlMgauqIBrXl5pxJf+/UXYdAv8s1sO+jcgnpLQ7S9frw ESKsrP20CjohrRFA0FntNSxNX6Z8+XHwIHIpVpOttAnCs/yONzzaOsDK6MDIhPFNYoxY aYQA== X-Received: by 10.180.99.34 with SMTP id en2mr17195919wib.34.1406810879631; Thu, 31 Jul 2014 05:47:59 -0700 (PDT) Received: from [192.168.2.100] (dhcp46-187-149-223.eaw.com.pl. [46.187.149.223]) by mx.google.com with ESMTPSA id q2sm8482466wia.19.2014.07.31.05.47.58 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 31 Jul 2014 05:47:58 -0700 (PDT) Message-ID: <53DA3B2B.8020003@gmail.com> Date: Thu, 31 Jul 2014 14:48:43 +0200 From: Adi User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: FreeBSD 10 swapping Idle processes :/ References: <53D962A7.5090105@gmail.com> <3526D2C767B54D3597CC1380AAC5E07E@multiplay.co.uk> In-Reply-To: <3526D2C767B54D3597CC1380AAC5E07E@multiplay.co.uk> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 12:48:01 -0000 Hello > Try patching your sources with: > MFC r265418 - vm_pageout stuck pages > MFC r260567 - vm_pageout stuck pages > > http://svnweb.freebsd.org/base?view=revision&revision=265944 > http://svnweb.freebsd.org/base?view=revision&revision=265945 This should help. But a very long time didn't compile sources. Is it any chance to avoid compile? Best Regards. From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 31 14:04:15 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8705F5B4 for ; Thu, 31 Jul 2014 14:04:15 +0000 (UTC) Received: from dmz-mailsec-scanner-2.mit.edu (dmz-mailsec-scanner-2.mit.edu [18.9.25.13]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2648A2400 for ; Thu, 31 Jul 2014 14:04:14 +0000 (UTC) X-AuditID: 1209190d-f79c06d000002f07-e8-53da4cdc7a9c Received: from mailhub-auth-4.mit.edu ( [18.7.62.39]) (using TLS with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by dmz-mailsec-scanner-2.mit.edu (Symantec Messaging Gateway) with SMTP id 32.23.12039.CDC4AD35; Thu, 31 Jul 2014 10:04:12 -0400 (EDT) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id s6VE4B3q032007; Thu, 31 Jul 2014 10:04:12 -0400 Received: from multics.mit.edu (system-low-sipb.mit.edu [18.187.2.37]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id s6VE4AQY018686 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 31 Jul 2014 10:04:11 -0400 Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id s6VE49jc026571; Thu, 31 Jul 2014 10:04:09 -0400 (EDT) Date: Thu, 31 Jul 2014 10:04:09 -0400 (EDT) From: Benjamin Kaduk To: Dutch Ingraham Subject: Re: Audio CDs Not Playing In-Reply-To: <53D9A676.6050303@gmx.us> Message-ID: References: <53D97A50.8090006@gmx.us> <53D9A676.6050303@gmx.us> User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrDIsWRmVeSWpSXmKPExsUixG6nrnvH51awwczLJhbbN/9jtNj/UcmB yWPGp/ksHs92r2MOYIrisklJzcksSy3St0vgyphy6gZbwRu+ijULF7A0MJ7g7mLk5JAQMJHo vHWVBcIWk7hwbz1bFyMXh5DAbCaJzRM/sUI4GxklPl/bxwzhHGKS2PV/KROE08AosebkPSaQ fhYBbYlPr2eyg9hsAioSM99sBJrFwSEioCDRcCoKJMwsIC9xYfMhRhBbWEBJYt+1DrDVnAJq Epf3LWADsXkFHCVa//wHGyMkUCoxec0HsLiogI7E6v1TWCBqBCVOznzCAjHTUuLcn+tsExgF ZyFJzUKSWsDItIpRNiW3Sjc3MTOnODVZtzg5MS8vtUjXSC83s0QvNaV0EyMoUDkleXcwvjuo dIhRgINRiYf3h8nNYCHWxLLiytxDjJIcTEqivDO/A4X4kvJTKjMSizPii0pzUosPMUpwMCuJ 8F50vBUsxJuSWFmVWpQPk5LmYFES531rbRUsJJCeWJKanZpakFoEk5Xh4FCS4D3vDdQoWJSa nlqRlplTgpBm4uAEGc4DNPw4SA1vcUFibnFmOkT+FKOilDjvR5CEAEgiozQPrheWSF4xigO9 Isx7DaSKB5iE4LpfAQ1mAhr8/NZ1kMEliQgpqQbGdsf6Bf17L59mc1v4pFp3OduL4rbqp6kO fzWYDvvb3l3lI7Wl8UGTj1Lw3J5XG1NOOeSG3HB2vxbtr1oeYLwxnuFs3YGS23qn7lzqvO4e Mvt8rZ3ArZaSpUeuNX6bERpVH8zxtGFv8if2VrXIVUDrEifPFeewjwk/OMdyqssqs3lvJ9nv uq3EUpyRaKjFXFScCABzGeRH/wIAAA== Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 14:04:15 -0000 On Wed, 30 Jul 2014, Dutch Ingraham wrote: > Thanks for the info, Ben. I had actually seen something on that issue in an > old (May 2008) daemonforums post while researching this problem. There, the > poster suggested using amarok or xmms - programs capable of "digital audio > extraction" ( I don't know what that is ) and in which it was implied that > cdcontrol was not capable of. > > xmms is deprecated and I don't want the hundreds of files that come with > amarok. I suppose I could try something like audacious, but as noted, vlc > doesn't work either, so I'd prefer to not get into the cycle of downloading a > bunch of similar programs just to find out there was a simple setting I was > missing. > > Also, the current man page for and the handbook don't adress such > a restriction that I could find. > > Does "digital audio extraction" mean anything to you or is it helpful in ay > way? I think I know what it means. The easiest way to think about it is probably to realize that in order to play audio, the bits recorded on the CD have to make it to the digital-analog converter somehow. If there's no direct line from the CD drive to the sound card (as we were discussing in the trimmed text), then that data has to be moved around in software. Some tool is needed to extract the audio data from the CD drive, and some tool is needed to send those off to the sound card; these tools can be the same, but need not be. I tend to use cdparanoia for the first step, and the play(1) utility provided by audio/sox for the second step, leaving the bits around on my (sizable) hard drive for later use. It sounds like xmms and amarok can combine the two steps into one, without leaving the bits on disk as an intermediate; I'm not sure offhand whether there are more lightweight utilities that can also do so. -Ben From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 31 14:10:57 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 317E47DB for ; Thu, 31 Jul 2014 14:10:57 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id EB921244F for ; Thu, 31 Jul 2014 14:10:56 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 5AD0120E7088D; Thu, 31 Jul 2014 14:10:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.2 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id 13C4B20E7088B; Thu, 31 Jul 2014 14:10:49 +0000 (UTC) Message-ID: From: "Steven Hartland" To: "Adi" , References: <53D962A7.5090105@gmail.com> <3526D2C767B54D3597CC1380AAC5E07E@multiplay.co.uk> <53DA3B2B.8020003@gmail.com> Subject: Re: FreeBSD 10 swapping Idle processes :/ Date: Thu, 31 Jul 2014 15:10:58 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 14:10:57 -0000 ----- Original Message ----- From: "Adi" To: Sent: Thursday, July 31, 2014 1:48 PM Subject: Re: FreeBSD 10 swapping Idle processes :/ > Hello > >> Try patching your sources with: >> MFC r265418 - vm_pageout stuck pages >> MFC r260567 - vm_pageout stuck pages >> >> http://svnweb.freebsd.org/base?view=revision&revision=265944 >> http://svnweb.freebsd.org/base?view=revision&revision=265945 > > This should help. But a very long time didn't compile sources. > Is it any chance to avoid compile? Kernel here takes ~5mins max on a decent machine so it shouldn't take that long. You don't need a full world build just kernel. Regards Steve From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 31 15:55:51 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2BF00DE0 for ; Thu, 31 Jul 2014 15:55:51 +0000 (UTC) Received: from smtp9.server.rpi.edu (gateway.canit.rpi.edu [128.113.2.229]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DA73521FE for ; Thu, 31 Jul 2014 15:55:50 +0000 (UTC) Received: from smtp-auth2.server.rpi.edu (route.canit.rpi.edu [128.113.2.232]) by smtp9.server.rpi.edu (8.14.3/8.14.3/Debian-9.4) with ESMTP id s6VFtltb027913; Thu, 31 Jul 2014 11:55:47 -0400 Received: from smtp-auth2.server.rpi.edu (localhost [127.0.0.1]) by smtp-auth2.server.rpi.edu (Postfix) with ESMTP id E81CD180FA; Thu, 31 Jul 2014 11:55:46 -0400 (EDT) Received: from [128.113.24.47] (gilead-qc124.netel.rpi.edu [128.113.124.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: drosih) by smtp-auth2.server.rpi.edu (Postfix) with ESMTPSA id 8695918072; Thu, 31 Jul 2014 11:55:46 -0400 (EDT) From: "Garance A Drosehn" To: "Zaro Korchev" Subject: Re: [GSoC] Machine readable output from userland utilities Date: Thu, 31 Jul 2014 11:55:45 -0400 Message-ID: <1F6CE510-95F2-46C8-AB0E-D5987CF13760@rpi.edu> In-Reply-To: <4C29B30D-6833-4F0E-B071-C8EA215C0A17@mail.bg> References: <0FCB749A-67F7-4C2F-AAC1-32D0BD67B502@theravensnest.org> <4C29B30D-6833-4F0E-B071-C8EA215C0A17@mail.bg> MIME-Version: 1.0 Content-Type: text/plain; format=flowed X-Mailer: MailMate (1.7.2r3905) X-Virus-Scanned: ClamAV using ClamSMTP X-Bayes-Prob: 0.0001 (Score 0, tokens from: outgoing, @@RPTN) X-Spam-Score: 0.00 () [Hold at 15.10] X-CanIt-Incident-Id: 02Mx3TLuc X-CanIt-Geo: ip=128.113.124.17; country=US; region=New York; city=Troy; latitude=42.7495; longitude=-73.5951; http://maps.google.com/maps?q=42.7495,-73.5951&z=6 X-CanItPRO-Stream: outgoing X-Canit-Stats-ID: Bayes signature not available X-Scanned-By: CanIt (www . roaringpenguin . com) on 128.113.2.229 Cc: Jonathan Anderson , Eitan Adler , Alfred Perlstein , David Chisnall , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 15:55:51 -0000 Apologies for quoting so much, but I do have something to add way down at the end of this, and I expect few people remember the original message. I see 'soc-status@freebsd.org' in one of the messages quoted here. Should I be sending this to that list too? Way back on 3 Jun 2014, Zaro Korchev wrote: > Hi everybody > > I see there are several different ideas about how the output > format should be specified. > > I first started using an option named -O with the idea that > this can be changed when the best variant is decided. > > > There is the idea with the environment variable that we > discussed with Eitan: > > On 29 May 2014 at 18:31, Eitan Adler wrote : >> On 29 May 2014 05:12, Zaro Korchev wrote: >>> I thought about whether it is better to use an option or environment >>> variable. I did it with option because it is easier to switch an >>> option on/off. It appears that the flag -O is free in almost all >>> tools. I have no problem making the use an environment variable. >> >> My concern is that future standards may require this option (or at >> least, would be precluded from using it). In addition, it may >> conflict with non-base utilities, such as coreutils ones. > > ---- > > There is the pipelining idea of Jonathan: > > On 23 May 2014 at 16:27, Jonathan Anderson wrote : >> Imagine: >> >> $ ifconfig | filterBy "ether" " 3c:07:.*" | sortBy "ether" | output >> my_ifconfig.format # or "json" or "xml" or ... >> >> A pipeline of little tools, each doing one thing well: how much more >> unix can you get? Currently, every command-line tool has to do two or >> three things: >> 1. its primary job, >> 2. output some arbitrary text format (that you're never allowed to >> change because other tools scrape it) and >> 3. (optionally) parse arbitrary text formats generated by users or >> some other tool. >> >> Task 2 is annoying: in order to usefully query command-line tools, I >> have >> to write a parser. The tool has binary data, I want binary data, but >> we >> have to go through a dump/parse dance in order for me to get the >> data. >> This is the approach (again, from Plan 9) that brings you Linux >> sysfs. >> Perhaps David would now like to comment on his cross-platform "how >> much >> battery do I have" experience. :) >> >> Task 3 isn't just annoying, however, it's risky. If every tool >> implements >> its own string protocol parsing, we greatly increase the risk of >> unnoticed >> bugs. Better to centralize as much string parsing as possible into a >> single library, which can be rigorously analyzed (and optimized!). >> >> Imagine if geom didn't have to speak XML natively, but rather used a >> supported-everywhere-in-base data structure that users could convert >> into >> XML if they need it. Desktop applications are going to start >> requiring >> structured data passing via kdbus-like interfaces (currently based on >> GLib's GVariant), so we might as well have a structured >> representation >> that we like and are able to provide ABI support for (and, in the >> kdbus >> case, can possibly be converted to/from GVariant as required). > > ---- > > There is the long option idea of David: > >> From : David Chisnall >> Date : 2 June 2014 16:31:11 >> To : Zaro Korchev >> Cc : soc-status@freebsd.org >> >> On 2 Jun 2014, at 12:43, Zaro Korchev wrote: >> >>> At the moment both ls and vmstat are told to output JSON by >>> specifying >>> the -O option. However as I discussed with my mentor, this will be >>> changed. The idea is to use an environment variable instead of >>> the -O flag. >> >> I don't like the idea of using an environment variable, because this >> is >> something that you might want to control on a per-command basis >> within >> a pipeline. Especially with respect to incremental adoption, if you >> have some commands that will emit their default format, which is sent >> to sed / awk / whatever, and some that will emit json natively, you >> don't want to suddenly have the output format from the legacy tools >> change once they gain machine-readable output support. >> >> One *very* important thing to do is standardise the command-line flag >> that is used to specify the output format. This may involve also >> converting some of the tools to use getopt_long if they don't already >> (lots of tools already use most single-digit options, so there's no >> possibility to define a single-letter flag that will be useable on >> all tools). >> >>> I understand your concerns about multi-threading. The idea is to >>> have functions that serialize the object in an allocated buffer as >>> it is constructed. Here is a more detailed example of what I mean: >> >> It would be better to has some stream output API as the default. If >> one back end only supports writing to buffers, then you can add an >> extra alloc / write / free sequence to hide it, but it would be good >> if the interface understands writing directly to file descriptors. >> If the back end natively supports streaming, then you don't need >> to buffer the output. > > > As you have more experience I believe you can decide which is the > best. > > I like the pipelining and the long option idea the most. At the moment > I'm working on porting more tools to use libsol so this decision is > not urgent. I can change how the format is specified easily. Last night it occurred to me that another way to do this would be to have a second version of each command. So "wc" and "mor-wc", "uptime" and "mro-uptime", etc. This might seem like some horrible bloat, but you could have the two commands hard-linked to each other, and the behavior of the executable would dependent upon what name it was executed as. The amount of disk space used would be pretty close to the same used by any of the other methods. Advantages of this: 1. The "mro" (machine-readable-output) version can start fresh with whatever command-line arguments it wants to support. You can make *all* "mro" commands use the same parameters and environment variables for formatting options. 2. It is trivial for a human to see if there is an "mro" version of a command that they're going to use in some script. They don't have to go reading through the 20-screen man page for 'ps' to see if it can be made to generate MRO. 3. It is trivial for some cross-platform package to use autoconfig to find out if a given OS has the "mro" version of whatever commands they wanted. If it seems pretty ugly to clutter /bin and /usr/bin with two versions of many commands, the MRO versions could also be moved into separate directories from the original commands. /bin/mro/df, /bin/mro/ps, etc. If someone *wants* to type in MRO commands at the command line, they can add that to their PATH. But I expect most usage of these commands will be in scripts, and scripts can set their own paths. (and yes, I think would still want separate command names with a common prefix even if all the MRO commands are in their own directory. I don't want someone to get the MRO-version of a command by mistake if they put the MRO directory before the standard directory). -- Garance Alistair Drosehn = drosih@rpi.edu Senior Systems Programmer or gad@FreeBSD.org Rensselaer Polytechnic Institute; Troy, NY; USA From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 31 16:16:21 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F0180B0F for ; Thu, 31 Jul 2014 16:16:21 +0000 (UTC) Received: from mail-wg0-x234.google.com (mail-wg0-x234.google.com [IPv6:2a00:1450:400c:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 889002591 for ; Thu, 31 Jul 2014 16:16:21 +0000 (UTC) Received: by mail-wg0-f52.google.com with SMTP id a1so2984882wgh.11 for ; Thu, 31 Jul 2014 09:16:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=MzWlX1qGQi1AzHdKz9RXa0NxkEZrPNavFI2UV7l3j8Q=; b=T2onuMcmeFZmkc7BdmK/Y4XSCYm3ZhCnjei91X+BPLePysaEUDMAWPbi6NrAeGmkug J9w5k33glksaIlNJcXIYwiAL+CFNgQ2c7vAvVgpyYgm4GF+U4K21aeV3UXf/Jh2ohVfS Lwiixy/Jml3C0PZBScwAeSkXGQeGLN1DhlqalzSajTjmpyLZfPb5sTnEv79WHNo7E2bt XYYrLb9LP/Bm7LblDAz3tR3b9iS7mtQExIyMLvP0nvtpZ/AbSkAqoco1ZLzE7wCDjyKU lsZ6p1OZ8FjSzTPodiGqaPGh4tWt5Gy49y9e2GtNxVd49285A/YBZY6HyWvWU8b8z/oi 0aQA== X-Received: by 10.194.202.165 with SMTP id kj5mr18814960wjc.50.1406823379081; Thu, 31 Jul 2014 09:16:19 -0700 (PDT) Received: from [192.168.2.100] (dhcp46-187-149-223.eaw.com.pl. [46.187.149.223]) by mx.google.com with ESMTPSA id cw6sm14366524wjb.19.2014.07.31.09.16.18 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 31 Jul 2014 09:16:18 -0700 (PDT) Message-ID: <53DA6BD9.8090906@gmail.com> Date: Thu, 31 Jul 2014 18:16:25 +0200 From: Adi User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: FreeBSD 10 swapping Idle processes :/ References: <53D962A7.5090105@gmail.com> <3526D2C767B54D3597CC1380AAC5E07E@multiplay.co.uk> <53DA3B2B.8020003@gmail.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 16:16:22 -0000 Hello > Kernel here takes ~5mins max on a decent machine so it shouldn't > take that long. Compiling process only kernel, yes is fast but I don't remember how do it :) 1. I should download full 10 RELEASE p7 sources 2. I should download sys/vm/vm_pageout.c file from STABLE tree And replace orginal vm_pageout.c file? Is it correct? But if changes (in Stable) vm_pageout.c file affect other source files in stable I don't want destroy production system :) Best Regards. From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 1 12:56:57 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 87140214 for ; Fri, 1 Aug 2014 12:56:57 +0000 (UTC) Received: from mail-wg0-x230.google.com (mail-wg0-x230.google.com [IPv6:2a00:1450:400c:c00::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1F6742A89 for ; Fri, 1 Aug 2014 12:56:56 +0000 (UTC) Received: by mail-wg0-f48.google.com with SMTP id x13so4224538wgg.19 for ; Fri, 01 Aug 2014 05:56:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=date:from:to:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; bh=1XLPNjxaeNvm5aeAooS5rQ3LOiPqHMv5/XIWFZ8R6IU=; b=dwrxaKKKDgXEWLK8OlGawH8RJj+WeplAVkg8LmwbAB8Oi9tppB1OKYutRJBNsX+bLc td1nFKzctQQ13TPwBtMbelJ4s0SmRfQnmx7E3FMxidpNNiymNdZhsUkiqaPggeFkq/9R +ycUoQMbIaqUV8ZiYsk3yXgd4WUyYHGwtCnH2k/sqsmrqySEaVerA/bxjEaIHCIkDCZi QTZwH6dE0xVGHNBUnVQGn+juMeBgwk21ww8TQ+pNTkHk/pwIV+8x10O2FxRh6NDqmpjL VLop2PrNA3ofadi0qN6iSN3lS6EzaVpitnZ6KY82yx6u6tr0itIe4CjyoVe/0TKwrEPW kU2Q== X-Received: by 10.194.22.166 with SMTP id e6mr7904297wjf.88.1406897814281; Fri, 01 Aug 2014 05:56:54 -0700 (PDT) Received: from gumby.homeunix.com ([94.195.197.120]) by mx.google.com with ESMTPSA id ex2sm8488726wic.24.2014.08.01.05.56.53 for (version=SSLv3 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 01 Aug 2014 05:56:53 -0700 (PDT) Date: Fri, 1 Aug 2014 13:56:52 +0100 From: RW To: freebsd-hackers@freebsd.org Subject: Re: FreeBSD 10 swapping Idle processes :/ Message-ID: <20140801135652.264bdd4d@gumby.homeunix.com> In-Reply-To: <53D9F04C.6080607@gmail.com> References: <53D962A7.5090105@gmail.com> <53D97C71.3050806@freebsd.org> <53D9F04C.6080607@gmail.com> X-Mailer: Claws Mail 3.10.1 (GTK+ 2.24.22; amd64-portbld-freebsd10.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2014 12:56:57 -0000 On Thu, 31 Jul 2014 09:29:16 +0200 Adi wrote: > Hello >=20 > > That looks odd, the output of top shows 0 swap usage, but also 0 > > resident memory for most of the processes... >=20 > Yes very odd. After server restart or after restart all deamons > processes working normally (in memory not in SWAP) for few (1-3) > hours. >=20 > Then start odd swapping. And this is not temporary (for the possible > disappearance of the "swapping trigger") only lasting behavior. >=20 > Process is moved from SWAP to memory if is not idle. And back > to SWAP after idle few seconds. =46rom what you showed before there was zero swap space in use. If what I wrote before is right then the processes marked as swap are still in RAM, they just don't show as active any more. I don't know what's changed, you only used to get this behaviour when memory got very low or vm.swap_idle_enabled was set. Possibly someones optimized it, or maybe it's a bug. Either way I doubt it's a significant problem, setting swap_idle_enabled is just a minor optimization. From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 1 13:55:49 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0A282B7C for ; Fri, 1 Aug 2014 13:55:49 +0000 (UTC) Received: from mail-wg0-x234.google.com (mail-wg0-x234.google.com [IPv6:2a00:1450:400c:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 969DE22D9 for ; Fri, 1 Aug 2014 13:55:48 +0000 (UTC) Received: by mail-wg0-f52.google.com with SMTP id a1so4322804wgh.11 for ; Fri, 01 Aug 2014 06:55:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=vze39IAQZBVr/oyd2fzTp2SVAlUTYFp6tppfB07cqTs=; b=YQ/STOE2+Ozv7Z41lNbF320ohOiVYcPN6GoR2A5zI46CHOQpMtQEn7USz/9Leywxbk 2YPHZrnHNS1G8iyTM5aON609sc95qn2bhCdHI+wOxf7hSj1+VLPYG7ObC8B7g2Jqw6o7 63DAUHI2oTbEw/AzWIxktn7SvUZe49MgHq8+eQ859D5mUnnz9c7tZDvOvQ3a9bXmcEHd 82J+BwkMjHQJ4NdustzLCa+1IUJ7nF7XDgvFfYpPZ6WEZr19xbU9StaeZlXCm1/Vzxwt hDPL4tCeWxWseU1QeQ5XejZFnETNrGHjDO2npznaNulSpmXxK4fdL6UyVWa5NOZ/xcWn 7HgA== X-Received: by 10.180.85.101 with SMTP id g5mr6997008wiz.51.1406901346023; Fri, 01 Aug 2014 06:55:46 -0700 (PDT) Received: from [192.168.2.100] (dhcp46-187-149-223.eaw.com.pl. [46.187.149.223]) by mx.google.com with ESMTPSA id m8sm21121574wjy.35.2014.08.01.06.55.45 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 01 Aug 2014 06:55:45 -0700 (PDT) Message-ID: <53DB9C76.2080208@gmail.com> Date: Fri, 01 Aug 2014 15:56:06 +0200 From: Adi User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: FreeBSD 10 swapping Idle processes :/ References: <53D962A7.5090105@gmail.com> <53D97C71.3050806@freebsd.org> <53D9F04C.6080607@gmail.com> <20140801135652.264bdd4d@gumby.homeunix.com> In-Reply-To: <20140801135652.264bdd4d@gumby.homeunix.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2014 13:55:49 -0000 Hello > I don't know what's changed, you only used to get this behaviour when > memory got very low or vm.swap_idle_enabled was set. Possibly someones > optimized it, or maybe it's a bug. Either way I doubt it's a significant > problem, setting swap_idle_enabled is just a minor optimization. # sysctl vm.swap_idle_enabled vm.swap_idle_enabled: 0 I have never changed this setting. After reset all processes (excluded one I forgot) 7 hours ago is still OK procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad0 ad1 in sy cs us sy id 0 0 1 14346M 1181M 306 0 0 0 321 511 0 0 86 1648 661 0 0 100 so only 1 processes is in SWAP idle. Mem: 419M Active, 27G Inact, 2990M Wired, 30M Cache, 1652M Buf, 1130M Free Swap: 32G Total, 10M Used, 32G Free Probably patches from Steven Hartland post: " MFC r265418 - vm_pageout stuck pages MFC r260567 - vm_pageout stuck pages http://svnweb.freebsd.org/base?view=revision&revision=265944 http://svnweb.freebsd.org/base?view=revision&revision=265945 " will help. Dimitry , author thread (with similar problem) http://lists.freebsd.org/pipermail/freebsd-hackers/2014-January/044040.html update FreeBSD to 10.0-STABLE and is better but not 100% ok. my system is not yet updated. I wonder whether it makes sense to checkout 10.0-Releng and replacement only file vm_pageout.c (form STABLE). And then compile the kernel. Instead of updating the entire system to STABLE. Best Regards. From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 1 15:30:58 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D1175751 for ; Fri, 1 Aug 2014 15:30:58 +0000 (UTC) Received: from mail-wg0-x22b.google.com (mail-wg0-x22b.google.com [IPv6:2a00:1450:400c:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6904B2FA1 for ; Fri, 1 Aug 2014 15:30:58 +0000 (UTC) Received: by mail-wg0-f43.google.com with SMTP id l18so4451723wgh.14 for ; Fri, 01 Aug 2014 08:30:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=date:from:to:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; bh=TD2XM4VqQw3NImNuEZmGeL3b71aQnei2kUjceGasObo=; b=Zu9XGug81WpH3TGnMSoSEgpUHaP6c7TkxvS9bWzB6pW03wJu73zI46RW+vXXiCth0e GbXBzqmof2/YFegrSYg4ilvlXYF2zFoxacxcGrLdaimPHxSYdmNrtAlVHzhmE5cYR2ml yQyOfN1oh+kUCllfWtr4EfRBo4x5qrNUPFNL1wVUQC3d1ue2qzXjMfx4KMUOvoHoLEKC x5wnsd/GOWofK71/Yn43iV98o2gJUsLEwsRilM/t9XKUsbY7fDTfdReWv2R8+zB5YMcF 2qcFfU/HXcBAO/PSeVAhSdOiHxGFTvQUdmutcPaNcE3bLRyHo1L2H0onqRNGOnBod36I yEBQ== X-Received: by 10.180.100.133 with SMTP id ey5mr8167471wib.16.1406907053646; Fri, 01 Aug 2014 08:30:53 -0700 (PDT) Received: from gumby.homeunix.com ([94.195.197.120]) by mx.google.com with ESMTPSA id di7sm21645061wjb.34.2014.08.01.08.30.52 for (version=SSLv3 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 01 Aug 2014 08:30:53 -0700 (PDT) Date: Fri, 1 Aug 2014 16:30:50 +0100 From: RW To: freebsd-hackers@freebsd.org Subject: Re: FreeBSD 10 swapping Idle processes :/ Message-ID: <20140801163050.66d32b85@gumby.homeunix.com> In-Reply-To: <53DB9C76.2080208@gmail.com> References: <53D962A7.5090105@gmail.com> <53D97C71.3050806@freebsd.org> <53D9F04C.6080607@gmail.com> <20140801135652.264bdd4d@gumby.homeunix.com> <53DB9C76.2080208@gmail.com> X-Mailer: Claws Mail 3.10.1 (GTK+ 2.24.22; amd64-portbld-freebsd10.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2014 15:30:58 -0000 On Fri, 01 Aug 2014 15:56:06 +0200 Adi wrote: > Hello > > > I don't know what's changed, you only used to get this behaviour > > when memory got very low or vm.swap_idle_enabled was set. Possibly > > someones optimized it, or maybe it's a bug. Either way I doubt it's > > a significant problem, setting swap_idle_enabled is just a minor > > optimization. > > > # sysctl vm.swap_idle_enabled > vm.swap_idle_enabled: 0 > > I have never changed this setting. I know, that setting was on pastebin. > After reset all processes (excluded one I forgot) 7 hours ago is > still OK > > > so only 1 processes is in SWAP idle. > ... > > update FreeBSD to 10.0-STABLE and is better but not 100% ok. but seeing processes marked as swapped is not in itself a significant problem. It's really just a way of reducing the priority of certain pages in case actual page-outs occur later. In your original post you had many idle processes marked as swapped and *zero* swap usage. From owner-freebsd-hackers@FreeBSD.ORG Fri Aug 1 18:08:55 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2AB8DBB1 for ; Fri, 1 Aug 2014 18:08:55 +0000 (UTC) Received: from smtp.kn-bremen.de (gruenbaer.kn-bremen.de [148.251.8.79]) by mx1.freebsd.org (Postfix) with ESMTP id DE0572171 for ; Fri, 1 Aug 2014 18:08:54 +0000 (UTC) Received: by smtp.kn-bremen.de (Postfix, from userid 10) id 9F2A33DE0DF9; Fri, 1 Aug 2014 20:02:44 +0200 (CEST) Received: from enceladus10.kn-bremen.de (noident@localhost [127.0.0.1]) by enceladus10.kn-bremen.de (8.14.5/8.14.5) with ESMTP id s71I19lk062662; Fri, 1 Aug 2014 20:01:09 +0200 (CEST) (envelope-from nox@enceladus10.kn-bremen.de) Received: (from nox@localhost) by enceladus10.kn-bremen.de (8.14.5/8.14.5/Submit) id s71I19Vr062661; Fri, 1 Aug 2014 20:01:09 +0200 (CEST) (envelope-from nox) Date: Fri, 1 Aug 2014 20:01:09 +0200 (CEST) From: Juergen Lock Message-Id: <201408011801.s71I19Vr062661@enceladus10.kn-bremen.de> To: kaduk@MIT.EDU Subject: Re: Audio CDs Not Playing X-Newsgroups: local.list.freebsd.hackers In-Reply-To: References: <53D97A50.8090006@gmx.us> <53D9A676.6050303@gmx.us> Organization: home X-Mailman-Approved-At: Fri, 01 Aug 2014 19:47:42 +0000 Cc: freebsd-hackers@freebsd.org, Dutch Ingraham X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2014 18:08:55 -0000 In article you write: >On Wed, 30 Jul 2014, Dutch Ingraham wrote: > >> Thanks for the info, Ben. I had actually seen something on that issue in an >> old (May 2008) daemonforums post while researching this problem. There, the >> poster suggested using amarok or xmms - programs capable of "digital audio >> extraction" ( I don't know what that is ) and in which it was implied that >> cdcontrol was not capable of. >> >> xmms is deprecated and I don't want the hundreds of files that come with >> amarok. I suppose I could try something like audacious, but as noted, vlc >> doesn't work either, so I'd prefer to not get into the cycle of downloading a >> bunch of similar programs just to find out there was a simple setting I was >> missing. >> >> Also, the current man page for and the handbook don't adress such >> a restriction that I could find. >> >> Does "digital audio extraction" mean anything to you or is it helpful in ay >> way? > >I think I know what it means. The easiest way to think about it is >probably to realize that in order to play audio, the bits recorded on the >CD have to make it to the digital-analog converter somehow. If there's no >direct line from the CD drive to the sound card (as we were discussing in >the trimmed text), then that data has to be moved around in software. >Some tool is needed to extract the audio data from the CD drive, and some >tool is needed to send those off to the sound card; these tools can be the >same, but need not be. > >I tend to use cdparanoia for the first step, and the play(1) utility >provided by audio/sox for the second step, leaving the bits around on my >(sizable) hard drive for later use. It sounds like xmms and amarok can >combine the two steps into one, without leaving the bits on disk as an >intermediate; I'm not sure offhand whether there are more lightweight >utilities that can also do so. > >-Ben You can use mplayer built with CDIO knob too, see this forum thread: https://forums.freebsd.org/viewtopic.php?&t=27167#p198402 (which also has info about xpt0 and passX permssions to be adjusted if you want to do this as non-root.) HTH, :) Juergen From owner-freebsd-hackers@FreeBSD.ORG Sat Aug 2 09:20:25 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2F4C034A for ; Sat, 2 Aug 2014 09:20:25 +0000 (UTC) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B96FB2695 for ; Sat, 2 Aug 2014 09:20:24 +0000 (UTC) Received: by mail-wi0-f171.google.com with SMTP id hi2so2524790wib.16 for ; Sat, 02 Aug 2014 02:20:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=UH02ww0jkVF1ktPBO8S9lqNxugt1TTSg1KQl0emidxg=; b=1B+PtOhCP37dH2b0hErPsenT875BwFT7RGMrJqY8fGkYhvnazszzkYVAM65ZTyFWFl gmEnmgQvHQBPniJE8bzL0nYNcRxOxrcNv3ykLvJTM1sEaeOLv3SpZyDRHLC4WuYua6ma 7nAOk2tN7ApELHlIatXNhyksiEcKQWMI0ru5Dbg6i3+e2uXFssLXuXMexY5kjAaHN/fX ERc3sfiqxg3FkNarfkxMaVcb/knnNjQzn3yxjFH0zDd0RxCL/tEzPpgRRLnd9yunKeZc Gvkss/i0WRc7Lx3zkhauknV7HisuoQzh1S37z0zSyww4dqdEjpEtOwg/X4lJMcqUbjTe TGKQ== X-Received: by 10.180.96.169 with SMTP id dt9mr14345535wib.62.1406971222982; Sat, 02 Aug 2014 02:20:22 -0700 (PDT) Received: from [192.168.2.100] (dhcp46-187-149-223.eaw.com.pl. [46.187.149.223]) by mx.google.com with ESMTPSA id p3sm27004311wjw.13.2014.08.02.02.20.21 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 02 Aug 2014 02:20:22 -0700 (PDT) Message-ID: <53DCAD70.3000900@gmail.com> Date: Sat, 02 Aug 2014 11:20:48 +0200 From: Adi User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: FreeBSD 10 swapping Idle processes :/ References: <53D962A7.5090105@gmail.com> <53D97C71.3050806@freebsd.org> <53D9F04C.6080607@gmail.com> <20140801135652.264bdd4d@gumby.homeunix.com> <53DB9C76.2080208@gmail.com> <20140801163050.66d32b85@gumby.homeunix.com> In-Reply-To: <20140801163050.66d32b85@gumby.homeunix.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2014 09:20:25 -0000 Hello > but seeing processes marked as swapped is not in itself a significant > problem. yes, but is very annoying and confusing > It's really just a way of reducing the priority of certain > pages in case actual page-outs occur later. whether this affects the performance? > In your original post you had many idle processes marked as swapped and > *zero* swap usage. Yes there where zero swap usage. if it is not broken and correctly indicates :) Best Regards. From owner-freebsd-hackers@FreeBSD.ORG Sat Aug 2 15:08:52 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2B4E434E for ; Sat, 2 Aug 2014 15:08:52 +0000 (UTC) Received: from mail-in-13.arcor-online.net (mail-in-13.arcor-online.net [151.189.21.53]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx.arcor.de", Issuer "Thawte SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D6F8B28E2 for ; Sat, 2 Aug 2014 15:08:51 +0000 (UTC) Received: from mail-in-15-z2.arcor-online.net (mail-in-15-z2.arcor-online.net [151.189.8.32]) by mx.arcor.de (Postfix) with ESMTP id 746F912616A for ; Sat, 2 Aug 2014 16:34:08 +0200 (CEST) Received: from mail-in-14.arcor-online.net (mail-in-14.arcor-online.net [151.189.21.54]) by mail-in-15-z2.arcor-online.net (Postfix) with ESMTP id 7BF7611209C for ; Sat, 2 Aug 2014 16:34:08 +0200 (CEST) X-Greylist: Passed host: 188.98.178.71 X-DKIM: Sendmail DKIM Filter v2.8.2 mail-in-14.arcor-online.net 4DC929BE31 Received: from lorvorc.mips.inka.de (dslb-188-098-178-071.188.098.pools.vodafone-ip.de [188.98.178.71]) by mail-in-14.arcor-online.net (Postfix) with ESMTPS id 4DC929BE31 for ; Sat, 2 Aug 2014 16:34:08 +0200 (CEST) Received: from lorvorc.mips.inka.de (localhost [127.0.0.1]) by lorvorc.mips.inka.de (8.14.9/8.14.7) with ESMTP id s72EY7nC047702 for ; Sat, 2 Aug 2014 16:34:07 +0200 (CEST) (envelope-from news@lorvorc.mips.inka.de) Received: (from news@localhost) by lorvorc.mips.inka.de (8.14.9/8.14.9/Submit) id s72EY7Yf047701 for freebsd-hackers@freebsd.org; Sat, 2 Aug 2014 16:34:07 +0200 (CEST) (envelope-from news) To: freebsd-hackers@freebsd.org From: Christian Weisgerber Newsgroups: list.freebsd.hackers Subject: Re: Audio CDs Not Playing Date: Sat, 2 Aug 2014 14:34:07 +0000 (UTC) Lines: 17 Message-ID: References: <53D97A50.8090006@gmx.us> <53D9A676.6050303@gmx.us> X-Trace: lorvorc.mips.inka.de 1406990047 46290 ::1 (2 Aug 2014 14:34:07 GMT) X-Complaints-To: usenet@mips.inka.de User-Agent: slrn/1.0.1 (FreeBSD) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2014 15:08:52 -0000 On 2014-07-31, Dutch Ingraham wrote: > Thanks for the info, Ben. I had actually seen something on that issue > in an old (May 2008) daemonforums post while researching this problem. > There, the poster suggested using amarok or xmms - programs capable of > "digital audio extraction" ( I don't know what that is ) and in which it > was implied that cdcontrol was not capable of. > > xmms is deprecated and I don't want the hundreds of files that come with > amarok. I suppose I could try something like audacious, but as noted, Playing audio CDs with xmms and digital audio extraction works just fine. It should also work with audacious but this appears to be broken. -- Christian "naddy" Weisgerber naddy@mips.inka.de From owner-freebsd-hackers@FreeBSD.ORG Sat Aug 2 18:11:11 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 29CC13BA for ; Sat, 2 Aug 2014 18:11:11 +0000 (UTC) Received: from mout.gmx.com (mout.gmx.com [74.208.4.200]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E08DC2D15 for ; Sat, 2 Aug 2014 18:11:10 +0000 (UTC) Received: from [192.168.1.81] ([172.15.184.248]) by mail.gmx.com (mrgmxus002) with ESMTPSA (Nemesis) id 0MH0ia-1XIZhq0uDi-00Dkbz for ; Sat, 02 Aug 2014 20:11:04 +0200 Message-ID: <53DD2A1C.7020009@gmx.us> Date: Sat, 02 Aug 2014 14:12:44 -0400 From: Dutch Ingraham User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.7.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: Audio CDs Not Playing References: <53D97A50.8090006@gmx.us> <53D9A676.6050303@gmx.us> <201408011801.s71I19Vr062661@enceladus10.kn-bremen.de> In-Reply-To: <201408011801.s71I19Vr062661@enceladus10.kn-bremen.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:bFUvUylJuJuzX4egjLW3LHiwYFVcyXwdastyS7rojsFB6tfH1pu Cy93DiJlkfD8znVlxOOjWkp4NdVAjuW9m03OKNf6ykfY77Jd3Jqmnj+/onAXN77V0hD/QdX VP2D8X4+dfoRT3Q52hu9z8tiG6c9w0jKwfJY64xDNzpNu/dfgiNbts00ejRFASV1A6Kdp6p MccAOY6ICJbbhx8v4PyGw== X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2014 18:11:11 -0000 On 08/01/2014 02:01 PM, Juergen Lock wrote: > In article you write: >> On Wed, 30 Jul 2014, Dutch Ingraham wrote: >> >>> Thanks for the info, Ben. I had actually seen something on that issue in an >>> old (May 2008) daemonforums post while researching this problem. There, the >>> poster suggested using amarok or xmms - programs capable of "digital audio >>> extraction" ( I don't know what that is ) and in which it was implied that >>> cdcontrol was not capable of. >>> >>> xmms is deprecated and I don't want the hundreds of files that come with >>> amarok. I suppose I could try something like audacious, but as noted, vlc >>> doesn't work either, so I'd prefer to not get into the cycle of downloading a >>> bunch of similar programs just to find out there was a simple setting I was >>> missing. >>> >>> Also, the current man page for and the handbook don't adress such >>> a restriction that I could find. >>> >>> Does "digital audio extraction" mean anything to you or is it helpful in ay >>> way? >> >> I think I know what it means. The easiest way to think about it is >> probably to realize that in order to play audio, the bits recorded on the >> CD have to make it to the digital-analog converter somehow. If there's no >> direct line from the CD drive to the sound card (as we were discussing in >> the trimmed text), then that data has to be moved around in software. >> Some tool is needed to extract the audio data from the CD drive, and some >> tool is needed to send those off to the sound card; these tools can be the >> same, but need not be. >> >> I tend to use cdparanoia for the first step, and the play(1) utility >> provided by audio/sox for the second step, leaving the bits around on my >> (sizable) hard drive for later use. It sounds like xmms and amarok can >> combine the two steps into one, without leaving the bits on disk as an >> intermediate; I'm not sure offhand whether there are more lightweight >> utilities that can also do so. >> >> -Ben > > You can use mplayer built with CDIO knob too, see this forum thread: > > https://forums.freebsd.org/viewtopic.php?&t=27167#p198402 > > (which also has info about xpt0 and passX permssions to be adjusted > if you want to do this as non-root.) > > HTH, :) > Juergen > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > Thanks, Juergen. Nice thread, also confirming Tijl's information on the use (or non-use) of on more current hardware. From owner-freebsd-hackers@FreeBSD.ORG Sat Aug 2 18:13:58 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 253A5594 for ; Sat, 2 Aug 2014 18:13:58 +0000 (UTC) Received: from mout.gmx.com (mout.gmx.com [74.208.4.200]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E47532EAE for ; Sat, 2 Aug 2014 18:13:57 +0000 (UTC) Received: from [192.168.1.81] ([172.15.184.248]) by mail.gmx.com (mrgmxus002) with ESMTPSA (Nemesis) id 0MexW3-1WpFJV3SbB-00OaP1 for ; Sat, 02 Aug 2014 20:13:56 +0200 Message-ID: <53DD2AC9.4040507@gmx.us> Date: Sat, 02 Aug 2014 14:15:37 -0400 From: Dutch Ingraham User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.7.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: Audio CDs Not Playing References: <53D97A50.8090006@gmx.us> <53D9A676.6050303@gmx.us> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:mq7dZdQ4GHsJ8ej+TTUb7k/v/yIn2ElcllOSss4ZgBMJ25t81wz FW6n9dG79xXOcVVZXPv/69FZ5bqOkObgqvnbMoba+dOtvEqXPvGoh+nNSJLPhfJsMjuFI2O zRHKSyo7diYU+hWf7hcqLjodGAV4hvo4V5BrVByhblGC0Cwn9+UhMwdFQJM6x7oUHCt4dRj 7+l4gqk00AMMZyENOAOIw== X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2014 18:13:58 -0000 On 08/02/2014 10:34 AM, Christian Weisgerber wrote: > On 2014-07-31, Dutch Ingraham wrote: > >> Thanks for the info, Ben. I had actually seen something on that issue >> in an old (May 2008) daemonforums post while researching this problem. >> There, the poster suggested using amarok or xmms - programs capable of >> "digital audio extraction" ( I don't know what that is ) and in which it >> was implied that cdcontrol was not capable of. >> >> xmms is deprecated and I don't want the hundreds of files that come with >> amarok. I suppose I could try something like audacious, but as noted, > > Playing audio CDs with xmms and digital audio extraction works just > fine. It should also work with audacious but this appears to be > broken. > Thanks - I wasn't aware there was a problem with that vis-a-vis audacious - which was my back-up if I couldn't get vlc working (which I did).