From owner-freebsd-arch@FreeBSD.ORG Sun May 29 02:48:53 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BF165106564A; Sun, 29 May 2011 02:48:53 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-yi0-f54.google.com (mail-yi0-f54.google.com [209.85.218.54]) by mx1.freebsd.org (Postfix) with ESMTP id 507248FC0C; Sun, 29 May 2011 02:48:52 +0000 (UTC) Received: by yie12 with SMTP id 12so1505070yie.13 for ; Sat, 28 May 2011 19:48:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=0eTAUQgQho0KpjQcgkjSA1pNkan8lLU+mZeSQZ4lZj8=; b=nHdH+jvqd97nQwklgCAnNcGhKz1hsm1pymRILlVaks/ccL5qi1aka+Z9NuxVYM0SXa Tx4KRnWEDq1zljTLjJjtfP/AVF4JA/HT+hn46eCeTylEYATbPacAtLoT8xM3TY7jVZHS YSc5FfsEnPCCl6DB24pYkhOQGclnaPqQljrWs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=QJ3l1VmxEB6KdICxSpamHZD0HvJhzrUs5wHVQ4kIRR4fnqwZ29PAhvY6OjpkfRaUEs eJmlBGvUjjOJEMQGLxTaUNWe7dXsM3G5sk5+DX/VQJ9l7Z16+acNJgXmOfS26WF7IyqX EP3xhcuyMdHqmOv3HPjvAr0etLIDkw34QKGSw= MIME-Version: 1.0 Received: by 10.236.175.38 with SMTP id y26mr4577500yhl.304.1306637332392; Sat, 28 May 2011 19:48:52 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.236.103.136 with HTTP; Sat, 28 May 2011 19:48:52 -0700 (PDT) In-Reply-To: <4DDD2B34.5070702@FreeBSD.org> References: <4DD3F662.9040603@FreeBSD.org> <4DD54C18.8050305@FreeBSD.org> <4DDA8B2A.6010500@FreeBSD.org> <4DDD2B34.5070702@FreeBSD.org> Date: Sat, 28 May 2011 22:48:52 -0400 X-Google-Sender-Auth: hyYqyUUVDjH52ldX2USa0amNndI Message-ID: From: Attilio Rao To: Andriy Gapon Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-current@freebsd.org" , "freebsd-arch@freebsd.org" Subject: Re: [rfc] remove hlt_cpus et al sysctls and related code X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 May 2011 02:48:53 -0000 2011/5/25 Andriy Gapon : > on 23/05/2011 19:28 Andriy Gapon said the following: >> I propose the following path for moving forward. >> - use hint.lapic.X.disabled to disable individual CPUs by their APIC ID >> - use machdep.hyperthreading_allowed tunable to disable second logical C= PU on each >> real core >> >> The above should already work as expected. =C2=A0One thing is that curre= ntly we have >> handling of machdep.hyperthreading_allowed tunable under SCHED_ULE. =C2= =A0I plan to >> make it unconditional. >> >> Things to remove: >> - all the related sysctls for dynamic onlining/offlining >> - machdep.hlt_logical_cpus tunable (it duplicates hint.lapic.X.disabled) >> >> It's possible to keep machdep.hlt_logical_cpus and just add some code to= convert >> hlt_logical_cpus mask to a set of individual hint.lapic.X.disabled, but = I don't >> see very much value in that. =C2=A0But if there is a good reason to keep= that tunable, >> I am prepared to jump through this hoop. >> >> If no one objects to this proposal, I will provide a patch soon. > > The patch is here: > http://people.freebsd.org/~avg/cpu-offline-sysctl.diff > It should implement the strategy described above. > I don't see the point in keeping alive mp_grab_cpu_hlt() and supporting, actually. On the top of your patch I made some modifies that use directly ap_watchdog() in cpu_idle() which I think is better for the time being: http://www.freebsd.org/~attilio/avg_rem_cpuhlt.diff If you are happy with it, just commit as long as Garrett tests that. On a second round of changes we can discuss mp_watchdog and eventual removal / improvements to it. Attilio --=20 Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Sun May 29 03:06:33 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E3EBB1065672; Sun, 29 May 2011 03:06:32 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-yi0-f54.google.com (mail-yi0-f54.google.com [209.85.218.54]) by mx1.freebsd.org (Postfix) with ESMTP id 724C18FC15; Sun, 29 May 2011 03:06:32 +0000 (UTC) Received: by yie12 with SMTP id 12so1508013yie.13 for ; Sat, 28 May 2011 20:06:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=ln1aS4EpWu2Ff3myz6p94p4tr36E1rJ0kO7si9kf/io=; b=pO2k4XNYTSwU5VB5OUYGJ4zXs/Z9WYRge8TgmYa8ySwazkcI+Yq5G8B/76PyvTht2u Sf0csGc+hpca8hsToqg1VhMV0bdsegR5aAa+uPA6tf2ErEXycurq8YGfIc/hkt1XjNWF /MJK467pDgZTHopZ9IB/GSHZWUZy9hlt3Ms3I= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=Wc0RaJa6AQwL5ZZMmVZSD5zyla9+r8dEF1bxSVaHKIH8d2kB9Gzah240xBDeL8gqHT 6B7g9yJFLIC/rPfgp26FB2QRtRamb0bbBPq9FSpijm8w0tqMRnNlUsN5DSjc0i0hi6gC Zpv85zb1+XJm+TM638V04eLgeousK23QTXdLk= MIME-Version: 1.0 Received: by 10.236.184.67 with SMTP id r43mr1456317yhm.103.1306638391582; Sat, 28 May 2011 20:06:31 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.236.103.136 with HTTP; Sat, 28 May 2011 20:06:31 -0700 (PDT) In-Reply-To: References: <4DD3F662.9040603@FreeBSD.org> <4DD54C18.8050305@FreeBSD.org> <4DDA8B2A.6010500@FreeBSD.org> <4DDD2B34.5070702@FreeBSD.org> Date: Sat, 28 May 2011 23:06:31 -0400 X-Google-Sender-Auth: IIc0Uygr-f8wMAEEUsr1T_8RmV0 Message-ID: From: Attilio Rao To: Andriy Gapon Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-current@freebsd.org" , "freebsd-arch@freebsd.org" Subject: Re: [rfc] remove hlt_cpus et al sysctls and related code X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 May 2011 03:06:33 -0000 2011/5/28 Attilio Rao : > 2011/5/25 Andriy Gapon : >> on 23/05/2011 19:28 Andriy Gapon said the following: >>> I propose the following path for moving forward. >>> - use hint.lapic.X.disabled to disable individual CPUs by their APIC ID >>> - use machdep.hyperthreading_allowed tunable to disable second logical = CPU on each >>> real core >>> >>> The above should already work as expected. =C2=A0One thing is that curr= ently we have >>> handling of machdep.hyperthreading_allowed tunable under SCHED_ULE. =C2= =A0I plan to >>> make it unconditional. >>> >>> Things to remove: >>> - all the related sysctls for dynamic onlining/offlining >>> - machdep.hlt_logical_cpus tunable (it duplicates hint.lapic.X.disabled= ) >>> >>> It's possible to keep machdep.hlt_logical_cpus and just add some code t= o convert >>> hlt_logical_cpus mask to a set of individual hint.lapic.X.disabled, but= I don't >>> see very much value in that. =C2=A0But if there is a good reason to kee= p that tunable, >>> I am prepared to jump through this hoop. >>> >>> If no one objects to this proposal, I will provide a patch soon. >> >> The patch is here: >> http://people.freebsd.org/~avg/cpu-offline-sysctl.diff >> It should implement the strategy described above. >> > > I don't see the point in keeping alive mp_grab_cpu_hlt() and > supporting, actually. > > On the top of your patch I made some modifies that use directly > ap_watchdog() in cpu_idle() which I think is better for the time > being: > http://www.freebsd.org/~attilio/avg_rem_cpuhlt.diff > > If you are happy with it, just commit as long as Garrett tests that. > > On a second round of changes we can discuss mp_watchdog and eventual > removal / improvements to it. I almost forgot: this change would also require an UPDATE entry, where you explicitly mention the "new" way to deal with CPUs. Use your prefer wording. Attilio --=20 Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Tue May 31 11:42:22 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 14125106566B; Tue, 31 May 2011 11:42:22 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id F0AAC8FC15; Tue, 31 May 2011 11:42:20 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA08487; Tue, 31 May 2011 14:42:19 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4DE4D41B.1000000@FreeBSD.org> Date: Tue, 31 May 2011 14:42:19 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.17) Gecko/20110504 Lightning/1.0b2 Thunderbird/3.1.10 MIME-Version: 1.0 To: Attilio Rao References: <4DD3F662.9040603@FreeBSD.org> <4DD54C18.8050305@FreeBSD.org> <4DDA8B2A.6010500@FreeBSD.org> <4DDD2B34.5070702@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: "freebsd-current@freebsd.org" , "freebsd-arch@freebsd.org" Subject: Re: [rfc] remove hlt_cpus et al sysctls and related code X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 May 2011 11:42:22 -0000 on 29/05/2011 06:06 Attilio Rao said the following: > 2011/5/28 Attilio Rao : >> 2011/5/25 Andriy Gapon : >>> The patch is here: >>> http://people.freebsd.org/~avg/cpu-offline-sysctl.diff >>> It should implement the strategy described above. >>> >> >> I don't see the point in keeping alive mp_grab_cpu_hlt() and >> supporting, actually. >> >> On the top of your patch I made some modifies that use directly >> ap_watchdog() in cpu_idle() which I think is better for the time >> being: >> http://www.freebsd.org/~attilio/avg_rem_cpuhlt.diff Yes, I agree, thank you. >> If you are happy with it, just commit as long as Garrett tests that. OK. Waiting for test feedback. >> On a second round of changes we can discuss mp_watchdog and eventual >> removal / improvements to it. > > I almost forgot: this change would also require an UPDATE entry, where > you explicitly mention the "new" way to deal with CPUs. Use your > prefer wording. Sure. Thank you! BTW, I guess there would be no reason to MFC this change? -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Tue May 31 13:34:46 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5EFD21065670; Tue, 31 May 2011 13:34:46 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id DEB808FC08; Tue, 31 May 2011 13:34:45 +0000 (UTC) Received: by gyg13 with SMTP id 13so2604228gyg.13 for ; Tue, 31 May 2011 06:34:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=LVSmB85qYlHDaL3fZa8ZoC5JsBnOZaedd7+vTmFvHlU=; b=wZvG5a/CyefhwZEVJiRJSXCcbU1qhoq4r5qAHCSry+zQY8mmP2o8oUzsTySD0v58zK hYMSuRy9To5SJtHf3BMyJGqPN/6U0d/ho+LOHCvzZEd7XqyqjPbNdnRKK7kLL6iujbsD LgAlAb8UgQuzhuPjdY+8Aer+UjKuynFzEK8B8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=lKhSKv0dkbePOQdP6Qzp/Yql4T4SkMuXPSbFIKV4JauzlBe8DLFySBz+Y2H3YAIH1h JkdB9kaTbqnamB0LDbXkXjRMgGNDLs6GNK/hEshrJL41Kn7vjCayx1s5mJBAILfbDu6l mMUAVq19sCyEvPf9Jn+pFQ+g5G6ljRilEqWSI= MIME-Version: 1.0 Received: by 10.236.161.194 with SMTP id w42mr6771364yhk.237.1306848885146; Tue, 31 May 2011 06:34:45 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.236.103.136 with HTTP; Tue, 31 May 2011 06:34:44 -0700 (PDT) In-Reply-To: <4DE4D41B.1000000@FreeBSD.org> References: <4DD3F662.9040603@FreeBSD.org> <4DD54C18.8050305@FreeBSD.org> <4DDA8B2A.6010500@FreeBSD.org> <4DDD2B34.5070702@FreeBSD.org> <4DE4D41B.1000000@FreeBSD.org> Date: Tue, 31 May 2011 09:34:44 -0400 X-Google-Sender-Auth: ArqOaQt3weQvcGly6vcpJn3F6AM Message-ID: From: Attilio Rao To: Andriy Gapon Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-current@freebsd.org" , "freebsd-arch@freebsd.org" Subject: Re: [rfc] remove hlt_cpus et al sysctls and related code X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 May 2011 13:34:46 -0000 2011/5/31 Andriy Gapon : > on 29/05/2011 06:06 Attilio Rao said the following: >> 2011/5/28 Attilio Rao : >>> 2011/5/25 Andriy Gapon : >>>> The patch is here: >>>> http://people.freebsd.org/~avg/cpu-offline-sysctl.diff >>>> It should implement the strategy described above. >>>> >>> >>> I don't see the point in keeping alive mp_grab_cpu_hlt() and >>> supporting, actually. >>> >>> On the top of your patch I made some modifies that use directly >>> ap_watchdog() in cpu_idle() which I think is better for the time >>> being: >>> http://www.freebsd.org/~attilio/avg_rem_cpuhlt.diff > > Yes, I agree, thank you. > >>> If you are happy with it, just commit as long as Garrett tests that. > > > OK. =C2=A0Waiting for test feedback. > >>> On a second round of changes we can discuss mp_watchdog and eventual >>> removal / improvements to it. >> >> I almost forgot: this change would also require an UPDATE entry, where >> you explicitly mention the "new" way to deal with CPUs. Use your >> prefer wording. > > Sure. =C2=A0Thank you! > > BTW, I guess there would be no reason to MFC this change? You mean no reason to not MFC it? In general, I think that users may expect those sysctls to be alive (IMHO we should consider sysctls to be part of the userland API) so that we can add some more, but we should not axe them. So probabilly MFC is not the best option here. Attilio --=20 Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Tue May 31 13:40:48 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD939106564A; Tue, 31 May 2011 13:40:48 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A56C18FC14; Tue, 31 May 2011 13:40:47 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA11207; Tue, 31 May 2011 16:40:46 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4DE4EFDD.8070803@FreeBSD.org> Date: Tue, 31 May 2011 16:40:45 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.17) Gecko/20110504 Lightning/1.0b2 Thunderbird/3.1.10 MIME-Version: 1.0 To: Attilio Rao References: <4DD3F662.9040603@FreeBSD.org> <4DD54C18.8050305@FreeBSD.org> <4DDA8B2A.6010500@FreeBSD.org> <4DDD2B34.5070702@FreeBSD.org> <4DE4D41B.1000000@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: "freebsd-current@freebsd.org" , "freebsd-arch@freebsd.org" Subject: Re: [rfc] remove hlt_cpus et al sysctls and related code X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 May 2011 13:40:48 -0000 on 31/05/2011 16:34 Attilio Rao said the following: > 2011/5/31 Andriy Gapon : >> on 29/05/2011 06:06 Attilio Rao said the following: >>> 2011/5/28 Attilio Rao : >>>> 2011/5/25 Andriy Gapon : >>>>> The patch is here: >>>>> http://people.freebsd.org/~avg/cpu-offline-sysctl.diff >>>>> It should implement the strategy described above. >>>>> >>>> >>>> I don't see the point in keeping alive mp_grab_cpu_hlt() and >>>> supporting, actually. >>>> >>>> On the top of your patch I made some modifies that use directly >>>> ap_watchdog() in cpu_idle() which I think is better for the time >>>> being: >>>> http://www.freebsd.org/~attilio/avg_rem_cpuhlt.diff >> >> Yes, I agree, thank you. >> >>>> If you are happy with it, just commit as long as Garrett tests that. >> >> >> OK. Waiting for test feedback. >> >>>> On a second round of changes we can discuss mp_watchdog and eventual >>>> removal / improvements to it. >>> >>> I almost forgot: this change would also require an UPDATE entry, where >>> you explicitly mention the "new" way to deal with CPUs. Use your >>> prefer wording. >> >> Sure. Thank you! >> >> BTW, I guess there would be no reason to MFC this change? > > You mean no reason to not MFC it? I meant exactly what I asked :-) As in: I didn't see any reason for MFC. > In general, I think that users may expect those sysctls to be alive > (IMHO we should consider sysctls to be part of the userland API) so > that we can add some more, but we should not axe them. > So probabilly MFC is not the best option here. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Wed Jun 1 10:14:59 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 79E1D1065678 for ; Wed, 1 Jun 2011 10:14:59 +0000 (UTC) (envelope-from plaxo@mx.plaxo.com) Received: from plaxomx1.plaxo.com (plaxomx1.plaxo.com [204.15.240.53]) by mx1.freebsd.org (Postfix) with ESMTP id 56E6F8FC15 for ; Wed, 1 Jun 2011 10:14:59 +0000 (UTC) Received: from 127.0.0.1 ([10.1.2.12]) by plaxomx1.plaxo.com (StrongMail Enterprise 4.1.2(4.1.2-51177)); Wed, 01 Jun 2011 02:59:56 -0700 X-VirtualServer: Pulse-Member, plaxomx1.plaxo.com, 10.1.6.53 X-MailingID: 00000::00000::00000::00000::79-0-0-244816035251-0-20110601::368165 X-SMHeaderMap: mid="X-MailingID" X-Destination-ID: freebsd-arch@freebsd.org X-SMFBL: ZnJlZWJzZC1hcmNoQGZyZWVic2Qub3Jn DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=mx.plaxo.com; s=plaxo; l=6977; t=1306922396; i=@mx.plaxo.com; h=From:To:Subject:Message-Id:X-VirtualServerGroup: X-PlaxoMailType:X-PlaxoMailID:List-Unsubscribe:Reply-To: Errors-To:Date:X-Mailer:Content-Type:MIME-Version; bh=49dffnqOVD UwOmCfcGl8WFyYDmQ=; b=EXLc3caTiw3A8LqqniE9ztUdzQyxr5/3QZa/3/Y6F1 PTthdHSWVLWRNZaIGQGpzHlsv8J5XeG2KJTMyLthDTz5zbjARvz12d44/Qb7xRHu Ol/ZIiBWL0wdLvwgwMmhKc From: "John Styler via Plaxo" To: freebsd-arch@freebsd.org Message-Id: <31517cc37eece63fca980a0aca66d08c@xpertmailer.com> X-VirtualServerGroup: Pulse-Member X-PlaxoMailType: Pulse-NonMember X-PlaxoMailID: 79-0-0-244816035251-0-20110601 Errors-To: plaxo@mx.plaxo.com Date: Wed, 01 Jun 2011 02:59:56 -0700 X-Mailer: XPM4 v.0.3 < www.xpertmailer.com > MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: John Styler has invited you to join Plaxo X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: noreply@plaxo.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jun 2011 10:14:59 -0000 John invited you to join Plaxo!=20 John Styler Web & Graphic Specialist, Get more people to see your business @ www.BizOppers.biz John uses Plaxo, a smart address book that keeps your address book up-to-date and notifies your important contacts when you change your information. Learn more. http://www.plaxo.com/products?src=3Demail&et=3D79&el=3Den_us Join now for free http://www.plaxo.com/join/taf?src=3Demail&et=3D79&el=3Den_us&rem=3DOpYKXoiPVmS= KhpDu3qDZbWOwZtigQgCiqtNQVihqlzM%3D&suid=3DWwCFRqYnbpMkAONcISCRzHWRMlgGub8lK2E= ZdQ09YYk%3D Plaxo, Inc. - 203 Ravendale Drive - Mountain View - CA - 94043 - USA=20 Never Send Me Emails http://www.plaxo.com/stop?src=3Demail&et=3D79&el=3Den_us&email=3Dfreebsd-arch%= 40freebsd.org