From owner-freebsd-arch@FreeBSD.ORG Tue May 17 23:21:09 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 816401065672 for ; Tue, 17 May 2011 23:21:09 +0000 (UTC) (envelope-from root@ds80-237-154-16.dedicated.hosteurope.de) Received: from sambanacht.de (ds80-237-154-16.dedicated.hosteurope.de [80.237.154.16]) by mx1.freebsd.org (Postfix) with ESMTP id 485868FC14 for ; Tue, 17 May 2011 23:21:09 +0000 (UTC) Received: from nag (unknown [184.22.115.146]) by sambanacht.de (Postfix) with SMTP id 36F601C469B; Wed, 18 May 2011 00:56:33 +0200 (CEST) To: arch@freebsd.org, arbuhexs@alvhifpp.com, arbujam.oi@gmail.com, arbell.camron@sonymusic.com From: "Luna Elliott" Message-Id: <20110517232109.816401065672@hub.freebsd.org> Date: Tue, 17 May 2011 23:21:09 +0000 (UTC) Cc: Subject: mailing/email lists for sale X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: root@ds80-237-154-16.dedicated.hosteurope.de List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 May 2011 23:21:09 -0000 Healthcare List Package: $295 for all 21 complete lists below - Also available, Business Lists, Foregin Business Lists and Consumer data Physicians (34 specialties) Database : 788k records, 17k emails, 200k fax numbers Chiropractors Database : 108,421 total records * 3,414 emails * 6,553 fax numbers Alternative Medicine Database : 1,141,602 total records with 36,320 emails and 38.935 fax numbers Dentists Database : 164k records, 45k emails, 77k fax numbers Dentists with Specialties Database : 30k records all with emails Veterinarians Database : 78,986 total records with 1,438 emails and 1,050 fax numbers Hospitals Database : 10,661 Emails for hospitals in the USA Nursing Homes Database : 31,589 Senior Administrators, 11,288 Nursing Directors in over 14,706 Nursing Homes (full contact info no emails) Pharmaceutical Companies Database : 47,000 emails of pharma company employees Physical Therapists Database : 125,460 total records with 5,483 emails and 4,405 fax numbers Oncology Doctors Database : 2,200 records all with emails US Surgery Centers Database : 85k records and 14k emails Acupuncturists Database : 23,988 records 1,826 emails Massage Therapists Database : 76,701 records and 8,305 emails Medical Equipment Suppliers Database : 67,425 total records with 6,940 emails and 5,812 fax numbers Mental Health Counselors Database : 283,184 records 7,206 emails Visiting Nurses & RN's Database : 91,386 total records with 2,788 emails and 2,390 fax numbers Optometrists Database : 63,837 records 2,015 emails Psychologists Database : 272,188 records and 9,874 emails Healthcare Recruiters Database : 11,370 emails for active healthcare recruiters in the USA Plastic Surgeons Database : 23,109 emails with full data Send me an email here Jenny.Sufei@gmx.com for counts and sample spreadsheets. We also sell lists individually at reduced prices. kill the ssp*m by going here please email remlisting@gmx.de From owner-freebsd-arch@FreeBSD.ORG Wed May 18 16:40:06 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD7BB106564A; Wed, 18 May 2011 16:40:06 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id CDD178FC13; Wed, 18 May 2011 16:40:04 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA25040; Wed, 18 May 2011 19:40:03 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4DD3F662.9040603@FreeBSD.org> Date: Wed, 18 May 2011 19:40:02 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.17) Gecko/20110504 Lightning/1.0b2 Thunderbird/3.1.10 MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org, freebsd-current@FreeBSD.org X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: [rfc] remove hlt_cpus et al sysctls and related code X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2011 16:40:06 -0000 I think that it is a well known fact that currently we do not have any support for dynamically offlining processors. Yet, we have some code that looks like it does provide that support and even provides a user interface to supposedly do that. What we don't currently do specifically: - rebinding interrupts away from an offlined processor - updating relevant cpu sets and masks - protecting the above for concurrent access - moving threads away from an offlined processor - notifying potentially interested parties - maybe more... The code has been in this shape for a long while and I would dare to say that it never really worked, not in "production ready" sense anyway. An example of troubles caused by using that code can be found e.g. in the followups to the following PR: http://www.freebsd.org/cgi/query-pr.cgi?pr=145385 And also discussed here: http://thread.gmane.org/gmane.os.freebsd.stable/74462/focus=74510 I think that there already have been a proposal to remove the systcls and the code. I would like to re-submit that proposal. Removing that code would: 1) prevent users from hurting themselves by executing broken code 2) potentially make things easier for largeSMP project Once we grow correct code for offlining CPUs, then we could re-introduce the sysctls without any problems. While the offlining code doesn't seem terribly hard to develop, it's a big piece of work and requires time and effort. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Wed May 18 16:49:03 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7CCEF1065670; Wed, 18 May 2011 16:49:03 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id EE3398FC12; Wed, 18 May 2011 16:49:02 +0000 (UTC) Received: by vxc34 with SMTP id 34so1734450vxc.13 for ; Wed, 18 May 2011 09:49:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=gZowLRemBSQ+DEqXAswmBiyJy7dF78EqgQWIbO+XtXg=; b=w3xck7nMXkmhaImQpQ/H6WLKVSnyyyiHONkelKBCQ7Pvv/EaOtusvHIa/HdUTBpu43 6w/6VScEHN+0u7L2aNLPiaF+b3e5hxgI3xLBnHxV8XGMwoGYsXmQNtm1nHNtaVVvc01l qZWXvYKg7ngbjs/RcqnC5bnA9TOOrrVOuwVJo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=JOCFrIuWvcniZSI8R4OmP16ohTS20Sq5cizNTw2ZKxuYP0T5wI8hnvHFC3WqJeHhr8 mKe6P1CdDsHT0j6JcI1jVzxoY+V0gbxBtkKENUQN/5jqKdc5fMNMu9HrL6cWhIxfPOaL npbNXMBPmNjYO/A8tOvcUajbWhcY8ydaQHOZ8= MIME-Version: 1.0 Received: by 10.52.95.203 with SMTP id dm11mr2987039vdb.213.1305737341821; Wed, 18 May 2011 09:49:01 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.220.201.3 with HTTP; Wed, 18 May 2011 09:49:01 -0700 (PDT) In-Reply-To: References: <4DD3F662.9040603@FreeBSD.org> Date: Wed, 18 May 2011 12:49:01 -0400 X-Google-Sender-Auth: IRtDwsPfj1ipkk9E6aqWWBUQ3GI Message-ID: From: Attilio Rao To: Garrett Cooper Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-current@freebsd.org, Andriy Gapon , freebsd-arch@freebsd.org Subject: Re: [rfc] remove hlt_cpus et al sysctls and related code X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2011 16:49:03 -0000 2011/5/18 Garrett Cooper : > On Wed, May 18, 2011 at 9:40 AM, Andriy Gapon wrote: >> >> I think that it is a well known fact that currently we do not have any s= upport for >> dynamically offlining processors. =C2=A0Yet, we have some code that look= s like it does >> provide that support and even provides a user interface to supposedly do= that. >> >> What we don't currently do specifically: >> - rebinding interrupts away from an offlined processor >> - updating relevant cpu sets and masks >> - protecting the above for concurrent access >> - moving threads away from an offlined processor >> - notifying potentially interested parties >> - maybe more... >> >> The code has been in this shape for a long while and I would dare to say= that it >> never really worked, not in "production ready" sense anyway. >> An example of troubles caused by using that code can be found e.g. in th= e >> followups to the following PR: >> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D145385 >> And also discussed here: >> http://thread.gmane.org/gmane.os.freebsd.stable/74462/focus=3D74510 >> >> I think that there already have been a proposal to remove the systcls an= d the >> code. =C2=A0I would like to re-submit that proposal. >> Removing that code would: >> 1) prevent users from hurting themselves by executing broken code >> 2) potentially make things easier for largeSMP project >> >> Once we grow correct code for offlining CPUs, then we could re-introduce= the >> sysctls without any problems. >> While the offlining code doesn't seem terribly hard to develop, it's a b= ig piece >> of work and requires time and effort. > > =C2=A0 =C2=A0What would be nice too (even though it might not be possible= ) is > to make this more MI than it is today (i.e. sysctls that work for > amd64, sparc64, etc), but that might be a pipe dream. > Thanks! > -Garrett That is actually the purpose. We should have a real online/offline system for hotplugging CPUs, not only tied to x86 hyperthreading. The htt specific parts are mostly hacks that don't take into account all the necessary handover for it. Andryi, I'll look into the patch asap, but I'm in favor of this change for = sure. Attilio --=20 Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Wed May 18 16:52:45 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6C911106566C; Wed, 18 May 2011 16:52:45 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id E0CDF8FC12; Wed, 18 May 2011 16:52:44 +0000 (UTC) Received: by vws18 with SMTP id 18so1708612vws.13 for ; Wed, 18 May 2011 09:52:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=EhXRXlZ8nSay9Px+S38WHZyx8P+G4wR7rNXw26tOz1E=; b=sdIYad8c4hnPB72fXZmiiR8yG4phec9M6rEzAjme+UQAC7/54xxD4dOS9z8rk23VcL daUsbpW0FjTTQwq5xT8Fbkv1FJ3yIW0ylNjfldU9Vs308LEijWTIe/dMiuPw2DfvuPHx qtYHXqtH5Yf0ha1l9JYyjlhuFtWB5frIMXej8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=hWlSTEXxuAUD56bM77uyDn028PJ+r4NkSNRnnY6BPR5Enc7Fbkh+Lldc6bxKRTOePi QQAfuUxYtEAHCXX8AhhB0OO+pWRC15uegY1fGltB0PS39Hh0ipvgCrKcxpEhKZurmhCQ d71B4UaySJZzjJfQHltA2FUIptwYxzbVZ53YI= MIME-Version: 1.0 Received: by 10.220.42.74 with SMTP id r10mr621813vce.132.1305737563927; Wed, 18 May 2011 09:52:43 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.220.201.3 with HTTP; Wed, 18 May 2011 09:52:43 -0700 (PDT) In-Reply-To: References: <4DD3F662.9040603@FreeBSD.org> Date: Wed, 18 May 2011 12:52:43 -0400 X-Google-Sender-Auth: asD9RAhrk3Z7Osbh5T6nMVum8Cg Message-ID: From: Attilio Rao To: Garrett Cooper Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-current@freebsd.org, Andriy Gapon , freebsd-arch@freebsd.org Subject: Re: [rfc] remove hlt_cpus et al sysctls and related code X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2011 16:52:45 -0000 2011/5/18 Attilio Rao : > 2011/5/18 Garrett Cooper : >> On Wed, May 18, 2011 at 9:40 AM, Andriy Gapon wrote: >>> >>> I think that it is a well known fact that currently we do not have any = support for >>> dynamically offlining processors. =C2=A0Yet, we have some code that loo= ks like it does >>> provide that support and even provides a user interface to supposedly d= o that. >>> >>> What we don't currently do specifically: >>> - rebinding interrupts away from an offlined processor >>> - updating relevant cpu sets and masks >>> - protecting the above for concurrent access >>> - moving threads away from an offlined processor >>> - notifying potentially interested parties >>> - maybe more... >>> >>> The code has been in this shape for a long while and I would dare to sa= y that it >>> never really worked, not in "production ready" sense anyway. >>> An example of troubles caused by using that code can be found e.g. in t= he >>> followups to the following PR: >>> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D145385 >>> And also discussed here: >>> http://thread.gmane.org/gmane.os.freebsd.stable/74462/focus=3D74510 >>> >>> I think that there already have been a proposal to remove the systcls a= nd the >>> code. =C2=A0I would like to re-submit that proposal. >>> Removing that code would: >>> 1) prevent users from hurting themselves by executing broken code >>> 2) potentially make things easier for largeSMP project >>> >>> Once we grow correct code for offlining CPUs, then we could re-introduc= e the >>> sysctls without any problems. >>> While the offlining code doesn't seem terribly hard to develop, it's a = big piece >>> of work and requires time and effort. >> >> =C2=A0 =C2=A0What would be nice too (even though it might not be possibl= e) is >> to make this more MI than it is today (i.e. sysctls that work for >> amd64, sparc64, etc), but that might be a pipe dream. >> Thanks! >> -Garrett > > That is actually the purpose. =C2=A0We should have a real online/offline > system for hotplugging CPUs, not only tied to x86 hyperthreading. > The htt specific parts are mostly hacks that don't take into account > all the necessary handover for it. For instance, I always promised to implement them and I never did, mostly because it is quite a bit of work in my idea and we don't really have a big pressure for it and there are really bigger things on my plate. If someone wants to step up and implementing offline/online CPUs I would be glad to discuss approaches I have in mind and help formalizing a plan for it, thus offering guidance for the implementation. Attilio --=20 Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Wed May 18 17:01:27 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7E2A9106564A; Wed, 18 May 2011 17:01:27 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3982B8FC12; Wed, 18 May 2011 17:01:27 +0000 (UTC) Received: by pwj8 with SMTP id 8so1084977pwj.13 for ; Wed, 18 May 2011 10:01:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:references:in-reply-to:mime-version :content-transfer-encoding:content-type:message-id:cc:x-mailer:from :subject:date:to; bh=lFG374beTWD8niyTMihkFgrXD5hw4AGaErMjy3i3syk=; b=ZPMbvhdWObaFcAtWLCg8fEZXli5rV54lUnsCNvVN9iUUUDuTw64FbI3vIeob5VE3zo E8BtoWwTUyt6mtcwMz/ven8Vhug/YGwIIZ1Bu+OU6PS0OccwQ18WSjJXtSjY1YCv3D5X 0VRcd45xbQ3zi2WhjY0oZhx6YTB+i0SuBpe4U= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=references:in-reply-to:mime-version:content-transfer-encoding :content-type:message-id:cc:x-mailer:from:subject:date:to; b=r2EkusCyAnYA8M2OFYywsebagFYn3823ZuaWXR7yZTsaaIuj4L5uq824nbHRtuhQnr 8jNtHIWKUEZQqedEpsmIc7R2M9xAbB5xeGaDaPkzGb28tjOBfcN/SIB5OOtigrpoVDod LYmKM7ePe/+kljQoeaVGeo5gOTRaV7lP4n0u4= Received: by 10.68.46.195 with SMTP id x3mr3173696pbm.442.1305737620640; Wed, 18 May 2011 09:53:40 -0700 (PDT) Received: from [192.168.20.56] ([24.6.49.154]) by mx.google.com with ESMTPS id b10sm1162323pbq.22.2011.05.18.09.53.39 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 18 May 2011 09:53:39 -0700 (PDT) References: <4DD3F662.9040603@FreeBSD.org> In-Reply-To: Mime-Version: 1.0 (iPhone Mail 8J2) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Message-Id: X-Mailer: iPhone Mail (8J2) From: Garrett Cooper Date: Wed, 18 May 2011 09:53:34 -0700 To: Attilio Rao Cc: "freebsd-current@freebsd.org" , Andriy Gapon , "freebsd-arch@freebsd.org" Subject: Re: [rfc] remove hlt_cpus et al sysctls and related code X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2011 17:01:27 -0000 On May 18, 2011, at 9:49 AM, Attilio Rao wrote: > 2011/5/18 Garrett Cooper : >> On Wed, May 18, 2011 at 9:40 AM, Andriy Gapon wrote: >>>=20 >>> I think that it is a well known fact that currently we do not have any s= upport for >>> dynamically offlining processors. Yet, we have some code that looks lik= e it does >>> provide that support and even provides a user interface to supposedly do= that. >>>=20 >>> What we don't currently do specifically: >>> - rebinding interrupts away from an offlined processor >>> - updating relevant cpu sets and masks >>> - protecting the above for concurrent access >>> - moving threads away from an offlined processor >>> - notifying potentially interested parties >>> - maybe more... >>>=20 >>> The code has been in this shape for a long while and I would dare to say= that it >>> never really worked, not in "production ready" sense anyway. >>> An example of troubles caused by using that code can be found e.g. in th= e >>> followups to the following PR: >>> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D145385 >>> And also discussed here: >>> http://thread.gmane.org/gmane.os.freebsd.stable/74462/focus=3D74510 >>>=20 >>> I think that there already have been a proposal to remove the systcls an= d the >>> code. I would like to re-submit that proposal. >>> Removing that code would: >>> 1) prevent users from hurting themselves by executing broken code >>> 2) potentially make things easier for largeSMP project >>>=20 >>> Once we grow correct code for offlining CPUs, then we could re-introduce= the >>> sysctls without any problems. >>> While the offlining code doesn't seem terribly hard to develop, it's a b= ig piece >>> of work and requires time and effort. >>=20 >> What would be nice too (even though it might not be possible) is >> to make this more MI than it is today (i.e. sysctls that work for >> amd64, sparc64, etc), but that might be a pipe dream. >> Thanks! >> -Garrett >=20 > That is actually the purpose. We should have a real online/offline > system for hotplugging CPUs, not only tied to x86 hyperthreading. > The htt specific parts are mostly hacks that don't take into account > all the necessary handover for it. >=20 > Andryi, I'll look into the patch asap, but I'm in favor of this change for= sure. We use this internally at work still with a software config that uses 4B= SD so as long as there is an equivalent tunable, that's good enough for us m= oving forward. Thanks! -Garrett= From owner-freebsd-arch@FreeBSD.ORG Wed May 18 17:04:14 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A64D3106564A; Wed, 18 May 2011 17:04:14 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3156A8FC08; Wed, 18 May 2011 17:04:14 +0000 (UTC) Received: by vxc34 with SMTP id 34so1750361vxc.13 for ; Wed, 18 May 2011 10:04:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=CQIaPuMYI9b7eS795opJ4SRZNJfv/Ta3oH9sN6WBzi8=; b=lnTD3/FXWxfk9IN5E1shaMhrPUu/IlWHJjaYaJEcpyccfTvXZkmnBPtFIvO7+M9Vx7 bTF/mkNXWerEtbETIQ8Vepi9jxSEDVbbgBOTkjGMzstoW/qlSq7qmzON3m71kHHTUznX Kvf3mkLg7y9PkvImvgeY13S1jzn8RMghoeYgI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=hU1RT7Zub8Dp6V7xENOzW8AS1JhNzbls9VuRxj9AfEvDJUtXHfLXSiokP5Y2HMJJtO YCAxTfkyx+35gusZFjaC2DKGzgZTMwEvdP/T4/eZubz41mxhj/wOQUHCpf9Qk6nsiDPa u3fi1esIbdP36oUkqDvJJ2tuaf/+QaTaMXkEA= MIME-Version: 1.0 Received: by 10.52.95.203 with SMTP id dm11mr3010286vdb.213.1305738253465; Wed, 18 May 2011 10:04:13 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.220.201.3 with HTTP; Wed, 18 May 2011 10:04:13 -0700 (PDT) In-Reply-To: References: <4DD3F662.9040603@FreeBSD.org> Date: Wed, 18 May 2011 13:04:13 -0400 X-Google-Sender-Auth: WR1UlXTHniDXxPwzAE-lGuHJEXw Message-ID: From: Attilio Rao To: Garrett Cooper Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-current@freebsd.org" , Andriy Gapon , "freebsd-arch@freebsd.org" Subject: Re: [rfc] remove hlt_cpus et al sysctls and related code X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2011 17:04:14 -0000 2011/5/18 Garrett Cooper : > On May 18, 2011, at 9:49 AM, Attilio Rao wrote: > >> 2011/5/18 Garrett Cooper : >>> On Wed, May 18, 2011 at 9:40 AM, Andriy Gapon wrote: >>>> >>>> I think that it is a well known fact that currently we do not have any= support for >>>> dynamically offlining processors. =C2=A0Yet, we have some code that lo= oks like it does >>>> provide that support and even provides a user interface to supposedly = do that. >>>> >>>> What we don't currently do specifically: >>>> - rebinding interrupts away from an offlined processor >>>> - updating relevant cpu sets and masks >>>> - protecting the above for concurrent access >>>> - moving threads away from an offlined processor >>>> - notifying potentially interested parties >>>> - maybe more... >>>> >>>> The code has been in this shape for a long while and I would dare to s= ay that it >>>> never really worked, not in "production ready" sense anyway. >>>> An example of troubles caused by using that code can be found e.g. in = the >>>> followups to the following PR: >>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D145385 >>>> And also discussed here: >>>> http://thread.gmane.org/gmane.os.freebsd.stable/74462/focus=3D74510 >>>> >>>> I think that there already have been a proposal to remove the systcls = and the >>>> code. =C2=A0I would like to re-submit that proposal. >>>> Removing that code would: >>>> 1) prevent users from hurting themselves by executing broken code >>>> 2) potentially make things easier for largeSMP project >>>> >>>> Once we grow correct code for offlining CPUs, then we could re-introdu= ce the >>>> sysctls without any problems. >>>> While the offlining code doesn't seem terribly hard to develop, it's a= big piece >>>> of work and requires time and effort. >>> >>> =C2=A0 =C2=A0What would be nice too (even though it might not be possib= le) is >>> to make this more MI than it is today (i.e. sysctls that work for >>> amd64, sparc64, etc), but that might be a pipe dream. >>> Thanks! >>> -Garrett >> >> That is actually the purpose. =C2=A0We should have a real online/offline >> system for hotplugging CPUs, not only tied to x86 hyperthreading. >> The htt specific parts are mostly hacks that don't take into account >> all the necessary handover for it. >> >> Andryi, I'll look into the patch asap, but I'm in favor of this change f= or sure. > > =C2=A0 =C2=A0We use this internally at work still with a software config = that uses 4BSD so as long as there is an equivalent tunable, that's good en= ough for us moving forward. Tunables are pretty much acceptable for this case. What is really broken is the on-the-fly ability to mark CPUs active/inactive and subsequent handovers. I thought Andriy attached a patch to the tree, but it doesn't seem so... anyway, yes, I think that adding tunables for this is very reasonable and not as dangerous as the current mechanism. Attilio --=20 Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Wed May 18 17:08:42 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 65D3E1065670 for ; Wed, 18 May 2011 17:08:42 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1F1E38FC15 for ; Wed, 18 May 2011 17:08:41 +0000 (UTC) Received: by vxc34 with SMTP id 34so1754971vxc.13 for ; Wed, 18 May 2011 10:08:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=/ctk06kF8iSm+jaH13/MBau//5kkKXr0aFQ1hE+aK54=; b=PhkV0NUbMA+bEHgUcywxXd3eULssNBdBF2ZK9iMS4k9R0q58KL3bnfijb4sypjTZ2v Zmo/1Kaq2pUE5tTqMtzD6uIIW99K4d02AvWP4tndXia4EgN42gvS7N4uIICntgCzKoVv j2mrPnjjalUm+Rq54N2zVEim/zGohMTD6BegQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=ls8saRNdrExG20lNoljlpMYbu4epL/An2gX7VSHfHNH+5uHC13SK/DY7w57nKQW8Fw pyfvPI+aFvAptiAFQaRA9+Fnb7DSCLueQGm/0V6V5tVdymFaEhfXglXA3Uahb0c5RFhG XYa+BEtFD6DLr2Vmy4HyjMTw//PshvyD6gq6w= MIME-Version: 1.0 Received: by 10.220.247.139 with SMTP id mc11mr620238vcb.107.1305736987000; Wed, 18 May 2011 09:43:07 -0700 (PDT) Received: by 10.220.188.202 with HTTP; Wed, 18 May 2011 09:43:06 -0700 (PDT) In-Reply-To: <4DD3F662.9040603@FreeBSD.org> References: <4DD3F662.9040603@FreeBSD.org> Date: Wed, 18 May 2011 09:43:06 -0700 Message-ID: From: Garrett Cooper To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [rfc] remove hlt_cpus et al sysctls and related code X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2011 17:08:42 -0000 On Wed, May 18, 2011 at 9:40 AM, Andriy Gapon wrote: > > I think that it is a well known fact that currently we do not have any su= pport for > dynamically offlining processors. =A0Yet, we have some code that looks li= ke it does > provide that support and even provides a user interface to supposedly do = that. > > What we don't currently do specifically: > - rebinding interrupts away from an offlined processor > - updating relevant cpu sets and masks > - protecting the above for concurrent access > - moving threads away from an offlined processor > - notifying potentially interested parties > - maybe more... > > The code has been in this shape for a long while and I would dare to say = that it > never really worked, not in "production ready" sense anyway. > An example of troubles caused by using that code can be found e.g. in the > followups to the following PR: > http://www.freebsd.org/cgi/query-pr.cgi?pr=3D145385 > And also discussed here: > http://thread.gmane.org/gmane.os.freebsd.stable/74462/focus=3D74510 > > I think that there already have been a proposal to remove the systcls and= the > code. =A0I would like to re-submit that proposal. > Removing that code would: > 1) prevent users from hurting themselves by executing broken code > 2) potentially make things easier for largeSMP project > > Once we grow correct code for offlining CPUs, then we could re-introduce = the > sysctls without any problems. > While the offlining code doesn't seem terribly hard to develop, it's a bi= g piece > of work and requires time and effort. What would be nice too (even though it might not be possible) is to make this more MI than it is today (i.e. sysctls that work for amd64, sparc64, etc), but that might be a pipe dream. Thanks! -Garrett From owner-freebsd-arch@FreeBSD.ORG Thu May 19 16:58:04 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 164081065670; Thu, 19 May 2011 16:58:04 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id E5EB78FC15; Thu, 19 May 2011 16:58:02 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA17489; Thu, 19 May 2011 19:58:01 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4DD54C18.8050305@FreeBSD.org> Date: Thu, 19 May 2011 19:58:00 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.17) Gecko/20110504 Lightning/1.0b2 Thunderbird/3.1.10 MIME-Version: 1.0 To: Attilio Rao , Garrett Cooper References: <4DD3F662.9040603@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: "freebsd-current@freebsd.org" , "freebsd-arch@freebsd.org" Subject: Re: [rfc] remove hlt_cpus et al sysctls and related code X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 May 2011 16:58:04 -0000 on 18/05/2011 20:04 Attilio Rao said the following: > 2011/5/18 Garrett Cooper : >> We use this internally at work still with a software config that uses 4BSD so >> as long as there is an equivalent tunable, that's good enough for us moving >> forward. Can you please clarify which exactly tunable(s) do you use/need? Just turning hyperthreading on/off or more? (BTW, doing that via BIOS is inconvenient / not feasible?) BTW, I think that if we switch hyperthreading off then we better off not sending Start IPI to the logical CPUs at all. > Tunables are pretty much acceptable for this case. What is really broken is the > on-the-fly ability to mark CPUs active/inactive and subsequent handovers. Yes, I completely agree. Static disabling of CPUs doesn't have any problems, and IMO, currently the best way to do it is with hint.lapic.X.disabled. > I thought Andriy attached a patch to the tree, but it doesn't seem so... > anyway, yes, I think that adding tunables for this is very reasonable and not > as dangerous as the current mechanism. I agree. I haven't sent a patch, because I don't have it yet :) I decided to solicit opinions before getting to hacking code. -- Andriy Gapon