From owner-freebsd-hackers@freebsd.org Sun Mar 3 05:21:06 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DF6A915244D5 for ; Sun, 3 Mar 2019 05:21:05 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-36.consmr.mail.ne1.yahoo.com (sonic317-36.consmr.mail.ne1.yahoo.com [66.163.184.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9747A6EDE5 for ; Sun, 3 Mar 2019 05:21:04 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: qYN4k3IVM1lUtekxELAbd42EJuBb7C7eA4P_EyTRQ_K6TdyTM1cbzUqDPu4KV3M gCAgqy6wc2BiQtk5oJNAIq01M_Uu4Asixqsb7ZN3hnyc69CwT_Msst4bN0GVoolnXqlb0rLIJPnj FXS2gaBHA1Isy7P0rBXxr4KIVv2_BMmb3KVqJfGOxjqMwW.y_sEXy30xBoP6SnY3OwiHv0IiPUxq b9eQRWZPz15hZaTBofYkBuyN96FULOQ_zhxgrVEcTWUFvCfv.Aik6oELWCjsDLf1iABFREPoTfXm MnM4sSEFx9j3n.xlYnHdZdFGMFU8UoHf2Zi38trFbU6aYre2z9qVvyeJrSt2AxfyUqedFroS9LBc 76NMtjfd24WYb1_Juzx5gFqqyi2H3COYAzSiDjV_WSMa6FptXaIgB9HT6UI0KlAlG9m9zLJeKPXQ 2L2wfD_VAl8e_ld73o091ws80Vigl0FZBKmlUBE5KyRyFAiQ4Dd37xATiStRGJZ7VKx8XXmcnook R7fpOBOjcr5gvJEN3WMQaUf9hl1NLvKw009PAPSvvjx8yDmtdpdT2HvQSVclpudBO8cKaTngu3Uc Ts6Ls9NtHb_Cbn2ZIwXH42CEhL8lgwX9sC282o8COW9JJSnd9FjU9UE4qoIba7REjzvIEEe5fuUS stMoDANNAyGzoGHmaD_3EWyHtK9_.bAGO_OLTi9siYT6cdo_zMCfywcG37E6o_1SNlzs8YQPEKjr 9CJBwkeYNRTLiw0lVPqa.QOtcNFlZgGgTDg7XHbamIxD6jdLoJ5u1lD6dGalUKYl7ucX3KRSYxUP idSJGzkneMgDgQEU6lRkUCrE0PxiAZYOpmWKzR2TXU8AYb.egbqYnMJIsyALShoDB8N7B6L_mMI1 hbHRXrwNWu9d7KTy.qc0gY5.ab2lZ2PrDqcJLJiSjUl0T5y9EfgZI_U5dUH2b2RNHVGvnsnkf0BH _MVAefJoZxjqY4TgAFPpbNVfAta4tY_D6Rv.8uE5eXOscoHxOEovK.x22KjM547i9jJzRgwyGfiq yTg1ryJr5cwkeW.zWR4ocR51z.CGrUbBMCRZ9zUx5GLYxUW25PdM5lhvl0nxz2GwWhYQ6lJQ- Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.ne1.yahoo.com with HTTP; Sun, 3 Mar 2019 05:21:02 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.115]) ([67.170.167.181]) by smtp421.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 44d2170f61a51d6c5540268ab4cad8d3; Sun, 03 Mar 2019 05:21:00 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: powerpc64 on PowerMac G5 4-core (system total): a hack that so far seem to avoid the stuck-sleeping issue Message-Id: Date: Sat, 2 Mar 2019 21:20:58 -0800 To: FreeBSD PowerPC ML , Mark Millard via freebsd-hackers X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: 9747A6EDE5 X-Spamd-Bar: +++ X-Spamd-Result: default: False [3.33 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.67)[0.672,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.33)[ip: (4.40), ipnet: 66.163.184.0/21(1.29), asn: 36646(1.03), country: US(-0.07)]; NEURAL_SPAM_MEDIUM(0.96)[0.956,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.88)[0.879,0]; RCVD_IN_DNSWL_NONE(0.00)[47.184.163.66.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Mar 2019 05:21:06 -0000 [This note goes in a different direction compared to my prior evidence report for overflows and the later activity that has been happening for it. This does *not* involve the patches associated with that report.] I view the following as an evidence-gathering hack: showing the change in behavior with the code changes, not as directly what FreeBSD should do for powerpc64. In code for defined(__powerpc64__) && defined(AIM) I freely use knowledge of the PowerMac G5 context instead of attempting general code. Also: the code is set up to record some information that I've been looking at via ddb. The recording is not part of what changes the behavior but I decided to show that code too. It is preliminary, but, so far, the hack has avoided buf*daemon* threads and pmac_thermal getting stuck sleeping (or, at least, far less frequently). The tbr-value hack: =46rom what I see the G5 various cores have each tbr running at the same rate but have some some offsets as far as the base time goes. cpu_mp_unleash does: ap_awake =3D 1; /* Provide our current DEC and TB values for APs */ ap_timebase =3D mftb() + 10; __asm __volatile("msync; isync"); /* Let APs continue */ atomic_store_rel_int(&ap_letgo, 1); platform_smp_timebase_sync(ap_timebase, 0); and machdep_ap_bootstrap does: /* * Set timebase as soon as possible to meet an implicit = rendezvous * from cpu_mp_unleash(), which sets ap_letgo and then = immediately * sets timebase. * * Note that this is instrinsically racy and is only relevant on * platforms that do not support better mechanisms. */ platform_smp_timebase_sync(ap_timebase, 1); which attempts to set the tbrs appropriately. But on small scales of differences the various tbr values from different cpus end up not well ordered relative to time, synchronizes with, and the like. Only large enough differences can well indicate an ordering of interest. Note: tc->tc_get_timecount(tc) only provides the least signficant 32 bits of the tbr value. th->th_offset_count is also 32 bits and based on truncated tbr values. So I made binuptime avoid finishing when it sees a small (<0x10) step backwards for a new tc->tc_get_timecount(tc) value vs. the existing th->th_offset_count value (values strongly tied to powerpc64 tbr values): void binuptime(struct bintime *bt) { struct timehands *th; u_int gen; struct bintime old_bt=3D *bt; // HACK!!! struct timecounter *tc; // HACK!!! u_int tim_cnt, tim_offset, tim_diff; // HACK!!! uint64_t freq, scale_factor, diff_scaled; // HACK!!! u_int try_cnt=3D 0ull; // HACK!!! do { do { // HACK!!! th =3D timehands; tc =3D th->th_counter; gen =3D atomic_load_acq_int(&th->th_generation); tim_cnt=3D tc->tc_get_timecount(tc); tim_offset=3D th->th_offset_count; } while (tim_cntth_offset; tim_diff=3D (tim_cnt - tim_offset) & = tc->tc_counter_mask; scale_factor=3D th->th_scale; diff_scaled=3D scale_factor * tim_diff; bintime_addx(bt, diff_scaled); freq=3D tc->tc_frequency; atomic_thread_fence_acq(); try_cnt++; } while (gen =3D=3D 0 || gen !=3D th->th_generation); if (*(volatile uint64_t*)0xc000000000000020=3D=3D0u && = (0xffffffffffffffffull/scale_factor)tc_get_timecount(tc) not actually indicating a useful < vs. =3D=3D vs. > ordering relation uniquely. (I make no claim that the hack is a proper way to deal with such.) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)