From owner-freebsd-hackers@freebsd.org Sun Nov 4 17:22:26 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7972310DDAE5 for ; Sun, 4 Nov 2018 17:22:26 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B99477B0A0; Sun, 4 Nov 2018 17:22:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id wA4HM73T010279 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 4 Nov 2018 19:22:11 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua wA4HM73T010279 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id wA4HM7V0010278; Sun, 4 Nov 2018 19:22:07 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 4 Nov 2018 19:22:07 +0200 From: Konstantin Belousov To: Daniel Braniss Cc: Dimitry Andric , freebsd-hackers@FreeBSD.org Subject: Re: help with dl module and clang Message-ID: <20181104172207.GB5335@kib.kiev.ua> References: <4B8101F4-9A89-4486-8DF3-93DC799EF6D4@cs.huji.ac.il> <2E58B6A3-B3E4-4266-9B40-5F3D64433460@cs.huji.ac.il> <08A14290-0A6E-4002-892E-D4254EB23076@cs.huji.ac.il> <20181103163508.GS5335@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-Rspamd-Queue-Id: B99477B0A0 X-Spamd-Result: default: False [-2.29 / 200.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.98)[-0.980,0]; IP_SCORE(-1.30)[ipnet: 2001:470::/32(-3.43), asn: 6939(-2.98), country: US(-0.07)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; RCVD_TLS_LAST(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Nov 2018 17:22:26 -0000 On Sat, Nov 03, 2018 at 07:31:33PM +0200, Daniel Braniss wrote: > > > > On 3 Nov 2018, at 18:35, Konstantin Belousov wrote: > > > > On Sat, Nov 03, 2018 at 06:11:19PM +0200, Daniel Braniss wrote: > >> > >> > >>> On 3 Nov 2018, at 17:58, Daniel Braniss wrote: > >>> > >>> > >>> > >>>> On 3 Nov 2018, at 17:47, Dimitry Andric wrote: > >>>> > >>>> On 3 Nov 2018, at 15:51, Daniel Braniss wrote: > >>>>> > >>>>> I have a program that loads some modules via dlopen(), these modules call some routines > >>>>> which are in the main program, this works when using gcc, but with cc it does not. > >>>>> > >>>>> when compiling the main program I use -export-dynamic, and the modules link fine when compiled with > >>>>> gcc, but when compiling with clang/cc i get dlerror: ...Undefined symbol … > >>>>> BTW, when linking the main program with cc I get > >>>>> /usr/bin/ld: warning: cannot find entry symbol xport-dynamic; defaulting to 0000000000402140 > >>>> > >>>> Instead of using -export-dynamic (which is a linker flag) as a flag to > >>>> cc, try using -Wl,-export-dynamic instead. Now, the linker interprets > >>>> this as the -e flag, which is something totally different. > >>>> > >>> > >>> i had tried that before but i had -WI (upper case i) but re-reading the manual > >>> it should be-Wl (lower case l as in lima :-))!!!!! > >>> > >>> thanks!!!!!! > >>> > >>> and now have to try profiling (gprof) which started this mess, > >>> > >>> thanks again!! > >>> > >>> danny > >>> > >>> > >>>> I think this will also help with the exporting of the symbols of your > >>>> main program. > >>>> > >>>> -Dimitry > >>>> > >>> > >> ok, new problem, it also happens with gcc: > >> > >> when compiling the main program with flag -pg (gprof) > >> dlopen fails with dlerror: Service unavailable > > This means that your binary is linked statically. > > dlopen(3) is not supported for static linking, ld-elf.so.1 is required > > for dynamic loading to work. > > as far as I can tell, it’s NOT statically linked: > l > e-kots-b# ldd /vol/src/libexec/idng/idngd/idngd > /vol/src/libexec/idng/idngd/idngd: > libpq.so.5 => /usr/local/lib/libpq.so.5 (0x800932000) > libcrypto.so.8 => /lib/libcrypto.so.8 (0x800c00000) > libldap-2.4.so.2 => /usr/local/lib/libldap-2.4.so.2 (0x801070000) > libthr.so.3 => /lib/libthr.so.3 (0x8012b7000) > libintl.so.8 => /usr/local/lib/libintl.so.8 (0x8014df000) > libssl.so.8 => /usr/lib/libssl.so.8 (0x8016ea000) > libc.so.7 => /lib/libc.so.7 (0x80195d000) > liblber-2.4.so.2 => /usr/local/lib/liblber-2.4.so.2 (0x801d18000) > e-kots-b# file !$ > file /vol/src/libexec/idng/idngd/idngd > /vol/src/libexec/idng/idngd/idngd: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 11.2 (1102501 ), FreeBSD-style, not stripped I suppose that dlopen(3) is called from the text of the idngd binary ? Put the readelf -a idngd output somewhere. From owner-freebsd-hackers@freebsd.org Sun Nov 4 17:31:04 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 711D810DF245 for ; Sun, 4 Nov 2018 17:31:04 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EA5EF7B8A3; Sun, 4 Nov 2018 17:30:55 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from imac.bk.cs.huji.ac.il ([132.65.179.42]) by kabab.cs.huji.ac.il with esmtp id 1gJMER-0005GX-Sr; Sun, 04 Nov 2018 19:30:39 +0200 From: Daniel Braniss Mime-Version: 1.0 (Mac OS X Mail 12.0 \(3445.100.39\)) Subject: Re: help with dl module and clang Date: Sun, 4 Nov 2018 19:30:39 +0200 References: <4B8101F4-9A89-4486-8DF3-93DC799EF6D4@cs.huji.ac.il> <2E58B6A3-B3E4-4266-9B40-5F3D64433460@cs.huji.ac.il> <08A14290-0A6E-4002-892E-D4254EB23076@cs.huji.ac.il> <20181103163508.GS5335@kib.kiev.ua> <20181104172207.GB5335@kib.kiev.ua> To: Dimitry Andric , freebsd-hackers@FreeBSD.org In-Reply-To: <20181104172207.GB5335@kib.kiev.ua> Message-Id: <7ACC89D8-ECFF-41D0-AAB4-6D88F1387117@cs.huji.ac.il> X-Mailer: Apple Mail (2.3445.100.39) X-Rspamd-Queue-Id: EA5EF7B8A3 X-Spamd-Result: default: False [3.09 / 200.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[huji.ac.il]; AUTH_NA(1.00)[]; URI_COUNT_ODD(1.00)[3]; IP_SCORE(0.01)[country: EU(0.04)]; NEURAL_SPAM_SHORT(0.70)[0.696,0]; MX_GOOD(-0.01)[kabab.cs.huji.ac.il,post.cs.huji.ac.il]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[210.116.65.132.list.dnswl.org : 127.0.10.0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:378, ipnet:132.64.0.0/13, country:EU]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Server: mx1.freebsd.org Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Nov 2018 17:31:04 -0000 > On 4 Nov 2018, at 19:22, Konstantin Belousov = wrote: >=20 > On Sat, Nov 03, 2018 at 07:31:33PM +0200, Daniel Braniss wrote: >>=20 >>=20 >>> On 3 Nov 2018, at 18:35, Konstantin Belousov = wrote: >>>=20 >>> On Sat, Nov 03, 2018 at 06:11:19PM +0200, Daniel Braniss wrote: >>>>=20 >>>>=20 >>>>> On 3 Nov 2018, at 17:58, Daniel Braniss = wrote: >>>>>=20 >>>>>=20 >>>>>=20 >>>>>> On 3 Nov 2018, at 17:47, Dimitry Andric wrote: >>>>>>=20 >>>>>> On 3 Nov 2018, at 15:51, Daniel Braniss = wrote: >>>>>>>=20 >>>>>>> I have a program that loads some modules via dlopen(), these = modules call some routines >>>>>>> which are in the main program, this works when using gcc, but = with cc it does not. >>>>>>>=20 >>>>>>> when compiling the main program I use -export-dynamic, and the = modules link fine when compiled with >>>>>>> gcc, but when compiling with clang/cc i get dlerror: = ...Undefined symbol =E2=80=A6 >>>>>>> BTW, when linking the main program with cc I get >>>>>>> /usr/bin/ld: warning: cannot find entry symbol = xport-dynamic; defaulting to 0000000000402140 >>>>>>=20 >>>>>> Instead of using -export-dynamic (which is a linker flag) as a = flag to >>>>>> cc, try using -Wl,-export-dynamic instead. Now, the linker = interprets >>>>>> this as the -e flag, which is something totally different. >>>>>>=20 >>>>>=20 >>>>> i had tried that before but i had -WI (upper case i) but = re-reading the manual >>>>> it should be-Wl (lower case l as in lima :-))!!!!! >>>>>=20 >>>>> thanks!!!!!! >>>>>=20 >>>>> and now have to try profiling (gprof) which started this mess, >>>>>=20 >>>>> thanks again!! >>>>>=20 >>>>> danny >>>>>=20 >>>>>=20 >>>>>> I think this will also help with the exporting of the symbols of = your >>>>>> main program. >>>>>>=20 >>>>>> -Dimitry >>>>>>=20 >>>>>=20 >>>> ok, new problem, it also happens with gcc: >>>>=20 >>>> when compiling the main program with flag -pg (gprof) >>>> dlopen fails with dlerror: Service unavailable >>> This means that your binary is linked statically. >>> dlopen(3) is not supported for static linking, ld-elf.so.1 is = required >>> for dynamic loading to work. >>=20 >> as far as I can tell, it=E2=80=99s NOT statically linked: >> l >> e-kots-b# ldd /vol/src/libexec/idng/idngd/idngd=20 >> /vol/src/libexec/idng/idngd/idngd: >> libpq.so.5 =3D> /usr/local/lib/libpq.so.5 (0x800932000) >> libcrypto.so.8 =3D> /lib/libcrypto.so.8 (0x800c00000) >> libldap-2.4.so.2 =3D> /usr/local/lib/libldap-2.4.so.2 = (0x801070000) >> libthr.so.3 =3D> /lib/libthr.so.3 (0x8012b7000) >> libintl.so.8 =3D> /usr/local/lib/libintl.so.8 (0x8014df000) >> libssl.so.8 =3D> /usr/lib/libssl.so.8 (0x8016ea000) >> libc.so.7 =3D> /lib/libc.so.7 (0x80195d000) >> liblber-2.4.so.2 =3D> /usr/local/lib/liblber-2.4.so.2 = (0x801d18000) >> e-kots-b# file !$ >> file /vol/src/libexec/idng/idngd/idngd >> /vol/src/libexec/idng/idngd/idngd: ELF 64-bit LSB executable, x86-64, = version 1 (FreeBSD), dynamically linked, interpreter = /libexec/ld-elf.so.1, for FreeBSD 11.2 (1102501 ), = FreeBSD-style, not stripped >=20 > I suppose that dlopen(3) is called from the text of the idngd binary ? correct > Put the readelf -a idngd output somewhere. www.cs.huji.ac.il/~danny/idngd.elf.out = From owner-freebsd-hackers@freebsd.org Mon Nov 5 17:34:57 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 631521107007 for ; Mon, 5 Nov 2018 17:34:57 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-io1-xd44.google.com (mail-io1-xd44.google.com [IPv6:2607:f8b0:4864:20::d44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B956E6A4FD for ; Mon, 5 Nov 2018 17:34:56 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-io1-xd44.google.com with SMTP id o19-v6so7121210iod.3 for ; Mon, 05 Nov 2018 09:34:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=al7cUUx8n+e48suvX/NjkjQwl5RzhSub4L6eOTheUpM=; b=AZLo6xvQApIPdT9uyVxQV5ywg2FlAyrMgRG/ck3W0kQqBwPHz3jdwnbcLL6p6k7s1N pf6HUSHaPGJkTVNxd6XN+64FpQEZk+rdHs2GiF+ZMXz1lo2wcKQ0SKqblwly0yPrew+i tR+A+vRzmWK283I5Ga/oz648HFrhQSl9sn0SED0n4e0IZoO77yGtzMzs9mKp7r5euP4N OJUmnuNrAZ+BnYckBHNAAfsvhmly8OT7xWg1iPReGE1jdnD8uez7OmyWO8LezyJ7NQ94 5nrFqDKwfBW55g0Q/2ydfd5UI5me3zAylSIYZcSpMs3yvAkdZ5kMU/qIUvIfFGq8J9o7 YQPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=al7cUUx8n+e48suvX/NjkjQwl5RzhSub4L6eOTheUpM=; b=GvLuSlehrK9AQXC7vRiV2dVHVjM4hz+oIO6yebNxPDpIoIW/2x43tR3sK+TFIVE/eI Ue4B9d1yhVq/VlIbMI98lWFLp5fhy4OEd7XlpzVm9h+IBkiDXDgvftZQaYP8oT8CGp9f Ua+e4ybaaclgQnZou+6ynvirZgMPW+kIahAtq3imcfJ2pM7m23FU5F47VYr0x7+MMF7w R5V7kazEHFUBc/AHp4v6SLbxogsxd+ew4E/E6l8njRrx1i2Th3sXZAo9SSBRvEy9v1af +OyhN0eVhKfX82pUNTcYt2l4vdbJYhBOt6RRQq1J8lUHr+ChO0KQgATfKbVwHDWREq1K br9w== X-Gm-Message-State: AGRZ1gLI0is1VFquERikwVGHpUksGcpHoR+mJWtPhDC1D9KePqpQBIRC jdTvnF6qByEK/vtSpMD9M49lkWgM X-Google-Smtp-Source: AJdET5fqoo3Ta7hYs6dW6XSb0PCnQ8KLnIAjRnyNXtb95P17qmlKKKIYOsy0J4atrrl2+NcBy7g1VQ== X-Received: by 2002:a6b:c7c7:: with SMTP id x190-v6mr19234420iof.30.1541439296140; Mon, 05 Nov 2018 09:34:56 -0800 (PST) Received: from raichu (toroon0560w-lp130-08-67-71-176-199.dsl.bell.ca. [67.71.176.199]) by smtp.gmail.com with ESMTPSA id z18-v6sm14235783iog.57.2018.11.05.09.34.55 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 05 Nov 2018 09:34:55 -0800 (PST) Sender: Mark Johnston Date: Mon, 5 Nov 2018 12:34:53 -0500 From: Mark Johnston To: "Bjoern A. Zeeb" Cc: freebsd-hackers@freebsd.org Subject: Re: [CFT] capsicum patches for rtsol(8) and rtsold(8) Message-ID: <20181105173453.GB78763@raichu> References: <20181015194212.GA2751@spy> <20181016165308.GB5066@raichu> <86D87437-BD34-489A-87B7-33F1089080EE@lists.zabbadoz.net> <20181016200414.GD5066@raichu> <2A564C8A-FB64-4D2A-9E3E-392F1FCA66BD@lists.zabbadoz.net> <20181024195627.GI45118@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Rspamd-Queue-Id: B956E6A4FD X-Spamd-Result: default: False [-1.93 / 200.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com]; NEURAL_HAM_MEDIUM(-0.99)[-0.993,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_SHORT(-0.82)[-0.821,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[4.4.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCVD_TLS_LAST(0.00)[]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; MID_RHS_NOT_FQDN(0.50)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; IP_SCORE(-0.41)[ipnet: 2607:f8b0::/32(-0.54), asn: 15169(-1.45), country: US(-0.06)] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2018 17:34:57 -0000 On Fri, Nov 02, 2018 at 11:25:40PM +0000, Bjoern A. Zeeb wrote: > On 24 Oct 2018, at 19:56, Mark Johnston wrote: > > Hi, > > sorry I lost track on this. No problem, thanks for testing. > > Yes, I just uploaded a new version of the patch to > > https://people.freebsd.org/~markj/patches/rtsold_capsicum.diff and > > would > > appreciate any further testing that you can do. > > The rtsol Makefile does not cleanly apply to my HEAD. I updated the patch yet again, it should apply this time. The newest version requires HEAD as of r340130. > Also there seems to be an empty .else case in both rtsold and rtsol > Makefiles. Fixed, thanks. > Also I couldn’t get rtsol to link until I realised that it was rescue > which didn’t want to link (-j24 output can be confusing). I guess > with -DSMALL gone and MK_CASPER not turned off for rescue or whatever it > is the result is not what we expect? In other cases (e.g., ping(8)) we simply disable libcasper support for rescue binaries, so I've done the same here. > >>> resolvconf -a will only update /etc/resolv.conf if the info in > >>> /var/run/resolvconf/interfaces/vtnet0 has changed, I believe. Try > >>> deleting that file too, and then try running rtsol. > >> > >> When I deleted /etc/resolv.conf and then rtsol manually it had > >> re-appeared. Unclear to me what was in /var/run; I just wanted to > >> point > >> out the difference in behaviour; maybe you are right; I’ll go and > >> check if deleting in /var/run/ as well makes a difference. > > > > I don't observe that behaviour with either the stock or patched > > rtsol(8): for resolvconf(8) to update /etc/resolv.conf (or re-generate > > it), something under /var/run/resolvconf/interfaces needs to have > > changed. So, in my case, deleting /etc/resolv.conf *and* > > /var/run/resolvconf/interfaces/re0:slaac will cause resolv.conf to be > > regenerated once rtsold(8) decides to re-run resolvconf(8), but > > deleting resolv.conf on its own will not. > > I wonder if that’s a bug (unrelated to yours). I also noticed that > when my nameservers changed /etc/resolv.conf did not always reflect > this. From owner-freebsd-hackers@freebsd.org Mon Nov 5 21:23:35 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0D75A110C24E for ; Mon, 5 Nov 2018 21:23:35 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-it1-x135.google.com (mail-it1-x135.google.com [IPv6:2607:f8b0:4864:20::135]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 683CB73983 for ; Mon, 5 Nov 2018 21:23:34 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: by mail-it1-x135.google.com with SMTP id k206-v6so14973103ite.0 for ; Mon, 05 Nov 2018 13:23:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=XoNWpjKvqUfNOjCRyjsHK3x9UBB23dmpsEi4WD0hQxc=; b=OzFBZJPwJrNvVAK4AzspBbXyEvTTIZ8HpMmZfKXag7YEQZGHivS+lDk20fVrUA965A 4vTeuSoSXrylpgGSsPxWKP0QibxsNvTDvzMGMEby1Tw9XrTSFDUdn4/Gg4A+/SpXr4CA bM/SfijJWFLHJzREeEei3I/UVb27oOce1o58rbVvMyH14JwyS6rk8joIGhnkCfI8lORI riTpcYlkACqPO1/TtUjFBsskc5U+H/gFoXFfnWwMOPVgt5iFOVHvyOXqgTJR4P2veO9H ixYN16MfI2hubL2GrgxB2yfdh3AufIb5HwWmLwXWb9Ot+7gBObO3AiSUaTy2ASmaoqmp OCuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=XoNWpjKvqUfNOjCRyjsHK3x9UBB23dmpsEi4WD0hQxc=; b=LEhLiWTrK+dbKkKMkxl0Q7CZbXZ7xH0e9R301yhJHPI8Ur3x/kIwFPGM+5tkKgnhFj PuA3BIcKZfzEyawY6AQuwQ7q7yadE2tfC0BQxxlcBGROjU98Kod+UHvT+E3HYF7XkBsp iqls6llf2SL+3Cr+qD51u2vbyddFYY8yCLjulnbiig+Iwuz9qRQVW1fG525sQyIK3wdX kWt8DeMU2d13eOVVniPI4cbNkRu26cgOHVLp6Mq96jGvcCfTWd8js9nok644L964PHy+ uIUuBpBuMYsqnqv9hHqDq9YZNhMr+ojOT8TDnT2JN82hTBs1vt1mo4sIKjLhPizSQWzk zU3A== X-Gm-Message-State: AGRZ1gJT6wMPVf2WviL17+CxqBwc8TiEj14S/AxYeNVsjRyQcKZcB8s6 D1Xe6qJNdK9bY5UO64s4L0pA0PH/5PC+73GwQFhzI9I= X-Google-Smtp-Source: AJdET5e7wDurD3MNwc0Q1cAaL5bUxv2YAbNtuNENEnHMrl6GBiZ/kNNm6yunSv49JFMi9MDxZBoTaNU2GjH5EOKejdA= X-Received: by 2002:a24:e0b:: with SMTP id 11-v6mr8400377ite.60.1541453013386; Mon, 05 Nov 2018 13:23:33 -0800 (PST) MIME-Version: 1.0 From: Zaphod Beeblebrox Date: Mon, 5 Nov 2018 16:23:22 -0500 Message-ID: Subject: Threadripper Thermal strangeness. To: FreeBSD Hackers X-Rspamd-Queue-Id: 683CB73983 X-Spamd-Result: default: False [-2.98 / 200.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.91)[-0.905,0]; R_DKIM_ALLOW(-0.20)[gmail.com]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-0.37)[ipnet: 2607:f8b0::/32(-0.49), asn: 15169(-1.29), country: US(-0.06)]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[5.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.70)[-0.696,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Server: mx1.freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2018 21:23:35 -0000 So ... I have the lowliest threadripper ... the 1900X in a motherboard that I dual-boot between Windows 10 and FreeBSD. Under windows 10, the variable fan stuff behaves. Under light load, the CPU sticks around 40C and under heavy load it can spike as high as 55C as the water pump and fans spool up. Even though the variable fan stuff seems to operate under FreeBSD (as a function of the motherboard's microcontroller or whatnot), as I was having problems, I disabled the feature and set all fans to simply "on" or 100%. kldloading "amdtemp.ko" reports an idle temp of 55C or so and heavy compiling (make world -j32) reports temperatures as high as 95C ... upto and including some instability and/or crash or rebooting. ... now... is this due to thermal throttling not kicking in? Is windows miss-reporting the temperature? Is FreeBSD miss-reporting the temperature? Has anyone else experienced this? From owner-freebsd-hackers@freebsd.org Mon Nov 5 22:21:12 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BD717110D9CA for ; Mon, 5 Nov 2018 22:21:12 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 04B2C772FC; Mon, 5 Nov 2018 22:21:12 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: by mail-ed1-x535.google.com with SMTP id y10-v6so2379000edr.4; Mon, 05 Nov 2018 14:21:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:date:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IhYcbedyvyTLZZmFqSjgiOS3MsnOSMuWXcu7RooAEyo=; b=OMzpc0xnCdcfsENnBkYaDTzWpq6O1qTKZXNIZOXtYYMrFhPI0AA0zxuqeQpAgvTFVg hWO1v8XzQlU/wVg0Ax1w6A4IjMrJbZdqBS4wvM9KwTQxIT4ACaWp49ILZgTuynz3fAJ+ pufcphRckmxJZ/yiCh2eiitbzCzZi/rT3LHXT490ppE9L98GcvyJ1ZW7b3nz0EY8VczN nt8FSLyUKJednYU1OhEmW4UcQA+40OoTqXM8PYyhC04D7yNTjIpXk8ybVtG2eHyuBDif kNY6AlhFgH8jwJBTLASmcD1naLjH/lNUEtKUgAn+FF+iYs7oZZzZ4bwwR8p0JXZGe3hG ouPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:date:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IhYcbedyvyTLZZmFqSjgiOS3MsnOSMuWXcu7RooAEyo=; b=ioQH0QLJ9OaE4F6qXevvP9A8IsTt4+pYO76gqAnz6gA/qikb5LFstoNL77x31pL68N BW8tQ6RUi/90R8pP5c1ZO598FNEuH7dQIhMlEZVotmGT+G1qpwoPxxkUeldzpiJ/k0fs qFV12/TXS674FOZfGiB2CrMS0xJ8A+Cx95xBdqXRJ5UY2+Z/p0RIt5A6uzlAIn3FIyBP 8zWcGV8QMC1cETqoytS/rLN916oYUzJxxoPO3Dh0zBTHDeaF0WOK9eImqmf4UQD3SJDr v/onv/3rpOrEv5moexdr6thmsKIC6lVll0oEd13BoNx5Rh1OSBkXcEBVP32PFLAgNATx 23OA== X-Gm-Message-State: AGRZ1gLFNz7g9w6/6RL//olAn53aAyUMAFThfmcownDDFb0NNXlg/8BA WWW84Epm/K/cWFnhIlPl40U= X-Google-Smtp-Source: AJdET5eQKle/WQr4hL8q6hHAEli/3vTqPa9D9dfdmSrcIPgjQ1x5QBjr4kAfF7SBw1PGeRT81luGRg== X-Received: by 2002:a50:8e8c:: with SMTP id w12-v6mr20249740edw.280.1541456470468; Mon, 05 Nov 2018 14:21:10 -0800 (PST) Received: from localhost ([2001:470:1f15:3d8:7285:c2ff:fe37:5722]) by smtp.gmail.com with ESMTPSA id q10-v6sm7333260ejs.73.2018.11.05.14.21.09 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 05 Nov 2018 14:21:09 -0800 (PST) From: Rozhuk Ivan X-Google-Original-From: Rozhuk Ivan Date: Tue, 6 Nov 2018 01:21:07 +0300 To: Konstantin Belousov Cc: Robert , Mark Millard , FreeBSD , Mark Johnston Subject: Re: Sudden grow of memory in "Laundry" state Message-ID: <20181106012107.2898f093@gmail.com> In-Reply-To: <20181027043819.GX5335@kib.kiev.ua> References: <20180911150849.GD92634@raichu> <104be96a-c16b-7e7c-7d0d-00338ab5a106@gmail.com> <20180928152550.GA3609@raichu> <20181024211237.302b72d9@gmail.com> <981C887D-78EB-46D2-AEE5-877E269AF066@yahoo.com> <42f6544f-830c-18c5-e1a8-0acc4c3f09cc@gmail.com> <20181027043819.GX5335@kib.kiev.ua> X-Mailer: Claws Mail 3.16.0 (GTK+ 2.24.32; amd64-portbld-freebsd11.2) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 04B2C772FC X-Spamd-Result: default: False [-4.61 / 200.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_SHORT(-0.93)[-0.933,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[5.3.5.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-0.67)[ipnet: 2a00:1450::/32(-1.97), asn: 15169(-1.32), country: US(-0.06)]; FREEMAIL_CC(0.00)[gmail.com] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2018 22:21:13 -0000 On Sat, 27 Oct 2018 07:38:19 +0300 Konstantin Belousov wrote: > > 1. When process allocates memory (mmap), "Active" memory increases, > > "Free" memory decreases (that's expected). > No, mmap(2) call only causes some small amount of the kernel memory > allocation, to track the mmaped region. > > Pages are allocated on the first touch, lets ignore the prefaulting > mechanism which is not too relevant for the discussion. Mapped page > can be either active or inactive, this is determined by the usage > history. > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=195882 This is still a proublem. I have 12.0b3 in vbox. 16 Gb hdd, 4 Gb RAM. Free 6+Gb on hdd. Build app to write 6 Gb. run, it was killed Nov 5 21:05:09 firewall kernel: pid 96603 (testvm), uid 0, was killed: out of swap space Nov 5 21:05:15 firewall kernel: Nov 5 21:05:09 firewall kernel: pid 96603 (testvm), uid 0, was killed: out of swap space Mem: 50M Active, 3331M Inact, 192M Laundry, 362M Wired, 208M Buf, 212M Free Swap: 16G Total, 151M Used, 16G Free Filesystem Size Used Avail Capacity iused ifree %iused Mounted on /dev/gptid/6f42a7d9-45b9-11e8-8fd8-0800278fa2e3 17G 14G 2.4G 85% 768k 1.4M 35% / 1. Program does not do it job / killed, but system have more than enough swap. 2. mem does not free after all disk io done. It free only after file deleted or readed. Also there is more strange behaviour if swap file disabled. From owner-freebsd-hackers@freebsd.org Mon Nov 5 23:13:48 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3DA2E110ECC4 for ; Mon, 5 Nov 2018 23:13:48 +0000 (UTC) (envelope-from rebecca@bluestop.org) Received: from muon.bluestop.org (muon.bluestop.org [96.73.9.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 93F8E79C16 for ; Mon, 5 Nov 2018 23:13:47 +0000 (UTC) (envelope-from rebecca@bluestop.org) Received: from muon.bluestop.org (localhost [127.0.0.1]) by muon.bluestop.org (Postfix) with ESMTP id 2FF1A33C72; Mon, 5 Nov 2018 16:14:27 -0700 (MST) Received: from muon.bluestop.org ([127.0.0.1]) by muon.bluestop.org (muon.bluestop.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nbjUor52UB8O; Mon, 5 Nov 2018 16:14:27 -0700 (MST) Received: from photon.int.bluestop.org (gw.bluestop.org [96.73.9.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by muon.bluestop.org (Postfix) with ESMTPSA; Mon, 5 Nov 2018 16:14:27 -0700 (MST) From: Rebecca Cran To: freebsd-hackers@freebsd.org Cc: Zaphod Beeblebrox Subject: Re: Threadripper Thermal strangeness. Date: Mon, 05 Nov 2018 16:13:45 -0700 Message-ID: <1952737.4WAli8B44Z@photon.int.bluestop.org> In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Rspamd-Queue-Id: 93F8E79C16 X-Spamd-Result: default: False [-3.22 / 200.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[bluestop.org:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[bluestop.org,quarantine]; MX_GOOD(-0.01)[cached: mail.bluestop.org]; NEURAL_HAM_SHORT(-0.83)[-0.827,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; CTE_CASE(0.50)[]; ASN(0.00)[asn:7922, ipnet:96.64.0.0/11, country:US]; IP_SCORE(-0.01)[country: US(-0.06)]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.93)[-0.933,0]; R_DKIM_ALLOW(-0.20)[bluestop.org]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-0.94)[-0.939,0]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FREEMAIL_CC(0.00)[gmail.com] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2018 23:13:48 -0000 On Monday, 5 November 2018 14:23:22 MST Zaphod Beeblebrox wrote: > kldloading "amdtemp.ko" reports an idle temp of 55C or so and heavy > compiling (make world -j32) reports temperatures as high as 95C ... upto > and including some instability and/or crash or rebooting. I have a new Threadripper 2990WX in an ASUS motherboard - and mine just reports ridiculous values: % sysctl dev.amdtemp | grep sensor dev.amdtemp.3.core0.sensor0: 112.6C dev.amdtemp.3.sensor_offset: 0 dev.amdtemp.2.core0.sensor0: 113.5C dev.amdtemp.2.sensor_offset: 0 dev.amdtemp.1.core0.sensor0: 111.7C dev.amdtemp.1.sensor_offset: 0 dev.amdtemp.0.core0.sensor0: 114.5C dev.amdtemp.0.sensor_offset: 0 -- Rebecca From owner-freebsd-hackers@freebsd.org Mon Nov 5 23:25:04 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D6AC0110F009 for ; Mon, 5 Nov 2018 23:25:03 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-21.consmr.mail.gq1.yahoo.com (sonic305-21.consmr.mail.gq1.yahoo.com [98.137.64.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CCC337A2DE for ; Mon, 5 Nov 2018 23:25:02 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: OdahRkgVM1n6HodPqKIkrwgDe35JM3uHVDXB0W65Es7Cpdhbz2pWChIexu3feeH Pw9f9ocfehEnRvsSXnn2uHKarzFRr0Auh5I.q5XJOgy2TPMCO.FqAu3N8ippIalblvtq2RsVAKfJ iTjrIEo5mt1UNEdBQpClFe_LH0GXuhx1SgQsjbNMJQkILKRxUH_JV7RPf_KQ0EhB0P6OomlADeJM eXCCp9IHamLCtUHAmH8q0jSZmbu60ezzseMuWYdrcXxgUcsHrqoZC249RE2F6RoZSXC5BuV8kNf7 ZokUBcJ.rftV3fCt38kQ97UvtqgOegsbjI2O7XaSnAfWMoyQolVGAhPKCOpxFkerJj_.FmqMy9bM d3ryiNAl3UemZtrzrwxz1SG_WSJAZGNJGRJEmTfLypy9UG5J4qRip.e04DU5mCv1C1GB1KB8tzT6 SvjgUVrZTOI6wbx2YNTMn5sdFPxLnKMesGzb6.3eO.s4jNR9cTeuOHCa1JhUr8NLyg1rQq02otPP SAHMZ8EBMcTiMPYv5KS08SRhd28DLnDYZ0OMz1zQcFdzA5_L6TqQ_E.3bwM3VfoWzllzGBkh4WJV vWRcnML2yknuLfRJ.UbzhshaeotUPMJzDB8rQk3IxlxM0ldoIvabdvFclwPBuumTFYgiGhFqWkfJ _WqrVV8p2tOoTtqGJ2t8bv6f0YuGJXvS0WlbkDRBUAs23xQJVE8Q4_FnnvD1vomRz_pk.icXds_u zM8yzLWx0hp_Q.bv4s3ZCa9PFIVknpUIZUhD1SDFBip1Y8OoR6FNV11AKr4PzwCcDIOZFKBwXYWg _5SZ3vShKyZkePQnBxEMapyKwjBtYBcA9NBBVakIzEl32uQ67V.3hUfVyNycrvcyZAKz9YNZqUFv 34sa9ub_i7vm9vj7xE4R9pqc_gCaO8vHPf1TYr6Aju9TT.CQuvDCkS3wGha0sLh6IHuTPNP4L.jS eJDxXMoirMHQ6V7ay5YTYW4zq4kT8ythQtutjxHwL7agHGLxn5lqg7oZMEgP1tHZMOdFeDhH5s28 hnHcR1Lyo_KtL.p2YpkrT.kU7ANQ1zFmZg54VQevZ1XlH7Jq0NLhZr_gdGa4nYWbA3N8r5WnP Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Mon, 5 Nov 2018 23:24:56 +0000 Received: from c-76-115-7-162.hsd1.or.comcast.net (EHLO [192.168.1.25]) ([76.115.7.162]) by smtp432.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID fd2c6f61bddc51ab850dd24b2590a1ff; Mon, 05 Nov 2018 23:04:38 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.1 \(3445.101.1\)) Subject: Re: Sudden grow of memory in "Laundry" state From: Mark Millard In-Reply-To: <20181106012107.2898f093@gmail.com> Date: Mon, 5 Nov 2018 15:04:38 -0800 Cc: Konstantin Belousov , Robert , FreeBSD , Mark Johnston Content-Transfer-Encoding: quoted-printable Message-Id: References: <20180911150849.GD92634@raichu> <104be96a-c16b-7e7c-7d0d-00338ab5a106@gmail.com> <20180928152550.GA3609@raichu> <20181024211237.302b72d9@gmail.com> <981C887D-78EB-46D2-AEE5-877E269AF066@yahoo.com> <42f6544f-830c-18c5-e1a8-0acc4c3f09cc@gmail.com> <20181027043819.GX5335@kib.kiev.ua> <20181106012107.2898f093@gmail.com> To: Rozhuk Ivan X-Mailer: Apple Mail (2.3445.101.1) X-Rspamd-Queue-Id: CCC337A2DE X-Spamd-Result: default: False [-3.22 / 200.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.94)[-0.943,0]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.13)[ipnet: 98.137.64.0/21(0.39), asn: 36647(0.31), country: US(-0.06)]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.95)[-0.950,0]; R_DKIM_ALLOW(-0.20)[yahoo.com]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-0.94)[-0.940,0]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[84.64.137.98.list.dnswl.org : 127.0.5.0]; FREEMAIL_CC(0.00)[gmail.com] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2018 23:25:04 -0000 On 2018-Nov-5, at 14:21, Rozhuk Ivan wrote: > On Sat, 27 Oct 2018 07:38:19 +0300 > Konstantin Belousov wrote: >=20 >>> 1. When process allocates memory (mmap), "Active" memory increases,=20= >>> "Free" memory decreases (that's expected). =20 >> No, mmap(2) call only causes some small amount of the kernel memory >> allocation, to track the mmaped region. >>=20 >> Pages are allocated on the first touch, lets ignore the prefaulting >> mechanism which is not too relevant for the discussion. Mapped page >> can be either active or inactive, this is determined by the usage >> history. >>=20 >=20 >=20 > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D195882 > This is still a proublem. >=20 > I have 12.0b3 in vbox. > 16 Gb hdd, 4 Gb RAM. > Free 6+Gb on hdd. > Build app to write 6 Gb. >=20 > run, it was killed=20 > Nov 5 21:05:09 firewall kernel: pid 96603 (testvm), uid 0, was = killed: out of swap space > Nov 5 21:05:15 firewall kernel: Nov 5 21:05:09 firewall kernel: pid = 96603 (testvm), uid 0, was killed: out of swap space Unfortunately, the wording of this message is a misnomer for what drives the kills: it is actually driven by being unable to gain more free memory but FreeBSD will not swap-out processes that stay runnable (or are running), only ones that are waiting. Even a single process that stays runnable and keeps lots of RAM in the active category can lead to kills, even if swap is unused or little used. So the kill-behavior is very workload dependent. Having more swap need not avoid such kills because there may not be sufficient other swap-eligible processes that can be used to gain sufficient free RAM to avoid the kills. Real "out of swap" conditions tend to also have messages similar to: Aug 5 17:54:01 sentinel kernel: swap_pager_getswapspace(32): failed If you are not seeing such messages, then it is likely that the mount of swap space that is still free is not the actual issue driving the kill(s). Instead it is tied to a sustained period of low free RAM based on processes that stay running/runnable. (The system simply leaves the dirty pages in memory when a swap_pager_getswapspace failed message is produced. Of itself, it does not cause a kill.) > Mem: 50M Active, 3331M Inact, 192M Laundry, 362M Wired, 208M Buf, 212M = Free > Swap: 16G Total, 151M Used, 16G Free Without extra evidence, do not beleive the "out of swap space" part of "killed: out of swap space". But it turns out there is a tunable setting to control how many tries at freeing memory before kills happen: so, indirectly, how "long" before the kills will start under sustained low free RAM conditions. The default vm.pageout_oom_seq=3D12 can be increased to increase how long a low-free-RAM condition is tolerated. I assign vm.pageout_oom_seq in /etc/sysctl.conf --but that may not be the best for your context. vm.pageout_oom_seq=3D120 has proved useful. In some extreme situations (buildworld buildkernel in a low RAM, slow context, including long I/O latencies) vm.pageout_oom_seq=3D1024 or more has been used to avoid kills when there was plenty of swap space but low free memory (via processes that stay runnable and keep large amounts of RAM active relative to what the system has). So you may want to try assigning vm.pageout_oom_seq based on workload characteristics as part of your experiments. You may find that the kills can be avoided. > Filesystem Size Used = Avail Capacity iused ifree %iused Mounted on > /dev/gptid/6f42a7d9-45b9-11e8-8fd8-0800278fa2e3 17G 14G = 2.4G 85% 768k 1.4M 35% / >=20 > 1. Program does not do it job / killed, but system have more than = enough swap. > 2. mem does not free after all disk io done. > It free only after file deleted or readed. >=20 > Also there is more strange behaviour if swap file disabled. The book The Design And Implementation Of The FreeBSD Operating System Second Edition has material describing that runnable processes are not swapped out: the policy is deliberate. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-hackers@freebsd.org Mon Nov 5 23:45:19 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AB760110F801 for ; Mon, 5 Nov 2018 23:45:19 +0000 (UTC) (envelope-from matt.joras@gmail.com) Received: from mail-ed1-f47.google.com (mail-ed1-f47.google.com [209.85.208.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E89D07B3E6 for ; Mon, 5 Nov 2018 23:45:18 +0000 (UTC) (envelope-from matt.joras@gmail.com) Received: by mail-ed1-f47.google.com with SMTP id w19-v6so9026541eds.1 for ; Mon, 05 Nov 2018 15:45:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Qdn+tuZoB6nE0WWi0M0omPRZ9Q5g53wNe+L56G8g0Wc=; b=k+WyCsEreNG8j4W3aEGAniqTd9AeF+ziyCaEd0dfn0qsUlQ4tf5xttTJds1aJRtYsh ukrTP/JcZDtPJsAYTwZEqlHwi4yvcQ2XCKxtl4xySWnriZCKtidenxag6P5U27+En4LU 8bspOv9QOggV4zcvZAqXxRgHLjjzpn9RQzOfitEOAxPLf7cGiCnHx8/yCChpiuQvsIIi DSbf275VeKl0+SOVS8oxdPFB/JvytgSYnlePpouzfvzbgI88tA18liRu1mJeKbFmgDPe ZmCQ/2lCjYBMAwlDFQ2okyucxE6suvRyKEGojRUPMWovlS+mqaDTjX5TWJPlv/IZ5MhC xKmw== X-Gm-Message-State: AGRZ1gIQ9yO6pS+nCSA9MABc/WysL3aV3cA925YW3+zd1+RFpSjh/JIv O1XvaQf3g+RHBejkheL+u40UOvBc X-Google-Smtp-Source: AJdET5dEfGrKZV8lTVzOdjOoVpz3/HG8K+Bcbj1vyTsnWd2PMCyNnd4uW6U/1wKxCYQdsgjYq2rl3A== X-Received: by 2002:a50:90a7:: with SMTP id c36-v6mr20131540eda.31.1541461512066; Mon, 05 Nov 2018 15:45:12 -0800 (PST) Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com. [209.85.221.51]) by smtp.gmail.com with ESMTPSA id w6-v6sm602774edb.37.2018.11.05.15.45.11 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Nov 2018 15:45:11 -0800 (PST) Received: by mail-wr1-f51.google.com with SMTP id j26-v6so11474078wre.1 for ; Mon, 05 Nov 2018 15:45:11 -0800 (PST) X-Received: by 2002:adf:aa05:: with SMTP id p5-v6mr22763301wrd.56.1541461511086; Mon, 05 Nov 2018 15:45:11 -0800 (PST) MIME-Version: 1.0 References: <1952737.4WAli8B44Z@photon.int.bluestop.org> In-Reply-To: <1952737.4WAli8B44Z@photon.int.bluestop.org> From: Matt Joras Date: Mon, 5 Nov 2018 15:44:59 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Threadripper Thermal strangeness. To: rebecca@bluestop.org Cc: freebsd-hackers@freebsd.org, zbeeble@gmail.com Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: E89D07B3E6 X-Spamd-Result: default: False [-4.00 / 200.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; IP_SCORE(-1.01)[ipnet: 209.85.128.0/17(-3.64), asn: 15169(-1.35), country: US(-0.07)]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.97)[-0.974,0]; RCVD_IN_DNSWL_NONE(0.00)[47.208.85.209.list.dnswl.org : 127.0.5.0]; RCVD_TLS_LAST(0.00)[]; FORGED_SENDER(0.30)[mjoras@freebsd.org,mattjoras@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[47.208.85.209.rep.mailspike.net : 127.0.0.17]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; TAGGED_FROM(0.00)[]; FROM_NEQ_ENVFROM(0.00)[mjoras@freebsd.org,mattjoras@gmail.com] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2018 23:45:19 -0000 You have to use the sensor_offset to get the actual value at the die. I believe for all Threadripper chips the offset is -27C. Matt On Mon, Nov 5, 2018 at 3:14 PM Rebecca Cran via freebsd-hackers wrote: > > On Monday, 5 November 2018 14:23:22 MST Zaphod Beeblebrox wrote: > > > kldloading "amdtemp.ko" reports an idle temp of 55C or so and heavy > > compiling (make world -j32) reports temperatures as high as 95C ... upto > > and including some instability and/or crash or rebooting. > > I have a new Threadripper 2990WX in an ASUS motherboard - and mine just > reports ridiculous values: > > % sysctl dev.amdtemp | grep sensor > dev.amdtemp.3.core0.sensor0: 112.6C > dev.amdtemp.3.sensor_offset: 0 > dev.amdtemp.2.core0.sensor0: 113.5C > dev.amdtemp.2.sensor_offset: 0 > dev.amdtemp.1.core0.sensor0: 111.7C > dev.amdtemp.1.sensor_offset: 0 > dev.amdtemp.0.core0.sensor0: 114.5C > dev.amdtemp.0.sensor_offset: 0 > > -- > Rebecca > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Mon Nov 5 23:52:20 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C3132110FC83 for ; Mon, 5 Nov 2018 23:52:20 +0000 (UTC) (envelope-from rebecca@bluestop.org) Received: from muon.bluestop.org (muon.bluestop.org [96.73.9.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2FA477BBFB; Mon, 5 Nov 2018 23:52:20 +0000 (UTC) (envelope-from rebecca@bluestop.org) Received: from muon.bluestop.org (localhost [127.0.0.1]) by muon.bluestop.org (Postfix) with ESMTP id EECC133D02; Mon, 5 Nov 2018 16:52:59 -0700 (MST) Received: from muon.bluestop.org ([127.0.0.1]) by muon.bluestop.org (muon.bluestop.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zeKH3Z3xxdkX; Mon, 5 Nov 2018 16:52:59 -0700 (MST) Received: from photon.int.bluestop.org (gw.bluestop.org [96.73.9.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by muon.bluestop.org (Postfix) with ESMTPSA; Mon, 5 Nov 2018 16:52:59 -0700 (MST) From: Rebecca Cran To: Matt Joras Cc: freebsd-hackers@freebsd.org, zbeeble@gmail.com Subject: Re: Threadripper Thermal strangeness. Date: Mon, 05 Nov 2018 16:52:18 -0700 Message-ID: <3138150.gzUHUCRiql@photon.int.bluestop.org> In-Reply-To: References: <1952737.4WAli8B44Z@photon.int.bluestop.org> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Rspamd-Queue-Id: 2FA477BBFB X-Spamd-Result: default: False [-3.40 / 200.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[bluestop.org]; NEURAL_HAM_MEDIUM(-0.98)[-0.980,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-0.98)[-0.983,0]; MIME_GOOD(-0.10)[text/plain]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bluestop.org:+]; DMARC_POLICY_ALLOW(-0.50)[bluestop.org,quarantine]; MX_GOOD(-0.01)[cached: mail.bluestop.org]; NEURAL_HAM_SHORT(-0.92)[-0.916,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; CTE_CASE(0.50)[]; ASN(0.00)[asn:7922, ipnet:96.64.0.0/11, country:US]; IP_SCORE(-0.01)[country: US(-0.07)] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2018 23:52:21 -0000 On Monday, 5 November 2018 16:44:59 MST Matt Joras wrote: > You have to use the sensor_offset to get the actual value at the die. > I believe for all Threadripper chips the offset is -27C. Oh, thanks! I'm guessing there's something different about mine though, because 114 - 27 is 87, and under Windows is reports 67C when it's completely maxed out. Maybe with its 4 units (32 cores) the offset is -54C (2*27) instead? -- Rebecca > On Mon, Nov 5, 2018 at 3:14 PM Rebecca Cran via freebsd-hackers > > wrote: > > On Monday, 5 November 2018 14:23:22 MST Zaphod Beeblebrox wrote: > > > kldloading "amdtemp.ko" reports an idle temp of 55C or so and heavy > > > compiling (make world -j32) reports temperatures as high as 95C ... upto > > > and including some instability and/or crash or rebooting. > > > > I have a new Threadripper 2990WX in an ASUS motherboard - and mine just > > reports ridiculous values: > > > > % sysctl dev.amdtemp | grep sensor > > dev.amdtemp.3.core0.sensor0: 112.6C > > dev.amdtemp.3.sensor_offset: 0 > > dev.amdtemp.2.core0.sensor0: 113.5C > > dev.amdtemp.2.sensor_offset: 0 > > dev.amdtemp.1.core0.sensor0: 111.7C > > dev.amdtemp.1.sensor_offset: 0 > > dev.amdtemp.0.core0.sensor0: 114.5C > > dev.amdtemp.0.sensor_offset: 0 > > > > -- > > Rebecca > > > > > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Tue Nov 6 00:53:53 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CFDD61121950 for ; Tue, 6 Nov 2018 00:53:53 +0000 (UTC) (envelope-from rwmaillists@googlemail.com) Received: from mail-wr1-x442.google.com (mail-wr1-x442.google.com [IPv6:2a00:1450:4864:20::442]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0BDDE7E4BA for ; Tue, 6 Nov 2018 00:53:53 +0000 (UTC) (envelope-from rwmaillists@googlemail.com) Received: by mail-wr1-x442.google.com with SMTP id 74-v6so11544081wrb.13 for ; Mon, 05 Nov 2018 16:53:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9l9kZyxN2rtJ9GufG3cV5KAsh3i+Kdt1Mw+ttQ6nZLg=; b=TS9o7xyDozUqYTDO/IcFbpSWgBPCslSXTR/9trXhoytX71Y2X2CH418nLnfycZSPKj gdG2S5BHmBsWG3/x7z4M325/Qy5/RP23EBME6IfeUdBGurXQrTwu0mo2i61bHiuPvcUV waGQfcS4uXDIJriGFTQ1v1zXjTAjcyGa0NUNu+9kv/QkrSe4CVhL0nukWy8IjjqFnykW 6rxUUnDNGVisMPBfj/+PbRw3+uJwLy/NwulVdBmWwzWVJFQAvI8KY3TClJ1jQMD7PWDs VlTdj8kgtpYGhWAzO/Y99/n6yc8KPLQvG2vhIycLOajGFZhBiRs3pMql9XUsfe0PwBOi ESDw== X-Gm-Message-State: AGRZ1gKet+AId6SQKh33OfA3ZZev3214eDFTNTijf65Nzs72M1J78Mdq ezK9/A1bK4Xo48RjpcxfPnYhRmh1rA4= X-Google-Smtp-Source: AJdET5ckmHOrqwIKh/QzDsDwNDYI8wuha1s0HFvwl6nkoTI8Vc2KMEPUpX96RGo8ClJ2kL+J7kVVEQ== X-Received: by 2002:a5d:4cc6:: with SMTP id c6-v6mr22085269wrt.75.1541465631232; Mon, 05 Nov 2018 16:53:51 -0800 (PST) Received: from gumby.homeunix.com ([2.121.229.57]) by smtp.gmail.com with ESMTPSA id z16-v6sm258698wmc.0.2018.11.05.16.53.49 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 05 Nov 2018 16:53:50 -0800 (PST) Date: Tue, 6 Nov 2018 00:53:47 +0000 From: RW To: freebsd-hackers@freebsd.org Subject: Re: Sudden grow of memory in "Laundry" state Message-ID: <20181106005347.2f035e34@gumby.homeunix.com> In-Reply-To: References: <20180911150849.GD92634@raichu> <104be96a-c16b-7e7c-7d0d-00338ab5a106@gmail.com> <20180928152550.GA3609@raichu> <20181024211237.302b72d9@gmail.com> <981C887D-78EB-46D2-AEE5-877E269AF066@yahoo.com> <42f6544f-830c-18c5-e1a8-0acc4c3f09cc@gmail.com> <20181027043819.GX5335@kib.kiev.ua> <20181106012107.2898f093@gmail.com> X-Mailer: Claws Mail 3.16.0 (GTK+ 2.24.32; amd64-portbld-freebsd11.1) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 0BDDE7E4BA X-Spamd-Result: default: False [-4.66 / 200.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[googlemail.com]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[googlemail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[googlemail.com:+]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCVD_IN_DNSWL_NONE(0.00)[2.4.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.96)[-0.963,0]; DMARC_POLICY_ALLOW(-0.50)[googlemail.com,quarantine]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[googlemail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; IP_SCORE(-0.69)[ipnet: 2a00:1450::/32(-2.00), asn: 15169(-1.37), country: US(-0.07)]; RECEIVED_SPAMHAUS_PBL(0.00)[57.229.121.2.zen.spamhaus.org : 127.0.0.10] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2018 00:53:54 -0000 On Mon, 5 Nov 2018 15:04:38 -0800 Mark Millard via freebsd-hackers wrote: > On 2018-Nov-5, at 14:21, Rozhuk Ivan wrote: > > > run, it was killed > > Nov 5 21:05:09 firewall kernel: pid 96603 (testvm), uid 0, was > > killed: out of swap space Nov 5 21:05:15 firewall kernel: Nov 5 > > 21:05:09 firewall kernel: pid 96603 (testvm), uid 0, was killed: > > out of swap space > > Unfortunately, the wording of this message is a misnomer for what > drives the kills: it is actually driven by being unable to gain more > free memory but FreeBSD will not swap-out processes that stay runnable > (or are running), only ones that are waiting. When you say it wont swap-out processes, do you mean that literally, or do you mean it wont page-out from runable processes? Swapping processes shouldn't be an essential part of recovering memory, Linux doesn't even support it. From owner-freebsd-hackers@freebsd.org Tue Nov 6 01:30:49 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 21A5E11253C5 for ; Tue, 6 Nov 2018 01:30:49 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-22.consmr.mail.gq1.yahoo.com (sonic310-22.consmr.mail.gq1.yahoo.com [98.137.69.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CCE2D81936 for ; Tue, 6 Nov 2018 01:30:47 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: vuElhIIVM1lBePjeJgKWyQQhaksoKTl.dR61FROAorwq3fJdiVv7pw5vZuJhD5z bP0j3WqqeBGdhQShpipRkxgXM2ZPXTMobNEfbRmGbdv.eJPPcG6hNxMg79lK0QXq_jWCF0c1c8LA cg5KEN1r7VFfaoBebCPyfWW6e.3cTMTvmogpIXRi1JCM_EIcviRvkmXUbpNkos8S9EJFd68jSlOT r6YXHazg8WFl4bakNIRAOSk7iZaiLHUzMAeBfpVWbusnxoVF9FCvbOnnq43EliNpi88Uml01MnWd zNBTF7D8qk2atWQGRIg20L_yQ50ZiIkb1hRIYO800Xi.EsjCt_3p6imQ2x7CzvmI9a.CJnVD_dX3 EP.FgVk1JWWA0XG_rjgM8ETa5T5i0HT1KWVpYuK.GDiYEcsTIkIUl9UDjuo4fDKhWGjSDK4e3GLi cSwit9O5xAVnF3gRbDP_fN.4G0dpSz1mcJgPV1zTXX_Z4niEnAWP34_.nxR.5Wyy1YoD588b1hrh mbzUo6vKoMS02T6WrtFK.DzGd9MFOL4e_bVSOThGSHkMDwCDPiIxq3fz04c039t6lxff.wvKtpfz LQ.8CduiciY6EWUERdOL4t9yKs3XSYQlx7CbQZiioOMfpXf2Dd6NJzZEYnKJuMLvmcs42MzIH49x uIK5yNu8M2_lVEmyKkGXaQJSiciys_i2tjtKu338Gst95cT0v62Q6J6FNygVh8TMtUUthCCrvZD1 RAgFgYPJW0i80pXVkoyjpsrAdd_MZ3roPsVDPBO5xbb3fg4RJlEIez8bLwwSRYac_p6ZXMh4uF9J WrlymOuyOEXCKbMT_EcSL0Pk_82ZcinGe3nFfrCt5DkAbaEkyUPk8hffX1VseWondDLHBsz2DjhN 1sDSQcCr178NzfohlbZQ2Aih18O8iC1D8H1FvolFEeILePaiWBAT1pSo46TsBjdQAoLQ2_Zcp8qg sVY4LwaDEFCEU5.uotf.xLgwBCtuiKUeNI9moCzxKjc_A0nRiYMecYtKihADSw4BKnuykYtCwV89 NSGem2ERlIRJBLjDPiFk6xra_Rdx.cCsV7MZULpLQLpVy_Vo- Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Tue, 6 Nov 2018 01:30:46 +0000 Received: from c-76-115-7-162.hsd1.or.comcast.net (EHLO [192.168.1.25]) ([76.115.7.162]) by smtp404.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 2da3d9b43aba330a088eca332cff48be; Tue, 06 Nov 2018 01:30:43 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.1 \(3445.101.1\)) Subject: Re: Sudden grow of memory in "Laundry" state From: Mark Millard In-Reply-To: <20181106005347.2f035e34@gumby.homeunix.com> Date: Mon, 5 Nov 2018 17:30:42 -0800 Cc: FreeBSD , Rozhuk Ivan Content-Transfer-Encoding: quoted-printable Message-Id: <3F9246AC-0CBA-4C73-A39E-A1F6149BA079@yahoo.com> References: <20180911150849.GD92634@raichu> <104be96a-c16b-7e7c-7d0d-00338ab5a106@gmail.com> <20180928152550.GA3609@raichu> <20181024211237.302b72d9@gmail.com> <981C887D-78EB-46D2-AEE5-877E269AF066@yahoo.com> <42f6544f-830c-18c5-e1a8-0acc4c3f09cc@gmail.com> <20181027043819.GX5335@kib.kiev.ua> <20181106012107.2898f093@gmail.com> <20181106005347.2f035e34@gumby.homeunix.com> To: RW X-Mailer: Apple Mail (2.3445.101.1) X-Rspamd-Queue-Id: CCE2D81936 X-Spamd-Result: default: False [-3.19 / 200.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.84)[-0.842,0]; FREEMAIL_TO(0.00)[googlemail.com]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.08)[ipnet: 98.137.64.0/21(0.27), asn: 36647(0.21), country: US(-0.07)]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.963,0]; R_DKIM_ALLOW(-0.20)[yahoo.com]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-0.95)[-0.954,0]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[148.69.137.98.list.dnswl.org : 127.0.5.0] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2018 01:30:49 -0000 On 2018-Nov-5, at 16:53, RW via freebsd-hackers wrote: > On Mon, 5 Nov 2018 15:04:38 -0800 > Mark Millard via freebsd-hackers wrote: >=20 >> On 2018-Nov-5, at 14:21, Rozhuk Ivan wrote: >>=20 >>> run, it was killed=20 >>> Nov 5 21:05:09 firewall kernel: pid 96603 (testvm), uid 0, was >>> killed: out of swap space Nov 5 21:05:15 firewall kernel: Nov 5 >>> 21:05:09 firewall kernel: pid 96603 (testvm), uid 0, was killed: >>> out of swap space =20 >>=20 >> Unfortunately, the wording of this message is a misnomer for what >> drives the kills: it is actually driven by being unable to gain more >> free memory but FreeBSD will not swap-out processes that stay = runnable >> (or are running), only ones that are waiting. >=20 > When you say it wont swap-out processes, do you mean that literally, = or > do you mean it wont page-out from runable processes? Swapping = processes > shouldn't be an essential part of recovering memory, Linux doesn't = even > support it. Paging is a different issue as I understand. Without paging, no process could have a memory area bigger than RAM. But, going in a different direction . . . Turns out the example program in bugzilla 195882 is interesting independent of any worries about kills or swap space use: I ran an a.out built from the source but with 16 instead of 4 as the size controlling constant, in order use 16 GiByte for an 8 GiByte aarch64 system. Before the a.out run (after a fresh boot, form top): Mem: 13M Active, 1240K Inact, 108M Wired, 28M Buf, 7757M Free Swap: 28G Total, 28G Free After the a.out had finished, from a separate top run: Mem: 2197M Active, 4937M Inact, 255M Wired, 186M Buf, 629M Free Swap: 28G Total, 11M Used, 28G Free No actual process (or group of them) was running using a large amount of RAM at that point. Active and Inact are rather large. With the system left idle, 15 minutes later or more, the Active and Inact had not changed significantly, nor had Free. The figures do seem odd. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-hackers@freebsd.org Tue Nov 6 01:39:41 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 53EA81126438 for ; Tue, 6 Nov 2018 01:39:41 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com [IPv6:2a00:1450:4864:20::130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 81CC982C84; Tue, 6 Nov 2018 01:39:40 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: by mail-lf1-x130.google.com with SMTP id u18so7615102lff.10; Mon, 05 Nov 2018 17:39:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:date:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qn/9YAYloMyUq5QKlRwkFAQ+w2SrbyWaL1Nnt/wbOEk=; b=PbRg8vu5+yiiNbxtamvneoSNQi9pR976PJ6YSUDeBWLwbDPOckdRK3sXXN1PPVOMXs +kmHZnKV/ev/yY3upTGJ4fPrVaCKox7837ToaJcVwvjAixhr8c9zIYqanNvK2hPA4oPi imlQsruDzQvv7Mw//tPcNb5v7jbNATAk7DmbJQ+yM2HbgS6t1Yqm9Ov783rwO4Zuu4j1 QrHDkkEcc2a7ndRjVjGf2aaQMEwxiWUf8JeDdemr8Fl5HbJtus/ox+ejX7Y65zos+qQq 6v4KKlBAArfXwmGqeqSC9NJ8fg8Xm9lm53/NjuHgmz4sozqc96M3ZSEOuQrAZn2/3XFI V+lA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:date:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qn/9YAYloMyUq5QKlRwkFAQ+w2SrbyWaL1Nnt/wbOEk=; b=dHBXxG+QqA1c6AqS502C/zgn/s+NCb62o+o/r7bP7wWudMNgS3qGwPTOj/7oSfZIxk WHAPvftp70pv6Qd/4qinqnMS0DjM2WLQleEi6CokHBThM+ZNipLQP0zpTq1zLSjVPZuE uOJvSoD695UKTIxsTq54AZPBTtmwbs/Dxb2KXCgl2chcPmmXFK1x8T7bA3vysJHXnEIo u/8OtaoW1Ykl5p89xR6P3GSyquQjtRPt/2pPBglInluQhCq5dqwnweEwS0c6ERod9Apc FjbFdg2zqmsvQzUGy1OI6nv0XwV6ib6BmBUb+itYCTLNQfmZq4mKDZLU+eg2mbnDi83U 4sow== X-Gm-Message-State: AGRZ1gJo7i+ClpU0kn6JkO/KntH1iNaetEWvxrJ0G+UKPvz5BlIRrdbX b6PJIP+dtvbpIN6iG9Yj61BZV3fqZgY= X-Google-Smtp-Source: AJdET5czu/vVQw+NqTSCyHeRHfg+4uW+TWqkwFQ6jznsrasrPGmu91iClC75Jt70UdlHmHmPTeOdaQ== X-Received: by 2002:a19:26ce:: with SMTP id m197mr11200889lfm.23.1541468378825; Mon, 05 Nov 2018 17:39:38 -0800 (PST) Received: from localhost ([2001:470:1f15:3d8:7285:c2ff:fe37:5722]) by smtp.gmail.com with ESMTPSA id j20sm4392636lfg.69.2018.11.05.17.39.37 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 05 Nov 2018 17:39:37 -0800 (PST) From: Rozhuk Ivan X-Google-Original-From: Rozhuk Ivan Date: Tue, 6 Nov 2018 04:39:35 +0300 To: Mark Millard Cc: Konstantin Belousov , Robert , FreeBSD , Mark Johnston Subject: Re: Sudden grow of memory in "Laundry" state Message-ID: <20181106043935.6398fbb5@gmail.com> In-Reply-To: References: <20180911150849.GD92634@raichu> <104be96a-c16b-7e7c-7d0d-00338ab5a106@gmail.com> <20180928152550.GA3609@raichu> <20181024211237.302b72d9@gmail.com> <981C887D-78EB-46D2-AEE5-877E269AF066@yahoo.com> <42f6544f-830c-18c5-e1a8-0acc4c3f09cc@gmail.com> <20181027043819.GX5335@kib.kiev.ua> <20181106012107.2898f093@gmail.com> X-Mailer: Claws Mail 3.16.0 (GTK+ 2.24.32; amd64-portbld-freebsd11.2) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 81CC982C84 X-Spamd-Result: default: False [-4.67 / 200.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; FREEMAIL_TO(0.00)[yahoo.com]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-0.69)[ipnet: 2a00:1450::/32(-2.00), asn: 15169(-1.38), country: US(-0.07)]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_SHORT(-0.97)[-0.973,0]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[0.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; FREEMAIL_CC(0.00)[gmail.com] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2018 01:39:41 -0000 On Mon, 5 Nov 2018 15:04:38 -0800 Mark Millard wrote: > > Mem: 50M Active, 3331M Inact, 192M Laundry, 362M Wired, 208M Buf, > > 212M Free Swap: 16G Total, 151M Used, 16G Free > > Without extra evidence, do not beleive the "out of swap space" > part of "killed: out of swap space". > > But it turns out there is a tunable setting to control how many > tries at freeing memory before kills happen: so, indirectly, > how "long" before the kills will start under sustained low > free RAM conditions. > I undestand that real kill reason is not enough free RAM, not swap. > The default vm.pageout_oom_seq=12 can be increased > to increase how long a low-free-RAM condition is tolerated. > I assign vm.pageout_oom_seq in /etc/sysctl.conf --but that may > not be the best for your context. I do not undestand why system keep that crap in my RAM (making big disk cache) and does not return it back, and prefer to move some other process/mem to swap. I want limit disk cache size / durty pages via sysctl, or let it scale to all free mem but if some app need memory - it can take from that cache. For now I see that system does not like to give to apps that mem. FreeBSD has worst mem system design for user purposes, linux and windiws mush better. Only FreeBSD may stick because all 32GB mem used by something (not user processes) and try to start swapping. I never use swap with windows and linux, I always know who eat my mem. I try many sysctl tunings but without success - disk cache/durty pages grow and system freeze (if swap is off). I see some where in past some patch to VM system that more active free mem, but lost link and not try it. :( Laundary and some other improvenets from FreeBSD 10 to 11.2 is big step, but still have no confort in use. From owner-freebsd-hackers@freebsd.org Tue Nov 6 02:05:00 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F25F01128F15 for ; Tue, 6 Nov 2018 02:04:59 +0000 (UTC) (envelope-from rwmaillists@googlemail.com) Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 192F084CA9 for ; Tue, 6 Nov 2018 02:04:59 +0000 (UTC) (envelope-from rwmaillists@googlemail.com) Received: by mail-wr1-x42c.google.com with SMTP id y15-v6so11663941wru.9 for ; Mon, 05 Nov 2018 18:04:59 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2VwX7TAoUYMVeaYl+8btEEeLy1FYSPsxpSfgcnh5fwY=; b=jcmE8XP5cs4iKwJ5LTZ2RsDvTGcsuzJHNNZUaQDYS4zkml46xuBFLU2xTnld1uzxW5 hMIZlamvx4RcdRVVggAFsnxNJLo/Maxp/UOuUgZ0Y5T3dGM7dg1ZP59MtA8q9ynjUUfN g6pE+lMhCtZbNAuLTQ9GEx893RfdtGrjQmecT13h+G4BqLZyOeSw2QKEhBPvwoDMGkxq ZX/VT2HG05sVqC4idNEdGhexVkF5KtQzu/4Lo+xcaYaJzK9OgMmXn3Hrukszp5M3X/GL P++WeXYoC11P3jyJjsZ7CvXlXSUB820f9iJWQuCYffS/nMkJdoXXptZj7WWjx3O84JB6 hpFg== X-Gm-Message-State: AGRZ1gKvHnBuQQuplG+VoGdSVWKvhpLtvkHw+U8w16DeUl4MKIIh1LpL xz53CeOD7MlP2B3nf/adJBnl4IeSEN4= X-Google-Smtp-Source: AJdET5cQ2WJjEIj+GVgI0ofm2tYvoLH8anKn7raqymaksuZLE/rmj4qtBhyIQXnodlTuFxqdMps+QQ== X-Received: by 2002:a5d:46ce:: with SMTP id g14-v6mr21166373wrs.263.1541469897690; Mon, 05 Nov 2018 18:04:57 -0800 (PST) Received: from gumby.homeunix.com ([2.121.229.57]) by smtp.gmail.com with ESMTPSA id l9-v6sm328308wme.36.2018.11.05.18.04.56 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 05 Nov 2018 18:04:57 -0800 (PST) Date: Tue, 6 Nov 2018 02:04:54 +0000 From: RW To: freebsd-hackers@freebsd.org Subject: Re: Sudden grow of memory in "Laundry" state Message-ID: <20181106020454.1200f69c@gumby.homeunix.com> In-Reply-To: <3F9246AC-0CBA-4C73-A39E-A1F6149BA079@yahoo.com> References: <20180911150849.GD92634@raichu> <104be96a-c16b-7e7c-7d0d-00338ab5a106@gmail.com> <20180928152550.GA3609@raichu> <20181024211237.302b72d9@gmail.com> <981C887D-78EB-46D2-AEE5-877E269AF066@yahoo.com> <42f6544f-830c-18c5-e1a8-0acc4c3f09cc@gmail.com> <20181027043819.GX5335@kib.kiev.ua> <20181106012107.2898f093@gmail.com> <20181106005347.2f035e34@gumby.homeunix.com> <3F9246AC-0CBA-4C73-A39E-A1F6149BA079@yahoo.com> X-Mailer: Claws Mail 3.16.0 (GTK+ 2.24.32; amd64-portbld-freebsd11.1) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 192F084CA9 X-Spamd-Result: default: False [-4.61 / 200.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[googlemail.com]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[googlemail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-0.997,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[googlemail.com:+]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DMARC_POLICY_ALLOW(-0.50)[googlemail.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[c.2.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.92)[-0.919,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[googlemail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; IP_SCORE(-0.68)[ipnet: 2a00:1450::/32(-1.97), asn: 15169(-1.36), country: US(-0.07)]; RECEIVED_SPAMHAUS_PBL(0.00)[57.229.121.2.zen.spamhaus.org : 127.0.0.10] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2018 02:05:00 -0000 On Mon, 5 Nov 2018 17:30:42 -0800 Mark Millard wrote: > On 2018-Nov-5, at 16:53, RW via freebsd-hackers freebsd.org> wrote: > > >> Unfortunately, the wording of this message is a misnomer for what > >> drives the kills: it is actually driven by being unable to gain > >> more free memory but FreeBSD will not swap-out processes that stay > >> runnable (or are running), only ones that are waiting. > > > > When you say it wont swap-out processes, do you mean that > > literally, or do you mean it wont page-out from runable processes? > > Swapping processes shouldn't be an essential part of recovering > > memory, Linux doesn't even support it. > > Paging is a different issue as I understand. Without paging, > no process could have a memory area bigger than RAM. Modern VM manages memory at the page level, and paging replaced true process swapping a long time ago. In FreeBSD a form of swapping was recreated on top of paging. Under low memory conditions the recovery of memory from whole processes is accelerated allowing more memory into the running/runable processes. This is a relatively minor tweak to ordinary paging - AFAIK it shouldn't prevent normal memory management at the page level. From owner-freebsd-hackers@freebsd.org Tue Nov 6 04:01:44 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 36BF5112E07A for ; Tue, 6 Nov 2018 04:01:44 +0000 (UTC) (envelope-from steevanxperia@gmail.com) Received: from mail-ot1-x32e.google.com (mail-ot1-x32e.google.com [IPv6:2607:f8b0:4864:20::32e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8B4AE8A3EE for ; Tue, 6 Nov 2018 04:01:43 +0000 (UTC) (envelope-from steevanxperia@gmail.com) Received: by mail-ot1-x32e.google.com with SMTP id g10so10209331otl.12 for ; Mon, 05 Nov 2018 20:01:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=iqMwWh+NMIZrllo0q/OgO/NI7g7PN8gikl8T+VHwDrE=; b=YFETp8113C5y4av3COjOvFik+1AqAc++g0D9HorjVqy5STm3u8J1IqahouXoHYG2lG 6TUstRFazkGcXZWcqRRdfJ+z5fMoo7anI1CqawuixPjx1rL8x4sUlWGGTBj/M2f8Nwds QdKK77aFfry/2hvalq74i3iH57jGZzy7eDD1m8q58F76CTRUnwnk24RHpZHUjj4Cd0Os 3dps2Bf+c144JlpxeqjghgnkDNhPSF3hih3oH+V754v4LKUOWvEFUVQcHU7o6vf7YAOp vfZ6alDDQZHCUVVP4JMp9sjj4LCm5/S8iQ6aHquFba0e56Og55cZWr76nzIWWb2HApLb dmTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=iqMwWh+NMIZrllo0q/OgO/NI7g7PN8gikl8T+VHwDrE=; b=ntjul51HpBMgf9TscJ/S7ZOM91knwnKYcE5zHbwawVTSD5C2gVdkWAk0tzmmZTIFSz DuODC1iO4Htn0G+vVw9rjawpkplUuJp+ET7CnrR1cb0J32GUtJmadrzokbZrZXACLGTT vHeBosB1J5QQ8s1UEfHuUSbNvm+5rWvFZsVlC36ZNsoLFuiv1BZlBEmWp1CgQWtvTPkQ MgeLTs3/u7Yj/kSBSDMPsjVBpJvJ28BLLuwis7Y1obLztYv1OOP10L0Slt1KmEMhOcfO lIlCMAsa1NmOZftpGVeYVuVFjJJLHvzXd2FE77MaWDkfxU2WFXsqjUl9tfcdqptZ3DLJ hBsw== X-Gm-Message-State: AGRZ1gLfyTX7FzsHyedvTkcllTCB/d/SgYZAYsyBP2IbpuLddIjjgo18 2C6D7KsofA+aK0nI9BwtmeYx+qftbj/IbyFzNUbAXkIi X-Google-Smtp-Source: AJdET5edmWcFrAiksHCV/N1eMXzjAJLSuT9o3sfIDQK3PUsE7j7SRLlhRFQEyYeLZUGknGMWzogw4XMkxsd/LivYXgk= X-Received: by 2002:a9d:7692:: with SMTP id j18mr15111030otl.217.1541476902426; Mon, 05 Nov 2018 20:01:42 -0800 (PST) MIME-Version: 1.0 From: Steevan Rodrigues Date: Tue, 6 Nov 2018 09:31:30 +0530 Message-ID: Subject: FreeBSD 11.x thread creation time is 9000+ microseconds on Intel Xeon Gold series CPU To: freebsd-hackers@freebsd.org X-Rspamd-Queue-Id: 8B4AE8A3EE X-Spamd-Result: default: False [-0.81 / 200.00]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MIME_BASE64_TEXT(0.10)[]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-0.46)[ipnet: 2607:f8b0::/32(-0.81), asn: 15169(-1.40), country: US(-0.07)]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MIME_UNKNOWN(0.10)[text/x-csrc]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.71)[-0.714,0]; R_DKIM_ALLOW(-0.20)[gmail.com]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.65)[-0.652,0]; MIME_GOOD(-0.10)[multipart/mixed,multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; MIME_BAD_ATTACHMENT(1.60)[c]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_SHORT(0.23)[0.225,0]; RCVD_IN_DNSWL_NONE(0.00)[e.2.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Server: mx1.freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2018 04:01:44 -0000 Hi , I am seeing a FreeBSD 11.x OS poor performance issue . CPU is Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz ( dual socket CPU with 10 cores per socket ) I have attached a simple program which creates thread and computes time taken to create this thread. On this CPU with FreeBSD 11.x OS it takes 9000 to 15000 micro seconds ( us) to create just one thread. On other platforms this thread creation time is usually 20 to 30 us only. Any idea why it takes so much more time with FreeBSD 11.x ? Is there any processor specific tuning that needs to be done ? Thanks, Steevan From owner-freebsd-hackers@freebsd.org Tue Nov 6 06:46:45 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0DBC21104F49 for ; Tue, 6 Nov 2018 06:46:45 +0000 (UTC) (envelope-from vhaisman@gmail.com) Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5E9FC8F234 for ; Tue, 6 Nov 2018 06:46:44 +0000 (UTC) (envelope-from vhaisman@gmail.com) Received: by mail-wr1-x429.google.com with SMTP id x12-v6so12113233wrw.8 for ; Mon, 05 Nov 2018 22:46:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:openpgp:autocrypt:message-id:date :user-agent:mime-version:in-reply-to; bh=K9vQMd2pg3ymto2BELLwVQPJpwOlH8XkxLyCdENn/8Y=; b=vRsWSAiHbtB2YSUVEGhNoRPyihd6znb4yDuyxhBFza0LdAe7ro1O9+1udfEG4S7KSQ OZcDBIspizIz1d1RRbgdEteG4/MNhwaaTC5AMVCQ+fENQUdj04HS1jUkJo2JFO0pB+KD 87N4dDGWI3MpWJ2h85PVCYyteHKLSCtEO2EIpqKkZ+zjsieFUqroYE2SJbbvO0PWobGe IgcvYT8KaGVrw+dUfQ5XK3rmnNAB+L9HvTl0g6HR/NrjWSFoEvR+294gRamoTKguvnxR jI5EOHl1SyNQkuIa/mt0j4ElbR/d+/eXSgu/6BndtUbhdaIOyqVo62hDLDW50YOc3k1y iDVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:openpgp:autocrypt :message-id:date:user-agent:mime-version:in-reply-to; bh=K9vQMd2pg3ymto2BELLwVQPJpwOlH8XkxLyCdENn/8Y=; b=W1Cc+X4IkOwFvHbcMJz/Iz+qttTyJ8PMm/IwJhzFdDSwntDvf9GB/JM0MRiVEuhNF0 DNOuUvAIlgEv2YgZIGoh3tjctQaWZfwuRK3+OfQE4Aq0X7SIn4D0v93nXvDlGCh5Qrot XQtr8jY/0b7vKlieCJZqkfVNOTFgJUOu6Han2JF0AExMVy05NYILONzj8SvV3TMxmnsj c6FGn+zHsblakfKhyZnrTGWEo01wK6F+xk1xa8Qo5pDce3mv1NnoxFZwupxc64IUK0of J71gDlCz5iy0brQrW44YfC5KRpzBxW8Mco1ZirDhFc/wJ4PmXpLaFN84V6QxNzBQVq55 8GTA== X-Gm-Message-State: AGRZ1gJcrRJUPkuTiaGL3wPsz5ejHU5ub+u1lV2pPyxTg60fS4Q26F1U +xMav3t0qZngkZDIhUEQV9M= X-Google-Smtp-Source: AJdET5ex+Udb1jwvtCeAg/yDGXR5Sss21rFxipueOOkRebvztPGfvUjf3vD6WLucb+lLoOM0YN7a6g== X-Received: by 2002:a5d:558a:: with SMTP id i10-v6mr8939647wrv.38.1541486802971; Mon, 05 Nov 2018 22:46:42 -0800 (PST) Received: from [192.168.0.122] (ip-86-49-239-100.net.upcbroadband.cz. [86.49.239.100]) by smtp.googlemail.com with ESMTPSA id v129-v6sm727407wme.45.2018.11.05.22.46.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Nov 2018 22:46:41 -0800 (PST) Subject: Re: FreeBSD 11.x thread creation time is 9000+ microseconds on Intel Xeon Gold series CPU To: freebsd-hackers@freebsd.org References: From: =?UTF-8?Q?V=c3=a1clav_Haisman?= Openpgp: preference=signencrypt Autocrypt: addr=vhaisman@gmail.com; prefer-encrypt=mutual; keydata= mDMEWf3f1RYJKwYBBAHaRw8BAQdA+8IStMkkgNpf51nGCCVfGgaiznOp+xPcQtrfgfGLS9K0 JFbDoWNsYXYgSGFpc21hbiA8dmhhaXNtYW5AZ21haWwuY29tPoh8BBMWCAAkAhsDBQsJCAcC BhUICQoLAgQWAgMBAh4BAheABQJZ/emtAhkBAAoJEAS4nVHf5aIV+XwA/0ksz6qVucX60Aju 2+NHaNAvCwDZ9VCE6sXwTCVl/7T2AQD36hCK4CXDscIPhZ/j+EpoU1nQlhVkVXv3gvnqDCBz Cbg4BFn939USCisGAQQBl1UBBQEBB0BPTwtPhiWgvubjRmH8c2TCzUfRwqbCE79wJRJPMynx EwMBCAeIZwQYFggADwIbDAUCWf3h0gUJAeE1fQAKCRAEuJ1R3+WiFf1lAP9liLHDNkn5xu1q 2XA0ivSPhXSBVdyQht35zB5qascEbAEAphYqkkB6EDEtadOYSF4eaGYxUJE4GnHRuMxNY3JP eQa4MwRZ/eHgFgkrBgEEAdpHDwEBB0CGctdGA5BtgIA5eWP9n9cvWQIZGTTsIDA2vJZgWdBk mojHBBgWCAAPBQJZ/eHgAhsCBQkB4TOAAGoJEAS4nVHf5aIVXyAEGRYIAAYFAln94eAACgkQ qJdP1ET5MFwf2QEA2AKskTdBJKQRVhqdK7LFOrQ/68Hu9VPd1ivCyP47WPYBAKF5OlmwdJAR tvM31BLUsJkBx984WMCT1KDvoK219AYL8VEBAN7goxU95eCGtQ/QvLBGPhrSX7i7FgsR2OVr qjfzRZ9uAP4qHRvzeUjr1EDCg6C9OZc+IPZgcS06kM9UZQSCNgb5BQ== Message-ID: <7bb8463e-aa5d-bade-7ae7-6c612bae65f9@gmail.com> Date: Tue, 6 Nov 2018 07:46:33 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="OpV3w5i4nFj9vxHOOI8hRdlXMGOMjcdZo" X-Rspamd-Queue-Id: 5E9FC8F234 X-Spamd-Result: default: False [-6.67 / 200.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; SIGNED_PGP(-2.00)[]; NEURAL_HAM_SHORT(-0.86)[-0.859,0]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-0.70)[ipnet: 2a00:1450::/32(-1.96), asn: 15169(-1.47), country: US(-0.08)]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[9.2.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2018 06:46:45 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --OpV3w5i4nFj9vxHOOI8hRdlXMGOMjcdZo Content-Type: multipart/mixed; boundary="3GItp3KyG4VyP6g7Qej0IVnTrtbcvhE9C"; protected-headers="v1" From: =?UTF-8?Q?V=c3=a1clav_Haisman?= To: freebsd-hackers@freebsd.org Message-ID: <7bb8463e-aa5d-bade-7ae7-6c612bae65f9@gmail.com> Subject: Re: FreeBSD 11.x thread creation time is 9000+ microseconds on Intel Xeon Gold series CPU References: In-Reply-To: --3GItp3KyG4VyP6g7Qej0IVnTrtbcvhE9C Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: quoted-printable On 06. 11. 18 5:01, Steevan Rodrigues wrote: > Hi , > I am seeing a FreeBSD 11.x OS poor performance issue . > CPU is Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz ( dual socket CPU with = 10 > cores per socket ) >=20 > I have attached a simple program which creates thread and computes time= There is no attachment. The mailing list software probably stripped that.= > taken to create this thread. On this CPU with FreeBSD 11.x OS it takes = 9000 > to 15000 micro seconds ( us) to create > just one thread. >=20 > On other platforms this thread creation time is usually 20 to 30 us onl= y. > Any idea why it takes so much more time with FreeBSD 11.x ? > Is there any processor specific tuning that needs to be done ? >=20 > Thanks, > Steevan > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o= rg" >=20 --=20 VH --3GItp3KyG4VyP6g7Qej0IVnTrtbcvhE9C-- --OpV3w5i4nFj9vxHOOI8hRdlXMGOMjcdZo Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQTkBikvfQi7sAhGExQEuJ1R3+WiFQUCW+E40AAKCRAEuJ1R3+Wi FaHXAP4gph0xiOdYnlG+wTcVhtpWrtWDOPMAz45PE9cUKYu12wEA/eFShbTvIfW/ qA7am8DBI5ADU2xmLa6jQyc88V6M4Qk= =bAAc -----END PGP SIGNATURE----- --OpV3w5i4nFj9vxHOOI8hRdlXMGOMjcdZo-- From owner-freebsd-hackers@freebsd.org Tue Nov 6 09:01:23 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 83F17110C680 for ; Tue, 6 Nov 2018 09:01:23 +0000 (UTC) (envelope-from steevanxperia@gmail.com) Received: from mail-oi1-x241.google.com (mail-oi1-x241.google.com [IPv6:2607:f8b0:4864:20::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 76E806D885 for ; Tue, 6 Nov 2018 09:01:22 +0000 (UTC) (envelope-from steevanxperia@gmail.com) Received: by mail-oi1-x241.google.com with SMTP id v83-v6so10064373oia.5 for ; Tue, 06 Nov 2018 01:01:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=hhy+C6Eqt+cvhZM560IDHNN8WrqfzoYVhW9ybjsAlCY=; b=kZRZYpRH+piZ4rGzR5Mr205YuF0RfiZfVgg6+2l5MXzryGGJ3dbzqk4fCkLSsffrVP 4sOYwW5Q5vYc3/yDwadjEg8JjkBo7pcGwVeDeHyH/qthglfS5rj5Dxg06GcrepbB5TZd o+JCmqGSl34qfzQuXGYS5O3dzm5oaVU5GMh1Nz0DDpz3xc6LiNgXDmHDhECfxebsAp3y UoEGkLYqTqiASa+zSBo3oQ8Mvubw9xbObZhXx2/BTXFVlkmdeaSgKgh9hWSl/oIpUENb OmFsZiIxNjSumTxGnD7m4o6Nz3OzUWvuKOzBXXrKe0fDK5wWkBnDWzGkBGoIGG6ACwNs kPIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=hhy+C6Eqt+cvhZM560IDHNN8WrqfzoYVhW9ybjsAlCY=; b=VvhwSMPjlv2ZXNUeYR69PLer+rg5HQxdJ/o1IQRd+lwSZNxmPPjiKPnVwDyFG16rnM 1w69kxAYESkz54wMg5z3tvFx1D0X6/RjXprDN0w85UWlqS04q7bpxEYnzhCBxYB7EoqH UMqEK+T+4ipndV3or5PHu7Yt34Qhqip46aHr82mT3HPA/gmdnl9vYl3kmc4nnFD+P7Sy lB8g/Ll9JEpscsQ5jQvHo/kImwv+80Nm6moXNDKy/5EdeoNc2vCZsKPrrAA56RNvfPwl sguXRFevZEkhmvwyypuCc599zYjj/RY5xsOtLMLpGV12YWciNQYpR0Mv6DTsiUTcVJz7 H/vA== X-Gm-Message-State: AGRZ1gJTkVKgE37DcuuEm4KOIcdGzVqd6ZjCHAA/GcROSRwOYf/fH3NC t8Xzb6ZK8LbovTAMi+U1DP12ZXMqxJOiOqFQ+5c= X-Google-Smtp-Source: AJdET5c140+MshuljqX9X5SiZMpWv6/J38Kc6ieCBtiCC2iVjQRtBSGNsVGN1HkYt44lDZWUyVCxUNvYBrH1mrO4tAc= X-Received: by 2002:aca:6005:: with SMTP id u5-v6mr15464786oib.201.1541494881136; Tue, 06 Nov 2018 01:01:21 -0800 (PST) MIME-Version: 1.0 References: <7bb8463e-aa5d-bade-7ae7-6c612bae65f9@gmail.com> In-Reply-To: <7bb8463e-aa5d-bade-7ae7-6c612bae65f9@gmail.com> From: Steevan Rodrigues Date: Tue, 6 Nov 2018 14:31:09 +0530 Message-ID: Subject: Re: FreeBSD 11.x thread creation time is 9000+ microseconds on Intel Xeon Gold series CPU To: vhaisman@gmail.com Cc: freebsd-hackers@freebsd.org X-Rspamd-Queue-Id: 76E806D885 X-Spamd-Result: default: False [-3.30 / 200.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.994,0]; R_DKIM_ALLOW(-0.20)[gmail.com]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; HAS_ATTACHMENT(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; MIME_GOOD(-0.10)[multipart/mixed,multipart/alternative,text/plain]; NEURAL_SPAM_SHORT(0.24)[0.237,0]; NEURAL_HAM_LONG(-1.00)[-0.997,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCVD_IN_DNSWL_NONE(0.00)[1.4.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; IP_SCORE(-0.53)[ipnet: 2607:f8b0::/32(-1.14), asn: 15169(-1.46), country: US(-0.08)] X-Rspamd-Server: mx1.freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2018 09:01:23 -0000 Ok. Sorry about that. Again I have attached that test program as a tar file. I am not sure whether even this gets stripped by mailing list software. Anyway here is the program given below. #define _GNU_SOURCE #include #include #include #include #include #include #include #include #define CPU_NUM 5 #define CPUBIND void *myThreadFun(void *vargp) { sleep(1); printf("Printing from demo for thread creation time \n"); return NULL; } void main(void) { pthread_t thread_id; unsigned long int usec, usec2, sec, sec2; struct timeval tv, tv2; struct timezone tz; pthread_attr_t attr; #ifdef CPUBIND cpuset_t cset; const pid_t pid =3D getpid(); CPU_ZERO(&cset); CPU_SET(CPU_NUM,&cset); cpusetid_t setid; cpuset(&setid); //sched_setaffinity(pid, sizeof(cpu_set_t), &cset); //cpuset_setaffinity(CPU_LEVEL_CPUSET, CPU_WHICH_CPUSET, setid, sizeof(cpuset_t), &cset); cpuset_setaffinity(CPU_LEVEL_WHICH, CPU_WHICH_PID, pid, sizeof(cpuset_t), &cset); #endif pthread_attr_init(&attr); pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE); gettimeofday(&tv, &tz); // take time before creating thread pthread_create(&thread_id, &attr, myThreadFun, NULL); gettimeofday(&tv2, &tz); // take time after creation of thread pthread_join(thread_id, NULL); usec =3D tv.tv_usec; sec =3D tv.tv_sec; usec2 =3D tv2.tv_usec; sec2 =3D tv2.tv_sec; printf("Thread creation time details are: \n"); printf("Before: Sec %lu usec %lu After: Sec %lu usec %lu : SecDiff %lu UsecDiff %lu \n", sec, usec, sec2, usec2, sec2 - sec , usec2 - usec); } --------------------------------------------- Thanks Steevan On Tue, Nov 6, 2018 at 12:34 PM V=C3=A1clav Haisman wr= ote: > On 06. 11. 18 5:01, Steevan Rodrigues wrote: > > Hi , > > I am seeing a FreeBSD 11.x OS poor performance issue . > > CPU is Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz ( dual socket CPU with = 10 > > cores per socket ) > > > > I have attached a simple program which creates thread and computes time > > There is no attachment. The mailing list software probably stripped that. > > > taken to create this thread. On this CPU with FreeBSD 11.x OS it takes > 9000 > > to 15000 micro seconds ( us) to create > > just one thread. > > > > On other platforms this thread creation time is usually 20 to 30 us onl= y. > > Any idea why it takes so much more time with FreeBSD 11.x ? > > Is there any processor specific tuning that needs to be done ? > > > > Thanks, > > Steevan > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to " > freebsd-hackers-unsubscribe@freebsd.org" > > > > > -- > VH > > From owner-freebsd-hackers@freebsd.org Tue Nov 6 09:47:39 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E976D110DB60 for ; Tue, 6 Nov 2018 09:47:38 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from mail.madpilot.net (vogon.madpilot.net [159.69.1.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A2E706F30C for ; Tue, 6 Nov 2018 09:47:35 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from mail (mail [192.168.254.3]) by mail.madpilot.net (Postfix) with ESMTP id 42q4TX56Jjz6dPH; Tue, 6 Nov 2018 10:47:28 +0100 (CET) Received: from mail.madpilot.net ([192.168.254.3]) by mail (mail.madpilot.net [192.168.254.3]) (amavisd-new, port 10024) with ESMTP id lRlvQVN02X8v; Tue, 6 Nov 2018 10:47:26 +0100 (CET) Received: from marvin.madpilot.net (host109-128-dynamic.16-87-r.retail.telecomitalia.it [87.16.128.109]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.madpilot.net (Postfix) with ESMTPSA; Tue, 6 Nov 2018 10:47:26 +0100 (CET) Subject: Re: FreeBSD 11.x thread creation time is 9000+ microseconds on Intel Xeon Gold series CPU To: Steevan Rodrigues , vhaisman@gmail.com Cc: freebsd-hackers@freebsd.org References: <7bb8463e-aa5d-bade-7ae7-6c612bae65f9@gmail.com> From: Guido Falsi Openpgp: preference=signencrypt Autocrypt: addr=mad@madpilot.net; prefer-encrypt=mutual; keydata= xsBNBE+G+l0BCADi/WBQ0aRJfnE7LBPsM0G3m/m3Yx7OPu4iYFvS84xawmRHtCNjWIntsxuX fptkmEo3Rsw816WUrek8dxoUAYdHd+EcpBcnnDzfDH5LW/TZ4gbrFezrHPdRp7wdxi23GN80 qPwHEwXuF0X4Wy5V0OO8B6VT/nA0ADYnBDhXS52HGIJ/GCUjgqJn+phDTdCFLvrSFdmgx4Wl c0W5Z1p5cmDF9l8L/hc959AeyNf7I9dXnjekGM9gVv7UDUYzCifR3U8T0fnfdMmS8NeI9NC+ wuREpRO4lKOkTnj9TtQJRiptlhcHQiAlG1cFqs7EQo57Tqq6cxD1FycZJLuC32bGbgalABEB AAHNHkd1aWRvIEZhbHNpIDxtYWRAbWFkcGlsb3QubmV0PsLAeQQTAQgAIwIbAwIeAQIXgAUL CQgHAwUVCgkICwQWAgMBBQJS79AgAhkBAAoJEBrmhg5Wy9KTc0kH/RO64ORBlTbTHaUaOj8F Je5O5NU2Pt9Cyt5ZWBRvxntr1zPTJGKRPS9ihlIfqT4ZvEngQGp57EUyFbCpI0UWasTerImM tt5WACnGmCzUTB39UXx8Oy4b1EgWeTJQ747e/F1mQLXTNa6ijRBE9fYlTb4gAkPN88/wVV9v 3PZozKLTg16ghBzHM/P7Lk8L7clPEZChX1FTa/6eSt3nvzfCuTMZbBPJF/ph+q1KyPqRgVfh tyhu5dvgMoPz/ni41IfeSrkJTD5RXzdyGR9q4Z1NYeBsLkRjC4LxKAP5KqUsvlOUjKvO1byj ApYdMarol+IGkaSk9e3zVYAJkWKjn/ni8XbOwU0EUxB7QQEQAKFhrDceoPdK/IHDSmoj6SQY isvM7VdhcleS7E9DoEAVt7yMbf6HbbMVTTY6ckvwTWQssywLBXNVqxgc4WLJjzfUhgef+WE7 5M3+WFYlOVQLGZY/zEVgma1raYnOHNAOzeHLDmEXjbZP6vGAeDyBbGfQPpE7qGYZ7ubeT3Xw QO+PklcCrvOPj2ZPcAxGNS2xVU/LzONqCrJqLMJSIcCdsbiSP4G5PnDFHtMokaTY6OEr8OEQ fOAerhcHUa/z7Uu8YtmaqKH+QGkE/WEgaRqSiTnv0JOTD+DxehaqvoKPPZ++2NpCZMHB2i6A /xifmQwEiIjEXtcueBRzkNUQkxhqZyS13SrhocL9ydtaVPBzZatAEjUDDEJmAMLVFs45qfyh MiNapHJo2n3MW/E5omqCvEkDdWX/en3P7CK2TemeaDghMsgkNKax/z0wNo5UZCkOPOz0xpNi UilOVbkuezZZNg65741qee2lfXhQIaZ66yT7hphc/N/z3PIAtLeze4u1VR2EXAuZ2sWAdlKC NTlJMsaU/x70BV11Wd/ypnVzM68dfdQIIAj1iMFAD/lXGlEUmKXg5Ov2VQDlTntQoanCYrAg +8CttPzjrydgLZFq3hrtQmfc0se5yv1WHS69+BsUOG09RvvawUDZxUjW19kyeN9THaNRgow3 kSuArUp6zSmJABEBAAHCwF8EGAEIAAkFAlMQe0ECGwwACgkQGuaGDlbL0pMN5wgA4bCkX/qw EVC06ToeR6C2putmSWQMgpDaqrv65Hubo+QGmg2P4ewTYQQ4g6oYWS03qHxqVVWhKz7FjfrV +dH8qbCLfSgIcvdBha7ayGZVrsiuMLKGbw36fcmkZPpSDOfHcP0XH8Z+u9CWj0xUkTxAlZ/7 i6gYSUpG2JWNtdmE/X8VVEyXusCLwy0K0BI60A/4dRTIX3C4QKrJ3ZbUXegz70ynjHf+lQMZ 9IZKASoRMuS5FozPQh6abvmwZEPdf5I9riUElzvHrqJ8Bx0t3Pujdoth+yNHpnBxrtO8LkQd rQ58P0SwcaIX33T2U9pG8bhu5YVR88FQ8OQ0cEsPBpDncg== Message-ID: Date: Tue, 6 Nov 2018 10:47:26 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: A2E706F30C X-Spamd-Result: default: False [-3.48 / 200.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.994,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-0.99)[-0.987,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[madpilot.net]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[eddie.gfratio.it,vogon.madpilot.net]; NEURAL_HAM_SHORT(-0.93)[-0.929,0]; IP_SCORE(-0.26)[ipnet: 159.69.0.0/16(1.53), asn: 24940(-2.81), country: DE(-0.01)]; FREEMAIL_TO(0.00)[gmail.com]; RECEIVED_SPAMHAUS_PBL(0.00)[109.128.16.87.zen.spamhaus.org : 127.0.0.10]; R_DKIM_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:24940, ipnet:159.69.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2018 09:47:39 -0000 On 06/11/18 10:01, Steevan Rodrigues wrote: > Ok. Sorry about that. Again I have attached that test program as a tar > file. I am not sure whether even this gets stripped by mailing list > software. > > Anyway here is the program given below. > I'm not a kernel guy or an expert in such things, but I see that flamegraphs are often used to analyze such issues. It gives an idea where the kernel is actually spending it's time. There's also a port to help with this: https://www.freshports.org/benchmarks/flamegraph/ I think if you could gather such information and make it available you'd make it easier for experts to explain what is happening and maybe also do something about this. -- Guido Falsi From owner-freebsd-hackers@freebsd.org Tue Nov 6 11:05:34 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3E4BF110F7EA for ; Tue, 6 Nov 2018 11:05:34 +0000 (UTC) (envelope-from steevanxperia@gmail.com) Received: from mail-ot1-x32a.google.com (mail-ot1-x32a.google.com [IPv6:2607:f8b0:4864:20::32a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0A0B3718E5 for ; Tue, 6 Nov 2018 11:05:33 +0000 (UTC) (envelope-from steevanxperia@gmail.com) Received: by mail-ot1-x32a.google.com with SMTP id n46so6766706otb.9 for ; Tue, 06 Nov 2018 03:05:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=614dQ35afF3HYNna+nV2fEhiDDfe9chjnt/ozXeAlJY=; b=f3IPY5yweM3CZ6av4uruGYclzzyEr8uIpFtAfUnwbhqKjQeW+ogViSwCJFW5AGYIpz wQJlNJX4C9uEyDs9TXSFXYEentUEd8/j/QTW9ne5J4/RyY6BW2oDibdFI3gBTEBp+aJl zupByPscWXcLXcwwoiWUmNm2nmPuYm1n0FHF3tayGBdiySbT9QGPHn3eJk0CY5z0uDcZ wphc3t4IR1J/S2OKeUePiBhWeNUP0Eod4jBZuf024GcruVO8kD78B++cKn9SPxsispBy OeLONj4+K8Z4/F5KhL+0uzlenxtDNcxwJ3FE1uZCKIBYbVh30sRIi93daXnFIZZvRWWQ GWog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=614dQ35afF3HYNna+nV2fEhiDDfe9chjnt/ozXeAlJY=; b=DyBeBI7qAQuZWx+VltdatxL6bJauICyoGhVtnkwgfLMBhrradvSVipcx+Gyf4zKC5g s0xuNJOry4EMSp7tGV9y08FERos7VNLjL22NWWDlcjEM33shU4QTxRPZB8NL5ZHU5aCE iw+dG+Hg16YxqXpdIpDqr0xao3FQO7+om1w3tETgbcuCtDgH5WBfQxOFtyLfSwDmCw08 ts+DtVF9k+IpPsSKCWgsiIyOyCKqC3pvaoqM/tico1dmpXGfKdo4ybScAOnW1dPNp8Gm a6z8kly8yyQItG5MJA4wP/PujsAymHvNMeFXBPTtkEpP1WMkWRjoyokppt5yhqmyauIZ Q0yQ== X-Gm-Message-State: AGRZ1gLnDL0gkwwtqUixXjKKgQeiDvQ2gB1I/2pKyxJs5igr8tEZr7p1 gR1lNUd7DeE67cDTJ0f6q5JHWOWV2keDd+aYzIU= X-Google-Smtp-Source: AJdET5eWr+xJwLK4AH+OFRCkaUl12NiOujJrksfMvpLNuKqW53DFWv30KXuaB/y2Z9Od3j8u0JZtQbv3gJxYnoeiLIU= X-Received: by 2002:a9d:128b:: with SMTP id g11mr15122761otg.130.1541502332021; Tue, 06 Nov 2018 03:05:32 -0800 (PST) MIME-Version: 1.0 References: <7bb8463e-aa5d-bade-7ae7-6c612bae65f9@gmail.com> In-Reply-To: From: Steevan Rodrigues Date: Tue, 6 Nov 2018 16:35:20 +0530 Message-ID: Subject: Re: FreeBSD 11.x thread creation time is 9000+ microseconds on Intel Xeon Gold series CPU To: mad@madpilot.net Cc: vhaisman@gmail.com, freebsd-hackers@freebsd.org X-Rspamd-Queue-Id: 0A0B3718E5 X-Spamd-Result: default: False [-4.20 / 200.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; R_DKIM_ALLOW(-0.20)[gmail.com]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.997,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[a.2.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; IP_SCORE(-0.56)[ipnet: 2607:f8b0::/32(-1.27), asn: 15169(-1.47), country: US(-0.08)]; NEURAL_HAM_SHORT(-0.63)[-0.630,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FREEMAIL_CC(0.00)[gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Server: mx1.freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2018 11:05:34 -0000 Thanks to all of you for the quick responses. I see this issue only on a particular server. I do not see this issue on another server with slightly older processor Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz (dual socket 6 core per socket) In this system the same program takes only about 800 usec I wonder whether it is problem with 11.x FreeBSD release on this particular processor. I came across two results (given below) from Phoronix OSbench test suite which also show poor thread creation time for FreeBSD 11.x https://openbenchmarking.org/result/1804094-AR-1804096AR58 https://openbenchmarking.org/result/1807114-RA-WINDOWSSE47 In this look for "Create Threads" row. Thanks Steevan On Tue, Nov 6, 2018 at 3:17 PM Guido Falsi wrote: > On 06/11/18 10:01, Steevan Rodrigues wrote: > > Ok. Sorry about that. Again I have attached that test program as a tar > > file. I am not sure whether even this gets stripped by mailing list > > software. > > > > Anyway here is the program given below. > > > > I'm not a kernel guy or an expert in such things, but I see that > flamegraphs are often used to analyze such issues. It gives an idea > where the kernel is actually spending it's time. > > There's also a port to help with this: > > https://www.freshports.org/benchmarks/flamegraph/ > > I think if you could gather such information and make it available you'd > make it easier for experts to explain what is happening and maybe also > do something about this. > > -- > Guido Falsi > From owner-freebsd-hackers@freebsd.org Tue Nov 6 12:48:31 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BC5CC1122B4C for ; Tue, 6 Nov 2018 12:48:30 +0000 (UTC) (envelope-from se@freebsd.org) Received: from mailout03.t-online.de (mailout03.t-online.de [194.25.134.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mailout00.t-online.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D4AD574FC2 for ; Tue, 6 Nov 2018 12:48:29 +0000 (UTC) (envelope-from se@freebsd.org) Received: from fwd10.aul.t-online.de (fwd10.aul.t-online.de [172.20.26.152]) by mailout03.t-online.de (Postfix) with SMTP id 0FF1E4208777; Tue, 6 Nov 2018 13:48:22 +0100 (CET) Received: from Stefans-MBP-LAN.fritz.box (bpREusZcoh-zhlDv+lz9LU9+mUIYXJSYZ7tubNpOnrON81CTsiyKGVLpoyacf40QXV@[80.128.111.25]) by fwd10.t-online.de with (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384 encrypted) esmtp id 1gK0mF-2RZnSS0; Tue, 6 Nov 2018 13:48:15 +0100 Subject: Re: FreeBSD 11.x thread creation time is 9000+ microseconds on Intel Xeon Gold series CPU To: Steevan Rodrigues , mad@madpilot.net Cc: freebsd-hackers@freebsd.org, vhaisman@gmail.com References: <7bb8463e-aa5d-bade-7ae7-6c612bae65f9@gmail.com> From: Stefan Esser Openpgp: preference=signencrypt Autocrypt: addr=se@freebsd.org; prefer-encrypt=mutual; keydata= xsBNBFVxiRIBCADOLNOZBsqlplHUQ3tG782FNtVT33rQli9EjNt2fhFERHIo4NxHlWBpHLnU b0s4L/eItx7au0i7Gegv01A9LUMwOnAc9EFAm4EW3Wmoa6MYrcP7xDClohg/Y69f7SNpEs3x YATBy+L6NzWZbJjZXD4vqPgZSDuMcLU7BEdJf0f+6h1BJPnGuwHpsSdnnMrZeIM8xQ8PPUVQ L0GZkVojHgNUngJH6e21qDrud0BkdiBcij0M3TCP4GQrJ/YMdurfc8mhueLpwGR2U1W8TYB7 4UY+NLw0McThOCLCxXflIeF/Y7jSB0zxzvb/H3LWkodUTkV57yX9IbUAGA5RKRg9zsUtABEB AAHNKVN0ZWZhbiBFw59lciAoWWFob28hKSA8c3QuZXNzZXJAeWFob28uZGU+wsCWBBMBCgBA AhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AWIQSjceplnAvsyCtxUxNH67XvWv31RAUC WvLvqwUJCyUBEwAKCRBH67XvWv31REySCACc6vqcSFQCRyBRc2CV5ZBjbbnTy7VBoXbUS3/c 4Hn8I0YQ39q7//2z8vYsgLeM1mMXL4PUIU/0f0dBAFBLpxV7bntGzyCJls6SeGS/qcQKhqaI 6I7NcWg8OkIJIhUL6q238cS1ql9pU65fyHe0PP8JS08m81PDpX2/4wTE6h2jgYUy55eXRzoF MEjr1S8SSnidsBem27o7iWu9ltJsUtE86071iZlLzbuHv2nvucrjAV9cK9tHrxYT/YiY8QhT L48iWj2xIjLjg1ebmgIFZ2k881we/KTIoUugqOOR1gDSc4qwM8CA388cN3frjtl98CwhAT5T UV8tIDqri+/Z1AKwzsBNBFVxiRIBCACxI/aglzGVbnI6XHd0MTP05VK/fJub4hHdc+LQpz1M kVnCAhFbY9oecTB/togdKtfiloavjbFrb0nJhJnx57K+3SdSuu+znaQ4SlWiZOtXnkbpRWNU eMm+gtTDMSvloGAfr76RtFHskdDOLgXsHD70bKuMhlBxUCrSwGzHaD00q8iQPhJZ5itb3WPq z3B4IjiDAWTO2obD1wtAvSuHuUj/XJRsiKDKW3x13cfavkad81bZW4cpNwUv8XHLv/vaZPSA ly+hkY7NrDZydMMXVNQ7AJQufWuTJ0q7sImRcEZ5EIa98esJPey4O7C0vY405wjeyxpVZkpq ThDMurqtQFn1ABEBAAHCwHwEGAEKACYCGwwWIQSjceplnAvsyCtxUxNH67XvWv31RAUCWvLv qwUJCyUBGQAKCRBH67XvWv31RLnrB/9gzcRlpx71sDMosoZULWn7wysBJ/8AIEfIByRaHQe3 pn/KwE57pB+zFbbQqB7YzeZb7/UUgR4zU2ZbOcEfwDZcHUbj0B3fGRsS3t0uiLlAd8w0sBZb SxrqzjdpDjIbOZkxssqUmvrsN67UG1AFWH9aD24keBS7YjPBS8hLxPeYV+Xz6vUL8fRZje/Z JgiBMIwyj6g2lH/zkdnxBdC0iG1xxJOLTaghMMeQyCdH6ef8+VMyAlAJsMckbOTvx63tY8z7 DFcrnTJfbe1EziRilVsEaK8tTzJzhcTfos+f3eBYWEilxe5HzIhYKJeC7lmsSUcGwa6+9VRg a0ctmi9Z8OgX Message-ID: <0f7e583f-9ad9-f210-4bdb-1ace13ab8c5f@freebsd.org> Date: Tue, 6 Nov 2018 13:48:12 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:60.0) Gecko/20100101 Thunderbird/60.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 7bit X-ID: bpREusZcoh-zhlDv+lz9LU9+mUIYXJSYZ7tubNpOnrON81CTsiyKGVLpoyacf40QXV X-TOI-MSGID: cf3ae718-8e93-4bf3-84e3-1e5f3a48c87f X-Rspamd-Queue-Id: D4AD574FC2 X-Spamd-Result: default: False [-103.14 / 200.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; ALLOW_DOMAIN_WHITELIST(-100.00)[freebsd.org]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-0.99)[-0.992,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; R_SPF_SOFTFAIL(0.00)[~all]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: mx66.freebsd.org]; NEURAL_HAM_SHORT(-1.00)[-0.995,0]; RCVD_IN_DNSWL_NONE(0.00)[81.134.25.194.list.dnswl.org : 127.0.5.0]; RWL_MAILSPIKE_GOOD(0.00)[81.134.25.194.rep.mailspike.net : 127.0.0.18]; IP_SCORE(-0.04)[asn: 3320(-0.20), country: DE(-0.01)]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:3320, ipnet:194.25.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2018 12:48:31 -0000 Am 06.11.18 um 12:05 schrieb Steevan Rodrigues: > Thanks to all of you for the quick responses. > > I see this issue only on a particular server. > I do not see this issue on another server with slightly older processor > Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz (dual socket 6 core per socket) > In this system the same program takes only about 800 usec > > I wonder whether it is problem with 11.x FreeBSD release on this particular > processor. > I came across two results (given below) from Phoronix OSbench test suite > which also show poor thread creation time for FreeBSD 11.x > > https://openbenchmarking.org/result/1804094-AR-1804096AR58 > https://openbenchmarking.org/result/1807114-RA-WINDOWSSE47 > > In this look for "Create Threads" row. This list shows, that FreeBSD-12 is fast, only FreeBSD-11 shows the long thread creation time (16000 vs 36 in whatever units), even though FreeBSD-12 was running with KPTI-Patches. This lets me think that the issue that caused these delays has been fixed, but not merged to FreeBSD-11, yet.# I guess that the cause for the delays is the synchronization IPIs between sockets, and that these might be very high due to some cores in deep sleep modes (with corresponding long wake-up delays). Regards, STefan From owner-freebsd-hackers@freebsd.org Tue Nov 6 13:43:33 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C6C5B112409D for ; Tue, 6 Nov 2018 13:43:33 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E2DCD76DF4 for ; Tue, 6 Nov 2018 13:43:22 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id wA6DhDvK019456 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 6 Nov 2018 14:43:14 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: steevanxperia@gmail.com Received: from [10.58.0.4] (dadv@[10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id wA6DhC9Q090287 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 6 Nov 2018 20:43:12 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: FreeBSD 11.x thread creation time is 9000+ microseconds on Intel Xeon Gold series CPU To: Steevan Rodrigues , freebsd-hackers@freebsd.org References: From: Eugene Grosbein Message-ID: Date: Tue, 6 Nov 2018 20:43:13 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: E2DCD76DF4 X-Spamd-Result: default: False [-2.50 / 200.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.993,0]; MX_INVALID(0.50)[greylisted]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-0.98)[-0.977,0]; R_BAD_CTE_7BIT(1.05)[7bit,utf8]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.94)[-0.942,0]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; IP_SCORE(-1.04)[ipnet: 2a01:4f8::/29(-2.30), asn: 24940(-2.89), country: DE(-0.01)]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2018 13:43:33 -0000 06.11.2018 11:01, Steevan Rodrigues пишет: > Hi , > I am seeing a FreeBSD 11.x OS poor performance issue . > CPU is Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz ( dual socket CPU with 10 > cores per socket ) > > I have attached a simple program which creates thread and computes time > taken to create this thread. On this CPU with FreeBSD 11.x OS it takes 9000 > to 15000 micro seconds ( us) to create > just one thread. > > On other platforms this thread creation time is usually 20 to 30 us only. > Any idea why it takes so much more time with FreeBSD 11.x ? > Is there any processor specific tuning that needs to be done ? Please set "sysctl kern.eventtimer.periodic=1" and retry your tests. From owner-freebsd-hackers@freebsd.org Tue Nov 6 13:48:33 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C8991112421B for ; Tue, 6 Nov 2018 13:48:33 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0B4AE76F88 for ; Tue, 6 Nov 2018 13:48:32 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id wA6DmQRx019507 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 6 Nov 2018 14:48:26 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: steevanxperia@gmail.com Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id wA6DmPHs090378 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 6 Nov 2018 20:48:25 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: FreeBSD 11.x thread creation time is 9000+ microseconds on Intel Xeon Gold series CPU To: Steevan Rodrigues , freebsd-hackers@freebsd.org References: From: Eugene Grosbein Message-ID: <157d9126-158f-d50b-0c38-672d08f199d7@grosbein.net> Date: Tue, 6 Nov 2018 20:48:21 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 0B4AE76F88 X-Spamd-Result: default: False [-3.61 / 200.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; MX_INVALID(0.50)[cached]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.998,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.98)[-0.983,0]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; IP_SCORE(-1.03)[ipnet: 2a01:4f8::/29(-2.26), asn: 24940(-2.88), country: DE(-0.01)]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2018 13:48:33 -0000 06.11.2018 20:43, Eugene Grosbein wrote: >> I am seeing a FreeBSD 11.x OS poor performance issue . >> CPU is Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz ( dual socket CPU with 10 >> cores per socket ) >> >> I have attached a simple program which creates thread and computes time >> taken to create this thread. On this CPU with FreeBSD 11.x OS it takes 9000 >> to 15000 micro seconds ( us) to create >> just one thread. >> >> On other platforms this thread creation time is usually 20 to 30 us only. >> Any idea why it takes so much more time with FreeBSD 11.x ? >> Is there any processor specific tuning that needs to be done ? > > Please set "sysctl kern.eventtimer.periodic=1" and retry your tests. Also "sysctl kern.timecounter.alloweddeviation=0" may help, too. From owner-freebsd-hackers@freebsd.org Tue Nov 6 16:58:32 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0A8811129F23 for ; Tue, 6 Nov 2018 16:58:32 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-io1-xd33.google.com (mail-io1-xd33.google.com [IPv6:2607:f8b0:4864:20::d33]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4A0D58020A; Tue, 6 Nov 2018 16:58:31 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: by mail-io1-xd33.google.com with SMTP id o19-v6so9742350iod.3; Tue, 06 Nov 2018 08:58:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=FdC60EvGZu6VkSRFNUb8c6Ghtf8BDzU3CxNtiHWzfDw=; b=OyBlrNmMV+9/oixmavm4FwzuiZNNvCiZxiuFeUoT1hlBOYNmcJLXiiEGYuX041oIx5 +czgAdn6iCrtyrGZr3SR92UtVUrLOcYbLZ+BbrAWu46kai+CAO+m15iT9VdK+3Fdb4Ly TbbI3w3+Y0Jkh5X3a+YD1GRGq2xoPDqhWBRaC+8Vh8BUQ/Fs3UDhf2GeWzBVkK2/1qNQ IedPYW+Yyf+DpdoeE1s8elA/yaZGquM5Hc4ZVz2A23xcdTfV8kYoSHwNG36bwSyOXjBs JuaI/jmsTcUX+HYk7pq7Cj6mLUQB/g4KE01hrwpUySzDN+zzIAYssGvN8/spbRsQaotn CUTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=FdC60EvGZu6VkSRFNUb8c6Ghtf8BDzU3CxNtiHWzfDw=; b=uAGOfNiebUxEyxdJ02l7uv4rUfqmv8mcE1n2LQEmRl05E7zL5NOEBT51cdnS2rJfRr GSQk+M3xJrGv6Yx5XKN7cusAqDjXpRUHBa2juGAq2P5t0RpDGcyfSwii8YPbzxUmEyaX lel6g/mrJkgLVqdJh3mqvELPjsEseubafEDKyqhfIzn+wPwenDwz3SxzcV5MgradHlP2 Tw6ysnBy11+6zOyrj2P7ItI4ZwHGZLXM+3xtyhcGolqSXXQAByZIG02wRpDSvEb8w9KA ZEaKJtXSKLyFtv4GPGKEJ8Av3p5a8vT/VJjDAtz/MBIr9CplrD17DVmUmDylRFuOHFHc qE1A== X-Gm-Message-State: AGRZ1gJlDLTd6aebIjXWl/jHBWzCV6X4LgHkqrzVm2F+DwYdNTOASk+u SpL7Ee5gl2cBT3zIiP72Li4nm3zQ1tLgHcYjOcSF X-Google-Smtp-Source: AJdET5d85/kKKaFYaLhImJjYBYk9MRZL4sPDLi6HXhvQrZES3NolrmKbDI1dF1lecQACdcv8IZzx1+SOwdGAmHpCxOk= X-Received: by 2002:a6b:cf18:: with SMTP id o24-v6mr9957677ioa.245.1541523510292; Tue, 06 Nov 2018 08:58:30 -0800 (PST) MIME-Version: 1.0 References: <1952737.4WAli8B44Z@photon.int.bluestop.org> In-Reply-To: From: Zaphod Beeblebrox Date: Tue, 6 Nov 2018 11:58:18 -0500 Message-ID: Subject: Re: Threadripper Thermal strangeness. To: mjoras@freebsd.org Cc: rebecca@bluestop.org, FreeBSD Hackers X-Rspamd-Queue-Id: 4A0D58020A X-Spamd-Result: default: False [-4.45 / 200.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.993,0]; R_DKIM_ALLOW(-0.20)[gmail.com]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; IP_SCORE(-0.54)[ipnet: 2607:f8b0::/32(-1.24), asn: 15169(-1.39), country: US(-0.08)]; NEURAL_HAM_LONG(-1.00)[-0.998,0]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[3.3.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.91)[-0.906,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Server: mx1.freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2018 16:58:32 -0000 1. Wouldn't it make sense to integrate this information in amdtemp.ko? 2. isn't sensor offset an item in the sysctl output of amdtemp.ko ... and it's value is 0 for me. On Mon, Nov 5, 2018 at 6:45 PM Matt Joras wrote: > You have to use the sensor_offset to get the actual value at the die. > I believe for all Threadripper chips the offset is -27C. > > Matt > On Mon, Nov 5, 2018 at 3:14 PM Rebecca Cran via freebsd-hackers > wrote: > > > > On Monday, 5 November 2018 14:23:22 MST Zaphod Beeblebrox wrote: > > > > > kldloading "amdtemp.ko" reports an idle temp of 55C or so and heavy > > > compiling (make world -j32) reports temperatures as high as 95C ... > upto > > > and including some instability and/or crash or rebooting. > > > > I have a new Threadripper 2990WX in an ASUS motherboard - and mine just > > reports ridiculous values: > > > > % sysctl dev.amdtemp | grep sensor > > dev.amdtemp.3.core0.sensor0: 112.6C > > dev.amdtemp.3.sensor_offset: 0 > > dev.amdtemp.2.core0.sensor0: 113.5C > > dev.amdtemp.2.sensor_offset: 0 > > dev.amdtemp.1.core0.sensor0: 111.7C > > dev.amdtemp.1.sensor_offset: 0 > > dev.amdtemp.0.core0.sensor0: 114.5C > > dev.amdtemp.0.sensor_offset: 0 > > > > -- > > Rebecca > > > > > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to " > freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@freebsd.org Tue Nov 6 17:32:32 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 72095112AFF4 for ; Tue, 6 Nov 2018 17:32:32 +0000 (UTC) (envelope-from rebecca@bluestop.org) Received: from muon.bluestop.org (muon.bluestop.org [96.73.9.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A50DC827EE; Tue, 6 Nov 2018 17:32:31 +0000 (UTC) (envelope-from rebecca@bluestop.org) Received: from muon.bluestop.org (localhost [127.0.0.1]) by muon.bluestop.org (Postfix) with ESMTP id 62C1A353BF; Tue, 6 Nov 2018 10:33:11 -0700 (MST) Received: from muon.bluestop.org ([127.0.0.1]) by muon.bluestop.org (muon.bluestop.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aP9I8N77EXl5; Tue, 6 Nov 2018 10:33:11 -0700 (MST) Received: from photon.int.bluestop.org (gw.bluestop.org [96.73.9.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by muon.bluestop.org (Postfix) with ESMTPSA; Tue, 6 Nov 2018 10:33:11 -0700 (MST) From: Rebecca Cran To: Zaphod Beeblebrox Cc: mjoras@freebsd.org, FreeBSD Hackers Subject: Re: Threadripper Thermal strangeness. Date: Tue, 06 Nov 2018 10:32:29 -0700 Message-ID: <1851518.EnoYUHA41c@photon.int.bluestop.org> In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Rspamd-Queue-Id: A50DC827EE X-Spamd-Result: default: False [-4.75 / 200.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[bluestop.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; TO_DN_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.986,0]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bluestop.org:+]; DMARC_POLICY_ALLOW(-0.50)[bluestop.org,quarantine]; MX_GOOD(-0.01)[cached: mail.bluestop.org]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; CTE_CASE(0.50)[]; ASN(0.00)[asn:7922, ipnet:96.64.0.0/11, country:US]; IP_SCORE(-1.25)[ipnet: 96.64.0.0/11(-4.87), asn: 7922(-1.30), country: US(-0.08)] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2018 17:32:32 -0000 On Tuesday, 6 November 2018 09:58:18 MST Zaphod Beeblebrox wrote: > 1. Wouldn't it make sense to integrate this information in amdtemp.ko? > 2. isn't sensor offset an item in the sysctl output of amdtemp.ko ... and > it's value is 0 for me. I think it would make sense: it looks like amdtemp is fairly basic at the moment, and the sensor offset is supposed to be set manually. >From amdtemp(4): SYSCTL VARIABLES The following variable is available as both sysctl(8) variable and loader(8) tunable: dev.amdtemp.%d.sensor_offset Add the given offset to the temperature of the sensor. Default is 0. -- Rebecca From owner-freebsd-hackers@freebsd.org Thu Nov 8 10:45:11 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D5BF1125127; Thu, 8 Nov 2018 10:45:11 +0000 (UTC) (envelope-from rajfbsd@gmail.com) Received: from mail-wr1-x443.google.com (mail-wr1-x443.google.com [IPv6:2a00:1450:4864:20::443]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A782C7807D; Thu, 8 Nov 2018 10:45:10 +0000 (UTC) (envelope-from rajfbsd@gmail.com) Received: by mail-wr1-x443.google.com with SMTP id z13-v6so18325161wrs.3; Thu, 08 Nov 2018 02:45:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=HvOtueL2/aekfryQWtOeFMaiUjkcdRffrBb3uGnrHPc=; b=jGZw1l6mXMekxd21yZ7SbjxE64qGHe7gjPPw/myrRDTn3xES1r8H0g8SsuMTYRkzaZ fG1NiJN9KfaxjNSiVSw9m9jd1X6GmKp790wLAR4sTSrN92ulvkfymu+spOdHp9WE3nB0 YIzaxCZOAHgldxTNfUKZTXen5fQGew57pJQH4g2xAMFCRegLnzQLogLygZE8ZuYSFuzG OEvwWRZSGK8c8AwzRItIqCuaPhAK+1hTy4/wLyvG7b47T0jT8EB1Vva6fJAZI/cNFwh1 xhi/LrSl34PLzazO9rMRcGhxAVBlsdd+MUTvDHCBDWUKvK5SmZr1QLOXSVJl+h56kUxy TLZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=HvOtueL2/aekfryQWtOeFMaiUjkcdRffrBb3uGnrHPc=; b=DeGCeqKTKItQkondLyV9jQ3s7LI1pR2VqmuGchl5q7NG62tIvmspGDu/r4ddnNDAsl rVxjYtpEmFf5NcuW12mufCenTb/o+xBwnzGf7B6EKznkSf98bbwpACzNGH3VGZJzuPyK DS3T0xm5uNBBu7/Sw2YRm+DAe8KPHe+pOTS/GmZPC632gApIDeNUxkOkoleafNqVuKDn FbsnmNDRU/fd6MD+eUdqhEkGnQHk/FVTIK6BR4tRs5XX2vtdQ38WJSciQqXKTPhmPOy0 BxE8i3H3Zx2VgZQ940Q3R8kMeni8GsTmk2XRFLyoc2lu/wohDkv2GSRVXFmMdEFVNjY8 N3gg== X-Gm-Message-State: AGRZ1gJkwCU/q0DIJFesTFmIdUYPHueMkFF+tnUwit9IYJs4kR41TBVM x9WZhsqygC3KlMd3ebpPadEDABDBnjHCFOikprDMyA== X-Google-Smtp-Source: AJdET5ezsVv+sHF5AEaP1U/2gBDNzGXTv9UcPk5n+CtV9LKAYIIMlfm12VwI4Ns6We28mnlw/6XRY3Iixaax9e8qgJg= X-Received: by 2002:adf:aa05:: with SMTP id p5-v6mr3930974wrd.56.1541673909279; Thu, 08 Nov 2018 02:45:09 -0800 (PST) MIME-Version: 1.0 From: Rajesh Kumar Date: Thu, 8 Nov 2018 16:14:57 +0530 Message-ID: Subject: Enabling a ACPI eMMC device is causing kernel reboots with FreeBSD 11.2 To: freebsd-amd64@freebsd.org, freebsd-acpi@freebsd.org, freebsd-hackers@freebsd.org X-Rspamd-Queue-Id: A782C7807D X-Spamd-Result: default: False [-4.53 / 200.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[gmail.com]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.998,0]; TO_MATCH_ENVRCPT_ALL(0.00)[]; IP_SCORE(-0.73)[ipnet: 2a00:1450::/32(-2.02), asn: 15169(-1.55), country: US(-0.09)]; DKIM_TRACE(0.00)[gmail.com:+]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCVD_IN_DNSWL_NONE(0.00)[3.4.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.80)[-0.795,0]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Server: mx1.freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2018 10:45:11 -0000 Hi, I have amd64 board, which has a eMMC device on the ACPI bus. When I try booting FreeBSD 11.2 on this board, with eMMC device disabled in BIOS, it boots without any issues. But, when eMMC device is enabled in BIOS, FreeBSD 11.2 continuously reboots. When I have the following tunable set, FreeBSD 11.2 boots without any issues with eMMC enabled. hw.pci.mcfg=0 Observations : a) We have verified the ACPI device addresses (from DSDT table) doesn't conflict with the PCI address base (configured in MCFG table). b) The PCI address base from MCFG table is read correctly by kernel (Verbose boot prints that address). So, I don't think MCFG table pointer is corrupted and getting read properly. Questions : a) What does hw.pci.mcfg=0 actually do to get the reboot issue resolved? b) Why enabling a device on ACPI bus needs hw.pci.mcfg (PCI specific) to boot? Looks like the system reboots when the ACPI and PCI devices enumeration starts during the boot. So, seems ACPI (or) PCI devices aren't getting enough resources and so reboots. Thanks, Rajesh. From owner-freebsd-hackers@freebsd.org Thu Nov 8 21:32:02 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 784641106D9B for ; Thu, 8 Nov 2018 21:32:02 +0000 (UTC) (envelope-from joerg@bec.de) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8BABA6DF6B for ; Thu, 8 Nov 2018 21:32:01 +0000 (UTC) (envelope-from joerg@bec.de) X-Originating-IP: 93.205.171.92 Received: from britannica.bec.de (p5DCDAB5C.dip0.t-ipconnect.de [93.205.171.92]) (Authenticated sender: joerg@bec.de) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 28109FF808 for ; Thu, 8 Nov 2018 21:31:53 +0000 (UTC) Date: Thu, 8 Nov 2018 22:31:52 +0100 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org Subject: Re: help with dl module and clang Message-ID: <20181108213152.GA22982@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org References: <4B8101F4-9A89-4486-8DF3-93DC799EF6D4@cs.huji.ac.il> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.9.5 (2018-04-13) X-Rspamd-Queue-Id: 8BABA6DF6B X-Spamd-Result: default: False [-2.25 / 200.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_XOIP(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.96)[-0.964,0]; FROM_EQ_ENVFROM(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_ONE(0.00)[1]; AUTH_NA(1.00)[]; DMARC_NA(0.00)[bec.de]; MX_GOOD(-0.01)[spool.mail.gandi.net]; NEURAL_HAM_SHORT(-0.25)[-0.248,0]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; R_SPF_NA(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[199.183.70.217.list.dnswl.org : 127.0.5.1]; R_DKIM_NA(0.00)[]; IP_SCORE(-0.83)[ipnet: 217.70.176.0/20(-2.07), asn: 29169(-2.04), country: FR(-0.02)]; ASN(0.00)[asn:29169, ipnet:217.70.176.0/20, country:FR]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[92.171.205.93.zen.spamhaus.org : 127.0.0.10] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2018 21:32:02 -0000 On Sat, Nov 03, 2018 at 04:47:37PM +0100, Dimitry Andric wrote: > On 3 Nov 2018, at 15:51, Daniel Braniss wrote: > > > > I have a program that loads some modules via dlopen(), these modules call some routines > > which are in the main program, this works when using gcc, but with cc it does not. > > > > when compiling the main program I use -export-dynamic, and the modules link fine when compiled with > > gcc, but when compiling with clang/cc i get dlerror: ...Undefined symbol … > > BTW, when linking the main program with cc I get > > /usr/bin/ld: warning: cannot find entry symbol xport-dynamic; defaulting to 0000000000402140 > > Instead of using -export-dynamic (which is a linker flag) as a flag to > cc, try using -Wl,-export-dynamic instead. Now, the linker interprets > this as the -e flag, which is something totally different. Or cc -rdynamic. Joerg