From owner-freebsd-arm@freebsd.org Mon Oct 22 15:25:29 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 05868FFFF0E for ; Mon, 22 Oct 2018 15:25:29 +0000 (UTC) (envelope-from the.lists@mgm51.com) Received: from oneyou.mgm51.net (oneyou.mgm51.net [IPv6:2607:f2f8:af30::100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "oneyou.mgm51.net", Issuer "RapidSSL RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8BFD97F59C for ; Mon, 22 Oct 2018 15:25:28 +0000 (UTC) (envelope-from the.lists@mgm51.com) Received: from sentry.24cl.com (sentry.24cl.com [IPv6:2001:558:6017:94:c582:1d99:a986:7609]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "sentry.24cl.com", Issuer "Mike's Certificate Authority" (verified OK)) by oneyou.mgm51.net (Postfix) with ESMTPS id 42f0hQ59SczZDyd for ; Mon, 22 Oct 2018 11:25:26 -0400 (EDT) Received: from [IPv6:fdcf:b715:2f4d:1:4035:9c79:a8d7:c4a7] (unknown [IPv6:fdcf:b715:2f4d:1:4035:9c79:a8d7:c4a7]) by sentry.24cl.com (Postfix) with ESMTP id 42f0hP5G3Qz1XwJ for ; Mon, 22 Oct 2018 11:25:25 -0400 (EDT) To: freebsd-arm@freebsd.org From: Mike Subject: 12.0 BETA1 - pkg command error Message-ID: <64148527-b8ca-deff-6042-483fb60f9f2d@mgm51.com> Date: Mon, 22 Oct 2018 11:25:23 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2018 15:25:29 -0000 I've installed the 12.0 BETA1 on a RPi3. When I try to use the pkg command, I get this error: The package management tool is not yet installed on your system. Do you want to fetch and install it now? [y/N]: y Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:12:aarch64/latest, please wait... Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done Installing pkg-1.10.5_3... Extracting pkg-1.10.5_3: 100% ld-elf.so.1: Shared object "libssl.so.8" not found, required by "pkg" How can I work around it? From owner-freebsd-arm@freebsd.org Mon Oct 22 15:30:28 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 470F0100842B for ; Mon, 22 Oct 2018 15:30:28 +0000 (UTC) (envelope-from hyun@caffeinated.codes) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E8F447FA1D for ; Mon, 22 Oct 2018 15:30:27 +0000 (UTC) (envelope-from hyun@caffeinated.codes) Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id F3AE521FC9; Mon, 22 Oct 2018 11:30:26 -0400 (EDT) Received: from web6 ([10.202.2.216]) by compute7.internal (MEProxy); Mon, 22 Oct 2018 11:30:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= caffeinated.codes; h=message-id:from:to:cc:mime-version :content-transfer-encoding:content-type:subject:in-reply-to:date :references; s=fm3; bh=5ilJCHcychXZHUiyUUtswD/NFyxA8eDtbSc0poFOi Ic=; b=LYyrfuda5GIWCuH9pisTphqQHJskGUM3jdWq+VVoyo1WUxefIYTp9JgzX Nr8MUWYSgeghhokyNweraWbt5aEWCUbMwyB9NjV/J5yEHLBKn2gVw/MP14bShSVp kipMJ7dvM0AusCEomXgvo4Bz+dOnHog2cDHG2uU91yLVbOuPkTVBYFqZlGLQLUt9 gQ01uEKkq2RernS6yTuir0Bhzd0SbCQVnULCJCdW9Z444h+07UpuuOgsoRuHLxmP 4TWsXQlxpOfkezAuPXpMUJEx7DZ5WHgjFLEKf3Z9SkYHiemBBTF1VuQrVj5pv0pr tfOclXokeP1pOz9ZJYSyrTQvBKrgg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=5ilJCHcychXZHUiyUUtswD/NFyxA8eDtbSc0poFOi Ic=; b=ufwdRkhNlmOnFvMGPgR+VcxBRSkz8mEohfeefJPWf4c8LqcOGFPT3fbIs BtwYdpSVKdhDVGhPZZyZWp5U4XxIXIOOIIGvxrhPafg8ibWED2VRIxOf+8RzNBxQ WuIb913nw+l7/w/9irjuaZabWX2RN1tCqUDWbojT2nb4miNch1uJ15o4t0qw2Dmv dT4KI7MtBB9DQEF6xaEhm+VJYX5L+HcxmlG5xyU4fnx/UtuiM3lZOVtqD/zFNQp2 ptO9fuGqdQn4Bt0JDgqmTfFKOsQFi/vThx8iT8Zv38S2/ypL93bc/NBHb07Nsx3Z 17OCZ5Thm8rCjgZQkU1YuIEWz88WQ== X-ME-Sender: X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 99) id D83C64274; Mon, 22 Oct 2018 11:30:25 -0400 (EDT) Message-Id: <1540222225.2684343.1550519584.308D8BA8@webmail.messagingengine.com> From: Hyun Hwang To: Mike Cc: freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="utf-8" X-Mailer: MessagingEngine.com Webmail Interface - ajax-b315a288 Subject: Re: 12.0 BETA1 - pkg command error In-Reply-To: <64148527-b8ca-deff-6042-483fb60f9f2d@mgm51.com> Date: Mon, 22 Oct 2018 11:30:25 -0400 References: <64148527-b8ca-deff-6042-483fb60f9f2d@mgm51.com> X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2018 15:30:28 -0000 On Monday, October 22, 2018, 11:25 AM (UTC-0400), "Mike via freebsd-arm" wrote: > > I've installed the 12.0 BETA1 on a RPi3. > > When I try to use the pkg command, I get this error: > > > The package management tool is not yet installed on your system. > Do you want to fetch and install it now? [y/N]: y > Bootstrapping pkg from > pkg+http://pkg.FreeBSD.org/FreeBSD:12:aarch64/latest, please wait... > Verifying signature with trusted certificate > pkg.freebsd.org.2013102301... done > Installing pkg-1.10.5_3... > Extracting pkg-1.10.5_3: 100% > ld-elf.so.1: Shared object "libssl.so.8" not found, required by "pkg" > > > > How can I work around it? > _______________________________________________ > freebsd-arm@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arm > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" Hi, I have experienced this too on RPi 2. When I manually compiled pkg out of ports-mgmt/pkg, the error went gone. Probably still some fallout after OpenSSL 1.1.1. So yeah, try to build one yourself instead of getting the prebuilt binary. -- Hyun Hwang From owner-freebsd-arm@freebsd.org Mon Oct 22 17:05:01 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 146F3103965A for ; Mon, 22 Oct 2018 17:05:00 +0000 (UTC) (envelope-from the.lists@mgm51.com) Received: from oneyou.mgm51.net (oneyou.mgm51.net [IPv6:2607:f2f8:af30::100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "oneyou.mgm51.net", Issuer "RapidSSL RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 04B8F85782 for ; Mon, 22 Oct 2018 17:05:00 +0000 (UTC) (envelope-from the.lists@mgm51.com) Received: from sentry.24cl.com (sentry.24cl.com [IPv6:2001:558:6017:94:c582:1d99:a986:7609]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "sentry.24cl.com", Issuer "Mike's Certificate Authority" (verified OK)) by oneyou.mgm51.net (Postfix) with ESMTPS id 42f2v76w7MzZDyk; Mon, 22 Oct 2018 13:04:51 -0400 (EDT) Received: from [IPv6:fdcf:b715:2f4d:1:4035:9c79:a8d7:c4a7] (unknown [IPv6:fdcf:b715:2f4d:1:4035:9c79:a8d7:c4a7]) by sentry.24cl.com (Postfix) with ESMTP id 42f2v665MHz1XwJ; Mon, 22 Oct 2018 13:04:50 -0400 (EDT) Subject: Re: 12.0 BETA1 - pkg command error To: Hyun Hwang Cc: freebsd-arm@freebsd.org References: <64148527-b8ca-deff-6042-483fb60f9f2d@mgm51.com> <1540222225.2684343.1550519584.308D8BA8@webmail.messagingengine.com> From: Mike Message-ID: Date: Mon, 22 Oct 2018 13:04:48 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <1540222225.2684343.1550519584.308D8BA8@webmail.messagingengine.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2018 17:05:01 -0000 On 10/22/2018 11:30 AM, Hyun Hwang wrote: > On Monday, October 22, 2018, 11:25 AM (UTC-0400), "Mike via freebsd-arm" wrote: >> >> I've installed the 12.0 BETA1 on a RPi3. >> >> When I try to use the pkg command, I get this error: >> >> >> The package management tool is not yet installed on your system. >> Do you want to fetch and install it now? [y/N]: y >> Bootstrapping pkg from >> pkg+http://pkg.FreeBSD.org/FreeBSD:12:aarch64/latest, please wait... >> Verifying signature with trusted certificate >> pkg.freebsd.org.2013102301... done >> Installing pkg-1.10.5_3... >> Extracting pkg-1.10.5_3: 100% >> ld-elf.so.1: Shared object "libssl.so.8" not found, required by "pkg" >> >> >> >> How can I work around it? >> _______________________________________________ >> freebsd-arm@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-arm >> To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" > > Hi, > > I have experienced this too on RPi 2. When I manually compiled pkg out of ports-mgmt/pkg, the error went gone. Probably still some fallout after OpenSSL 1.1.1. > > So yeah, try to build one yourself instead of getting the prebuilt binary. > That seems to work. Thanks! From owner-freebsd-arm@freebsd.org Mon Oct 22 17:11:13 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0DDF11039E0D for ; Mon, 22 Oct 2018 17:11:13 +0000 (UTC) (envelope-from rj@obsigna.com) Received: from mail.cyclaero.com (ec2-18-195-62-44.eu-central-1.compute.amazonaws.com [18.195.62.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A586185E18 for ; Mon, 22 Oct 2018 17:11:12 +0000 (UTC) (envelope-from rj@obsigna.com) Received: from mail.obsigna.com (unknown [189.29.185.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.cyclaero.com (Postfix) with ESMTPSA id 9ADB167; Mon, 22 Oct 2018 19:07:57 +0200 (CEST) Received: from rolf-mini.obsigna.com (rolf-mini.obsigna.com [192.168.222.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.obsigna.com (Postfix) with ESMTPSA id 883081350FA79; Mon, 22 Oct 2018 14:07:53 -0300 (-03) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: 12.0 BETA1 - pkg command error From: "Dr. Rolf Jansen" In-Reply-To: <1540222225.2684343.1550519584.308D8BA8@webmail.messagingengine.com> Date: Mon, 22 Oct 2018 14:07:52 -0300 Cc: Mike , freebsd-arm@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <93184F28-95A9-424C-8F34-3E6973E89A62@obsigna.com> References: <64148527-b8ca-deff-6042-483fb60f9f2d@mgm51.com> <1540222225.2684343.1550519584.308D8BA8@webmail.messagingengine.com> To: Hyun Hwang X-Mailer: Apple Mail (2.3445.9.1) X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2018 17:11:13 -0000 > Am 22.10.2018 um 12:30 schrieb Hyun Hwang : >=20 > On Monday, October 22, 2018, 11:25 AM (UTC-0400), "Mike via = freebsd-arm" wrote: >>=20 >> I've installed the 12.0 BETA1 on a RPi3. >>=20 >> When I try to use the pkg command, I get this error: >>=20 >>=20 >> The package management tool is not yet installed on your system. >> Do you want to fetch and install it now? [y/N]: y >> Bootstrapping pkg from >> pkg+http://pkg.FreeBSD.org/FreeBSD:12:aarch64/latest, please wait... >> Verifying signature with trusted certificate >> pkg.freebsd.org.2013102301... done >> Installing pkg-1.10.5_3... >> Extracting pkg-1.10.5_3: 100% >> ld-elf.so.1: Shared object "libssl.so.8" not found, required by "pkg" >>=20 >> How can I work around it? >=20 > Hi, >=20 > I have experienced this too on RPi 2. When I manually compiled pkg out = of ports-mgmt/pkg, the error went gone. Probably still some fallout = after OpenSSL 1.1.1. >=20 > So yeah, try to build one yourself instead of getting the prebuilt = binary. > --=20 > Hyun Hwang In addition, note, that this does not only affect pkg. All ports and = other 3rdparty software which somehow depend on OpenSSL need to be = recompiled. In order to know which ports/packages are affected, I issued the = following command. After some minutes this would spill out a list of = packages which must be re-installed: find /usr/local/bin /usr/local/sbin /usr/local/libexec /usr/local/lib = -type f | xargs -n1 file -F =E2=80=9A ' | grep ELF | cut -f1 -d' ' | = xargs ldd -f '%A %o\n' | grep "libssl.so\|libcrypto.so" | cut -f1 -d' ' = | sort -u | xargs -n1 pkg which -p | cut -f6 -d' ' | sort -u Best regards Rolf From owner-freebsd-arm@freebsd.org Mon Oct 22 19:46:19 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2FCD105EC79 for ; Mon, 22 Oct 2018 19:46:19 +0000 (UTC) (envelope-from freebsd.asc@strcmp.org) Received: from onager.schwarzes.net (onager.schwarzes.net [IPv6:2a03:4000:8:2bb::5d22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2F5B08F4A1 for ; Mon, 22 Oct 2018 19:46:18 +0000 (UTC) (envelope-from freebsd.asc@strcmp.org) Received: from [172.30.250.35] (x4dba4b26.dyn.telefonica.de [77.186.75.38]) (authenticated bits=0) by onager.schwarzes.net (8.15.2/8.15.2) with ESMTPA id w9MJkEZl077639 for ; Mon, 22 Oct 2018 21:46:15 +0200 (CEST) (envelope-from freebsd.asc@strcmp.org) From: Andreas Schwarz To: freebsd-arm@FreeBSD.org Mail-Reply-To: Andreas Schwarz Mail-Followup-To: freebsd-arm@FreeBSD.org Date: Mon, 22 Oct 2018 21:46:14 +0200 (CEST) Message-ID: <4cc206182ca.7dd26cb4@mail.schwarzes.net> In-Reply-To: <64148527-b8ca-deff-6042-483fb60f9f2d@mgm51.com> References: <64148527-b8ca-deff-6042-483fb60f9f2d@mgm51.com> User-Agent: YAM/2.9p1 (MorphOS; PPC; rv:20140418r7798) Subject: Re: 12.0 BETA1 - pkg command error MIME-Version: 1.0 Content-Type: text/plain X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (onager.schwarzes.net [37.221.194.76]); Mon, 22 Oct 2018 21:46:15 +0200 (CEST) X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2018 19:46:19 -0000 On 22.10.18, Mike via freebsd-arm wrote: > > I've installed the 12.0 BETA1 on a RPi3. > > When I try to use the pkg command, I get this error: > > > The package management tool is not yet installed on your system. > Do you want to fetch and install it now? [y/N]: y > Bootstrapping pkg from > pkg+http://pkg.FreeBSD.org/FreeBSD:12:aarch64/latest, please wait... > Verifying signature with trusted certificate > pkg.freebsd.org.2013102301... done > Installing pkg-1.10.5_3... > Extracting pkg-1.10.5_3: 100% > ld-elf.so.1: Shared object "libssl.so.8" not found, required by "pkg" You can use the static linked version "pkg-static" instead (which is also part of the pkg package). -asc From owner-freebsd-arm@freebsd.org Tue Oct 23 15:46:16 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B157FFEA759 for ; Tue, 23 Oct 2018 15:46:16 +0000 (UTC) (envelope-from greg@unrelenting.technology) Received: from out.migadu.com (out.migadu.com [91.121.223.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.migadu.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 309D782EA2 for ; Tue, 23 Oct 2018 15:46:15 +0000 (UTC) (envelope-from greg@unrelenting.technology) Received: (Migadu outbound); Tue, 23 Oct 2018 15:46:07 +0000 Authentication-Results: out.migadu.com; auth=pass (plain) Received: from [192.168.1.141] ([62.122.208.146]) by out.migadu.com (Haraka/2.8.16) with ESMTPSA id A944EB7C-E6C0-41C4-8E68-13F9CB93A68F.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 verify=FAIL); Tue, 23 Oct 2018 15:46:07 +0000 Date: Tue, 23 Oct 2018 18:46:05 +0300 From: Greg V Subject: Re: Poor virtio performance on Scaleway ARM systems To: Peter Jeremy Cc: freebsd-arm@freebsd.org Message-Id: <1540309565.1771.1@smtp.migadu.com> In-Reply-To: <20180916184657.GB24416@server.rulingia.com> References: <20180916184657.GB24416@server.rulingia.com> X-Mailer: geary/0.12.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed DKIM-Signature: v=1; a=rsa-sha256; bh=wcIJGX08xd7okFOrML54+qceg3m1WIv0JTICn8cfBgQ=; c=relaxed/simple; d=unrelenting.technology; h=from:subject:date:to; s=default; b=TkUuif791mDcdEsrF6VU5QlBxNmTVeOC3X+t5bymb3j32dz6vjSsIHAgpeBruSAmRmI21yIX9JCThQiuw0SzY3NorDSWMMdDfwHPFfIbsIP9bwdSHEwqmLCeQOfnMaG1a1pAee52Spc6cv3VrFejegbfE1WXx7lCTjupiLnwmBk= X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2018 15:46:16 -0000 On Sun, Sep 16, 2018 at 9:46 PM, Peter Jeremy wrote: > I have been playing with the 4-core ARM64 VPSs on > https://www.scaleway.com > and notice that the disk I/O performance (using virtio_blk) is > abyssmal. > Using "dd if=/dev/{vda|vtbd0} of=/dev/null bs=256k count=4096", I get > 400-500MBps, whilst under FreeBSD-12, I get about 5MBps. I've > checked on a > couple of instances and both Linux & FreeBSD on the same instance and > get > similar results. Linux & FreeBSD are both using a virtio block device > attached to the PCI bus. Rebuilding FreeBSD to turn off all the > debugging > has no effect. > > The only oddity I've found is that FreeBSD is reporting very high > interrupt > rates on gic0,p11, gic0,s4 and gic0,s5 whilst disk I/O is occurring. > Unfortunately, I can't tell what is attached to those interrupts (it's > not obvious from the dmesg and reported as "+"). gic0,s4 correlates to network activity. gic0,s5 correlates to disk activity. > I've done some searching and have only found general FUD ("FreeBSD > virtio > isn't any good") and nothing specifically related to Scaleway. > > Can anyone suggest where to look for a solution? I FIGURED IT OUT!! echo 'hw.pci.honor_msi_blacklist="0"' >> /boot/loader.conf SOMEHOW FreeBSD ended up using legacy interrupts instead of MSI-X. I have no idea how. Looking at pci_quirks, MSI disable is for old Intel stuff, some old VMWare stuff... NOTHING about QEMU/KVM or Cavium! With the fix, we're getting ~1000 interrupts on 'its0,4: virtio_pci1' (instead of ~30000 on gic), latency is no longer awful, and bandwidth is about 220 MB/s. YAY!! From owner-freebsd-arm@freebsd.org Tue Oct 23 17:04:07 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62136FECA1A for ; Tue, 23 Oct 2018 17:04:07 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0DC1285B5C for ; Tue, 23 Oct 2018 17:04:07 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro-2.local (ralph.baldwin.cx [66.234.199.215]) by mail.baldwin.cx (Postfix) with ESMTPSA id C4D6210B429; Tue, 23 Oct 2018 13:04:05 -0400 (EDT) Subject: Re: Poor virtio performance on Scaleway ARM systems To: Greg V , Peter Jeremy References: <20180916184657.GB24416@server.rulingia.com> <1540309565.1771.1@smtp.migadu.com> Cc: freebsd-arm@freebsd.org From: John Baldwin Message-ID: <966ed99b-4709-5693-dca2-77e3e6d7d830@FreeBSD.org> Date: Tue, 23 Oct 2018 10:04:04 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <1540309565.1771.1@smtp.migadu.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Tue, 23 Oct 2018 13:04:06 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2018 17:04:07 -0000 On 10/23/18 8:46 AM, Greg V wrote: > > > On Sun, Sep 16, 2018 at 9:46 PM, Peter Jeremy > wrote: >> I have been playing with the 4-core ARM64 VPSs on >> https://www.scaleway.com >> and notice that the disk I/O performance (using virtio_blk) is >> abyssmal. >> Using "dd if=/dev/{vda|vtbd0} of=/dev/null bs=256k count=4096", I get >> 400-500MBps, whilst under FreeBSD-12, I get about 5MBps. I've >> checked on a >> couple of instances and both Linux & FreeBSD on the same instance and >> get >> similar results. Linux & FreeBSD are both using a virtio block device >> attached to the PCI bus. Rebuilding FreeBSD to turn off all the >> debugging >> has no effect. >> >> The only oddity I've found is that FreeBSD is reporting very high >> interrupt >> rates on gic0,p11, gic0,s4 and gic0,s5 whilst disk I/O is occurring. >> Unfortunately, I can't tell what is attached to those interrupts (it's >> not obvious from the dmesg and reported as "+"). > gic0,s4 correlates to network activity. > gic0,s5 correlates to disk activity. > >> I've done some searching and have only found general FUD ("FreeBSD >> virtio >> isn't any good") and nothing specifically related to Scaleway. >> >> Can anyone suggest where to look for a solution? > > I FIGURED IT OUT!! > > echo 'hw.pci.honor_msi_blacklist="0"' >> /boot/loader.conf > > SOMEHOW FreeBSD ended up using legacy interrupts instead of MSI-X. > > I have no idea how. > Looking at pci_quirks, MSI disable is for old Intel stuff, some old > VMWare stuff... NOTHING about QEMU/KVM or Cavium! > > With the fix, we're getting ~1000 interrupts on 'its0,4: virtio_pci1' > (instead of ~30000 on gic), latency is no longer awful, and bandwidth > is about 220 MB/s. YAY!! Can you make 'pciconf -lc' output available? There is a "generic" blacklist that assumes we will see a host bridge device with PCI-e or PCI-X and if neither of those is present we also blacklist MSI. However, we can certainly figure out how to make this work out of the box. -- John Baldwin From owner-freebsd-arm@freebsd.org Tue Oct 23 18:50:15 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62191FF1294 for ; Tue, 23 Oct 2018 18:50:15 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vtr.rulingia.com (vtr.rulingia.com [IPv6:2001:19f0:5801:ebe:5400:1ff:fe53:30fd]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vtr.rulingia.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A3E6C8B2B4; Tue, 23 Oct 2018 18:50:14 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from server.rulingia.com (ppp59-167-167-3.static.internode.on.net [59.167.167.3]) by vtr.rulingia.com (8.15.2/8.15.2) with ESMTPS id w9NInvko050198 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Oct 2018 05:50:04 +1100 (AEDT) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.15.2/8.15.2) with ESMTPS id w9NInnX8010795 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 24 Oct 2018 05:49:49 +1100 (AEDT) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.15.2/8.15.2/Submit) id w9NInn4e010794; Wed, 24 Oct 2018 05:49:49 +1100 (AEDT) (envelope-from peter) Date: Wed, 24 Oct 2018 05:49:49 +1100 From: Peter Jeremy To: John Baldwin Cc: Greg V , freebsd-arm@freebsd.org Subject: Re: Poor virtio performance on Scaleway ARM systems Message-ID: <20181023184949.GB20902@server.rulingia.com> References: <20180916184657.GB24416@server.rulingia.com> <1540309565.1771.1@smtp.migadu.com> <966ed99b-4709-5693-dca2-77e3e6d7d830@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="W/nzBZO5zC0uMSeA" Content-Disposition: inline In-Reply-To: <966ed99b-4709-5693-dca2-77e3e6d7d830@FreeBSD.org> X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.10.0 (2018-05-17) X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2018 18:50:15 -0000 --W/nzBZO5zC0uMSeA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2018-Oct-23 10:04:04 -0700, John Baldwin wrote: >On 10/23/18 8:46 AM, Greg V wrote: >> On Sun, Sep 16, 2018 at 9:46 PM, Peter Jeremy =20 >> wrote: >>> I have been playing with the 4-core ARM64 VPSs on=20 >>> https://www.scaleway.com >>> and notice that the disk I/O performance (using virtio_blk) is=20 >>> abyssmal. >>> >>> The only oddity I've found is that FreeBSD is reporting very high=20 >>> interrupt >>> rates on gic0,p11, gic0,s4 and gic0,s5 whilst disk I/O is occurring. >>> Unfortunately, I can't tell what is attached to those interrupts (it's >>> not obvious from the dmesg and reported as "+"). >> gic0,s4 correlates to network activity. >> gic0,s5 correlates to disk activity. >>=20 >> echo 'hw.pci.honor_msi_blacklist=3D"0"' >> /boot/loader.conf >Can you make 'pciconf -lc' output available? There is a "generic" >blacklist that assumes we will see a host bridge device with PCI-e >or PCI-X and if neither of those is present we also blacklist MSI. >However, we can certainly figure out how to make this work out of >the box. $ sudo pciconf -lc hostb0@pci0:0:0:0: class=3D0x060000 card=3D0x11001af4 chip=3D0x00081b3= 6 rev=3D0x00 hdr=3D0x00 virtio_pci0@pci0:0:1:0: class=3D0x020000 card=3D0x00011af4 chip=3D0x10001af= 4 rev=3D0x00 hdr=3D0x00 cap 11[98] =3D MSI-X supports 3 messages Table in map 0x14[0x0], PBA in map 0x14[0x800] cap 09[84] =3D vendor (length 20) cap 09[70] =3D vendor (length 20) cap 09[60] =3D vendor (length 16) cap 09[50] =3D vendor (length 16) cap 09[40] =3D vendor (length 16) virtio_pci1@pci0:0:2:0: class=3D0x010000 card=3D0x00021af4 chip=3D0x10011af= 4 rev=3D0x00 hdr=3D0x00 cap 11[98] =3D MSI-X supports 2 messages Table in map 0x14[0x0], PBA in map 0x14[0x800] cap 09[84] =3D vendor (length 20) cap 09[70] =3D vendor (length 20) cap 09[60] =3D vendor (length 16) cap 09[50] =3D vendor (length 16) cap 09[40] =3D vendor (length 16) --=20 Peter Jeremy --W/nzBZO5zC0uMSeA Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEE7rKYbDBnHnTmXCJ+FqWXoOSiCzQFAlvPbU1fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEVF QjI5ODZDMzA2NzFFNzRFNjVDMjI3RTE2QTU5N0EwRTRBMjBCMzQACgkQFqWXoOSi CzQydw/9ElOshtL2CBAxaQzzGtBFLfiCusmGc3PYZG3S8GoTxc9ylylYN/rbqzEs P9CRKfJJmXhxDsphH0neKbxH+L/uSpU6+RCB21zKRRJpf5w4swg8i8dZ6bi5RTlC Xm1A7JBCOV+3ZxtJ0r7qGXKf0G+5TZhvpzoEpfkIyyCROGf48jpwPGOMQcHjGLip V4guNHBis79n1FAY0iSsdC00WGQkwyU0H95nssMF1byrBdUklmdzsON5JF+0U6i1 I9lijJ0G+I96E+UG6hMilQ+gIxgs2DgmHUG9w1P8x1fTNTYZvrFHXn9Ie5ZlbOo/ ShbONpe+rVVejl5OeVeQgKUSuVAocux4dZiEPHVqKO3I/WmWBrjBpTYbV02/WbXS Ec767hYA4VQi7ReKGQqxdVmS7DqdPLkpjcMghWwT93TTQ78PkbtVf8jkczkiKTFN kT1HyXCqDK9TaJGoVpUhE7fLBOjLwng3BTr+WYUJ3lE+xUCwalmPIzhKJf9p0Rwz qvBiLjkrWyut1a9A7A3JOekOwH+6qdBe+mEw+7N0wkoj5egkXqChcmLUJGQJx9bS Sai9u48zUOj83Pl7nQxbFkr+J2KFFLMufMGlDTTWOPnpTT79Pm0exkPb+Wrg/GLb S+ZDcpbpK+KclqbNtZO/jlfEufV34NY5bucH6lihvENXOo0Ds8s= =oRM+ -----END PGP SIGNATURE----- --W/nzBZO5zC0uMSeA-- From owner-freebsd-arm@freebsd.org Thu Oct 25 13:13:44 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 302C71075922; Thu, 25 Oct 2018 13:13:44 +0000 (UTC) (envelope-from ronald@klop.ws) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CAE3781618; Thu, 25 Oct 2018 13:13:43 +0000 (UTC) (envelope-from ronald@klop.ws) Received: from smtp.greenhost.nl ([213.108.110.112]) by smarthost1.greenhost.nl with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1gFfSB-0007BD-L9; Thu, 25 Oct 2018 15:13:35 +0200 Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: freebsd-arm@freebsd.org, "FreeBSD Current" , freebsd-fs@freebsd.org Date: Thu, 25 Oct 2018 15:13:36 +0200 Subject: tmpfs corrupt file (aarch64/12.0-ALPHA5) MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Ronald Klop" Message-ID: User-Agent: Opera Mail/12.16 (FreeBSD) X-Authenticated-As-Hash: bdb49c4ff80bd276e321aade33e76e02752072e2 X-Virus-Scanned: by clamav at smarthost1.samage.net X-Spam-Level: / X-Spam-Score: -0.2 X-Spam-Status: No, score=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 autolearn=disabled version=3.4.0 X-Scan-Signature: 76f3589a93270604ea078d468a2051b3 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2018 13:13:44 -0000 Hello, I'm running this small script which generates changelog file in gzipped JSON format. TMPDIR=/var/tmp START=1 END=$(( $START + $STEP - 1 )) while test $START -lt 300000 do svnlite log --xml -r $START:$END -v ~/src/freebsd-base > $TMPDIR/bla.xml bin/xml2json.py > $TMPDIR/commits.$START-$END.json gzip -f $TMPDIR/commits.$START-$END.json START=$(( $START + $STEP )) END=$(( $START + $STEP - 1 )) done So $TMPDIR/bla.xml is overwritten every iteration. Well: - If $TMPDIR=/var/tmp which is on UFS the script works. - If %TMPDIR=/tmp which is tmpfs the xml2json.py script very often gives errors about the XML structure of bla.xml like if the file is truncated or empty. So it looks like something is happening on tmpfs that the data is not yet 'synced'. Can this be a threading/cpu-cache issue on aarch64? This is on an RPI3B+. FreeBSD rpi3 12.0-ALPHA5 FreeBSD 12.0-ALPHA5 #5 r338616: Wed Sep 12 22:23:05 CEST 2018 builder@rpi3:/data/src/obj-head/data/src/head/arm64.aarch64/sys/GENERIC-NODEBUG arm64 Can I do more testing or provide more information before creating a PR about this? Regards, Ronald. From owner-freebsd-arm@freebsd.org Thu Oct 25 14:36:23 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 580901078D95 for ; Thu, 25 Oct 2018 14:36:23 +0000 (UTC) (envelope-from sparvu@kronometrix.org) Received: from mail.kronometrix.org (mail.kronometrix.org [95.85.46.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.kronometrix.org", Issuer "mail.kronometrix.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id DA67485F4F for ; Thu, 25 Oct 2018 14:36:22 +0000 (UTC) (envelope-from sparvu@kronometrix.org) Received: from [192.168.1.101] (82-203-189-118.bb.dnainternet.fi [82.203.189.118]) (authenticated bits=0) by mail.kronometrix.org (8.15.2/8.15.2) with ESMTPSA id w9PE7vac011504 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Thu, 25 Oct 2018 14:07:58 GMT (envelope-from sparvu@kronometrix.org) X-Authentication-Warning: mail.kronometrix.org: Host 82-203-189-118.bb.dnainternet.fi [82.203.189.118] claimed to be [192.168.1.101] From: Stefan Parvu Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.0 \(3445.100.39\)) Subject: boot loader serial console speed to 9600 Message-Id: Date: Thu, 25 Oct 2018 17:07:51 +0300 To: freebsd-arm@freebsd.org X-Mailer: Apple Mail (2.3445.100.39) X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2018 14:36:23 -0000 Hi, Under RBPI3B+ FreeBSD 12 ALPHA8, I see the serial console is set to = 115200. Is this correct ? Shouldn't this be set to 9600 ? How can I change the serial console = speed to 9600 under loader.conf ?=20 Currently my loader.conf looks like this: autoboot_delay=3D0 hw.usb.template=3D3 umodem_load=3D"YES" # Multiple console (serial+efi gop) enabled. boot_multicons=3D"YES" boot_serial=3D"YES" # Disable the beastie menu and color beastie_disable=3D"YES" loader_color=3D"NO" mac_portacl_load=3D=E2=80=9CYES" I tried:=20 comconsole_speed=3D"9600" console=3D=E2=80=9Ccomconsole" no go. somehow it does stay set to 115200. Why is that ?=20 Thanks, Stefan= From owner-freebsd-arm@freebsd.org Thu Oct 25 15:25:22 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0DF8B107B155 for ; Thu, 25 Oct 2018 15:25:22 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7FD4889467 for ; Thu, 25 Oct 2018 15:25:21 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1540480151; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=WFETv8BcEc3uAERKb0rMDEKbZtuNFxi9O1shFQ4suXdPRJtFULcDgVmlya3BMoXf8dsGRzn1/0Ikb v8+RruKL2AafkwhB0wU0e9DT++3ZP7hc6frA0MrWucYI8u7cj2daJwStFM2JAj/3nT2I1LVqlroj6O j6VmZsXEpaFqcjeEgXCbbjj+zqsHw+QL2sgBOcf9f138lBqDPXTJx9sqJf9gLGXU92HdVVTPBDOBFP l76WLDGNIk5foB0SgOHygNXb3v8dNUW8i4v31hPeWWHqgQWOLtTDfR4N4+6cj1k7wuzARfIvz+ZeLz so32qQBSSg03J2CFe54QhVywXbbaGIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:to:from:subject:message-id:dkim-signature:from; bh=nEBmGJVVlEr1ND6jBWFv79jCb9xGkObbTozK5rb7MEc=; b=Q9ERZWeUkY73XLVdKCrAHgHbE27ltIEhPsTcUqqpXUouc8LIDoi4YKRFVkNgqtXlhYRJtAGl7QSRJ XKGiH+XtUFWOWi3b8gfHA8I12JVnBxpbzhDKEHmIMtsqMWIJ/Du6YgNaslCpFG4XKZ2UoptFcMRbNr ShXeZ4OIzW6K+JndL14LIg5CvHopNlLCmWSNuLSyRE6bDjR9hu/KvPr6EtIPN26c/B5GtraI/x5Sou KNoNegYG3uyMDu0/5v4aAZ3KdUMODeevMfSjhVYvHh7Jz0Hfy4R0eixz7aiLMvZiPUiS1/xOv9yX6J HdQejwEjvZFZBsEglR7Nd0GrWkI6org== ARC-Authentication-Results: i=1; outbound1.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:to:from:subject:message-id:from; bh=nEBmGJVVlEr1ND6jBWFv79jCb9xGkObbTozK5rb7MEc=; b=UXukQVGhFe/szOA9603BAnsXyJujkm7vFJTLig1LHS1Xed9a6t7k9BW5cRpQQqgyMnsnis0a47gOd nZmzlPJhgRLMbm9b49t3jm4QSPhChB8GCB5jhtz1GxIUosKhJpzP9rSJEcepW3vn+j7zGNVwVB/vjc imsObKzq9LoRTJQYBBy9rU2OyV4eSxj8ThKl4MECwzpx6gfIQ9RJ2X1xJjPr0WkaedOCGrXMRid3Cd +NK7OsHZdCyyTf+5CQpM4cyXU6hC4nBx2oxEleC0AWXojkgzPfpGoGe36ZPX2Q+l69QDNvXiQgQNHY LSVXO9y/hZmnqgnhJeKe8QL10ePrnZQ== X-MHO-RoutePath: aGlwcGll X-MHO-User: eadd2c41-d867-11e8-a93b-310333596487 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound1.ore.mailhop.org (Halon) with ESMTPSA id eadd2c41-d867-11e8-a93b-310333596487; Thu, 25 Oct 2018 15:09:07 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id w9PF961v052762; Thu, 25 Oct 2018 09:09:06 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1540480146.22340.79.camel@freebsd.org> Subject: Re: boot loader serial console speed to 9600 From: Ian Lepore To: Stefan Parvu , freebsd-arm@freebsd.org Date: Thu, 25 Oct 2018 09:09:06 -0600 In-Reply-To: References: Content-Type: text/plain; charset="iso-8859-13" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2018 15:25:22 -0000 On Thu, 2018-10-25 at 17:07 +0300, Stefan Parvu wrote: > Hi, > > Under RBPI3B+ FreeBSD 12 ALPHA8, I see the serial console is set to > 115200. Is this correct ? > Shouldn't this be set to 9600 ? How can I change the serial console > speed to 9600 under loader.conf ?  > > Currently my loader.conf looks like this: > > autoboot_delay=0 > hw.usb.template=3 > umodem_load="YES" > # Multiple console (serial+efi gop) enabled. > boot_multicons="YES" > boot_serial="YES" > # Disable the beastie menu and color > beastie_disable="YES" > loader_color="NO" > mac_portacl_load=´YES" > > I tried:  > > comconsole_speed="9600" > console=´comconsole" > > no go. somehow it does stay set to 115200. Why is that ?  > > Thanks, > Stefan The ability to change serial console speed from loader(8) only works with x86 BIOS calls (and maybe UEFI, not sure). In general, when uboot is involved, loader will use whatever serial speed was set up by uboot. Why do you want to set the console to such a slow speed? It actually pisses me off that the standard is as slow as 115200 in the 21st century; modern uart chips have no problem running at speeds in the 1- 12mbps range. -- Ian From owner-freebsd-arm@freebsd.org Thu Oct 25 15:28:48 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8ECC1107B2B5 for ; Thu, 25 Oct 2018 15:28:48 +0000 (UTC) (envelope-from sparvu@kronometrix.org) Received: from mail.kronometrix.org (mail.kronometrix.org [95.85.46.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.kronometrix.org", Issuer "mail.kronometrix.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1A5708957F for ; Thu, 25 Oct 2018 15:28:47 +0000 (UTC) (envelope-from sparvu@kronometrix.org) Received: from [192.168.1.101] (82-203-189-118.bb.dnainternet.fi [82.203.189.118]) (authenticated bits=0) by mail.kronometrix.org (8.15.2/8.15.2) with ESMTPSA id w9PFSjt5012795 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Thu, 25 Oct 2018 15:28:46 GMT (envelope-from sparvu@kronometrix.org) X-Authentication-Warning: mail.kronometrix.org: Host 82-203-189-118.bb.dnainternet.fi [82.203.189.118] claimed to be [192.168.1.101] From: Stefan Parvu Mime-Version: 1.0 (Mac OS X Mail 12.0 \(3445.100.39\)) Subject: Re: boot loader serial console speed to 9600 Date: Thu, 25 Oct 2018 18:28:39 +0300 References: <1540480146.22340.79.camel@freebsd.org> To: freebsd-arm@freebsd.org In-Reply-To: <1540480146.22340.79.camel@freebsd.org> Message-Id: <02DDF4DE-2735-4B81-9BB7-E07F60DCB44B@kronometrix.org> X-Mailer: Apple Mail (2.3445.100.39) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2018 15:28:48 -0000 > The ability to change serial console speed from loader(8) only works > with x86 BIOS calls (and maybe UEFI, not sure). In general, when uboot > is involved, loader will use whatever serial speed was set up by = uboot. ok, right. Thats exactly probable what we are seeing. We are using a RBPI3B+ with Strato Base [1] which connects to a Vaisala logger [2] which cannot speak 115200 due its restrictions to operate on solar = panels etc. So we are experimenting with FreeBSD 12 on this setup to fetch data from the Vaisala logger.=20 > Why do you want to set the console to such a slow speed? It actually > pisses me off that the standard is as slow as 115200 in the 21st > century; modern uart chips have no problem running at speeds in the 1- > 12mbps range. Because I need to connect the Vaisala logger (which cannot speak 115200)=20= to the RBPI3B+ to read data. Using a FDTI cable is simple and probable = will work for us. Is there anything else we could try to configure ? You mean we could = configure the uboot module ? How ? Any pointers ? Thanks a lot for tips, Stefan [1] https://www.sferalabs.cc/strato-pi/#models = [2] = https://www.vaisala.com/sites/default/files/documents/QML201C-Datasheet-B2= 10718EN.pdf = =20 From owner-freebsd-arm@freebsd.org Thu Oct 25 15:41:00 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D6DFF107B9E3 for ; Thu, 25 Oct 2018 15:40:59 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5A49B89F70 for ; Thu, 25 Oct 2018 15:40:59 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1540482058; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=jC1DHiR+q6p+XL+qmRAO9MFR13kTCOFMnDKrGaraGY4EPonLKUClJhnyjAyQ2lmuGraysu7Fp+/bZ 0I7isJUbf36X+lcJ8AYBMm4LF9T1n4o9NMEocMQHxfFEQOaq5FEmKFLPA3BhM3bf2wS3CGv0HyBi3K IrTuVBx3Qg8DzYQRSqlJGF7TOBM/IsZD2H5RA2mCLFM3PyrTc7DBirEMcHqf7Lg4wPgaX4f2kBjRIF G7DAVJe3+VLM0evw8Eido21JEZ8NIqNBd/JXFQvb/q07+I9zVRfz4F/Lm7oS7tf8mIAv9xnJcJSUKM HPuxHGcpz9vClD8ALAjhZQJ9WkN3dbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:to:from:subject:message-id:dkim-signature:from; bh=EgySd2eRyos78FQWBB/PCUSpa9zzQYDWG4Pzl6QeBLw=; b=MQtXo15pXJFEzi1oTkqw36VYfJcrofKeFd3OTEM4eDSoi1lrCf9tonrKqdOo1HUmA2jdmDLq+M2C7 98m17skZe3OuMHzalfp/0ODZSaoLmDaACotpB57rLADmlDt1alwBP4cNNJqy3UV9eukxDeWtjZUwaT L8r6cxxTmAnWVQu9Me4+EiFGhEnT0Jg7OmvJ7qcVqS06T4I19iuQx5vuOoMstUys29oTcv8+Rx2Q/5 EY+wZX5SHfPuKJFioh1a4ViJ49KYqJD20h2i7zmOIwswSqA5z2hM4M24nhx4R3fzGI0lgrGa/uEYrc LJqgnK63zFRdZzctlolN+IgJaA/MjzQ== ARC-Authentication-Results: i=1; outbound1.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:to:from:subject:message-id:from; bh=EgySd2eRyos78FQWBB/PCUSpa9zzQYDWG4Pzl6QeBLw=; b=JfUMpSP2IRX3K1fqpKww1WQ5gttEu2wgIKZM3Kalb1ET1TyUU1sh+YyG/aG712Vx/fSYm3DNlOzpx X2QUpr81UJs3wb5tOdzZ/LhU0Q8rD1FYNOKXpZDiCa0ebZa30hbwWnOue3RUvMy7kqclLkll9RsaYw BAG00FoXEyON+5J7uj6HmT94Ov/0FrbdMNFdC7vn519WMYYfHmjszHXfmNeu/N9OPkFE+o6QLm1USt Qhk7iROp69CD4dJO/GRSJ6ZlNAhNiuYlflCgIdGVuq5t3colA7+jzv2d9VEoDpnzZZyLNr0rQyvrQQ C87eyMnbg252T9ieqGOx5FFnvr2xX/g== X-MHO-RoutePath: aGlwcGll X-MHO-User: 5ce079a4-d86c-11e8-a93b-310333596487 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound1.ore.mailhop.org (Halon) with ESMTPSA id 5ce079a4-d86c-11e8-a93b-310333596487; Thu, 25 Oct 2018 15:40:57 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id w9PFetm0052817; Thu, 25 Oct 2018 09:40:55 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1540482055.22340.83.camel@freebsd.org> Subject: Re: boot loader serial console speed to 9600 From: Ian Lepore To: Stefan Parvu , freebsd-arm@freebsd.org Date: Thu, 25 Oct 2018 09:40:55 -0600 In-Reply-To: <02DDF4DE-2735-4B81-9BB7-E07F60DCB44B@kronometrix.org> References: <1540480146.22340.79.camel@freebsd.org> <02DDF4DE-2735-4B81-9BB7-E07F60DCB44B@kronometrix.org> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2018 15:41:00 -0000 On Thu, 2018-10-25 at 18:28 +0300, Stefan Parvu wrote: > > > > The ability to change serial console speed from loader(8) only > > works > > with x86 BIOS calls (and maybe UEFI, not sure). In general, when > > uboot > > is involved, loader will use whatever serial speed was set up by > > uboot. > ok, right. Thats exactly probable what we are seeing. We are using a > RBPI3B+ with Strato Base [1] which connects to a Vaisala logger [2] > which cannot speak 115200 due its restrictions to operate on solar > panels etc. > So we are experimenting with FreeBSD 12 on this setup to fetch data > from > the Vaisala logger.  > > > > > Why do you want to set the console to such a slow speed? It > > actually > > pisses me off that the standard is as slow as 115200 in the 21st > > century; modern uart chips have no problem running at speeds in the > > 1- > > 12mbps range. > Because I need to connect the Vaisala logger (which cannot speak > 115200)  > to the RBPI3B+ to read data. Using a FDTI cable is simple and > probable will > work for us. > > Is there anything else we could try to configure ? You mean we could > configure > the uboot module ? How ? Any pointers ? > > Thanks a lot for tips, > Stefan > > [1] https://www.sferalabs.cc/strato-pi/#models > > [2] https://www.vaisala.com/sites/default/files/documents/QML201C-Dat > asheet-B210718EN.pdf > Datasheet-B210718EN.pdf>  Normally you set a uboot env var named "baudrate".  If you do it interactively, it will prompt you to change your terminal program rate to the new value then hit enter.  If you do saveenv it'll just run at that speed from startup on future boots. But, everything about RPi is non-standard, so probably none of that works, in which case you'll have to hunt around for the rpi way of doing the same thing. I would think if you're emitting data to a logger via uart, you would want to disable the serial console completely, so that your data generating application can have exclusive use of the uart to output data. -- Ian From owner-freebsd-arm@freebsd.org Thu Oct 25 15:58:43 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 10272107CC79 for ; Thu, 25 Oct 2018 15:58:43 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A008F8B3EF for ; Thu, 25 Oct 2018 15:58:42 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro-2.local (ralph.baldwin.cx [66.234.199.215]) by mail.baldwin.cx (Postfix) with ESMTPSA id 2EE3110A87D; Thu, 25 Oct 2018 11:58:41 -0400 (EDT) Subject: Re: Poor virtio performance on Scaleway ARM systems To: Peter Jeremy References: <20180916184657.GB24416@server.rulingia.com> <1540309565.1771.1@smtp.migadu.com> <966ed99b-4709-5693-dca2-77e3e6d7d830@FreeBSD.org> <20181023184949.GB20902@server.rulingia.com> Cc: Greg V , freebsd-arm@freebsd.org From: John Baldwin Message-ID: Date: Thu, 25 Oct 2018 08:58:40 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <20181023184949.GB20902@server.rulingia.com> Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Thu, 25 Oct 2018 11:58:41 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2018 15:58:43 -0000 On 10/23/18 11:49 AM, Peter Jeremy wrote: > On 2018-Oct-23 10:04:04 -0700, John Baldwin wrote: >> On 10/23/18 8:46 AM, Greg V wrote: >>> On Sun, Sep 16, 2018 at 9:46 PM, Peter Jeremy >>> wrote: >>>> I have been playing with the 4-core ARM64 VPSs on >>>> https://www.scaleway.com >>>> and notice that the disk I/O performance (using virtio_blk) is >>>> abyssmal. >>>> >>>> The only oddity I've found is that FreeBSD is reporting very high >>>> interrupt >>>> rates on gic0,p11, gic0,s4 and gic0,s5 whilst disk I/O is occurring. >>>> Unfortunately, I can't tell what is attached to those interrupts (it's >>>> not obvious from the dmesg and reported as "+"). >>> gic0,s4 correlates to network activity. >>> gic0,s5 correlates to disk activity. >>> >>> echo 'hw.pci.honor_msi_blacklist="0"' >> /boot/loader.conf > >> Can you make 'pciconf -lc' output available? There is a "generic" >> blacklist that assumes we will see a host bridge device with PCI-e >> or PCI-X and if neither of those is present we also blacklist MSI. >> However, we can certainly figure out how to make this work out of >> the box. > > $ sudo pciconf -lc > hostb0@pci0:0:0:0: class=0x060000 card=0x11001af4 chip=0x00081b36 rev=0x00 hdr=0x00 > virtio_pci0@pci0:0:1:0: class=0x020000 card=0x00011af4 chip=0x10001af4 rev=0x00 hdr=0x00 > cap 11[98] = MSI-X supports 3 messages > Table in map 0x14[0x0], PBA in map 0x14[0x800] > cap 09[84] = vendor (length 20) > cap 09[70] = vendor (length 20) > cap 09[60] = vendor (length 16) > cap 09[50] = vendor (length 16) > cap 09[40] = vendor (length 16) > virtio_pci1@pci0:0:2:0: class=0x010000 card=0x00021af4 chip=0x10011af4 rev=0x00 hdr=0x00 > cap 11[98] = MSI-X supports 2 messages > Table in map 0x14[0x0], PBA in map 0x14[0x800] > cap 09[84] = vendor (length 20) > cap 09[70] = vendor (length 20) > cap 09[60] = vendor (length 16) > cap 09[50] = vendor (length 16) > cap 09[40] = vendor (length 16) The relevant code is here in sys/dev/pci/pci.c: /* * Determine if MSI is blacklisted globally on this system. Currently, * we just check for blacklisted chipsets as represented by the * host-PCI bridge at device 0:0:0. In the future, it may become * necessary to check other system attributes, such as the kenv values * that give the motherboard manufacturer and model number. */ static int pci_msi_blacklisted(void) { device_t dev; if (!pci_honor_msi_blacklist) return (0); /* Blacklist all non-PCI-express and non-PCI-X chipsets. */ if (!(pcie_chipset || pcix_chipset)) { if (vm_guest != VM_GUEST_NO) { /* * Whitelist older chipsets in virtual * machines known to support MSI. */ dev = pci_find_bsf(0, 0, 0); if (dev != NULL) return (!pci_has_quirk(pci_get_devid(dev), PCI_QUIRK_ENABLE_MSI_VM)); } return (1); } dev = pci_find_bsf(0, 0, 0); if (dev != NULL) return (pci_msi_device_blacklisted(dev)); return (0); } In this case no devices have PCI-express or PCI-X capabilities. We could perhaps whitelist your hostb device, but I think a more general solution might be to whitelist MSI if there are any virtio devices as virtio devices can always do MSI. I can work on a patch unless someone else beats me to it, but for the virtio approach what I would do is have a global 'virtio_seen' similar to 'pcie_chipset' that gets set to true if we find any virtio devices during the initial bus scans and then add '|| virtio_seen' to the second 'if' there. -- John Baldwin From owner-freebsd-arm@freebsd.org Thu Oct 25 16:21:27 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C8DAC107E2EE for ; Thu, 25 Oct 2018 16:21:27 +0000 (UTC) (envelope-from sparvu@kronometrix.org) Received: from mail.kronometrix.org (mail.kronometrix.org [95.85.46.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.kronometrix.org", Issuer "mail.kronometrix.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5F84B8CB95; Thu, 25 Oct 2018 16:21:27 +0000 (UTC) (envelope-from sparvu@kronometrix.org) Received: from [192.168.1.101] (82-203-189-118.bb.dnainternet.fi [82.203.189.118]) (authenticated bits=0) by mail.kronometrix.org (8.15.2/8.15.2) with ESMTPSA id w9PGLOFJ013635 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 25 Oct 2018 16:21:25 GMT (envelope-from sparvu@kronometrix.org) X-Authentication-Warning: mail.kronometrix.org: Host 82-203-189-118.bb.dnainternet.fi [82.203.189.118] claimed to be [192.168.1.101] From: Stefan Parvu Message-Id: <17904788-30B5-43D0-8768-570B291113F5@kronometrix.org> Mime-Version: 1.0 (Mac OS X Mail 12.0 \(3445.100.39\)) Subject: Re: boot loader serial console speed to 9600 Date: Thu, 25 Oct 2018 19:21:16 +0300 In-Reply-To: <1540482055.22340.83.camel@freebsd.org> Cc: freebsd-arm@freebsd.org To: Ian Lepore References: <1540480146.22340.79.camel@freebsd.org> <02DDF4DE-2735-4B81-9BB7-E07F60DCB44B@kronometrix.org> <1540482055.22340.83.camel@freebsd.org> X-Mailer: Apple Mail (2.3445.100.39) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2018 16:21:28 -0000 > I would think if you're emitting data to a logger via uart, you would > want to disable the serial console completely, so that your data > generating application can have exclusive use of the uart to output > data. btw: how can u do that ? How can I completely disable the serial console on FBSD12 ? thanks, Stefan From owner-freebsd-arm@freebsd.org Thu Oct 25 16:33:29 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6CD3F107EFCD for ; Thu, 25 Oct 2018 16:33:29 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1a.eu.mailhop.org (outbound1a.eu.mailhop.org [52.58.109.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E49388D9B8 for ; Thu, 25 Oct 2018 16:33:28 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1540485201; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=BW+0UspuZdso4v9nRIL3nd/a1dyMFotYa7aQbahJe0jkNPFm3AXpU7v+GrS+d81XaxeoJYvSYUUGR BZ4ccoj/JQrJtofhj9oy7nG2CBt/atMcfi3ri8XIU8bj0yVwD7FiWRG5eFDAqJom8AsnJO8+v11scJ cJ3tIe1uywe4G+EeB/v94EcUJFaeA2HffKV7bvQ35JdxakS4chvQfJMfUCssOOV4j39S1U5DSJYu0v LVXg5wT/zqAO7PCVpnE7RPDSJupGQ0yaW5G7Zi99xcn0oTv7ig8FEnzBi9sr41QCLtXy6Gil6WskCV ZtPBVKD2msZo38SZIUHBzNYBU0zlusg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=Wz9tS2F6annCgh47I1o43I/Wz2K0wSTfo/7Uyb84C5o=; b=Mwz6fKe4SDdLp7DFHX+1thBIqUrCebFfJilV46NWEy+DhsNm7pE9ZnlXcLcdV1erKJtxD6eXZrs0t IWL4AnkPFJ73W/+mjkCtf+C7sYLOmSMx64pG8dfRmaV9BDHbPolJ56z/5WbGjU04YNk6qcIJG5uGAI EF7ly0eZRHa4Tu1bbWc07xAW9P2Xg3Ra2A0CKc9WQGbYzlQ7GSmNoHNoWWu77Gpz9WjJ9D1k06+WLj G+cdKFeUQYrUNcANe3hVAq7DPlZxO6taZ3KV1cYIr/qkHfP4v7eQxefwYQ0wbLYjShUzS3dhLLJ+8h ff/0ccH41+cnryoc9Al+YI0bFcFF00g== ARC-Authentication-Results: i=1; outbound1.eu.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=Wz9tS2F6annCgh47I1o43I/Wz2K0wSTfo/7Uyb84C5o=; b=rMduz6eKsPFE20ZzowqTJ4cb3Mvn4uB2vZqS6a7IPYz57TGWlg+O4aNrOdGMef1pTcHqc3oGC0Noc VO00euT2jyWe7vcyYJe/EFeJR6RtRiyH8QNE+bPy9jHLQ7T6Wou2+22snyAbcin7Cl6bnMb1jlIKBo CERb9e4+Du9FADTpJm0ggKhWSLmqi2lKLcd790KrKLFn5lzc0i45xwaqVWXIj1nBcT8BpM30+JR1rq Ow3MQ8AaZssv3JZBEJ53ChJ8REcO0HV5aXYowzs6iOBlVC++TDNAgN9m2kstdioG1CkI9JFj3lzxp3 TlCkHce0Nc9hKFnQlTkPr2jhD2Bn55A== X-MHO-RoutePath: aGlwcGll X-MHO-User: ab713f2a-d873-11e8-9048-075f73944867 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound1.eu.mailhop.org (Halon) with ESMTPSA id ab713f2a-d873-11e8-9048-075f73944867; Thu, 25 Oct 2018 16:33:17 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id w9PGXEfW052952; Thu, 25 Oct 2018 10:33:14 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1540485194.22340.87.camel@freebsd.org> Subject: Re: boot loader serial console speed to 9600 From: Ian Lepore To: Stefan Parvu Cc: freebsd-arm@freebsd.org Date: Thu, 25 Oct 2018 10:33:14 -0600 In-Reply-To: <17904788-30B5-43D0-8768-570B291113F5@kronometrix.org> References: <1540480146.22340.79.camel@freebsd.org> <02DDF4DE-2735-4B81-9BB7-E07F60DCB44B@kronometrix.org> <1540482055.22340.83.camel@freebsd.org> <17904788-30B5-43D0-8768-570B291113F5@kronometrix.org> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2018 16:33:29 -0000 On Thu, 2018-10-25 at 19:21 +0300, Stefan Parvu wrote: > > > > I would think if you're emitting data to a logger via uart, you > > would > > want to disable the serial console completely, so that your data > > generating application can have exclusive use of the uart to output > > data. > btw: how can u do that ?  How can I completely disable the serial > console > on FBSD12 ? > > thanks, > Stefan > Here's an old mail thread describing how to disable the console so that a gps add-on board would work: https://lists.freebsd.org/pipermail/freebsd-arm/2017-January/015438.html That's pretty old, so I'm not sure everything in it still applies, but I suspect there is still useful info there. Also, a few months ago we got a new driver for the mini-uart that's in an RPi3, it uses a separate set of IO pins from the console uart, maybe that's a viable solution for you? I don't know much about the second uart, I've just seen people mention it. -- Ian From owner-freebsd-arm@freebsd.org Thu Oct 25 19:10:31 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A14B210C55C3 for ; Thu, 25 Oct 2018 19:10:31 +0000 (UTC) (envelope-from sparvu@kronometrix.org) Received: from mail.kronometrix.org (mail.kronometrix.org [95.85.46.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.kronometrix.org", Issuer "mail.kronometrix.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 313F96DF64 for ; Thu, 25 Oct 2018 19:10:30 +0000 (UTC) (envelope-from sparvu@kronometrix.org) Received: from [192.168.1.101] (82-203-189-118.bb.dnainternet.fi [82.203.189.118]) (authenticated bits=0) by mail.kronometrix.org (8.15.2/8.15.2) with ESMTPSA id w9PJASE0016385 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Thu, 25 Oct 2018 19:10:29 GMT (envelope-from sparvu@kronometrix.org) X-Authentication-Warning: mail.kronometrix.org: Host 82-203-189-118.bb.dnainternet.fi [82.203.189.118] claimed to be [192.168.1.101] From: Stefan Parvu Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Mac OS X Mail 12.0 \(3445.100.39\)) Subject: Re: boot loader serial console speed to 9600 Date: Thu, 25 Oct 2018 22:10:22 +0300 References: <1540480146.22340.79.camel@freebsd.org> <02DDF4DE-2735-4B81-9BB7-E07F60DCB44B@kronometrix.org> <1540482055.22340.83.camel@freebsd.org> <17904788-30B5-43D0-8768-570B291113F5@kronometrix.org> <1540485194.22340.87.camel@freebsd.org> To: freebsd-arm@freebsd.org In-Reply-To: <1540485194.22340.87.camel@freebsd.org> Message-Id: <4F0CAF1D-2A7D-4561-8CFD-34AA5C429CD0@kronometrix.org> X-Mailer: Apple Mail (2.3445.100.39) X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2018 19:10:31 -0000 > > https://lists.freebsd.org/pipermail/freebsd-arm/2017-January/015438.html > > That's pretty old, so I'm not sure everything in it still applies, but > I suspect there is still useful info there. 10x thanks. will try that soon Stefan From owner-freebsd-arm@freebsd.org Thu Oct 25 19:45:41 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 26F731086383 for ; Thu, 25 Oct 2018 19:45:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B91EC7014C for ; Thu, 25 Oct 2018 19:45:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id EACD8194D1 for ; Thu, 25 Oct 2018 19:45:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w9PJjddT069912 for ; Thu, 25 Oct 2018 19:45:39 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w9PJjdSI069909 for freebsd-arm@FreeBSD.org; Thu, 25 Oct 2018 19:45:39 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: freebsd-arm@FreeBSD.org Subject: [Bug 232688] suspend/resume does not work on a thinkpad/lenove X280 Date: Thu, 25 Oct 2018 19:45:39 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: arm X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jacques+freebsdbugzilla@foucry.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-arm@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter attachments.created Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2018 19:45:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D232688 Bug ID: 232688 Summary: suspend/resume does not work on a thinkpad/lenove X280 Product: Base System Version: 11.2-RELEASE Hardware: amd64 OS: Any Status: New Severity: Affects Only Me Priority: --- Component: arm Assignee: freebsd-arm@FreeBSD.org Reporter: jacques+freebsdbugzilla@foucry.net Created attachment 198626 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D198626&action= =3Dedit dmesg file of my configuration Hello, My new Thinkpad/Lenovo X280 does not support suspend/resume operation. I checked the page https://wiki.freebsd.org/SuspendResume and the command `acpiconf -s 3` freeze my computer. I try to load with kldload `/boot/kernel/acpi_ibm.ko` with to succes. You'll attached dmesg, devinfo and pciconf files as requested in the page. I stil ready to test more. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-arm@freebsd.org Fri Oct 26 03:56:10 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0F3E910D4A3B for ; Fri, 26 Oct 2018 03:56:10 +0000 (UTC) (envelope-from sparvu@kronometrix.org) Received: from mail.kronometrix.org (mail.kronometrix.org [95.85.46.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.kronometrix.org", Issuer "mail.kronometrix.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 91F2481422 for ; Fri, 26 Oct 2018 03:56:09 +0000 (UTC) (envelope-from sparvu@kronometrix.org) Received: from [192.168.1.101] (82-203-189-118.bb.dnainternet.fi [82.203.189.118]) (authenticated bits=0) by mail.kronometrix.org (8.15.2/8.15.2) with ESMTPSA id w9Q3u6B7024902 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Fri, 26 Oct 2018 03:56:07 GMT (envelope-from sparvu@kronometrix.org) X-Authentication-Warning: mail.kronometrix.org: Host 82-203-189-118.bb.dnainternet.fi [82.203.189.118] claimed to be [192.168.1.101] From: Stefan Parvu Mime-Version: 1.0 (Mac OS X Mail 12.0 \(3445.100.39\)) Subject: Re: boot loader serial console speed to 9600 Date: Fri, 26 Oct 2018 06:56:00 +0300 References: <1540480146.22340.79.camel@freebsd.org> <02DDF4DE-2735-4B81-9BB7-E07F60DCB44B@kronometrix.org> <1540482055.22340.83.camel@freebsd.org> <17904788-30B5-43D0-8768-570B291113F5@kronometrix.org> <1540485194.22340.87.camel@freebsd.org> <4F0CAF1D-2A7D-4561-8CFD-34AA5C429CD0@kronometrix.org> To: freebsd-arm@freebsd.org In-Reply-To: <4F0CAF1D-2A7D-4561-8CFD-34AA5C429CD0@kronometrix.org> Message-Id: <65F04B5D-FB7B-4A9A-A546-21E37553EB91@kronometrix.org> X-Mailer: Apple Mail (2.3445.100.39) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Oct 2018 03:56:10 -0000 So here are the final comments on this: 1. Disable entire serial console, as documented here [1] echo 'hw.fdt.console=3D"NO"' >> /boot/loader.conf.local BTW the man page of loader.conf or loader must sy about this = undocumented=20 variable setting.=20 2. Set a lower speed of the serial console from 115200 to 9600 for = example uboot config.txt init_uart_baud=3D9600 It would be nice whoever wrote this part to properly document these such = way this is not a mission impossible to find. I will test in coming days these things to = see if they actually=20 work.=20 Thanks Ian for tips and support Stefan= From owner-freebsd-arm@freebsd.org Fri Oct 26 04:03:35 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D54F10D4EB4 for ; Fri, 26 Oct 2018 04:03:35 +0000 (UTC) (envelope-from sparvu@kronometrix.org) Received: from mail.kronometrix.org (mail.kronometrix.org [95.85.46.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.kronometrix.org", Issuer "mail.kronometrix.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id CBD8E8196F for ; Fri, 26 Oct 2018 04:03:34 +0000 (UTC) (envelope-from sparvu@kronometrix.org) Received: from [192.168.1.101] (82-203-189-118.bb.dnainternet.fi [82.203.189.118]) (authenticated bits=0) by mail.kronometrix.org (8.15.2/8.15.2) with ESMTPSA id w9Q43WZR025026 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Fri, 26 Oct 2018 04:03:33 GMT (envelope-from sparvu@kronometrix.org) X-Authentication-Warning: mail.kronometrix.org: Host 82-203-189-118.bb.dnainternet.fi [82.203.189.118] claimed to be [192.168.1.101] From: Stefan Parvu Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.0 \(3445.100.39\)) Subject: RBPI3B+ clock speed Message-Id: Date: Fri, 26 Oct 2018 07:03:26 +0300 To: freebsd-arm@freebsd.org X-Mailer: Apple Mail (2.3445.100.39) X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Oct 2018 04:03:35 -0000 Hi, Coming back to the clock speed of the RBPI3B+ I have some questions: - How can I see the current clock speed my RBPI is using at any moment ? - How would you change the frequency of the clock speed which by default = seems=20 set to 600MHz ?=20 I hope changing the cpu frequency speed would not affect how the uart is = operating. I do recall they were issues on a previous RBPI3B+ version regarding = this. Thanks, Stefan= From owner-freebsd-arm@freebsd.org Fri Oct 26 04:34:50 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3D5B510D5CF7 for ; Fri, 26 Oct 2018 04:34:50 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vtr.rulingia.com (vtr.rulingia.com [IPv6:2001:19f0:5801:ebe:5400:1ff:fe53:30fd]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vtr.rulingia.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8190283481 for ; Fri, 26 Oct 2018 04:34:49 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from server.rulingia.com (ppp59-167-167-3.static.internode.on.net [59.167.167.3]) by vtr.rulingia.com (8.15.2/8.15.2) with ESMTPS id w9Q4YabT069156 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Fri, 26 Oct 2018 15:34:42 +1100 (AEDT) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.15.2/8.15.2) with ESMTPS id w9Q4YTfi065779 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Fri, 26 Oct 2018 15:34:29 +1100 (AEDT) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.15.2/8.15.2/Submit) id w9Q4YTrg065778 for freebsd-arm@freebsd.org; Fri, 26 Oct 2018 15:34:29 +1100 (AEDT) (envelope-from peter) Date: Fri, 26 Oct 2018 15:34:29 +1100 From: Peter Jeremy To: freebsd-arm@freebsd.org Subject: Re: Poor virtio performance on Scaleway ARM systems Message-ID: <20181026043429.GA64669@server.rulingia.com> References: <20180916184657.GB24416@server.rulingia.com> <1540309565.1771.1@smtp.migadu.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="BOKacYhQ+x31HxR3" Content-Disposition: inline In-Reply-To: <1540309565.1771.1@smtp.migadu.com> X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.10.0 (2018-05-17) X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Oct 2018 04:34:50 -0000 --BOKacYhQ+x31HxR3 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2018-Oct-23 18:46:05 +0300, Greg V wrote: >echo 'hw.pci.honor_msi_blacklist=3D"0"' >> /boot/loader.conf > >With the fix, we're getting ~1000 interrupts on 'its0,4: virtio_pci1'=20 >(instead of ~30000 on gic), latency is no longer awful, and bandwidth=20 >is about 220 MB/s. YAY!! Unfortunately, this isn't working for me on 13-CURRENT. I am still seeing ~4.5MBps disk I/O with ~30,000 interrupts/sec on gic0,s5. I don't even see an its0,4. =20 Looking at a verbose boot, the relevant errors are: its0: mem 0x8080000-0x809ffff on gi= c0 its0: GITS_BASER0: unable to be updated: a907000040a40607 !=3D a907000040ee= 0607 device_attach: its0 attach returned 6 =2E.. virtio_pci0: port 0xf040-0xf05f mem 0x3ee09000= -0x3ee09fff,0x3ee00000-0x3ee03fff at device 1.0 on pci0 vtnet0: on virtio_pci0 virtio_pci0: host features: 0x79bfffe7 virtio_pci0: negotiated features: 0x308fbbe3 vtnet0: bpf attached vtnet0: Ethernet address: de:2b:88:06:a0:4c virtio_pci0: attempting to allocate 3 MSI-X vectors (3 supported) virtio_pci0: attempting to allocate 2 MSI-X vectors (3 supported) virtio_pci0: using legacy interrupt virtio_pci1: port 0xf000-0xf03f mem 0x3ee08000-0= x3ee08fff,0x3ee04000-0x3ee07fff at device 2.0 on pci0 vtblk0: on virtio_pci1 virtio_pci1: host features: 0x79000e54 virtio_pci1: negotiated features: 0x10000e54 virtio_pci1: attempting to allocate 2 MSI-X vectors (2 supported) virtio_pci1: attempting to allocate 2 MSI-X vectors (2 supported) virtio_pci1: using legacy interrupt vtblk0: 47683MB (97656250 512 byte sectors) I haven't had a chance to dig into either of these any further but hope to = do so over the weekend. -- Peter Jeremy --BOKacYhQ+x31HxR3 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEE7rKYbDBnHnTmXCJ+FqWXoOSiCzQFAlvSmVRfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEVF QjI5ODZDMzA2NzFFNzRFNjVDMjI3RTE2QTU5N0EwRTRBMjBCMzQACgkQFqWXoOSi CzQ4Yg/+Nap/BL1gukJpj5bO5V7EBC0aijNpgq72zLXqFzmtjvfUjIoAZmdIqtq4 F/U1hRz1oMA932x83fvMlChO45bjBq/BymZG508cGuL6NvlSuy4lFlQxBzR1XFsC sW/AXYBCdavZFdN6qMY3WGKOrfvqzvqCVYGAkcVwmvukBUbvoDy5amy3TXcRPKyL Tr/Bp4woIIZ7LZfbpDdur3u0PAVAfuCNYMQ8esUWGkxGZ43VqNbnmrWGQWPHqCcY 5Ozr8Y53KwdQqsruBAmCWKhcsvBg7YYAkTEfnzt9a8FTw/cc+i5NPIeS0nvaMroT NAJKoh+kt4d0xCHk2MkqIWMnk634XFczzDaOoNU94FbgezAByaou7u7padkFht+B 4tRSsf3KQtJFL6DMvNnxFcfoYdBZ00WGWBgUrFI+CtEyfsxZt990m6GySNsLJwRm h21UnS1Ro6Kne2rrvBeFnCap14qvtePdQlvOMyipcgXX8q5pLxFQ+HKSx7KsJ29v 7q/F0z8NqB0ziAO2w2axPahj9vh7nuER+9LxYq3j2sF5QY9vP6Uj9dRg8EiIEq0Q /kYwCy1I4fS+5YFyvrL1vATP/3E1RuUXn5DcdpiN2g9xhP1RaGTT089rLkyE+jr/ vdzFNKLTDLEUFPIGvc2gyT43qpi3Em/hx11Y6ul6pp37JBrcEBM= =2p1g -----END PGP SIGNATURE----- --BOKacYhQ+x31HxR3-- From owner-freebsd-arm@freebsd.org Fri Oct 26 04:41:08 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7039310D5E08 for ; Fri, 26 Oct 2018 04:41:08 +0000 (UTC) (envelope-from sparvu@kronometrix.org) Received: from mail.kronometrix.org (mail.kronometrix.org [95.85.46.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.kronometrix.org", Issuer "mail.kronometrix.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EBA898358E for ; Fri, 26 Oct 2018 04:41:07 +0000 (UTC) (envelope-from sparvu@kronometrix.org) Received: from [192.168.1.101] (82-203-189-118.bb.dnainternet.fi [82.203.189.118]) (authenticated bits=0) by mail.kronometrix.org (8.15.2/8.15.2) with ESMTPSA id w9Q4f3N7025615 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Fri, 26 Oct 2018 04:41:06 GMT (envelope-from sparvu@kronometrix.org) X-Authentication-Warning: mail.kronometrix.org: Host 82-203-189-118.bb.dnainternet.fi [82.203.189.118] claimed to be [192.168.1.101] From: Stefan Parvu Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.0 \(3445.100.39\)) Subject: Re: RBPI3B+ clock speed Date: Fri, 26 Oct 2018 07:40:57 +0300 References: To: freebsd-arm@freebsd.org In-Reply-To: Message-Id: <98EE4128-8377-40F4-966C-1CA15F7C644D@kronometrix.org> X-Mailer: Apple Mail (2.3445.100.39) X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Oct 2018 04:41:08 -0000 > - How can I see the current clock speed my RBPI is using at any moment = ? by checking system=E2=80=99s variables: dev.cpu.0.freq_levels: 1400/-1 600/-1 dev.cpu.0.freq: 1400 this means now my system is using 1400 frequency vs 600.=20 >=20 > - How would you change the frequency of the clock speed which by = default seems=20 > set to 600MHz ?=20 by simple setting powerd during rc.conf or manually start it.=20 Are these correct ?= From owner-freebsd-arm@freebsd.org Fri Oct 26 06:17:32 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3257B10D80B2 for ; Fri, 26 Oct 2018 06:17:32 +0000 (UTC) (envelope-from iz-rpi03@hs-karlsruhe.de) Received: from smtp.hs-karlsruhe.de (smtp.HS-Karlsruhe.DE [193.196.64.25]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CC9FE861E4 for ; Fri, 26 Oct 2018 06:17:31 +0000 (UTC) (envelope-from iz-rpi03@hs-karlsruhe.de) Received: from iz-wera01.hs-karlsruhe.de ([193.196.65.46]) by smtp.hs-karlsruhe.de with esmtp (Exim 4.80.1) (envelope-from ) id 1gFvQx-009JaE-DL; Fri, 26 Oct 2018 07:17:23 +0100 X-Mailer: exmh version 2.8.0 04/21/2012 with nmh-1.6 From: Ralf Wenk To: Stefan Parvu cc: freebsd-arm@freebsd.org Subject: Re: RBPI3B+ clock speed In-reply-to: <98EE4128-8377-40F4-966C-1CA15F7C644D@kronometrix.org> References: <98EE4128-8377-40F4-966C-1CA15F7C644D@kronometrix.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Date: Fri, 26 Oct 2018 08:17:22 +0200 Message-Id: X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Oct 2018 06:17:32 -0000 On 2018-10-26 at 7:40 +0300 Stefan Parvu wrote: > > - How can I see the current clock speed my RBPI is using at any momen= t ? >=20 > by checking system’s variables: >=20 > dev.cpu.0.freq_levels: 1400/-1 600/-1 > dev.cpu.0.freq: 1400 >=20 > this means now my system is using 1400 frequency vs 600.=20 >=20 > >=20 > > - How would you change the frequency of the clock speed which by > > default seems=20 > > set to 600MHz ?=20 >=20 > by simple setting powerd during rc.conf or manually start it.=20 >=20 > Are these correct ? Last time I tried powerd(8) on a RPi3B+ it was complaining about missing a CPU clock driver. Meanwhile I did not read anything in the repository log about work on that issue. So probably powerd(8) will not work. A RPi3B shows more information about CPU frequency(s) than you provided: =24 sysctl -a =7C fgrep cpu =7C fgrep freq debug.cpufreq.verbose: 0 debug.cpufreq.lowest: 0 hw.cpufreq.temperature: 49388 hw.cpufreq.voltage_sdram_p: 1200000 hw.cpufreq.voltage_sdram_i: 1200000 hw.cpufreq.voltage_sdram_c: 1200000 hw.cpufreq.voltage_core: 1256250 hw.cpufreq.turbo: 1 hw.cpufreq.sdram_freq: 450000000 hw.cpufreq.core_freq: 400000000 hw.cpufreq.arm_freq: 1200000000 dev.cpufreq.0.%parent: cpu0 dev.cpufreq.0.%pnpinfo:=20 dev.cpufreq.0.%location:=20 dev.cpufreq.0.%driver: cpufreq dev.cpufreq.0.%desc:=20 dev.cpufreq.%parent:=20 dev.bcm2835_cpufreq.0.freq_settings: 1200/-1 600/-1 dev.bcm2835_cpufreq.0.%parent: cpu0 dev.bcm2835_cpufreq.0.%pnpinfo:=20 dev.bcm2835_cpufreq.0.%location:=20 dev.bcm2835_cpufreq.0.%driver: bcm2835_cpufreq dev.bcm2835_cpufreq.0.%desc: CPU Frequency Control dev.bcm2835_cpufreq.%parent:=20 dev.cpu.0.freq_levels: 1200/-1 600/-1 dev.cpu.0.freq: 1200 =24=20 Occasionally powerd(8) gets into trouble on this RPi3B. mbox0: mbox response error bcm2835_cpufreq0: can't set clock rate (id=3D3) mbox0: mbox response error bcm2835_cpufreq0: can't get clock rate (id=3D3) bcm2835_cpufreq0: wrong freq bcm2835_cpufreq0: set freq failed, err 5 mbox0: mbox response error bcm2835_cpufreq0: can't set clock rate (id=3D8) Which seems to depend on the kernel release. Some are working for weeks without that issue, others only 24 hours. The samples came from revision 339388. Regards Ralf From owner-freebsd-arm@freebsd.org Fri Oct 26 07:26:56 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 99FCF10D9CEB for ; Fri, 26 Oct 2018 07:26:56 +0000 (UTC) (envelope-from freebsd-arm@sentry.org) Received: from shadow.sentry.org (shadow.sentry.org [210.8.237.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "shadow.sentry.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CCCCE6A765 for ; Fri, 26 Oct 2018 07:26:55 +0000 (UTC) (envelope-from freebsd-arm@sentry.org) Received: from shadow.sentry.org (localhost [127.0.0.1]) by shadow.sentry.org (8.15.2/8.15.2) with ESMTP id w9Q7MY6q001365 for ; Fri, 26 Oct 2018 18:22:35 +1100 (AEDT) (envelope-from freebsd-arm@sentry.org) Subject: Re: RBPI3B+ clock speed To: freebsd-arm@freebsd.org References: <98EE4128-8377-40F4-966C-1CA15F7C644D@kronometrix.org> From: Trev Message-ID: <5cc1cca0-9d1e-ff82-d485-56d2766a780b@sentry.org> Date: Fri, 26 Oct 2018 18:22:34 +1100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 SeaMonkey/2.49.4 MIME-Version: 1.0 In-Reply-To: <98EE4128-8377-40F4-966C-1CA15F7C644D@kronometrix.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (shadow.sentry.org [0.0.0.0]); Fri, 26 Oct 2018 18:22:35 +1100 (AEDT) X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Oct 2018 07:26:56 -0000 Stefan Parvu wrote on 26/10/2018 15:40: >> - How would you change the frequency of the clock speed which by default seems >> set to 600MHz ? > > by simple setting powerd during rc.conf or manually start it. > > Are these correct ? Yes - powerd works fine on the Rpi3 (tested with 12-Current and now 13-Current) - I used it to keep the freq at 600 for example to run some tests to compare with Rpi2, but it also works adaptively scaling from 600 straight to 1400. From owner-freebsd-arm@freebsd.org Fri Oct 26 09:39:26 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1682E10DD306 for ; Fri, 26 Oct 2018 09:39:26 +0000 (UTC) (envelope-from greg@unrelenting.technology) Received: from out.migadu.com (out.migadu.com [91.121.223.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.migadu.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8A2EB6E93C for ; Fri, 26 Oct 2018 09:39:25 +0000 (UTC) (envelope-from greg@unrelenting.technology) Received: (Migadu outbound); Fri, 26 Oct 2018 09:39:18 +0000 Authentication-Results: out.migadu.com; auth=pass (plain) Received: from [192.168.1.141] ([62.122.208.146]) by out.migadu.com (Haraka/2.8.16) with ESMTPSA id D5F0E0B7-18AF-42C4-9189-4223983F3511.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 verify=FAIL); Fri, 26 Oct 2018 09:39:18 +0000 Date: Fri, 26 Oct 2018 12:39:16 +0300 From: Greg V Subject: Re: Poor virtio performance on Scaleway ARM systems To: Peter Jeremy Cc: freebsd-arm@freebsd.org Message-Id: <1540546756.1759.0@smtp.migadu.com> In-Reply-To: <20181026043429.GA64669@server.rulingia.com> References: <20180916184657.GB24416@server.rulingia.com> <1540309565.1771.1@smtp.migadu.com> <20181026043429.GA64669@server.rulingia.com> X-Mailer: geary/0.12.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed DKIM-Signature: v=1; a=rsa-sha256; bh=PsgLwMSBK8VYQWTSupuXHXMpaF+I7zPYXjgbGuWqaIM=; c=relaxed/simple; d=unrelenting.technology; h=from:subject:date:to; s=default; b=Q2p6cXmAqA1cFLwY5fYYK0eCKbxQoPezsGeyHIcoL+quyvJ0QqvZ09KLaMWrNt1128roBv33NJWsdjoe59V8zEDMSwKuazHqIQtDF+tMSLocIZ5CM7kptIsQ2LTtLKwQ4y9A7893gDYr2hLVG3QUpVmt8uRYeG3NbptasjiaMQ8= X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Oct 2018 09:39:26 -0000 On Fri, Oct 26, 2018 at 7:34 AM, Peter Jeremy wrote: > On 2018-Oct-23 18:46:05 +0300, Greg V > wrote: >> echo 'hw.pci.honor_msi_blacklist="0"' >> /boot/loader.conf >> >> With the fix, we're getting ~1000 interrupts on 'its0,4: virtio_pci1' >> (instead of ~30000 on gic), latency is no longer awful, and bandwidth >> is about 220 MB/s. YAY!! > > Unfortunately, this isn't working for me on 13-CURRENT. I am still > seeing ~4.5MBps disk I/O with ~30,000 interrupts/sec on gic0,s5. I > don't even see an its0,4. > > Looking at a verbose boot, the relevant errors are: > > its0: mem 0x8080000-0x809ffff > on gic0 > its0: GITS_BASER0: unable to be updated: a907000040a40607 != > a907000040ee0607 > device_attach: its0 attach returned 6 Yeah, I think I mentioned this above, sometimes its0 fails to attach. Just reboot. From owner-freebsd-arm@freebsd.org Fri Oct 26 19:03:00 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C01181085C9F for ; Fri, 26 Oct 2018 19:03:00 +0000 (UTC) (envelope-from lisa.connor@galaxymediazone.com) Received: from n1nlsmtp01.shr.prod.ams1.secureserver.net (n1nlsmtp01.shr.prod.ams1.secureserver.net [188.121.43.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "relay-hosting.secureserver.net", Issuer "Starfield Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3FD96839B3 for ; Fri, 26 Oct 2018 19:03:00 +0000 (UTC) (envelope-from lisa.connor@galaxymediazone.com) Received: from n3plcpnl0187.prod.ams3.secureserver.net ([160.153.155.38]) by : HOSTING RELAY : with ESMTP id G7MmgyXCzdePfG7Mmgh2Vo; Fri, 26 Oct 2018 12:01:52 -0700 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=galaxymediazone.com; s=default; h=Content-Type:MIME-Version:Message-ID:Date :Subject:To:From:Sender:Reply-To:Cc:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=560xnwFYqZy7aoWWFE1E0xKhJ9+eA0Ebc3KTB5cQLGY=; b=I2k6WjVDqzlTynWczIJdy3xGK0 9tDjbtXPSZYxQtUSRDlh2mboVthPyKNc3SfoPzPwol8w9EG+Fav1SiNtVcXv7USgYXxRPWT3L9yPR ExgMjZnm8LsbzbflQEUVmmGyrROXmLxL4BbnrdCr3ZF71oQkAKjucmOglcWsyym1C8Lu28S9j1B6j Up4TtP5vlX97LrqBr5iHnC3O/gWPc/SdFJ0znHL1OBn+6d9wo6AO89kyg3A1DNrCRzXCGKjqiRJzm S01AyIWy0diyFKM4oO8FSZ0PbZhVByzFD5zn4CWZEU+zKg96anwrMUxkmaR2qKjLhddZW0F92bBSu mdcZzeWg==; Received: from [207.244.100.145] (port=52935 helo=HIPL3472) by n3plcpnl0187.prod.ams3.secureserver.net with esmtpsa (TLSv1:ECDHE-RSA-AES256-SHA:256) (Exim 4.91) (envelope-from ) id 1gG7Mm-00881g-7K for freebsd-arm@freebsd.org; Fri, 26 Oct 2018 12:01:52 -0700 From: "Lisa Connor" To: Subject: Bloomberg Potential Business Lead Date: Sat, 27 Oct 2018 00:31:50 +0530 Message-ID: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Thread-Index: AdRtXllk3/0C0mAoSz+1/VjE5973CA== Content-Language: en-us X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - n3plcpnl0187.prod.ams3.secureserver.net X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - galaxymediazone.com X-Get-Message-Sender-Via: n3plcpnl0187.prod.ams3.secureserver.net: authenticated_id: lisa.connor@galaxymediazone.com X-Authenticated-Sender: n3plcpnl0187.prod.ams3.secureserver.net: lisa.connor@galaxymediazone.com X-Source: X-Source-Args: X-Source-Dir: X-CMAE-Envelope: MS4wfEyFfHnbkv9UcAzIFw6WxNXTPXE56q6xD65zGxdyjC58fpHjdwJ769JB5+Eb7RjQmcu5iS6nKleNIwzszc3h2gASPJb2Fh/FG9kxPCf9b2kE9wctJphU zNCRSNlqIo5O/ABXBd+Oh5zE6b0e0fIyV0zzZi0Iq0nbo6KWz7gsUxTGTBiIKeWYgoojQl748avk6Qik8jNOb2tY10tTKvAfrzFV/+0XDyRnNd4O0BT55XeQ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Oct 2018 19:03:01 -0000 Hi, I would like to know if you are interested in acquiring Bloomberg Users List. Each Contact Includes: Names, Title, Email, Phone, Company Name, Company URL, and Company physical address, SIC Code, Industry, Company Size (Revenue and Employee). We also have related technology users like: QuickBooks, QuickBooks Online, QuickBooks Payroll many more. Let me know if you are interested, so that I shall get back to you with the counts and pricing. Regards, Lisa Connor Sr. Marketing Manager To delist from this mailing list reply with subject line "No"