From owner-freebsd-arm@freebsd.org Wed Jul 1 17:45:10 2020 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3CF6035858C for ; Wed, 1 Jul 2020 17:45:10 +0000 (UTC) (envelope-from greg@unrelenting.technology) Received: from out0.migadu.com (out0.migadu.com [94.23.1.103]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 49xpXP0WMjz4p2n for ; Wed, 1 Jul 2020 17:45:07 +0000 (UTC) (envelope-from greg@unrelenting.technology) Date: Wed, 01 Jul 2020 17:44:55 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=unrelenting.technology; s=default; t=1593625500; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0OIN3EK1UlRcVt33kxQV+SxnBtjcCYFdFkLRY81Lrxc=; b=gXxA5uqeLLvQI5G1rkABrK+T9WWdpv5NRFNa1vnVtwkSrViHsMMoQvDQ8QMO5h1IiuQliE PvwsQgtSv5KIXDJQPgf2SM46NQfMDCUa4k/JXDdmSLvwRl5Rgha1HX8HrHyITOqPZ4WB2B KO/7x9Neqaygcax/2PVXtTjhU4MRJAY= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: myfreeweb To: Dan Kotowski CC: freebsd-arm Subject: Re: FreeBSD on Layerscape/QorIQ LX2160X In-Reply-To: References: _____<71481BF5-3972-45A3-8287-FCEB1FCCDC41@unrelenting.technology>_?= Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.10 X-Rspamd-Queue-Id: 49xpXP0WMjz4p2n X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=unrelenting.technology header.s=default header.b=gXxA5uqe; dmarc=pass (policy=none) header.from=unrelenting.technology; spf=pass (mx1.freebsd.org: domain of greg@unrelenting.technology designates 94.23.1.103 as permitted sender) smtp.mailfrom=greg@unrelenting.technology X-Spamd-Result: default: False [-3.71 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.03)[-1.030]; R_DKIM_ALLOW(-0.20)[unrelenting.technology:s=default]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:94.23.1.103]; NEURAL_HAM_LONG(-0.99)[-0.987]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[unrelenting.technology:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[unrelenting.technology,none]; NEURAL_HAM_SHORT(-0.69)[-0.689]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RWL_MAILSPIKE_VERYGOOD(0.00)[94.23.1.103:from]; ASN(0.00)[asn:16276, ipnet:94.23.0.0/16, country:FR]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jul 2020 17:45:10 -0000 On June 30, 2020 7:40:26 PM UTC, Dan Kotowski wrote: >> > We do not support the SMMU at all=2E (There is a patch for SMMUv3 sup= port but this chip has a v1/v2=2E) >> >> Appologies for still being new to this, but the following revision impl= ies to me that we do support SMMU? >> >> https://reviews=2Efreebsd=2Eorg/D18002 >> >> I only barely know what I'm doing down at this level, so maybe I'm just= not reading it properly=2E I guess we could throw some debug prints in the= re too? > >Appologies - you were right, the n00b was wrong=2E > >``` >#ifdef notyet >/* > * Not implemented, map a PCIe device to the SMMU it is associated with= =2E > */ >int >acpi_iort_map_smmu(u_int seg, u_int devid, void **smmu, u_int *sid) >{ > /* XXX: convert oref to SMMU device */ > return (ENXIO); >} >#endif >``` > >That's been effectively a stub since Nov 2018=2E=2E=2E > >I've been reading DEN 0029C on SBSA 6=2E0 but it's not clear to me whethe= r we need this for all SBSA-compliant systems or if this is just a decision= Jon and SR are making and claiming it's necessary=2E > >From section 4=2E1=2E3: >""" >Non-secure off-chip devices that cannot directly address all of the Non-s= ecure address space must be placed >behind a stage 1 System MMU compatible with the Arm SMMUv2 or SMMUv3 spec= ification >""" Well, normally PCIe devices can address all the space, unless you have ter= ribly quirky hardware (on the RPi4, PCIe can address 3GB only=2E=2E) Also I would guess that like=2E=2E yes, even if PCIe goes through the SMMU= , if the SMMU is untouched by the OS, interrupts should arrive where the OS= expects it normally=2E >And section 4=2E1=2E6 System MMU and Device Assignment >""" >If a device is assigned and passed through to an operating system under a= hypervisor, then the memory We're running on bare metal, this is about VMs=2E >I'm almost surprised that nobody has bumped into this before=2E How does = the IORT look on the MACCHIATObin if interrupts are NOT mapped behind SMMU? There is no IORT on the mcbin=2E=2E armada8k has a GICv2 not v3 :D Socionext SynQuacer has the PCI node pointed to the ITS node=2E If I'm reading the table disassembly correctly, the Ampere eMAG has PCI no= des pointing to SMMU (v1/v2)=2E But FreeBSD works there fine! That means the "fallback" non-IORT calculation of RIDs works fine there=2E= But maybe it doesn't on the NXP chip because it's buggy=2E But there was *some* way to get the interrupts (without supporting SMMU) o= n this NXP chip =E2=80=93 as evidenced by NetBSD working! Again=2E=2E can you flash the firmware **where PCIe was working under NetB= SD**, dump the IORT there and also try patched (with D25179, e=2Eg=2E any o= f my recent builds I guess) FreeBSD there?