Date: Mon, 6 Jun 2005 13:38:14 -0700 (PDT) From: Doug White <dwhite@gumbysoft.com> To: Hiroki Sato <hrs@FreeBSD.org> Cc: sparc64@FreeBSD.org Subject: Re: E4500 with 24GB RAM Message-ID: <20050606132756.X16994@carver.gumbysoft.com> In-Reply-To: <20050607.015510.21897573.hrs@allbsd.org> References: <20050530.025302.64839649.hrs@allbsd.org> <429A7B4D.3080102@samsco.org> <20050607.015510.21897573.hrs@allbsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 7 Jun 2005, Hiroki Sato wrote: > Scott Long <scottl@samsco.org> wrote > in <429A7B4D.3080102@samsco.org>: > > sc> I'd guess that you're the first to have access to so much memory > sc> and a machine to hold it. The error means that vm_map_find() > sc> returned KERN_NO_SPACE. It could be that there is a 64-bit bug > sc> in the code, or it could be that the page tables to index so much > sc> memory consume all available space in the kernel map. > > Thanks. Jake gave me an advice about kern.maxbcache tunable, and > I finally make it boot with kern.maxbcache=524288000. > However, the hme driver seems to have a problem: > > |panic: iommu_enter: XXX: physical address too large (0x5e3fcc000) This panic occurs since the IOMMU can only handle physical addresses up to 34 bits (16GB), it appears. From src/sys/sparc64/include/iommureg.h: 67 #define IOMMU_BITS 34 68 #define IOMMU_MAXADDR (1UL << IOMMU_BITS) Since the panic is in a KASSERT thats why it only pops up under INVARIANTS. However the bus_dma tags specify that hme can only handle 32 bit addresses so it should be using a bounce buffer. I'm guessing that bus_dma needs to be taught about this limitation and use bounce buffers for the IOMMU on >16GB systems. Someone should also research this value a bit more ... I wonder if its from some older system (E450?), and newer machines have larger limits. I would hope they'd build E4500s with IOMMUs that can address all of physical memory. > |cpuid = 0 > |KDB: enter: panic > |[thread pid 442 tid 100215 ] > |Stopped at kdb_enter+0x3c: ta %xcc, 1 > |db> tr > |Tracing pid 442 tid 100215 td 0xfffff800af657b80 > |panic() at panic+0x16c > |iommu_enter() at iommu_enter+0x3c > |iommu_dvmamap_load_buffer() at iommu_dvmamap_load_buffer+0x118 > |iommu_dvmamap_load_mbuf() at iommu_dvmamap_load_mbuf+0x170 > |hme_load_txmbuf() at hme_load_txmbuf+0x68 > |hme_start_locked() at hme_start_locked+0x1bc > |hme_start() at hme_start+0x2c > |if_start() at if_start+0xa0 > |ether_output_frame() at ether_output_frame+0x244 > |ether_output() at ether_output+0x474 > |ip_output() at ip_output+0xa98 > |udp_output() at udp_output+0x5b4 > |udp_send() at udp_send+0x14 > |sosend() at sosend+0x654 > > When INVARIANTS enabled this panic occurs instantly. > > -- Doug White | FreeBSD: The Power to Serve dwhite@gumbysoft.com | www.FreeBSD.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20050606132756.X16994>