Date: Thu, 25 Apr 2019 13:37:43 -0700 From: John Baldwin <jhb@FreeBSD.org> To: Ryan Stone <rysto32@gmail.com>, Tycho Nightingale <tychon@freebsd.org> Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org, Ed Maste <emaste@freebsd.org> Subject: Re: svn commit: r346386 - in head/sys: dev/bge dev/pci dev/twa x86/iommu Message-ID: <5c43013c-1fc6-57c2-6dec-1fdfc5213fb3@FreeBSD.org> In-Reply-To: <CAFMmRNywSPckXRi0oATi3AL%2BBX2MBUQd7t36TCncWiPf06%2BbkQ@mail.gmail.com> References: <201904191343.x3JDhYVF010453@repo.freebsd.org> <CAFMmRNxaJ6gWaFpxMFdYx_T2%2BjTRGrvZpvptXFgDr=Z-Zj2=Eg@mail.gmail.com> <CAFMmRNywSPckXRi0oATi3AL%2BBX2MBUQd7t36TCncWiPf06%2BbkQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
I had looked for the aac change, but wasn't able to find it, perhaps because I looked at tags created in aac.c rather than aac_pci.c. I agree aac will need to be re-patched. I'm not really certain how many other devices are actually broken. They would all be due to a firmware bug, nothing inherent in PCI. I believe twa(4) and bge(4) issues predated aac(4) FWIW. Unfortunately, the main bit of discussion about moving the limit into the PCI bus itself seems to be an IRC discussion on 2/28/12 that resulted in revision r232267 as a quick MFC'able fix, but I don't have a log of that conversation. :( I couldn't find anything in e-mail either that was definitive for why this might have been inherent in PCI-e vs a few firmware writers having similar bugs. On 4/25/19 12:20 PM, Ryan Stone wrote: > Following up, this is what will have to be re-instated in the aac driver: > > http://svn.freebsd.org/changeset/base/232260 > > However, my biggest concern is that we have no idea how many new > devices with the broken behaviour might have been introduced since we > fixed the problem in general. How does Linux handle the issue? > > On Thu, Apr 25, 2019 at 3:17 PM Ryan Stone <rysto32@gmail.com> wrote: >> >> This change makes me *very* uncomfortable. It was originally brought >> in due to issues with Adaptec RAID cards using the aac(9) driver. The >> symptoms of the bug included silent corruption of data as it was >> written to disk. Are we sure that this change is a good idea, given >> how catastrophic it is when a device gets this wrong? >> >> On Fri, Apr 19, 2019 at 9:43 AM Tycho Nightingale <tychon@freebsd.org> wrote: >>> >>> Author: tychon >>> Date: Fri Apr 19 13:43:33 2019 >>> New Revision: 346386 >>> URL: https://svnweb.freebsd.org/changeset/base/346386 >>> >>> Log: >>> remove the 4GB boundary requirement on PCI DMA segments >>> >>> Reviewed by: kib >>> Discussed with: jhb >>> Sponsored by: Dell EMC Isilon >>> Differential Revision: https://reviews.freebsd.org/D19867 >>> >>> Modified: >>> head/sys/dev/bge/if_bgereg.h >>> head/sys/dev/pci/pci.c >>> head/sys/dev/pci/pcivar.h >>> head/sys/dev/twa/tw_osl.h >>> head/sys/dev/twa/tw_osl_freebsd.c >>> head/sys/x86/iommu/intel_ctx.c >>> >>> Modified: head/sys/dev/bge/if_bgereg.h >>> ============================================================================== >>> --- head/sys/dev/bge/if_bgereg.h Fri Apr 19 13:23:41 2019 (r346385) >>> +++ head/sys/dev/bge/if_bgereg.h Fri Apr 19 13:43:33 2019 (r346386) >>> @@ -3067,3 +3067,11 @@ struct bge_softc { >>> #define BGE_LOCK_ASSERT(_sc) mtx_assert(&(_sc)->bge_mtx, MA_OWNED) >>> #define BGE_UNLOCK(_sc) mtx_unlock(&(_sc)->bge_mtx) >>> #define BGE_LOCK_DESTROY(_sc) mtx_destroy(&(_sc)->bge_mtx) >>> + >>> +#ifdef BUS_SPACE_MAXADDR >>> +#if (BUS_SPACE_MAXADDR > 0xFFFFFFFF) >>> +#define BGE_DMA_BOUNDARY (0x100000000) >>> +#else >>> +#define BGE_DMA_BOUNDARY 0 >>> +#endif >>> +#endif >>> >>> Modified: head/sys/dev/pci/pci.c >>> ============================================================================== >>> --- head/sys/dev/pci/pci.c Fri Apr 19 13:23:41 2019 (r346385) >>> +++ head/sys/dev/pci/pci.c Fri Apr 19 13:43:33 2019 (r346386) >>> @@ -4343,9 +4343,6 @@ pci_attach_common(device_t dev) >>> { >>> struct pci_softc *sc; >>> int busno, domain; >>> -#ifdef PCI_DMA_BOUNDARY >>> - int error, tag_valid; >>> -#endif >>> #ifdef PCI_RES_BUS >>> int rid; >>> #endif >>> @@ -4365,23 +4362,7 @@ pci_attach_common(device_t dev) >>> if (bootverbose) >>> device_printf(dev, "domain=%d, physical bus=%d\n", >>> domain, busno); >>> -#ifdef PCI_DMA_BOUNDARY >>> - tag_valid = 0; >>> - if (device_get_devclass(device_get_parent(device_get_parent(dev))) != >>> - devclass_find("pci")) { >>> - error = bus_dma_tag_create(bus_get_dma_tag(dev), 1, >>> - PCI_DMA_BOUNDARY, BUS_SPACE_MAXADDR, BUS_SPACE_MAXADDR, >>> - NULL, NULL, BUS_SPACE_MAXSIZE, BUS_SPACE_UNRESTRICTED, >>> - BUS_SPACE_MAXSIZE, 0, NULL, NULL, &sc->sc_dma_tag); >>> - if (error) >>> - device_printf(dev, "Failed to create DMA tag: %d\n", >>> - error); >>> - else >>> - tag_valid = 1; >>> - } >>> - if (!tag_valid) >>> -#endif >>> - sc->sc_dma_tag = bus_get_dma_tag(dev); >>> + sc->sc_dma_tag = bus_get_dma_tag(dev); >>> return (0); >>> } >>> >>> >>> Modified: head/sys/dev/pci/pcivar.h >>> ============================================================================== >>> --- head/sys/dev/pci/pcivar.h Fri Apr 19 13:23:41 2019 (r346385) >>> +++ head/sys/dev/pci/pcivar.h Fri Apr 19 13:43:33 2019 (r346386) >>> @@ -693,14 +693,6 @@ int pcie_link_reset(device_t port, int pcie_location); >>> >>> void pci_print_faulted_dev(void); >>> >>> -#ifdef BUS_SPACE_MAXADDR >>> -#if (BUS_SPACE_MAXADDR > 0xFFFFFFFF) >>> -#define PCI_DMA_BOUNDARY 0x100000000 >>> -#else >>> -#define PCI_DMA_BOUNDARY 0 >>> -#endif >>> -#endif >>> - >>> #endif /* _SYS_BUS_H_ */ >>> >>> /* >>> >>> Modified: head/sys/dev/twa/tw_osl.h >>> ============================================================================== >>> --- head/sys/dev/twa/tw_osl.h Fri Apr 19 13:23:41 2019 (r346385) >>> +++ head/sys/dev/twa/tw_osl.h Fri Apr 19 13:43:33 2019 (r346386) >>> @@ -57,6 +57,12 @@ >>> #define TW_OSLI_MAX_NUM_IOS (TW_OSLI_MAX_NUM_REQUESTS - 2) >>> #define TW_OSLI_MAX_NUM_AENS 0x100 >>> >>> +#ifdef PAE >>> +#define TW_OSLI_DMA_BOUNDARY (1u << 31) >>> +#else >>> +#define TW_OSLI_DMA_BOUNDARY ((bus_size_t)((uint64_t)1 << 32)) >>> +#endif >>> + >>> /* Possible values of req->state. */ >>> #define TW_OSLI_REQ_STATE_INIT 0x0 /* being initialized */ >>> #define TW_OSLI_REQ_STATE_BUSY 0x1 /* submitted to CL */ >>> >>> Modified: head/sys/dev/twa/tw_osl_freebsd.c >>> ============================================================================== >>> --- head/sys/dev/twa/tw_osl_freebsd.c Fri Apr 19 13:23:41 2019 (r346385) >>> +++ head/sys/dev/twa/tw_osl_freebsd.c Fri Apr 19 13:43:33 2019 (r346386) >>> @@ -551,7 +551,7 @@ tw_osli_alloc_mem(struct twa_softc *sc) >>> /* Create the parent dma tag. */ >>> if (bus_dma_tag_create(bus_get_dma_tag(sc->bus_dev), /* parent */ >>> sc->alignment, /* alignment */ >>> - 0, /* boundary */ >>> + TW_OSLI_DMA_BOUNDARY, /* boundary */ >>> BUS_SPACE_MAXADDR, /* lowaddr */ >>> BUS_SPACE_MAXADDR, /* highaddr */ >>> NULL, NULL, /* filter, filterarg */ >>> >>> Modified: head/sys/x86/iommu/intel_ctx.c >>> ============================================================================== >>> --- head/sys/x86/iommu/intel_ctx.c Fri Apr 19 13:23:41 2019 (r346385) >>> +++ head/sys/x86/iommu/intel_ctx.c Fri Apr 19 13:43:33 2019 (r346386) >>> @@ -130,7 +130,7 @@ ctx_tag_init(struct dmar_ctx *ctx, device_t dev) >>> maxaddr = MIN(ctx->domain->end, BUS_SPACE_MAXADDR); >>> ctx->ctx_tag.common.ref_count = 1; /* Prevent free */ >>> ctx->ctx_tag.common.impl = &bus_dma_dmar_impl; >>> - ctx->ctx_tag.common.boundary = PCI_DMA_BOUNDARY; >>> + ctx->ctx_tag.common.boundary = 0; >>> ctx->ctx_tag.common.lowaddr = maxaddr; >>> ctx->ctx_tag.common.highaddr = maxaddr; >>> ctx->ctx_tag.common.maxsize = maxaddr; >>> > -- John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5c43013c-1fc6-57c2-6dec-1fdfc5213fb3>