From owner-freebsd-arm@FreeBSD.ORG Mon Aug 29 11:07:04 2011 Return-Path: Delivered-To: freebsd-arm@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8C29C1065693 for ; Mon, 29 Aug 2011 11:07:04 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 7B4538FC20 for ; Mon, 29 Aug 2011 11:07:04 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p7TB74J7089217 for ; Mon, 29 Aug 2011 11:07:04 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p7TB73MD089215 for freebsd-arm@FreeBSD.org; Mon, 29 Aug 2011 11:07:03 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 29 Aug 2011 11:07:03 GMT Message-Id: <201108291107.p7TB73MD089215@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-arm@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-arm@FreeBSD.org X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to the StrongARM Processor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Aug 2011 11:07:04 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o arm/158950 arm arm/sheevaplug fails fsx when mmap operations are enab o arm/156814 arm OpenRD Ultimate does not boot on DB-88F6XXX or SHEEVAP o arm/156496 arm [patch] Minor bugfixes and enhancements to mmc and mmc o arm/155894 arm [patch] Enable at91 booting from SDHC (high capacity) o arm/155214 arm [patch] MMC/SD IO slow on Atmel ARM with modern large o arm/154227 arm [geli] using GELI leads to panic on ARM o arm/154189 arm lang/perl5.12 doesn't build on arm o arm/153380 arm Panic / translation fault with wlan on ARM o arm/150581 arm [irq] Unknown error generates IRQ address decoding err o arm/149288 arm mail/dovecot causes panic during configure on Sheevapl o arm/134368 arm [patch] nslu2_led driver for the LEDs on the NSLU2 p arm/134338 arm [patch] Lock GPIO accesses on ixp425 12 problems total. From owner-freebsd-arm@FreeBSD.ORG Sat Sep 3 17:20:12 2011 Return-Path: Delivered-To: freebsd-arm@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A26AC1065675 for ; Sat, 3 Sep 2011 17:20:12 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 77F248FC1A for ; Sat, 3 Sep 2011 17:20:12 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p83HKCeN076305 for ; Sat, 3 Sep 2011 17:20:12 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p83HKCZl076304; Sat, 3 Sep 2011 17:20:12 GMT (envelope-from gnats) Resent-Date: Sat, 3 Sep 2011 17:20:12 GMT Resent-Message-Id: <201109031720.p83HKCZl076304@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-arm@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Ian Lepore Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0CED6106564A for ; Sat, 3 Sep 2011 17:19:20 +0000 (UTC) (envelope-from ilepore@damnhippie.dyndns.org) Received: from qmta06.emeryville.ca.mail.comcast.net (qmta06.emeryville.ca.mail.comcast.net [76.96.30.56]) by mx1.freebsd.org (Postfix) with ESMTP id E61638FC08 for ; Sat, 3 Sep 2011 17:19:19 +0000 (UTC) Received: from omta24.emeryville.ca.mail.comcast.net ([76.96.30.92]) by qmta06.emeryville.ca.mail.comcast.net with comcast id UG9d1h0031zF43QA6HKEf0; Sat, 03 Sep 2011 17:19:14 +0000 Received: from damnhippie.dyndns.org ([24.8.232.202]) by omta24.emeryville.ca.mail.comcast.net with comcast id UHM31h01y4NgCEG8kHM4ak; Sat, 03 Sep 2011 17:21:04 +0000 Received: from revolution.hippie.lan (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id p83HJHTt033984 for ; Sat, 3 Sep 2011 11:19:17 -0600 (MDT) (envelope-from ilepore@damnhippie.dyndns.org) Received: (from ilepore@localhost) by revolution.hippie.lan (8.14.4/8.14.4/Submit) id p83HJHHO098751; Sat, 3 Sep 2011 11:19:17 -0600 (MDT) (envelope-from ilepore) Message-Id: <201109031719.p83HJHHO098751@revolution.hippie.lan> Date: Sat, 3 Sep 2011 11:19:17 -0600 (MDT) From: Ian Lepore To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Cc: Subject: arm/160431: [patch] Disable interrupts during busdma cache sync operations. X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Ian Lepore List-Id: Porting FreeBSD to the StrongARM Processor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Sep 2011 17:20:12 -0000 >Number: 160431 >Category: arm >Synopsis: [patch] Disable interrupts during busdma cache sync operations. >Confidential: no >Severity: critical >Priority: high >Responsible: freebsd-arm >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Sat Sep 03 17:20:12 UTC 2011 >Closed-Date: >Last-Modified: >Originator: Ian Lepore >Release: FreeBSD 8.2-RC3 arm >Organization: none >Environment: FreeBSD dvb 8.2-RC3 FreeBSD 8.2-RC3 #49: Tue Feb 15 22:52:14 UTC 2011 root@revolution.hippie.lan:/usr/obj/arm/usr/src/sys/DVB arm >Description: Data can be corrupted when an interrupt occurs while busdma_sync_buf() is handling a buffer that partially overlaps a cache line. One scenario, seen in the real world, was a small IO buffer allocated in the same cache line as an instance of a struct intr_handler. The dma sync code copied the non-DMA data (the intr_handler struct) to a temporary buffer prior to the cache sync, then an interrupt occurs that results in setting the it_need flag in the struct. When control returns to the dma sync code it finishes by copying the saved partial cache line from the temporary buffer back to the intr_handler struct, restoring the it_need flag to zero, and resulting in a threaded interrupt handler not running as needed. Similar sequences can be imagined that would lead to corruption of either the DMA'd data or non-DMA data sharing the same cache line, depending on the timing of the interrupt, and I can't quite convince myself that the problem only occurs in this partial-cacheline-overlap scenario. For example, what happens if execution is in the middle of a cpu_dcache_wbinv_range() operation and an interrupt leads to a context switch wherein the pmap code decides to call cpu_dcache_inv_range()? So to be conservatively safe, this patch disables interrupts for the entire duration bus_dmamap_sync_buf(), not just when partial cache lines are being handled. >How-To-Repeat: It would be very difficult to set up a repeatable test of this condition. We were lucky (!) enough to have it happen repeatably enough to diagnose. >Fix: Problem was discovered in an 8.2 environment, but this diff is to -current. --- diff.tmp begins here --- --- busdma_machdep.c.orig 2010-03-11 14:16:54.000000000 -0700 +++ busdma_machdep.c 2011-09-03 10:15:16.000000000 -0600 @@ -1091,6 +1091,14 @@ static void bus_dmamap_sync_buf(void *buf, int len, bus_dmasync_op_t op) { char _tmp_cl[arm_dcache_align], _tmp_clend[arm_dcache_align]; + uint32_t intr; + + /* Interrupts MUST be disabled when handling partial cacheline flush + * and most likely should be disabled for all flushes. (I know for + * certain interrupts can cause failures on partial flushes, and suspect + * problems could also happen in other scenarios.) + */ + intr = intr_disable(); if ((op & BUS_DMASYNC_PREWRITE) && !(op & BUS_DMASYNC_PREREAD)) { cpu_dcache_wb_range((vm_offset_t)buf, len); @@ -1129,6 +1137,8 @@ bus_dmamap_sync_buf(void *buf, int len, arm_dcache_align - (((vm_offset_t)(buf) + len) & arm_dcache_align_mask)); } + + intr_restore(intr); } static void --- diff.tmp ends here --- >Release-Note: >Audit-Trail: >Unformatted: From owner-freebsd-arm@FreeBSD.ORG Sat Sep 3 19:40:09 2011 Return-Path: Delivered-To: freebsd-arm@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B394D1065670 for ; Sat, 3 Sep 2011 19:40:09 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 991458FC13 for ; Sat, 3 Sep 2011 19:40:09 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p83Je91B004191 for ; Sat, 3 Sep 2011 19:40:09 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p83Je9fo004190; Sat, 3 Sep 2011 19:40:09 GMT (envelope-from gnats) Date: Sat, 3 Sep 2011 19:40:09 GMT Message-Id: <201109031940.p83Je9fo004190@freefall.freebsd.org> To: freebsd-arm@FreeBSD.org From: Mark Tinguely Cc: Subject: Re: arm/160431: [patch] Disable interrupts during busdma cache sync operations. X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Mark Tinguely List-Id: Porting FreeBSD to the StrongARM Processor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Sep 2011 19:40:09 -0000 The following reply was made to PR arm/160431; it has been noted by GNATS. From: Mark Tinguely To: Ian Lepore Cc: FreeBSD-gnats-submit@FreeBSD.org Subject: Re: arm/160431: [patch] Disable interrupts during busdma cache sync operations. Date: Sat, 03 Sep 2011 14:05:32 -0500 On 9/3/2011 12:19 PM, Ian Lepore wrote: >> Number: 160431 >> Category: arm >> Synopsis: [patch] Disable interrupts during busdma cache sync operations. >> Confidential: no >> Severity: critical >> Priority: high >> Responsible: freebsd-arm >> State: open >> Quarter: >> Keywords: >> Date-Required: >> Class: sw-bug >> Submitter-Id: current-users >> Arrival-Date: Sat Sep 03 17:20:12 UTC 2011 >> Closed-Date: >> Last-Modified: >> Originator: Ian Lepore >> Release: FreeBSD 8.2-RC3 arm >> Organization: > none >> Environment: > FreeBSD dvb 8.2-RC3 FreeBSD 8.2-RC3 #49: Tue Feb 15 22:52:14 UTC 2011 root@revolution.hippie.lan:/usr/obj/arm/usr/src/sys/DVB arm > >> Description: > Data can be corrupted when an interrupt occurs while busdma_sync_buf() is > handling a buffer that partially overlaps a cache line. One scenario, seen > in the real world, was a small IO buffer allocated in the same cache line > as an instance of a struct intr_handler. The dma sync code copied the non-DMA > data (the intr_handler struct) to a temporary buffer prior to the cache sync, > then an interrupt occurs that results in setting the it_need flag in the > struct. When control returns to the dma sync code it finishes by copying > the saved partial cache line from the temporary buffer back to the > intr_handler struct, restoring the it_need flag to zero, and resulting in > a threaded interrupt handler not running as needed. > > Similar sequences can be imagined that would lead to corruption of either > the DMA'd data or non-DMA data sharing the same cache line, depending on the > timing of the interrupt, and I can't quite convince myself that the problem > only occurs in this partial-cacheline-overlap scenario. For example, what > happens if execution is in the middle of a cpu_dcache_wbinv_range() operation > and an interrupt leads to a context switch wherein the pmap code decides to > call cpu_dcache_inv_range()? So to be conservatively safe, this patch > disables interrupts for the entire duration bus_dmamap_sync_buf(), not just > when partial cache lines are being handled. > >> How-To-Repeat: > It would be very difficult to set up a repeatable test of this condition. We > were lucky (!) enough to have it happen repeatably enough to diagnose. > >> Fix: > Problem was discovered in an 8.2 environment, but this diff is to -current. > > --- diff.tmp begins here --- > --- busdma_machdep.c.orig 2010-03-11 14:16:54.000000000 -0700 > +++ busdma_machdep.c 2011-09-03 10:15:16.000000000 -0600 > @@ -1091,6 +1091,14 @@ static void > bus_dmamap_sync_buf(void *buf, int len, bus_dmasync_op_t op) > { > char _tmp_cl[arm_dcache_align], _tmp_clend[arm_dcache_align]; > + uint32_t intr; > + > + /* Interrupts MUST be disabled when handling partial cacheline flush > + * and most likely should be disabled for all flushes. (I know for > + * certain interrupts can cause failures on partial flushes, and suspect > + * problems could also happen in other scenarios.) > + */ > + intr = intr_disable(); > > if ((op& BUS_DMASYNC_PREWRITE)&& !(op& BUS_DMASYNC_PREREAD)) { > cpu_dcache_wb_range((vm_offset_t)buf, len); > @@ -1129,6 +1137,8 @@ bus_dmamap_sync_buf(void *buf, int len, > arm_dcache_align - (((vm_offset_t)(buf) + len)& > arm_dcache_align_mask)); > } > + > + intr_restore(intr); > } > > static void > --- diff.tmp ends here --- > >> Release-Note: >> Audit-Trail: >> Unformatted: > _______________________________________________ > Which processor are you using (for my curiosity)? If this is easily reproducible, would you please put the interrupt disable/restore just around the BUS_DMASYNC_POSTREAD option? (for my curiosity again). Thank-you --Mark. From owner-freebsd-arm@FreeBSD.ORG Sat Sep 3 20:00:23 2011 Return-Path: Delivered-To: freebsd-arm@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E3DFB106566B for ; Sat, 3 Sep 2011 20:00:23 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id BABD68FC13 for ; Sat, 3 Sep 2011 20:00:23 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p83K0N4u021253 for ; Sat, 3 Sep 2011 20:00:23 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p83K0N75021242; Sat, 3 Sep 2011 20:00:23 GMT (envelope-from gnats) Date: Sat, 3 Sep 2011 20:00:23 GMT Message-Id: <201109032000.p83K0N75021242@freefall.freebsd.org> To: freebsd-arm@FreeBSD.org From: Ian Lepore Cc: Subject: Re: arm/160431: [patch] Disable interrupts during busdma cache sync operations. X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Ian Lepore List-Id: Porting FreeBSD to the StrongARM Processor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Sep 2011 20:00:24 -0000 The following reply was made to PR arm/160431; it has been noted by GNATS. From: Ian Lepore To: Mark Tinguely Cc: FreeBSD-gnats-submit@FreeBSD.org Subject: Re: arm/160431: [patch] Disable interrupts during busdma cache sync operations. Date: Sat, 03 Sep 2011 13:43:38 -0600 On Sat, 2011-09-03 at 14:05 -0500, Mark Tinguely wrote: > On 9/3/2011 12:19 PM, Ian Lepore wrote: > > [bug report] > > Which processor are you using (for my curiosity)? > > If this is easily reproducible, would you please put the interrupt > disable/restore just around the BUS_DMASYNC_POSTREAD option? (for my > curiosity again). > > Thank-you > > --Mark. It's an Atmel at91rm9200. It's been weeks since we were actively working this problem (I'm just way behind on submitting fixes back to the community), so it would be pretty hard at this point to get back to where the problem is easily reproducible. But when we were still investigating it, I remember instrumenting the code to conclusively prove it was the handling of partially-overlapping cache lines within the POSTREAD block that was leading to the ih_need flag of the nearby intr_handler struct getting reset to zero. We actually discovered all this on code that was mostly 6.2 with some 6.4 stuff merged in. We're now almost up and running on 8.2 on our embedded arm products, so the whole 6.2 thing is feeling like a bad nightmare that won't quite fade from memory. :) My putting the intr_disable() at the next-outer layer of code is just an abundance of caution. But given that the pmap code and the busdma code can both lead to writeback and/or invalidate activity, and that those two pieces of code don't know what each other are doing, I've had a growing quesy feeling about this stuff for a while. For example, the pmap code can decide to do a writeback that could overwrite DMA-in-progress data, especially in the cases of partial overlap of a DMA operation with a cache line. But I didn't want to start raising any alarms until I've learned more about the code in its current form, and I'm still working on learning that. -- Ian