From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 14 11:44:05 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1DCB9106 for ; Sat, 14 Mar 2015 11:44:05 +0000 (UTC) Received: from mail-wi0-f174.google.com (mail-wi0-f174.google.com [209.85.212.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9E5F27C7 for ; Sat, 14 Mar 2015 11:44:03 +0000 (UTC) Received: by wibg7 with SMTP id g7so5771467wib.1 for ; Sat, 14 Mar 2015 04:44:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type; bh=OiaYJj7g6UA3x0QA32/lKsLIIoTFZZqLGO4j2/0pcr0=; b=kKEcmIWKqASN8m/M6i/PXYQ1jz2qkyU+p0s0andYIzx9D3/JmMstLoX0/rw2SvJ30Q +p9GR3esgDFA2uae0btWe7urMOaDbQ8HWCVh2hDWTwso6DAyHe4m04rVxUUnLyf5/Fvo 8ZCoCoA0v4cY1tOIHdNURupkpEvNswkzxV4umOhO2lMk6eotMAg+BnNCeUq1HrEiFKKu t3p/5hD2zNiflyx5S27b6HuJ404fl4rx9trXJz6RwnNPyyzubwSa6nUkXOmdhLyIcvSt VfLHSo20Yx3JoDMTZg4tuABAZzUU7obp5RpLEcD69it0PEnOgS0P3OMPsapbNQ4dJwR7 pLog== X-Gm-Message-State: ALoCoQkwxeldEpV5AEQZg+zbrrmjtBwdKoQ1a/R5crgS/UrqweckUDnkozWJZ6GaPn7ioIs5qCU+ X-Received: by 10.180.106.197 with SMTP id gw5mr56477615wib.58.1426333441925; Sat, 14 Mar 2015 04:44:01 -0700 (PDT) Received: from [10.10.1.68] (82-69-141-170.dsl.in-addr.zen.co.uk. [82.69.141.170]) by mx.google.com with ESMTPSA id vq9sm6711077wjc.6.2015.03.14.04.44.00 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 14 Mar 2015 04:44:00 -0700 (PDT) Message-ID: <55041EF5.9080200@multiplay.co.uk> Date: Sat, 14 Mar 2015 11:43:49 +0000 From: Steven Hartland User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Michael Fuckner , Ryan Stone Subject: Re: Server with 3TB Crashing at boot References: <550046F7.1050205@fuckner.net> <550093DD.5040603@freebsd.org> <55015D3B.6030605@fuckner.net> <5501DD57.7030305@freebsd.org> <5503234A.4060103@fuckner.net> <55032C1B.5030004@multiplay.co.uk> <275339388.395931.1426279324879.JavaMail.open-xchange@ptangptang.store> <5503DC66.40409@fuckner.net> In-Reply-To: <5503DC66.40409@fuckner.net> Content-Type: multipart/mixed; boundary="------------090702070101020508050506" Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Mar 2015 11:44:05 -0000 This is a multi-part message in MIME format. --------------090702070101020508050506 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit On 14/03/2015 06:59, Michael Fuckner wrote: > On 3/13/2015 10:17 PM, Ryan Stone wrote: >> On Fri, Mar 13, 2015 at 4:42 PM, Michael Fuckner >> wrote: >> >>> Now I can kldload zfs without exploding kernel. I'll do some more >>> tests >>> tomorrow, but this looks fine! >>> >> >> Excellent news! I'd be interested to know whether this fixes the panics >> that you saw when zfs.ko was loaded by the bootloader. It's definitely >> possible, as the symptoms of this bug are likely to be random memory >> corruption after zfs initializes, but your crash happened pretty >> early on >> and I'm not sure whether zfs would have had a chance to do anything that >> early. >> >> Thanks for all of the work that you did to debug this. > > > Currently there is another issue that prevents me from testing ZFS: > only one HBA gets initialized. > > mpr0: 9300-8i with 8x Intel S3700 > mpr1: 9300-4i4e with 4x Intel S3700 and an external JBOD with 24HDD. > > mpr0 initializes fine, mpr1 fails > > root@s4l:~ # dmesg |grep mpr > mpr1: port 0xf000-0xf0ff mem 0xfb100000-0xfb10ffff irq > 112 at device 0.0 on pci195 > mpr1: IOCFacts : > mpr1: Firmware: 07.00.01.00, Driver: 05.255.05.00-fbsd > mpr1: IOCCapabilities: > 7a85c > mpr1: Cannot allocate queues memory > mpr1: mpr_iocfacts_allocate failed to alloc queues with error 12 > mpr1: mpr_attach IOC Facts based allocation failed with error 12 > device_attach: mpr1 attach returned 12 > > Any idea? Should I post to freebsd-scsi? Thats failing in bus_dmamem_alloc so as a guess I'd say there's no space below the 4GB boundary for the allocation of size qsize. The attached patch will print out the maxsize of the attempted allocation which may be interesting, although I wouldn't expect it to be anything strange as its pretty much device specific and not connected to total memory that I can see. Given that I suspect something else in the earlier part of the boot process has allocated a large chunk of memory making available space below 4GB scarce. A verbose boot up to this point may provide some indication as to this. Regards Steve --------------090702070101020508050506 Content-Type: text/x-patch; name="mpr.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="mpr.patch" --- sys/dev/mpr/mpr.c.orig 2015-03-14 11:32:43.256662663 +0000 +++ sys/dev/mpr/mpr.c 2015-03-14 11:33:23.682658734 +0000 @@ -1125,7 +1125,7 @@ mpr_alloc_queues(struct mpr_softc *sc) } if (bus_dmamem_alloc(sc->queues_dmat, (void **)&queues, BUS_DMA_NOWAIT, &sc->queues_map)) { - device_printf(sc->mpr_dev, "Cannot allocate queues memory\n"); + device_printf(sc->mpr_dev, "Cannot allocate queues size %d memory\n", qsize); return (ENOMEM); } bzero(queues, qsize); --------------090702070101020508050506--