From owner-freebsd-scsi@freebsd.org Sun Jun 3 21:00:58 2018 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 49E91FD53E7 for ; Sun, 3 Jun 2018 21:00:58 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id CF8387C51C for ; Sun, 3 Jun 2018 21:00:57 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 943D1FD53E2; Sun, 3 Jun 2018 21:00:57 +0000 (UTC) Delivered-To: scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7F79BFD53D9 for ; Sun, 3 Jun 2018 21:00:57 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E19457C514 for ; Sun, 3 Jun 2018 21:00:56 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 38B3C2CCD2 for ; Sun, 3 Jun 2018 21:00:56 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w53L0uDO086977 for ; Sun, 3 Jun 2018 21:00:56 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w53L0uFD086967 for scsi@FreeBSD.org; Sun, 3 Jun 2018 21:00:56 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201806032100.w53L0uFD086967@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: scsi@FreeBSD.org Subject: Problem reports for scsi@FreeBSD.org that need special attention Date: Sun, 3 Jun 2018 21:00:56 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jun 2018 21:00:58 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 221952 | cam iosched: Fix trim statistics 1 problems total for which you should take action. From owner-freebsd-scsi@freebsd.org Mon Jun 4 04:14:44 2018 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54D6BFEB755 for ; Mon, 4 Jun 2018 04:14:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id E14626F3A9 for ; Mon, 4 Jun 2018 04:14:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 97973FEB754; Mon, 4 Jun 2018 04:14:43 +0000 (UTC) Delivered-To: scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 702F2FEB753 for ; Mon, 4 Jun 2018 04:14:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0A9276F3A5 for ; Mon, 4 Jun 2018 04:14:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 46D808A53 for ; Mon, 4 Jun 2018 04:14:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w544EgqW065542 for ; Mon, 4 Jun 2018 04:14:42 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w544EgCb065540 for scsi@FreeBSD.org; Mon, 4 Jun 2018 04:14:42 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: scsi@FreeBSD.org Subject: [Bug 219857] panic in scsi_cd code Date: Mon, 04 Jun 2018 04:14:42 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: decui@microsoft.com X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: scsi@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jun 2018 04:14:44 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D219857 --- Comment #3 from Dexuan Cui --- Our test team sees a similar call-trace: panic: sleepq_add: td 0xfffff8010581a000 to sleep on wchan 0xfffff8010f9240= 48 with sleeping prohibited cpuid =3D 1 time =3D 1527752111 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0000581= 5e0 vpanic() at vpanic+0x1a3/frame 0xfffffe0000581640 doadump() at doadump/frame 0xfffffe00005816c0 sleepq_add() at sleepq_add+0x378/frame 0xfffffe0000581710 _sleep() at _sleep+0x2c3/frame 0xfffffe00005817b0 cam_periph_runccb() at cam_periph_runccb+0x1ad/frame 0xfffffe0000581900 cdcheckmedia() at cdcheckmedia+0x8a/frame 0xfffffe0000581980 cdstrategy() at cdstrategy+0x55/frame 0xfffffe00005819b0 g_disk_start() at g_disk_start+0x48e/frame 0xfffffe0000581a20 g_io_schedule_down() at g_io_schedule_down+0x105/frame 0xfffffe0000581a60 g_down_procbody() at g_down_procbody+0x6d/frame 0xfffffe0000581a70 fork_exit() at fork_exit+0x84/frame 0xfffffe0000581ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000581ab0 --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- KDB: enter: panic [ thread pid 13 tid 100113 ] Stopped at kdb_enter+0x3b: movq $0,kdb_why db>=20 But our test team is just running some fio test with RAID (we run "kldload geom_stripe). I'm not sure why our test is triggering cdstrategy() and cdcheckmedia(), which seems to be related to CD? Our test team says the pan= ic happens to FreeBSD 10.3 and the latest HEAD code, meaning this should be a longstanding issue... Anyway, do we have a plan to fix the panic originally reported? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-scsi@freebsd.org Mon Jun 4 06:23:58 2018 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2ECE4FD13F9 for ; Mon, 4 Jun 2018 06:23:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id BC31D7554E for ; Mon, 4 Jun 2018 06:23:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 77E7EFD13F4; Mon, 4 Jun 2018 06:23:57 +0000 (UTC) Delivered-To: scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 651FDFD13F3 for ; Mon, 4 Jun 2018 06:23:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F07F875548 for ; Mon, 4 Jun 2018 06:23:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 2CBA59C4B for ; Mon, 4 Jun 2018 06:23:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w546Nul6011810 for ; Mon, 4 Jun 2018 06:23:56 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w546Nuwn011809 for scsi@FreeBSD.org; Mon, 4 Jun 2018 06:23:56 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: scsi@FreeBSD.org Subject: [Bug 219857] panic in scsi_cd code Date: Mon, 04 Jun 2018 06:23:56 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: scsi@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jun 2018 06:23:58 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D219857 --- Comment #4 from Andriy Gapon --- (In reply to Dexuan Cui from comment #3) > I'm not sure why our test is triggering cdstrategy() and cdcheckmedia(),= which seems to be related to CD? It's probably because the new geom class wants to read media to check for on-disk metadata and it ends up trying to read from the empty CD drive too. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-scsi@freebsd.org Mon Jun 4 10:52:04 2018 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A1438FF3E8C for ; Mon, 4 Jun 2018 10:52:04 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 3CEA67FF7E for ; Mon, 4 Jun 2018 10:52:04 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: by mailman.ysv.freebsd.org (Postfix) id F0302FF3E8B; Mon, 4 Jun 2018 10:52:03 +0000 (UTC) Delivered-To: scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CB47BFF3E8A for ; Mon, 4 Jun 2018 10:52:03 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FAEA7FF76 for ; Mon, 4 Jun 2018 10:52:03 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (mh0.gentlemail.de [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id w54Aq0Nk018668; Mon, 4 Jun 2018 12:52:00 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 0CD25797; Mon, 4 Jun 2018 12:51:59 +0200 (CEST) Subject: =?UTF-8?Q?What_is_ENXIO_=e2=80=93_MSI_allocation_regression_in_:[Wa?= =?UTF-8?Q?s_Re:_svn_commit:_r321714_-_in_head/sys/dev:_mpr_mps]?= From: Harry Schmalzbauer To: Scott Long , scsi@freebsd.org References: <201707300653.v6U6rwLN099096@repo.freebsd.org> <597DA578.6030101@omnilan.de> <597F56A8.1060603@omnilan.de> <59804C8C.1020003@omnilan.de> Organization: OmniLAN Message-ID: Date: Mon, 4 Jun 2018 12:51:59 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <59804C8C.1020003@omnilan.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Mon, 04 Jun 2018 12:52:00 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jun 2018 10:52:04 -0000 Am 01.08.2017 um 11:40 schrieb Harry Schmalzbauer: > Bezüglich Scott Long's Nachricht vom 31.07.2017 18:56 (localtime): > > … >>> I'd like to report one I hadn't expected: >>> >>> mps0: port 0x4000-0x40ff mem 0xc3bc0000-0xc3bc3fff,0xc3b80000-0xc3bbffff irq 19 at device 0.0 on pci7 >>> >>> mps0: Firmware: 20.00.04.00, Driver: 21.02.00.00-fbsd >>> mps0: IOCCapabilities: >>> 185c >>> mps0: Cannot allocate INTx interrupt >>> mps0: mps_iocfacts_allocate failed to setup interrupts >>> mps0: mps_attach IOC Facts based allocation failed with error 6 >>> panic: resource_list_release: resource entry is not busy >>> cpuid = 6 >>> KDB: stack backtrace: >>> #0 0xffffffff805e32d7 at kdb_backtrace+0x67 >>> #1 0xffffffff805a1d26 at vpanic+0x186 >>> #2 0xffffffff805a1b93 at panic+0x43 >>> #3 0xffffffff805d71c6 at resource_list_release+0x1c6 >>> #4 0xffffffff8040fef1 at mps_pci_free+0xe1 >>> #5 0xffffffff8040fa23 at mps_pci_attach+0x1b3 >>> #6 0xffffffff805d6594 at device_attach+0x3a4 >>> #7 0xffffffff805d774d at bus_generic_attach+0x3d >>> #8 0xffffffff8044ac05 at pci_attach+0xd5 >>> #9 0xffffffff805d6594 at device_attach+0x3a4 >>> #10 0xffffffff805d774d at bus_generic_attach+0x3d >>> #11 0xffffffff80364761 at acpi_pcib_pci_attach+0xa1 >>> #12 0xffffffff805d6594 at device_attach+0x3a4 >>> #13 0xffffffff805d774d at bus_generic_attach+0x3d >>> #14 0xffffffff8044ac05 at pci_attach+0xd5 >>> #15 0xffffffff805d6594 at device_attach+0x3a4 >>> #16 0xffffffff805d774d at bus_generic_attach+0x3d >>> #17 0xffffffff80363e4d at acpi_pcib_acpi_attach+0x42d >>> Uptime: 1s > … > >> Fixed in r321799, thanks for the report. > Fix confiremd; merged together with r321733 (and 321737) to 11.1 and > panic vanished. Late in the 11.2 phase, I identified this commit as a regression for MSI (non-x) alloctaion. I have an idea what probably causes the problem here (INTx allocation, although MSI (and MSI-x) capability): disable_msix is not 0 (I need to disable MSI-x because of ESXi-passthru…). Corresponding lines: {         device_t dev;         int error, msgs;         dev = sc->mps_dev;         error = 0;         msgs = 0;         if ((sc->disable_msix == 0) &&             ((msgs = pci_msix_count(dev)) >= MPS_MSI_COUNT))                 error = mps_alloc_msix(sc, MPS_MSI_COUNT);         if ((error != 0) && (sc->disable_msi == 0) &&             ((msgs = pci_msi_count(dev)) >= MPS_MSI_COUNT))                 error = mps_alloc_msi(sc, MPS_MSI_COUNT);         if (error != 0)                 msgs = 0;         sc->msi_msgs = msgs;         return (error); } Before r321714, error was assigned ENXIO, which, if != 0, could help make me understand the problem. Unfortunately I have no idea what ENXIO means, where it's defined and most important, how to find the place where the declaration/definition happens.  Only joe and vi available here, any hints highly appreciated. I can confirm that MSI allocation works with mps.ko_21.02.00.00-fbsd-r321415 with my ESXi-passthru-non_msi-x setup. Although the dirver emits no message that an MSI was allocated, like toher drivers do.  That's a cosmetic one though. But the MSI->INTx regression is a severe one for me, which I'd like to fix myself but I'm missing so many fundamental skills :-( Thanks, -harry From owner-freebsd-scsi@freebsd.org Mon Jun 4 22:22:05 2018 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EF4F0FDB24F for ; Mon, 4 Jun 2018 22:22:04 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 6CC816A010 for ; Mon, 4 Jun 2018 22:22:04 +0000 (UTC) (envelope-from scottl@samsco.org) Received: by mailman.ysv.freebsd.org (Postfix) id 2B704FDB235; Mon, 4 Jun 2018 22:22:04 +0000 (UTC) Delivered-To: scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 04025FDB234 for ; Mon, 4 Jun 2018 22:22:04 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9A5796A00C for ; Mon, 4 Jun 2018 22:22:03 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 2FFAD21D4A; Mon, 4 Jun 2018 18:22:03 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Mon, 04 Jun 2018 18:22:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsco.org; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc; s=fm3; bh=nlbYyoAUIjYjK0Xu8BacoLzzgnVP+ 5yVzZyo/TIvl1M=; b=tQ0QKQJcTxQDccqycqekpFKOOHfWsj8SdKDq02vFNXndU ILa5awkw/Jlc5BECfhSoG4crhBYBKlXCObnHZB3tFk8WpihDP/fEl5L838ZYV+Rt ijaEsEBAyC4AyKHapd/ZUmwCBo8zc29i4iNNVpvGdOhfCZv0mbnSEc4c+9eNDE/W 8XlU8FVJKBovaLAPYS820+EyyQ+6DMiTS3GOBMjTBXSCtmJSL+M6tIc+U4LDw9A3 qU3aeH/ZpCTGSczLUirWRLg3WFjw2jYfJfpsrqUoUGlrJWY89iX3DQ38edQarjZH Edq7AalqRkDVov/rD6tkGBCos3Odeklk+V8Vpukuw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=nlbYyo AUIjYjK0Xu8BacoLzzgnVP+5yVzZyo/TIvl1M=; b=kBiD8PZQVBX3KIzqzclDlE aguN6o6EKWjdzuqkF4o7OymyH6lLcLFltJPyqcvIYA9Y61728TSuxpJfWqcYjXZY Gc7QkZyNQN1/Lrk+u9Q+klizAWHfpTV2bEzBxWJNzY5iNK2fzD8Kr7h4j4DdFSq5 ld6wUDqRI0OP4f0ofLA85gUbTdvJg9HkkGnWB9vsDQ0rgcu0713NNkJfkGuVacFY 094rMS20Bkt7WOgN8O27Bj6wWVyozg0TwKG1xeKxIP6nd7JyaetSC4qVmTaH0S2j HSfGpfSp2ZAUrFELXFocOiD1Rtss95/JgKX0Jkpm/VOOIGQjwgW3OZzv+2MLcy7Q == X-ME-Proxy: X-ME-Proxy: X-ME-Proxy: X-ME-Proxy: X-ME-Proxy: X-ME-Proxy: X-ME-Sender: Received: from [192.168.100.41] (unknown [50.227.106.226]) by mail.messagingengine.com (Postfix) with ESMTPA id 8C886E479F; Mon, 4 Jun 2018 18:22:02 -0400 (EDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.3 \(3445.6.18\)) Subject: =?utf-8?Q?Re=3A_What_is_ENXIO_=E2=80=93_MSI_allocation_regression?= =?utf-8?Q?_in_=3A=5BWas_Re=3A_svn_commit=3A_r321714_-_in_head/sys/dev=3A_?= =?utf-8?Q?mpr_mps=5D?= From: Scott Long In-Reply-To: Date: Mon, 4 Jun 2018 16:22:01 -0600 Cc: scsi@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <78611650-D7A4-4B1D-A254-DB058E1AC1C6@samsco.org> References: <201707300653.v6U6rwLN099096@repo.freebsd.org> <597DA578.6030101@omnilan.de> <597F56A8.1060603@omnilan.de> <59804C8C.1020003@omnilan.de> To: Harry Schmalzbauer X-Mailer: Apple Mail (2.3445.6.18) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jun 2018 22:22:05 -0000 > On Jun 4, 2018, at 4:51 AM, Harry Schmalzbauer = wrote: >=20 > Am 01.08.2017 um 11:40 schrieb Harry Schmalzbauer: >> Bez=C3=BCglich Scott Long's Nachricht vom 31.07.2017 18:56 = (localtime): >>=20 >> =E2=80=A6 >>>> I'd like to report one I hadn't expected: >>>>=20 >>>> mps0: port 0x4000-0x40ff mem = 0xc3bc0000-0xc3bc3fff,0xc3b80000-0xc3bbffff irq 19 at device 0.0 on pci7 >>>>=20 >>>> mps0: Firmware: 20.00.04.00, Driver: 21.02.00.00-fbsd >>>> mps0: IOCCapabilities: >>>> 185c >>>> mps0: Cannot allocate INTx interrupt >>>> mps0: mps_iocfacts_allocate failed to setup interrupts >>>> mps0: mps_attach IOC Facts based allocation failed with error 6 >>>> panic: resource_list_release: resource entry is not busy >>>> cpuid =3D 6 >>>> KDB: stack backtrace: >>>> #0 0xffffffff805e32d7 at kdb_backtrace+0x67 >>>> #1 0xffffffff805a1d26 at vpanic+0x186 >>>> #2 0xffffffff805a1b93 at panic+0x43 >>>> #3 0xffffffff805d71c6 at resource_list_release+0x1c6 >>>> #4 0xffffffff8040fef1 at mps_pci_free+0xe1 >>>> #5 0xffffffff8040fa23 at mps_pci_attach+0x1b3 >>>> #6 0xffffffff805d6594 at device_attach+0x3a4 >>>> #7 0xffffffff805d774d at bus_generic_attach+0x3d >>>> #8 0xffffffff8044ac05 at pci_attach+0xd5 >>>> #9 0xffffffff805d6594 at device_attach+0x3a4 >>>> #10 0xffffffff805d774d at bus_generic_attach+0x3d >>>> #11 0xffffffff80364761 at acpi_pcib_pci_attach+0xa1 >>>> #12 0xffffffff805d6594 at device_attach+0x3a4 >>>> #13 0xffffffff805d774d at bus_generic_attach+0x3d >>>> #14 0xffffffff8044ac05 at pci_attach+0xd5 >>>> #15 0xffffffff805d6594 at device_attach+0x3a4 >>>> #16 0xffffffff805d774d at bus_generic_attach+0x3d >>>> #17 0xffffffff80363e4d at acpi_pcib_acpi_attach+0x42d >>>> Uptime: 1s >> =E2=80=A6 >>=20 >>> Fixed in r321799, thanks for the report. >> Fix confiremd; merged together with r321733 (and 321737) to 11.1 and >> panic vanished. >=20 > Late in the 11.2 phase, I identified this commit as a regression for = MSI (non-x) alloctaion. > I have an idea what probably causes the problem here (INTx allocation, = although MSI (and MSI-x) capability): > disable_msix is not 0 (I need to disable MSI-x because of = ESXi-passthru=E2=80=A6). >=20 > Corresponding lines: > { > device_t dev; > int error, msgs; >=20 > dev =3D sc->mps_dev; > error =3D 0; > msgs =3D 0; >=20 > if ((sc->disable_msix =3D=3D 0) && > ((msgs =3D pci_msix_count(dev)) >=3D MPS_MSI_COUNT)) > error =3D mps_alloc_msix(sc, MPS_MSI_COUNT); > if ((error !=3D 0) && (sc->disable_msi =3D=3D 0) && > ((msgs =3D pci_msi_count(dev)) >=3D MPS_MSI_COUNT)) > error =3D mps_alloc_msi(sc, MPS_MSI_COUNT); > if (error !=3D 0) > msgs =3D 0; >=20 > sc->msi_msgs =3D msgs; > return (error); > } >=20 > Before r321714, error was assigned ENXIO, which, if !=3D 0, could help = make me understand the problem. > Unfortunately I have no idea what ENXIO means, where it's defined and = most important, how to find the place where the declaration/definition = happens. Only joe and vi available here, any hints highly appreciated. >=20 > I can confirm that MSI allocation works with = mps.ko_21.02.00.00-fbsd-r321415 with my ESXi-passthru-non_msi-x setup. > Although the dirver emits no message that an MSI was allocated, like = toher drivers do. That's a cosmetic one though. > But the MSI->INTx regression is a severe one for me, which I'd like to = fix myself but I'm missing so many fundamental skills :-( >=20 Hi Harry, You are correct about the bug. Please change the line at the top of the = function that reads error =3D 0; to error =3D ENXIO; Let me know if that fixes the MSI problem for you. Scott From owner-freebsd-scsi@freebsd.org Tue Jun 5 07:18:15 2018 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D1AA1FE81C7 for ; Tue, 5 Jun 2018 07:18:14 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 6950A856A1 for ; Tue, 5 Jun 2018 07:18:14 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: by mailman.ysv.freebsd.org (Postfix) id 22EFEFE81BF; Tue, 5 Jun 2018 07:18:14 +0000 (UTC) Delivered-To: scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F244EFE81BE for ; Tue, 5 Jun 2018 07:18:13 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7661F8569E for ; Tue, 5 Jun 2018 07:18:13 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (mh0.gentlemail.de [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id w557IA5u031476; Tue, 5 Jun 2018 09:18:10 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id E9B2C941; Tue, 5 Jun 2018 09:18:09 +0200 (CEST) Subject: =?UTF-8?Q?Re:_What_is_ENXIO_=e2=80=93_MSI_allocation_regression_in_?= =?UTF-8?Q?:[Was_Re:_svn_commit:_r321714_-_in_head/sys/dev:_mpr_mps]?= To: Scott Long Cc: scsi@freebsd.org References: <201707300653.v6U6rwLN099096@repo.freebsd.org> <597DA578.6030101@omnilan.de> <597F56A8.1060603@omnilan.de> <59804C8C.1020003@omnilan.de> <78611650-D7A4-4B1D-A254-DB058E1AC1C6@samsco.org> From: Harry Schmalzbauer Organization: OmniLAN Message-ID: Date: Tue, 5 Jun 2018 09:18:06 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <78611650-D7A4-4B1D-A254-DB058E1AC1C6@samsco.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Tue, 05 Jun 2018 09:18:10 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jun 2018 07:18:15 -0000 Am 05.06.2018 um 00:22 schrieb Scott Long: > > >> On Jun 4, 2018, at 4:51 AM, Harry Schmalzbauer wrote: >> >> Am 01.08.2017 um 11:40 schrieb Harry Schmalzbauer: >>> Bezüglich Scott Long's Nachricht vom 31.07.2017 18:56 (localtime): >>> >>> … >>>>> I'd like to report one I hadn't expected: >>>>> >>>>> mps0: port 0x4000-0x40ff mem 0xc3bc0000-0xc3bc3fff,0xc3b80000-0xc3bbffff irq 19 at device 0.0 on pci7 >>>>> >>>>> mps0: Firmware: 20.00.04.00, Driver: 21.02.00.00-fbsd >>>>> mps0: IOCCapabilities: >>>>> 185c >>>>> mps0: Cannot allocate INTx interrupt >>>>> mps0: mps_iocfacts_allocate failed to setup interrupts >>>>> mps0: mps_attach IOC Facts based allocation failed with error 6 >>>>> panic: resource_list_release: resource entry is not busy >>>>> cpuid = 6 >>>>> KDB: stack backtrace: >>>>> #0 0xffffffff805e32d7 at kdb_backtrace+0x67 >>>>> #1 0xffffffff805a1d26 at vpanic+0x186 >>>>> #2 0xffffffff805a1b93 at panic+0x43 >>>>> #3 0xffffffff805d71c6 at resource_list_release+0x1c6 >>>>> #4 0xffffffff8040fef1 at mps_pci_free+0xe1 >>>>> #5 0xffffffff8040fa23 at mps_pci_attach+0x1b3 >>>>> #6 0xffffffff805d6594 at device_attach+0x3a4 >>>>> #7 0xffffffff805d774d at bus_generic_attach+0x3d >>>>> #8 0xffffffff8044ac05 at pci_attach+0xd5 >>>>> #9 0xffffffff805d6594 at device_attach+0x3a4 >>>>> #10 0xffffffff805d774d at bus_generic_attach+0x3d >>>>> #11 0xffffffff80364761 at acpi_pcib_pci_attach+0xa1 >>>>> #12 0xffffffff805d6594 at device_attach+0x3a4 >>>>> #13 0xffffffff805d774d at bus_generic_attach+0x3d >>>>> #14 0xffffffff8044ac05 at pci_attach+0xd5 >>>>> #15 0xffffffff805d6594 at device_attach+0x3a4 >>>>> #16 0xffffffff805d774d at bus_generic_attach+0x3d >>>>> #17 0xffffffff80363e4d at acpi_pcib_acpi_attach+0x42d >>>>> Uptime: 1s >>> … >>> >>>> Fixed in r321799, thanks for the report. >>> Fix confiremd; merged together with r321733 (and 321737) to 11.1 and >>> panic vanished. >> >> Late in the 11.2 phase, I identified this commit as a regression for MSI (non-x) alloctaion. >> I have an idea what probably causes the problem here (INTx allocation, although MSI (and MSI-x) capability): >> disable_msix is not 0 (I need to disable MSI-x because of ESXi-passthru…). >> >> Corresponding lines: >> { >> device_t dev; >> int error, msgs; >> >> dev = sc->mps_dev; >> error = 0; >> msgs = 0; >> >> if ((sc->disable_msix == 0) && >> ((msgs = pci_msix_count(dev)) >= MPS_MSI_COUNT)) >> error = mps_alloc_msix(sc, MPS_MSI_COUNT); >> if ((error != 0) && (sc->disable_msi == 0) && >> ((msgs = pci_msi_count(dev)) >= MPS_MSI_COUNT)) >> error = mps_alloc_msi(sc, MPS_MSI_COUNT); >> if (error != 0) >> msgs = 0; >> >> sc->msi_msgs = msgs; >> return (error); >> } >> >> Before r321714, error was assigned ENXIO, which, if != 0, could help make me understand the problem. >> Unfortunately I have no idea what ENXIO means, where it's defined and most important, how to find the place where the declaration/definition happens. Only joe and vi available here, any hints highly appreciated. >> >> I can confirm that MSI allocation works with mps.ko_21.02.00.00-fbsd-r321415 with my ESXi-passthru-non_msi-x setup. >> Although the dirver emits no message that an MSI was allocated, like toher drivers do. That's a cosmetic one though. >> But the MSI->INTx regression is a severe one for me, which I'd like to fix myself but I'm missing so many fundamental skills :-( >> > > Hi Harry, > > You are correct about the bug. Please change the line at the top of the function that reads > > error = 0; > > to > > error = ENXIO; > > Let me know if that fixes the MSI problem for you. Hello Scott, thanks for your hint. Unfortunately I have a lot more problems – the system (11.2-RC1) deadlocks for some soconds with iSCSI load... This is far easyer reproducable / heavier impact with mps(4) and INTx allocation than with MSI, but backup runs over night triggered that extreme slowdown although mps(4) was allocating MSI – up to 20 sec locks, where even no terminal update happes. All those update ar queued though, so after about 10-20 sedonds, the screen flickers, showing all queued output. One symptom is that systat(1) shows 25% intr usage which is one core. It's a ZFS machine, so high sys usage is normal, but intr usually is about 10% with GbE traffic. Only when the slowdown/lockup happens, intr usage constantly stays at 25%. Can't imagine ctld(8) or zfs is causing this, but who knows – I don't at the moment. Will have to revert to 11.1 and see if things change, the machine was 10.? before – without such problems. BTW, does anybody have a link where I can get info about ENXIO? Thanks, -harry From owner-freebsd-scsi@freebsd.org Tue Jun 5 17:43:58 2018 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F60CFE86D2 for ; Tue, 5 Jun 2018 17:43:58 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from weser.webweaving.org (weser.webweaving.org [148.251.234.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "weser.webweaving.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 17EF086F15 for ; Tue, 5 Jun 2018 17:43:57 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from beeb.leiden.webweaving.org (5ED29A06.cm-7-3c.dynamic.ziggo.nl [94.210.154.6]) (authenticated bits=0) by weser.webweaving.org (8.15.2/8.15.2) with ESMTPSA id w55HSt3h056276 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 5 Jun 2018 19:28:56 +0200 (CEST) (envelope-from dirkx@webweaving.org) X-Authentication-Warning: weser.webweaving.org: Host 5ED29A06.cm-7-3c.dynamic.ziggo.nl [94.210.154.6] claimed to be beeb.leiden.webweaving.org Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: Scalar i40 tapechanger LUN appearing as a second tape drive From: Dirk-Willem van Gulik In-Reply-To: <64CB99E1-9A12-41EB-9D11-42C1AECED2F0@webweaving.org> Date: Tue, 5 Jun 2018 19:28:54 +0200 Cc: freebsd-scsi@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <47CCCC93-7168-47B8-8C80-248C90F8085C@webweaving.org> <20180515201939.GA35140@mithlond.kdm.org> <3E5345E0-2F97-47A8-8BFB-94523B65B0A4@webweaving.org> <20180516165307.GB35140@mithlond.kdm.org> <64CB99E1-9A12-41EB-9D11-42C1AECED2F0@webweaving.org> To: "Kenneth D. Merry" X-Mailer: Apple Mail (2.3273) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (weser.webweaving.org [148.251.234.232]); Tue, 05 Jun 2018 19:28:57 +0200 (CEST) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jun 2018 17:43:58 -0000 > On 16 May 2018, at 19:30, Dirk-Willem van Gulik = wrote: >=20 > On 16 May 2018, at 18:53, Kenneth D. Merry wrote: >=20 >>> Ok. good advice. Will do exactly that. >>=20 >> The best controllers to use are LSI (now Broadcom) 6Gb or 12Gb SAS >> controllers. They have TLR support, which helps insure data = integrity with >> tape drives. >=20 > Ok - we surely have a few SAS9201 and LSIA SAS9202=E2=80=99s around. Ok - So the summary is that 1) indeed the LSI controller see the Scalar = i40 changer with no issue (and gives us a nice speed update). And that = 2) that there may be some regression in the cam/scsi stack -- restoring = a FreeBSD-8-RELEASE makes the P222 see the changer again. Next up - finding out what is wrong with LEOM (and amanda). Dw. From owner-freebsd-scsi@freebsd.org Tue Jun 5 17:54:38 2018 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 17EDBFEA819 for ; Tue, 5 Jun 2018 17:54:38 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 9CB14877BB for ; Tue, 5 Jun 2018 17:54:37 +0000 (UTC) (envelope-from scottl@samsco.org) Received: by mailman.ysv.freebsd.org (Postfix) id 604A3FEA811; Tue, 5 Jun 2018 17:54:37 +0000 (UTC) Delivered-To: scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38BA1FEA810 for ; Tue, 5 Jun 2018 17:54:37 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CFCBB877BA for ; Tue, 5 Jun 2018 17:54:36 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id CED3F21C4B; Tue, 5 Jun 2018 13:54:35 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Tue, 05 Jun 2018 13:54:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsco.org; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc; s=fm3; bh=v5+q3MpqstpVp4EqldjOQp8nQmYyt 2Ia9FZUG0nOlXM=; b=gN6sENpy5Mh+6/fZUOz/bQ42GMNdGH+jHmXqj6ooznQlz slo4KeTv/Jcp4RBvdqFCTMznrSRM1xveS+o8tczI3fdLaAYLbWLExfhzT9G+C6ZY yvaQpyfXoqH5/dz3JXSJq9Y4+MIFm0bu3OT/kqE15u+zHVOUkY/laVGpfLXeAOqz Lnw5oxZSOViSsowi7PWdlHhzeKh9YDcAsdyrHEORvteNukItkQ13DOOaYt1cMvk3 8kTEJimx49a/uD/hk7qIVudskjpm2asaZwSR0GSw7inVGOaOQzTMqPGr0CKacO+3 0lcojbaDF3qYk0gRCyXQPEVubodlORQ9Mtog2xfdg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=v5+q3M pqstpVp4EqldjOQp8nQmYyt2Ia9FZUG0nOlXM=; b=WXcFD3CMkOKZ8BglI1cfUZ gkTIs8vHAf1P40850oVx6aedXwF0IhZwAui/j6v11Gu+Jq1XNbbip2Ckj4qW5pbn IIfB+W1IUY/LerlD+5OV/3jWVmzOtpZEI/AnqK6JScBq2m9ICND2AUkN+iXzyQ2p S0P0lFDVf/ZG+eBzJm+vnTyyIXV0egm+FddQhg48i+U7DAeMq7QcX3gY35hZsmC0 EoVvcwYMun0lUknV4+RxcgTNblU8cxxDh29DYIQq/tjm7nJBQ/a+nz0NuChaY1IC iLJIaPWW0qmJoL+G4odnIE+44A6bXJtd5AqT/2nWnrlpbG4stn/wfD353Ql+gGUQ == X-ME-Proxy: X-ME-Proxy: X-ME-Proxy: X-ME-Proxy: X-ME-Proxy: X-ME-Proxy: X-ME-Sender: Received: from [172.27.7.190] (unknown [75.104.68.209]) by mail.messagingengine.com (Postfix) with ESMTPA id A6632E4120; Tue, 5 Jun 2018 13:54:31 -0400 (EDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.3 \(3445.6.18\)) Subject: =?utf-8?Q?Re=3A_What_is_ENXIO_=E2=80=93_MSI_allocation_regression?= =?utf-8?Q?_in_=3A=5BWas_Re=3A_svn_commit=3A_r321714_-_in_head/sys/dev=3A_?= =?utf-8?Q?mpr_mps=5D?= From: Scott Long In-Reply-To: Date: Tue, 5 Jun 2018 11:54:32 -0600 Cc: scsi@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <201707300653.v6U6rwLN099096@repo.freebsd.org> <597DA578.6030101@omnilan.de> <597F56A8.1060603@omnilan.de> <59804C8C.1020003@omnilan.de> <78611650-D7A4-4B1D-A254-DB058E1AC1C6@samsco.org> To: Harry Schmalzbauer X-Mailer: Apple Mail (2.3445.6.18) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jun 2018 17:54:38 -0000 > On Jun 5, 2018, at 1:18 AM, Harry Schmalzbauer = wrote: >=20 > Am 05.06.2018 um 00:22 schrieb Scott Long: >>> On Jun 4, 2018, at 4:51 AM, Harry Schmalzbauer = wrote: >>>=20 >>> Am 01.08.2017 um 11:40 schrieb Harry Schmalzbauer: >>>> Bez=C3=BCglich Scott Long's Nachricht vom 31.07.2017 18:56 = (localtime): >>>>=20 >>>> =E2=80=A6 >>>>>> I'd like to report one I hadn't expected: >>>>>>=20 >>>>>> mps0: port 0x4000-0x40ff mem = 0xc3bc0000-0xc3bc3fff,0xc3b80000-0xc3bbffff irq 19 at device 0.0 on pci7 >>>>>>=20 >>>>>> mps0: Firmware: 20.00.04.00, Driver: 21.02.00.00-fbsd >>>>>> mps0: IOCCapabilities: >>>>>> 185c >>>>>> mps0: Cannot allocate INTx interrupt >>>>>> mps0: mps_iocfacts_allocate failed to setup interrupts >>>>>> mps0: mps_attach IOC Facts based allocation failed with error 6 >>>>>> panic: resource_list_release: resource entry is not busy >>>>>> cpuid =3D 6 >>>>>> KDB: stack backtrace: >>>>>> #0 0xffffffff805e32d7 at kdb_backtrace+0x67 >>>>>> #1 0xffffffff805a1d26 at vpanic+0x186 >>>>>> #2 0xffffffff805a1b93 at panic+0x43 >>>>>> #3 0xffffffff805d71c6 at resource_list_release+0x1c6 >>>>>> #4 0xffffffff8040fef1 at mps_pci_free+0xe1 >>>>>> #5 0xffffffff8040fa23 at mps_pci_attach+0x1b3 >>>>>> #6 0xffffffff805d6594 at device_attach+0x3a4 >>>>>> #7 0xffffffff805d774d at bus_generic_attach+0x3d >>>>>> #8 0xffffffff8044ac05 at pci_attach+0xd5 >>>>>> #9 0xffffffff805d6594 at device_attach+0x3a4 >>>>>> #10 0xffffffff805d774d at bus_generic_attach+0x3d >>>>>> #11 0xffffffff80364761 at acpi_pcib_pci_attach+0xa1 >>>>>> #12 0xffffffff805d6594 at device_attach+0x3a4 >>>>>> #13 0xffffffff805d774d at bus_generic_attach+0x3d >>>>>> #14 0xffffffff8044ac05 at pci_attach+0xd5 >>>>>> #15 0xffffffff805d6594 at device_attach+0x3a4 >>>>>> #16 0xffffffff805d774d at bus_generic_attach+0x3d >>>>>> #17 0xffffffff80363e4d at acpi_pcib_acpi_attach+0x42d >>>>>> Uptime: 1s >>>> =E2=80=A6 >>>>=20 >>>>> Fixed in r321799, thanks for the report. >>>> Fix confiremd; merged together with r321733 (and 321737) to 11.1 = and >>>> panic vanished. >>>=20 >>> Late in the 11.2 phase, I identified this commit as a regression for = MSI (non-x) alloctaion. >>> I have an idea what probably causes the problem here (INTx = allocation, although MSI (and MSI-x) capability): >>> disable_msix is not 0 (I need to disable MSI-x because of = ESXi-passthru=E2=80=A6). >>>=20 >>> Corresponding lines: >>> { >>> device_t dev; >>> int error, msgs; >>>=20 >>> dev =3D sc->mps_dev; >>> error =3D 0; >>> msgs =3D 0; >>>=20 >>> if ((sc->disable_msix =3D=3D 0) && >>> ((msgs =3D pci_msix_count(dev)) >=3D MPS_MSI_COUNT)) >>> error =3D mps_alloc_msix(sc, MPS_MSI_COUNT); >>> if ((error !=3D 0) && (sc->disable_msi =3D=3D 0) && >>> ((msgs =3D pci_msi_count(dev)) >=3D MPS_MSI_COUNT)) >>> error =3D mps_alloc_msi(sc, MPS_MSI_COUNT); >>> if (error !=3D 0) >>> msgs =3D 0; >>>=20 >>> sc->msi_msgs =3D msgs; >>> return (error); >>> } >>>=20 >>> Before r321714, error was assigned ENXIO, which, if !=3D 0, could = help make me understand the problem. >>> Unfortunately I have no idea what ENXIO means, where it's defined = and most important, how to find the place where the = declaration/definition happens. Only joe and vi available here, any = hints highly appreciated. >>>=20 >>> I can confirm that MSI allocation works with = mps.ko_21.02.00.00-fbsd-r321415 with my ESXi-passthru-non_msi-x setup. >>> Although the dirver emits no message that an MSI was allocated, like = toher drivers do. That's a cosmetic one though. >>> But the MSI->INTx regression is a severe one for me, which I'd like = to fix myself but I'm missing so many fundamental skills :-( >>>=20 >> Hi Harry, >> You are correct about the bug. Please change the line at the top of = the function that reads >> error =3D 0; >> to >> error =3D ENXIO; >> Let me know if that fixes the MSI problem for you. >=20 > Hello Scott, >=20 > thanks for your hint. > Unfortunately I have a lot more problems =E2=80=93 the system = (11.2-RC1) deadlocks for some soconds with iSCSI load... > This is far easyer reproducable / heavier impact with mps(4) and INTx = allocation than with MSI, but backup runs over night triggered that = extreme slowdown although mps(4) was allocating MSI =E2=80=93 up to 20 = sec locks, where even no terminal update happes. > All those update ar queued though, so after about 10-20 sedonds, the = screen flickers, showing all queued output. >=20 > One symptom is that systat(1) shows 25% intr usage which is one core. > It's a ZFS machine, so high sys usage is normal, but intr usually is = about 10% with GbE traffic. > Only when the slowdown/lockup happens, intr usage constantly stays at = 25%. >=20 > Can't imagine ctld(8) or zfs is causing this, but who knows =E2=80=93 = I don't at the moment. > Will have to revert to 11.1 and see if things change, the machine was = 10.? before =E2=80=93 without such problems. >=20 > BTW, does anybody have a link where I can get info about ENXIO? ENXIO means that the device is not available. I use it in the driver to = signal when the hardware cannot be accessed. The manual page for error = codes is =E2=80=9Cman errno" Scott From owner-freebsd-scsi@freebsd.org Tue Jun 5 18:07:21 2018 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 20917FECFD3 for ; Tue, 5 Jun 2018 18:07:21 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mithlond.kdm.org (mithlond.kdm.org [96.89.93.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mithlond.kdm.org", Issuer "mithlond.kdm.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B1C8E683A8 for ; Tue, 5 Jun 2018 18:07:20 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.15.2/8.14.9) with ESMTPS id w55I7IUj002796 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 5 Jun 2018 14:07:18 -0400 (EDT) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.15.2/8.14.9/Submit) id w55I7IjT002795; Tue, 5 Jun 2018 14:07:18 -0400 (EDT) (envelope-from ken) Date: Tue, 5 Jun 2018 14:07:18 -0400 From: "Kenneth D. Merry" To: Dirk-Willem van Gulik Cc: freebsd-scsi@freebsd.org Subject: Re: Scalar i40 tapechanger LUN appearing as a second tape drive Message-ID: <20180605180718.GA2459@mithlond.kdm.org> References: <47CCCC93-7168-47B8-8C80-248C90F8085C@webweaving.org> <20180515201939.GA35140@mithlond.kdm.org> <3E5345E0-2F97-47A8-8BFB-94523B65B0A4@webweaving.org> <20180516165307.GB35140@mithlond.kdm.org> <64CB99E1-9A12-41EB-9D11-42C1AECED2F0@webweaving.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Tue, 05 Jun 2018 14:07:19 -0400 (EDT) X-Spam-Status: No, score=-2.7 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mithlond.kdm.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jun 2018 18:07:21 -0000 On Tue, Jun 05, 2018 at 19:28:54 +0200, Dirk-Willem van Gulik wrote: > > > On 16 May 2018, at 19:30, Dirk-Willem van Gulik wrote: > > > > On 16 May 2018, at 18:53, Kenneth D. Merry wrote: > > > >>> Ok. good advice. Will do exactly that. > >> > >> The best controllers to use are LSI (now Broadcom) 6Gb or 12Gb SAS > >> controllers. They have TLR support, which helps insure data integrity with > >> tape drives. > > > > Ok - we surely have a few SAS9201 and LSIA SAS9202???s around. > > Ok - So the summary is that 1) indeed the LSI controller see the Scalar i40 changer with no issue (and gives us a nice speed update). And that 2) that there may be some regression in the cam/scsi stack -- restoring a FreeBSD-8-RELEASE makes the P222 see the changer again. > I'm glad things work with the LSI controller! That is strange that FreeBSD 8 works with the ciss(4) controller and 11.1 doesn't. I would guess that something changed in the ciss(4) driver, but I don't see anything really obvious in the logs. > Next up - finding out what is wrong with LEOM (and amanda). You're having problems with Amanda? Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-scsi@freebsd.org Tue Jun 5 20:26:44 2018 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4AE1FF526C for ; Tue, 5 Jun 2018 20:26:44 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from weser.webweaving.org (weser.webweaving.org [148.251.234.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "weser.webweaving.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5617F7063C for ; Tue, 5 Jun 2018 20:26:44 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from [192.168.2.134] (a83-163-200-191.adsl.xs4all.nl [83.163.200.191]) (authenticated bits=0) by weser.webweaving.org (8.15.2/8.15.2) with ESMTPSA id w55KPQkh061042 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 5 Jun 2018 22:25:26 +0200 (CEST) (envelope-from dirkx@webweaving.org) X-Authentication-Warning: weser.webweaving.org: Host a83-163-200-191.adsl.xs4all.nl [83.163.200.191] claimed to be [192.168.2.134] From: Dirk-Willem van Gulik Message-Id: Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: Scalar i40 tapechanger LUN appearing as a second tape drive Date: Tue, 5 Jun 2018 22:25:20 +0200 In-Reply-To: <20180605180718.GA2459@mithlond.kdm.org> Cc: freebsd-scsi@freebsd.org To: "Kenneth D. Merry" References: <47CCCC93-7168-47B8-8C80-248C90F8085C@webweaving.org> <20180515201939.GA35140@mithlond.kdm.org> <3E5345E0-2F97-47A8-8BFB-94523B65B0A4@webweaving.org> <20180516165307.GB35140@mithlond.kdm.org> <64CB99E1-9A12-41EB-9D11-42C1AECED2F0@webweaving.org> <20180605180718.GA2459@mithlond.kdm.org> X-Mailer: Apple Mail (2.3273) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (weser.webweaving.org [148.251.234.232]); Tue, 05 Jun 2018 22:25:26 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jun 2018 20:26:45 -0000 On 5 Jun 2018, at 20:07, Kenneth D. Merry wrote: >=20 >> Next up - finding out what is wrong with LEOM (and amanda). >=20 > You're having problems with Amanda? Somehow amtapetype does not detect LEOM on LTO5 with amanda 3.3.9. Which = seems a bit silly. Checking with 3.5.1 now (as ports stops at 3.3.9); just in case. Dw.= From owner-freebsd-scsi@freebsd.org Tue Jun 5 20:34:42 2018 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 792D7FF583C for ; Tue, 5 Jun 2018 20:34:42 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mithlond.kdm.org (mithlond.kdm.org [96.89.93.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mithlond.kdm.org", Issuer "mithlond.kdm.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 14BC470CBE for ; Tue, 5 Jun 2018 20:34:41 +0000 (UTC) (envelope-from ken@kdm.org) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.15.2/8.14.9) with ESMTPS id w55KYe9R005163 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 5 Jun 2018 16:34:40 -0400 (EDT) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.15.2/8.14.9/Submit) id w55KYe7F005162; Tue, 5 Jun 2018 16:34:40 -0400 (EDT) (envelope-from ken) Date: Tue, 5 Jun 2018 16:34:40 -0400 From: "Kenneth D. Merry" To: Dirk-Willem van Gulik Cc: freebsd-scsi@freebsd.org Subject: Re: Scalar i40 tapechanger LUN appearing as a second tape drive Message-ID: <20180605203439.GF2459@mithlond.kdm.org> References: <47CCCC93-7168-47B8-8C80-248C90F8085C@webweaving.org> <20180515201939.GA35140@mithlond.kdm.org> <3E5345E0-2F97-47A8-8BFB-94523B65B0A4@webweaving.org> <20180516165307.GB35140@mithlond.kdm.org> <64CB99E1-9A12-41EB-9D11-42C1AECED2F0@webweaving.org> <20180605180718.GA2459@mithlond.kdm.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Tue, 05 Jun 2018 16:34:40 -0400 (EDT) X-Spam-Status: No, score=-2.7 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mithlond.kdm.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jun 2018 20:34:42 -0000 On Tue, Jun 05, 2018 at 22:25:20 +0200, Dirk-Willem van Gulik wrote: > On 5 Jun 2018, at 20:07, Kenneth D. Merry wrote: > > > >> Next up - finding out what is wrong with LEOM (and amanda). > > > > You're having problems with Amanda? > > Somehow amtapetype does not detect LEOM on LTO5 with amanda 3.3.9. Which seems a bit silly. > Amanda shouldn't be doing anything unusual in reading the tape, but it's possible that its detection logic isn't quite right. For what it's worth, here's what I'm using in my amanda.conf as a tapetype with an IBM LTO-5 drive on 3.3.9: define tapetype LTO-5 { comment "LTO 5" length 2000000 mbytes filemark 0 kbytes speed 150000 kps part_size 150000 mbytes part_cache_type disk part_cache_dir "/scratch/amanda" part_cache_max_size 150000 mbytes } Note that the length is intentionally longer than the 1.5TB raw tape capacity. With compression, we get at least that or more, and that helps keep Amanda from thinking it needs too many tapes. > Checking with 3.5.1 now (as ports stops at 3.3.9); just in case. I would suggest just use the above tapetype or a derivative and don't worry about whether amtapetype is doing the right thing. Ken -- Kenneth Merry ken@FreeBSD.ORG