From owner-freebsd-arch@FreeBSD.ORG Sun Sep 2 02:08:35 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0FE961065673; Sun, 2 Sep 2012 02:08:35 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id CEFCB8FC08; Sun, 2 Sep 2012 02:08:34 +0000 (UTC) Received: by dadr6 with SMTP id r6so2723339dad.13 for ; Sat, 01 Sep 2012 19:08:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=NVp04Iyfui9YRlVUz0oZLhuiqRWJXM11LIJxuzYMohw=; b=q2t+d5xI2Yd8PlXoIEOHdOMaI4I6NfiYeGWm7Fm5LKZcvbcjl3XMSy+IHhtEfiQar+ nPiClH48FlllQ+ChS6ShnMnodsyw9QeCpXp+skFyNjQqtMupb01fddso8oiFPgk6woFL JqUjRuTfL9eYztv5P6arRLYMfl3JQRxpKsW37T0ZSufteQ3JAQIZ4ifuhullzU6iO12p gQxCxbA5J+UB35+RKGNNnbXEL1eDZfoH9iwvpXyspOX3uWDZzSeBwn+zGRagkH09vS03 nIGm3Xqvt8txzZwkcNwCwGQXNj9VDlV30zpGe6kOAEoF2KwW2nyBgTiEHDkBz5W5NzJS N34g== MIME-Version: 1.0 Received: by 10.68.136.40 with SMTP id px8mr27640406pbb.153.1346551708468; Sat, 01 Sep 2012 19:08:28 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.68.36.106 with HTTP; Sat, 1 Sep 2012 19:08:28 -0700 (PDT) In-Reply-To: <02C549DE-D912-4D1A-A08D-650B918E9B99@bsdimp.com> References: <20120831154241.82902856.ray@dlink.ua> <02C549DE-D912-4D1A-A08D-650B918E9B99@bsdimp.com> Date: Sat, 1 Sep 2012 19:08:28 -0700 X-Google-Sender-Auth: _Y5Or6HTU8E9Lb0BduYOrHMW_cY Message-ID: From: Adrian Chadd To: Warner Losh Content-Type: text/plain; charset=ISO-8859-1 Cc: Aleksandr Rybalko , arch@freebsd.org, embedded@freebsd.org Subject: Re: [RFC] hintmode switch patch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Sep 2012 02:08:35 -0000 Please commit. Approved by, etc. Adrian On 1 September 2012 16:13, Warner Losh wrote: > This patch looks good to me. It contained none of the stuff I was worried about before, so please commit this. > > Warner > > On Sep 1, 2012, at 8:01 AM, Adrian Chadd wrote: > >> Once others have stopped giving you reasons not to commit, please commit. >> >> >> >> Adrian >> >> >> On 31 August 2012 05:42, Aleksandr Rybalko wrote: >>> Hi hackers, >>> >>> I already post that patch some time ago with proposed "dynamic attach >>> of hinted devices" patch. [1] >>> >>> But will try to do it again, step-by-step :) >>> >>> So that patch allow switch from static hints to dynamic hints. >>> That way embedded systems, which usually compiled with hints (static) >>> will be able to see/edit/add hints and/or kenv variables. >>> >>> If nobody have objections I will commit it soon. >>> Hope 2-3 days enough for that :) >>> >>> [1] >>> http://lists.freebsd.org/pipermail/freebsd-arch/2012-January/012295.html >>> [2] http://people.freebsd.org/~ray/subr_hints.c.patch >>> >>> Thanks. >>> >>> WBW >>> -- >>> Alexandr Rybalko >>> aka Alex RAY >>> _______________________________________________ >>> freebsd-embedded@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-embedded >>> To unsubscribe, send any mail to "freebsd-embedded-unsubscribe@freebsd.org" >> _______________________________________________ >> freebsd-embedded@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-embedded >> To unsubscribe, send any mail to "freebsd-embedded-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Sun Sep 2 12:37:35 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E3BFA106566C for ; Sun, 2 Sep 2012 12:37:35 +0000 (UTC) (envelope-from loos.br@gmail.com) Received: from mail-gg0-f182.google.com (mail-gg0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 931238FC12 for ; Sun, 2 Sep 2012 12:37:35 +0000 (UTC) Received: by ggnk4 with SMTP id k4so838175ggn.13 for ; Sun, 02 Sep 2012 05:37:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:subject:date:message-id:to:mime-version:x-mailer; bh=4mvb+U5UWfYIQu/rzEMTKbzrYFSACeI9Buwnw5eFRyI=; b=ueLz5yBQEXuA0tkBwUqqXcAiGrPtanq5uif1F6j44MC3r9FExV0G22kiGgC4JwWZag aWYVBBleswunG28pmBFxGU6zf+mQRuAEnu4zT46dsFjpSLbO/LL5nxmL8BNr/eWcKjoU qmnFuhmCUjZRY+M00P0MwUxvBXnZTOQN/nz8y3/2xPZJjceiQrOq3Sov9AwaLzPAv52G J225eX1l1MYwKczz0O2TOihVRJ/7+X8FFUCItL3be0RYnm1cTMr3pdZHwjdZSFupCtT0 Ed6g+C7VcoAVhUaNrhGRq8Y/kyqA0dd6WdobCVSV5o6BtoLGOtFjJ+mIwVsQkD1cCa0U uW8w== Received: by 10.236.115.138 with SMTP id e10mr12723618yhh.79.1346589448693; Sun, 02 Sep 2012 05:37:28 -0700 (PDT) Received: from [192.168.0.53] ([187.120.131.141]) by mx.google.com with ESMTPS id v8sm18930927yhi.15.2012.09.02.05.37.26 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 02 Sep 2012 05:37:27 -0700 (PDT) From: Luiz Otavio O Souza Content-Type: multipart/mixed; boundary=Apple-Mail-58-1062540785 Date: Sun, 2 Sep 2012 09:37:24 -0300 Message-Id: To: freebsd-arch@freebsd.org Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) Subject: spibus access serialization X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Sep 2012 12:37:36 -0000 --Apple-Mail-58-1062540785 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi, I've some embedded systems with spi devices that share the same spibus = and because of that i'm working to get some kind of access serialization = on spibus and also protection to devices which need a series of spi = transfers to achieve some goal. The SPI usage (with all the patches applied) goes like this: /* Wait until the spibus is free. When free, acquire the bus and select = the device */ SPIBUS_ACQUIRE_BUS(spibus, device); /* Program the CPLD to read data from NAND */ SPIBUS_TRANSFER(spibus, device, &cmd); /* Read 'n' bytes from CPLD */ SPIBUS_TRANSFER(spibus, device, &cmd); /* Release the spibus and deselect the device */ SPIBUS_RELEASE_BUS(spibus, device); While today everything is done inside SPIBUS_TRANSFER(). The patch spibus-01.diff adds the bus methods and the default methods. The spibus-02-devices.diff adds the needed glue to all spi devices = (dev/flash/at45d.c, dev/flash/mx25l.c, arm/lpc/ssd1289.c, = mips/atheros/pcf2123_rtc.c). As the default methods are just nops, there = are no functional changes. On spibus-03-controllers.diff we finally add the serialization methods = to spi controllers (mips/atheros/ar71xx_spi.c and arm/lpc/lpc_spi.c) and = change the device CS to happen on bus acquire and release and not on = start and end of each transfer. The spi controller on arm/at91/at91_spi.c wasn't changed because looks = like it will be need to move the device CS control from the controller = and use it as a GPIO pin. I need to read more of at91 code before i can = suggest some change there. Until there it will work just as now (no = serialization and selecting/deselecting the device within each = transfer). Comments ? Thanks, Luiz --Apple-Mail-58-1062540785 Content-Disposition: attachment; filename=spibus-01.diff Content-Type: application/octet-stream; name="spibus-01.diff" Content-Transfer-Encoding: 7bit Index: dev/spibus/spibus.c =================================================================== --- dev/spibus/spibus.c (revision 239916) +++ dev/spibus/spibus.c (working copy) @@ -155,6 +155,20 @@ resource_int_value(dname, dunit, "cs", &devi->cs); } +static void +spibus_acquire_bus_impl(device_t dev, device_t child) +{ + + SPIBUS_ACQUIRE_BUS(device_get_parent(dev), child); +} + +static void +spibus_release_bus_impl(device_t dev, device_t child) +{ + + SPIBUS_RELEASE_BUS(device_get_parent(dev), child); +} + static int spibus_transfer_impl(device_t dev, device_t child, struct spi_command *cmd) { @@ -180,6 +194,8 @@ DEVMETHOD(bus_hinted_child, spibus_hinted_child), /* spibus interface */ + DEVMETHOD(spibus_acquire_bus, spibus_acquire_bus_impl), + DEVMETHOD(spibus_release_bus, spibus_release_bus_impl), DEVMETHOD(spibus_transfer, spibus_transfer_impl), DEVMETHOD_END Index: dev/spibus/spibus_if.m =================================================================== --- dev/spibus/spibus_if.m (revision 239916) +++ dev/spibus/spibus_if.m (working copy) @@ -32,6 +32,37 @@ INTERFACE spibus; # +# Default implementation +# +CODE { + static void + null_acquire_bus(device_t dev, device_t child) + { + } + + static void + null_release_bus(device_t dev, device_t child) + { + } +}; + +# +# Acquire bus and select the device +# +METHOD void acquire_bus { + device_t dev; + device_t child; +} DEFAULT null_acquire_bus; + +# +# Release bus and deselect the device +# +METHOD void release_bus { + device_t dev; + device_t child; +} DEFAULT null_release_bus; + +# # Do a spi command # METHOD int transfer { --Apple-Mail-58-1062540785 Content-Disposition: attachment; filename=spibus-02-devices.diff Content-Type: application/octet-stream; name="spibus-02-devices.diff" Content-Transfer-Encoding: 7bit Index: dev/flash/at45d.c =================================================================== --- dev/flash/at45d.c (revision 239916) +++ dev/flash/at45d.c (working copy) @@ -132,7 +132,9 @@ cmd.tx_cmd = txBuf; cmd.rx_cmd = rxBuf; cmd.rx_cmd_sz = cmd.tx_cmd_sz = 2; + SPIBUS_ACQUIRE_BUS(device_get_parent(dev), dev); err = SPIBUS_TRANSFER(device_get_parent(dev), dev, &cmd); + SPIBUS_RELEASE_BUS(device_get_parent(dev), dev); *status = rxBuf[1]; return (err); } @@ -152,7 +154,9 @@ cmd.tx_cmd = &txBuf; cmd.rx_cmd = &rxBuf; cmd.tx_cmd_sz = cmd.rx_cmd_sz = 5; + SPIBUS_ACQUIRE_BUS(device_get_parent(dev), dev); err = SPIBUS_TRANSFER(device_get_parent(dev), dev, &cmd); + SPIBUS_RELEASE_BUS(device_get_parent(dev), dev); if (err) return (err); memcpy(resp, rxBuf + 1, 4); @@ -365,8 +369,10 @@ txBuf[2] = ((addr >> 8) & 0xff); txBuf[3] = 0; cmd.tx_data_sz = cmd.rx_data_sz = 0; + SPIBUS_ACQUIRE_BUS(pdev, dev); err = SPIBUS_TRANSFER(pdev, dev, &cmd); + SPIBUS_RELEASE_BUS(pdev, dev); if (err == 0) err = at45d_wait_ready(dev, &status); @@ -383,7 +389,9 @@ txBuf[2] = ((addr >> 8) & 0xff); txBuf[3] = (addr & 0xff); cmd.tx_data_sz = cmd.rx_data_sz = len; + SPIBUS_ACQUIRE_BUS(pdev, dev); err = SPIBUS_TRANSFER(pdev, dev, &cmd); + SPIBUS_RELEASE_BUS(pdev, dev); if (err == 0 && bp->bio_cmd != BIO_READ) err = at45d_wait_ready(dev, &status); if (err != 0) { @@ -397,7 +405,9 @@ txBuf[2] = ((addr >> 8) & 0xff); txBuf[3] = 0; cmd.tx_data_sz = cmd.rx_data_sz = 0; + SPIBUS_ACQUIRE_BUS(pdev, dev); err = SPIBUS_TRANSFER(pdev, dev, &cmd); + SPIBUS_RELEASE_BUS(pdev, dev); if (err == 0) err = at45d_wait_ready(dev, &status); if (err != 0 || (status & 0x40) != 0) { Index: dev/flash/mx25l.c =================================================================== --- dev/flash/mx25l.c (revision 239916) +++ dev/flash/mx25l.c (working copy) @@ -123,7 +123,9 @@ cmd.rx_cmd = rxBuf; cmd.rx_cmd_sz = 2; cmd.tx_cmd_sz = 2; + SPIBUS_ACQUIRE_BUS(device_get_parent(dev), dev); err = SPIBUS_TRANSFER(device_get_parent(dev), dev, &cmd); + SPIBUS_RELEASE_BUS(device_get_parent(dev), dev); return (rxBuf[1]); } @@ -157,7 +159,9 @@ */ cmd.tx_cmd_sz = 4; cmd.rx_cmd_sz = 4; + SPIBUS_ACQUIRE_BUS(device_get_parent(dev), dev); err = SPIBUS_TRANSFER(device_get_parent(dev), dev, &cmd); + SPIBUS_RELEASE_BUS(device_get_parent(dev), dev); if (err) return (NULL); @@ -192,7 +196,9 @@ cmd.rx_cmd = rxBuf; cmd.rx_cmd_sz = 1; cmd.tx_cmd_sz = 1; + SPIBUS_ACQUIRE_BUS(device_get_parent(dev), dev); err = SPIBUS_TRANSFER(device_get_parent(dev), dev, &cmd); + SPIBUS_RELEASE_BUS(device_get_parent(dev), dev); } static void @@ -217,7 +223,9 @@ txBuf[1] = ((sector >> 16) & 0xff); txBuf[2] = ((sector >> 8) & 0xff); txBuf[3] = (sector & 0xff); + SPIBUS_ACQUIRE_BUS(device_get_parent(dev), dev); err = SPIBUS_TRANSFER(device_get_parent(dev), dev, &cmd); + SPIBUS_RELEASE_BUS(device_get_parent(dev), dev); } static int @@ -290,7 +298,9 @@ mx25l_wait_for_device_ready(dev); mx25l_set_writable(dev, 1); + SPIBUS_ACQUIRE_BUS(pdev, dev); err = SPIBUS_TRANSFER(pdev, dev, &cmd); + SPIBUS_RELEASE_BUS(pdev, dev); if (err) break; @@ -339,7 +349,9 @@ cmd.rx_data = data; cmd.rx_data_sz = count; + SPIBUS_ACQUIRE_BUS(pdev, dev); err = SPIBUS_TRANSFER(pdev, dev, &cmd); + SPIBUS_RELEASE_BUS(pdev, dev); return (err); } @@ -480,7 +492,7 @@ DEVMETHOD(device_attach, mx25l_attach), DEVMETHOD(device_detach, mx25l_detach), - { 0, 0 } + DEVMETHOD_END }; static driver_t mx25l_driver = { Index: arm/lpc/ssd1289.c =================================================================== --- arm/lpc/ssd1289.c (revision 239916) +++ arm/lpc/ssd1289.c (working copy) @@ -179,6 +179,7 @@ { uint8_t buffer[2]; + SPIBUS_ACQUIRE_BUS(device_get_parent(sc->ss_dev), sc->ss_dev); ssd1289_set_dc(sc, 0); buffer[0] = 0x00; buffer[1] = addr & 0xff; @@ -188,6 +189,7 @@ buffer[0] = (value >> 8) & 0xff; buffer[1] = value & 0xff; ssd1289_spi_send(sc, buffer, 2); + SPIBUS_RELEASE_BUS(device_get_parent(sc->ss_dev), sc->ss_dev); } static device_method_t ssd1289_methods[] = { @@ -195,7 +197,7 @@ DEVMETHOD(device_probe, ssd1289_probe), DEVMETHOD(device_attach, ssd1289_attach), - { 0, 0 } + DEVMETHOD_END }; static devclass_t ssd1289_devclass; Index: mips/atheros/pcf2123_rtc.c =================================================================== --- mips/atheros/pcf2123_rtc.c (revision 239916) +++ mips/atheros/pcf2123_rtc.c (working copy) @@ -91,7 +91,9 @@ cmd.tx_cmd = txBuf; cmd.rx_cmd_sz = sizeof(rxBuf); cmd.tx_cmd_sz = sizeof(txBuf); + SPIBUS_ACQUIRE_BUS(device_get_parent(dev), dev); err = SPIBUS_TRANSFER(device_get_parent(dev), dev, &cmd); + SPIBUS_RELEASE_BUS(device_get_parent(dev), dev); DELAY(PCF2123_DELAY); return (0); @@ -120,7 +122,9 @@ cmd.tx_cmd = txTimedate; cmd.rx_cmd_sz = sizeof(rxTimedate); cmd.tx_cmd_sz = sizeof(txTimedate); + SPIBUS_ACQUIRE_BUS(device_get_parent(dev), dev); err = SPIBUS_TRANSFER(device_get_parent(dev), dev, &cmd); + SPIBUS_RELEASE_BUS(device_get_parent(dev), dev); DELAY(PCF2123_DELAY); ct.nsec = 0; @@ -178,7 +182,9 @@ txTimedate[6] = TOBCD(ct.mon); txTimedate[7] = TOBCD(ct.year - YEAR_BASE); + SPIBUS_ACQUIRE_BUS(device_get_parent(dev), dev); err = SPIBUS_TRANSFER(device_get_parent(dev), dev, &cmd); + SPIBUS_RELEASE_BUS(device_get_parent(dev), dev); DELAY(PCF2123_DELAY); return (err); @@ -191,7 +197,7 @@ DEVMETHOD(clock_gettime, pcf2123_rtc_gettime), DEVMETHOD(clock_settime, pcf2123_rtc_settime), - { 0, 0 }, + DEVMETHOD_END }; static driver_t pcf2123_rtc_driver = { --Apple-Mail-58-1062540785 Content-Disposition: attachment; filename=spibus-03-controllers.diff Content-Type: application/octet-stream; name="spibus-03-controllers.diff" Content-Transfer-Encoding: 7bit Index: mips/atheros/ar71xx_spi.c =================================================================== --- mips/atheros/ar71xx_spi.c (revision 239916) +++ mips/atheros/ar71xx_spi.c (working copy) @@ -35,7 +35,9 @@ #include #include #include +#include #include +#include #include #include @@ -76,10 +78,21 @@ struct ar71xx_spi_softc { device_t sc_dev; + device_t sc_owner; + struct mtx sc_mtx; struct resource *sc_mem_res; uint32_t sc_reg_ctrl; }; +#define AR71XX_SPI_LOCK(_sc) mtx_lock(&(_sc)->sc_mtx) +#define AR71XX_SPI_UNLOCK(_sc) mtx_unlock(&(_sc)->sc_mtx) +#define AR71XX_SPI_LOCK_INIT(_sc) \ + mtx_init(&_sc->sc_mtx, device_get_nameunit(_sc->sc_dev), \ + "ar71xx_spi", MTX_DEF) +#define AR71XX_SPI_LOCK_DESTROY(_sc) mtx_destroy(&_sc->sc_mtx) +#define AR71XX_SPI_ASSERT_LOCKED(_sc) mtx_assert(&_sc->sc_mtx, MA_OWNED) +#define AR71XX_SPI_ASSERT_UNLOCKED(_sc) mtx_assert(&_sc->sc_mtx, MA_NOTOWNED) + static int ar71xx_spi_probe(device_t dev) { @@ -102,6 +115,7 @@ return (ENXIO); } + AR71XX_SPI_LOCK_INIT(sc); SPI_WRITE(sc, AR71XX_SPI_FS, 1); sc->sc_reg_ctrl = SPI_READ(sc, AR71XX_SPI_CTRL); @@ -133,6 +147,49 @@ SPI_WRITE(sc, AR71XX_SPI_IO_CTRL, SPI_IO_CTRL_CSMASK); } +static void +ar71xx_spi_acquire_bus(device_t dev, device_t child) +{ + struct spibus_ivar *devi = SPIBUS_IVAR(child); + struct ar71xx_spi_softc *sc; + + sc = device_get_softc(dev); + AR71XX_SPI_ASSERT_UNLOCKED(sc); + + /* Wait until bus is free and then set the new bus owner. */ + AR71XX_SPI_LOCK(sc); + while (sc->sc_owner != NULL) { + mtx_sleep(sc, &sc->sc_mtx, 0, "ar71xx_spi", 0); + } + sc->sc_owner = child; + AR71XX_SPI_UNLOCK(sc); + + /* Select the SPI device. */ + ar71xx_spi_chip_activate(sc, devi->cs); +} + +static void +ar71xx_spi_release_bus(device_t dev, device_t child) +{ + struct spibus_ivar *devi = SPIBUS_IVAR(child); + struct ar71xx_spi_softc *sc; + + sc = device_get_softc(dev); + AR71XX_SPI_ASSERT_UNLOCKED(sc); + + /* Free the SPI BUS. */ + AR71XX_SPI_LOCK(sc); + if (sc->sc_owner == NULL) + panic("ar71xx_spi: releasing unowned bus."); + if (sc->sc_owner != child) + panic("ar71xx_spi: you don't own the bus. game over."); + sc->sc_owner = NULL; + AR71XX_SPI_UNLOCK(sc); + + /* Deselect the SPI device. */ + ar71xx_spi_chip_deactivate(sc, devi->cs); +} + static uint8_t ar71xx_spi_txrx(struct ar71xx_spi_softc *sc, int cs, uint8_t data) { @@ -173,7 +230,7 @@ sc = device_get_softc(dev); - ar71xx_spi_chip_activate(sc, devi->cs); + KASSERT(sc->sc_owner != NULL, ("SPI transfer on unowned bus")); KASSERT(cmd->tx_cmd_sz == cmd->rx_cmd_sz, ("TX/RX command sizes should be equal")); @@ -196,8 +253,6 @@ for (i = 0; i < cmd->tx_data_sz; i++) buf_in[i] = ar71xx_spi_txrx(sc, devi->cs, buf_out[i]); - ar71xx_spi_chip_deactivate(sc, devi->cs); - return (0); } @@ -209,6 +264,8 @@ SPI_WRITE(sc, AR71XX_SPI_CTRL, sc->sc_reg_ctrl); SPI_WRITE(sc, AR71XX_SPI_FS, 0); + AR71XX_SPI_LOCK_DESTROY(sc); + if (sc->sc_mem_res) bus_release_resource(dev, SYS_RES_MEMORY, 0, sc->sc_mem_res); @@ -221,9 +278,11 @@ DEVMETHOD(device_attach, ar71xx_spi_attach), DEVMETHOD(device_detach, ar71xx_spi_detach), + DEVMETHOD(spibus_acquire_bus, ar71xx_spi_acquire_bus), + DEVMETHOD(spibus_release_bus, ar71xx_spi_release_bus), DEVMETHOD(spibus_transfer, ar71xx_spi_transfer), - {0, 0} + DEVMETHOD_END }; static driver_t ar71xx_spi_driver = { Index: arm/lpc/lpc_spi.c =================================================================== --- arm/lpc/lpc_spi.c (revision 239916) +++ arm/lpc/lpc_spi.c (working copy) @@ -67,6 +67,8 @@ struct lpc_spi_softc { device_t ls_dev; + device_t ls_owner; + struct mtx ls_mtx; struct resource * ls_mem_res; struct resource * ls_irq_res; bus_space_tag_t ls_bst; @@ -83,6 +85,15 @@ #define lpc_spi_write_4(_sc, _reg, _val) \ bus_space_write_4(_sc->ls_bst, _sc->ls_bsh, _reg, _val) +#define LPC_SPI_LOCK(_sc) mtx_lock(&(_sc)->ls_mtx) +#define LPC_SPI_UNLOCK(_sc) mtx_unlock(&(_sc)->ls_mtx) +#define LPC_SPI_LOCK_INIT(_sc) \ + mtx_init(&_sc->sc_mtx, device_get_nameunit(_sc->ls_dev), \ + "lpc_spi", MTX_DEF) +#define LPC_SPI_LOCK_DESTROY(_sc) mtx_destroy(&_sc->ls_mtx) +#define LPC_SPI_ASSERT_LOCKED(_sc) mtx_assert(&_sc->ls_mtx, MA_OWNED) +#define LPC_SPI_ASSERT_UNLOCKED(_sc) mtx_assert(&_sc->ls_mtx, MA_NOTOWNED) + static int lpc_spi_probe(device_t dev) { @@ -126,6 +137,8 @@ lpc_spi_write_4(sc, LPC_SSP_CR1, LPC_SSP_CR1_SSE); lpc_spi_write_4(sc, LPC_SSP_CPSR, 128); + LPC_SPI_LOCK_INIT(sc); + device_add_child(dev, "spibus", 0); return (bus_generic_attach(dev)); } @@ -136,6 +149,49 @@ return (EBUSY); } +static void +lpc_spi_acquire_bus(device_t dev, device_t child) +{ + struct spibus_ivar *devi = SPIBUS_IVAR(child); + struct lpc_spi_softc *sc; + + sc = device_get_softc(dev); + LPC_SPI_ASSERT_UNLOCKED(sc); + + /* Wait until bus is free and then set the new bus owner. */ + LPC_SPI_LOCK(sc); + while (sc->ls_owner != NULL) { + mtx_sleep(sc, &sc->ls_mtx, 0, "lpc_spi", 0); + } + sc->ls_owner = child; + LPC_SPI_UNLOCK(sc); + + /* Set CS active */ + lpc_gpio_set_state(child, devi->cs, 0); +} + +static void +lpc_spi_release_bus(device_t dev, device_t child) +{ + struct spibus_ivar *devi = SPIBUS_IVAR(child); + struct ar71xx_spi_softc *sc; + + sc = device_get_softc(dev); + LPC_SPI_ASSERT_UNLOCKED(sc); + + /* Free the SPI BUS. */ + LPC_SPI_LOCK(sc); + if (sc->ls_owner == NULL) + panic("lpc_spi: releasing unowned bus."); + if (sc->ls_owner != child) + panic("lpc_spi: you don't own the bus. game over."); + sc->ls_owner = NULL; + LPC_SPI_UNLOCK(sc); + + /* Set CS inactive */ + lpc_gpio_set_state(child, devi->cs, 1); +} + static int lpc_spi_transfer(device_t dev, device_t child, struct spi_command *cmd) { @@ -144,8 +200,7 @@ uint8_t *in_buf, *out_buf; int i; - /* Set CS active */ - lpc_gpio_set_state(child, devi->cs, 0); + KASSERT(sc->ls_owner != NULL, ("SPI transfer on unowned bus")); /* Wait for FIFO to be ready */ while ((lpc_spi_read_4(sc, LPC_SSP_SR) & LPC_SSP_SR_TNF) == 0); @@ -166,9 +221,6 @@ in_buf[i] = lpc_spi_read_4(sc, LPC_SSP_DR); } - /* Set CS inactive */ - lpc_gpio_set_state(child, devi->cs, 1); - return (0); } @@ -179,9 +231,11 @@ DEVMETHOD(device_detach, lpc_spi_detach), /* SPI interface */ + DEVMETHOD(spibus_acquire_bus, lpc_spi_acquire_bus), + DEVMETHOD(spibus_release_bus, lpc_spi_release_bus), DEVMETHOD(spibus_transfer, lpc_spi_transfer), - { 0, 0 } + DEVMETHOD_END }; static devclass_t lpc_spi_devclass; --Apple-Mail-58-1062540785-- From owner-freebsd-arch@FreeBSD.ORG Sun Sep 2 14:47:18 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DEF8E106566B for ; Sun, 2 Sep 2012 14:47:17 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9DEA18FC1A for ; Sun, 2 Sep 2012 14:47:17 +0000 (UTC) Received: by iebc12 with SMTP id c12so3564970ieb.13 for ; Sun, 02 Sep 2012 07:47:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=+JTmYZHQ0QK9DQ65sX02rU6l/WvGRMKln1xITpW3tU0=; b=pUU1OMEZc9EmS4scwJG/YU7lQ94TkZXe0ObpQiaNUSpYyMwSrhAY+JEPEMuWzgJJv8 T0wERbYwDvecykG4HlSMLT135+q3DIAfX7kED0Aa+U7Okm5SOs/QfUCTZI5C7eJg3Wxx 8AeqBuAYav4ZBlUA73qHOgCewKUIrT83rPZrrcd2agkrPPxX7a/ddpqPDiO0P1o1LVTS nL9P7XjH4avNne9J18d9SNgIsXWZ0oq6J7Rb7yz2tDhKQg48ynuZpuHLGN0y8JKR53DH p2JH8+8dQZSyzp9jHa0xFT3fnKmOUcmkGbIbEk8W14GUN9IsjkMiA0vGPg2w6zXdptvt Vzag== Received: by 10.50.77.131 with SMTP id s3mr8243006igw.20.1346597236800; Sun, 02 Sep 2012 07:47:16 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id wn5sm6391745igc.7.2012.09.02.07.47.15 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 02 Sep 2012 07:47:15 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: Date: Sun, 2 Sep 2012 08:47:14 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Luiz Otavio O Souza X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQk3gTRq0FwKrb4MAaHnhwm8IgybkkcKnVHuCder6SrXLFlAZWp7eScAL1RIf6NAcdQyle+i Cc: freebsd-arch@freebsd.org Subject: Re: spibus access serialization X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Sep 2012 14:47:18 -0000 On Sep 2, 2012, at 6:37 AM, Luiz Otavio O Souza wrote: > Hi, >=20 > I've some embedded systems with spi devices that share the same spibus = and because of that i'm working to get some kind of access serialization = on spibus and also protection to devices which need a series of spi = transfers to achieve some goal. >=20 > The SPI usage (with all the patches applied) goes like this: >=20 > /* Wait until the spibus is free. When free, acquire the bus and = select the device */ > SPIBUS_ACQUIRE_BUS(spibus, device); Does it do this with a sleep? > /* Program the CPLD to read data from NAND */ > SPIBUS_TRANSFER(spibus, device, &cmd); >=20 > /* Read 'n' bytes from CPLD */ > SPIBUS_TRANSFER(spibus, device, &cmd); >=20 > /* Release the spibus and deselect the device */ > SPIBUS_RELEASE_BUS(spibus, device); >=20 > While today everything is done inside SPIBUS_TRANSFER(). >=20 > The patch spibus-01.diff adds the bus methods and the default methods. >=20 > The spibus-02-devices.diff adds the needed glue to all spi devices = (dev/flash/at45d.c, dev/flash/mx25l.c, arm/lpc/ssd1289.c, = mips/atheros/pcf2123_rtc.c). As the default methods are just nops, there = are no functional changes. >=20 > On spibus-03-controllers.diff we finally add the serialization methods = to spi controllers (mips/atheros/ar71xx_spi.c and arm/lpc/lpc_spi.c) and = change the device CS to happen on bus acquire and release and not on = start and end of each transfer. >=20 > The spi controller on arm/at91/at91_spi.c wasn't changed because looks = like it will be need to move the device CS control from the controller = and use it as a GPIO pin. I need to read more of at91 code before i can = suggest some change there. Until there it will work just as now (no = serialization and selecting/deselecting the device within each = transfer). The at91_spi controller controls which CS line is asserted. So long as = you don't change the bits in the registers, it will stay asserted. I = think it can fit with this scheme. I'll have to take a closer look. = All my AT91 devices have only one chip on the spi bus. > Comments ? I like the idea.... I'll review the code in more detail to see if I see = any gotchas in it. Not sure why we need so many different flash modules, but that's a = different issue. > Thanks, > Luiz >=20 >=20 >=20 > = ______= _________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Sun Sep 2 16:16:25 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B1554106564A for ; Sun, 2 Sep 2012 16:16:25 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id EA0E78FC0A for ; Sun, 2 Sep 2012 16:16:06 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id q82GFrBx012901 for ; Sun, 2 Sep 2012 10:15:59 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q82GFUY1039224; Sun, 2 Sep 2012 10:15:30 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: Warner Losh In-Reply-To: References: Content-Type: text/plain; charset="us-ascii" Date: Sun, 02 Sep 2012 10:15:29 -0600 Message-ID: <1346602529.1140.563.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org, Luiz Otavio O Souza Subject: Re: spibus access serialization X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Sep 2012 16:16:25 -0000 On Sun, 2012-09-02 at 08:47 -0600, Warner Losh wrote: > On Sep 2, 2012, at 6:37 AM, Luiz Otavio O Souza wrote: > > > Hi, > > > > I've some embedded systems with spi devices that share the same spibus and because of that i'm working to get some kind of access serialization on spibus and also protection to devices which need a series of spi transfers to achieve some goal. > > > > The SPI usage (with all the patches applied) goes like this: > > > > /* Wait until the spibus is free. When free, acquire the bus and select the device */ > > SPIBUS_ACQUIRE_BUS(spibus, device); > > Does it do this with a sleep? > > > /* Program the CPLD to read data from NAND */ > > SPIBUS_TRANSFER(spibus, device, &cmd); > > > > /* Read 'n' bytes from CPLD */ > > SPIBUS_TRANSFER(spibus, device, &cmd); > > > > /* Release the spibus and deselect the device */ > > SPIBUS_RELEASE_BUS(spibus, device); > > > > While today everything is done inside SPIBUS_TRANSFER(). > > > > The patch spibus-01.diff adds the bus methods and the default methods. > > > > The spibus-02-devices.diff adds the needed glue to all spi devices (dev/flash/at45d.c, dev/flash/mx25l.c, arm/lpc/ssd1289.c, mips/atheros/pcf2123_rtc.c). As the default methods are just nops, there are no functional changes. > > > > On spibus-03-controllers.diff we finally add the serialization methods to spi controllers (mips/atheros/ar71xx_spi.c and arm/lpc/lpc_spi.c) and change the device CS to happen on bus acquire and release and not on start and end of each transfer. > > > > The spi controller on arm/at91/at91_spi.c wasn't changed because looks like it will be need to move the device CS control from the controller and use it as a GPIO pin. I need to read more of at91 code before i can suggest some change there. Until there it will work just as now (no serialization and selecting/deselecting the device within each transfer). > > The at91_spi controller controls which CS line is asserted. So long as you don't change the bits in the registers, it will stay asserted. I think it can fit with this scheme. I'll have to take a closer look. All my AT91 devices have only one chip on the spi bus. Unfortunately, the atmel SPI controller will de-assert the chip select when the transmit register (or PDC) runs out of data. It's a bug that may only affect rm9200; I haven't checked the errata and docs for the sam9 series to see if they fixed it. The way to fix the problem is indeed to take over the handling of chip selects in the driver, by taking the pins away from the device and managing them as GPIOs. Doing that is on my to-do list, but I've been waiting for some resolution of the "how do we manage device/gpio pin setup across the atmel SoC family" questions. This auto-deassert in rm9200 is also the bug that causes us to run the SPI bus at roughly 1/20th of its rated speed, to ensure that we never get a spurios chip de-select in the middle of a transfer due to momentary PDC bus starvation. Speaking of SPI bus speed, that's the other wart in our SPI support. We need a speed limit member in the spi request structure, so that drivers of individual devices on the bus can indicate the limit for their specific devices. I'd be happy to see that get added sooner rather than later, even if it takes a while to get all the low-level drivers supporting it. I think the new field in the struct should indicate the fastest speed in Hz that the device can handle the bus running, with zero meaning no limit. -- Ian From owner-freebsd-arch@FreeBSD.ORG Sun Sep 2 19:38:10 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9A883106566B for ; Sun, 2 Sep 2012 19:38:10 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id 585268FC14 for ; Sun, 2 Sep 2012 19:38:10 +0000 (UTC) Received: by iebc12 with SMTP id c12so3707876ieb.13 for ; Sun, 02 Sep 2012 12:38:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=v0kmt/IO7bJy3i/ZW3aVFrQBCIf9b3USDxId7Nc/e0c=; b=bvMLU51/6ngb92wPH5OmBjhxiDs4jgtCGpNwX6JWECYFKKS9zpjcrvjUZhgFEKSMTZ jCLgeGdNlxYsBMMP7bQCQp9mermfPxfojEaVBx3V0jJ2xh5fVCUvdH3tYGWZZa6748mM H+2YCLqB8xQ68fVkNNlpsWj2hcvIML1DCdswnzP9cSF4L7DrjvsFHlwyK96heq9s84Hv H1bPXjZf47fT3eXj6tjxf5aBx/0nfeBysf879uzax1Fpvj3A+4sbXnzvBjZAX9xGZdtm alINF+lvL1RKk05v0BivxPHrbahnCfzLYxq3Jf/k7kv90AQXGEyNqjmIwXb/9QTQPHST LzNA== Received: by 10.50.220.169 with SMTP id px9mr8692202igc.8.1346614689524; Sun, 02 Sep 2012 12:38:09 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id fu4sm7010600igc.4.2012.09.02.12.38.07 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 02 Sep 2012 12:38:08 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <1346602529.1140.563.camel@revolution.hippie.lan> Date: Sun, 2 Sep 2012 13:38:06 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <3C91462E-1423-4AFC-B4C7-6239E698A98E@bsdimp.com> References: <1346602529.1140.563.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQnBoRfzEsr/qqwwroNwtjldahj1MfqrJ/HBjRdGLWxCGR5+f+DxtLRIVcfQrARWRA7ZwahN Cc: freebsd-arch@freebsd.org, Luiz Otavio O Souza Subject: Re: spibus access serialization X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Sep 2012 19:38:10 -0000 On Sep 2, 2012, at 10:15 AM, Ian Lepore wrote: > On Sun, 2012-09-02 at 08:47 -0600, Warner Losh wrote: >> On Sep 2, 2012, at 6:37 AM, Luiz Otavio O Souza wrote: >>=20 >>> Hi, >>>=20 >>> I've some embedded systems with spi devices that share the same = spibus and because of that i'm working to get some kind of access = serialization on spibus and also protection to devices which need a = series of spi transfers to achieve some goal. >>>=20 >>> The SPI usage (with all the patches applied) goes like this: >>>=20 >>> /* Wait until the spibus is free. When free, acquire the bus and = select the device */ >>> SPIBUS_ACQUIRE_BUS(spibus, device); >>=20 >> Does it do this with a sleep? >>=20 >>> /* Program the CPLD to read data from NAND */ >>> SPIBUS_TRANSFER(spibus, device, &cmd); >>>=20 >>> /* Read 'n' bytes from CPLD */ >>> SPIBUS_TRANSFER(spibus, device, &cmd); >>>=20 >>> /* Release the spibus and deselect the device */ >>> SPIBUS_RELEASE_BUS(spibus, device); >>>=20 >>> While today everything is done inside SPIBUS_TRANSFER(). >>>=20 >>> The patch spibus-01.diff adds the bus methods and the default = methods. >>>=20 >>> The spibus-02-devices.diff adds the needed glue to all spi devices = (dev/flash/at45d.c, dev/flash/mx25l.c, arm/lpc/ssd1289.c, = mips/atheros/pcf2123_rtc.c). As the default methods are just nops, there = are no functional changes. >>>=20 >>> On spibus-03-controllers.diff we finally add the serialization = methods to spi controllers (mips/atheros/ar71xx_spi.c and = arm/lpc/lpc_spi.c) and change the device CS to happen on bus acquire and = release and not on start and end of each transfer. >>>=20 >>> The spi controller on arm/at91/at91_spi.c wasn't changed because = looks like it will be need to move the device CS control from the = controller and use it as a GPIO pin. I need to read more of at91 code = before i can suggest some change there. Until there it will work just as = now (no serialization and selecting/deselecting the device within each = transfer). >>=20 >> The at91_spi controller controls which CS line is asserted. So long = as you don't change the bits in the registers, it will stay asserted. I = think it can fit with this scheme. I'll have to take a closer look. = All my AT91 devices have only one chip on the spi bus. >=20 > Unfortunately, the atmel SPI controller will de-assert the chip select > when the transmit register (or PDC) runs out of data. It's a bug that > may only affect rm9200; I haven't checked the errata and docs for the > sam9 series to see if they fixed it. Yes, that's been fixed (which is why I didn't think it would be a = problem, but it is). > The way to fix the problem is indeed to take over the handling of chip > selects in the driver, by taking the pins away from the device and > managing them as GPIOs. Doing that is on my to-do list, but I've been > waiting for some resolution of the "how do we manage device/gpio pin > setup across the atmel SoC family" questions. That's the question still, alas. I've been looking at how FDT does it = in Linux land, and they punt on this issue. We'll likely have to put = some effort into defining these things with atmel's FDT efforts. Their = FDT comes close by defining groups, but doesn't seem to take it down to = the individual pin to signal mapping. Otherwise we'll have to fall back to arrays of pins that somehow get = associated with the device... > This auto-deassert in rm9200 is also the bug that causes us to run the > SPI bus at roughly 1/20th of its rated speed, to ensure that we never > get a spurios chip de-select in the middle of a transfer due to > momentary PDC bus starvation. Yes. The SAM9 processors have a new bit, CSAAT, which forces the line = to be asserted until you deassert it, not just during data transfers. A = quick grep of the code indicates that we don't use it, but I haven't = checked in detail... > Speaking of SPI bus speed, that's the other wart in our SPI support. = We > need a speed limit member in the spi request structure, so that = drivers > of individual devices on the bus can indicate the limit for their > specific devices. I'd be happy to see that get added sooner rather = than > later, even if it takes a while to get all the low-level drivers > supporting it. I think the new field in the struct should indicate = the > fastest speed in Hz that the device can handle the bus running, with > zero meaning no limit. Yes. And SPI mode too. These items should be programmed when the bus = is requested, imho, but perhaps there's a reason not to do this. Warner= From owner-freebsd-arch@FreeBSD.ORG Sun Sep 2 19:50:49 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0A3C11065670 for ; Sun, 2 Sep 2012 19:50:49 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id BDF998FC15 for ; Sun, 2 Sep 2012 19:50:48 +0000 (UTC) Received: by iebc12 with SMTP id c12so3713363ieb.13 for ; Sun, 02 Sep 2012 12:50:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=/Oz5t3qm+FvYGJP0HxWurDUcFgEN5rCWVzOX68KI5U4=; b=cBGax0gx1QAFDWHySAirgBHq1j9WInMgVzcjShtN+GS2F9T2+cMvMi2Ifmr10w7gBb farc601jD1qEYaJnh2IoauZqRKoqAwiG45CstQIhaDnZgCdQiZI+85JKMYE5MTw3LUCn gOLklnMO3U6x7ZoJ24GSxi3pmqO07afJ+MfMkJnZPGqB/nel7aNCmKSpNoaoJLBzrW6A At7tqyLnEpXEE83+Gilfb7B0K/HZMo7xQOY95ZligYwtQIfL1lK0dLqOLzNSaQHIAL7z e+RDJhNwAw9vZRax+lJSfSfVBx+2thoDX2mVvG92m8OTKKhOQnWxApBY0tOPv52GP3Gi nWqQ== Received: by 10.50.151.199 with SMTP id us7mr8879689igb.14.1346615448198; Sun, 02 Sep 2012 12:50:48 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id ud8sm15033722igb.4.2012.09.02.12.50.47 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 02 Sep 2012 12:50:47 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: Date: Sun, 2 Sep 2012 13:50:46 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <99A4DDAC-7A6E-4FA5-B55A-5C30210D9981@bsdimp.com> References: To: Luiz Otavio O Souza X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQmjaf9ES4gpu2iMwyAJoOx4dIN5HB9+ln2ZDSd2NEpEEziwGrfFEdLQxnoputMeHzAsnm9B Cc: freebsd-arch@freebsd.org Subject: Re: spibus access serialization X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Sep 2012 19:50:49 -0000 On Sep 2, 2012, at 6:37 AM, Luiz Otavio O Souza wrote: > This looks good to me as is. spibus-02.diff This looks good as well. There's another change mixed in = (DEVMETHOD_END), but it is likely benign enough to go in with this. If = you really want, split it out, but if you don't really want, things are = OK as is. spibus-03.diff + mtx_sleep(sc, &sc->sc_mtx, 0, "ar71xx_spi", 0); I'm not sure I see what wakes this up. Am I missing something? Same is = true in all the drivers. The rest is decent enough, but does mean we = can only do acquire/release in a sleepable context. Is that going to be = OK? Warner From owner-freebsd-arch@FreeBSD.ORG Mon Sep 3 06:01:11 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B067D106564A; Mon, 3 Sep 2012 06:01:11 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5B4D48FC0C; Mon, 3 Sep 2012 06:01:11 +0000 (UTC) Received: by obbun3 with SMTP id un3so11248909obb.13 for ; Sun, 02 Sep 2012 23:01:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=mPJ1HHBky1ux+1DNSGl2nsn3gFPP+e2X+JjOZiQc/yQ=; b=Q9fqZdOh1eCq3iIyIKs6OZqYOAtNb4dCUZgWikdQbZtsfmJjkzfyo+WQDWd2TFcDiE cyjgjBtXRna7whfgT0Yt7AOfDNPNxtx0xqU9f09/pO4+YxcZEGFxbQqR/StXcF/lKLNZ iXhh/bZ8kpEiWF18TazJfLOk6Eil9YC69dPDnRrmM1FWLNGhY9LNvKsCmQOJ+7o7b8tq 8+SKJtmCRyq8lf3Q83hj363oUFoYF32gHZ15mlxlhM9I5EaZe+rtyLkfE3APWWpM0bV2 72XKABdcGB79gZVQE7ooJtQXmxLXbgl0PTkhYOcsRCTfMWe50eX76iBLFvlJhREH3wA+ kVKw== MIME-Version: 1.0 Received: by 10.182.218.37 with SMTP id pd5mr13251048obc.24.1346652070443; Sun, 02 Sep 2012 23:01:10 -0700 (PDT) Received: by 10.76.142.201 with HTTP; Sun, 2 Sep 2012 23:01:09 -0700 (PDT) Date: Sun, 2 Sep 2012 23:01:09 -0700 Message-ID: From: Garrett Cooper To: freebsd-hackers@freebsd.org Content-Type: multipart/mixed; boundary=f46d04447dff4b7a4504c8c5de50 Cc: "freebsd-arch@FreeBSD.org Arch" Subject: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 06:01:11 -0000 --f46d04447dff4b7a4504c8c5de50 Content-Type: text/plain; charset=ISO-8859-1 Hello, I've been a bit busy working on porting over ATF from NetBSD, and one of the pieces that's currently not available in FreeBSD that's available in NetBSD is the ability to understand and compile multiple programs. In order to do this I had to refactor bsd.prog.mk (a lot). The attached patch is the end result so far, and I was wondering if anyone could please review it and/or test it (outside of me doing so). I wrote over 40 tests, but it's not exercising everything, and I would like for someone to please review/test this out who has an interest in NLS support (ala bsd.nls.mk) in particular. AFAICT this is the only gap that I couldn't resolve right away (there isn't a ton of recent documentation on how to use bsd.nls.mk). I'll run a micro benchmark and buildworld a few times (in progress) with and without the change to measure the performance effect. Any assistance would be much appreciated. Thanks! -Garrett --f46d04447dff4b7a4504c8c5de50 Content-Type: application/octet-stream; name="share-mk-bsd-prog-mk-CURRENT.patch" Content-Disposition: attachment; filename="share-mk-bsd-prog-mk-CURRENT.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_h6n5s8au0 LS0tIHNoYXJlL21rL2JzZC5wcm9nLm1rCTIwMTItMDktMDEgMTQ6MzA6MTUuOTIxNzk1NTU0IC0w NzAwCisrKyBzaGFyZS9tay9ic2QucHJvZy5tawkyMDEyLTA5LTAyIDIyOjMxOjU5LjkxNTc0MzI2 OCAtMDcwMApAQCAtNSwxOTUgKzUsMjc3IEBACiAKIC5TVUZGSVhFUzogLm91dCAubyAuYyAuY2Mg LmNwcCAuY3h4IC5DIC5tIC55IC5sIC5sbiAucyAuUyAuYXNtCiAKLSMgWFhYIFRoZSB1c2Ugb2Yg Q09QVFMgaW4gbW9kZXJuIG1ha2VmaWxlcyBpcyBkaXNjb3VyYWdlZC4KLS5pZiBkZWZpbmVkKENP UFRTKQotQ0ZMQUdTKz0ke0NPUFRTfQotLmVuZGlmCisjIExlZ2FjeSBrbm9icworLmlmIGRlZmlu ZWQoUFJPRykgfHwgZGVmaW5lZChQUk9HX0NYWCkKKy4gaWYgZGVmaW5lZChQUk9HKQorUFJPR1M9 CQkke1BST0d9CisuIGVuZGlmCisuIGlmIGRlZmluZWQoUFJPR19DWFgpCitQUk9HUz0JCSR7UFJP R19DWFh9CitQUk9HU19DWFg9CSR7UFJPR19DWFh9CisuIGVuZGlmCisjIExvb3Agb25jZSB0byBr ZWVwIHBhdHRlcm4gYW5kIGF2b2lkIG5hbWVzcGFjZSBwb2xsdXRpb24KKy4gZm9yIF9QIGluICR7 UFJPR1N9CisuICBpZiAhZGVmaW5lZChNQU4pCisuICAgZm9yIHNlY3QgaW4gMSAxYW91dCAyIDMg NCA1IDYgNyA4IDkKKy4gICAgaWYgZGVmaW5lZChNQU4ke3NlY3R9KQorTUFOLiR7X1B9PQkke01B TiR7c2VjdH19CisuICAgIGVuZGlmCisuICAgZW5kZm9yCisuICBlbmRpZiAjICFkZWZpbmVkKE1B TikKK01BTi4ke19QfT89CSR7X1B9LjEKKy4gIGlmIGRlZmluZWQoTkxTTkFNRSkgJiYgIWVtcHR5 KE5MU05BTUUpCitOTFNOQU1FLiR7UH06PQkke05MU05BTUV9CisuICBlbmRpZgorLiAgaWYgZGVm aW5lZChQUkVDSU9VU1BST0cpCitQUkVDSU9VU1BST0cuJHtfUH09CisuICBlbmRpZgorLiAgaWYg ZGVmaW5lZChQUk9HTkFNRSkKK1BST0dOQU1FLiR7X1B9PQkke1BST0dOQU1FfQorLiAgZW5kaWYK Ky4gIGlmIGRlZmluZWQoU1JDUykKK1NSQ1MuJHtfUH06PQkke1NSQ1N9CisuICBlbmRpZgorLiBl bmRmb3IKKy5lbHNlICMgIWRlZmluZWQoUFJPRykgJiYgIWRlZmluZWQoUFJPR19DWFgpCisuIGlm IGRlZmluZWQoUFJPR1NfQ1hYKSAmJiAhZW1wdHkoUFJPR1NfQ1hYKQorUFJPR1MrPQkJJHtQUk9H U19DWFh9CisuIGVuZGlmCisuZW5kaWYgIyBkZWZpbmVkKFBST0cpIHx8IGRlZmluZWQoUFJPR19D WFgpCisKKy5pZiBkZWZpbmVkKFBST0dTX0NYWCkgJiYgIWVtcHR5KFBST0dTX0NYWCkKKy4gZm9y IF9QIGluICR7UFJPR1NfQ1hYfQorUFJPR19DWFguJHtfUH09CisuIGVuZGZvcgorLmVuZGlmCisK KyMgQXZvaWQgcmVjdXJzaXZlIHZhcmlhYmxlcy4gSXQgc2hvdWxkbid0IGJlIHVzZWQgdGhpcyB3 YXkgd2l0aCAke1BST0dTfSBlaXRoZXIKKy51bmRlZiBOTFNOQU1FCisKKyMgU2V0dXAgZ2xvYmFs IENGTEFHUywgQ1hYRkxBR1MsIGFuZCBMREZMQUdTCitDRkxBR1M/PQorQ1hYRkxBR1M/PQorRFBB REQ/PQorTERBREQ/PQorTERGTEFHUz89CiAKLS5pZiAke01LX0FTU0VSVF9ERUJVR30gPT0gIm5v IgotQ0ZMQUdTKz0gLUROREVCVUcKLU5PX1dFUlJPUj0KKy5pZiBkZWZpbmVkKENPUFRTKQorQ0ZM QUdTKz0JJHtDT1BUU30KIC5lbmRpZgogCiAuaWYgZGVmaW5lZChERUJVR19GTEFHUykKLUNGTEFH Uys9JHtERUJVR19GTEFHU30KLUNYWEZMQUdTKz0ke0RFQlVHX0ZMQUdTfQotCi0uaWYgJHtNS19D VEZ9ICE9ICJubyIgJiYgJHtERUJVR19GTEFHUzpNLWd9ICE9ICIiCi1DVEZGTEFHUys9IC1nCi0u ZW5kaWYKKy4gaWYgJHtNS19DVEZ9ICE9ICJubyIgJiYgJHtERUJVR19GTEFHUzpNLWd9ICE9ICIi CitDVEZGTEFHUys9CS1nCisuIGVuZGlmCitDRkxBR1MrPQktRE5ERUJVRworQ0ZMQUdTKz0JJHtE RUJVR19GTEFHU30KK0NYWEZMQUdTKz0JJHtERUJVR19GTEFHU30KIC5lbmRpZgogCi0uaWYgZGVm aW5lZChDUlVOQ0hfQ0ZMQUdTKQotQ0ZMQUdTKz0ke0NSVU5DSF9DRkxBR1N9Ci0uZW5kaWYKK1NU UklQPz0JCS1zCiAKLS5pZiAhZGVmaW5lZChERUJVR19GTEFHUykKLVNUUklQPz0JLXMKLS5lbmRp ZgotCi0uaWYgZGVmaW5lZChOT19TSEFSRUQpICYmICgke05PX1NIQVJFRH0gIT0gIm5vIiAmJiAk e05PX1NIQVJFRH0gIT0gIk5PIikKLUxERkxBR1MrPSAtc3RhdGljCi0uZW5kaWYKLQotLmlmIGRl ZmluZWQoUFJPR19DWFgpCi1QUk9HPQkke1BST0dfQ1hYfQotLmVuZGlmCi0KLS5pZiBkZWZpbmVk KFBST0cpCi0uaWYgZGVmaW5lZChTUkNTKQotCi1PQkpTKz0gICR7U1JDUzpOKi5oOlI6Uy8kLy5v L2d9Ci0KLS5pZiB0YXJnZXQoYmVmb3JlbGlua2luZykKLSR7UFJPR306ICR7T0JKU30gYmVmb3Jl bGlua2luZwotLmVsc2UKLSR7UFJPR306ICR7T0JKU30KLS5lbmRpZgotLmlmIGRlZmluZWQoUFJP R19DWFgpCi0JJHtDWFh9ICR7Q1hYRkxBR1N9ICR7TERGTEFHU30gLW8gJHsuVEFSR0VUfSAke09C SlN9ICR7TERBRER9Ci0uZWxzZQotCSR7Q0N9ICR7Q0ZMQUdTfSAke0xERkxBR1N9IC1vICR7LlRB UkdFVH0gJHtPQkpTfSAke0xEQUREfQotLmVuZGlmCi0uaWYgJHtNS19DVEZ9ICE9ICJubyIKLQkk e0NURk1FUkdFfSAke0NURkZMQUdTfSAtbyAkey5UQVJHRVR9ICR7T0JKU30KKy5pZiAke01LX0FT U0VSVF9ERUJVR30gPT0gIm5vIgorQ0ZMQUdTKz0JLUROREVCVUcKK05PX1dFUlJPUj0KIC5lbmRp ZgogCi0uZWxzZQkjICFkZWZpbmVkKFNSQ1MpCisuZm9yIF9QIGluICR7UFJPR1N9CiAKLS5pZiAh dGFyZ2V0KCR7UFJPR30pCi0uaWYgZGVmaW5lZChQUk9HX0NYWCkKLVNSQ1M9CSR7UFJPR30uY2MK LS5lbHNlCi1TUkNTPQkke1BST0d9LmMKLS5lbmRpZgorQ0ZMQUdTLiR7X1B9CT89CitDWFhGTEFH Uy4ke19QfQk/PQorTERGTEFHUy4ke19QfQk/PQorLiBpZiAhZGVmaW5lZChOTFNESVIuJHtfUH0p CitOTFNESVIuJHtfUH06PQkke05MU0RJUn0KKy4gZW5kaWYKKy4gdW5kZWYgTkxTRElSCisKKy4g aWYgKGRlZmluZWQoTk9fU0hBUkVEKSAmJiAke05PX1NIQVJFRH0gIT0gIm5vIikgfHwgKGRlZmlu ZWQoTk9fU0hBUkVELiR7X1B9KSAmJiAke05PX1NIQVJFRC4ke19QfX0gIT0gIm5vIikKK0xERkxB R1MuJHtfUH0rPSAtc3RhdGljCisuIGVuZGlmCisKKy4gaWYgZGVmaW5lZChTUkNTLiR7X1B9KQor CitPQkpTLiR7X1B9PQkke1NSQ1MuJHtfUH06TiouaDpSOlMvJC8uby9nfQorCisuICBpZiB0YXJn ZXQoYmVmb3JlbGlua2luZykKKyR7X1B9OiAke09CSlMuJHtfUH19IGJlZm9yZWxpbmtpbmcKKy4g IGVsc2UKKyR7X1B9OiAke09CSlMuJHtfUH19CisuICBlbmRpZgorLiAgaWYgZGVmaW5lZChQUk9H X0NYWC4ke19QfSkKKwkke0NYWH0gJHtDWFhGTEFHU30gJHtDWFhGTEFHUy4ke19QfX0gJHtMREZM QUdTfSAke0xERkxBR1MuJHtfUH19IC1vICR7LlRBUkdFVH0gJHtPQkpTLiR7X1B9fSAke0xEQURE fSAke0xEQURELiR7X1B9fQorLiAgZWxzZQorCSR7Q0N9ICR7Q0ZMQUdTfSAke0NGTEFHUy4ke19Q fX0gJHtMREZMQUdTfSAke0xERkxBR1MuJHtfUH19IC1vICR7LlRBUkdFVH0gJHtPQkpTLiR7X1B9 fSAke0xEQUREfSAke0xEQURELiR7X1B9fQorLiAgZW5kaWYKKy4gIGlmICR7TUtfQ1RGfSAhPSAi bm8iCisJJHtDVEZNRVJHRX0gJHtDVEZGTEFHU30gLW8gJHsuVEFSR0VUfSAke09CSlMuJHtfUH19 CisuICBlbmRpZgorCisuIGVsc2UgIyAhZGVmaW5lZChTUkNTLiR7X1B9KQorCisuICBpZiAhdGFy Z2V0KCR7X1B9KQorLiAgIGlmIGRlZmluZWQoUFJPR19DWFguJHtfUH0pCitTUkNTLiR7X1B9Pz0J JHtfUH0uY2MKKy4gICBlbHNlCitTUkNTLiR7X1B9Pz0JJHtfUH0uYworLiAgIGVuZGlmCiAKICMg QWx3YXlzIG1ha2UgYW4gaW50ZXJtZWRpYXRlIG9iamVjdCBmaWxlIGJlY2F1c2U6CiAjIC0gaXQg c2F2ZXMgdGltZSByZWJ1aWxkaW5nIHdoZW4gb25seSB0aGUgbGlicmFyeSBoYXMgY2hhbmdlZAog IyAtIHRoZSBuYW1lIG9mIHRoZSBvYmplY3QgZ2V0cyBwdXQgaW50byB0aGUgZXhlY3V0YWJsZSBz eW1ib2wgdGFibGUgaW5zdGVhZCBvZgogIyAgIHRoZSBuYW1lIG9mIGEgdmFyaWFibGUgdGVtcG9y YXJ5IG9iamVjdC4KICMgLSBpdCdzIHVzZWZ1bCB0byBrZWVwIG9iamVjdHMgYXJvdW5kIGZvciBj cnVuY2hpbmcuCi1PQkpTPQkke1BST0d9Lm8KK09CSlMuJHtfUH09CSR7X1B9Lm8KIAotLmlmIHRh cmdldChiZWZvcmVsaW5raW5nKQotJHtQUk9HfTogJHtPQkpTfSBiZWZvcmVsaW5raW5nCi0uZWxz ZQotJHtQUk9HfTogJHtPQkpTfQotLmVuZGlmCi0uaWYgZGVmaW5lZChQUk9HX0NYWCkKLQkke0NY WH0gJHtDWFhGTEFHU30gJHtMREZMQUdTfSAtbyAkey5UQVJHRVR9ICR7T0JKU30gJHtMREFERH0K LS5lbHNlCi0JJHtDQ30gJHtDRkxBR1N9ICR7TERGTEFHU30gLW8gJHsuVEFSR0VUfSAke09CSlN9 ICR7TERBRER9Ci0uZW5kaWYKLS5pZiAke01LX0NURn0gIT0gIm5vIgotCSR7Q1RGTUVSR0V9ICR7 Q1RGRkxBR1N9IC1vICR7LlRBUkdFVH0gJHtPQkpTfQotLmVuZGlmCi0uZW5kaWYKKy4gICBpZiB0 YXJnZXQoYmVmb3JlbGlua2luZykKKyR7X1B9OiAke09CSlMuJHtfUH19IGJlZm9yZWxpbmtpbmcK Ky4gICBlbHNlCiske19QfTogJHtPQkpTLiR7X1B9fQorLiAgIGVuZGlmICMgdGFyZ2V0KGJlZm9y ZWxpbmtpbmcpCisuICAgaWYgZGVmaW5lZChQUk9HX0NYWC4ke19QfSkKKwkke0NYWH0gJHtDWFhG TEFHU30gJHtDWFhGTEFHUy4ke19QfX0gJHtMREZMQUdTfSAke0xERkxBR1MuJHtfUH19IC1vICR7 LlRBUkdFVH0gJHtPQkpTLiR7X1B9fSAke0xEQUREfSAke0xEQURELiR7X1B9fQorLiAgIGVsc2UK Kwkke0NDfSAke0NGTEFHU30gJHtDRkxBR1MuJHtfUH19ICR7TERGTEFHU30gJHtMREZMQUdTLiR7 X1B9fSAtbyAkey5UQVJHRVR9ICR7T0JKUy4ke19QfX0gJHtMREFERH0gJHtMREFERC4ke19QfX0K Ky4gICBlbmRpZgorLiAgIGlmICR7TUtfQ1RGfSAhPSAibm8iCisJJHtDVEZNRVJHRX0gJHtDVEZG TEFHU30gLW8gJHsuVEFSR0VUfSAke09CSlMuJHtfUH19CisuICAgZW5kaWYKKworLiAgZW5kaWYg IyAhdGFyZ2V0KCR7X1B9KQorCisuIGVuZGlmICMgZGVmaW5lZChTUkNTLiR7X1B9KQorCisjIE5l ZWRlZCBpbiBic2QuZGVwLm1rIC4KKy4gaWYgIWVtcHR5KFNSQ1MuJHtfUH0pCitTUkNTKz0JJHtT UkNTLiR7X1B9fQorLiBlbmRpZgogCi0uZW5kaWYKK0NMRUFORklMRVMrPSAJJHtPQkpTLiR7X1B9 fQogCi0uaWYJJHtNS19NQU59ICE9ICJubyIgJiYgIWRlZmluZWQoTUFOKSAmJiBcCi0JIWRlZmlu ZWQoTUFOMSkgJiYgIWRlZmluZWQoTUFOMikgJiYgIWRlZmluZWQoTUFOMykgJiYgXAotCSFkZWZp bmVkKE1BTjQpICYmICFkZWZpbmVkKE1BTjUpICYmICFkZWZpbmVkKE1BTjYpICYmIFwKLQkhZGVm aW5lZChNQU43KSAmJiAhZGVmaW5lZChNQU44KSAmJiAhZGVmaW5lZChNQU45KSAmJiBcCi0JIWRl ZmluZWQoTUFOMWFvdXQpCi1NQU49CSR7UFJPR30uMQotTUFOMT0JJHtNQU59Ci0uZW5kaWYKLS5l bmRpZgorLmVuZGZvciAjIGZvciBfUCBpbiAke1BST0dTfQorCithbGw6IG9iandhcm4gJHtQUk9H U30gJHtTQ1JJUFRTfQogCi1hbGw6IG9iandhcm4gJHtQUk9HfSAke1NDUklQVFN9CiAuaWYgJHtN S19NQU59ICE9ICJubyIKKy4gZm9yIF9QIGluICR7UFJPR1N9CisuICBpZiBkZWZpbmVkKE1BTi4k e19QfSkKK01BTis9CSR7TUFOLiR7X1B9fQorLiAgZW5kaWYKKy4gZW5kZm9yCiBhbGw6IF9tYW5w YWdlcwogLmVuZGlmCiAKLS5pZiBkZWZpbmVkKFBST0cpCi1DTEVBTkZJTEVTKz0gJHtQUk9HfQot LmVuZGlmCi0KLS5pZiBkZWZpbmVkKE9CSlMpCi1DTEVBTkZJTEVTKz0gJHtPQkpTfQotLmVuZGlm CitDTEVBTkZJTEVTKz0gJHtQUk9HU30KIAogLmluY2x1ZGUgPGJzZC5saWJuYW1lcy5taz4KIAot LmlmIGRlZmluZWQoUFJPRykKIF9FWFRSQURFUEVORDoKLS5pZiBkZWZpbmVkKExERkxBR1MpICYm ICFlbXB0eShMREZMQUdTOk0tbm9zdGRsaWIpCi0uaWYgZGVmaW5lZChEUEFERCkgJiYgIWVtcHR5 KERQQUREKQotCWVjaG8gJHtQUk9HfTogJHtEUEFERH0gPj4gJHtERVBFTkRGSUxFfQotLmVuZGlm Ci0uZWxzZQotCWVjaG8gJHtQUk9HfTogJHtMSUJDfSAke0RQQUREfSA+PiAke0RFUEVOREZJTEV9 Ci0uaWYgZGVmaW5lZChQUk9HX0NYWCkKLS5pZiAhZW1wdHkoQ1hYRkxBR1M6TS1zdGRsaWI9bGli YysrKQotCWVjaG8gJHtQUk9HfTogJHtMSUJDUExVU1BMVVN9ID4+ICR7REVQRU5ERklMRX0KLS5l bHNlCi0JZWNobyAke1BST0d9OiAke0xJQlNURENQTFVTUExVU30gPj4gJHtERVBFTkRGSUxFfQot LmVuZGlmCi0uZW5kaWYKLS5lbmRpZgotLmVuZGlmCisuZm9yIF9QIGluICR7UFJPR1N9CisuIGlm ICFlbXB0eShMREZMQUdTOk0tbm9zdGRsaWIpIHx8IChkZWZpbmVkKExERkxBR1MuJHtfUH0pICYm ICFlbXB0eShMREZMQUdTLiR7X1B9Ok0tbm9zdGRsaWIpKQorCWVjaG8gJHtfUH06ICR7RFBBRER9 ICR7RFBBREQuJHtfUH19ID4+ICR7REVQRU5ERklMRX0KKy4gZWxzZQorCWVjaG8gJHtfUH06ICR7 TElCQ30gJHtEUEFERH0gJHtEUEFERC4ke19QfX0gPj4gJHtERVBFTkRGSUxFfQorLiAgaWYgZGVm aW5lZChQUk9HX0NYWC4ke19QfSkKKy4gICBpZiAhZW1wdHkoQ1hYRkxBR1M6TS1zdGRsaWI9bGli YysrKSB8fCAhZW1wdHkoQ1hYRkxBR1MuJHtfUH06TS1zdGRsaWI9bGliYysrKQorCWVjaG8gJHtf UH06ICR7TElCQ1BMVVNQTFVTfSA+PiAke0RFUEVOREZJTEV9CisuICAgZWxzZQorCWVjaG8gJHtf UH06ICR7TElCU1REQ1BMVVNQTFVTfSA+PiAke0RFUEVOREZJTEV9CisuICAgZW5kaWYKKy4gIGVu ZGlmCisuIGVuZGlmCisuZW5kZm9yCiAKLS5pZiAhdGFyZ2V0KGluc3RhbGwpCisuZm9yIF9QIGlu ICR7UFJPR1N9CitJTlNUQUxMRkxBR1MuJHtfUH0/PSAke0lOU1RBTExGTEFHU30KIAotLmlmIGRl ZmluZWQoUFJFQ0lPVVNQUk9HKQotLmlmICFkZWZpbmVkKE5PX0ZTQ0hHKQotSU5TVEFMTEZMQUdT Kz0gLWZzY2hnCi0uZW5kaWYKLUlOU1RBTExGTEFHUys9IC1TCi0uZW5kaWYKKy4gaWYgIXRhcmdl dChpbnN0YWxsKQorLiAgaWYgZGVmaW5lZChQUkVDSU9VU1BST0cuJHtfUH0pCisuICAgaWYgIWRl ZmluZWQoTk9fRlNDSEcpICYmICFkZWZpbmVkKE5PX0ZTQ0hHLiR7X1B9KQorSU5TVEFMTEZMQUdT LiR7X1B9Kz0gLWZzY2hnCisuICAgZW5kaWYKK0lOU1RBTExGTEFHUy4ke19QfSs9IC1TCisuICBl bmRpZgorLiBlbmRpZgorCitCSU5ESVIuJHtfUH0/PQkJJHtCSU5ESVJ9CitCSU5HUlAuJHtfUH0/ PQkJJHtCSU5HUlB9CitCSU5NT0RFLiR7X1B9Pz0JCSR7QklOTU9ERX0KK0JJTk9XTi4ke19QfT89 CQkke0JJTk9XTn0KIAotX0lOU1RBTExGTEFHUzo9CSR7SU5TVEFMTEZMQUdTfQotLmZvciBpZSBp biAke0lOU1RBTExGTEFHU19FRElUfQotX0lOU1RBTExGTEFHUzo9CSR7X0lOU1RBTExGTEFHUyR7 aWV9fQogLmVuZGZvcgogCi0uaWYgIXRhcmdldChyZWFsaW5zdGFsbCkgJiYgIWRlZmluZWQoSU5U RVJOQUxQUk9HKQotcmVhbGluc3RhbGw6IF9wcm9naW5zdGFsbAotLk9SREVSOiBiZWZvcmVpbnN0 YWxsIF9wcm9naW5zdGFsbAotX3Byb2dpbnN0YWxsOgotLmlmIGRlZmluZWQoUFJPRykKLS5pZiBk ZWZpbmVkKFBST0dOQU1FKQotCSR7SU5TVEFMTH0gJHtTVFJJUH0gLW8gJHtCSU5PV059IC1nICR7 QklOR1JQfSAtbSAke0JJTk1PREV9IFwKLQkgICAgJHtfSU5TVEFMTEZMQUdTfSAke1BST0d9ICR7 REVTVERJUn0ke0JJTkRJUn0vJHtQUk9HTkFNRX0KLS5lbHNlCi0JJHtJTlNUQUxMfSAke1NUUklQ fSAtbyAke0JJTk9XTn0gLWcgJHtCSU5HUlB9IC1tICR7QklOTU9ERX0gXAotCSAgICAke19JTlNU QUxMRkxBR1N9ICR7UFJPR30gJHtERVNURElSfSR7QklORElSfQotLmVuZGlmCi0uZW5kaWYKLS5l bmRpZgkjICF0YXJnZXQocmVhbGluc3RhbGwpCi0KLS5pZiBkZWZpbmVkKFNDUklQVFMpICYmICFl bXB0eShTQ1JJUFRTKQotcmVhbGluc3RhbGw6IF9zY3JpcHRzaW5zdGFsbAotLk9SREVSOiBiZWZv cmVpbnN0YWxsIF9zY3JpcHRzaW5zdGFsbAotCi1TQ1JJUFRTRElSPz0JJHtCSU5ESVJ9Ci1TQ1JJ UFRTT1dOPz0JJHtCSU5PV059Ci1TQ1JJUFRTR1JQPz0JJHtCSU5HUlB9Ci1TQ1JJUFRTTU9ERT89 CSR7QklOTU9ERX0KLQotLmZvciBzY3JpcHQgaW4gJHtTQ1JJUFRTfQotLmlmIGRlZmluZWQoU0NS SVBUU05BTUUpCi1TQ1JJUFRTTkFNRV8ke3NjcmlwdDpUfT89CSR7U0NSSVBUU05BTUV9Ci0uZWxz ZQotU0NSSVBUU05BTUVfJHtzY3JpcHQ6VH0/PQkke3NjcmlwdDpUOlJ9Ci0uZW5kaWYKLVNDUklQ VFNESVJfJHtzY3JpcHQ6VH0/PQkke1NDUklQVFNESVJ9Ci1TQ1JJUFRTT1dOXyR7c2NyaXB0OlR9 Pz0JJHtTQ1JJUFRTT1dOfQotU0NSSVBUU0dSUF8ke3NjcmlwdDpUfT89CSR7U0NSSVBUU0dSUH0K LVNDUklQVFNNT0RFXyR7c2NyaXB0OlR9Pz0JJHtTQ1JJUFRTTU9ERX0KLV9zY3JpcHRzaW5zdGFs bDogX1NDUklQVFNJTlNfJHtzY3JpcHQ6VH0KLV9TQ1JJUFRTSU5TXyR7c2NyaXB0OlR9OiAke3Nj cmlwdH0KKy5pZiAhdGFyZ2V0KGluc3RhbGwpCisKKy4gZm9yIF9QIGluICR7UFJPR1N9CisKKy4g IGlmICFkZWZpbmVkKElOVEVSTkFMUFJPRy4ke19QfSkKKworLk9SREVSOiBiZWZvcmVpbnN0YWxs IF9wcm9naW5zdGFsbC4ke19QfQorX3Byb2dpbnN0YWxsLiR7X1B9OgorLiAgIGlmIGRlZmluZWQo UFJPR05BTUUuJHtfUH0pCisJJHtJTlNUQUxMfSAke1NUUklQfSAtbyAke0JJTk9XTi4ke19QfX0g LWcgJHtCSU5HUlAuJHtfUH19IFwKKwkgICAgLW0gJHtCSU5NT0RFLiR7X1B9fSAke0lOU1RBTExG TEFHUy4ke19QfX0gJHtfUH0gXAorCSAgICAke0RFU1RESVJ9JHtCSU5ESVIuJHtfUH19LyR7UFJP R05BTUUuJHtfUH19CisuICAgZWxzZQorCSR7SU5TVEFMTH0gJHtTVFJJUH0gLW8gJHtCSU5PV04u JHtfUH19IC1nICR7QklOR1JQLiR7X1B9fSBcCisJICAgIC1tICR7QklOTU9ERS4ke19QfX0gJHtJ TlNUQUxMRkxBR1MuJHtfUH19ICR7X1B9IFwKKwkgICAgJHtERVNURElSfSR7QklORElSLiR7X1B9 fQorLiAgIGVuZGlmIAorCityZWFsaW5zdGFsbDogX3Byb2dpbnN0YWxsLiR7X1B9CisKKy4gIGVu ZGlmICMgIWRlZmluZWQoSU5URVJOQUxQUk9HLiR7X1B9KQorCisuIGVuZGZvciAjIGZvciBfUCBp biAke1BST0dTfQorCisuIGlmIGRlZmluZWQoU0NSSVBUUykgJiYgIWVtcHR5KFNDUklQVFMpCitT Q1JJUFRTRElSPz0JCSR7QklORElSfQorU0NSSVBUU09XTj89CQkke0JJTk9XTn0KK1NDUklQVFNH UlA/PQkJJHtCSU5HUlB9CitTQ1JJUFRTTU9ERT89CQkke0JJTk1PREV9CisKK19fc2NyaXB0c2lu c3RhbGw6IC5VU0UKIAkke0lOU1RBTEx9IC1vICR7U0NSSVBUU09XTl8key5BTExTUkM6VH19IFwK LQkgICAgLWcgJHtTQ1JJUFRTR1JQXyR7LkFMTFNSQzpUfX0gLW0gJHtTQ1JJUFRTTU9ERV8key5B TExTUkM6VH19IFwKKwkgICAgLWcgJHtTQ1JJUFRTR1JQXyR7LkFMTFNSQzpUfX0gXAorCSAgICAt bSAke1NDUklQVFNNT0RFXyR7LkFMTFNSQzpUfX0gXAogCSAgICAkey5BTExTUkN9IFwKLQkgICAg JHtERVNURElSfSR7U0NSSVBUU0RJUl8key5BTExTUkM6VH19LyR7U0NSSVBUU05BTUVfJHsuQUxM U1JDOlR9fQotLmVuZGZvcgotLmVuZGlmCisJICAgICR7LlRBUkdFVH0KKworLiAgZm9yIFMgaW4g JHtTQ1JJUFRTfQorCityZWFsaW5zdGFsbDogc2NyaXB0c2luc3RhbGwKKy5PUkRFUjogYmVmb3Jl aW5zdGFsbCBzY3JpcHRzaW5zdGFsbAogCi1OTFNOQU1FPz0JJHtQUk9HfQorLiAgIGlmIGRlZmlu ZWQoU0NSSVBUU05BTUUpCitTQ1JJUFRTTkFNRV8ke1N9Pz0JJHtTQ1JJUFRTTkFNRX0KKy4gICBl bHNlCitTQ1JJUFRTTkFNRV8ke1N9Pz0JJHtTOlQ6Un0KKy4gICBlbmRpZgorCitTQ1JJUFRTRElS XyR7U30/PQkke1NDUklQVFNESVJ9CitTQ1JJUFRTT1dOXyR7U30/PQkke1NDUklQVFNPV059CitT Q1JJUFRTR1JQXyR7U30/PQkke1NDUklQVFNHUlB9CitTQ1JJUFRTTU9ERV8ke1N9Pz0JJHtTQ1JJ UFRTTU9ERX0KKworc2NyaXB0c2luc3RhbGw6ICR7REVTVERJUn0ke1NDUklQVFNESVJfJHtTfX0v JHtTQ1JJUFRTTkFNRV8ke1N9fQorCiske0RFU1RESVJ9JHtTQ1JJUFRTRElSXyR7U319LyR7U0NS SVBUU05BTUVfJHtTfX06IF9fc2NyaXB0c2luc3RhbGwgJHtTfQorCisuICBlbmRmb3IgIyBmb3Ig UyBpbiAke1NDUklQVFN9CisKKy4gZW5kaWYgIyBkZWZpbmVkKFNDUklQVFMpICYmICFlbXB0eShT Q1JJUFRTKQorCisuZW5kaWYgIyAhdGFyZ2V0KGluc3RhbGwpCisKKyMgV3JhcCBic2QubmxzLm1r IGJlY2F1c2UgSSBjYW4ndCBmb3JjZSB0aGF0IE1ha2VmaWxlIHNuaXBwZXQgdG8gd29yayBvbmx5 IHdpdGgKKyMgJHtQUk9HU30uCisuZm9yIF9QIGluICR7UFJPR1N9CitOTFNOQU1FLiR7X1B9Pz0J JHtfUH0KK05MUzo9CQkke05MUy4ke19QfX0KK05MU0RJUjo9CSR7TkxTRElSLiR7X1B9fQorTkxT TkFNRTo9CSR7TkxTTkFNRS4ke19QfX0KIC5pbmNsdWRlIDxic2QubmxzLm1rPgorLmVuZGZvcgog CiAuaW5jbHVkZSA8YnNkLmZpbGVzLm1rPgogLmluY2x1ZGUgPGJzZC5pbmNzLm1rPgpAQCAtMjAy LDI1ICsyODQsMzIgQEAKIC5pZiAke01LX01BTn0gIT0gIm5vIgogcmVhbGluc3RhbGw6IF9tYW5p bnN0YWxsCiAuT1JERVI6IGJlZm9yZWluc3RhbGwgX21hbmluc3RhbGwKLS5lbmRpZgogCisuaW5j bHVkZSA8YnNkLm1hbi5taz4KIC5lbmRpZgogCiAuaWYgIXRhcmdldChsaW50KQotbGludDogJHtT UkNTOk0qLmN9Ci0uaWYgZGVmaW5lZChQUk9HKQotCSR7TElOVH0gJHtMSU5URkxBR1N9ICR7Q0ZM QUdTOk0tW0RJVV0qfSAkey5BTExTUkN9Ci0uZW5kaWYKLS5lbmRpZgorLiBmb3IgX1AgaW4gJHtQ Uk9HU30KKy4gIGlmICF0YXJnZXQobGludC4ke19QfSkKKy4gICBpZiBkZWZpbmVkKFBST0dfQ1hY LiR7X1B9KQorbGludC4ke19QfToKKy4gICBlbHNlCitsaW50LiR7X1B9OiAke1NSQ1MuJHtfUH06 TSouY30KKwkke0xJTlR9ICR7TElOVEZMQUdTfSAke0NGTEFHUzpNLVtESVVdKn0gJHtDRkxBR1Mu JHtfUH06TS1bRElVXSp9ICR7LkFMTFNSQ30KKy4gICBlbmRpZgorLiAgZW5kaWYKIAotLmlmICR7 TUtfTUFOfSAhPSAibm8iCi0uaW5jbHVkZSA8YnNkLm1hbi5taz4KLS5lbmRpZgorbGludDogbGlu dC4ke19QfQorCisuIGVuZGZvcgorLmVuZGlmICMgIXRhcmdldChsaW50KQogCiAuaW5jbHVkZSA8 YnNkLmRlcC5taz4KIAotLmlmIGRlZmluZWQoUFJPRykgJiYgIWV4aXN0cygkey5PQkpESVJ9LyR7 REVQRU5ERklMRX0pCi0ke09CSlN9OiAke1NSQ1M6TSouaH0KKy5pZiAhZXhpc3RzKCR7Lk9CSkRJ Un0vJHtERVBFTkRGSUxFfSkKKy4gZm9yIF9QIGluICR7UFJPR1N9Ciske09CSlMuJHtfUH19OiAk e1NSQ1MuJHtfUH06TSouaH0KKy4gZW5kZm9yCiAuZW5kaWYKIAogLmluY2x1ZGUgPGJzZC5vYmou bWs+Ci0tLSAvdXNyL3NyYy9zaGFyZS9tay9ic2QuUkVBRE1FCTIwMTItMDUtMjAgMTU6Mzc6NDAu ODkzMDkyNTAxIC0wNzAwCisrKyBzaGFyZS9tay9ic2QuUkVBRE1FCTIwMTItMDktMDIgMjI6MjU6 NDguNDEwNzQ1MTYxIC0wNzAwCkBAIC0xLDUgKzEsNSBAQAogIwlAKCMpYnNkLlJFQURNRQk4LjIg KEJlcmtlbGV5KSA0LzIvOTQKLSMgJEZyZWVCU0Q6IHN0YWJsZS85L3NoYXJlL21rL2JzZC5SRUFE TUUgMjM1NTM0IDIwMTItMDUtMTcgMDc6NTk6MTVaIGpsaCAkCisjICRGcmVlQlNEOiBzcmMvc2hh cmUvbWsvYnNkLlJFQURNRSx2IDEuMzcgMjAxMi8wNS8yNCAyMDowMDo1OCBtYXJjZWwgRXhwICQK IAogVGhpcyBpcyB0aGUgUkVBRE1FIGZpbGUgZm9yIHRoZSAiaW5jbHVkZSIgZmlsZXMgZm9yIHRo ZSBGcmVlQlNECiBzb3VyY2UgdHJlZS4gIFRoZSBmaWxlcyBhcmUgaW5zdGFsbGVkIGluIC91c3Iv c2hhcmUvbWssIGFuZCBhcmUgYnkKQEAgLTIwNyw5OCArMjA3LDE1NyBAQAogCiBJdCBzZXRzL3Vz ZXMgdGhlIGZvbGxvd2luZyB2YXJpYWJsZXM6CiAKLUJJTkdSUAkJQmluYXJ5IGdyb3VwLgorVmFy aWFibGVzIGdsb2JhbCB0byBhbGwgcHJvZ3JhbXM6CiAKLUJJTk9XTgkJQmluYXJ5IG93bmVyLgor Q0xFQU5GSUxFUwkJQWRkaXRpb25hbCBmaWxlcyB0byByZW1vdmUgYW5kCitDTEVBTkRJUlMJCWFk ZGl0aW9uYWwgZGlyZWN0b3JpZXMgdG8gcmVtb3ZlIGR1cmluZyBjbGVhbiBhbmQgY2xlYW5kaXIK KwkJCXRhcmdldHMuICAicm0gLWYiIGFuZCAicm0gLXJmIiB1c2VkIHJlc3BlY3RpdmVseS4KKwor REVCVUdfRkxBR1MJCWRlYnVnZ2luZyBzcGVjaWZpYyBDRkxBR1MvTERGTEFHUzsgaWYgc2V0IGFu eSBiaW5hcmllcworIAkJCXByb2R1Y2VkIHdpbGwgbm90IGJlIHN0cmlwcGVkLiAgU2VlIGFsc286 IFdJVEhfQ1RGCisKK0ZJTEVTCQkJQSBsaXN0IG9mIG5vbi1leGVjdXRhYmxlIGZpbGVzLgorCQkJ VGhlIGluc3RhbGxhdGlvbiBpcyBjb250cm9sbGVkIGJ5IHRoZSBGSUxFU05BTUUsIEZJTEVTT1dO LAorCQkJRklMRVNHUlAsIEZJTEVTTU9ERSwgRklMRVNESVIgdmFyaWFibGVzIHRoYXQgY2FuIGJl CisJCQlmdXJ0aGVyIHNwZWNpYWxpemVkIGJ5IEZJTEVTPFZBUj5fPGZpbGU+LgorCitMSU5LUwkJ CVRoZSBsaXN0IG9mIGJpbmFyeSBsaW5rczsgc2hvdWxkIGJlIGZ1bGwgcGF0aG5hbWVzLCB0aGUK KwkJCWxpbmtlZC10byBmaWxlIGNvbWluZyBmaXJzdCwgZm9sbG93ZWQgYnkgdGhlIGxpbmtlZAor CQkJZmlsZS4gIFRoZSBmaWxlcyBhcmUgaGFyZC1saW5rZWQuICBGb3IgZXhhbXBsZSwgdG8gbGlu aworCQkJL2Jpbi90ZXN0IGFuZCAvYmluL1ssIHVzZToKIAotQklOTU9ERQkJQmluYXJ5IG1vZGUu CisJCQlMSU5LUz0JJHtERVNURElSfS9iaW4vdGVzdCAke0RFU1RESVJ9L2Jpbi9bCiAKLUNMRUFO RklMRVMJQWRkaXRpb25hbCBmaWxlcyB0byByZW1vdmUgYW5kCi1DTEVBTkRJUlMJYWRkaXRpb25h bCBkaXJlY3RvcmllcyB0byByZW1vdmUgZHVyaW5nIGNsZWFuIGFuZCBjbGVhbmRpcgotCQl0YXJn ZXRzLiAgInJtIC1mIiBhbmQgInJtIC1yZiIgdXNlZCByZXNwZWN0aXZlbHkuCitQUk9HCQkJVGhl IG5hbWUgb2YgdGhlIHByb2dyYW0gdG8gYnVpbGQuICBJZiBub3Qgc3VwcGxpZWQsCisJCQlub3Ro aW5nIGlzIGJ1aWx0LgogCi1DRkxBR1MJCUZsYWdzIHRvIHRoZSBjb21waWxlciB3aGVuIGNyZWF0 aW5nIEMgb2JqZWN0cy4KK1BST0dfQ1hYCQlJZiBkZWZpbmVkLCB0aGUgbmFtZSBvZiB0aGUgQysr IHByb2dyYW0gdG8gYnVpbGQuICBBbHNvCisJCQljYXVzZXMgPGJzZC5wcm9nLm1rPiB0byBsaW5r IHRoZSBwcm9ncmFtIHdpdGggdGhlCisJCQlzdGFuZGFyZCBDKysgbGlicmFyeS4gIFBST0dfQ1hY IG92ZXJyaWRlcyB0aGUgdmFsdWUKKwkJCW9mIFBST0cgaWYgUFJPRyBpcyBhbHNvIHNldC4KIAot RklMRVMJCUEgbGlzdCBvZiBub24tZXhlY3V0YWJsZSBmaWxlcy4KLQkJVGhlIGluc3RhbGxhdGlv biBpcyBjb250cm9sbGVkIGJ5IHRoZSBGSUxFU05BTUUsIEZJTEVTT1dOLAotCQlGSUxFU0dSUCwg RklMRVNNT0RFLCBGSUxFU0RJUiB2YXJpYWJsZXMgdGhhdCBjYW4gYmUKLQkJZnVydGhlciBzcGVj aWFsaXplZCBieSBGSUxFUzxWQVI+XzxmaWxlPi4KK1NUUklQCQkJVGhlIGZsYWcgcGFzc2VkIHRv IHRoZSBpbnN0YWxsIHByb2dyYW0gdG8gY2F1c2UgdGhlCisJCQliaW5hcnkgdG8gYmUgc3RyaXBw ZWQuICBUaGlzIGlzIHRvIGJlIHVzZWQgd2hlbgorCQkJYnVpbGRpbmcgeW91ciBvd24gaW5zdGFs bCBzY3JpcHQgc28gdGhhdCB0aGUgZW50aXJlCisJCQlzeXN0ZW0gY2FuIGJlIG1hZGUgc3RyaXBw ZWQvbm90LXN0cmlwcGVkIHVzaW5nIGEKKwkJCXNpbmdsZSBrbm9iLgogCi1MREFERAkJQWRkaXRp b25hbCBsb2FkZXIgb2JqZWN0cy4gIFVzdWFsbHkgdXNlZCBmb3IgbGlicmFyaWVzLgotCQlGb3Ig ZXhhbXBsZSwgdG8gbG9hZCB3aXRoIHRoZSBjb21wYXRpYmlsaXR5IGFuZCB1dGlsaXR5Ci0JCWxp YnJhcmllcywgdXNlOgorU1VCRElSCQkJQSBsaXN0IG9mIHN1YmRpcmVjdG9yaWVzIHRoYXQgc2hv dWxkIGJlIGJ1aWx0IGFzIHdlbGwuCisJCQlFYWNoIG9mIHRoZSB0YXJnZXRzIHdpbGwgZXhlY3V0 ZSB0aGUgc2FtZSB0YXJnZXQgaW4gdGhlCisJCQlzdWJkaXJlY3Rvcmllcy4KIAotCQkJTERGSUxF Uz0tbHV0aWwgLWxjb21wYXQKK1RoZSBmb2xsb3dpbmcgdmFyaWFibGVzIGNhbiBiZSB0dW5lZCBm b3Igc3BlY2lmaWMgcHJvZ3JhbSB1c2UgKHlvdSB3aWxsIG5lZWQKK3RvIHN1ZmZpeCB2YXJpYWJs ZSBkZWNsYXJhdGlvbnMgd2l0aCAuJHtQUk9HfSwgZS5nLiBQUk9HPSBmb28gLT4gU1JDUy5mb28p OgogCi1MREZMQUdTCQlBZGRpdGlvbmFsIGxvYWRlciBmbGFncy4KK0JJTkRJUgkJCUJpbmFyeSBk aXJlY3RvcnkuCiAKLUxJTktTCQlUaGUgbGlzdCBvZiBiaW5hcnkgbGlua3M7IHNob3VsZCBiZSBm dWxsIHBhdGhuYW1lcywgdGhlCi0JCWxpbmtlZC10byBmaWxlIGNvbWluZyBmaXJzdCwgZm9sbG93 ZWQgYnkgdGhlIGxpbmtlZAotCQlmaWxlLiAgVGhlIGZpbGVzIGFyZSBoYXJkLWxpbmtlZC4gIEZv ciBleGFtcGxlLCB0byBsaW5rCi0JCS9iaW4vdGVzdCBhbmQgL2Jpbi9bLCB1c2U6CitCSU5HUlAJ CQlCaW5hcnkgZ3JvdXAuCiAKLQkJCUxJTktTPQkke0RFU1RESVJ9L2Jpbi90ZXN0ICR7REVTVERJ Un0vYmluL1sKK0JJTk9XTgkJCUJpbmFyeSBvd25lci4KKworQklOTU9ERQkJCUJpbmFyeSBtb2Rl LgorCitDRkxBR1MJCQlGbGFncyB0byB0aGUgY29tcGlsZXIgd2hlbiBjcmVhdGluZyBDIG9iamVj dHMuCisKK0RQQURECQkJQWRkaXRpb25hbCBkZXBlbmRlbmNpZXMgZm9yIHRoZSBwcm9ncmFtLiAg VXN1YWxseSB1c2VkCisJCQlmb3IgbGlicmFyaWVzLiAgRm9yIGV4YW1wbGUsIHRvIGRlcGVuZCBv biB0aGUKKwkJCWNvbXBhdGliaWxpdHkgYW5kIHV0aWxpdHkgbGlicmFyaWVzIHVzZToKKworCQkJ CURQQUREPSR7TElCQ09NUEFUfSAke0xJQlVUSUx9CisKKwkJCVRoZXJlIGlzIGEgcHJlZGVmaW5l ZCBpZGVudGlmaWVyIGZvciBlYWNoCisJCQkobm9uLXByb2ZpbGVkLCBub24tc2hhcmVkKSBsaWJy YXJ5IGFuZCBvYmplY3QuICBMaWJyYXJ5CisJCQlmaWxlIG5hbWVzIGFyZSB0cmFuc2Zvcm1lZCB0 byBpZGVudGlmaWVycyBieSByZW1vdmluZworCQkJdGhlIGV4dGVuc2lvbiBhbmQgY29udmVydGlu ZyB0byB1cHBlciBjYXNlLgorCisJCQlUaGVyZSBhcmUgbm8gc3BlY2lhbCBpZGVudGlmaWVycyBm b3IgcHJvZmlsZWQgb3Igc2hhcmVkCisJCQlsaWJyYXJpZXMgb3Igb2JqZWN0cy4gIFRoZSBpZGVu dGlmaWVycyBmb3IgdGhlIHN0YW5kYXJkCisJCQlsaWJyYXJpZXMgYXJlIHVzZWQgaW4gRFBBREQu ICBUaGlzIHdvcmtzIGNvcnJlY3RseSBpZmYKKwkJCWFsbCB0aGUgbGlicmFyaWVzIGFyZSBidWls dCBhdCB0aGUgc2FtZSB0aW1lLgorCQkJVW5mb3J0dW5hdGVseSwgaXQgY2F1c2VzIHVubmVjZXNz YXJ5IHJlbGlua3MgdG8gc2hhcmVkCisJCQlsaWJyYXJpZXMgd2hlbiBvbmx5IHRoZSBzdGF0aWMg bGlicmFyaWVzIGhhdmUgY2hhbmdlZC4KKwkJCURlcGVuZGVuY2llcyBvbiBzaGFyZWQgbGlicmFy aWVzIHNob3VsZCBiZSBvbmx5IG9uIHRoZQorCQkJbGlicmFyeSB2ZXJzaW9uIG51bWJlcnMuCisK K0lOU1RBTExGTEFHUwkJaW5zdGFsbCgxKSBmbGFncy4KKworTERBREQJCQlBZGRpdGlvbmFsIGxv YWRlciBvYmplY3RzLiAgVXN1YWxseSB1c2VkIGZvciBsaWJyYXJpZXMuCisJCQlGb3IgZXhhbXBs ZSwgdG8gbG9hZCB3aXRoIHRoZSBjb21wYXRpYmlsaXR5IGFuZCB1dGlsaXR5CisJCQlsaWJyYXJp ZXMsIHVzZToKKworCQkJTERBREQ9LWx1dGlsIC1sY29tcGF0CisKK0xERkxBR1MJCQlBZGRpdGlv bmFsIGxvYWRlciBmbGFncy4KKworTUFOCQkJTWFudWFsIHBhZ2VzIChzaG91bGQgZW5kIGluIC4x IC0gLjkpLiAgSWYgbm8gTUFOCisJCQl2YXJpYWJsZSBpcyBkZWZpbmVkLCAiTUFOPTxQUk9HPi4x IiBpcyBhc3N1bWVkLgorCitOT19GU0NIRwkJRG8gbm90IGNhbGwgaW5zdGFsbCgxKSB3aXRoIC1m IHNjaGcuCisKK1BSRUNJT1VTUFJPRwkJSW5zdGFsbCB3aXRoIC1TIChzYWZlIGNvcHkpIGFuZCBo YW5kbGUgTk9fRlNDSEcKKwkJCXZhcmlhYmxlIGNhc2UgYXMgd2VsbC4KKworUFJPR19DWFgJCUMr KyBhcHBsaWNhdGlvbiBuYW1lKHMpOyB0aGUgc291cmNlcyBmb3IgdGhlCisJCQlhcHBsaWNhdGlv bihzKSBhcmUgY29tcGlsZWQgdmlhICR7Q1hYfQorCitQUk9HTkFNRQkJVGhlIG5hbWUgdGhhdCB0 aGUgYWJvdmUgcHJvZ3JhbSB3aWxsIGJlIGluc3RhbGxlZCwgaWYKKwkJCWRpZmZlcmVudCBmcm9t IDxQUk9HPi4KKworU1JDUwkJCUxpc3Qgb2Ygc291cmNlIGZpbGVzIHRvIGJ1aWxkIHRoZSBwcm9n cmFtLiAgSWYgU1JDUyBpcworCQkJbm90IGRlZmluZWQsIGl0J3MgYXNzdW1lZCB0byBiZSAke1BS T0d9LmMgb3IsIGlmCisJCQlQUk9HX0NYWCBpcyBkZWZpbmVkLCAke1BST0dfQ1hYfS5jYy4KKwor U0NSSVBUUwkJCUEgbGlzdCBvZiBpbnRlcnByZXRlciBzY3JpcHRzIFtmaWxlLntzaCxjc2gscGws YXdrLC4uLn1dLgorCQkJVGhlIGluc3RhbGxhdGlvbiBpcyBjb250cm9sbGVkIGJ5IHRoZSBTQ1JJ UFRTTkFNRSwgU0NSSVBUU09XTiwKKwkJCVNDUklQVFNHUlAsIFNDUklQVFNNT0RFLCBTQ1JJUFRT RElSIHZhcmlhYmxlcyB0aGF0IGNhbiBiZQorCQkJZnVydGhlciBzcGVjaWFsaXplZCBieSBTQ1JJ UFRTPFZBUj5fPHNjcmlwdD4uCisKK1RoZSBmb2xsb3dpbmcgdmFyaWFibGVzIGNhbiBiZSB0dW5l ZCBmb3Igc3BlY2lmaWMgc2NyaXB0czsgeW91IHdpbGwgbmVlZAordG8gc3VmZml4IHZhcmlhYmxl IGRlY2xhcmF0aW9ucyB3aXRoIF88U0NSSVBUPiwgZS5nLgorU0NSSVBUIC0+IGZvbywgLT4gU0NS SVBURElSX2Zvby4gIEluIGFsbCBjYXNlcywgdGhlIHBlci1zY3JpcHQgdmFyaWFibGVzCitkZWZh dWx0IHRvIHRoZSBlcXVpdmFsZW50IGdsb2JhbCBTQ1JJUFQqIHZhcmlhYmxlIGFuZCBmYWxsYmFj ayB0byB0aGUKK0JJTiogZXF1aXZhbGVudHMsIGUuZy4gU0NSSVBURElSX2ZvbyBmYWxscyBiYWNr IHRvIFNDUklQVERJUiBpZiBTQ1JJUFRESVJfZm9vCitpcyBub3QgZGVmaW5lZCwgdGhlbiBCSU5E SVIgaWYgU0NSSVBURElSIGlzIG5vdCBkZWZpbmVkOgorCitTQ1JJUFRTRElSCQlEaXJlY3Rvcnkg dG8gaW5zdGFsbCBhIHNjcmlwdCB0by4KKworU0NSSVBUR1JQCQlHcm91cCB0byBzZXQgZm9yIGEg c2NyaXB0LgorCitTQ1JJUFRNT0RFCQlNb2RlIHRvIGFwcGx5IHRvIGEgc2NyaXB0LgogCi1NQU4J CU1hbnVhbCBwYWdlcyAoc2hvdWxkIGVuZCBpbiAuMSAtIC45KS4gIElmIG5vIE1BTiB2YXJpYWJs ZQotCQlpcyBkZWZpbmVkLCAiTUFOPSR7UFJPR30uMSIgaXMgYXNzdW1lZC4KK1NDUklQVE9XTgkJ T3duZXIgdG8gc2V0IGZvciBhIHNjcmlwdC4KIAotUFJPRwkJVGhlIG5hbWUgb2YgdGhlIHByb2dy YW0gdG8gYnVpbGQuICBJZiBub3Qgc3VwcGxpZWQsIG5vdGhpbmcKLQkJaXMgYnVpbHQuCitMZWdh Y3kgdmFyaWFibGVzIGFyZSBhcyBmb2xsb3dzOgogCi1QUk9HX0NYWAlJZiBkZWZpbmVkLCB0aGUg bmFtZSBvZiB0aGUgcHJvZ3JhbSB0byBidWlsZC4gIEFsc28KLQkJY2F1c2VzIDxic2QucHJvZy5t az4gdG8gbGluayB0aGUgcHJvZ3JhbSB3aXRoIHRoZQotCQlzdGFuZGFyZCBDKysgbGlicmFyeS4g IFBST0dfQ1hYIG92ZXJyaWRlcyB0aGUgdmFsdWUKLQkJb2YgUFJPRyBpZiBQUk9HIGlzIGFsc28g c2V0LgotCi1QUk9HTkFNRQlUaGUgbmFtZSB0aGF0IHRoZSBhYm92ZSBwcm9ncmFtIHdpbGwgYmUg aW5zdGFsbGVkIGFzLCBpZgotCQlkaWZmZXJlbnQgZnJvbSAke1BST0d9LgotCi1TUkNTCQlMaXN0 IG9mIHNvdXJjZSBmaWxlcyB0byBidWlsZCB0aGUgcHJvZ3JhbS4gIElmIFNSQ1MgaXMgbm90Ci0J CWRlZmluZWQsIGl0J3MgYXNzdW1lZCB0byBiZSAke1BST0d9LmMgb3IsIGlmIFBST0dfQ1hYIGlz Ci0JCWRlZmluZWQsICR7UFJPR19DWFh9LmNjLgotCi1EUEFERAkJQWRkaXRpb25hbCBkZXBlbmRl bmNpZXMgZm9yIHRoZSBwcm9ncmFtLiAgVXN1YWxseSB1c2VkIGZvcgotCQlsaWJyYXJpZXMuICBG b3IgZXhhbXBsZSwgdG8gZGVwZW5kIG9uIHRoZSBjb21wYXRpYmlsaXR5IGFuZAotCQl1dGlsaXR5 IGxpYnJhcmllcyB1c2U6Ci0KLQkJCVNSQ0xJQj0ke0xJQkNPTVBBVH0gJHtMSUJVVElMfQotCi0J CVRoZXJlIGlzIGEgcHJlZGVmaW5lZCBpZGVudGlmaWVyIGZvciBlYWNoIChub24tcHJvZmlsZWQs Ci0JCW5vbi1zaGFyZWQpIGxpYnJhcnkgYW5kIG9iamVjdC4gIExpYnJhcnkgZmlsZSBuYW1lcyBh cmUKLQkJdHJhbnNmb3JtZWQgdG8gaWRlbnRpZmllcnMgYnkgcmVtb3ZpbmcgdGhlIGV4dGVuc2lv biBhbmQKLQkJY29udmVydGluZyB0byB1cHBlciBjYXNlLgotCi0JCVRoZXJlIGFyZSBubyBzcGVj aWFsIGlkZW50aWZpZXJzIGZvciBwcm9maWxlZCBvciBzaGFyZWQKLQkJbGlicmFyaWVzIG9yIG9i amVjdHMuICBUaGUgaWRlbnRpZmllcnMgZm9yIHRoZSBzdGFuZGFyZAotCQlsaWJyYXJpZXMgYXJl IHVzZWQgaW4gRFBBREQuICBUaGlzIHdvcmtzIGNvcnJlY3RseSBpZmYgYWxsCi0JCXRoZSBsaWJy YXJpZXMgYXJlIGJ1aWx0IGF0IHRoZSBzYW1lIHRpbWUuICBVbmZvcnR1bmF0ZWx5LAotCQlpdCBj YXVzZXMgdW5uZWNlc3NhcnkgcmVsaW5rcyB0byBzaGFyZWQgbGlicmFyaWVzIHdoZW4KLQkJb25s eSB0aGUgc3RhdGljIGxpYnJhcmllcyBoYXZlIGNoYW5nZWQuICBEZXBlbmRlbmNpZXMgb24KLQkJ c2hhcmVkIGxpYnJhcmllcyBzaG91bGQgYmUgb25seSBvbiB0aGUgbGlicmFyeSB2ZXJzaW9uCi0J CW51bWJlcnMuCi0KLVNUUklQCQlUaGUgZmxhZyBwYXNzZWQgdG8gdGhlIGluc3RhbGwgcHJvZ3Jh bSB0byBjYXVzZSB0aGUgYmluYXJ5Ci0JCXRvIGJlIHN0cmlwcGVkLiAgVGhpcyBpcyB0byBiZSB1 c2VkIHdoZW4gYnVpbGRpbmcgeW91cgotCQlvd24gaW5zdGFsbCBzY3JpcHQgc28gdGhhdCB0aGUg ZW50aXJlIHN5c3RlbSBjYW4gYmUgbWFkZQotCQlzdHJpcHBlZC9ub3Qtc3RyaXBwZWQgdXNpbmcg YSBzaW5nbGUgbm9iLgotCi1TVUJESVIJCUEgbGlzdCBvZiBzdWJkaXJlY3RvcmllcyB0aGF0IHNo b3VsZCBiZSBidWlsdCBhcyB3ZWxsLgotCQlFYWNoIG9mIHRoZSB0YXJnZXRzIHdpbGwgZXhlY3V0 ZSB0aGUgc2FtZSB0YXJnZXQgaW4gdGhlCi0JCXN1YmRpcmVjdG9yaWVzLgotCi1TQ1JJUFRTCQlB IGxpc3Qgb2YgaW50ZXJwcmV0ZXIgc2NyaXB0cyBbZmlsZS57c2gsY3NoLHBsLGF3aywuLi59XS4K LQkJVGhlIGluc3RhbGxhdGlvbiBpcyBjb250cm9sbGVkIGJ5IHRoZSBTQ1JJUFRTTkFNRSwgU0NS SVBUU09XTiwKLQkJU0NSSVBUU0dSUCwgU0NSSVBUU01PREUsIFNDUklQVFNESVIgdmFyaWFibGVz IHRoYXQgY2FuIGJlCi0JCWZ1cnRoZXIgc3BlY2lhbGl6ZWQgYnkgU0NSSVBUUzxWQVI+XzxzY3Jp cHQ+LgorQ09QVFMJCQlPcHRpbWl6YXRpb24gZmxhZ3MgYXBwZW5kZWQgdG8gQ0ZMQUdTOyBoaWdo bHkKKyAJCQlkaXNjb3VyYWdlZCBpbiBtb2Rlcm4gTWFrZWZpbGVzLgorCitQUk9HCQkJTGVnYWN5 IGZvcm0gb2YgYFBST0dTYAorCitQUk9HX0NYWAkJTGVnYWN5IGZvcm0gb2YgYFBST0dTX0NYWGAK KworVGhlIGZvbGxvd2luZyB2YXJpYWJsZXMgYXJlIHdyYXBwZWQgdG8gYmUgbWFkZSBgUFJPR1Mg YXdhcmVgOgorCitOTFMJCQlTZWUgYnNkLm5scy5tayBkZXNjcmlwdGlvbiBmb3IgbW9yZSBkZXRh aWxzLgorTkxTRElSCitOTFNOQU1FCisKK0RldmVsb3BlciBjb252ZW5pZW5jZSB2YXJpYWJsZXM6 CisKK1dJVEhfQVNTRVJUX0RFQlVHCUNvbXBpbGUgd2l0aCBDRkxBR1MrPS1ETkRFQlVHCiAKIFRo ZSBpbmNsdWRlIGZpbGUgPGJzZC5wcm9nLm1rPiBpbmNsdWRlcyB0aGUgZmlsZSBuYW1lZCAiLi4v TWFrZWZpbGUuaW5jIgotaWYgaXQgZXhpc3RzLCBhcyB3ZWxsIGFzIHRoZSBpbmNsdWRlIGZpbGUg PGJzZC5tYW4ubWs+LgoraWYgaXQgZXhpc3RzLCBhcyB3ZWxsIGFzIHRoZSBpbmNsdWRlIGZpbGUg PGJzZC5tYW4ubWs+LiAgRnVydGhlcm1vcmUsIGlmCitXSVRIX05MUyBpcyBkZWZpbmVkIGFuZCBO TFMuPFBST0c+IGlzIGFsc28gZGVmaW5lZCwgdGhlbiBic2QubmxzLm1rIGlzCitpbmNsdWRlZCBh cyB3ZWxsLgogCiBTb21lIHNpbXBsZSBleGFtcGxlczoKIAorMS4gU2luZ2xlIEMgUHJvZ3JhbToK KwogVG8gYnVpbGQgZm9vIGZyb20gZm9vLmMgd2l0aCBhIG1hbnVhbCBwYWdlIGZvby4xLCB1c2U6 CiAKLQlQUk9HPQlmb28KKwlQUk9HUz0JZm9vCiAKIAkuaW5jbHVkZSA8YnNkLnByb2cubWs+CiAK QEAgLTMxNCw2ICszNzMsMzQgQEAKIAogCVNSQ1M9CWEuYyBiLmMgYy5jIGQuYwogCisyLiBTaW5n bGUgQysrIFByb2dyYW06CisKK1RvIGJ1aWxkIGJhciB3aXRoIGJhci5jYyB3aXRoIGEgbWFucGFn ZSBwYWdlIGJhci4xLCB1c2U6CisKKwlQUk9HU19DWFg9CWJhcgorCisJLmluY2x1ZGUgPGJzZC5w cm9nLm1rPgorCitBbGwgb3RoZXIgY29uc3RydWN0cyBhcmUgdGhlIHNhbWUgYXMgaW4gdGhlIFNp bmdsZSBDIFByb2dyYW0gY2FzZS4KKworMy4gTXVsdGlwbGUgQyBwcm9ncmFtczoKKworVG8gYnVp bGQgZm9vIGZyb20gZm9vLmMgYW5kIGJhciB3aXRoIGJhci5jLCB1c2U6CisKKwlQUk9HUz0JZm9v IGJhcgorCisJLmluY2x1ZGUgPGJzZC5wcm9nLm1rPgorCitUbyBidWlsZCBmb28gZnJvbSBiYXIu YyBhbmQgYmFyIHdpdGggZm9vLmMsIHVzZToKKworCVBST0dTPQlmb28gYmFyCisKKwlTUkNTLmZv bz0gYmFyLmMKKworCVNSQ1MuYmFyPSBmb28uYworCisJLmluY2x1ZGUgPGJzZC5wcm9nLm1rPgor CiA9LT0tPS09LT0tPS09LT0tPS09LT0tPS09LT0tPS09LT0tPS09LT0tPS09LT0tPS09LT0tPS09 LT0tPS09LT0tPS09LT0tPS09CiAKIFRoZSBpbmNsdWRlIGZpbGUgPGJzZC5zdWJkaXIubWs+IGNv bnRhaW5zIHRoZSBkZWZhdWx0IHRhcmdldHMgZm9yIGJ1aWxkaW5nCg== --f46d04447dff4b7a4504c8c5de50-- From owner-freebsd-arch@FreeBSD.ORG Mon Sep 3 09:02:35 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 10086106566B; Mon, 3 Sep 2012 09:02:35 +0000 (UTC) (envelope-from ray@dlink.ua) Received: from smtp.dlink.ua (smtp.dlink.ua [193.138.187.146]) by mx1.freebsd.org (Postfix) with ESMTP id B6F338FC0A; Mon, 3 Sep 2012 09:02:34 +0000 (UTC) Received: from terran.dlink.ua (unknown [192.168.10.90]) (Authenticated sender: ray) by smtp.dlink.ua (Postfix) with ESMTPSA id 62E7EC492D; Mon, 3 Sep 2012 12:02:33 +0300 (EEST) Date: Mon, 3 Sep 2012 12:04:31 +0300 From: Aleksandr Rybalko To: Warner Losh Message-Id: <20120903120431.23058038.ray@dlink.ua> In-Reply-To: <02C549DE-D912-4D1A-A08D-650B918E9B99@bsdimp.com> References: <20120831154241.82902856.ray@dlink.ua> <02C549DE-D912-4D1A-A08D-650B918E9B99@bsdimp.com> Organization: D-Link X-Mailer: Sylpheed 2.7.1 (GTK+ 2.20.1; i386-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, Adrian Chadd , embedded@freebsd.org Subject: Re: [RFC] hintmode switch patch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 09:02:35 -0000 On Sat, 1 Sep 2012 17:13:46 -0600 Warner Losh wrote: >> This patch looks good to me. It contained none of the stuff I was >> worried about before, so please commit this. Thank you for review! >> >> Warner >> >> On Sep 1, 2012, at 8:01 AM, Adrian Chadd wrote: >> >> > Once others have stopped giving you reasons not to commit, please >> > commit. >> > >> > >> > >> > Adrian >> > >> > >> > On 31 August 2012 05:42, Aleksandr Rybalko wrote: >> >> Hi hackers, >> >> >> >> I already post that patch some time ago with proposed "dynamic >> >> attach of hinted devices" patch. [1] >> >> >> >> But will try to do it again, step-by-step :) >> >> >> >> So that patch allow switch from static hints to dynamic hints. >> >> That way embedded systems, which usually compiled with hints >> >> (static) will be able to see/edit/add hints and/or kenv variables. >> >> >> >> If nobody have objections I will commit it soon. >> >> Hope 2-3 days enough for that :) >> >> >> >> [1] >> >> http://lists.freebsd.org/pipermail/freebsd-arch/2012-January/012295.html >> >> [2] http://people.freebsd.org/~ray/subr_hints.c.patch >> >> >> >> Thanks. >> >> >> >> WBW >> >> -- >> >> Alexandr Rybalko >> >> aka Alex RAY >> >> _______________________________________________ >> >> freebsd-embedded@freebsd.org mailing list >> >> http://lists.freebsd.org/mailman/listinfo/freebsd-embedded >> >> To unsubscribe, send any mail to >> >> "freebsd-embedded-unsubscribe@freebsd.org" >> > _______________________________________________ >> > freebsd-embedded@freebsd.org mailing list >> > http://lists.freebsd.org/mailman/listinfo/freebsd-embedded >> > To unsubscribe, send any mail to >> > "freebsd-embedded-unsubscribe@freebsd.org" >> >> _______________________________________________ >> freebsd-embedded@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-embedded >> To unsubscribe, send any mail to >> "freebsd-embedded-unsubscribe@freebsd.org" -- Alexandr Rybalko aka Alex RAY From owner-freebsd-arch@FreeBSD.ORG Mon Sep 3 09:03:47 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0CA0C106564A; Mon, 3 Sep 2012 09:03:47 +0000 (UTC) (envelope-from ray@dlink.ua) Received: from smtp.dlink.ua (smtp.dlink.ua [193.138.187.146]) by mx1.freebsd.org (Postfix) with ESMTP id B0CA78FC12; Mon, 3 Sep 2012 09:03:46 +0000 (UTC) Received: from terran.dlink.ua (unknown [192.168.10.90]) (Authenticated sender: ray) by smtp.dlink.ua (Postfix) with ESMTPSA id 7860CC492D; Mon, 3 Sep 2012 12:03:45 +0300 (EEST) Date: Mon, 3 Sep 2012 12:05:43 +0300 From: Aleksandr Rybalko To: Adrian Chadd Message-Id: <20120903120543.0fdd4860.ray@dlink.ua> In-Reply-To: References: <20120831154241.82902856.ray@dlink.ua> <02C549DE-D912-4D1A-A08D-650B918E9B99@bsdimp.com> Organization: D-Link X-Mailer: Sylpheed 2.7.1 (GTK+ 2.20.1; i386-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, embedded@freebsd.org Subject: Re: [RFC] hintmode switch patch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 09:03:47 -0000 On Sat, 1 Sep 2012 19:08:28 -0700 Adrian Chadd wrote: >> Please commit. Approved by, etc. >> Thank you Adrian! Commited, r240067. >> >> Adrian >> >> >> On 1 September 2012 16:13, Warner Losh wrote: >> > This patch looks good to me. It contained none of the stuff I was >> > worried about before, so please commit this. >> > >> > Warner >> > >> > On Sep 1, 2012, at 8:01 AM, Adrian Chadd wrote: >> > >> >> Once others have stopped giving you reasons not to commit, please >> >> commit. >> >> >> >> >> >> >> >> Adrian >> >> >> >> >> >> On 31 August 2012 05:42, Aleksandr Rybalko wrote: >> >>> Hi hackers, >> >>> >> >>> I already post that patch some time ago with proposed "dynamic >> >>> attach of hinted devices" patch. [1] >> >>> >> >>> But will try to do it again, step-by-step :) >> >>> >> >>> So that patch allow switch from static hints to dynamic hints. >> >>> That way embedded systems, which usually compiled with hints >> >>> (static) will be able to see/edit/add hints and/or kenv >> >>> variables. >> >>> >> >>> If nobody have objections I will commit it soon. >> >>> Hope 2-3 days enough for that :) >> >>> >> >>> [1] >> >>> http://lists.freebsd.org/pipermail/freebsd-arch/2012-January/012295.html >> >>> [2] http://people.freebsd.org/~ray/subr_hints.c.patch >> >>> >> >>> Thanks. >> >>> >> >>> WBW >> >>> -- >> >>> Alexandr Rybalko >> >>> aka Alex RAY >> >>> _______________________________________________ >> >>> freebsd-embedded@freebsd.org mailing list >> >>> http://lists.freebsd.org/mailman/listinfo/freebsd-embedded >> >>> To unsubscribe, send any mail to >> >>> "freebsd-embedded-unsubscribe@freebsd.org" >> >> _______________________________________________ >> >> freebsd-embedded@freebsd.org mailing list >> >> http://lists.freebsd.org/mailman/listinfo/freebsd-embedded >> >> To unsubscribe, send any mail to >> >> "freebsd-embedded-unsubscribe@freebsd.org" >> > -- Alexandr Rybalko aka Alex RAY From owner-freebsd-arch@FreeBSD.ORG Mon Sep 3 13:06:34 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9C6421065674 for ; Mon, 3 Sep 2012 13:06:34 +0000 (UTC) (envelope-from loos.br@gmail.com) Received: from mail-gg0-f182.google.com (mail-gg0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4FC9D8FC0A for ; Mon, 3 Sep 2012 13:06:34 +0000 (UTC) Received: by ggnk4 with SMTP id k4so937164ggn.13 for ; Mon, 03 Sep 2012 06:06:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=FX6XKDu/5n8jrmgstL+flfLia+vD9BpClfdBjndmLJc=; b=PRHkpAUTSPIIe5CVUJVNuT1+unLzKqRlPx525bAm7cBBt9ADEjKQ2loakHp0dbXGN4 CuMuw171xkda6AM809lIhLJ047TeEIHyfxl0lbhoObkHalH5cRhnv3zGPfLxGtlLuw8o kPuVbSypKRbB4fRHpc2B3/8/wfa6ChFMOIgPqymX137ne/Oahrmg3nVn1s4YrHzUxZR+ vOnNvgWD97kHQdNue711OAUovFitQDseoLsiZeNT41DJ8TM7LA+lG71WQYHRJPoUw3+f oEY5kp9SOPICL/8PXx92X09l923WCBw7nKcPJ/Nz/0iscH5AO6NmJHa1As1/4TJ7YUDI kDmQ== Received: by 10.236.141.42 with SMTP id f30mr15076839yhj.120.1346677587845; Mon, 03 Sep 2012 06:06:27 -0700 (PDT) Received: from [10.10.1.152] (200-205-66-186.adseguros.com.br. [200.205.66.186]) by mx.google.com with ESMTPS id l1sm24146660yhm.19.2012.09.03.06.06.25 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 03 Sep 2012 06:06:27 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Luiz Otavio O Souza In-Reply-To: <3C91462E-1423-4AFC-B4C7-6239E698A98E@bsdimp.com> Date: Mon, 3 Sep 2012 10:06:21 -0300 Content-Transfer-Encoding: quoted-printable Message-Id: <6583C0FF-CC67-4C92-AD5F-87878703AA22@gmail.com> References: <1346602529.1140.563.camel@revolution.hippie.lan> <3C91462E-1423-4AFC-B4C7-6239E698A98E@bsdimp.com> To: Warner Losh X-Mailer: Apple Mail (2.1084) Cc: Ian Lepore , freebsd-arch@freebsd.org Subject: Re: spibus access serialization X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 13:06:34 -0000 On Sep 2, 2012, at 4:38 PM, Warner Losh wrote: >=20 > On Sep 2, 2012, at 10:15 AM, Ian Lepore wrote: >=20 >> On Sun, 2012-09-02 at 08:47 -0600, Warner Losh wrote: >>> On Sep 2, 2012, at 6:37 AM, Luiz Otavio O Souza wrote: >>>=20 >>>>=20 >>>> The spi controller on arm/at91/at91_spi.c wasn't changed because = looks like it will be need to move the device CS control from the = controller and use it as a GPIO pin. I need to read more of at91 code = before i can suggest some change there. Until there it will work just as = now (no serialization and selecting/deselecting the device within each = transfer). >>>=20 >>> The at91_spi controller controls which CS line is asserted. So long = as you don't change the bits in the registers, it will stay asserted. I = think it can fit with this scheme. I'll have to take a closer look. = All my AT91 devices have only one chip on the spi bus. >>=20 >> Unfortunately, the atmel SPI controller will de-assert the chip = select >> when the transmit register (or PDC) runs out of data. It's a bug = that >> may only affect rm9200; I haven't checked the errata and docs for the >> sam9 series to see if they fixed it. >=20 > Yes, that's been fixed (which is why I didn't think it would be a = problem, but it is). >=20 >> The way to fix the problem is indeed to take over the handling of = chip >> selects in the driver, by taking the pins away from the device and >> managing them as GPIOs. Doing that is on my to-do list, but I've = been >> waiting for some resolution of the "how do we manage device/gpio pin >> setup across the atmel SoC family" questions. >=20 > That's the question still, alas. I've been looking at how FDT does it = in Linux land, and they punt on this issue. We'll likely have to put = some effort into defining these things with atmel's FDT efforts. Their = FDT comes close by defining groups, but doesn't seem to take it down to = the individual pin to signal mapping. >=20 > Otherwise we'll have to fall back to arrays of pins that somehow get = associated with the device... >=20 >> This auto-deassert in rm9200 is also the bug that causes us to run = the >> SPI bus at roughly 1/20th of its rated speed, to ensure that we never >> get a spurios chip de-select in the middle of a transfer due to >> momentary PDC bus starvation. >=20 > Yes. The SAM9 processors have a new bit, CSAAT, which forces the line = to be asserted until you deassert it, not just during data transfers. A = quick grep of the code indicates that we don't use it, but I haven't = checked in detail... Hmm. I was thinking that using CSAAT was the only way to control the CS = pin from GPIO, but i didn't knew that CSAAT was something new (i was = reading the SAM9260 datasheet). Now I have no idea how to overcome that on older chips. >=20 >> Speaking of SPI bus speed, that's the other wart in our SPI support. = We >> need a speed limit member in the spi request structure, so that = drivers >> of individual devices on the bus can indicate the limit for their >> specific devices. I'd be happy to see that get added sooner rather = than >> later, even if it takes a while to get all the low-level drivers >> supporting it. I think the new field in the struct should indicate = the >> fastest speed in Hz that the device can handle the bus running, with >> zero meaning no limit. >=20 > Yes. And SPI mode too. These items should be programmed when the bus = is requested, imho, but perhaps there's a reason not to do this. The actual patch is just one piece of Patrick Kelsey's work on MMC/SD = SPI driver = (http://freebsd.1045724.n5.nabble.com/PATCH-MMC-SD-SPI-mode-driver-td55540= 34.html) which include new methods to limit the SPI speed for a device. I'm breaking the patch in smaller parts so it can be easily reviewed. Luiz= From owner-freebsd-arch@FreeBSD.ORG Mon Sep 3 13:20:58 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 912CC1065670 for ; Mon, 3 Sep 2012 13:20:58 +0000 (UTC) (envelope-from loos.br@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 46F258FC08 for ; Mon, 3 Sep 2012 13:20:58 +0000 (UTC) Received: by yenl7 with SMTP id l7so945960yen.13 for ; Mon, 03 Sep 2012 06:20:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=GxDYku+2wyC1QmOA+gBXrhCySCNHpwtuEVbLGWujW38=; b=v/a7ikuhIBJrH/vK1oPJeVx4mUt5PG0Jlh/1b1AJwJJ1GEAW2TaVk28YkN7lJ64QNT tXpXMvEAwecynaHGrBjBd6a4ulYqSavUY0a5drl2Utq0acMtXo8yWBFrBjEEsBX2kYL5 gIBluTc2kJRR1jc7kDiwd7pDUZqlB81BmYi3zb1TT8oNCuGElsoanNgk+SPJoEMjWUBD zj5QcpK/ebjRa/qVwGlm7wOkkN6Y+1euaoRKQt8QTqc6ueJHzHF0PJ1dv8lmypunfgZo EDX07tNrGybG7gWYSryPXkE88HfKFBk1iUvR91xRlul3vC5w02fTPOTV0CKAsrJ9OF8v z4FA== Received: by 10.236.200.201 with SMTP id z49mr15080000yhn.41.1346678457607; Mon, 03 Sep 2012 06:20:57 -0700 (PDT) Received: from [10.10.1.152] (200-205-66-186.adseguros.com.br. [200.205.66.186]) by mx.google.com with ESMTPS id a4sm11910286anm.14.2012.09.03.06.20.56 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 03 Sep 2012 06:20:57 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Luiz Otavio O Souza In-Reply-To: <99A4DDAC-7A6E-4FA5-B55A-5C30210D9981@bsdimp.com> Date: Mon, 3 Sep 2012 10:20:52 -0300 Content-Transfer-Encoding: quoted-printable Message-Id: <6AAF2B85-B720-45B5-8F92-FBD9F49F1E6F@gmail.com> References: <99A4DDAC-7A6E-4FA5-B55A-5C30210D9981@bsdimp.com> To: Warner Losh X-Mailer: Apple Mail (2.1084) Cc: freebsd-arch@freebsd.org Subject: Re: spibus access serialization X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 13:20:58 -0000 On Sep 2, 2012, at 4:50 PM, Warner Losh wrote: >=20 > On Sep 2, 2012, at 6:37 AM, Luiz Otavio O Souza wrote: >=20 >> >=20 > This looks good to me as is. >=20 > spibus-02.diff >=20 > This looks good as well. There's another change mixed in = (DEVMETHOD_END), but it is likely benign enough to go in with this. If = you really want, split it out, but if you don't really want, things are = OK as is. >=20 > spibus-03.diff >=20 > + mtx_sleep(sc, &sc->sc_mtx, 0, "ar71xx_spi", 0); >=20 > I'm not sure I see what wakes this up. Am I missing something? Same = is true in all the drivers. The rest is decent enough, but does mean we = can only do acquire/release in a sleepable context. Is that going to be = OK? No, you're right, the patch is incomplete (my bad) :-/ For now doing the acquire/release only on sleepable context isn't = causing any problems, do you think i should change this ? I'll update the patch and do some more work to move some of common code = to default methods or to spibus code as suggested by ray@. Thanks for the review! Luiz= From owner-freebsd-arch@FreeBSD.ORG Mon Sep 3 14:41:18 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2159810657A8 for ; Mon, 3 Sep 2012 14:41:18 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 97D168FC14 for ; Mon, 3 Sep 2012 14:41:17 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id q83Ef51b035656 for ; Mon, 3 Sep 2012 08:41:11 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q83Ef2JG040298; Mon, 3 Sep 2012 08:41:02 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: Warner Losh In-Reply-To: <3C91462E-1423-4AFC-B4C7-6239E698A98E@bsdimp.com> References: <1346602529.1140.563.camel@revolution.hippie.lan> <3C91462E-1423-4AFC-B4C7-6239E698A98E@bsdimp.com> Content-Type: text/plain; charset="us-ascii" Date: Mon, 03 Sep 2012 08:41:02 -0600 Message-ID: <1346683262.1140.582.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org, Luiz Otavio O Souza Subject: Re: spibus access serialization X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 14:41:18 -0000 On Sun, 2012-09-02 at 13:38 -0600, Warner Losh wrote: > > Unfortunately, the atmel SPI controller will de-assert the chip > select > > when the transmit register (or PDC) runs out of data. It's a bug > that > > may only affect rm9200; I haven't checked the errata and docs for > the > > sam9 series to see if they fixed it. > > Yes, that's been fixed (which is why I didn't think it would be a > problem, but it is). > > > The way to fix the problem is indeed to take over the handling of > chip > > selects in the driver, by taking the pins away from the device and > > managing them as GPIOs. Doing that is on my to-do list, but I've > been > > waiting for some resolution of the "how do we manage device/gpio pin > > setup across the atmel SoC family" questions. > > That's the question still, alas. I've been looking at how FDT does it > in Linux land, and they punt on this issue. We'll likely have to put > some effort into defining these things with atmel's FDT efforts. > Their FDT comes close by defining groups, but doesn't seem to take it > down to the individual pin to signal mapping. Hmmm, if the workaround is needed only on rm9200 hardware, then there are only two choices for where the chip select pins live (PIOA or PIOD banks). I think it should be possible to sniff out whether the chip select pins have been configured to PIOAn or PIODn by reading the periphid registers, so we could potentially get the SPI code fixed without solving the difficult problem of managing pin assignments across different SoCs in the family. I should have sam9 hardware arriving in a couple weeks, then I'll be able to make changes and test that they work on rm9200 and sam9. -- Ian From owner-freebsd-arch@FreeBSD.ORG Mon Sep 3 16:19:30 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9DE7A106566B for ; Mon, 3 Sep 2012 16:19:30 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 782898FC16 for ; Mon, 3 Sep 2012 16:19:30 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id q83GJGma037239 for ; Mon, 3 Sep 2012 10:19:23 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q83GJEY1040387 for ; Mon, 3 Sep 2012 10:19:14 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: freebsd-arch@freebsd.org Content-Type: text/plain; charset="us-ascii" Date: Mon, 03 Sep 2012 10:19:14 -0600 Message-ID: <1346689154.1140.601.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Subject: Some busdma stats X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 16:19:30 -0000 I decided that a good way to learn more about the busdma subsystem would be to actually work with the code rather than just reading it. Regardless of whether we eventually fix every driver to eliminate transfers that aren't aligned to cache line boundaries, or somehow change the busdma code to automatically bounce unaligned requests, we need efficient allocation of small buffers aligned and sized to cache lines. I wrote some code to use uma(9) to manage pools of aligned buffers based on size, and set up a pool of uncachable/coherent buffers and a pool of "regular memory" buffers. One thing that jumps right out at you is that pretty much every call to bus_dmamem_alloc() these days uses the BUS_DMA_COHERENT flag, at least for the devices on my dreamplug unit: root@dpcur:/root # vmstat -z | egrep "ITEM|dma" ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP dma maps: 48, 0, 1838, 34, 2022, 0, 0 dma buffer 32: 32, 0, 0, 0, 0, 0, 0 dma buffer 64: 64, 0, 0, 0, 0, 0, 0 dma buffer 128: 128, 0, 0, 0, 0, 0, 0 dma buffer 256: 256, 0, 2, 28, 2, 0, 0 dma buffer 512: 512, 0, 0, 0, 0, 0, 0 dma buffer 1024: 1024, 0, 0, 0, 0, 0, 0 dma buffer 2048: 2048, 0, 0, 0, 0, 0, 0 dma buffer 4096: 4096, 0, 0, 0, 0, 0, 0 dma coherent 32: 32, 0, 1024, 106, 1024, 0, 0 dma coherent 64: 64, 0, 0, 0, 0, 0, 0 dma coherent 128: 128, 0, 129, 51, 129, 0, 0 dma coherent 256: 256, 0, 297, 33, 333, 0, 0 dma coherent 512: 512, 0, 8, 16, 16, 0, 0 dma coherent 1024: 1024, 0, 16, 4, 32, 0, 0 dma coherent 2048: 2048, 0, 12, 0, 24, 0, 0 dma coherent 4096: 4096, 0, 13, 1, 13, 0, 0 These stats represent every call to bus_dmamem_alloc() except for the SATA devices, which do a single allocation of just over 17K -- also using BUS_DMA_COHERENT -- which is large enough that it gets handled by kmem_alloc_contig() rather than by the pool allocator. The 'dma maps' number is the number of maps created by bus_dmamap_create() -- that is, it doesn't include maps automatically created by bus_dmamem_alloc(). Efficient allocation of small buffers (not wasting a page per allocation due to either alignment or cachability issues) should help pave the way for fixing existing drivers to allocate individual buffers for each DMA transfer rather than allocating a huge area and attempting to sub-divide it internally (which is bad, since a driver doesn't have all the info needed to sub-divide it safely and avoid partial cache line flushes). The new allocator code is architecture-agnostic and could be used by any busdma implementation that wants to manage pools of small aligned buffers. It's not clear to me where the .c and .h file for such a thing should live within src/sys (right now for testing I've got them in arm/arm and arm/include). -- Ian From owner-freebsd-arch@FreeBSD.ORG Tue Sep 4 04:35:00 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D10CB106564A; Tue, 4 Sep 2012 04:35:00 +0000 (UTC) (envelope-from gonzo@hq.bluezbox.com) Received: from hq.bluezbox.com (hq.bluezbox.com [70.38.37.145]) by mx1.freebsd.org (Postfix) with ESMTP id 94ACB8FC12; Tue, 4 Sep 2012 04:35:00 +0000 (UTC) Received: from [207.6.240.242] (helo=[192.168.1.64]) by hq.bluezbox.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.73 (FreeBSD)) (envelope-from ) id 1T8kMI-0003DP-CV; Mon, 03 Sep 2012 21:03:26 -0700 From: Oleksandr Tymoshenko Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Mon, 3 Sep 2012 21:03:32 -0700 Message-Id: To: "arch@freebsd.org" Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1463\)) X-Mailer: Apple Mail (2.1463) Sender: gonzo@hq.bluezbox.com X-Spam-Level: ---- X-Spam-Report: Spam detection software, running on the system "hq.bluezbox.com", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see The administrator of that system for details. Content preview: Hello, Proposed patch[1] splits sdhci driver in two parts: reusable SDHCI logic and PCI driver that provides access to registers. I tested it on X220 with Ricoh controller - no performance regression: reading 4Gb card takes 180 seconds (+/- 2) for both versions, old and new. [...] Content analysis details: (-4.4 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.8 ALL_TRUSTED Passed through trusted hosts only via SMTP -2.6 BAYES_00 BODY: Bayesian spam probability is 0 to 1% [score: 0.0000] Cc: mav@freebsd.org Subject: [RFC] SDHCI driver split X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 04:35:00 -0000 Hello, Proposed patch[1] splits sdhci driver in two parts: reusable SDHCI logic and PCI driver that provides access to registers. I tested it on X220 with Ricoh controller - no performance regression: reading 4Gb card takes 180 seconds (+/- 2) for both versions, old and new. If there are no objections - I'll commit it later this week. [1] http://people.freebsd.org/~gonzo/patches/sdhci-pci-3.diff From owner-freebsd-arch@FreeBSD.ORG Tue Sep 4 13:31:36 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D4FA21065674 for ; Tue, 4 Sep 2012 13:31:36 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id 935478FC0C for ; Tue, 4 Sep 2012 13:31:36 +0000 (UTC) Received: by iebc12 with SMTP id c12so6068072ieb.13 for ; Tue, 04 Sep 2012 06:31:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=iBduvwPO8fme3RDk89iS6ctVIzMvcInQZi/2FDEFf7s=; b=SYBhH2bcsNrfPr6Yp0AwniCAgB9IoHOtbpUoywrR74qo+6aX9AyfrBDu8Ko+H2V/VV EU+A9OSGu3AxElegVdbUWwA2MTKiD9xMIbXl/a84acJGh3EWlSuB7neoy7wHPQXC/QlK +vLGXlqM2icqVLaHkKu8yRbJv3Ou+71CpL14ygUemC6o6wt2SMCzHgOxC7XWuU4rcXUG OvZtVVIN5D5NFUjK/1fuBu5M/vpkC4zFUr16dzbHCB8j/uS8A0EXi5oi72XUK7X3Bpxm honjC7wWzP+F1TBR03tJqhhiyUZvcjXDaTBC28+xFb8jO544CcX4vQiZgMF4y+CKUnsc hLVw== Received: by 10.50.188.130 with SMTP id ga2mr14001938igc.32.1346765495970; Tue, 04 Sep 2012 06:31:35 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id i17sm12334043igd.5.2012.09.04.06.31.34 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 04 Sep 2012 06:31:34 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: Date: Tue, 4 Sep 2012 07:31:33 -0600 Content-Transfer-Encoding: 7bit Message-Id: References: To: Oleksandr Tymoshenko X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQn+HiUuP2rMNya+5MYZEFmciHeYB1tdUTd2kjoI+bZ2QG2t/EA7i68Sdk20slHxOyxZ1AJR Cc: "arch@freebsd.org" , mav@freebsd.org Subject: Re: [RFC] SDHCI driver split X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 13:31:36 -0000 On Sep 3, 2012, at 10:03 PM, Oleksandr Tymoshenko wrote: > Hello, > > Proposed patch[1] splits sdhci driver in two parts: reusable SDHCI > logic and PCI driver that provides access to registers. > > I tested it on X220 with Ricoh controller - no performance regression: > reading 4Gb card takes 180 seconds (+/- 2) for both versions, old and new. > > If there are no objections - I'll commit it later this week. > > [1] http://people.freebsd.org/~gonzo/patches/sdhci-pci-3.diff I don't see anything overtly broken in this patch. Warner From owner-freebsd-arch@FreeBSD.ORG Tue Sep 4 16:45:30 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E8073106566B; Tue, 4 Sep 2012 16:45:30 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id E36DB8FC14; Tue, 4 Sep 2012 16:45:29 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA07316; Tue, 04 Sep 2012 19:45:27 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <50463026.8000506@FreeBSD.org> Date: Tue, 04 Sep 2012 19:45:26 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120830 Thunderbird/15.0 MIME-Version: 1.0 To: Andrey Zonov References: <503DD433.2030108@FreeBSD.org> <201208290906.q7T96C9j032802@gw.catspoiler.org> <20120829092318.GW33100@deviant.kiev.zoral.com.ua> <503F2D24.8050103@FreeBSD.org> In-Reply-To: <503F2D24.8050103@FreeBSD.org> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: freebsd-arch@FreeBSD.org Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 16:45:31 -0000 on 30/08/2012 12:06 Andrey Zonov said the following: > Hi, > > So, I've got the first version of the patch (attached) which fixes > memory locked limit checking and accounting. Andrey, your mlock.patch looks good to me, but I haven't verified pieces under RACCT. Please try to get a review from a person who is knee-deep in the VM code like alc or your mentor. The code should also be sent for vetoing to security@. Not sure if you would get a review there, but absence of nays would be good. When the code is ready to be committed, please remember about memorylocked=unlimited in the default entry of the default login.conf. A big warning about it will have to be posted (in UPDATING and current@/stable@ at the very least). Thank you very much for doing this work. P.S. It would probably make sense to provide some HTTP home for this patch as well. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Tue Sep 4 17:10:44 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 16B7E1065670; Tue, 4 Sep 2012 17:10:44 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id BA3728FC18; Tue, 4 Sep 2012 17:10:42 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA07436; Tue, 04 Sep 2012 20:10:41 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <50463610.6070805@FreeBSD.org> Date: Tue, 04 Sep 2012 20:10:40 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120830 Thunderbird/15.0 MIME-Version: 1.0 To: Andrey Zonov , Edward Tomasz Napierala References: <503DD433.2030108@FreeBSD.org> <201208290906.q7T96C9j032802@gw.catspoiler.org> <20120829092318.GW33100@deviant.kiev.zoral.com.ua> <503F2D24.8050103@FreeBSD.org> <503F476E.1010505@FreeBSD.org> In-Reply-To: <503F476E.1010505@FreeBSD.org> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 17:10:44 -0000 on 30/08/2012 13:58 Andrey Zonov said the following: > [1] http://people.freebsd.org/~zont/vm_mmap.c.patch This patch is correct. > [2] http://people.freebsd.org/~zont/racct.patch This patch looks correct. And it also makes me wonder why kern/kern_racct.c is marked as standard while all(?) uses of racct API are placed under RACCT option. Ditto for kern_rctl.c/RCTL. I think that excluding these file if the options are not used would help to catch cases where the API is used unconditionally and it would also help to reduce kernel sizes a tiny bit too. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Tue Sep 4 17:48:35 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3E2C61065674 for ; Tue, 4 Sep 2012 17:48:35 +0000 (UTC) (envelope-from andrey@zonov.org) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9B0948FC0C for ; Tue, 4 Sep 2012 17:48:34 +0000 (UTC) Received: by lage12 with SMTP id e12so5477748lag.13 for ; Tue, 04 Sep 2012 10:48:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :x-gm-message-state; bh=egkq+pO5hqrcO8UMNoJLHTwap/IL3Olm7GgHqi0iWPU=; b=IFHMwUXIrRUQRv09Smz/b24wip5ORTwi3cmdCeLWi7NX6PDXK0F9dCooGzkArWazRx VMCiZzzd96c7FXBMSmdcnT8l8qTsMX6Cmfyf53I+uQy6Evo381yUlCmIaN6xkoVTeyek sPFkKccKs9YTM7iOsIDDDo11jmBPA1pBD2E2yotc2sTWt6BunZWnT9BqxqqOGzhPPLGo 5DQvC31bU20a+Litkly/oyjTgxWGJHkCZAgvf3SFAjI1CxY51rfK2dWA8SOBMFvUdAwM 7hRIonb3uk/BrTHMVgZesRwxvZJCdp/UksFPszBS7cD3vXvmcmGiZO4x50MfRPtKQbHV dLDQ== Received: by 10.152.144.134 with SMTP id sm6mr17880387lab.5.1346780913115; Tue, 04 Sep 2012 10:48:33 -0700 (PDT) Received: from zont-osx.local (ppp95-165-143-86.pppoe.spdop.ru. [95.165.143.86]) by mx.google.com with ESMTPS id fz8sm4214474lbb.9.2012.09.04.10.48.31 (version=SSLv3 cipher=OTHER); Tue, 04 Sep 2012 10:48:32 -0700 (PDT) Sender: Andrey Zonov Message-ID: <50463EEB.60207@FreeBSD.org> Date: Tue, 04 Sep 2012 21:48:27 +0400 From: Andrey Zonov User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20120824 Thunderbird/15.0 MIME-Version: 1.0 To: Andriy Gapon References: <503DD433.2030108@FreeBSD.org> <201208290906.q7T96C9j032802@gw.catspoiler.org> <20120829092318.GW33100@deviant.kiev.zoral.com.ua> <503F2D24.8050103@FreeBSD.org> <503F476E.1010505@FreeBSD.org> <50463610.6070805@FreeBSD.org> In-Reply-To: <50463610.6070805@FreeBSD.org> X-Enigmail-Version: 1.4.4 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig6E3D58686ACA8B5D436D610A" X-Gm-Message-State: ALoCoQmoIq6xaWeBTv+0IW3oYQ5VUd6nf0nHhiYKijYpB+MF20EEjUeEceUc9ou5gOznP1ON88/J Cc: Edward Tomasz Napierala , freebsd-arch@FreeBSD.org Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 17:48:35 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig6E3D58686ACA8B5D436D610A Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 9/4/12 9:10 PM, Andriy Gapon wrote: > on 30/08/2012 13:58 Andrey Zonov said the following: >> [1] http://people.freebsd.org/~zont/vm_mmap.c.patch >=20 > This patch is correct. >=20 Thanks! >> [2] http://people.freebsd.org/~zont/racct.patch >=20 > This patch looks correct. >=20 There is no need for this patch as I mentioned earlier. racct_set() doesn't add any additional locking here. > And it also makes me wonder why kern/kern_racct.c is marked as standard= while > all(?) uses of racct API are placed under RACCT option. Not all. I think some code was not easy to put under RACCT. > Ditto for kern_rctl.c/RCTL. > I think that excluding these file if the options are not used would hel= p to catch > cases where the API is used unconditionally and it would also help to r= educe > kernel sizes a tiny bit too. >=20 --=20 Andrey Zonov --------------enig6E3D58686ACA8B5D436D610A Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.18 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJQRj7uAAoJEBWLemxX/CvT7xUH/iL45f2olmOFCbV4dhxukTx0 +x7e6fRlJHxzb9rfglsfcOMdHLHd+8z2Qewq7VaJvyHvu5WouLNynE6TSs9oa4ru a+kdawS9bXc7pG3RHuXb+V638Xs4MiKtxwgbD2pI+2kRVrVyLutLCJ9bve09ihLA VVwCDpIwK7GgyfDGsHJisYxcb7QCChKz+UGkHZKLmma5PLt+DfTzpkAs1rR+vZ43 jjMSb5FsYa5qfbJQslU3dRMzRiU/PqEyBrPcOSjUHgvEGN9Kr5o1xRn73NFqKO1y avYj2gCTYSWLC2CFkqpoxEn18Rjx/s8azzz9g+TY69iBCd2DWpCZCYKczNl5NEk= =jDvD -----END PGP SIGNATURE----- --------------enig6E3D58686ACA8B5D436D610A-- From owner-freebsd-arch@FreeBSD.ORG Tue Sep 4 19:17:41 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B85F81065688 for ; Tue, 4 Sep 2012 19:17:41 +0000 (UTC) (envelope-from andrey@zonov.org) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id A54E48FC15 for ; Tue, 4 Sep 2012 19:17:39 +0000 (UTC) Received: by lage12 with SMTP id e12so5555458lag.13 for ; Tue, 04 Sep 2012 12:17:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :x-gm-message-state; bh=4mySdaGVwr5G22BxmJ4R8cggc1OIXg492zpcTc66bG0=; b=X4lJqOS3CgxCGyK+JoIyunBzozTAvi7RnUiW3GOpveBIE7WwMaBhLCUdfvafH6Su7N JWEeB6Bop+mmjw6fZnSaLkmCruHQ9c8pFk2H2KWyvaa2EzdN1CPohMW0pxrWi0iLnwrT QNztRnXxMYo/wOmmwsbRtYARzrYhoOtabe9YD52eK8Gs5lIw0aeJ1Zao3WhlcUJjPhmq f0MtYfhynlV2Q0kvr8dWasYowMaT60EsCEpT4HFAy8l07Gc/IHN9G9sSkISTHO963uoS GlyH0zFzR6a4vz3zbYfHbqlpHbbN80VUW60DYkC2PYnZz7MXCLiA7ZLghWFut8IXPO98 GHNw== Received: by 10.152.144.134 with SMTP id sm6mr18082031lab.5.1346786258441; Tue, 04 Sep 2012 12:17:38 -0700 (PDT) Received: from zont-osx.local (ppp95-165-143-86.pppoe.spdop.ru. [95.165.143.86]) by mx.google.com with ESMTPS id y4sm4268461lbg.5.2012.09.04.12.17.37 (version=SSLv3 cipher=OTHER); Tue, 04 Sep 2012 12:17:37 -0700 (PDT) Sender: Andrey Zonov Message-ID: <504653CD.2000707@FreeBSD.org> Date: Tue, 04 Sep 2012 23:17:33 +0400 From: Andrey Zonov User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20120824 Thunderbird/15.0 MIME-Version: 1.0 To: Andriy Gapon References: <503DD433.2030108@FreeBSD.org> <201208290906.q7T96C9j032802@gw.catspoiler.org> <20120829092318.GW33100@deviant.kiev.zoral.com.ua> <503F2D24.8050103@FreeBSD.org> <50463026.8000506@FreeBSD.org> In-Reply-To: <50463026.8000506@FreeBSD.org> X-Enigmail-Version: 1.4.4 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigD26EEE188AE2DD9859E2F28D" X-Gm-Message-State: ALoCoQkEuFXHN2N5ljDMKZ8qYVKeWS6OIA8Tbkf9QfZV+sYXMlCVtDnqxTPwo/lsRJePog8QsPNZ Cc: freebsd-arch@FreeBSD.org Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 19:17:41 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigD26EEE188AE2DD9859E2F28D Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 9/4/12 8:45 PM, Andriy Gapon wrote: > on 30/08/2012 12:06 Andrey Zonov said the following: >> Hi, >> >> So, I've got the first version of the patch (attached) which fixes >> memory locked limit checking and accounting. >=20 > Andrey, >=20 > your mlock.patch looks good to me, but I haven't verified pieces under = RACCT. > Please try to get a review from a person who is knee-deep in the VM cod= e like alc > or your mentor. >=20 Thanks for review! > The code should also be sent for vetoing to security@. Not sure if you= would get > a review there, but absence of nays would be good. >=20 > When the code is ready to be committed, please remember about > memorylocked=3Dunlimited in the default entry of the default login.conf= =2E A big > warning about it will have to be posted (in UPDATING and current@/stabl= e@ at the > very least). >=20 After that amd(8), geli(8) and watchdogd(8) will be broken, because they call mlockall(2). ntpd(8) won't, it already raises its RLIMIT_MEMLOCK. I will prepare patches for raising limits if there is no other solution.= > Thank you very much for doing this work. >=20 > P.S. It would probably make sense to provide some HTTP home for this p= atch as well. >=20 Updated patch is here [1]. [1] http://people.freebsd.org/~zont/mlock1.patch --=20 Andrey Zonov --------------enigD26EEE188AE2DD9859E2F28D Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.18 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJQRlPQAAoJEBWLemxX/CvTczAH/3wO0/IVDWfWfcqIJYr1i2J1 z745vwut6TjF6czsSCmMHCIf9nvj0wOGr/YbLl8YYHLZy6aWSD+En1O65ZemsLYX Dt0jopdVDGXJSSh2QPbcYGfaOJfmCAteRPRVEfV75QoavsT3ZVETXD/Sz8Mjl1Dp o/qqnw91irVVoEQA7c6dxzvnYWOEQ4TiGZlPa57qjvRSXBuWq/9P8jEHVjNW9xJ+ 2KanYCRH8iNUCigN94aqvcA19l/cKRQc+P4i0LpDKPibUlqoH7gnyOeR3WYFXOW+ SaBsApEaaULwcqFRz4L0LZTvAT9ctUL9oO1kt/OnbGc6H71aHypuuMWsgE5EGpo= =vU6j -----END PGP SIGNATURE----- --------------enigD26EEE188AE2DD9859E2F28D-- From owner-freebsd-arch@FreeBSD.ORG Tue Sep 4 22:17:44 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 555961065689; Tue, 4 Sep 2012 22:17:43 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 292008FC12; Tue, 4 Sep 2012 22:17:43 +0000 (UTC) Received: by mail-pb0-f54.google.com with SMTP id rp2so10217631pbb.13 for ; Tue, 04 Sep 2012 15:17:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=WFV/jdSomBAX4oy1gw3KP28+aIFmBzeDiyGyn8f5a9Y=; b=0Mn9lDvQ3HPEA7LUuBUXDamkh/0LTnSGbM2+vZnEGrg1ZnC+qxf1TS3lVoT2pV7YjO kcWbvc84+fo4wDw3lb6feDut2iCrHcZ81Erhv4xAQjoSsAxwPcWw1vtD24FJ7TxhhKvh i0Zvs86EVah9j/dbb5Xh1ydrDXF2M9l4yDvcV3XojfYXljqmnYymxm0fxB/DnV845ezm 8AUhzmmxnc8NM0nxn2hCppu/EivSoJTSLrxBI9KBm/AglNmYXs1peTQR5+L4xoWt/DAA V6eiHHnTVCAe3l7kT1PsG06leAssk6xtnglYvFfPE7ndNwb6YZ1a5vfidRTd2ebmI+00 ZoLg== MIME-Version: 1.0 Received: by 10.68.227.233 with SMTP id sd9mr48939697pbc.48.1346797063028; Tue, 04 Sep 2012 15:17:43 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.68.36.106 with HTTP; Tue, 4 Sep 2012 15:17:42 -0700 (PDT) In-Reply-To: <50463610.6070805@FreeBSD.org> References: <503DD433.2030108@FreeBSD.org> <201208290906.q7T96C9j032802@gw.catspoiler.org> <20120829092318.GW33100@deviant.kiev.zoral.com.ua> <503F2D24.8050103@FreeBSD.org> <503F476E.1010505@FreeBSD.org> <50463610.6070805@FreeBSD.org> Date: Tue, 4 Sep 2012 15:17:42 -0700 X-Google-Sender-Auth: pMXQwYEWCftFqbS1eSHLhgraL04 Message-ID: From: Adrian Chadd To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Cc: Andrey Zonov , Edward Tomasz Napierala , freebsd-arch@freebsd.org Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 22:17:44 -0000 Hi, On 4 September 2012 10:10, Andriy Gapon wrote: > And it also makes me wonder why kern/kern_racct.c is marked as standard while > all(?) uses of racct API are placed under RACCT option. > Ditto for kern_rctl.c/RCTL. > I think that excluding these file if the options are not used would help to catch > cases where the API is used unconditionally and it would also help to reduce > kernel sizes a tiny bit too. Thank you very much for considering kernel bloat here. It's this exact kind of tiny bloat that has caused FreeBSD to blow up through "death by a thousand cuts". Anything that we can keep from growing the kernel size is going to make it easier to embed on smaller devices - and there's many, many more embedded targets out there these days versus "PCs" (or macs. :-) Adrian From owner-freebsd-arch@FreeBSD.ORG Tue Sep 4 23:37:28 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9D738106573C; Tue, 4 Sep 2012 23:37:28 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id EC7678FC12; Tue, 4 Sep 2012 23:37:27 +0000 (UTC) Received: by iayy25 with SMTP id y25so240193iay.13 for ; Tue, 04 Sep 2012 16:37:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=GxysXVR+EzCS6fPeNPbcAlSWzkgP3gefxl5myc8GRtA=; b=DDwSOdGEci4bFRN/1iA17O9T2hHAqDcE93aTJBzWL3kq6ajBdwfUljMaH2f03xfLjK NSsAEA9sOEhIUP1gFiFoCTpZrk/uhQe0B/Ai0nKbk/XEoBjXQ0/P/Gtqu1P6nQLzfjv6 XP20qVhelFJ+rPqE+dShcl3N1V2J46cnIpF/s2nlEOzwX1RgMhqkXF4wIEt3w9S0+Fm5 O9SPJ31C9sOu3RyHTTuwIPkCgqU4xhfp4B8tUKRL8ToF32SF9zlY6F9IIlCiZyEzlXR6 KXEef6J1eUY8wAzadB6OpcARZpXyoo3tqcH4OrpWZiI+os+pDwH1w9bTuOLcMKcJ4QuG 1wOA== MIME-Version: 1.0 Received: by 10.60.13.232 with SMTP id k8mr16231826oec.81.1346801847419; Tue, 04 Sep 2012 16:37:27 -0700 (PDT) Received: by 10.76.142.201 with HTTP; Tue, 4 Sep 2012 16:37:27 -0700 (PDT) In-Reply-To: References: <503DD433.2030108@FreeBSD.org> <201208290906.q7T96C9j032802@gw.catspoiler.org> <20120829092318.GW33100@deviant.kiev.zoral.com.ua> <503F2D24.8050103@FreeBSD.org> <503F476E.1010505@FreeBSD.org> <50463610.6070805@FreeBSD.org> Date: Tue, 4 Sep 2012 16:37:27 -0700 Message-ID: From: Garrett Cooper To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 Cc: Andrey Zonov , Edward Tomasz Napierala , Andriy Gapon , freebsd-arch@freebsd.org Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 23:37:28 -0000 On Tue, Sep 4, 2012 at 3:17 PM, Adrian Chadd wrote: > Hi, > > On 4 September 2012 10:10, Andriy Gapon wrote: > >> And it also makes me wonder why kern/kern_racct.c is marked as standard while >> all(?) uses of racct API are placed under RACCT option. >> Ditto for kern_rctl.c/RCTL. >> I think that excluding these file if the options are not used would help to catch >> cases where the API is used unconditionally and it would also help to reduce >> kernel sizes a tiny bit too. > > Thank you very much for considering kernel bloat here. It's this exact > kind of tiny bloat that has caused FreeBSD to blow up through "death > by a thousand cuts". > > Anything that we can keep from growing the kernel size is going to > make it easier to embed on smaller devices - and there's many, many > more embedded targets out there these days versus "PCs" (or macs. :-) Hear hear! -Garrett From owner-freebsd-arch@FreeBSD.ORG Wed Sep 5 06:33:07 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 07E55106564A; Wed, 5 Sep 2012 06:33:07 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id C19648FC14; Wed, 5 Sep 2012 06:33:05 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id JAA12902; Wed, 05 Sep 2012 09:33:04 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1T99Ad-0002MC-VQ; Wed, 05 Sep 2012 09:33:04 +0300 Message-ID: <5046F21F.2080603@FreeBSD.org> Date: Wed, 05 Sep 2012 09:33:03 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120901 Thunderbird/15.0 MIME-Version: 1.0 To: Andrey Zonov References: <503DD433.2030108@FreeBSD.org> <201208290906.q7T96C9j032802@gw.catspoiler.org> <20120829092318.GW33100@deviant.kiev.zoral.com.ua> <503F2D24.8050103@FreeBSD.org> <503F476E.1010505@FreeBSD.org> <50463610.6070805@FreeBSD.org> <50463EEB.60207@FreeBSD.org> In-Reply-To: <50463EEB.60207@FreeBSD.org> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Edward Tomasz Napierala , freebsd-arch@FreeBSD.org Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 06:33:07 -0000 on 04/09/2012 20:48 Andrey Zonov said the following: > On 9/4/12 9:10 PM, Andriy Gapon wrote: >> on 30/08/2012 13:58 Andrey Zonov said the following: [snip] >>> [2] http://people.freebsd.org/~zont/racct.patch >> >> This patch looks correct. >> > > There is no need for this patch as I mentioned earlier. racct_set() > doesn't add any additional locking here. I thought that hiding the racct call behind RACCT was a worthy target on its own. >> And it also makes me wonder why kern/kern_racct.c is marked as standard while >> all(?) uses of racct API are placed under RACCT option. > > Not all. I think some code was not easy to put under RACCT. But perhaps it should still have been a goal for this optional feature. Unfortunately, Edward hasn't replied yet. >> Ditto for kern_rctl.c/RCTL. >> I think that excluding these file if the options are not used would help to catch >> cases where the API is used unconditionally and it would also help to reduce >> kernel sizes a tiny bit too. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Wed Sep 5 06:44:53 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 39662106566C; Wed, 5 Sep 2012 06:44:53 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 0A1CE8FC08; Wed, 5 Sep 2012 06:44:51 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id JAA12987; Wed, 05 Sep 2012 09:44:50 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1T99M2-0002N2-5l; Wed, 05 Sep 2012 09:44:50 +0300 Message-ID: <5046F4E0.6000606@FreeBSD.org> Date: Wed, 05 Sep 2012 09:44:48 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120901 Thunderbird/15.0 MIME-Version: 1.0 To: Andrey Zonov References: <503DD433.2030108@FreeBSD.org> <201208290906.q7T96C9j032802@gw.catspoiler.org> <20120829092318.GW33100@deviant.kiev.zoral.com.ua> <503F2D24.8050103@FreeBSD.org> <50463026.8000506@FreeBSD.org> <504653CD.2000707@FreeBSD.org> In-Reply-To: <504653CD.2000707@FreeBSD.org> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org Subject: Re: [patch] unprivileged mlock(2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 06:44:53 -0000 on 04/09/2012 22:17 Andrey Zonov said the following: > On 9/4/12 8:45 PM, Andriy Gapon wrote: >> on 30/08/2012 12:06 Andrey Zonov said the following: >>> Hi, >>> >>> So, I've got the first version of the patch (attached) which fixes >>> memory locked limit checking and accounting. >> >> Andrey, >> >> your mlock.patch looks good to me, but I haven't verified pieces under >> RACCT. Please try to get a review from a person who is knee-deep in the >> VM code like alc or your mentor. >> > > Thanks for review! > >> The code should also be sent for vetoing to security@. Not sure if you >> would get a review there, but absence of nays would be good. >> >> When the code is ready to be committed, please remember about >> memorylocked=unlimited in the default entry of the default login.conf. A >> big warning about it will have to be posted (in UPDATING and >> current@/stable@ at the very least). >> > > After that amd(8), geli(8) and watchdogd(8) will be broken, because they > call mlockall(2). ntpd(8) won't, it already raises its RLIMIT_MEMLOCK. I > will prepare patches for raising limits if there is no other solution. Thanks for working on this. BTW, I am not sure why those applications would get broken... We could/should still have memorylocked=unlimited for the 'root' class. Or is it about something else? >> Thank you very much for doing this work. >> >> P.S. It would probably make sense to provide some HTTP home for this >> patch as well. >> > > Updated patch is here [1]. > > [1] http://people.freebsd.org/~zont/mlock1.patch > Thank you! One additional thing - we probably should retire PRIV_VM_MLOCK and PRIV_VM_MUNLOCK. That would include making changes to sys/i386/ibcs2/ibcs2_misc.c and sys/ofed/drivers/infiniband/core/umem.c. P.S. PRIV_VM_MUNLOCK _privilege_ feels a little bit weird. I wonder what was the intended use for it (if any)... -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Wed Sep 5 10:33:20 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [69.147.83.53]) by hub.freebsd.org (Postfix) with ESMTP id 57EBE106566B; Wed, 5 Sep 2012 10:33:20 +0000 (UTC) (envelope-from melifaro@FreeBSD.org) Received: from dhcp170-36-red.yandex.net (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx2.freebsd.org (Postfix) with ESMTP id 0019814D862; Wed, 5 Sep 2012 10:32:57 +0000 (UTC) Message-ID: <50472A18.5050909@FreeBSD.org> Date: Wed, 05 Sep 2012 14:31:52 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120627 Thunderbird/13.0.1 MIME-Version: 1.0 To: net@freebsd.org Content-Type: multipart/mixed; boundary="------------000105030205080004040409" Cc: arch@freebsd.org, Gleb Smirnoff Subject: [PATCH] Using PFIL lock in packet filters X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 10:33:20 -0000 This is a multi-part message in MIME format. --------------000105030205080004040409 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hello list! Currently we have the following locking strategy in our firewalls: On every input/output IP packet PFIL acquires read lock while traversing list and after that reader (e.g. firewall handler) acquires its own lock (rwlock in case of ipfw). Pfil framework currently uses per-head rmlock (e.g. different lock for IPv4 and IPv6 per each VNET instance). Since most popular firewalls (ipfw and pf) uses per-VNET rwlock(*), per-AF pfil locks does not give us much benefit. Proposed idea is to: 1) Use single pfil lock per VNET instance by default. 2) Permit filters to use this lock via pfil_[rw]_[un]lock functions. Patch for pfil and ipfw changes attached. I've got several production routers running for a week with previous version of this patch without any (ipfw-related) problems. Performance difference is quite significant, more details here: http://lists.freebsd.org/pipermail/freebsd-net/2012-July/032842.html I'm planning to commit these patches (with minor changes) at the beginning of next week if nobody objects. (*) currently pf uses mutex, but hopefully glebius@ is working on much more scalable solution -- WBR, Alexander --------------000105030205080004040409 Content-Type: text/plain; charset=UTF-8; name="0001-Export-pfil-lock.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-Export-pfil-lock.patch" >From 347690db1c4ecaf267f3c027e18ea38734d28d42 Mon Sep 17 00:00:00 2001 From: "Alexander V. Chernikov" Date: Wed, 5 Sep 2012 13:09:16 +0400 Subject: [PATCH 1/2] Export pfil lock --- share/man/man9/pfil.9 | 55 ++++++++++++++++++++++++++++++++++++++++++++-- sys/net/pfil.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++++ sys/net/pfil.h | 46 ++++++++++++++++++++++++++++++--------- 3 files changed, 147 insertions(+), 12 deletions(-) diff --git a/share/man/man9/pfil.9 b/share/man/man9/pfil.9 index 36a4d47..a76168b 100644 --- a/share/man/man9/pfil.9 +++ b/share/man/man9/pfil.9 @@ -28,7 +28,7 @@ .\" .\" $FreeBSD: stable/8/share/man/man9/pfil.9 162404 2006-09-18 15:24:20Z ru $ .\" -.Dd September 29, 2004 +.Dd September 5, 2012 .Dt PFIL 9 .Os .Sh NAME @@ -39,7 +39,11 @@ .Nm pfil_hook_get , .Nm pfil_add_hook , .Nm pfil_remove_hook , -.Nm pfil_run_hooks +.Nm pfil_run_hooks , +.Nm pfil_rlock +.Nm pfil_runlock +.Nm pfil_wlock +.Nm pfil_wunlock .Nd packet filter interface .Sh SYNOPSIS .In sys/param.h @@ -62,6 +66,14 @@ .Fn (*func) "void *arg" "struct mbuf **mp" "struct ifnet *" "int dir" "struct inpcb *" .Ft int .Fn pfil_run_hooks "struct pfil_head *head" "struct mbuf **mp" "struct ifnet *" "int dir" "struct inpcb *" +.Ft void +.Fn pfil_rlock "struct pfil_head *" "struct rm_priotracker *" +.Ft void +.Fn pfil_runlock "struct pfil_head *" "struct rm_priotracker *" +.Ft void +.Fn pfil_wlock "struct pfil_head *" +.Ft void +.Fn pfil_wunlock "struct pfil_head *" .Sh DESCRIPTION The .Nm @@ -86,6 +98,16 @@ The data link type is a .Xr bpf 4 DLT constant indicating what kind of header is present on the packet at the filtering point. +Each filtering point uses common per-VNET rmlock by default. +This can be changed by specifying +.Vt PFIL_FLAG_PRIVATE_LOCK +as +.Vt "flags" +field in the +.Vt pfil_head +structure. +Note that specifying private lock can break filters sharing the same +ruleset and/or state between different data link types. Filtering points may be unregistered with the .Fn pfil_head_unregister function. @@ -122,6 +144,31 @@ The filter returns an error (errno) if the packet processing is to stop, or 0 if the processing is to continue. If the packet processing is to stop, it is the responsibility of the filter to free the packet. +.Pp +Every filter hook is called with +.Nm +read lock held. +All heads uses the same lock within the same VNET instance. +Packet filter can use this lock instead of own locking model to +improve performance. +Since +.Nm +uses +.Xr rmlock 9 +.Fn pfil_rlock +and +.Fn pfil_runlock +require +.Va struct rm_priotracker +to be passed as argument. +Filter can acquire and release writer lock via +.Fn pfil_wlock +and +.Fn pfil_wunlock +functions. +See +.Xr rmlock 9 +for more details. .Sh RETURN VALUES If successful, .Fn pfil_head_get @@ -146,6 +193,7 @@ might sleep! .Sh SEE ALSO .Xr bpf 4 , .Xr if_bridge 4 +.Xr rmlock 4 .Sh HISTORY The .Nm @@ -181,6 +229,9 @@ as well as be less IP-centric. .Pp Fine-grained locking was added in .Fx 5.2 . +.Nm +lock export was added in +.Fx 10.0 . .Sh BUGS The .Fn pfil_hook_get diff --git a/sys/net/pfil.c b/sys/net/pfil.c index 788ca24..6c48334 100644 --- a/sys/net/pfil.c +++ b/sys/net/pfil.c @@ -61,6 +61,8 @@ static int pfil_list_remove(pfil_list_t *, LIST_HEAD(pfilheadhead, pfil_head); VNET_DEFINE(struct pfilheadhead, pfil_head_list); #define V_pfil_head_list VNET(pfil_head_list) +VNET_DEFINE(struct rmlock, pfil_lock); +#define V_pfil_lock VNET(pfil_lock) /* * pfil_run_hooks() runs the specified packet filter hooks. @@ -91,6 +93,60 @@ pfil_run_hooks(struct pfil_head *ph, struct mbuf **mp, struct ifnet *ifp, } /* + * pfil_try_rlock() acquires rm reader lock for specified head + * if this is immediately possible, + */ +int +pfil_try_rlock(struct pfil_head *ph, struct rm_priotracker *tracker) +{ + return PFIL_TRY_RLOCK(ph, tracker); +} + +/* + * pfil_rlock() acquires rm reader lock for specified head. + */ +void +pfil_rlock(struct pfil_head *ph, struct rm_priotracker *tracker) +{ + PFIL_RLOCK(ph, tracker); +} + +/* + * pfil_runlock() releases reader lock for specified head. + */ +void +pfil_runlock(struct pfil_head *ph, struct rm_priotracker *tracker) +{ + PFIL_RUNLOCK(ph, tracker); +} + +/* + * pfil_wlock() acquires writer lock for specified head. + */ +void +pfil_wlock(struct pfil_head *ph) +{ + PFIL_WLOCK(ph); +} + +/* + * pfil_wunlock() releases writer lock for specified head. + */ +void +pfil_wunlock(struct pfil_head *ph) +{ + PFIL_WUNLOCK(ph); +} + +/* + * pfil_wowned() releases writer lock for specified head. + */ +int +pfil_wowned(struct pfil_head *ph) +{ + return PFIL_WOWNED(ph); +} +/* * pfil_head_register() registers a pfil_head with the packet filter hook * mechanism. */ @@ -295,6 +351,7 @@ vnet_pfil_init(const void *unused) { LIST_INIT(&V_pfil_head_list); + PFIL_LOCK_INIT_REAL(&V_pfil_lock, "shared"); return (0); } @@ -306,6 +363,7 @@ vnet_pfil_uninit(const void *unused) { /* XXX should panic if list is not empty */ + PFIL_LOCK_DESTROY_REAL(&V_pfil_lock); return (0); } diff --git a/sys/net/pfil.h b/sys/net/pfil.h index d314a72..7c99148 100644 --- a/sys/net/pfil.h +++ b/sys/net/pfil.h @@ -64,6 +64,8 @@ typedef TAILQ_HEAD(pfil_list, packet_filter_hook) pfil_list_t; #define PFIL_TYPE_AF 1 /* key is AF_* type */ #define PFIL_TYPE_IFNET 2 /* key is ifnet pointer */ +#define PFIL_FLAG_PRIVATE_LOCK 0x01 /* Personal lock instead of global */ + struct pfil_head { pfil_list_t ph_in; pfil_list_t ph_out; @@ -72,7 +74,9 @@ struct pfil_head { #if defined( __linux__ ) || defined( _WIN32 ) rwlock_t ph_mtx; #else - struct rmlock ph_lock; + struct rmlock *ph_plock; /* Pointer to the used lock */ + struct rmlock ph_lock; /* Private lock storage */ + int flags; #endif union { u_long phu_val; @@ -90,21 +94,43 @@ int pfil_remove_hook(int (*func)(void *, struct mbuf **, struct ifnet *, int pfil_run_hooks(struct pfil_head *, struct mbuf **, struct ifnet *, int, struct inpcb *inp); +struct rm_priotracker; /* Do not require including rmlock header */ +int pfil_try_rlock(struct pfil_head *, struct rm_priotracker *); +void pfil_rlock(struct pfil_head *, struct rm_priotracker *); +void pfil_runlock(struct pfil_head *, struct rm_priotracker *); +void pfil_wlock(struct pfil_head *); +void pfil_wunlock(struct pfil_head *); +int pfil_wowned(struct pfil_head *ph); + int pfil_head_register(struct pfil_head *); int pfil_head_unregister(struct pfil_head *); struct pfil_head *pfil_head_get(int, u_long); #define PFIL_HOOKED(p) ((p)->ph_nhooks > 0) -#define PFIL_LOCK_INIT(p) \ - rm_init_flags(&(p)->ph_lock, "PFil hook read/write mutex", RM_RECURSE) -#define PFIL_LOCK_DESTROY(p) rm_destroy(&(p)->ph_lock) -#define PFIL_RLOCK(p, t) rm_rlock(&(p)->ph_lock, (t)) -#define PFIL_WLOCK(p) rm_wlock(&(p)->ph_lock) -#define PFIL_RUNLOCK(p, t) rm_runlock(&(p)->ph_lock, (t)) -#define PFIL_WUNLOCK(p) rm_wunlock(&(p)->ph_lock) -#define PFIL_LIST_LOCK() mtx_lock(&pfil_global_lock) -#define PFIL_LIST_UNLOCK() mtx_unlock(&pfil_global_lock) +#define PFIL_LOCK_INIT_REAL(l, t) \ + rm_init_flags(l, "PFil " t " rmlock", RM_RECURSE) +#define PFIL_LOCK_DESTROY_REAL(l) \ + rm_destroy(l) +#define PFIL_LOCK_INIT(p) do { \ + if ((p)->flags & PFIL_FLAG_PRIVATE_LOCK) { \ + PFIL_LOCK_INIT_REAL(&(p)->ph_lock, "private"); \ + (p)->ph_plock = &(p)->ph_lock; \ + } else \ + (p)->ph_plock = &V_pfil_lock; \ +} while (0) +#define PFIL_LOCK_DESTROY(p) do { \ + if ((p)->flags & PFIL_FLAG_PRIVATE_LOCK) \ + PFIL_LOCK_DESTROY_REAL((p)->ph_plock); \ +} while (0) +#define PFIL_TRY_RLOCK(p, t) rm_try_rlock((p)->ph_plock, (t)) +#define PFIL_RLOCK(p, t) rm_rlock((p)->ph_plock, (t)) +#define PFIL_WLOCK(p) rm_wlock((p)->ph_plock) +#define PFIL_RUNLOCK(p, t) rm_runlock((p)->ph_plock, (t)) +#define PFIL_WUNLOCK(p) rm_wunlock((p)->ph_plock) +#define PFIL_WOWNED(p) rm_wowned((p)->ph_plock) +#define PFIL_LIST_LOCK() mtx_lock(&pfil_global_lock) +#define PFIL_LIST_UNLOCK() mtx_unlock(&pfil_global_lock) static __inline struct packet_filter_hook * pfil_hook_get(int dir, struct pfil_head *ph) -- 1.7.9.4 --------------000105030205080004040409 Content-Type: text/plain; charset=UTF-8; name="0002-Make-ipfw-use-pfil-hooks.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0002-Make-ipfw-use-pfil-hooks.patch" >From 808f85d5bc8b0ac12bd9b11c6fa1f8b44ad936cd Mon Sep 17 00:00:00 2001 From: "Alexander V. Chernikov" Date: Wed, 5 Sep 2012 13:10:15 +0400 Subject: [PATCH 2/2] Make ipfw use pfil hooks --- sys/netinet/ipfw/ip_fw2.c | 22 ++++++++++++++++++++++ sys/netinet/ipfw/ip_fw_nat.c | 18 ++++++++++++++++++ sys/netinet/ipfw/ip_fw_private.h | 17 +++++++++-------- sys/netinet/ipfw/ip_fw_sockopt.c | 4 ++++ sys/netinet/ipfw/ip_fw_table.c | 1 + 5 files changed, 54 insertions(+), 8 deletions(-) diff --git a/sys/netinet/ipfw/ip_fw2.c b/sys/netinet/ipfw/ip_fw2.c index 352fd4a..bee16a3 100644 --- a/sys/netinet/ipfw/ip_fw2.c +++ b/sys/netinet/ipfw/ip_fw2.c @@ -62,6 +62,7 @@ __FBSDID("$FreeBSD: head/sys/netinet/ipfw/ip_fw2.c 240099 2012-09-04 19:43:26Z m #include #include #include +#include #include #include @@ -785,6 +786,10 @@ set_match(struct ip_fw_args *args, int slot, * All arguments are in args so we can modify them and return them * back to the caller. * + * We assume ipfw_chk is ALWAYS called from PFIL hook so + * read lock is alredy held. If this is not the case, PFIL + * read lock has to be acquired manually before calling ipfw_chk() + * * Parameters: * * args->m (in/out) The packet; we set to NULL when/if we nuke it. @@ -1203,9 +1208,13 @@ do { \ args->f_id.dst_port = dst_port = ntohs(dst_port); } +#if defined(__linux__) || defined(_WIN32) IPFW_RLOCK(chain); +#endif if (! V_ipfw_vnet_ready) { /* shutting down, leave NOW. */ +#if defined(__linux__) || defined(_WIN32) IPFW_RUNLOCK(chain); +#endif return (IP_FW_PASS); /* accept */ } if (args->rule.slot) { @@ -2476,7 +2485,9 @@ do { \ retval = IP_FW_DENY; printf("ipfw: ouch!, skip past end of rules, denying packet\n"); } +#if defined(__linux__) || defined(_WIN32) IPFW_RUNLOCK(chain); +#endif #ifdef __FreeBSD__ if (ucred_cache != NULL) crfree(ucred_cache); @@ -2639,6 +2650,17 @@ vnet_ipfw_init(const void *unused) chain->id = rule->id = 1; IPFW_LOCK_INIT(chain); +#ifdef __FreeBSD__ +#ifdef INET + chain->phead = pfil_head_get(PFIL_TYPE_AF, AF_INET); +#else + chain->phead = pfil_head_get(PFIL_TYPE_AF, AF_INET6); +#endif + if (error) { + printf("ipfw2: PFIL lock setup failed"); + return (ENOENT); + } +#endif ipfw_dyn_init(); /* First set up some values that are compile time options */ diff --git a/sys/netinet/ipfw/ip_fw_nat.c b/sys/netinet/ipfw/ip_fw_nat.c index 7b3f3a3..1e9b6af 100644 --- a/sys/netinet/ipfw/ip_fw_nat.c +++ b/sys/netinet/ipfw/ip_fw_nat.c @@ -42,6 +42,7 @@ __FBSDID("$FreeBSD: head/sys/netinet/ipfw/ip_fw_nat.c 231991 2012-02-22 04:19:33 #include #include +#include #include #include #include @@ -201,6 +202,13 @@ add_redir_spool_cfg(char *buf, struct cfg_nat *ptr) } } +/* + * ipfw_nat - perform mbuf header translation. + * + * We assume ipfw_nat is ALWAYS called from ipfw_chk so + * PFIL read lock is alredy held. If this is not the case, + * read lock has to be acquired manually before calling ipfw_nat() + */ static int ipfw_nat(struct ip_fw_args *args, struct cfg_nat *t, struct mbuf *m) { @@ -268,7 +276,9 @@ ipfw_nat(struct ip_fw_args *args, struct cfg_nat *t, struct mbuf *m) found = 0; chain = &V_layer3_chain; +#if defined(__linux__) || defined(_WIN32) IPFW_RLOCK(chain); +#endif /* Check every nat entry... */ LIST_FOREACH(t, &chain->nat, _next) { if ((t->mode & PKT_ALIAS_SKIP_GLOBAL) != 0) @@ -281,7 +291,9 @@ ipfw_nat(struct ip_fw_args *args, struct cfg_nat *t, struct mbuf *m) break; } } +#if defined(__linux__) || defined(_WIN32) IPFW_RUNLOCK(chain); +#endif if (found != 1) { /* No instance found, return ignore */ args->m = mcl; @@ -493,6 +505,9 @@ ipfw_nat_get_cfg(struct sockopt *sopt) struct cfg_spool *s; char *data; int gencnt, nat_cnt, len, error; +#ifdef __FreeBSD__ + struct rm_priotracker tracker; +#endif nat_cnt = 0; len = sizeof(nat_cnt); @@ -551,6 +566,9 @@ ipfw_nat_get_log(struct sockopt *sopt) struct cfg_nat *ptr; int i, size; struct ip_fw_chain *chain; +#ifdef __FreeBSD__ + struct rm_priotracker tracker; +#endif chain = &V_layer3_chain; diff --git a/sys/netinet/ipfw/ip_fw_private.h b/sys/netinet/ipfw/ip_fw_private.h index 1cb1115..44908d8 100644 --- a/sys/netinet/ipfw/ip_fw_private.h +++ b/sys/netinet/ipfw/ip_fw_private.h @@ -212,6 +212,8 @@ VNET_DECLARE(int, autoinc_step); VNET_DECLARE(unsigned int, fw_tables_max); #define V_fw_tables_max VNET(fw_tables_max) +struct pfil_head; + struct ip_fw_chain { struct ip_fw *rules; /* list of rules */ struct ip_fw *reap; /* list of rules to reap */ @@ -227,7 +229,7 @@ struct ip_fw_chain { spinlock_t rwmtx; spinlock_t uh_lock; #else - struct rwlock rwmtx; + struct pfil_head *phead; /* PFIL head used for locking */ struct rwlock uh_lock; /* lock for upper half */ #endif uint32_t id; /* ruleset id */ @@ -241,22 +243,21 @@ struct sockopt; /* used by tcp_var.h */ * so the variable and the macros must be here. */ +/* Additional locking init is done in vnet_ipfw_init() */ #define IPFW_LOCK_INIT(_chain) do { \ - rw_init(&(_chain)->rwmtx, "IPFW static rules"); \ rw_init(&(_chain)->uh_lock, "IPFW UH lock"); \ } while (0) #define IPFW_LOCK_DESTROY(_chain) do { \ - rw_destroy(&(_chain)->rwmtx); \ rw_destroy(&(_chain)->uh_lock); \ } while (0) -#define IPFW_WLOCK_ASSERT(_chain) rw_assert(&(_chain)->rwmtx, RA_WLOCKED) +#define IPFW_WLOCK_ASSERT(_chain) -#define IPFW_RLOCK(p) rw_rlock(&(p)->rwmtx) -#define IPFW_RUNLOCK(p) rw_runlock(&(p)->rwmtx) -#define IPFW_WLOCK(p) rw_wlock(&(p)->rwmtx) -#define IPFW_WUNLOCK(p) rw_wunlock(&(p)->rwmtx) +#define IPFW_RLOCK(p) pfil_rlock((p)->phead, &tracker) +#define IPFW_RUNLOCK(p) pfil_runlock((p)->phead, &tracker) +#define IPFW_WLOCK(p) pfil_wlock((p)->phead) +#define IPFW_WUNLOCK(p) pfil_wunlock((p)->phead) #define IPFW_UH_RLOCK(p) rw_rlock(&(p)->uh_lock) #define IPFW_UH_RUNLOCK(p) rw_runlock(&(p)->uh_lock) diff --git a/sys/netinet/ipfw/ip_fw_sockopt.c b/sys/netinet/ipfw/ip_fw_sockopt.c index a245143..b67b25d 100644 --- a/sys/netinet/ipfw/ip_fw_sockopt.c +++ b/sys/netinet/ipfw/ip_fw_sockopt.c @@ -56,6 +56,7 @@ __FBSDID("$FreeBSD: head/sys/netinet/ipfw/ip_fw_sockopt.c 233745 2012-03-31 11:2 #include #include #include +#include #include #include /* hooks */ @@ -953,6 +954,9 @@ ipfw_ctl(struct sockopt *sopt) u_int32_t rulenum[2]; uint32_t opt; char xbuf[128]; +#ifdef __FreeBSD__ + struct rm_priotracker tracker; +#endif ip_fw3_opheader *op3 = NULL; error = priv_check(sopt->sopt_td, PRIV_NETINET_IPFW); diff --git a/sys/netinet/ipfw/ip_fw_table.c b/sys/netinet/ipfw/ip_fw_table.c index d05c916..3eb412d 100644 --- a/sys/netinet/ipfw/ip_fw_table.c +++ b/sys/netinet/ipfw/ip_fw_table.c @@ -55,6 +55,7 @@ __FBSDID("$FreeBSD: head/sys/netinet/ipfw/ip_fw_table.c 238265 2012-07-08 21:13: #include #include /* ip_fw.h requires IFNAMSIZ */ #include +#include #include #include -- 1.7.9.4 --------------000105030205080004040409-- From owner-freebsd-arch@FreeBSD.ORG Wed Sep 5 14:26:54 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 780B0106564A for ; Wed, 5 Sep 2012 14:26:54 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id 346568FC12 for ; Wed, 5 Sep 2012 14:26:53 +0000 (UTC) Received: by iebc12 with SMTP id c12so1580466ieb.13 for ; Wed, 05 Sep 2012 07:26:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=llCQJXE1CKAl3BXFYPV+KVu5wNwU9OC8bBCXzCgtCJw=; b=H9s0iRZo1Gl+MT9lPFlD/5UJZJz90ONLIzy8/tEIbAUKj+WONPspqHv9PXOFnllU7a /Wi62S4xtsxchpCLqDLsFQZ+f/rPjYKB2GaV9WJbUdvemU2feVKVRF+4Kr6luKy7XVUN 79j1KdUlnHqGfofBD63X45EdvAjRecOZfFCc9MY7kUfjHlEGyy4O/Bha2IH9F/pkC4gE aZlEnF7DB5rvElL+G/E1coXKTBSnxVxLOITDhdlUjFaMgTKvo7+r+TXp7sy7HgSSUx1g O1pYkFKbK6k/HRyospe06rkyRpR16wsZ1XGoKnv9MZAQ1FYqG/dWI0G/Z7VvTpidzwiO MWzQ== Received: by 10.43.131.193 with SMTP id hr1mr21414011icc.31.1346855213589; Wed, 05 Sep 2012 07:26:53 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id p5sm16226531igm.13.2012.09.05.07.26.48 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 05 Sep 2012 07:26:49 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <1346683262.1140.582.camel@revolution.hippie.lan> Date: Wed, 5 Sep 2012 08:26:46 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1346602529.1140.563.camel@revolution.hippie.lan> <3C91462E-1423-4AFC-B4C7-6239E698A98E@bsdimp.com> <1346683262.1140.582.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQloURbVyax81cjaCvxcf10vaAbAOiWZhD0ByRDYUGBxtih1h3Rb/69aHM+equxJRaCoDz1p Cc: freebsd-arch@freebsd.org, Luiz Otavio O Souza Subject: Re: spibus access serialization X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 14:26:54 -0000 On Sep 3, 2012, at 8:41 AM, Ian Lepore wrote: > On Sun, 2012-09-02 at 13:38 -0600, Warner Losh wrote: >>> Unfortunately, the atmel SPI controller will de-assert the chip >> select >>> when the transmit register (or PDC) runs out of data. It's a bug >> that >>> may only affect rm9200; I haven't checked the errata and docs for >> the >>> sam9 series to see if they fixed it. >>=20 >> Yes, that's been fixed (which is why I didn't think it would be a >> problem, but it is). >>=20 >>> The way to fix the problem is indeed to take over the handling of >> chip >>> selects in the driver, by taking the pins away from the device and >>> managing them as GPIOs. Doing that is on my to-do list, but I've >> been >>> waiting for some resolution of the "how do we manage device/gpio pin >>> setup across the atmel SoC family" questions. >>=20 >> That's the question still, alas. I've been looking at how FDT does = it >> in Linux land, and they punt on this issue. We'll likely have to put >> some effort into defining these things with atmel's FDT efforts. >> Their FDT comes close by defining groups, but doesn't seem to take it >> down to the individual pin to signal mapping.=20 >=20 > Hmmm, if the workaround is needed only on rm9200 hardware, then there > are only two choices for where the chip select pins live (PIOA or PIOD > banks). I think it should be possible to sniff out whether the chip > select pins have been configured to PIOAn or PIODn by reading the > periphid registers, so we could potentially get the SPI code fixed > without solving the difficult problem of managing pin assignments = across > different SoCs in the family. >=20 > I should have sam9 hardware arriving in a couple weeks, then I'll be > able to make changes and test that they work on rm9200 and sam9. I think a full test for this serialization might be the ENC28J60 SPI = Ethernet module. You can get one here on ebay: = http://www.ebay.com/itm/Mini-ENC28J60-Ethernet-LAN-Network-Module-For-51-A= VR-STM32-LPC-/140843724204?pt=3DBI_Electrical_Equipment_Tools&hash=3Ditem2= 0caf0adac for < $5.00. There are Linux drivers available, and the datasheet for = this item appears to be fairly complete. This board would be easy to = connect to the SPI bus of the G45 board you are getting with standard = jumpers. I recently bought a bunch on Amazon for about $5.00. If you = could support this on a board that also had a dataflash, you'd be sure = that the spi system was good. However, in looking at it, we'd need to beef up the at91 gpio stuff a = bit, since it needs an interrupt line and a reset line. Warner= From owner-freebsd-arch@FreeBSD.ORG Wed Sep 5 14:30:47 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7931B1065670 for ; Wed, 5 Sep 2012 14:30:47 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id 37CC68FC0C for ; Wed, 5 Sep 2012 14:30:46 +0000 (UTC) Received: by iebc12 with SMTP id c12so1588714ieb.13 for ; Wed, 05 Sep 2012 07:30:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=UF35nJTuAgSgj8n2f5foIsjGJceRm+InzWcKZykZ1hc=; b=OXhFETjFEqCq+PdSHxb/3eVl/ApycoBQgoaHG7vqmX8qYqjCPzfx/Wvm2wfLEmXplf CsHDLTXrV1nUDoUa5T5LMCY407uN3RrKK/gVayySQ8n4MPklNtf/jZ/yRNqRf+dCTXgz tdNLEfHHHT+f7onkXLDbaqYRy+YJ1+tkD7/V2AQVgN/Aq+7VuLA7oe0FXLPykEhpT4sU RKVC6gGLDI+i95L/Qc1/qBFogX5TqwLdyP8ebubls6jiLeJrxfxeczkAwM8f4dSJTcXS gjH8olRCMIF4GVxFljZiM5PS5Zv73n9CkF9l8nM9iC/PnmcfPW5soPKBHW7fmWlZHauk eO2g== Received: by 10.50.33.138 with SMTP id r10mr18327799igi.31.1346855446355; Wed, 05 Sep 2012 07:30:46 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id ce10sm16435034igb.1.2012.09.05.07.30.44 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 05 Sep 2012 07:30:45 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <1346689154.1140.601.camel@revolution.hippie.lan> Date: Wed, 5 Sep 2012 08:30:42 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <3AFC763F-011C-46B4-B500-FE21B704259F@bsdimp.com> References: <1346689154.1140.601.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQkpc3uXVS7e9Qj8IXpmCtsfj1NiPz6O0NBVIeIcXw6RKCxY7cq+ZtDkFaJC+8FdXlIFVt5i Cc: freebsd-arch@freebsd.org Subject: Re: Some busdma stats X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 14:30:47 -0000 On Sep 3, 2012, at 10:19 AM, Ian Lepore wrote: > I decided that a good way to learn more about the busdma subsystem = would > be to actually work with the code rather than just reading it.=20 Tis true. > Regardless of whether we eventually fix every driver to eliminate > transfers that aren't aligned to cache line boundaries, or somehow > change the busdma code to automatically bounce unaligned requests, we > need efficient allocation of small buffers aligned and sized to cache > lines. The issue can't be fixed in the busdma code because partial, unaligned = transfers are fine, so long as the calling code avoids the entire cache = line during the transfer. Returning cache-line aligned buffers from the = allocator will do that, of course, but it is also valid for the code to = only use part of the buffer for the transfer. > I wrote some code to use uma(9) to manage pools of aligned > buffers based on size, and set up a pool of uncachable/coherent = buffers > and a pool of "regular memory" buffers. Very cool stats. Need to review the code you posted though... Warner From owner-freebsd-arch@FreeBSD.ORG Wed Sep 5 15:10:35 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 61E751065758; Wed, 5 Sep 2012 15:10:35 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id CBC8D8FC08; Wed, 5 Sep 2012 15:10:22 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id q85FAMIF078927; Wed, 5 Sep 2012 09:10:22 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q85F9xhj042697; Wed, 5 Sep 2012 09:09:59 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: Warner Losh In-Reply-To: <3AFC763F-011C-46B4-B500-FE21B704259F@bsdimp.com> References: <1346689154.1140.601.camel@revolution.hippie.lan> <3AFC763F-011C-46B4-B500-FE21B704259F@bsdimp.com> Content-Type: text/plain; charset="us-ascii" Date: Wed, 05 Sep 2012 09:09:59 -0600 Message-ID: <1346857799.59094.66.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Some busdma stats X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 15:10:35 -0000 On Wed, 2012-09-05 at 08:30 -0600, Warner Losh wrote: > > Regardless of whether we eventually fix every driver to eliminate > > transfers that aren't aligned to cache line boundaries, or somehow > > change the busdma code to automatically bounce unaligned requests, > we > > need efficient allocation of small buffers aligned and sized to > cache > > lines. > > The issue can't be fixed in the busdma code because partial, unaligned > transfers are fine, so long as the calling code avoids the entire > cache line during the transfer. Returning cache-line aligned buffers > from the allocator will do that, of course, but it is also valid for > the code to only use part of the buffer for the transfer. Right. My goal with the dma buffer pool changes isn't some sort of magical automatic fix in the busdma layer, it's just a whittling away of one small roadblock on the path to fixing this stuff. When I first started asking about how we should address these problems, the experts said to keep platform-specific alignment and padding information encapsulated within the busdma layer rather than inventing a new mechanism to export that info to drivers. That implies that drivers should be allocating DMA buffers from busdma instead of allocating big chunks of memory and sub-dividing them into smaller buffers. For that to work, the busdma implementation needs to be able to efficiently allocate buffers that are properly aligned and padded and especially that are guaranteed not to share a cache line with some other unrelated data. The busdma implementation can't get those guarantees from malloc(9), and the alternatives (contigmalloc(), and the kmem_alloc family) only work in page-sized chunks. We're asking drivers to allocate individual buffers of sometimes no more than a few bytes each. So that's all I'm addressing in the patchset I submitted: make sure that when we start fixing drivers to allocate 256 individual 16-byte IO descriptors that it shares with the hardware, that doesn't result in allocating 256 pages of memory. Also, if the request is for BUS_DMA_COHERENT memory, make sure that doesn't result in turning off caching in up to 256 pages that each contain a 16 byte IO buffer and 4080 bytes of unrelated data. -- Ian From owner-freebsd-arch@FreeBSD.ORG Wed Sep 5 15:28:14 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6535E106566C for ; Wed, 5 Sep 2012 15:28:14 +0000 (UTC) (envelope-from jonathan.robert.anderson@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 2007F8FC08 for ; Wed, 5 Sep 2012 15:28:13 +0000 (UTC) Received: by obbun3 with SMTP id un3so543749obb.13 for ; Wed, 05 Sep 2012 08:28:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=AOLr3z5JgipOIdrXA12w0NxQC1zSV+6k+XXrD61BFXs=; b=N8OQxYnHhB9ITH0Zv1JLCAzlOMFbN3qN0vE+e3PKnozFvOHUQmpX7d0v58m7fke3gt amaq/rexl8H2IhgZDbSAovaCu1sL4R0TinZxzi2LjViZ71qsYaEN/W1re8pHeO7m24QN 7yx01QPBUADmh9J4RjSUJToGqLN14qbUMdudcK9qUMKnR7xIsIJCGzj38uebDuh6T4em v5z6MBwbiG4aCPtudKOB7Ps6HHHtPeinlOJg3piecsj3QDoku3AMS+vg2hrVxc4UO+qZ CKBxkqLnI56Nvbmc/hykbF4GZLJ6kfOUzXr6PcHBuLtzvqo+Z1kwCN8AvEen0h7Vyoec na9Q== MIME-Version: 1.0 Received: by 10.60.0.161 with SMTP id 1mr17957162oef.83.1346858893504; Wed, 05 Sep 2012 08:28:13 -0700 (PDT) Sender: jonathan.robert.anderson@gmail.com Received: by 10.60.24.164 with HTTP; Wed, 5 Sep 2012 08:28:13 -0700 (PDT) Date: Wed, 5 Sep 2012 16:28:13 +0100 X-Google-Sender-Auth: lNhTu3zbd32ryu1QqaSzlrHJl-E Message-ID: From: Jonathan Anderson To: freebsd-arch@freebsd.org Content-Type: multipart/mixed; boundary=e89a8fb1ffc2e8f05c04c8f605cf Subject: Call graphs with bsd.obj.mk X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 15:28:14 -0000 --e89a8fb1ffc2e8f05c04c8f605cf Content-Type: text/plain; charset=UTF-8 Hi all, While doing some hacking recently, I really wanted a call graph. Clang and LLVM make this pretty easy, assuming you have the right CFLAGS, so a little build system integration seemed to be in order. I did this, it's pretty small and modular, and I find it useful, so I'm sharing this work with the list in the hope of putting it into -CURRENT. I've attached two BSD makefiles that let me produce call graphs from any directory that includes bsd.obj.mk (which notably includes binaries and libraries in src): - bsd.analysis.mk contains a 'callgraph' target that produces ${.OBJDIR}/_callgraph_.dot - bsd.llvm.mk contains a few LLVM helpers (e.g. to generate LLVM IR) If you have clang, llvm-link and opt installed, this allows you to generate a complete call graph for C code; assembler files are ignored because we don't currently have assembly -> IR transformations. The resulting graph can be pretty large and ugly (e.g. LLVM shows an "external node" pseudo-function that calls everything), but it can be filtered with scripts like those found at https://github.com/trombonehero/dot-tools to produce really useful graphs. llvm-link and opt aren't included in the bootstrap tools, so I'm currently doing this by installing the llvm-devel package. Perhaps we might copy llvm-link and opt over to /usr/obj/usr/src/tmp, possibly governed by a WITH_LLVM_TOOLS flag in src.conf or something? Jon -- Jonathan Anderson jonathan@FreeBSD.org http://freebsd.org/~jonathan/ --e89a8fb1ffc2e8f05c04c8f605cf Content-Type: application/octet-stream; name="bsd.analysis.mk" Content-Disposition: attachment; filename="bsd.analysis.mk" Content-Transfer-Encoding: base64 X-Attachment-Id: f_h6qkstw70 IyAkRnJlZUJTRCQKIwojIFRoZSBpbmNsdWRlIGZpbGUgPGJzZC5hbmFseXNpcy5taz4gcHJvdmlk ZXMgc291cmNlIGNvZGUgYW5hbHlzaXMgdG9vbHMgbGlrZQojIEMgY2FsbCBncmFwaHMuCiMKIyAr KysgdGFyZ2V0cyArKysKIwojCWNhbGxncmFwaDoKIwkJR2VuZXJhdGUgYSBHcmFwaFZpeiAuZG90 IGZpbGUgaW4gJHsuT0JKRElSfS9fY2FsbGdyYXBoXy5kb3QgY29udGFpbmluZwojCQl0aGUgY2Fs bCBncmFwaCBvZiBhbGwgQyAoYW5kIG1heWJlIEMrKykgZnVuY3Rpb25zLgojCiMJCVRoZSByZXN1 bHRpbmcgZ3JhcGggbWF5IGJlIHF1aXRlIGxhcmdlLCBzbyB5b3UgbWF5IGxpa2UgdG8gZmlsdGVy IGl0CiMJCXVzaW5nIHNjcmlwdHMgZnJvbSBodHRwczovL2dpdGh1Yi5jb20vdHJvbWJvbmVoZXJv L2RvdC10b29scy4KIwoKLmluY2x1ZGUgPGJzZC5pbml0Lm1rPgouaW5jbHVkZSA8YnNkLmxsdm0u bWs+CgouU1VGRklYRVM6IC5kb3QKCkRPVEZJTEVTPSAgJHtMTFZNX0JDOlI6Uy8kLy5kb3QvZ30K Q0xFQU5GSUxFUys9ICR7RE9URklMRVN9IF9jYWxsZ3JhcGhfLmRvdCBfY2FsbGdyYXBoXy5iYwoK Y2FsbGdyYXBoOiBfY2FsbGdyYXBoXy5kb3QKCl9jYWxsZ3JhcGhfLmJjOiAke0xMVk1fQkN9Cgls bHZtLWxpbmsgLW8gJEAgJD8KCiMgRXh0cmFjdCBjYWxsIGdyYXBoIGZyb20gTExWTSBJUiAoYmlu YXJ5IC5iYyBvciB0ZXh0dWFsIC5sbCkuCi5iYy5kb3Q6CglvcHQgLWFuYWx5emUgLWRvdC1jYWxs Z3JhcGggJDwKCW12IGNhbGxncmFwaC5kb3QgJEAgICAgICMgTExWTSBjdXJyZW50bHkgaGFyZGNv ZGVzIHRoZSAuZG90IG91dHB1dCBmaWxlCgoubGwuZG90OgoJb3B0IC1hbmFseXplIC1kb3QtY2Fs bGdyYXBoICQ8CgltdiBjYWxsZ3JhcGguZG90ICRAICAgICAjIExMVk0gY3VycmVudGx5IGhhcmRj b2RlcyB0aGUgLmRvdCBvdXRwdXQgZmlsZQoK --e89a8fb1ffc2e8f05c04c8f605cf Content-Type: application/octet-stream; name="bsd.llvm.mk" Content-Disposition: attachment; filename="bsd.llvm.mk" Content-Transfer-Encoding: base64 X-Attachment-Id: f_h6qkstw91 IyAkRnJlZUJTRCQKIwojIFRoZSBpbmNsdWRlIGZpbGUgPGJzZC5sbHZtLm1rPiBoYW5kbGVzIExM Vk0tc3BlY2lmaWMgdGFza3MgbGlrZSBjb21waWxpbmcKIyBDIHRvIExMVk0gSVIuCiMKIyArKysg dGFyZ2V0cyArKysKIwojCWJjOgojCQljb21waWxlIGFsbCBDIHNvdXJjZXMgaW4gJHtTUkNTfSB0 byBiaW5hcnkgTExWTSBJUiBmb3JtYXQKIwojCWlyOgojCQljb21waWxlIGFsbCBDIHNvdXJjZXMg aW4gJHtTUkNTfSB0byB0ZXh0dWFsIExMVk0gSVIgZm9ybWF0CiMKCi5pbmNsdWRlIDxic2QuaW5p dC5taz4KCi5TVUZGSVhFUzogLmMgLmNjIC5jcHAgLmN4eCAuQyAuYmMgLmxsCgojIExMVk0gYnl0 ZWNvZGUgaXMgYSBiaW5hcnkgZm9ybWF0LgpMTFZNX0JDPSAgICAgJHtTUkNTOk4qLmg6TiouUzpS OlMvJC8uYmMvZ30KCiMgTExWTSBJUiBjb250YWlucyB0aGUgc2FtZSBpbmZvcm1hdGlvbiwgYnV0 IGluIGFuIGFzc2VtYmx5LWxpa2UgZm9ybWF0LgpMTFZNX0lSPSAgICAgJHtTUkNTOk4qLmg6Tiou UzpSOlMvJC8ubGwvZ30KCkNMRUFORklMRVMrPSAke0xMVk1fQkN9ICR7TExWTV9JUn0KCiMgQnVp bGQgYWxsIExMVk0gSVIgKGJpbmFyeSAuYmMgb3IgdGV4dHVhbCAubGwpLgpiYzogJHtMTFZNX0JD fQppcjogJHtMTFZNX0lSfQoKIyBDb21waWxlIEMgY29kZSB0byBiaW5hcnkgTExWTSBJUiBieXRl Y29kZS4KLmMuYmM6IHVzaW5nY2xhbmcKLmlmIGRlZmluZWQoUFJPR19DWFgpCgljbGFuZysrIC1j IC1lbWl0LWxsdm0gJHtDRkxBR1N9IC1vICRAICQ8Ci5lbHNlCgljbGFuZyAtYyAtZW1pdC1sbHZt ICR7Q0ZMQUdTfSAtbyAkQCAkPAouZW5kaWYKCiMgQ29tcGlsZSBDIGNvZGUgdG8gdGV4dHVhbCBM TFZNIElSIGZvcm1hdC4KLmMubGw6IHVzaW5nY2xhbmcKLmlmIGRlZmluZWQoUFJPR19DWFgpCglj bGFuZysrIC1TIC1lbWl0LWxsdm0gJHtDRkxBR1N9IC1vICRAICQ8Ci5lbHNlCgljbGFuZyAtUyAt ZW1pdC1sbHZtICR7Q0ZMQUdTfSAtbyAkQCAkPAouZW5kaWYKCiMgRW5zdXJlIHRoYXQgd2UgYXJl IGNvbXBpbGluZyB3aXRoIGNsYW5nLgp1c2luZ2NsYW5nOgouaWYgJHtNS19DTEFOR30gIT0gInll cyIKCUBlY2hvICI9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PSIKCUBlY2hvICJDIC0+IExMVk0gSVIgY29tcGlsYXRpb24g cmVxdWlyZXMgY2xhbmcuIgoJQGVjaG8gIllvdSBtYXkgd2lzaCB0byBlbmFibGUgV0lUSF9DTEFO RyBpbiAvZXRjL21ha2UuY29uZi4iCglAZWNobyAiPT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0iCglAZXhpdCAxCi5lbmRp ZgoK --e89a8fb1ffc2e8f05c04c8f605cf-- From owner-freebsd-arch@FreeBSD.ORG Wed Sep 5 20:54:10 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5BA7E106564A; Wed, 5 Sep 2012 20:54:10 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) by mx1.freebsd.org (Postfix) with ESMTP id 2AF558FC0C; Wed, 5 Sep 2012 20:54:09 +0000 (UTC) Received: from [209.249.190.124] (port=63358 helo=gnnmac.hudson-trading.com) by vps.hungerhost.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1T9Mbx-0001L9-DY; Wed, 05 Sep 2012 16:54:09 -0400 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) From: George Neville-Neil In-Reply-To: Date: Wed, 5 Sep 2012 16:54:09 -0400 Content-Transfer-Encoding: 7bit Message-Id: <1135E0EA-319C-4AB9-B282-59CDEA9020C8@neville-neil.com> References: To: Jonathan Anderson X-Mailer: Apple Mail (2.1486) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com Cc: freebsd-arch@freebsd.org Subject: Re: Call graphs with bsd.obj.mk X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 20:54:10 -0000 On Sep 5, 2012, at 11:28 , Jonathan Anderson wrote: > Hi all, > > While doing some hacking recently, I really wanted a call graph. Clang > and LLVM make this pretty easy, assuming you have the right CFLAGS, so > a little build system integration seemed to be in order. I did this, > it's pretty small and modular, and I find it useful, so I'm sharing > this work with the list in the hope of putting it into -CURRENT. > > I've attached two BSD makefiles that let me produce call graphs from > any directory that includes bsd.obj.mk (which notably includes > binaries and libraries in src): > > - bsd.analysis.mk contains a 'callgraph' target that produces > ${.OBJDIR}/_callgraph_.dot > - bsd.llvm.mk contains a few LLVM helpers (e.g. to generate LLVM IR) > > If you have clang, llvm-link and opt installed, this allows you to > generate a complete call graph for C code; assembler files are ignored > because we don't currently have assembly -> IR transformations. The > resulting graph can be pretty large and ugly (e.g. LLVM shows an > "external node" pseudo-function that calls everything), but it can be > filtered with scripts like those found at > https://github.com/trombonehero/dot-tools to produce really useful > graphs. > > llvm-link and opt aren't included in the bootstrap tools, so I'm > currently doing this by installing the llvm-devel package. Perhaps we > might copy llvm-link and opt over to /usr/obj/usr/src/tmp, possibly > governed by a WITH_LLVM_TOOLS flag in src.conf or something? > This would be great to have in the tree as a real build target. Best, George From owner-freebsd-arch@FreeBSD.ORG Thu Sep 6 04:00:36 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E4213106566B; Thu, 6 Sep 2012 04:00:36 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id 9F5368FC12; Thu, 6 Sep 2012 04:00:36 +0000 (UTC) Received: from JRE-MBP-2.local (ppp121-45-230-94.lns20.per1.internode.on.net [121.45.230.94]) (authenticated bits=0) by vps1.elischer.org (8.14.5/8.14.5) with ESMTP id q8640Rwx076704 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Wed, 5 Sep 2012 21:00:29 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <50481FD5.2090802@freebsd.org> Date: Thu, 06 Sep 2012 12:00:21 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:15.0) Gecko/20120824 Thunderbird/15.0 MIME-Version: 1.0 To: Jonathan Anderson References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-arch@freebsd.org Subject: Re: Call graphs with bsd.obj.mk X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 04:00:37 -0000 On 9/5/12 11:28 PM, Jonathan Anderson wrote: > Hi all, > > While doing some hacking recently, I really wanted a call graph. Clang > and LLVM make this pretty easy, assuming you have the right CFLAGS, so > a little build system integration seemed to be in order. I did this, > it's pretty small and modular, and I find it useful, so I'm sharing > this work with the list in the hope of putting it into -CURRENT. > > I've attached two BSD makefiles that let me produce call graphs from > any directory that includes bsd.obj.mk (which notably includes > binaries and libraries in src): > > - bsd.analysis.mk contains a 'callgraph' target that produces > ${.OBJDIR}/_callgraph_.dot > - bsd.llvm.mk contains a few LLVM helpers (e.g. to generate LLVM IR) > > If you have clang, llvm-link and opt installed, this allows you to > generate a complete call graph for C code; assembler files are ignored > because we don't currently have assembly -> IR transformations. The > resulting graph can be pretty large and ugly (e.g. LLVM shows an > "external node" pseudo-function that calls everything), but it can be > filtered with scripts like those found at > https://github.com/trombonehero/dot-tools to produce really useful > graphs. > > llvm-link and opt aren't included in the bootstrap tools, so I'm > currently doing this by installing the llvm-devel package. Perhaps we > might copy llvm-link and opt over to /usr/obj/usr/src/tmp, possibly > governed by a WITH_LLVM_TOOLS flag in src.conf or something? can you put a sample output file up somewhere? (.png or something?) > > Jon > > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Thu Sep 6 10:24:00 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 334271065675; Thu, 6 Sep 2012 10:24:00 +0000 (UTC) (envelope-from jonathan.robert.anderson@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id DF8878FC08; Thu, 6 Sep 2012 10:23:59 +0000 (UTC) Received: by iayy25 with SMTP id y25so2360921iay.13 for ; Thu, 06 Sep 2012 03:23:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=fgczAWYxl/BLrkwX03M+pcDqHa4VqFhuwFhq2O9gj6I=; b=RQZUCh9x9HaITaBPinWorLnI3vIO+H9smgYSgRLRjMjWwswCmBe+Jy6PacNk0EGfDJ cOpPl3n1Bnj1jD+UqYluvcTFdDkmgRWow7uZh1pRBTBDckCTQ3J8KC1f7OcqwsVra9Yc Wh2TaVmLfK0sBcR38ELi7T8bhmwRHAsTOKdr4KQ6znIJmXmv3cMPYsA0J9cpl9C4h2wX ZDUvoUuCOZ0+nUvi7ESho3ynY1U8OWRabZNeGTUW1IW+JohdH++JDWRKVotttqAQZp9+ CLrWUVUXkvgqCL4Pw/c10ohhRL3FkLtu7QkIqAu7Pgj6XcetjklHbRJB99yMpoEnS2vy c8IQ== MIME-Version: 1.0 Received: by 10.50.195.194 with SMTP id ig2mr21837912igc.24.1346927033746; Thu, 06 Sep 2012 03:23:53 -0700 (PDT) Sender: jonathan.robert.anderson@gmail.com Received: by 10.64.147.2 with HTTP; Thu, 6 Sep 2012 03:23:53 -0700 (PDT) In-Reply-To: <50481FD5.2090802@freebsd.org> References: <50481FD5.2090802@freebsd.org> Date: Thu, 6 Sep 2012 11:23:53 +0100 X-Google-Sender-Auth: 26hs3FQVzEf3QQ9Gj5mkP3ahfSE Message-ID: From: Jonathan Anderson To: Julian Elischer Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: Call graphs with bsd.obj.mk X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 10:24:00 -0000 On 6 September 2012 05:00, Julian Elischer wrote: > can you put a sample output file up somewhere? (.png or something?) I've placed some PDFs at: http://www.cl.cam.ac.uk/~jra40/dot/ As you can see, the unfiltered output is pretty messy, but filtering and colouring improves the readability substantially. The graphs shown (gzip, libgeom and rtld-elf) are three out of the first four things that I thought to try my makefiles on. Clang died when compiling the fourth =E2=80=94 libc =E2=80=94 to LLVM bytecode; I've f= orwarded this to theraven@ so we can find out what's going on there. Jon --=20 Jonathan Anderson jonathan@FreeBSD.org http://freebsd.org/~jonathan/ From owner-freebsd-arch@FreeBSD.ORG Thu Sep 6 10:41:41 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EB09B1065670 for ; Thu, 6 Sep 2012 10:41:41 +0000 (UTC) (envelope-from jonathan.robert.anderson@gmail.com) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id B10238FC16 for ; Thu, 6 Sep 2012 10:41:41 +0000 (UTC) Received: by iebc12 with SMTP id c12so3511237ieb.13 for ; Thu, 06 Sep 2012 03:41:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=7AUkjV69lUTG91hAS1J/WFHTC0MDiVba9VXF4TtTdAk=; b=rcauLI9atUSp0117ogp+KE3LirPmh4zeXCTPMRHkAeFWxMeD/fig7b0aeph5+g+M8y FMVk1i5g7DuX9Y3i+4/hC3empGe91dq5QAFSWUY20H3p0JnW1C5D3LPkiuxmPp73O/hu NsocP2PvZG65uxMk5CXByHNO3pmfwU+sHPfl4VLWVTjF5co+g/Jotqd+7fH4/xBlVsqT v2F0bJlPyzVPrnLMefrucD8VmZmQW8oLEf+kgOeBYHFjjyNaPjJiMcdeaAtFdGBs8NLz qsMu1hwywl+LIbH6xDIONtxYf5XxJdIuQLFpx6iDr8ELSbDtLVMEEQOAb2VUpnY4ZTw3 Gs+A== MIME-Version: 1.0 Received: by 10.50.242.3 with SMTP id wm3mr9985913igc.0.1346928100976; Thu, 06 Sep 2012 03:41:40 -0700 (PDT) Sender: jonathan.robert.anderson@gmail.com Received: by 10.64.147.2 with HTTP; Thu, 6 Sep 2012 03:41:40 -0700 (PDT) In-Reply-To: <1135E0EA-319C-4AB9-B282-59CDEA9020C8@neville-neil.com> References: <1135E0EA-319C-4AB9-B282-59CDEA9020C8@neville-neil.com> Date: Thu, 6 Sep 2012 11:41:40 +0100 X-Google-Sender-Auth: FkIJxshraxdfQlX6twQUG0EWPes Message-ID: From: Jonathan Anderson To: George Neville-Neil Content-Type: text/plain; charset=UTF-8 Cc: freebsd-arch@freebsd.org Subject: Re: Call graphs with bsd.obj.mk X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 10:41:42 -0000 On 5 September 2012 21:54, George Neville-Neil wrote: > This would be great to have in the tree as a real build target. Do you mean: a) a manual build target for binaries and libraries (as-is), b) an automatically-built target (for systems with llvm), c) a top-level build target (e.g. "make callgraphs") or d) something else entirely? I'd be very happy to commit (a) almost immediately if there is a consensus to do so; something like (b) would require a more careful approach so as to not break builds on boxes/architectures that don't have llvm-link or opt available. Also, while (a) could be committed almost immediately, using it requires either: - PREFIX_TOOLS not suppressing llvm-link and opt in the bootstrap tools dir (chasing this angle with brooks@) or - installing the llvm-devel port Jon -- Jonathan Anderson jonathan@FreeBSD.org http://freebsd.org/~jonathan/ From owner-freebsd-arch@FreeBSD.ORG Fri Sep 7 16:24:32 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D6CEE106566B for ; Fri, 7 Sep 2012 16:24:32 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 9830B8FC12 for ; Fri, 7 Sep 2012 16:24:32 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 0D03FB93B for ; Fri, 7 Sep 2012 12:24:32 -0400 (EDT) From: John Baldwin To: arch@freebsd.org Date: Fri, 7 Sep 2012 12:17:47 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p17; KDE/4.5.5; amd64; ; ) MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201209071217.47439.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 07 Sep 2012 12:24:32 -0400 (EDT) Cc: Subject: [PATCH] Close a race between exit1() and SIGSTOP X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 16:24:33 -0000 We ran into a bug at work recently where processes were reporting an exit status from wait(2) of WIFSIGNALED() with WTERMSIG() of SIGSTOP. I narrowed this down to a race in exit1(). Specifically, exit1() sets p->p_xstat to the passed in exit status and sets P_WEXIT. It then drops the PROC_LOCK() to do other work such as freeing file descriptors, etc. Later it reacquires the PROC_LOCK(), marks the process as a zombie, and terminates. During the window where the PROC_LOCK() is not held, if a stop signal arrives (SIGSTOP, SIGTSTP, etc.), then the signal code will overwrite p->p_xstat with the signal that initiated the stop. The end result is that setting from SIGSTOP stays in p->p_xstat and is reported via wait(2). My first attempt at a fix was to simply ignore all signals once P_WEXIT is set by adding a check for P_WEXIT to the existing check for PRS_ZOMBIE. However, I think this is not quite right. For example, if an exiting process gets hung on an interruptible NFS mount while doing fdfree() I think we want a user to be able to kill the process to let it break out of NFS and finish exiting. The second fix I have is to explicitly clear P_STOPPED_SIGNAL to discard any pending SIGSTOP when setting P_WEXIT and to explicitly disallow any stop signals for a process that is currently exiting (that is, P_WEXIT is set). This fixes the race I ran into while still allowing other signals during process exit. The patch to do that is below. Below that is a test program that reproduces the issue. I would really like to get some feedback on which approach is best and if my concerns about allowing other signals during exit1() is a legitimate concern. (For some reason I feel like I've seen an argument for allowing that before.) Index: kern/kern_sig.c =================================================================== --- kern/kern_sig.c (revision 240144) +++ kern/kern_sig.c (working copy) @@ -2134,6 +2134,8 @@ * We try do the per-process part here. */ if (P_SHOULDSTOP(p)) { + KASSERT(!(p->p_flag & P_WEXIT), + ("signal to stopped but exiting process")); if (sig == SIGKILL) { /* * If traced process is already stopped, @@ -2248,7 +2250,7 @@ MPASS(action == SIG_DFL); if (prop & SA_STOP) { - if (p->p_flag & P_PPWAIT) + if (p->p_flag & (P_PPWAIT|P_WEXIT)) goto out; p->p_flag |= P_STOPPED_SIG; p->p_xstat = sig; @@ -2410,6 +2412,7 @@ struct proc *p = td->td_proc; PROC_LOCK_ASSERT(p, MA_OWNED); + KASSERT(!(p->p_flag & P_WEXIT), ("Stopping exiting process")); WITNESS_WARN(WARN_GIANTOK | WARN_SLEEPOK, &p->p_mtx.lock_object, "Stopping for traced signal"); @@ -2648,7 +2651,7 @@ * process group, ignore tty stop signals. */ if (prop & SA_STOP) { - if (p->p_flag & P_TRACED || + if (p->p_flag & (P_TRACED|P_WEXIT) || (p->p_pgrp->pg_jobc == 0 && prop & SA_TTYSTOP)) break; /* == ignore */ Index: kern/kern_exit.c =================================================================== --- kern/kern_exit.c (revision 240204) +++ kern/kern_exit.c (working copy) @@ -200,6 +200,16 @@ _STOPEVENT(p, S_EXIT, rv); /* + * Ignore any pending request to stop due to a stop signal. + * Once P_WEXIT is set, future requests will be ignored as + * well. + * + * XXX: Is this correct? + */ + p->p_flag &= ~P_STOPPED_SIG; + KASSERT(!P_SHOULDSTOP(p), ("exiting process is stopped")); + + /* * Note that we are exiting and do another wakeup of anyone in * PIOCWAIT in case they aren't listening for S_EXIT stops or * decided to wait again after we told them we are exiting. /*- * Test program to expose a race where SIGSTOP can be delivered to an * exiting process after it has set p_xstat to the exit status in * exit1() resulting in the p_xstat being overwritten. */ #include #include #include #include #include #include #include #include #include #include static volatile sig_atomic_t info; static void handler(int sig) { info = 1; } static void print_status(int status) { if (WIFCONTINUED(status)) printf("CONTINUED"); else if (WIFEXITED(status)) printf("EXITED: %d", WEXITSTATUS(status)); else if (WIFSIGNALED(status)) printf("SIGNALED: %d%s", WTERMSIG(status), WCOREDUMP(status) ? " (core dumped" : ""); else if (WIFSTOPPED(status)) printf("STOPPED: %d", WSTOPSIG(status)); else printf("UNKNOWN: %#x\n", status); } static void * kill_thread(void *arg) { pid_t pid; pid = (intptr_t)arg; for (;;) { if (kill(pid, SIGSTOP) < 0) { if (errno == ESRCH) break; err(1, "kill(SIGSTOP)"); } if (kill(pid, SIGCONT) < 0) { if (errno == ESRCH) break; err(1, "kill(SIGCONT)"); } } return (NULL); } int main(int ac, char **av) { pthread_t thread; pid_t pid, wpid; int count, error, status; if (signal(SIGINFO, handler) == SIG_ERR) errx(1, "signal(SIGINFO)"); for (count = 0;; count++) { if (info) { printf("count %d\n", count); info = 0; } pid = fork(); switch (pid) { case -1: err(1, "fork"); case 0: usleep(1000); exit(0); } error = pthread_create(&thread, NULL, kill_thread, (void *)(intptr_t)pid); if (error) errc(1, error, "pthread_create"); wpid = waitpid(pid, &status, 0); if (wpid == -1) err(1, "waitpid"); if (wpid != pid) errx(1, "waitpid mismatch"); if (!WIFEXITED(status) || WEXITSTATUS(status) != 0) { printf("abnormal status: "); print_status(status); printf(" after %d loops\n", count); } error = pthread_join(thread, NULL); if (error) errc(1, error, "pthread_join"); } return (0); } -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Sep 7 16:35:54 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5DDA0106564A; Fri, 7 Sep 2012 16:35:54 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0DA808FC0A; Fri, 7 Sep 2012 16:35:53 +0000 (UTC) Received: by obbun3 with SMTP id un3so6245914obb.13 for ; Fri, 07 Sep 2012 09:35:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=vG18D5HaLZacX9N297J/xlM4lTYwj78HmhbIIjCNiXQ=; b=tVmR1/DNdNr1Dsi/VN/krJdmUFFwOns6guLn0olheZ/x/fLq5jpz6VZSEMxlp3dDsW e7eeMIOCMfox/EMqlc20gVRYDVFOpY5YJchEg3VPTcfVWHQiTB47Wp8zwm/x/auKjvt2 aHcwdny4wyKvo7I2LI3Bc+OZ7JE6uygghWBWp0PTjQ1jHuPd31zY2pNUpDSQugWpF2+a QFDQ/1KReOLrdoTXUrirGhmwE8Q30cawX3th/jNeDJLN5jagNGoj3HDgBIgpRfBKvoRC viShk5qAgzWXAYSp2Il9FSTlyF1UG9kXw4p9MHMjAamJhB4r0cehhItbeOXSXxUZW9ul RKng== MIME-Version: 1.0 Received: by 10.60.20.69 with SMTP id l5mr6642057oee.114.1347035752616; Fri, 07 Sep 2012 09:35:52 -0700 (PDT) Received: by 10.76.142.201 with HTTP; Fri, 7 Sep 2012 09:35:52 -0700 (PDT) In-Reply-To: <201209071217.47439.jhb@freebsd.org> References: <201209071217.47439.jhb@freebsd.org> Date: Fri, 7 Sep 2012 09:35:52 -0700 Message-ID: From: Garrett Cooper To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: arch@freebsd.org Subject: Re: [PATCH] Close a race between exit1() and SIGSTOP X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 16:35:54 -0000 On Fri, Sep 7, 2012 at 9:17 AM, John Baldwin wrote: > We ran into a bug at work recently where processes were reporting an exit > status from wait(2) of WIFSIGNALED() with WTERMSIG() of SIGSTOP. I narrowed > this down to a race in exit1(). Specifically, exit1() sets p->p_xstat to the > passed in exit status and sets P_WEXIT. It then drops the PROC_LOCK() to do > other work such as freeing file descriptors, etc. Later it reacquires the > PROC_LOCK(), marks the process as a zombie, and terminates. During the window > where the PROC_LOCK() is not held, if a stop signal arrives (SIGSTOP, SIGTSTP, > etc.), then the signal code will overwrite p->p_xstat with the signal that > initiated the stop. The end result is that setting from SIGSTOP stays in > p->p_xstat and is reported via wait(2). > > My first attempt at a fix was to simply ignore all signals once P_WEXIT is > set by adding a check for P_WEXIT to the existing check for PRS_ZOMBIE. > However, I think this is not quite right. For example, if an exiting process > gets hung on an interruptible NFS mount while doing fdfree() I think we want a > user to be able to kill the process to let it break out of NFS and finish > exiting. > > The second fix I have is to explicitly clear P_STOPPED_SIGNAL to discard any > pending SIGSTOP when setting P_WEXIT and to explicitly disallow any stop > signals for a process that is currently exiting (that is, P_WEXIT is set). > This fixes the race I ran into while still allowing other signals during > process exit. The patch to do that is below. Below that is a test program > that reproduces the issue. > > I would really like to get some feedback on which approach is best and if my > concerns about allowing other signals during exit1() is a legitimate concern. > (For some reason I feel like I've seen an argument for allowing that before.) Does this case continue to function normally? $ sh -c 'while :; do sleep 1; done' ^Z [1]+ Stopped sh -c 'while :; do sleep 1; done' $ kill %1 $ fg bash: fg: job has terminated [1]+ Terminated: 15 sh -c 'while :; do sleep 1; done' $ sh -c 'while :; do sleep 1; done' ^Z [1]+ Stopped sh -c 'while :; do sleep 1; done' $ kill %1 $ kill %1 bash: kill: (3522) - No such process [1]+ Terminated: 15 sh -c 'while :; do sleep 1; done' In particular, a single kill results in the signal being sent after the process wakes up, and a double kill results in the immediate death of the process (in this case a shell job) (in part because of the signals /bin/sh masks). Thanks! -Garrett From owner-freebsd-arch@FreeBSD.ORG Fri Sep 7 16:52:16 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 282001065675; Fri, 7 Sep 2012 16:52:16 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 87A348FC18; Fri, 7 Sep 2012 16:52:15 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q87GqNDe021324; Fri, 7 Sep 2012 19:52:23 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q87GqAA7071689; Fri, 7 Sep 2012 19:52:10 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q87GqAne071688; Fri, 7 Sep 2012 19:52:10 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 7 Sep 2012 19:52:10 +0300 From: Konstantin Belousov To: John Baldwin Message-ID: <20120907165210.GC33100@deviant.kiev.zoral.com.ua> References: <201209071217.47439.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="mcEcqdJNNSWmslGf" Content-Disposition: inline In-Reply-To: <201209071217.47439.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: arch@freebsd.org Subject: Re: [PATCH] Close a race between exit1() and SIGSTOP X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 16:52:16 -0000 --mcEcqdJNNSWmslGf Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Sep 07, 2012 at 12:17:47PM -0400, John Baldwin wrote: > We ran into a bug at work recently where processes were reporting an > exit status from wait(2) of WIFSIGNALED() with WTERMSIG() of SIGSTOP. > I narrowed this down to a race in exit1(). Specifically, exit1() > sets p->p_xstat to the passed in exit status and sets P_WEXIT. It > then drops the PROC_LOCK() to do other work such as freeing file > descriptors, etc. Later it reacquires the PROC_LOCK(), marks the > process as a zombie, and terminates. During the window where the > PROC_LOCK() is not held, if a stop signal arrives (SIGSTOP, SIGTSTP, > etc.), then the signal code will overwrite p->p_xstat with the signal > that initiated the stop. The end result is that setting from SIGSTOP > stays in p->p_xstat and is reported via wait(2). > > My first attempt at a fix was to simply ignore all signals once > P_WEXIT is set by adding a check for P_WEXIT to the existing check for > PRS_ZOMBIE. However, I think this is not quite right. For example, if > an exiting process gets hung on an interruptible NFS mount while doing > fdfree() I think we want a user to be able to kill the process to let > it break out of NFS and finish exiting. > > The second fix I have is to explicitly clear P_STOPPED_SIGNAL to > discard any pending SIGSTOP when setting P_WEXIT and to explicitly > disallow any stop signals for a process that is currently exiting > (that is, P_WEXIT is set). This fixes the race I ran into while still > allowing other signals during process exit. The patch to do that is > below. Below that is a test program that reproduces the issue. > > I would really like to get some feedback on which approach is best > and if my concerns about allowing other signals during exit1() is a > legitimate concern. (For some reason I feel like I've seen an argument > for allowing that before.) > Argument ? Please see r163541. I hope you agree with the author, who basically provided the same reasoning about interruptible NFS mounts. So I agree with r163541, which log message says that I requested it. And I do not see any other possible solution except case 2. Shouldn't the fix actually prevent any overwrite of p_xstat for P_WEXIT process from the signal delivery code ? > Index: kern/kern_sig.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- kern/kern_sig.c (revision 240144) > +++ kern/kern_sig.c (working copy) > @@ -2134,6 +2134,8 @@ > * We try do the per-process part here. > */ > if (P_SHOULDSTOP(p)) { > + KASSERT(!(p->p_flag & P_WEXIT), > + ("signal to stopped but exiting process")); > if (sig =3D=3D SIGKILL) { > /* > * If traced process is already stopped, > @@ -2248,7 +2250,7 @@ > MPASS(action =3D=3D SIG_DFL); > =20 > if (prop & SA_STOP) { > - if (p->p_flag & P_PPWAIT) > + if (p->p_flag & (P_PPWAIT|P_WEXIT)) > goto out; > p->p_flag |=3D P_STOPPED_SIG; > p->p_xstat =3D sig; > @@ -2410,6 +2412,7 @@ > struct proc *p =3D td->td_proc; > =20 > PROC_LOCK_ASSERT(p, MA_OWNED); > + KASSERT(!(p->p_flag & P_WEXIT), ("Stopping exiting process")); > WITNESS_WARN(WARN_GIANTOK | WARN_SLEEPOK, > &p->p_mtx.lock_object, "Stopping for traced signal"); > =20 > @@ -2648,7 +2651,7 @@ > * process group, ignore tty stop signals. > */ > if (prop & SA_STOP) { > - if (p->p_flag & P_TRACED || > + if (p->p_flag & (P_TRACED|P_WEXIT) || > (p->p_pgrp->pg_jobc =3D=3D 0 && > prop & SA_TTYSTOP)) > break; /* =3D=3D ignore */ > Index: kern/kern_exit.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- kern/kern_exit.c (revision 240204) > +++ kern/kern_exit.c (working copy) > @@ -200,6 +200,16 @@ > _STOPEVENT(p, S_EXIT, rv); > =20 > /* > + * Ignore any pending request to stop due to a stop signal. > + * Once P_WEXIT is set, future requests will be ignored as > + * well. > + * > + * XXX: Is this correct? > + */ > + p->p_flag &=3D ~P_STOPPED_SIG; > + KASSERT(!P_SHOULDSTOP(p), ("exiting process is stopped")); > + > + /* > * Note that we are exiting and do another wakeup of anyone in > * PIOCWAIT in case they aren't listening for S_EXIT stops or > * decided to wait again after we told them we are exiting. >=20 >=20 > /*- > * Test program to expose a race where SIGSTOP can be delivered to an > * exiting process after it has set p_xstat to the exit status in > * exit1() resulting in the p_xstat being overwritten. > */ >=20 > #include > #include > #include > #include > #include > #include > #include > #include > #include > #include >=20 > static volatile sig_atomic_t info; >=20 > static void > handler(int sig) > { >=20 > info =3D 1; > } >=20 > static void > print_status(int status) > { >=20 > if (WIFCONTINUED(status)) > printf("CONTINUED"); > else if (WIFEXITED(status)) > printf("EXITED: %d", WEXITSTATUS(status)); > else if (WIFSIGNALED(status)) > printf("SIGNALED: %d%s", WTERMSIG(status), WCOREDUMP(status) ? > " (core dumped" : ""); > else if (WIFSTOPPED(status)) > printf("STOPPED: %d", WSTOPSIG(status)); > else > printf("UNKNOWN: %#x\n", status); > } >=20 > static void * > kill_thread(void *arg) > { > pid_t pid; >=20 > pid =3D (intptr_t)arg; > for (;;) { > if (kill(pid, SIGSTOP) < 0) { > if (errno =3D=3D ESRCH) > break; > err(1, "kill(SIGSTOP)"); > } > if (kill(pid, SIGCONT) < 0) { > if (errno =3D=3D ESRCH) > break; > err(1, "kill(SIGCONT)"); > } > } > return (NULL); > } >=20 > int > main(int ac, char **av) > { > pthread_t thread; > pid_t pid, wpid; > int count, error, status; >=20 > if (signal(SIGINFO, handler) =3D=3D SIG_ERR) > errx(1, "signal(SIGINFO)"); > for (count =3D 0;; count++) { > if (info) { > printf("count %d\n", count); > info =3D 0; > } > pid =3D fork(); > switch (pid) { > case -1: > err(1, "fork"); > case 0: > usleep(1000); > exit(0); > } >=20 > error =3D pthread_create(&thread, NULL, kill_thread, > (void *)(intptr_t)pid); > if (error) > errc(1, error, "pthread_create"); >=20 > wpid =3D waitpid(pid, &status, 0); > if (wpid =3D=3D -1) > err(1, "waitpid"); > if (wpid !=3D pid) > errx(1, "waitpid mismatch"); > if (!WIFEXITED(status) || WEXITSTATUS(status) !=3D 0) { > printf("abnormal status: "); > print_status(status); > printf(" after %d loops\n", count); > } >=20 > error =3D pthread_join(thread, NULL); > if (error) > errc(1, error, "pthread_join"); > } >=20 > return (0); > } >=20 > --=20 > John Baldwin > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" --mcEcqdJNNSWmslGf Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlBKJjoACgkQC3+MBN1Mb4idMQCgmsHdbxM3kPjDs3W9k8WZmuYY DlUAoOz4Uz5cY6SR7rZsJsw6wXwFhAho =NI4s -----END PGP SIGNATURE----- --mcEcqdJNNSWmslGf-- From owner-freebsd-arch@FreeBSD.ORG Fri Sep 7 18:19:23 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 71A991065673 for ; Fri, 7 Sep 2012 18:19:23 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 448738FC16 for ; Fri, 7 Sep 2012 18:19:23 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 7455CB97A; Fri, 7 Sep 2012 14:19:22 -0400 (EDT) From: John Baldwin To: Garrett Cooper Date: Fri, 7 Sep 2012 14:16:24 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p17; KDE/4.5.5; amd64; ; ) References: <201209071217.47439.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201209071416.24970.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 07 Sep 2012 14:19:22 -0400 (EDT) Cc: arch@freebsd.org Subject: Re: [PATCH] Close a race between exit1() and SIGSTOP X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 18:19:23 -0000 On Friday, September 07, 2012 12:35:52 pm Garrett Cooper wrote: > Does this case continue to function normally? > > $ sh -c 'while :; do sleep 1; done' > ^Z > [1]+ Stopped sh -c 'while :; do sleep 1; done' > $ kill %1 > $ fg > bash: fg: job has terminated > [1]+ Terminated: 15 sh -c 'while :; do sleep 1; done' > > $ sh -c 'while :; do sleep 1; done' > ^Z > [1]+ Stopped sh -c 'while :; do sleep 1; done' > $ kill %1 > $ kill %1 > bash: kill: (3522) - No such process > [1]+ Terminated: 15 sh -c 'while :; do sleep 1; done' > > In particular, a single kill results in the signal being sent after > the process wakes up, and a double kill results in the immediate death > of the process (in this case a shell job) (in part because of the > signals /bin/sh masks). This still works in bash: > bash $ sh -c 'while :; do sleep 1; done' ^Z [1]+ Stopped sh -c 'while :; do sleep 1; done' $ kill %1 $ fg bash: fg: job has terminated [1]+ Terminated: 15 sh -c 'while :; do sleep 1; done' $ sh -c 'while :; do sleep 1; done' ^Z [1]+ Stopped sh -c 'while :; do sleep 1; done' $ kill %1 $ kill %1 bash: kill: (61009) - No such process [1]+ Terminated: 15 sh -c 'while :; do sleep 1; done' I get different behavior in tcsh: > sh -c 'while :; do sleep 1; done' ^Z Suspended > kill %1 > [1] Terminated sh -c while :; do sleep 1; done Note that this patch merey inhibits SIGSTOP from having an effect while a process is exiting, it does not affect SIGTERM at all, so I don't think it could have any effect on this case anyway. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Sep 7 18:19:23 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E9382106564A for ; Fri, 7 Sep 2012 18:19:23 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id BE8D28FC18 for ; Fri, 7 Sep 2012 18:19:23 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 19477B97C; Fri, 7 Sep 2012 14:19:23 -0400 (EDT) From: John Baldwin To: Konstantin Belousov Date: Fri, 7 Sep 2012 14:18:58 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p17; KDE/4.5.5; amd64; ; ) References: <201209071217.47439.jhb@freebsd.org> <20120907165210.GC33100@deviant.kiev.zoral.com.ua> In-Reply-To: <20120907165210.GC33100@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201209071418.58576.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 07 Sep 2012 14:19:23 -0400 (EDT) Cc: arch@freebsd.org Subject: Re: [PATCH] Close a race between exit1() and SIGSTOP X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 18:19:24 -0000 On Friday, September 07, 2012 12:52:10 pm Konstantin Belousov wrote: > On Fri, Sep 07, 2012 at 12:17:47PM -0400, John Baldwin wrote: > > We ran into a bug at work recently where processes were reporting an > > exit status from wait(2) of WIFSIGNALED() with WTERMSIG() of SIGSTOP. > > I narrowed this down to a race in exit1(). Specifically, exit1() > > sets p->p_xstat to the passed in exit status and sets P_WEXIT. It > > then drops the PROC_LOCK() to do other work such as freeing file > > descriptors, etc. Later it reacquires the PROC_LOCK(), marks the > > process as a zombie, and terminates. During the window where the > > PROC_LOCK() is not held, if a stop signal arrives (SIGSTOP, SIGTSTP, > > etc.), then the signal code will overwrite p->p_xstat with the signal > > that initiated the stop. The end result is that setting from SIGSTOP > > stays in p->p_xstat and is reported via wait(2). > > > > My first attempt at a fix was to simply ignore all signals once > > P_WEXIT is set by adding a check for P_WEXIT to the existing check for > > PRS_ZOMBIE. However, I think this is not quite right. For example, if > > an exiting process gets hung on an interruptible NFS mount while doing > > fdfree() I think we want a user to be able to kill the process to let > > it break out of NFS and finish exiting. > > > > The second fix I have is to explicitly clear P_STOPPED_SIGNAL to > > discard any pending SIGSTOP when setting P_WEXIT and to explicitly > > disallow any stop signals for a process that is currently exiting > > (that is, P_WEXIT is set). This fixes the race I ran into while still > > allowing other signals during process exit. The patch to do that is > > below. Below that is a test program that reproduces the issue. > > > > I would really like to get some feedback on which approach is best > > and if my concerns about allowing other signals during exit1() is a > > legitimate concern. (For some reason I feel like I've seen an argument > > for allowing that before.) > > > > Argument ? Please see r163541. I hope you agree with the author, who > basically provided the same reasoning about interruptible NFS mounts. I was using "argument" as in definition 4 here: http://dictionary.reference.com/browse/argument rather than definitions 1 or 2. As I said, I thought I recalled there being a good reason for allowing this. :) > So I agree with r163541, which log message says that I requested it. > And I do not see any other possible solution except case 2. Shouldn't > the fix actually prevent any overwrite of p_xstat for P_WEXIT process > from the signal delivery code ? The solution does do that AFAICT. SIGCONT should be a nop since the process should never be in a stopped state once P_WEXIT is set, and the patch explicitly turns SIGTOP into a nop. I added assertions to make sure that no signal is delivered to a stopped process while P_WEXIT is set as well as in the other places that overwrite p_xstat but that I think are unreachable (e.g. in ptracestop()). -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Sep 7 18:22:30 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 064521065675; Fri, 7 Sep 2012 18:22:29 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 821268FC0C; Fri, 7 Sep 2012 18:22:29 +0000 (UTC) Received: by obbun3 with SMTP id un3so6469764obb.13 for ; Fri, 07 Sep 2012 11:22:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=kxM3mO77U7ZNp0utZtVcyUCwrRQHDHGGoi6IzJR9dCI=; b=vPXmydu6u6E+AmRYcc7hDcw7Sqtc+WRnwHA+sxkPJKsc2mxb0NJ+BhwtFtnkn4YfOA AuRFehdkY+H3QiC3ljXfz46XWB+nM3FxgvgA0Mm7JCmxVGYaAfzXQYArecLnpsjFN6yL rwv3Th09PJ5V1ialqHbQqEptE9Tr0wViRHuRP7bqQuNBMKP2EoWtuewPpVRvHCImFFe/ HcFWxlk5yk/gaB5uZkkteRHsheDLxcwaBaRH2UWnp2dgxXIG0odbDccpcPQKz41Np/hP xF8KD65/p3j1bZCovo6UXDL5d1gJBLcc0AnKRAVeNEwlgrGXq3j9kW/po+6FhdBYwePH 9iDA== MIME-Version: 1.0 Received: by 10.182.222.39 with SMTP id qj7mr7144912obc.16.1347042143309; Fri, 07 Sep 2012 11:22:23 -0700 (PDT) Received: by 10.76.142.201 with HTTP; Fri, 7 Sep 2012 11:22:23 -0700 (PDT) In-Reply-To: <201209071416.24970.jhb@freebsd.org> References: <201209071217.47439.jhb@freebsd.org> <201209071416.24970.jhb@freebsd.org> Date: Fri, 7 Sep 2012 11:22:23 -0700 Message-ID: From: Garrett Cooper To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: arch@freebsd.org Subject: Re: [PATCH] Close a race between exit1() and SIGSTOP X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 18:22:30 -0000 On Fri, Sep 7, 2012 at 11:16 AM, John Baldwin wrote: > On Friday, September 07, 2012 12:35:52 pm Garrett Cooper wrote: >> Does this case continue to function normally? >> >> $ sh -c 'while :; do sleep 1; done' >> ^Z >> [1]+ Stopped sh -c 'while :; do sleep 1; done' >> $ kill %1 >> $ fg >> bash: fg: job has terminated >> [1]+ Terminated: 15 sh -c 'while :; do sleep 1; done' >> >> $ sh -c 'while :; do sleep 1; done' >> ^Z >> [1]+ Stopped sh -c 'while :; do sleep 1; done' >> $ kill %1 >> $ kill %1 >> bash: kill: (3522) - No such process >> [1]+ Terminated: 15 sh -c 'while :; do sleep 1; done' >> >> In particular, a single kill results in the signal being sent after >> the process wakes up, and a double kill results in the immediate death >> of the process (in this case a shell job) (in part because of the >> signals /bin/sh masks). > > This still works in bash: > >> bash > $ sh -c 'while :; do sleep 1; done' > ^Z > [1]+ Stopped sh -c 'while :; do sleep 1; done' > $ kill %1 > $ fg > bash: fg: job has terminated > [1]+ Terminated: 15 sh -c 'while :; do sleep 1; done' > $ sh -c 'while :; do sleep 1; done' > ^Z > [1]+ Stopped sh -c 'while :; do sleep 1; done' > $ kill %1 > $ kill %1 > bash: kill: (61009) - No such process > [1]+ Terminated: 15 sh -c 'while :; do sleep 1; done' > > I get different behavior in tcsh: > >> sh -c 'while :; do sleep 1; done' > ^Z > Suspended >> kill %1 >> > [1] Terminated sh -c while :; do sleep 1; done > > Note that this patch merey inhibits SIGSTOP from having an effect while > a process is exiting, it does not affect SIGTERM at all, so I don't think > it could have any effect on this case anyway. I should have said /bin/sh (I don't use tcsh by choice anymore because of its fun bugs). Good point.. I'd need to do more digging to better understand what is trying to be resolved here. Thanks! -Garrett From owner-freebsd-arch@FreeBSD.ORG Sat Sep 8 16:52:36 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 82E491065674; Sat, 8 Sep 2012 16:52:36 +0000 (UTC) (envelope-from marcel@xcllnt.net) Received: from mail.xcllnt.net (mail.xcllnt.net [70.36.220.4]) by mx1.freebsd.org (Postfix) with ESMTP id D7F418FC12; Sat, 8 Sep 2012 16:52:35 +0000 (UTC) Received: from marcelm-sslvpn-nc.jnpr.net (natint3.juniper.net [66.129.224.36]) (authenticated bits=0) by mail.xcllnt.net (8.14.5/8.14.5) with ESMTP id q88GqOsO028614 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sat, 8 Sep 2012 09:52:24 -0700 (PDT) (envelope-from marcel@xcllnt.net) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) From: Marcel Moolenaar In-Reply-To: <1346857799.59094.66.camel@revolution.hippie.lan> Date: Sat, 8 Sep 2012 09:52:23 -0700 Content-Transfer-Encoding: 7bit Message-Id: <01CF0B3B-1F24-4BAA-A662-356796DE2EDA@xcllnt.net> References: <1346689154.1140.601.camel@revolution.hippie.lan> <3AFC763F-011C-46B4-B500-FE21B704259F@bsdimp.com> <1346857799.59094.66.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1486) Cc: freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Some busdma stats X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2012 16:52:36 -0000 On Sep 5, 2012, at 8:09 AM, Ian Lepore wrote: > For that to work, the busdma implementation needs to be able to > efficiently allocate buffers that are properly aligned and padded and > especially that are guaranteed not to share a cache line with some other > unrelated data. The busdma implementation can't get those guarantees > from malloc(9), and the alternatives (contigmalloc(), and the kmem_alloc > family) only work in page-sized chunks. We're asking drivers to > allocate individual buffers of sometimes no more than a few bytes each. In my new busdma implementation this is indeed one of the things that get addressed. In fact, it contains a reverse mapping from the busdma tag as we know it (representing the device's perspective) to a new memory tag used to represent the corresponding CPU perspective. It's the memory tag that can only be used for allocating DMA-capable memory, as bus implementation may have non-trivial address transfor- mations. Note that the memory tag is where you assert CPU constraints, like cacheline alignment. See also: http://wiki.freebsd.org/UnifiedBusDma I'm working on the projects/altix2 branch right now. > So that's all I'm addressing in the patchset I submitted: make sure > that when we start fixing drivers to allocate 256 individual 16-byte IO > descriptors that it shares with the hardware, that doesn't result in > allocating 256 pages of memory. Yup. > Also, if the request is for > BUS_DMA_COHERENT memory, make sure that doesn't result in turning off > caching in up to 256 pages that each contain a 16 byte IO buffer and > 4080 bytes of unrelated data. Or worse: you end up with multiple incompatible translations for the same physical page. This tends to cause machine checks. The busdma infrastructure needs to maintain it's own list of physical pages and do so per "attribute". BTW: Are you interested in fixing bounce buffering as well? -- Marcel Moolenaar marcel@xcllnt.net From owner-freebsd-arch@FreeBSD.ORG Sat Sep 8 22:10:30 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8749C106566C for ; Sat, 8 Sep 2012 22:10:30 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 1A3568FC0A for ; Sat, 8 Sep 2012 22:10:29 +0000 (UTC) Received: by pbbrp2 with SMTP id rp2so1258565pbb.13 for ; Sat, 08 Sep 2012 15:10:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:from:date:message-id:subject:to:content-type; bh=2tX1tbtDNEOapWc2QyCMG7bd3qEzkMx2pNVBtRo9Bjw=; b=AjWwFhllwsgH94NPi072fKZHltIxm0hHofHBs+e5vOi7R0gVugGuCB6i3+9ZfG2x7J /wZKeqjDvYL4Xi/LZvDp+LF92taXQtt/Lhm+L4WiUojaYDwkOO+N3FTGvjp8B5epNIhM 0YZw7oo5FhcVFa/vlQO9B8oSgtGXfSrf8g+ds= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type :x-gm-message-state; bh=2tX1tbtDNEOapWc2QyCMG7bd3qEzkMx2pNVBtRo9Bjw=; b=mYSzO3JDQTFUaY0WOp4MG5PT73oDc9E1P6yuhu+QOoaDDa+lkX7WS18rhVVS7mG0EI wnM7NwVZ5nqrnZ64fspSpQ+K8hsbT2izPycXRb8jZISXniiSLh+O296NRKdeyjOraQwz ZHAGzXaAMjcoaTTmFOHFBsQ2ljSPzTw8iUOG8NL/Bdqbp6PBUdJggoxa4h0XopHKywlm JPBa4fkxVCeIqu3XAA+3y69OgfY6MOaNoki05rdSaKPnfkwr1AtLOTJ+nk4RbgAsD707 W9OWZCvJmNM6jqb8EvFr3Wbwttd0LYwEYsrkwkyiTONqfYaDsyFOMZfdEPD52rNnyofs ZHRg== Received: by 10.68.229.73 with SMTP id so9mr17962988pbc.66.1347142227713; Sat, 08 Sep 2012 15:10:27 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.193.101 with HTTP; Sat, 8 Sep 2012 15:09:55 -0700 (PDT) From: Eitan Adler Date: Sat, 8 Sep 2012 18:09:55 -0400 Message-ID: To: arch@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQlxeBCobTUUopmYD0WmSN1wcI/oYgapleEy8q6taJHqW+Qda40NGVcItBLKkoTH8aIcH25x Cc: Subject: Removing CVS from HEAD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2012 22:10:30 -0000 Hi all, CVS is obsolete. Virtually everyone that uses a version control system chooses git, mercurial, subversion, etc. FreeBSD has finally migrated from CVS for all of the repositories [2]. The ports management team has announced the end of CVS support in six months time (the end of February 2013). The documentation project does not export subversion to CVS. The source repository will eventually cease support of CVS as well. As such I propose that we treat CVS as deprecated in 9.x and remove CVS from HEAD [0]. There is already a port devel/cvs which uses a virtually identical copy of the CVS sources in HEAD as of today. [0] http://people.freebsd.org/~eadler/files/svn-remove-cvs-from-base.diff [1] http://www.mail-archive.com/freebsd-ports@freebsd.org/msg44029.html [2] projcvs does not count -- Eitan Adler From owner-freebsd-arch@FreeBSD.ORG Sat Sep 8 23:47:20 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C0BF2106564A for ; Sat, 8 Sep 2012 23:47:20 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (host-122-100-2-194.octopus.com.au [122.100.2.194]) by mx1.freebsd.org (Postfix) with ESMTP id 13A4E8FC08 for ; Sat, 8 Sep 2012 23:47:19 +0000 (UTC) Received: from server.rulingia.com (c220-239-249-137.belrs5.nsw.optusnet.com.au [220.239.249.137]) by vps.rulingia.com (8.14.5/8.14.5) with ESMTP id q88Nl6Vu017184 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Sun, 9 Sep 2012 09:47:07 +1000 (EST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.14.5/8.14.5) with ESMTP id q88Nl0uw011608 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 9 Sep 2012 09:47:00 +1000 (EST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.14.5/8.14.5/Submit) id q88Nkx6B011606; Sun, 9 Sep 2012 09:46:59 +1000 (EST) (envelope-from peter) Date: Sun, 9 Sep 2012 09:46:59 +1000 From: Peter Jeremy To: Eitan Adler Message-ID: <20120908234659.GA10489@server.rulingia.com> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="5vNYLRcllDrimb99" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.5.21 (2010-09-15) Cc: arch@freebsd.org Subject: Re: Removing CVS from HEAD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2012 23:47:20 -0000 --5vNYLRcllDrimb99 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2012-Sep-08 18:09:55 -0400, Eitan Adler wrote: >As such I propose that we treat CVS as deprecated in 9.x and remove >CVS from HEAD [0]. peter@ hinted at this in http://lists.freebsd.org/pipermail/freebsd-stable/2012-August/069251.html and this lead to a subthread where jhb@ and Julian H. Stacey disagreed whilst I agreed and further suggested that RCS also be removed. As I said before: CVS (and RCS) are both GPL-licensed tools that (as of 10.x) no longer serve any purpose in the base system. ... They are [therefore] not needed in the FreeBSD base. --=20 Peter Jeremy --5vNYLRcllDrimb99 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlBL2PMACgkQ/opHv/APuIco5ACfb//iYY6frOUdtS8w0fgc6dpC 7AYAn2/yjjrfG+Pq1bwedAKkTWS8O3Ah =MeJA -----END PGP SIGNATURE----- --5vNYLRcllDrimb99-- From owner-freebsd-arch@FreeBSD.ORG Sat Sep 8 23:50:30 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 69D48106566B for ; Sat, 8 Sep 2012 23:50:30 +0000 (UTC) (envelope-from bryan@shatow.net) Received: from secure.xzibition.com (secure.xzibition.com [173.160.118.92]) by mx1.freebsd.org (Postfix) with ESMTP id 0FC2E8FC0C for ; Sat, 8 Sep 2012 23:50:29 +0000 (UTC) DomainKey-Signature: a=rsa-sha1; c=nofws; d=shatow.net; h=message-id :date:from:mime-version:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; q=dns; s=sweb; b=WFUPJK I71tpNQgHlI31IWmdQmsCuJsph8xQ6A42kVRQTcRojx312iMNGoqcplEpD9q+Q6O bs/yB+Y5KEAtkqu4WqRIQ6aaNKPtmwkhI5K8Egb9AHhlSKMNWBNC4VxOG6XHm+ha 0a55xMNi062dzEwNfKyH/Lz52uKSOafKRpwKk= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=shatow.net; h=message-id :date:from:mime-version:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; s=sweb; bh=fckBj+D+byqp 3mRprJ7e6c1vr6hqfDTiTsSatLcL+f4=; b=MsQxaCDV40LvZKBWbbtEUDQGeB0x VD9vW9QxRXOS7fAMv/lXR+UKvDcpg2ZOLiReDFqfirAwoCpVW+y1ncRVb7qUFfQP Kr9l5WG0VMfaBzDn2XA2jAkUB5KLcN+5UWdDxaMEhxaxXJY/DeeftHvi3JPPiYh8 5fv1y/W/tDgpsFI= Received: (qmail 21336 invoked from network); 8 Sep 2012 18:50:27 -0500 Received: from unknown (HELO ?10.10.0.115?) (bryan@shatow.net@10.10.0.115) by sweb.xzibition.com with ESMTPA; 8 Sep 2012 18:50:27 -0500 Message-ID: <504BD9B5.20001@shatow.net> Date: Sat, 08 Sep 2012 18:50:13 -0500 From: Bryan Drewery User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) Gecko/20120824 Thunderbird/15.0 MIME-Version: 1.0 References: <20120908234659.GA10489@server.rulingia.com> In-Reply-To: <20120908234659.GA10489@server.rulingia.com> X-Enigmail-Version: 1.4.4 OpenPGP: id=3C9B0CF9; url=http://www.shatow.net/bryan/bryan.asc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Eitan Adler , arch@freebsd.org Subject: Re: Removing CVS from HEAD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2012 23:50:30 -0000 On 9/8/2012 6:46 PM, Peter Jeremy wrote: > As I said before: CVS (and RCS) are both GPL-licensed tools that (as > of 10.x) no longer serve any purpose in the base system. ... They are > [therefore] not needed in the FreeBSD base. Yes please, deprecate and remove both. -- Regards, Bryan Drewery bdrewery@freenode/EFNet