Date: Thu, 14 Nov 2019 23:31:20 +0000 (UTC) From: Josh Paetzel <jpaetzel@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r354715 - in head: share/man/man4 sys/amd64/conf sys/conf sys/dev/vmware/pvscsi sys/i386/conf sys/modules/vmware sys/modules/vmware/pvscsi Message-ID: <201911142331.xAENVKEU004510@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: jpaetzel Date: Thu Nov 14 23:31:20 2019 New Revision: 354715 URL: https://svnweb.freebsd.org/changeset/base/354715 Log: Add the pvscsi driver to the tree. This driver allows to usage of the paravirt SCSI controller in VMware products like ESXi. The pvscsi driver provides a substantial performance improvement in block devices versus the emulated mpt and mps SCSI/SAS controllers. Error handling in this driver has not been extensively tested yet. Submitted by: vbhakta@vmware.com Relnotes: yes Sponsored by: VMware, Panzura Differential Revision: D18613 Added: head/share/man/man4/pvscsi.4 (contents, props changed) head/sys/dev/vmware/pvscsi/ head/sys/dev/vmware/pvscsi/LICENSE (contents, props changed) head/sys/dev/vmware/pvscsi/pvscsi.c (contents, props changed) head/sys/dev/vmware/pvscsi/pvscsi.h (contents, props changed) head/sys/modules/vmware/pvscsi/ head/sys/modules/vmware/pvscsi/Makefile (contents, props changed) Modified: head/sys/amd64/conf/GENERIC head/sys/conf/files.amd64 head/sys/conf/files.i386 head/sys/i386/conf/GENERIC head/sys/modules/vmware/Makefile Added: head/share/man/man4/pvscsi.4 ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/share/man/man4/pvscsi.4 Thu Nov 14 23:31:20 2019 (r354715) @@ -0,0 +1,74 @@ +.\" Copyright (c) 2018 VMware, Inc. +.\" +.\" SPDX-License-Identifier: (BSD-2-Clause OR GPL-2.0) +.\" +.\" $FreeBSD$ +.Dd December 5, 2018 +.Dt PVSCSI 4 +.Os +.Sh NAME +.Nm pvscsi +.Nd VMware Paravirtual SCSI Controller +.Sh SYNOPSIS +To compile this driver into the kernel, +place the following line in your +kernel configuration file: +.Bd -ragged -offset indent +.Cd "device pci" +.Cd "device scbus" +.Cd "device pvscsi" +.Ed +.Pp +Alternatively, to load the driver as a +module at boot time, place the following line in +.Xr loader.conf 5 : +.Bd -literal -offset indent +pvscsi_load="YES" +.Ed +.Pp +The following tunables are settable from the +.Xr loader 8 : +.Bl -ohang +.It Va hw.pvscsi.request_ring_pages +controls how many pages are allocated for the device request ring. +A non-positive value will cause the driver to choose the value based on device +capabilities. +A non-zero value will use that many number of pages up to a maximum of 32. +The default setting is 0. +.It Va hw.pvscsi.max_queue_depth +controls the queue size for the adapter. +A non-positive value will cause the driver to choose the value based on number +of request ring pages. +A non-zero value will set the queue size up to a maximum allowed by the number +of request ring pages. +Default is 0. +.It Va hw.pvscsi.use_msg +setting to nonzero value enables the use of the PVSCSI message queue allowing +for disk hot-add and remove without manual rescan needed. +Default is 1. +.It Va hw.pvscsi.use_msi +setting to nonzero value enables the use of MSI interrupts. +Default is 1. +.It Va hw.pvscsi.use_msix +setting to nonzero value enables the use of MSI-X interrupts. +Default is 1. +.It Va hw.pvscsi.use_req_call_threshold +setting to nonzero value enables the request call threshold functionality. +TODO. +Default is 1. +.El +.Sh DESCRIPTION +The +.Nm +driver provides support for the VMware Paravirtual SCSI Controller (PVSCSI) in +virtual machines by VMware. +.Sh SEE ALSO +.Xr cam 4 , +.Xr da 4 +.Sh HISTORY +The +.Nm +driver first appeared in +.Fx 13.0 . +.Sh AUTHORS +.An Vishal Bhakta Aq Mt vbhakta@vmware.com . Modified: head/sys/amd64/conf/GENERIC ============================================================================== --- head/sys/amd64/conf/GENERIC Thu Nov 14 21:58:40 2019 (r354714) +++ head/sys/amd64/conf/GENERIC Thu Nov 14 23:31:20 2019 (r354715) @@ -152,6 +152,7 @@ device sym # NCR/Symbios Logic device trm # Tekram DC395U/UW/F DC315U adapters device isci # Intel C600 SAS controller device ocs_fc # Emulex FC adapters +device pvscsi # VMware PVSCSI # ATA/SCSI peripherals device scbus # SCSI bus (required for ATA/SCSI) Modified: head/sys/conf/files.amd64 ============================================================================== --- head/sys/conf/files.amd64 Thu Nov 14 21:58:40 2019 (r354714) +++ head/sys/conf/files.amd64 Thu Nov 14 23:31:20 2019 (r354715) @@ -345,6 +345,7 @@ dev/vmware/vmci/vmci_kernel_if.c optional vmci dev/vmware/vmci/vmci_qpair.c optional vmci dev/vmware/vmci/vmci_queue_pair.c optional vmci dev/vmware/vmci/vmci_resource.c optional vmci +dev/vmware/pvscsi/pvscsi.c optional pvscsi dev/vmd/vmd.c optional vmd dev/vmd/vmd_bus.c optional vmd_bus dev/wbwd/wbwd.c optional wbwd Modified: head/sys/conf/files.i386 ============================================================================== --- head/sys/conf/files.i386 Thu Nov 14 21:58:40 2019 (r354714) +++ head/sys/conf/files.i386 Thu Nov 14 23:31:20 2019 (r354715) @@ -162,6 +162,7 @@ dev/vmware/vmci/vmci_kernel_if.c optional vmci dev/vmware/vmci/vmci_qpair.c optional vmci dev/vmware/vmci/vmci_queue_pair.c optional vmci dev/vmware/vmci/vmci_resource.c optional vmci +dev/vmware/pvscsi/pvscsi.c optional pvscsi dev/acpi_support/acpi_wmi_if.m standard dev/wbwd/wbwd.c optional wbwd i386/acpica/acpi_machdep.c optional acpi Added: head/sys/dev/vmware/pvscsi/LICENSE ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/dev/vmware/pvscsi/LICENSE Thu Nov 14 23:31:20 2019 (r354715) @@ -0,0 +1,51 @@ +$FreeBSD$ + +These files are provided under a dual BSD-2 Clause/GPLv2 license. When +using or redistributing this file, you may do so under either license. + +BSD-2 Clause License + +Copyright (c) 2018 VMware, Inc. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +GPL License Summary + +Copyright (c) 2018 VMware, Inc. + +This program is free software; you can redistribute it and/or modify +it under the terms of version 2 of the GNU General Public License as +published by the Free Software Foundation. + +This program is distributed in the hope that it will be useful, but +WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program; if not, write to the Free Software +Foundation, Inc., 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA. +The full GNU General Public License is included in this distribution +in the file called LICENSE.GPL. Added: head/sys/dev/vmware/pvscsi/pvscsi.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/dev/vmware/pvscsi/pvscsi.c Thu Nov 14 23:31:20 2019 (r354715) @@ -0,0 +1,1804 @@ +/*- + * Copyright (c) 2018 VMware, Inc. + * + * SPDX-License-Identifier: (BSD-2-Clause OR GPL-2.0) + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include <sys/param.h> +#include <sys/bus.h> +#include <sys/errno.h> +#include <sys/kernel.h> +#include <sys/malloc.h> +#include <sys/module.h> +#include <sys/queue.h> +#include <sys/rman.h> +#include <sys/sysctl.h> +#include <sys/systm.h> + +#include <machine/bus.h> +#include <machine/resource.h> + +#include <dev/pci/pcireg.h> +#include <dev/pci/pcivar.h> + +#include <cam/cam.h> +#include <cam/cam_ccb.h> +#include <cam/cam_debug.h> +#include <cam/cam_sim.h> +#include <cam/cam_xpt_sim.h> +#include <cam/scsi/scsi_message.h> + +#include "pvscsi.h" + +#define PVSCSI_DEFAULT_NUM_PAGES_REQ_RING 8 +#define PVSCSI_SENSE_LENGTH 256 + +MALLOC_DECLARE(M_PVSCSI); +MALLOC_DEFINE(M_PVSCSI, "pvscsi", "PVSCSI memory"); + +#ifdef PVSCSI_DEBUG_LOGGING +#define DEBUG_PRINTF(level, dev, fmt, ...) \ + do { \ + if (pvscsi_log_level >= (level)) { \ + device_printf((dev), (fmt), ##__VA_ARGS__); \ + } \ + } while(0) +#else +#define DEBUG_PRINTF(level, dev, fmt, ...) +#endif /* PVSCSI_DEBUG_LOGGING */ + +#define ccb_pvscsi_hcb spriv_ptr0 +#define ccb_pvscsi_sc spriv_ptr1 + +struct pvscsi_softc; +static timeout_t pvscsi_timeout; +struct pvscsi_hcb; +struct pvscsi_dma; + +static inline uint32_t pvscsi_reg_read(struct pvscsi_softc *sc, + uint32_t offset); +static inline void pvscsi_reg_write(struct pvscsi_softc *sc, uint32_t offset, + uint32_t val); +static inline uint32_t pvscsi_read_intr_status(struct pvscsi_softc *sc); +static inline void pvscsi_write_intr_status(struct pvscsi_softc *sc, + uint32_t val); +static inline void pvscsi_intr_enable(struct pvscsi_softc *sc); +static inline void pvscsi_intr_disable(struct pvscsi_softc *sc); +static void pvscsi_kick_io(struct pvscsi_softc *sc, uint8_t cdb0); +static void pvscsi_write_cmd(struct pvscsi_softc *sc, uint32_t cmd, void *data, + uint32_t len); +static uint32_t pvscsi_get_max_targets(struct pvscsi_softc *sc); +static int pvscsi_setup_req_call(struct pvscsi_softc *sc, uint32_t enable); +static void pvscsi_setup_rings(struct pvscsi_softc *sc); +static void pvscsi_setup_msg_ring(struct pvscsi_softc *sc); +static int pvscsi_hw_supports_msg(struct pvscsi_softc *sc); + +static void pvscsi_timeout(void *arg); +static void pvscsi_freeze(struct pvscsi_softc *sc); +static void pvscsi_adapter_reset(struct pvscsi_softc *sc); +static void pvscsi_bus_reset(struct pvscsi_softc *sc); +static void pvscsi_device_reset(struct pvscsi_softc *sc, uint32_t target); +static void pvscsi_abort(struct pvscsi_softc *sc, uint32_t target, + union ccb *ccb); + +static void pvscsi_process_completion(struct pvscsi_softc *sc, + struct pvscsi_ring_cmp_desc *e); +static void pvscsi_process_cmp_ring(struct pvscsi_softc *sc); +static void pvscsi_process_msg(struct pvscsi_softc *sc, + struct pvscsi_ring_msg_desc *e); +static void pvscsi_process_msg_ring(struct pvscsi_softc *sc); + +static void pvscsi_intr_locked(struct pvscsi_softc *sc); +static void pvscsi_intr(void *xsc); +static void pvscsi_poll(struct cam_sim *sim); + +static void pvscsi_execute_ccb(void *arg, bus_dma_segment_t *segs, int nseg, + int error); +static void pvscsi_action(struct cam_sim *sim, union ccb *ccb); + +static inline uint64_t pvscsi_hcb_to_context(struct pvscsi_softc *sc, + struct pvscsi_hcb *hcb); +static inline struct pvscsi_hcb* pvscsi_context_to_hcb(struct pvscsi_softc *sc, + uint64_t context); +static struct pvscsi_hcb * pvscsi_hcb_get(struct pvscsi_softc *sc); +static void pvscsi_hcb_put(struct pvscsi_softc *sc, struct pvscsi_hcb *hcb); + +static void pvscsi_dma_cb(void *arg, bus_dma_segment_t *segs, int nseg, + int error); +static void pvscsi_dma_free(struct pvscsi_softc *sc, struct pvscsi_dma *dma); +static int pvscsi_dma_alloc(struct pvscsi_softc *sc, struct pvscsi_dma *dma, + bus_size_t size, bus_size_t alignment); +static int pvscsi_dma_alloc_ppns(struct pvscsi_softc *sc, + struct pvscsi_dma *dma, uint64_t *ppn_list, uint32_t num_pages); +static void pvscsi_dma_free_per_hcb(struct pvscsi_softc *sc, + uint32_t hcbs_allocated); +static int pvscsi_dma_alloc_per_hcb(struct pvscsi_softc *sc); +static void pvscsi_free_rings(struct pvscsi_softc *sc); +static int pvscsi_allocate_rings(struct pvscsi_softc *sc); +static void pvscsi_free_interrupts(struct pvscsi_softc *sc); +static int pvscsi_setup_interrupts(struct pvscsi_softc *sc); +static void pvscsi_free_all(struct pvscsi_softc *sc); + +static int pvscsi_attach(device_t dev); +static int pvscsi_detach(device_t dev); +static int pvscsi_probe(device_t dev); +static int pvscsi_shutdown(device_t dev); +static int pvscsi_get_tunable(struct pvscsi_softc *sc, char *name, int value); + + +#ifdef PVSCSI_DEBUG_LOGGING +static int pvscsi_log_level = 0; +static SYSCTL_NODE(_hw, OID_AUTO, pvscsi, CTLFLAG_RD, 0, + "PVSCSI driver parameters"); +SYSCTL_INT(_hw_pvscsi, OID_AUTO, log_level, CTLFLAG_RWTUN, &pvscsi_log_level, + 0, "PVSCSI debug log level"); +#endif + +static int pvscsi_request_ring_pages = 0; +TUNABLE_INT("hw.pvscsi.request_ring_pages", &pvscsi_request_ring_pages); + +static int pvscsi_use_msg = 1; +TUNABLE_INT("hw.pvscsi.use_msg", &pvscsi_use_msg); + +static int pvscsi_use_msi = 1; +TUNABLE_INT("hw.pvscsi.use_msi", &pvscsi_use_msi); + +static int pvscsi_use_msix = 1; +TUNABLE_INT("hw.pvscsi.use_msix", &pvscsi_use_msix); + +static int pvscsi_use_req_call_threshold = 1; +TUNABLE_INT("hw.pvscsi.use_req_call_threshold", &pvscsi_use_req_call_threshold); + +static int pvscsi_max_queue_depth = 0; +TUNABLE_INT("hw.pvscsi.max_queue_depth", &pvscsi_max_queue_depth); + + +struct pvscsi_sg_list { + struct pvscsi_sg_element sge[PVSCSI_MAX_SG_ENTRIES_PER_SEGMENT]; +}; + + +#define PVSCSI_ABORT_TIMEOUT 2 +#define PVSCSI_RESET_TIMEOUT 10 + +#define PVSCSI_HCB_NONE 0 +#define PVSCSI_HCB_ABORT 1 +#define PVSCSI_HCB_DEVICE_RESET 2 +#define PVSCSI_HCB_BUS_RESET 3 + +struct pvscsi_hcb { + union ccb *ccb; + struct pvscsi_ring_req_desc *e; + int recovery; + SLIST_ENTRY(pvscsi_hcb) links; + + struct callout callout; + bus_dmamap_t dma_map; + void *sense_buffer; + bus_addr_t sense_buffer_paddr; + struct pvscsi_sg_list *sg_list; + bus_addr_t sg_list_paddr; +}; + +struct pvscsi_dma +{ + bus_dma_tag_t tag; + bus_dmamap_t map; + void *vaddr; + bus_addr_t paddr; + bus_size_t size; +}; + +struct pvscsi_softc { + device_t dev; + struct mtx lock; + struct cam_sim *sim; + struct cam_path *bus_path; + int frozen; + struct pvscsi_rings_state *rings_state; + struct pvscsi_ring_req_desc *req_ring; + struct pvscsi_ring_cmp_desc *cmp_ring; + struct pvscsi_ring_msg_desc *msg_ring; + uint32_t hcb_cnt; + struct pvscsi_hcb *hcbs; + SLIST_HEAD(, pvscsi_hcb) free_list; + bus_dma_tag_t parent_dmat; + bus_dma_tag_t buffer_dmat; + + bool use_msg; + uint32_t max_targets; + int mm_rid; + struct resource *mm_res; + int irq_id; + struct resource *irq_res; + void *irq_handler; + int use_req_call_threshold; + int use_msi_or_msix; + + uint64_t rings_state_ppn; + uint32_t req_ring_num_pages; + uint64_t req_ring_ppn[PVSCSI_MAX_NUM_PAGES_REQ_RING]; + uint32_t cmp_ring_num_pages; + uint64_t cmp_ring_ppn[PVSCSI_MAX_NUM_PAGES_CMP_RING]; + uint32_t msg_ring_num_pages; + uint64_t msg_ring_ppn[PVSCSI_MAX_NUM_PAGES_MSG_RING]; + + struct pvscsi_dma rings_state_dma; + struct pvscsi_dma req_ring_dma; + struct pvscsi_dma cmp_ring_dma; + struct pvscsi_dma msg_ring_dma; + + struct pvscsi_dma sg_list_dma; + struct pvscsi_dma sense_buffer_dma; +}; + +static int pvscsi_get_tunable(struct pvscsi_softc *sc, char *name, int value) +{ + char cfg[64]; + + snprintf(cfg, sizeof(cfg), "hw.pvscsi.%d.%s", device_get_unit(sc->dev), + name); + TUNABLE_INT_FETCH(cfg, &value); + + return (value); +} + +static void +pvscsi_freeze(struct pvscsi_softc *sc) +{ + + if (!sc->frozen) { + xpt_freeze_simq(sc->sim, 1); + sc->frozen = 1; + } +} + +static inline uint32_t +pvscsi_reg_read(struct pvscsi_softc *sc, uint32_t offset) +{ + + return (bus_read_4(sc->mm_res, offset)); +} + +static inline void +pvscsi_reg_write(struct pvscsi_softc *sc, uint32_t offset, uint32_t val) +{ + + bus_write_4(sc->mm_res, offset, val); +} + +static inline uint32_t +pvscsi_read_intr_status(struct pvscsi_softc *sc) +{ + + return (pvscsi_reg_read(sc, PVSCSI_REG_OFFSET_INTR_STATUS)); +} + +static inline void +pvscsi_write_intr_status(struct pvscsi_softc *sc, uint32_t val) +{ + + pvscsi_reg_write(sc, PVSCSI_REG_OFFSET_INTR_STATUS, val); +} + +static inline void +pvscsi_intr_enable(struct pvscsi_softc *sc) +{ + uint32_t mask; + + mask = PVSCSI_INTR_CMPL_MASK; + if (sc->use_msg) { + mask |= PVSCSI_INTR_MSG_MASK; + } + + pvscsi_reg_write(sc, PVSCSI_REG_OFFSET_INTR_MASK, mask); +} + +static inline void +pvscsi_intr_disable(struct pvscsi_softc *sc) +{ + + pvscsi_reg_write(sc, PVSCSI_REG_OFFSET_INTR_MASK, 0); +} + +static void +pvscsi_kick_io(struct pvscsi_softc *sc, uint8_t cdb0) +{ + struct pvscsi_rings_state *s; + + if (cdb0 == READ_6 || cdb0 == READ_10 || + cdb0 == READ_12 || cdb0 == READ_16 || + cdb0 == WRITE_6 || cdb0 == WRITE_10 || + cdb0 == WRITE_12 || cdb0 == WRITE_16) { + s = sc->rings_state; + + if (!sc->use_req_call_threshold || + (s->req_prod_idx - s->req_cons_idx) >= + s->req_call_threshold) { + pvscsi_reg_write(sc, PVSCSI_REG_OFFSET_KICK_RW_IO, 0); + } + } else { + pvscsi_reg_write(sc, PVSCSI_REG_OFFSET_KICK_NON_RW_IO, 0); + } +} + +static void +pvscsi_write_cmd(struct pvscsi_softc *sc, uint32_t cmd, void *data, + uint32_t len) +{ + uint32_t *data_ptr; + int i; + + KASSERT(len % sizeof(uint32_t) == 0, + ("command size not a multiple of 4")); + + data_ptr = data; + len /= sizeof(uint32_t); + + pvscsi_reg_write(sc, PVSCSI_REG_OFFSET_COMMAND, cmd); + for (i = 0; i < len; ++i) { + pvscsi_reg_write(sc, PVSCSI_REG_OFFSET_COMMAND_DATA, + data_ptr[i]); + } +} + +static inline uint64_t pvscsi_hcb_to_context(struct pvscsi_softc *sc, + struct pvscsi_hcb *hcb) +{ + + /* Offset by 1 because context must not be 0 */ + return (hcb - sc->hcbs + 1); +} + +static inline struct pvscsi_hcb* pvscsi_context_to_hcb(struct pvscsi_softc *sc, + uint64_t context) +{ + + return (sc->hcbs + (context - 1)); +} + +static struct pvscsi_hcb * +pvscsi_hcb_get(struct pvscsi_softc *sc) +{ + struct pvscsi_hcb *hcb; + + mtx_assert(&sc->lock, MA_OWNED); + + hcb = SLIST_FIRST(&sc->free_list); + if (hcb) { + SLIST_REMOVE_HEAD(&sc->free_list, links); + } + + return (hcb); +} + +static void +pvscsi_hcb_put(struct pvscsi_softc *sc, struct pvscsi_hcb *hcb) +{ + + mtx_assert(&sc->lock, MA_OWNED); + hcb->ccb = NULL; + hcb->e = NULL; + hcb->recovery = PVSCSI_HCB_NONE; + SLIST_INSERT_HEAD(&sc->free_list, hcb, links); +} + +static uint32_t +pvscsi_get_max_targets(struct pvscsi_softc *sc) +{ + uint32_t max_targets; + + pvscsi_write_cmd(sc, PVSCSI_CMD_GET_MAX_TARGETS, NULL, 0); + + max_targets = pvscsi_reg_read(sc, PVSCSI_REG_OFFSET_COMMAND_STATUS); + + if (max_targets == ~0) { + max_targets = 16; + } + + return (max_targets); +} + +static int pvscsi_setup_req_call(struct pvscsi_softc *sc, uint32_t enable) +{ + uint32_t status; + struct pvscsi_cmd_desc_setup_req_call cmd; + + if (!pvscsi_get_tunable(sc, "pvscsi_use_req_call_threshold", + pvscsi_use_req_call_threshold)) { + return (0); + } + + pvscsi_reg_write(sc, PVSCSI_REG_OFFSET_COMMAND, + PVSCSI_CMD_SETUP_REQCALLTHRESHOLD); + status = pvscsi_reg_read(sc, PVSCSI_REG_OFFSET_COMMAND_STATUS); + + if (status != -1) { + bzero(&cmd, sizeof(cmd)); + cmd.enable = enable; + pvscsi_write_cmd(sc, PVSCSI_CMD_SETUP_REQCALLTHRESHOLD, + &cmd, sizeof(cmd)); + status = pvscsi_reg_read(sc, PVSCSI_REG_OFFSET_COMMAND_STATUS); + + return (status != 0); + } else { + return (0); + } +} + +static void +pvscsi_dma_cb(void *arg, bus_dma_segment_t *segs, int nseg, int error) +{ + bus_addr_t *dest; + + KASSERT(nseg == 1, ("more than one segment")); + + dest = arg; + + if (!error) { + *dest = segs->ds_addr; + } +} + +static void +pvscsi_dma_free(struct pvscsi_softc *sc, struct pvscsi_dma *dma) +{ + + if (dma->tag != NULL) { + if (dma->paddr != 0) { + bus_dmamap_unload(dma->tag, dma->map); + } + + if (dma->vaddr != NULL) { + bus_dmamem_free(dma->tag, dma->vaddr, dma->map); + } + + bus_dma_tag_destroy(dma->tag); + } + + bzero(dma, sizeof(*dma)); +} + +static int +pvscsi_dma_alloc(struct pvscsi_softc *sc, struct pvscsi_dma *dma, + bus_size_t size, bus_size_t alignment) +{ + int error; + + bzero(dma, sizeof(*dma)); + + error = bus_dma_tag_create(sc->parent_dmat, alignment, 0, + BUS_SPACE_MAXADDR, BUS_SPACE_MAXADDR, NULL, NULL, size, 1, size, + BUS_DMA_ALLOCNOW, NULL, NULL, &dma->tag); + if (error) { + device_printf(sc->dev, "error creating dma tag, error %d\n", + error); + goto fail; + } + + error = bus_dmamem_alloc(dma->tag, &dma->vaddr, + BUS_DMA_NOWAIT | BUS_DMA_ZERO, &dma->map); + if (error) { + device_printf(sc->dev, "error allocating dma mem, error %d\n", + error); + goto fail; + } + + error = bus_dmamap_load(dma->tag, dma->map, dma->vaddr, size, + pvscsi_dma_cb, &dma->paddr, BUS_DMA_NOWAIT); + if (error) { + device_printf(sc->dev, "error mapping dma mam, error %d\n", + error); + goto fail; + } + + dma->size = size; + +fail: + if (error) { + pvscsi_dma_free(sc, dma); + } + return (error); +} + +static int +pvscsi_dma_alloc_ppns(struct pvscsi_softc *sc, struct pvscsi_dma *dma, + uint64_t *ppn_list, uint32_t num_pages) +{ + int error; + uint32_t i; + uint64_t ppn; + + error = pvscsi_dma_alloc(sc, dma, num_pages * PAGE_SIZE, PAGE_SIZE); + if (error) { + device_printf(sc->dev, "Error allocating pages, error %d\n", + error); + return (error); + } + + ppn = dma->paddr >> PAGE_SHIFT; + for (i = 0; i < num_pages; i++) { + ppn_list[i] = ppn + i; + } + + return (0); +} + +static void +pvscsi_dma_free_per_hcb(struct pvscsi_softc *sc, uint32_t hcbs_allocated) +{ + int i; + int lock_owned; + struct pvscsi_hcb *hcb; + + lock_owned = mtx_owned(&sc->lock); + + if (lock_owned) { + mtx_unlock(&sc->lock); + } + for (i = 0; i < hcbs_allocated; ++i) { + hcb = sc->hcbs + i; + callout_drain(&hcb->callout); + }; + if (lock_owned) { + mtx_lock(&sc->lock); + } + + for (i = 0; i < hcbs_allocated; ++i) { + hcb = sc->hcbs + i; + bus_dmamap_destroy(sc->buffer_dmat, hcb->dma_map); + }; + + pvscsi_dma_free(sc, &sc->sense_buffer_dma); + pvscsi_dma_free(sc, &sc->sg_list_dma); +} + +static int +pvscsi_dma_alloc_per_hcb(struct pvscsi_softc *sc) +{ + int i; + int error; + struct pvscsi_hcb *hcb; + + i = 0; + + error = pvscsi_dma_alloc(sc, &sc->sg_list_dma, + sizeof(struct pvscsi_sg_list) * sc->hcb_cnt, 1); + if (error) { + device_printf(sc->dev, + "Error allocation sg list DMA memory, error %d\n", error); + goto fail; + } + + error = pvscsi_dma_alloc(sc, &sc->sense_buffer_dma, + PVSCSI_SENSE_LENGTH * sc->hcb_cnt, 1); + if (error) { + device_printf(sc->dev, + "Error allocation sg list DMA memory, error %d\n", error); + goto fail; + } + + for (i = 0; i < sc->hcb_cnt; ++i) { + hcb = sc->hcbs + i; + + error = bus_dmamap_create(sc->buffer_dmat, 0, &hcb->dma_map); + if (error) { + device_printf(sc->dev, + "Error creating dma map for hcb %d, error %d\n", + i, error); + goto fail; + } + + hcb->sense_buffer = + (void *)((caddr_t)sc->sense_buffer_dma.vaddr + + PVSCSI_SENSE_LENGTH * i); + hcb->sense_buffer_paddr = + sc->sense_buffer_dma.paddr + PVSCSI_SENSE_LENGTH * i; + + hcb->sg_list = + (struct pvscsi_sg_list *)((caddr_t)sc->sg_list_dma.vaddr + + sizeof(struct pvscsi_sg_list) * i); + hcb->sg_list_paddr = + sc->sg_list_dma.paddr + sizeof(struct pvscsi_sg_list) * i; + + callout_init_mtx(&hcb->callout, &sc->lock, 0); + } + + SLIST_INIT(&sc->free_list); + for (i = (sc->hcb_cnt - 1); i >= 0; --i) { + hcb = sc->hcbs + i; + SLIST_INSERT_HEAD(&sc->free_list, hcb, links); + } + +fail: + if (error) { + pvscsi_dma_free_per_hcb(sc, i); + } + + return (error); +} + +static void +pvscsi_free_rings(struct pvscsi_softc *sc) +{ + + pvscsi_dma_free(sc, &sc->rings_state_dma); + pvscsi_dma_free(sc, &sc->req_ring_dma); + pvscsi_dma_free(sc, &sc->cmp_ring_dma); + if (sc->use_msg) { + pvscsi_dma_free(sc, &sc->msg_ring_dma); + } +} + +static int +pvscsi_allocate_rings(struct pvscsi_softc *sc) +{ + int error; + + error = pvscsi_dma_alloc_ppns(sc, &sc->rings_state_dma, + &sc->rings_state_ppn, 1); + if (error) { + device_printf(sc->dev, + "Error allocating rings state, error = %d\n", error); + goto fail; + } + sc->rings_state = sc->rings_state_dma.vaddr; + + error = pvscsi_dma_alloc_ppns(sc, &sc->req_ring_dma, sc->req_ring_ppn, + sc->req_ring_num_pages); + if (error) { + device_printf(sc->dev, + "Error allocating req ring pages, error = %d\n", error); + goto fail; + } + sc->req_ring = sc->req_ring_dma.vaddr; + + error = pvscsi_dma_alloc_ppns(sc, &sc->cmp_ring_dma, sc->cmp_ring_ppn, + sc->cmp_ring_num_pages); + if (error) { + device_printf(sc->dev, + "Error allocating cmp ring pages, error = %d\n", error); + goto fail; + } + sc->cmp_ring = sc->cmp_ring_dma.vaddr; + + sc->msg_ring = NULL; + if (sc->use_msg) { + error = pvscsi_dma_alloc_ppns(sc, &sc->msg_ring_dma, + sc->msg_ring_ppn, sc->msg_ring_num_pages); + if (error) { + device_printf(sc->dev, + "Error allocating cmp ring pages, error = %d\n", + error); + goto fail; + } + sc->msg_ring = sc->msg_ring_dma.vaddr; + } + + DEBUG_PRINTF(1, sc->dev, "rings_state: %p\n", sc->rings_state); + DEBUG_PRINTF(1, sc->dev, "req_ring: %p - %u pages\n", sc->req_ring, + sc->req_ring_num_pages); + DEBUG_PRINTF(1, sc->dev, "cmp_ring: %p - %u pages\n", sc->cmp_ring, + sc->cmp_ring_num_pages); + DEBUG_PRINTF(1, sc->dev, "msg_ring: %p - %u pages\n", sc->msg_ring, + sc->msg_ring_num_pages); + +fail: + if (error) { + pvscsi_free_rings(sc); + } + return (error); +} + +static void +pvscsi_setup_rings(struct pvscsi_softc *sc) +{ + struct pvscsi_cmd_desc_setup_rings cmd; + uint32_t i; + + bzero(&cmd, sizeof(cmd)); + + cmd.rings_state_ppn = sc->rings_state_ppn; + + cmd.req_ring_num_pages = sc->req_ring_num_pages; + for (i = 0; i < sc->req_ring_num_pages; ++i) { + cmd.req_ring_ppns[i] = sc->req_ring_ppn[i]; + } + + cmd.cmp_ring_num_pages = sc->cmp_ring_num_pages; + for (i = 0; i < sc->cmp_ring_num_pages; ++i) { + cmd.cmp_ring_ppns[i] = sc->cmp_ring_ppn[i]; + } + + pvscsi_write_cmd(sc, PVSCSI_CMD_SETUP_RINGS, &cmd, sizeof(cmd)); +} + +static int +pvscsi_hw_supports_msg(struct pvscsi_softc *sc) +{ + uint32_t status; + + pvscsi_reg_write(sc, PVSCSI_REG_OFFSET_COMMAND, + PVSCSI_CMD_SETUP_MSG_RING); + status = pvscsi_reg_read(sc, PVSCSI_REG_OFFSET_COMMAND_STATUS); + + return (status != -1); +} + +static void +pvscsi_setup_msg_ring(struct pvscsi_softc *sc) +{ + struct pvscsi_cmd_desc_setup_msg_ring cmd; + uint32_t i; + + KASSERT(sc->use_msg, ("msg is not being used")); + + bzero(&cmd, sizeof(cmd)); + + cmd.num_pages = sc->msg_ring_num_pages; + for (i = 0; i < sc->msg_ring_num_pages; ++i) { + cmd.ring_ppns[i] = sc->msg_ring_ppn[i]; + } + + pvscsi_write_cmd(sc, PVSCSI_CMD_SETUP_MSG_RING, &cmd, sizeof(cmd)); +} + +static void +pvscsi_adapter_reset(struct pvscsi_softc *sc) +{ + uint32_t val; + + device_printf(sc->dev, "Adapter Reset\n"); + + pvscsi_write_cmd(sc, PVSCSI_CMD_ADAPTER_RESET, NULL, 0); + val = pvscsi_read_intr_status(sc); + + DEBUG_PRINTF(2, sc->dev, "adapter reset done: %u\n", val); +} + +static void +pvscsi_bus_reset(struct pvscsi_softc *sc) +{ + + device_printf(sc->dev, "Bus Reset\n"); + + pvscsi_write_cmd(sc, PVSCSI_CMD_RESET_BUS, NULL, 0); + pvscsi_process_cmp_ring(sc); + + DEBUG_PRINTF(2, sc->dev, "bus reset done\n"); +} + +static void +pvscsi_device_reset(struct pvscsi_softc *sc, uint32_t target) +{ + struct pvscsi_cmd_desc_reset_device cmd; + + memset(&cmd, 0, sizeof(cmd)); + + cmd.target = target; + + device_printf(sc->dev, "Device reset for target %u\n", target); + + pvscsi_write_cmd(sc, PVSCSI_CMD_RESET_DEVICE, &cmd, sizeof cmd); + pvscsi_process_cmp_ring(sc); + + DEBUG_PRINTF(2, sc->dev, "device reset done\n"); +} + +static void +pvscsi_abort(struct pvscsi_softc *sc, uint32_t target, union ccb *ccb) +{ + struct pvscsi_cmd_desc_abort_cmd cmd; + struct pvscsi_hcb *hcb; + uint64_t context; + + pvscsi_process_cmp_ring(sc); + + hcb = ccb->ccb_h.ccb_pvscsi_hcb; + + if (hcb != NULL) { + context = pvscsi_hcb_to_context(sc, hcb); + + memset(&cmd, 0, sizeof cmd); + cmd.target = target; + cmd.context = context; + + device_printf(sc->dev, "Abort for target %u context %llx\n", + target, (unsigned long long)context); + + pvscsi_write_cmd(sc, PVSCSI_CMD_ABORT_CMD, &cmd, sizeof(cmd)); + pvscsi_process_cmp_ring(sc); + + DEBUG_PRINTF(2, sc->dev, "abort done\n"); + } else { + DEBUG_PRINTF(1, sc->dev, + "Target %u ccb %p not found for abort\n", target, ccb); + } +} + +static int +pvscsi_probe(device_t dev) +{ + + if (pci_get_vendor(dev) == PCI_VENDOR_ID_VMWARE && + pci_get_device(dev) == PCI_DEVICE_ID_VMWARE_PVSCSI) { + device_set_desc(dev, "VMware Paravirtual SCSI Controller"); + return (BUS_PROBE_DEFAULT); + } + return (ENXIO); +} + +static int +pvscsi_shutdown(device_t dev) +{ *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201911142331.xAENVKEU004510>