Date: Wed, 6 Sep 2017 15:33:23 +0000 (UTC) From: Hans Petter Selasky <hselasky@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r323223 - in stable/10: share/man/man4 sys/conf sys/dev/mlx5/mlx5_ib sys/modules sys/modules/mlx5ib Message-ID: <201709061533.v86FXNRc015857@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: hselasky Date: Wed Sep 6 15:33:23 2017 New Revision: 323223 URL: https://svnweb.freebsd.org/changeset/base/323223 Log: MFC r322810 and r322830: Add new mlx5ib(4) driver to the kernel source tree which supports Remote DMA over Converged Ethernet, RoCE, for the ConnectX-4 series of PCI express network cards. There is currently no user-space support and this driver only supports kernel side non-routable RoCE V1. The krping kernel module can be used to test this driver. Full user-space support including RoCE V2 will be added as part of the ongoing upgrade to ibcore from Linux 4.9. Otherwise this driver is feature equivalent to mlx4ib(4). The mlx5ib(4) kernel module will only be built when WITH_OFED=YES is specified. Approved by: re (marius) Sponsored by: Mellanox Technologies Added: stable/10/share/man/man4/mlx5ib.4 - copied unchanged from r322810, head/share/man/man4/mlx5ib.4 stable/10/sys/dev/mlx5/mlx5_ib/ - copied from r322810, head/sys/dev/mlx5/mlx5_ib/ stable/10/sys/modules/mlx5ib/ - copied from r322810, head/sys/modules/mlx5ib/ Modified: stable/10/share/man/man4/Makefile stable/10/sys/conf/files stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib.h stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_ah.c stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_cq.c stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_doorbell.c stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_mad.c stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_mem.c stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_mr.c stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_qp.c stable/10/sys/modules/Makefile Directory Properties: stable/10/ (props changed) Modified: stable/10/share/man/man4/Makefile ============================================================================== --- stable/10/share/man/man4/Makefile Wed Sep 6 15:07:53 2017 (r323222) +++ stable/10/share/man/man4/Makefile Wed Sep 6 15:33:23 2017 (r323223) @@ -868,6 +868,7 @@ MAN+= iscsi_initiator.4 .if ${MK_OFED} != "no" MAN+= mlx4ib.4 +MAN+= mlx5ib.4 .endif .if ${MK_TESTS} != "no" Copied: stable/10/share/man/man4/mlx5ib.4 (from r322810, head/share/man/man4/mlx5ib.4) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ stable/10/share/man/man4/mlx5ib.4 Wed Sep 6 15:33:23 2017 (r323223, copy of r322810, head/share/man/man4/mlx5ib.4) @@ -0,0 +1,124 @@ +.\" Copyright (c) 2017 Mellanox Technologies +.\" All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS `AS IS' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.Dd August 23, 2017 +.Dt MLX5IB 4 +.Os +.Sh NAME +.Nm mlx5ib +.Nd "Mellanox ConnectX-4 and ConnectX-4 LX based 100Gb, 50Gb, 40Gb, 25Gb and 10Gb network adapter driver" +.Sh SYNOPSIS +To compile this driver into the kernel, +place these lines in your kernel configuration file: +.Bd -ragged -offset indent +.Cd "options COMPAT_LINUXKPI" +.Cd "device mlx5" +.Cd "device mlx5ib" +.Ed +.Pp +To load the driver as a module at run-time, +run this command as root: +.Bd -literal -offset indent +kldload mlx5ib +.Ed +.Pp +To load the driver as a +module at boot time, place this line in +.Xr loader.conf 5 : +.Bd -literal -offset indent +mlx5ib_load="YES" +.Ed +.Sh DESCRIPTION +The +.Nm +driver provides support for infiniband and Remote DMA over Converged Ethernet, +RoCE, for PCI Express network adapters based on ConnectX-4 and ConnectX-4 LX. +.br +For further hardware information and questions related to hardware +requirements, see +.Pa http://www.mellanox.com/ . +.Pp +For more information on configuring this device, see +.Xr ifconfig 8 . +.Sh HARDWARE +The +.Nm +driver supports 100Gb, 50Gb, 40Gb, 25Gb and 10Gb network adapters. +ConnectX-4 supports: 10/20/25/40/50/56/100Gb/s speeds. +ConnectX-4 LX supports: 10/25/40/50Gb/s speeds (and reduced power consumption): +.Pp +.Bl -bullet -compact +.It +Mellanox MCX455A-ECAT +.It +Mellanox MCX456A-ECAT +.It +Mellanox MCX415A-CCAT +.It +Mellanox MCX416A-CCAT +.It +Mellanox MCX455A-FCAT +.It +Mellanox MCX456A-FCAT +.It +Mellanox MCX415A-BCAT +.It +Mellanox MCX416A-BCAT +.It +Mellanox MCX4131A-GCAT +.It +Mellanox MCX4131A-BCAT +.It +Mellanox MCX4121A-ACAT +.It +Mellanox MCX4111A-ACAT +.It +Mellanox MCX4121A-XCAT +.It +Mellanox MCX4111A-XCAT +.El +.Sh SUPPORT +For general information and support, +go to the Mellanox support website at: +.Pa http://www.mellanox.com/ . +.Pp +If an issue is identified with this driver with a supported adapter, +email all the specific information related to the issue to +.Aq Mt freebsd-drivers@mellanox.com . +.Sh SEE ALSO +.Xr mlx5en 4 , +.Xr ifconfig 8 +.Sh HISTORY +The +.Nm +device driver first appeared in +.Fx 12.x . +.Sh AUTHORS +.An -nosplit +The +.Nm +driver was written by +.An Mellanox Technologies <freebsd-drivers@mellanox.com> . Modified: stable/10/sys/conf/files ============================================================================== --- stable/10/sys/conf/files Wed Sep 6 15:07:53 2017 (r323222) +++ stable/10/sys/conf/files Wed Sep 6 15:33:23 2017 (r323223) @@ -3933,6 +3933,27 @@ ofed/drivers/net/mlx4/en_tx.c optional mlxen \ no-depend obj-prefix "mlx4_" \ compile-with "${OFED_C_NOIMP} -I$S/ofed/drivers/net/mlx4/" +dev/mlx5/mlx5_ib/mlx5_ib_ah.c optional mlx5ib pci ofed \ + compile-with "${OFED_C}" +dev/mlx5/mlx5_ib/mlx5_ib_cq.c optional mlx5ib pci ofed \ + compile-with "${OFED_C}" +dev/mlx5/mlx5_ib/mlx5_ib_doorbell.c optional mlx5ib pci ofed \ + compile-with "${OFED_C}" +dev/mlx5/mlx5_ib/mlx5_ib_mad.c optional mlx5ib pci ofed \ + compile-with "${OFED_C}" +dev/mlx5/mlx5_ib/mlx5_ib_main.c optional mlx5ib pci ofed \ + compile-with "${OFED_C}" +dev/mlx5/mlx5_ib/mlx5_ib_mem.c optional mlx5ib pci ofed \ + compile-with "${OFED_C}" +dev/mlx5/mlx5_ib/mlx5_ib_mr.c optional mlx5ib pci ofed \ + compile-with "${OFED_C}" +dev/mlx5/mlx5_ib/mlx5_ib_qp.c optional mlx5ib pci ofed \ + compile-with "${OFED_C}" +dev/mlx5/mlx5_ib/mlx5_ib_roce.c optional mlx5ib pci ofed \ + compile-with "${OFED_C}" +dev/mlx5/mlx5_ib/mlx5_ib_srq.c optional mlx5ib pci ofed \ + compile-with "${OFED_C}" + dev/mlx5/mlx5_core/mlx5_alloc.c optional mlx5 pci \ compile-with "${OFED_C}" dev/mlx5/mlx5_core/mlx5_cmd.c optional mlx5 pci \ Modified: stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib.h ============================================================================== --- head/sys/dev/mlx5/mlx5_ib/mlx5_ib.h Wed Aug 23 12:09:37 2017 (r322810) +++ stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib.h Wed Sep 6 15:33:23 2017 (r323223) @@ -653,6 +653,7 @@ void mlx5_ib_free_srq_wqe(struct mlx5_ib_srq *srq, int int mlx5_MAD_IFC(struct mlx5_ib_dev *dev, int ignore_mkey, int ignore_bkey, u8 port, struct ib_wc *in_wc, struct ib_grh *in_grh, void *in_mad, void *response_mad); +int mlx5_ib_resolve_grh(const struct ib_ah_attr *ah_attr, u8 *mac, int *is_mcast); struct ib_ah *create_ib_ah(struct mlx5_ib_dev *dev, struct ib_ah_attr *ah_attr, struct mlx5_ib_ah *ah, enum rdma_link_layer ll); struct ib_ah *mlx5_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr); @@ -681,13 +682,13 @@ int mlx5_ib_post_recv(struct ib_qp *ibqp, struct ib_re struct ib_recv_wr **bad_wr); void *mlx5_get_send_wqe(struct mlx5_ib_qp *qp, int n); struct ib_cq *mlx5_ib_create_cq(struct ib_device *ibdev, - struct ib_cq_init_attr *attr, + int entries, int vector, struct ib_ucontext *context, struct ib_udata *udata); int mlx5_ib_destroy_cq(struct ib_cq *cq); int mlx5_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc); int mlx5_ib_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags); -int mlx5_ib_modify_cq(struct ib_cq *cq, struct ib_cq_attr *attr, int cq_attr_mask); +int mlx5_ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period); int mlx5_ib_resize_cq(struct ib_cq *ibcq, int entries, struct ib_udata *udata); struct ib_mr *mlx5_ib_get_dma_mr(struct ib_pd *pd, int acc); struct ib_mr *mlx5_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length, Modified: stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_ah.c ============================================================================== --- head/sys/dev/mlx5/mlx5_ib/mlx5_ib_ah.c Wed Aug 23 12:09:37 2017 (r322810) +++ stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_ah.c Wed Sep 6 15:33:23 2017 (r323223) @@ -29,6 +29,27 @@ #define IPV6_DEFAULT_HOPLIMIT 64 +int mlx5_ib_resolve_grh(const struct ib_ah_attr *ah_attr, u8 *mac, int *is_mcast) +{ + struct in6_addr in6; + + if (is_mcast != NULL) + *is_mcast = 0; + + memcpy(&in6, ah_attr->grh.dgid.raw, sizeof in6); + if (rdma_link_local_addr(&in6)) { + rdma_get_ll_mac(&in6, mac); + } else if (rdma_is_multicast_addr(&in6)) { + rdma_get_mcast_mac(&in6, mac); + if (is_mcast != NULL) + *is_mcast = 1; + } else { + return -EINVAL; + } + return 0; +} + + struct ib_ah *create_ib_ah(struct mlx5_ib_dev *dev, struct ib_ah_attr *ah_attr, struct mlx5_ib_ah *ah, enum rdma_link_layer ll) @@ -54,7 +75,7 @@ struct ib_ah *create_ib_ah(struct mlx5_ib_dev *dev, if (err) return ERR_PTR(err); - memcpy(ah->av.rmac, ah_attr->dmac, sizeof(ah_attr->dmac)); + mlx5_ib_resolve_grh(ah_attr, ah->av.rmac, NULL); ah->av.udp_sport = mlx5_get_roce_udp_sport( dev, ah_attr->port_num, Modified: stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_cq.c ============================================================================== --- head/sys/dev/mlx5/mlx5_ib/mlx5_ib_cq.c Wed Aug 23 12:09:37 2017 (r322810) +++ stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_cq.c Wed Sep 6 15:33:23 2017 (r323223) @@ -25,6 +25,7 @@ * $FreeBSD$ */ +#include <linux/compiler.h> #include <linux/kref.h> #include <rdma/ib_umem.h> #include <rdma/ib_user_verbs.h> @@ -822,15 +823,13 @@ static void destroy_cq_kernel(struct mlx5_ib_dev *dev, } struct ib_cq *mlx5_ib_create_cq(struct ib_device *ibdev, - struct ib_cq_init_attr *attr, + int entries, int vector, struct ib_ucontext *context, struct ib_udata *udata) { struct mlx5_create_cq_mbox_in *cqb = NULL; struct mlx5_ib_dev *dev = to_mdev(ibdev); struct mlx5_ib_cq *cq; - int entries = attr->cqe; - int vector = attr->comp_vector; int uninitialized_var(index); int uninitialized_var(inlen); int cqe_size; @@ -1006,13 +1005,11 @@ void mlx5_ib_cq_clean(struct mlx5_ib_cq *cq, u32 qpn, spin_unlock_irq(&cq->lock); } -int mlx5_ib_modify_cq(struct ib_cq *cq, struct ib_cq_attr *attr, int cq_attr_mask) +int mlx5_ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period) { struct mlx5_modify_cq_mbox_in *in; struct mlx5_ib_dev *dev = to_mdev(cq->device); struct mlx5_ib_cq *mcq = to_mcq(cq); - u16 cq_count = attr->moderation.cq_count; - u16 cq_period = attr->moderation.cq_period; int err; u32 fsel = 0; @@ -1022,7 +1019,7 @@ int mlx5_ib_modify_cq(struct ib_cq *cq, struct ib_cq_a return -ENOMEM; in->cqn = cpu_to_be32(mcq->mcq.cqn); - if (cq_attr_mask & IB_CQ_MODERATION) { + if (1) { if (MLX5_CAP_GEN(dev->mdev, cq_moderation)) { fsel |= (MLX5_CQ_MODIFY_PERIOD | MLX5_CQ_MODIFY_COUNT); if (cq_period & 0xf000) { Modified: stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_doorbell.c ============================================================================== --- head/sys/dev/mlx5/mlx5_ib/mlx5_ib_doorbell.c Wed Aug 23 12:09:37 2017 (r322810) +++ stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_doorbell.c Wed Sep 6 15:33:23 2017 (r323223) @@ -25,6 +25,7 @@ * $FreeBSD$ */ +#include <linux/compiler.h> #include <linux/kref.h> #include <linux/slab.h> #include <rdma/ib_umem.h> @@ -42,6 +43,7 @@ int mlx5_ib_db_map_user(struct mlx5_ib_ucontext *conte struct mlx5_db *db) { struct mlx5_ib_user_db_page *page; + struct ib_umem_chunk *chunk; int err = 0; mutex_lock(&context->db_page_mutex); @@ -69,7 +71,9 @@ int mlx5_ib_db_map_user(struct mlx5_ib_ucontext *conte list_add(&page->list, &context->db_page_list); found: - db->dma = sg_dma_address(page->umem->sg_head.sgl) + (virt & ~PAGE_MASK); + chunk = list_entry(page->umem->chunk_list.next, + struct ib_umem_chunk, list); + db->dma = sg_dma_address(chunk->page_list) + (virt & ~PAGE_MASK); db->u.user_page = page; ++page->refcnt; Modified: stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_mad.c ============================================================================== --- head/sys/dev/mlx5/mlx5_ib/mlx5_ib_mad.c Wed Aug 23 12:09:37 2017 (r322810) +++ stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_mad.c Wed Sep 6 15:33:23 2017 (r323223) @@ -31,6 +31,25 @@ #include "mlx5_ib.h" #include <dev/mlx5/vport.h> +#define MAX_U32 0xffffffffULL +#define MAX_U16 0xffffUL + +/* Counters should be saturate once they reach their maximum value */ +#define ASSIGN_32BIT_COUNTER(counter, value) do { \ + if ((value) > MAX_U32) \ + counter = cpu_to_be32(MAX_U32); \ + else \ + counter = cpu_to_be32(value); \ +} while (0) + +/* Counters should be saturate once they reach their maximum value */ +#define ASSIGN_16BIT_COUNTER(counter, value) do { \ + if ((value) > MAX_U16) \ + counter = cpu_to_be16(MAX_U16); \ + else \ + counter = cpu_to_be16(value); \ +} while (0) + enum { MLX5_IB_VENDOR_CLASS1 = 0x9, MLX5_IB_VENDOR_CLASS2 = 0xa Modified: stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c ============================================================================== --- head/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c Wed Aug 23 12:09:37 2017 (r322810) +++ stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c Wed Sep 6 15:33:23 2017 (r323223) @@ -33,6 +33,7 @@ #include <linux/sched.h> #include <linux/netdevice.h> #include <linux/etherdevice.h> +#include <net/ipv6.h> #include <linux/list.h> #include <dev/mlx5/driver.h> #include <dev/mlx5/vport.h> @@ -47,16 +48,17 @@ #include "mlx5_ib.h" #include <sys/unistd.h> -#include <sys/kthread.h> #define DRIVER_NAME "mlx5_ib" #define DRIVER_VERSION "3.2-rc1" #define DRIVER_RELDATE "May 2016" +#undef MODULE_VERSION +#include <sys/module.h> + MODULE_AUTHOR("Eli Cohen <eli@mellanox.com>"); MODULE_DESCRIPTION("Mellanox Connect-IB HCA IB driver"); MODULE_LICENSE("Dual BSD/GPL"); -MODULE_DEPEND(mlx5ib, linuxkpi, 1, 1, 1); MODULE_DEPEND(mlx5ib, mlx5, 1, 1, 1); MODULE_DEPEND(mlx5ib, ibcore, 1, 1, 1); MODULE_VERSION(mlx5ib, 1); @@ -856,7 +858,7 @@ static struct ib_ucontext *mlx5_ib_alloc_ucontext(stru else if (reqlen == sizeof(struct mlx5_ib_alloc_ucontext_req_v2)) ver = 2; else { - mlx5_ib_err(dev, "request malformed, reqlen: %ld\n", reqlen); + mlx5_ib_err(dev, "request malformed, reqlen: %ld\n", (long)reqlen); return ERR_PTR(-EINVAL); } @@ -1573,7 +1575,6 @@ static int create_dev_resources(struct mlx5_ib_resourc struct ib_srq_init_attr attr; struct mlx5_ib_dev *dev; int ret = 0; - struct ib_cq_init_attr cq_attr = { .cqe = 1 }; dev = container_of(devr, struct mlx5_ib_dev, devr); @@ -1586,7 +1587,7 @@ static int create_dev_resources(struct mlx5_ib_resourc devr->p0->uobject = NULL; atomic_set(&devr->p0->usecnt, 0); - devr->c0 = mlx5_ib_create_cq(&dev->ib_dev, &cq_attr, NULL, NULL); + devr->c0 = mlx5_ib_create_cq(&dev->ib_dev, 1, 0, NULL, NULL); if (IS_ERR(devr->c0)) { ret = PTR_ERR(devr->c0); goto error1; @@ -2098,7 +2099,6 @@ static void *mlx5_ib_add(struct mlx5_core_dev *mdev) dev->ib_dev.query_device = mlx5_ib_query_device; dev->ib_dev.query_port = mlx5_ib_query_port; dev->ib_dev.get_link_layer = mlx5_ib_port_link_layer; - dev->ib_dev.get_netdev = mlx5_ib_get_netdev; dev->ib_dev.query_gid = mlx5_ib_query_gid; dev->ib_dev.query_pkey = mlx5_ib_query_pkey; dev->ib_dev.modify_device = mlx5_ib_modify_device; Modified: stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_mem.c ============================================================================== --- head/sys/dev/mlx5/mlx5_ib/mlx5_ib_mem.c Wed Aug 23 12:09:37 2017 (r322810) +++ stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_mem.c Wed Sep 6 15:33:23 2017 (r323223) @@ -29,6 +29,8 @@ #include <rdma/ib_umem.h> #include "mlx5_ib.h" +CTASSERT(sizeof(uintptr_t) == sizeof(unsigned long)); + /* @umem: umem object to scan * @addr: ib virtual address requested by the user * @count: number of PAGE_SIZE pages covered by umem @@ -40,9 +42,10 @@ void mlx5_ib_cont_pages(struct ib_umem *umem, u64 addr, int *count, int *shift, int *ncont, int *order) { - uintptr_t tmp; + struct ib_umem_chunk *chunk; + unsigned long tmp; unsigned long m; - int i, k; + int i, j, k; u64 base = 0; int p = 0; int skip; @@ -50,7 +53,6 @@ void mlx5_ib_cont_pages(struct ib_umem *umem, u64 addr u64 len; u64 pfn; struct scatterlist *sg; - int entry; unsigned long page_shift = ilog2(umem->page_size); addr = addr >> page_shift; @@ -60,7 +62,9 @@ void mlx5_ib_cont_pages(struct ib_umem *umem, u64 addr mask = skip - 1; i = 0; - for_each_sg(umem->sg_head.sgl, sg, umem->nmap, entry) { + list_for_each_entry(chunk, &umem->chunk_list, list) { + for (j = 0; j < chunk->nmap; j++) { + sg = chunk->page_list + j; len = sg_dma_len(sg) >> page_shift; pfn = sg_dma_address(sg) >> page_shift; for (k = 0; k < len; k++) { @@ -85,6 +89,7 @@ void mlx5_ib_cont_pages(struct ib_umem *umem, u64 addr p++; i++; } + } } if (i) { @@ -124,17 +129,19 @@ static void __mlx5_ib_populate_pas(struct mlx5_ib_dev __be64 *pas, int access_flags) { unsigned long umem_page_shift = ilog2(umem->page_size); + struct ib_umem_chunk *chunk; int shift = page_shift - umem_page_shift; int mask = (1 << shift) - 1; - int i, k; + int i, j, k; u64 cur = 0; u64 base; int len; struct scatterlist *sg; - int entry; i = 0; - for_each_sg(umem->sg_head.sgl, sg, umem->nmap, entry) { + list_for_each_entry(chunk, &umem->chunk_list, list) { + for (j = 0; j < chunk->nmap; j++) { + sg = chunk->page_list + j; len = sg_dma_len(sg) >> umem_page_shift; base = sg_dma_address(sg); for (k = 0; k < len; k++) { @@ -152,6 +159,7 @@ static void __mlx5_ib_populate_pas(struct mlx5_ib_dev (base + (k << umem_page_shift))); i++; } + } } } Modified: stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_mr.c ============================================================================== --- head/sys/dev/mlx5/mlx5_ib/mlx5_ib_mr.c Wed Aug 23 12:09:37 2017 (r322810) +++ stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_mr.c Wed Sep 6 15:33:23 2017 (r323223) @@ -25,7 +25,7 @@ * $FreeBSD$ */ - +#include <linux/compiler.h> #include <linux/kref.h> #include <linux/random.h> #include <linux/fs.h> Modified: stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_qp.c ============================================================================== --- head/sys/dev/mlx5/mlx5_ib/mlx5_ib_qp.c Wed Aug 23 12:09:37 2017 (r322810) +++ stable/10/sys/dev/mlx5/mlx5_ib/mlx5_ib_qp.c Wed Sep 6 15:33:23 2017 (r323223) @@ -1591,7 +1591,9 @@ static int mlx5_set_path(struct mlx5_ib_dev *dev, cons &gid_type); if (err) return err; - memcpy(path->rmac, ah->dmac, sizeof(ah->dmac)); + err = mlx5_ib_resolve_grh(ah, path->rmac, NULL); + if (err) + return err; path->udp_sport = mlx5_get_roce_udp_sport(dev, port, ah->grh.sgid_index, 0); @@ -2079,20 +2081,14 @@ int mlx5_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp enum ib_qp_state cur_state, new_state; int err = -EINVAL; int port; - enum rdma_link_layer ll = IB_LINK_LAYER_UNSPECIFIED; mutex_lock(&qp->mutex); cur_state = attr_mask & IB_QP_CUR_STATE ? attr->cur_qp_state : qp->state; new_state = attr_mask & IB_QP_STATE ? attr->qp_state : cur_state; - if (!(cur_state == new_state && cur_state == IB_QPS_RESET)) { - port = attr_mask & IB_QP_PORT ? attr->port_num : qp->port; - ll = dev->ib_dev.get_link_layer(&dev->ib_dev, port); - } - if (!ignored_ts_check(ibqp->qp_type) && - !ib_modify_qp_is_ok(cur_state, new_state, ibqp->qp_type, attr_mask, ll)) + !ib_modify_qp_is_ok(cur_state, new_state, ibqp->qp_type, attr_mask)) goto out; if ((attr_mask & IB_QP_PORT) && Modified: stable/10/sys/modules/Makefile ============================================================================== --- stable/10/sys/modules/Makefile Wed Sep 6 15:07:53 2017 (r323222) +++ stable/10/sys/modules/Makefile Wed Sep 6 15:33:23 2017 (r323223) @@ -225,6 +225,7 @@ SUBDIR= \ ${_mlxen} \ ${_mlx5} \ ${_mlx5en} \ + ${_mlx5ib} \ ${_mly} \ mmc \ mmcsd \ @@ -762,6 +763,7 @@ _mly= mly .if ${MK_OFED} != "no" || defined(ALL_MODULES) _mlx4= mlx4 _mlx4ib= mlx4ib +_mlx5ib= mlx5ib _mlxen= mlxen _mthca= mthca .endif
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201709061533.v86FXNRc015857>