From owner-freebsd-drivers@freebsd.org Tue Nov 27 11:08:11 2018 Return-Path: Delivered-To: freebsd-drivers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 13BBA1145D66; Tue, 27 Nov 2018 11:08:11 +0000 (UTC) (envelope-from rajfbsd@gmail.com) Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5757671AA0; Tue, 27 Nov 2018 11:08:10 +0000 (UTC) (envelope-from rajfbsd@gmail.com) Received: by mail-wm1-x343.google.com with SMTP id c126so21650815wmh.0; Tue, 27 Nov 2018 03:08:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=pTOI4uxkm8PbZSHbiFzvlHtN2oBUtqapNAPLpy1x6Vs=; b=m/mx6Npds/kzR/waYIUYPvTOQZTXqOTio8YKSC6G0ytyV3dKMlnrFcyku2wgbuOryo MHbIu0Z/IUWNOL8z3kVyoaIakWXOrgia5I99D4wkIuOoAPuco42rI43MkZ9IWc0/r8Cu oKSLlFvLgdZ1LvLXre6786iohSO05ijRy6BB9C2URx0Jl/QV/Omf4tp7Z+Lu+CS7Z4/e aO+q2AY3e1W0G96MIIDCtBz6uQXoLOMQalkamTpQQCrKBfyCggABkm/kKAioxZKIDO8O fmh/qd+Fk+e2IRS5/NsKXfMh3ocqBeXtsenBIrHrrVDEGeyEhGlQbSXNEX5WMnDTdNwc 9qvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=pTOI4uxkm8PbZSHbiFzvlHtN2oBUtqapNAPLpy1x6Vs=; b=gLpSTiG+HbZe9aVCGUFCkh87kewaSqHXO9pWqzVr0kvNrX7Afkh99F6a24g6VWMJ8K lE+bhgjOZHOaEminVkaPuZ2DihCRJhfproCA2DnJ2HfS0hn59HeOIzQJJjUE7CzSjioL 5dye2Jc7uwFvaTiCQHPPcZMrERAZG6bPfXZ0cDwksDovomezXEn0oxB/9TP7EcqSY/td nJdRgMqu5MavR3JGgXeeTv3UskJd1YBlC37abJdLUpY1VGK5pUma0iW0MGMSES4NMcfx Lx/PhRBVYVPIuZRGbIxgIdjqENQLJmItk88GmmmoCkhW9dT6tqKAyrVhSI8jyzaIQFYx PsSg== X-Gm-Message-State: AA+aEWa200UsX4VIn4UZTb0zRQCsqiqKa/ZBF44kb5dOCGnvaGVljG7/ W7aaib1gO3Ll0rVAhN13Jf5blqfwB+XR59mQs0b8Nbkk X-Google-Smtp-Source: AFSGD/UCTe/WcuND6T3UCTxmOOLUov3nW2rxakJxJgo/yXADtsk3SMMTy7j8nnPo+bcc5NUdyOOEvm8GjQPZFulqYUQ= X-Received: by 2002:a1c:5702:: with SMTP id l2mr25051423wmb.31.1543316888911; Tue, 27 Nov 2018 03:08:08 -0800 (PST) MIME-Version: 1.0 From: Rajesh Kumar Date: Tue, 27 Nov 2018 16:37:57 +0530 Message-ID: Subject: bus_dmamem_alloc doesn't follow alignment specified To: freebsd-drivers@freebsd.org, freebsd-hackers@freebsd.org X-Rspamd-Queue-Id: 5757671AA0 X-Spamd-Result: default: False [-2.86 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.983,0]; R_DKIM_ALLOW(-0.20)[gmail.com]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-0.99)[-0.992,0]; TO_MATCH_ENVRCPT_ALL(0.00)[]; IP_SCORE(-0.07)[ip: (2.72), ipnet: 2a00:1450::/32(-1.56), asn: 15169(-1.40), country: US(-0.09)]; NEURAL_SPAM_SHORT(0.19)[0.190,0]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[3.4.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Server: mx1.freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Nov 2018 11:08:11 -0000 Hi, I am writing a PCI-E driver, where I am trying to allocate DMA buffers. But seems, like the alignment requirements are not satisfied by bus_dmamem_alloc function. I am using stable/11 branch. *Code :* dma_setup(size, align, alimit) { printf ("size 0x%x align 0x%x alimit 0x%x\n", size, align, alimit); if (bus_dma_tag_create(parent, align, 0, alimit, BUS_SPACE_MAXADDR, NULL, NULL, size, 1, size, 0, NULL, NULL, &dma_tag)) { return (ENOMEM); } if (bus_dmamem_alloc(dma_tag, &virt_addr, BUS_DMA_WAITOK | BUS_DMA_ZERO, &dma_map)) { bus_dma_tag_destroy(dma_tag); return (ENOMEM); } if (bus_dmamap_load(dma_tag, dma_map, virt_addr, size, dma_cbfn, &arg, BUS_DMA_NOWAIT)) { bus_dmamem_free(dma_tag, virt_addr, dma_map); bus_dma_tag_destroy(mw->dma_tag); return (ENOMEM); } printf("dma_addr 0x%lx virt_addr 0x%lx\n", (uint64_t)arg->addr, (uint64_t)virt_addr); } dma_cbfn(void *tmp, bus_dma_segment_t *segs, int nsegs, int error) { struct dma_args *arg = (struct dma_args *)tmp; arg->addr = segs[0].ds_addr; } *Logs: * size 0x100000 align 0x100000 alimit 0xffffffff dma_addr *0x78e00000* virt_addr *0xfffffe1c3665f000* As seen above, dma_addr is as per alignment requirement, but virtual address from bus_dmamem_alloc doesn't seem to satisfy the alignment requirement). My understanding is both the DMA addr and the virtual address should be as per the alignment specified. Can anyone please clarify? Thanks, Rajesh. From owner-freebsd-drivers@freebsd.org Tue Nov 27 12:24:57 2018 Return-Path: Delivered-To: freebsd-drivers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5196811496E9; Tue, 27 Nov 2018 12:24:57 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 583B676619; Tue, 27 Nov 2018 12:24:56 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id wARCOjQ0046572 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 27 Nov 2018 14:24:48 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua wARCOjQ0046572 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id wARCOiMH046571; Tue, 27 Nov 2018 14:24:44 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 27 Nov 2018 14:24:44 +0200 From: Konstantin Belousov To: Rajesh Kumar Cc: freebsd-drivers@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: bus_dmamem_alloc doesn't follow alignment specified Message-ID: <20181127122444.GB2378@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-Rspamd-Queue-Id: 583B676619 X-Spamd-Result: default: False [-5.12 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[gmail.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.97)[-0.967,0]; IP_SCORE(-2.14)[ip: (-2.62), ipnet: 2001:470::/32(-4.50), asn: 6939(-3.49), country: US(-0.09)]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; RCVD_TLS_LAST(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Nov 2018 12:24:57 -0000 On Tue, Nov 27, 2018 at 04:37:57PM +0530, Rajesh Kumar wrote: > Hi, > > I am writing a PCI-E driver, where I am trying to allocate DMA buffers. But > seems, like the alignment requirements are not satisfied by > bus_dmamem_alloc function. I am using stable/11 branch. > > *Code :* > dma_setup(size, align, alimit) { > > printf ("size 0x%x align 0x%x alimit 0x%x\n", size, align, alimit); > > if (bus_dma_tag_create(parent, align, 0, > alimit, BUS_SPACE_MAXADDR, > NULL, NULL, size, 1, size, > 0, NULL, NULL, &dma_tag)) { > return (ENOMEM); > } > > if (bus_dmamem_alloc(dma_tag, &virt_addr, > BUS_DMA_WAITOK | BUS_DMA_ZERO, &dma_map)) { > bus_dma_tag_destroy(dma_tag); > return (ENOMEM); > } > > if (bus_dmamap_load(dma_tag, dma_map, virt_addr, > size, dma_cbfn, &arg, BUS_DMA_NOWAIT)) { > bus_dmamem_free(dma_tag, virt_addr, dma_map); > bus_dma_tag_destroy(mw->dma_tag); > return (ENOMEM); > } > > printf("dma_addr 0x%lx virt_addr 0x%lx\n", (uint64_t)arg->addr, > (uint64_t)virt_addr); > > } > > dma_cbfn(void *tmp, bus_dma_segment_t *segs, int nsegs, int error) > { > struct dma_args *arg = (struct dma_args *)tmp; > arg->addr = segs[0].ds_addr; > } > > *Logs: * > size 0x100000 align 0x100000 alimit 0xffffffff > dma_addr *0x78e00000* virt_addr *0xfffffe1c3665f000* > > As seen above, dma_addr is as per alignment requirement, but virtual > address from bus_dmamem_alloc doesn't seem to satisfy the alignment > requirement). > > My understanding is both the DMA addr and the virtual address should be as > per the alignment specified. Can anyone please clarify? Only bus address is aligned. For single-segment i.e. contiguous allocations, amd64 could benefit from using dmap VA instead of kmem_alloc. This would be only an optimization, not done right now.