From owner-freebsd-hackers@freebsd.org Fri May 11 20:11:24 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0CBDDFC9F5C for ; Fri, 11 May 2018 20:11:24 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x22e.google.com (mail-io0-x22e.google.com [IPv6:2607:f8b0:4001:c06::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 845B2842DE for ; Fri, 11 May 2018 20:11:23 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x22e.google.com with SMTP id d11-v6so8355708iof.11 for ; Fri, 11 May 2018 13:11:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=U8no60hfNY+daL4DSJBxkV3lr8BfXzTHbm4NBjiMHkk=; b=uE3cWzegEDj6jiuI9FQP0ehglCtTFb1vWxfXRKdhMEJ/AszI5cR/SNV63RzYQaHXNH wL+H+Smlo22f2O2LvDbhqNsGPX7Y22b/4w6eVBJz+yQ7P+pIs0+TxsJi+Exk7zFgxh9Z vYPqKdTDd5ASGKjM1RHPJMtzu0tW9x3WxiuQvfvuHkISed9X06lwuRIPOUBSk2b+uDP5 a4q0h4VbPNRtbR/Rq+iPtvNxf8REA7MM7AEqhz+nuQgRWGWldmPlz0upx+URkUvzRwKi sTIfwxHuKLGSg+XLkqMkIWQSJ+/Mm1eKEUJ2g/IvQ3zFb/E1VNyqOcAI3v04nrztaEZp PzkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=U8no60hfNY+daL4DSJBxkV3lr8BfXzTHbm4NBjiMHkk=; b=TtWFvghaWMIcaZ3MFu9HKmylK3a+EUUVvEgCeqi6uHukBbKIXKIUJoOUmuRPTipAyD EnHNbasZp2V+cLTRmcYOSdKUfHsekptHNT2yZZwiusrowTW8wMq/lZ+alCVkqYBJk5+d /PRuUem9qemCRmdkVH44Vu1hNaBykBD9jVrJcT/GGsI59WKGMIUkrO2FC2kCKAkovI7E ny9mJ8853nUrDd12drNExddGdIoNuBABqNVdVN1Kmk/NVOyBRV/iuK94Ceox5HUsdjH8 vXE3pb/UNbo0NLYPZ6acfYCPsP/UsY57jjtQm0cC3oLPX0DLEZgZo42+jQH/D2jRyL2b P2+Q== X-Gm-Message-State: ALKqPwcLS+4K/1qKMo1640VXXZQ046Xz7r/+kX1LXG/otrS5xsNRx6+D +J60Kv41/VvzhtnVQ8ssOuALucRC/3w0YFHh8naAvA== X-Google-Smtp-Source: AB8JxZrjYzQ1dKVAHx3CmIUGhHF2zKnKwgpkUk8xeWTDL49/11hsK9J7vA1npMM6Xo9N9ykpJcC2UZ168bmbZSSkEh8= X-Received: by 2002:a6b:12a3:: with SMTP id 35-v6mr7424683ios.168.1526069482761; Fri, 11 May 2018 13:11:22 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 2002:a4f:a649:0:0:0:0:0 with HTTP; Fri, 11 May 2018 13:11:22 -0700 (PDT) X-Originating-IP: [2603:300b:6:5100:1052:acc7:f9de:2b6d] In-Reply-To: References: From: Warner Losh Date: Fri, 11 May 2018 14:11:22 -0600 X-Google-Sender-Auth: SIwOxazy3NP0l9wFHuzCOWHBDvU Message-ID: Subject: Re: PCIe multipliers, how do they work? To: Wojciech Puchar Cc: Dieter BSD , "freebsd-hackers@freebsd.org" , freebsd-hardware@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 May 2018 20:11:24 -0000 On Fri, May 11, 2018 at 1:03 PM, Wojciech Puchar wrote: > There are also PCIe cards which provide multiple slots, typically >> connected with a usb cable. These tend to convert 1 PCIe_x1 slot >> into multiple PCIe_x1 slots. I get the impression that these do >> not require bifurcation support. They seem to be aimed at "miners" >> > > these cards consist of PCIe switch which is supported out of the box. > former ones require BIOS to reconfigure CPUs PCI lanes so instead of eg > one 16x lane there will be 4 4x lanes. > Usually they require a driver for management functions, like if you want to turn off one of the ports (because you know there's a bad card in it, for example). But for normal probe / attach they are usually drop in. Some mobos can require some BIOS adjustment, though to properly setup lane bifurcation and such... The ones that are true switches, and not just signal retimers, usually don't: they take the full 8 or 16 lanes and multiplex it amongst the 16-32 downstream lanes they typically provide. We're looking at one that does x4 lanes and expands to 4 cards at x4 lanes. The devices on the other side fill just over a lane each, but we'd run out of lanes if we did x2 fan-out. This the cards we're looking at, we can get x4 combined rates form the 4 cards that on their own are kinda crappy. Warner