From owner-freebsd-hardware@freebsd.org Fri May 11 02:04:38 2018 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0A4F2FAFDF3; Fri, 11 May 2018 02:04:38 +0000 (UTC) (envelope-from dieterbsd@gmail.com) Received: from mail-it0-x242.google.com (mail-it0-x242.google.com [IPv6:2607:f8b0:4001:c0b::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 994368703E; Fri, 11 May 2018 02:04:37 +0000 (UTC) (envelope-from dieterbsd@gmail.com) Received: by mail-it0-x242.google.com with SMTP id n202-v6so371264ita.1; Thu, 10 May 2018 19:04:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=/uJFpaEp2Oq1CBUINjp9tKROg/rZgE1i304fj5C5rtI=; b=Jj6WsTot233bJglP7FtA6mmkksUg9yX994QUvA6nFFPHN/TLlRkEm7R1oj9xPCW3BF P3aQdu52NTnoqIKQ717K535+iKQb3QrGup3qsUsdJ2zesdz5dueIeNfTuiJMU9Tavzb5 YuFOxFrrM1cFv97CyQiNj9P5vRkcS2tT91zLzo045w7ULlL9KhW6/RQKr6sjAiGxAgNm Sppj8etJKxfcbM2y5Lyt3oADUe17UaOF7OvdrgTZIK9as1qE22fgRdMD021gwzCaKNWu cJNVvZaBgXBPhvI549Vxa2KwP49XWiQ7VYHmPtz8bensdtPURIdH6hJSbsXzTy3AVWfe Lseg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=/uJFpaEp2Oq1CBUINjp9tKROg/rZgE1i304fj5C5rtI=; b=O+oGqSLNoEkIxXc3tQw9w27jCiSeHbkqsULU95Bz08aj/lUKhdYrAHeOWeBM1n9BUw wYWSJQxSuLSpos0MNTZ7ywDiHIsPOKyKhGqgYdoDRLJfIHtkltMkGKJBkNldVTyx5jbM Hs4iVnhYg3YbZCf3gMq8cKH8sGmYZgOfRRtdb9l/MQTZxQH43SC6Q8fd/iJoTZyvnszC e+TvsrxlwT3ymYo9KJhYl0yztl47yp2RH+9Lx9UXG7fZizcJva36Oe31or3NQyaBn5dH W+WpDPkrKIhzrArp3YJmAMMonHcPzzXt041YKUppOoJGvG5gB72lXFHsv80dF47rFgwz GSfQ== X-Gm-Message-State: ALKqPweBZYG2/zVXfYRc2FyQzM5WQUsdEE8c9BtASaGiaJePDfuComxC lKfZi1Ek8QfjJvBzAEtEclWt3iYk84nBscPSdyc= X-Google-Smtp-Source: AB8JxZqZM3Xv5cazBaCVh2DJ81oWObgqLxZ+rm51z6rqmvSqHxL/U1kFdB27kB8mCk5e/xiL3VhZyyMr2ucnKzzNHIw= X-Received: by 2002:a24:4c55:: with SMTP id a82-v6mr1336086itb.1.1526004277018; Thu, 10 May 2018 19:04:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.192.132.228 with HTTP; Thu, 10 May 2018 19:04:36 -0700 (PDT) From: Dieter BSD Date: Thu, 10 May 2018 19:04:36 -0700 Message-ID: Subject: PCIe multipliers, how do they work? To: freebsd-hardware@freebsd.org Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 May 2018 02:04:38 -0000 Looking into ways to get additional expansion slots, since board designers cannot count past 7 and often not even that high. :-( There are PCIe riser cards that split a wide slot into 2 or more narrower slots, for example 1 x8 slot becomes 2 x4 slots. These would be very useful, except it appears that they require "bifurcation" support in the mainboard's firmware. Which most boards do not provide. And most boards are not supported by FLOSS firmware, so adding bifurcation support would be rather difficult. There are also PCIe cards which provide multiple slots, typically connected with a usb cable. These tend to convert 1 PCIe_x1 slot into multiple PCIe_x1 slots. I get the impression that these do not require bifurcation support. They seem to be aimed at "miners" for attaching multiple gpu cards. I'm not interested in mining or in attaching multiple gpu cards. I'm interesting in adding additional sata cards, Ethernet cards, and such. Unlike the bifurcation type riser splitter cards, which seem to conserve PCIe lanes, these are more like a sata port multiplier, with the same type of bandwidth limitation. I'm wondering how these things work. The wikipedia PCIe page [1] says: "PCI Express switches can create multiple endpoints out of one endpoint to allow sharing one endpoint with multiple devices." So maybe they use a PCIe switch? Poking around wikipedia and google has thus far uncovered very little info about PCIe switches. Wikipedia is less helpful than usual, and they keep making google less and less useful for no apparent reason. I don't see any other obvious keywords to google for. It isn't obvious how slot id/address is handled. How do commands and data get routed to/from the correct card? Is any firmware or OS support required? Is there some other solution that I haven't stumbled across? I'd really like to split an x8 slot into 4 x2 slots, which doesn't seem to be an off-the-shelf option either way. [1] en.wikipedia.org/wiki/PCI_Express