From owner-svn-src-head@FreeBSD.ORG Sun Aug 12 17:53:07 2012 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B0309106566B; Sun, 12 Aug 2012 17:53:07 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 97EEF8FC15; Sun, 12 Aug 2012 17:53:07 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q7CHr7qO017199; Sun, 12 Aug 2012 17:53:07 GMT (envelope-from hselasky@svn.freebsd.org) Received: (from hselasky@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q7CHr7VU017178; Sun, 12 Aug 2012 17:53:07 GMT (envelope-from hselasky@svn.freebsd.org) Message-Id: <201208121753.q7CHr7VU017178@svn.freebsd.org> From: Hans Petter Selasky Date: Sun, 12 Aug 2012 17:53:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r239214 - in head/sys: dev/usb dev/usb/controller sys X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Aug 2012 17:53:07 -0000 Author: hselasky Date: Sun Aug 12 17:53:06 2012 New Revision: 239214 URL: http://svn.freebsd.org/changeset/base/239214 Log: Add support for the so-called streams feature of BULK endpoints in SUPER-speed mode, USB 3.0. This feature has not been tested yet, due to lack of hardware. This feature is useful when implementing protocols like UASP, USB attached SCSI which promises higher USB mass storage throughput. This patch also implements support for hardware processing of endpoints for increased performance. The switching to hardware processing of an endpoint is done via a callback to the USB controller driver. The stream feature is implemented like a variant of a hardware USB protocol. USB controller drivers implementing device mode needs to be updated to implement the new "xfer_stall" USB controller method and remove the "xfer" argument from the "set_stall" method. The API's toward existing USB drivers are preserved. To setup a USB transfer in stream mode, set the "stream_id" field of the USB config structure to the desired value. The maximum number of BULK streams is currently hardcoded and limited to 8 via a define in usb_freebsd.h. All USB drivers should be re-compiled after this change. LibUSB will be updated next week to support streams mode. A new IOCTL to setup BULK streams as already been implemented. The ugen device nodes currently only supports stream ID zero. The FreeBSD version has been bumped. MFC after: 2 weeks Modified: head/sys/dev/usb/controller/at91dci.c head/sys/dev/usb/controller/atmegadci.c head/sys/dev/usb/controller/avr32dci.c head/sys/dev/usb/controller/dwc_otg.c head/sys/dev/usb/controller/musb_otg.c head/sys/dev/usb/controller/uss820dci.c head/sys/dev/usb/controller/xhci.c head/sys/dev/usb/controller/xhci.h head/sys/dev/usb/usb.h head/sys/dev/usb/usb_controller.h head/sys/dev/usb/usb_core.h head/sys/dev/usb/usb_debug.c head/sys/dev/usb/usb_device.c head/sys/dev/usb/usb_freebsd.h head/sys/dev/usb/usb_generic.c head/sys/dev/usb/usb_hub.c head/sys/dev/usb/usb_ioctl.h head/sys/dev/usb/usb_request.c head/sys/dev/usb/usb_transfer.c head/sys/dev/usb/usbdi.h head/sys/sys/param.h Modified: head/sys/dev/usb/controller/at91dci.c ============================================================================== --- head/sys/dev/usb/controller/at91dci.c Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/controller/at91dci.c Sun Aug 12 17:53:06 2012 (r239214) @@ -1226,7 +1226,14 @@ at91dci_device_done(struct usb_xfer *xfe } static void -at91dci_set_stall(struct usb_device *udev, struct usb_xfer *xfer, +at91dci_xfer_stall(struct usb_xfer *xfer) +{ + USB_BUS_LOCK_ASSERT(udev->bus, MA_OWNED); + at91dci_device_done(xfer, USB_ERR_STALLED); +} + +static void +at91dci_set_stall(struct usb_device *udev, struct usb_endpoint *ep, uint8_t *did_stall) { struct at91dci_softc *sc; @@ -1237,10 +1244,6 @@ at91dci_set_stall(struct usb_device *ude DPRINTFN(5, "endpoint=%p\n", ep); - if (xfer) { - /* cancel any ongoing transfers */ - at91dci_device_done(xfer, USB_ERR_STALLED); - } /* set FORCESTALL */ sc = AT9100_DCI_BUS2SC(udev->bus); csr_reg = (ep->edesc->bEndpointAddress & UE_ADDR); @@ -2332,6 +2335,7 @@ struct usb_bus_methods at91dci_bus_metho .xfer_unsetup = &at91dci_xfer_unsetup, .get_hw_ep_profile = &at91dci_get_hw_ep_profile, .set_stall = &at91dci_set_stall, + .xfer_stall = &at91dci_xfer_stall, .clear_stall = &at91dci_clear_stall, .roothub_exec = &at91dci_roothub_exec, .xfer_poll = &at91dci_do_poll, Modified: head/sys/dev/usb/controller/atmegadci.c ============================================================================== --- head/sys/dev/usb/controller/atmegadci.c Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/controller/atmegadci.c Sun Aug 12 17:53:06 2012 (r239214) @@ -1113,7 +1113,13 @@ atmegadci_device_done(struct usb_xfer *x } static void -atmegadci_set_stall(struct usb_device *udev, struct usb_xfer *xfer, +atmegadci_xfer_stall(struct usb_xfer *xfer) +{ + atmegadci_device_done(xfer, USB_ERR_STALLED); +} + +static void +atmegadci_set_stall(struct usb_device *udev, struct usb_endpoint *ep, uint8_t *did_stall) { struct atmegadci_softc *sc; @@ -1123,10 +1129,6 @@ atmegadci_set_stall(struct usb_device *u DPRINTFN(5, "endpoint=%p\n", ep); - if (xfer) { - /* cancel any ongoing transfers */ - atmegadci_device_done(xfer, USB_ERR_STALLED); - } sc = ATMEGA_BUS2SC(udev->bus); /* get endpoint number */ ep_no = (ep->edesc->bEndpointAddress & UE_ADDR); @@ -2151,6 +2153,7 @@ struct usb_bus_methods atmegadci_bus_met .xfer_setup = &atmegadci_xfer_setup, .xfer_unsetup = &atmegadci_xfer_unsetup, .get_hw_ep_profile = &atmegadci_get_hw_ep_profile, + .xfer_stall = &atmegadci_xfer_stall, .set_stall = &atmegadci_set_stall, .clear_stall = &atmegadci_clear_stall, .roothub_exec = &atmegadci_roothub_exec, Modified: head/sys/dev/usb/controller/avr32dci.c ============================================================================== --- head/sys/dev/usb/controller/avr32dci.c Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/controller/avr32dci.c Sun Aug 12 17:53:06 2012 (r239214) @@ -1080,7 +1080,13 @@ avr32dci_device_done(struct usb_xfer *xf } static void -avr32dci_set_stall(struct usb_device *udev, struct usb_xfer *xfer, +avr32dci_xfer_stall(struct usb_xfer *xfer) +{ + avr32dci_device_done(xfer, USB_ERR_STALLED); +} + +static void +avr32dci_set_stall(struct usb_device *udev, struct usb_endpoint *pipe, uint8_t *did_stall) { struct avr32dci_softc *sc; @@ -1090,10 +1096,6 @@ avr32dci_set_stall(struct usb_device *ud DPRINTFN(5, "pipe=%p\n", pipe); - if (xfer) { - /* cancel any ongoing transfers */ - avr32dci_device_done(xfer, USB_ERR_STALLED); - } sc = AVR32_BUS2SC(udev->bus); /* get endpoint number */ ep_no = (pipe->edesc->bEndpointAddress & UE_ADDR); @@ -2096,6 +2098,7 @@ struct usb_bus_methods avr32dci_bus_meth .xfer_setup = &avr32dci_xfer_setup, .xfer_unsetup = &avr32dci_xfer_unsetup, .get_hw_ep_profile = &avr32dci_get_hw_ep_profile, + .xfer_stall = &avr32dci_xfer_stall, .set_stall = &avr32dci_set_stall, .clear_stall = &avr32dci_clear_stall, .roothub_exec = &avr32dci_roothub_exec, Modified: head/sys/dev/usb/controller/dwc_otg.c ============================================================================== --- head/sys/dev/usb/controller/dwc_otg.c Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/controller/dwc_otg.c Sun Aug 12 17:53:06 2012 (r239214) @@ -1524,7 +1524,13 @@ dwc_otg_device_done(struct usb_xfer *xfe } static void -dwc_otg_set_stall(struct usb_device *udev, struct usb_xfer *xfer, +dwc_otg_xfer_stall(struct usb_xfer *xfer) +{ + dwc_otg_device_done(xfer, USB_ERR_STALLED); +} + +static void +dwc_otg_set_stall(struct usb_device *udev, struct usb_endpoint *ep, uint8_t *did_stall) { struct dwc_otg_softc *sc; @@ -1534,10 +1540,6 @@ dwc_otg_set_stall(struct usb_device *ude USB_BUS_LOCK_ASSERT(udev->bus, MA_OWNED); - if (xfer) { - /* cancel any ongoing transfers */ - dwc_otg_device_done(xfer, USB_ERR_STALLED); - } sc = DWC_OTG_BUS2SC(udev->bus); /* get endpoint address */ @@ -2625,6 +2627,7 @@ struct usb_bus_methods dwc_otg_bus_metho .xfer_setup = &dwc_otg_xfer_setup, .xfer_unsetup = &dwc_otg_xfer_unsetup, .get_hw_ep_profile = &dwc_otg_get_hw_ep_profile, + .xfer_stall = &dwc_otg_xfer_stall, .set_stall = &dwc_otg_set_stall, .clear_stall = &dwc_otg_clear_stall, .roothub_exec = &dwc_otg_roothub_exec, Modified: head/sys/dev/usb/controller/musb_otg.c ============================================================================== --- head/sys/dev/usb/controller/musb_otg.c Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/controller/musb_otg.c Sun Aug 12 17:53:06 2012 (r239214) @@ -1472,7 +1472,13 @@ musbotg_device_done(struct usb_xfer *xfe } static void -musbotg_set_stall(struct usb_device *udev, struct usb_xfer *xfer, +musbotg_xfer_stall(struct usb_xfer *xfer) +{ + musbotg_device_done(xfer, USB_ERR_STALLED); +} + +static void +musbotg_set_stall(struct usb_device *udev, struct usb_endpoint *ep, uint8_t *did_stall) { struct musbotg_softc *sc; @@ -1482,10 +1488,6 @@ musbotg_set_stall(struct usb_device *ude DPRINTFN(4, "endpoint=%p\n", ep); - if (xfer) { - /* cancel any ongoing transfers */ - musbotg_device_done(xfer, USB_ERR_STALLED); - } /* set FORCESTALL */ sc = MUSBOTG_BUS2SC(udev->bus); @@ -2801,6 +2803,7 @@ struct usb_bus_methods musbotg_bus_metho .xfer_setup = &musbotg_xfer_setup, .xfer_unsetup = &musbotg_xfer_unsetup, .get_hw_ep_profile = &musbotg_get_hw_ep_profile, + .xfer_stall = &musbotg_xfer_stall, .set_stall = &musbotg_set_stall, .clear_stall = &musbotg_clear_stall, .roothub_exec = &musbotg_roothub_exec, Modified: head/sys/dev/usb/controller/uss820dci.c ============================================================================== --- head/sys/dev/usb/controller/uss820dci.c Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/controller/uss820dci.c Sun Aug 12 17:53:06 2012 (r239214) @@ -1210,7 +1210,13 @@ uss820dci_device_done(struct usb_xfer *x } static void -uss820dci_set_stall(struct usb_device *udev, struct usb_xfer *xfer, +uss820dci_xfer_stall(struct usb_xfer *xfer) +{ + uss820dci_device_done(xfer, USB_ERR_STALLED); +} + +static void +uss820dci_set_stall(struct usb_device *udev, struct usb_endpoint *ep, uint8_t *did_stall) { struct uss820dci_softc *sc; @@ -1223,10 +1229,6 @@ uss820dci_set_stall(struct usb_device *u DPRINTFN(5, "endpoint=%p\n", ep); - if (xfer) { - /* cancel any ongoing transfers */ - uss820dci_device_done(xfer, USB_ERR_STALLED); - } /* set FORCESTALL */ sc = USS820_DCI_BUS2SC(udev->bus); ep_no = (ep->edesc->bEndpointAddress & UE_ADDR); @@ -2385,6 +2387,7 @@ struct usb_bus_methods uss820dci_bus_met .xfer_setup = &uss820dci_xfer_setup, .xfer_unsetup = &uss820dci_xfer_unsetup, .get_hw_ep_profile = &uss820dci_get_hw_ep_profile, + .xfer_stall = &uss820dci_xfer_stall, .set_stall = &uss820dci_set_stall, .clear_stall = &uss820dci_clear_stall, .roothub_exec = &uss820dci_roothub_exec, Modified: head/sys/dev/usb/controller/xhci.c ============================================================================== --- head/sys/dev/usb/controller/xhci.c Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/controller/xhci.c Sun Aug 12 17:53:06 2012 (r239214) @@ -128,7 +128,7 @@ static usb_proc_callback_t xhci_configur static usb_error_t xhci_configure_device(struct usb_device *); static usb_error_t xhci_configure_endpoint(struct usb_device *, struct usb_endpoint_descriptor *, uint64_t, uint16_t, - uint8_t, uint8_t, uint8_t, uint16_t, uint16_t); + uint8_t, uint8_t, uint8_t, uint16_t, uint16_t, uint8_t); static usb_error_t xhci_configure_mask(struct usb_device *, uint32_t, uint8_t); static usb_error_t xhci_cmd_evaluate_ctx(struct xhci_softc *, @@ -1245,7 +1245,7 @@ xhci_set_address(struct usb_device *udev &udev->ctrl_ep_desc); err = xhci_configure_endpoint(udev, &udev->ctrl_ep_desc, pepext->physaddr, - 0, 1, 1, 0, mps, mps); + 0, 1, 1, 0, mps, mps, USB_EP_MODE_DEFAULT); if (err != 0) { DPRINTF("Could not configure default endpoint\n"); @@ -1800,7 +1800,8 @@ xhci_setup_generic_chain(struct usb_xfer /* compute multiplier for ISOCHRONOUS transfers */ mult = xfer->endpoint->ecomp ? - (xfer->endpoint->ecomp->bmAttributes & 3) : 0; + UE_GET_SS_ISO_MULT(xfer->endpoint->ecomp->bmAttributes) + : 0; /* check for USB 2.0 multiplier */ if (mult == 0) { mult = (xfer->endpoint->edesc-> @@ -2055,7 +2056,8 @@ static usb_error_t xhci_configure_endpoint(struct usb_device *udev, struct usb_endpoint_descriptor *edesc, uint64_t ring_addr, uint16_t interval, uint8_t max_packet_count, uint8_t mult, - uint8_t fps_shift, uint16_t max_packet_size, uint16_t max_frame_size) + uint8_t fps_shift, uint16_t max_packet_size, + uint16_t max_frame_size, uint8_t ep_mode) { struct usb_page_search buf_inp; struct xhci_softc *sc = XHCI_BUS2SC(udev->bus); @@ -2090,9 +2092,20 @@ xhci_configure_endpoint(struct usb_devic if (mult == 0) return (USB_ERR_BAD_BUFSIZE); - temp = XHCI_EPCTX_0_EPSTATE_SET(0) | - XHCI_EPCTX_0_MAXP_STREAMS_SET(0) | - XHCI_EPCTX_0_LSA_SET(0); + if (ep_mode == USB_EP_MODE_STREAMS) { + temp = XHCI_EPCTX_0_EPSTATE_SET(0) | + XHCI_EPCTX_0_MAXP_STREAMS_SET(XHCI_MAX_STREAMS_LOG - 1) | + XHCI_EPCTX_0_LSA_SET(1); + + ring_addr += sizeof(struct xhci_trb) * + XHCI_MAX_TRANSFERS * XHCI_MAX_STREAMS; + } else { + temp = XHCI_EPCTX_0_EPSTATE_SET(0) | + XHCI_EPCTX_0_MAXP_STREAMS_SET(0) | + XHCI_EPCTX_0_LSA_SET(0); + + ring_addr |= XHCI_EPCTX_2_DCS_SET(1); + } switch (udev->speed) { case USB_SPEED_FULL: @@ -2160,9 +2173,6 @@ xhci_configure_endpoint(struct usb_devic temp |= XHCI_EPCTX_1_EPTYPE_SET(4); xhci_ctx_set_le32(sc, &pinp->ctx_ep[epno - 1].dwEpCtx1, temp); - - ring_addr |= XHCI_EPCTX_2_DCS_SET(1); - xhci_ctx_set_le64(sc, &pinp->ctx_ep[epno - 1].qwEpCtx2, ring_addr); switch (edesc->bmAttributes & UE_XFERTYPE) { @@ -2195,21 +2205,42 @@ xhci_configure_endpoint_by_xfer(struct u { struct xhci_endpoint_ext *pepext; struct usb_endpoint_ss_comp_descriptor *ecomp; + usb_stream_t x; pepext = xhci_get_endpoint_ext(xfer->xroot->udev, xfer->endpoint->edesc); ecomp = xfer->endpoint->ecomp; - pepext->trb[0].dwTrb3 = 0; /* halt any transfers */ + for (x = 0; x != XHCI_MAX_STREAMS; x++) { + uint64_t temp; + + /* halt any transfers */ + pepext->trb[x * XHCI_MAX_TRANSFERS].dwTrb3 = 0; + + /* compute start of TRB ring for stream "x" */ + temp = pepext->physaddr + + (x * XHCI_MAX_TRANSFERS * sizeof(struct xhci_trb)) + + XHCI_SCTX_0_SCT_SEC_TR_RING; + + /* make tree structure */ + pepext->trb[(XHCI_MAX_TRANSFERS * + XHCI_MAX_STREAMS) + x].qwTrb0 = htole64(temp); + + /* reserved fields */ + pepext->trb[(XHCI_MAX_TRANSFERS * + XHCI_MAX_STREAMS) + x].dwTrb2 = 0; + pepext->trb[(XHCI_MAX_TRANSFERS * + XHCI_MAX_STREAMS) + x].dwTrb3 = 0; + } usb_pc_cpu_flush(pepext->page_cache); return (xhci_configure_endpoint(xfer->xroot->udev, xfer->endpoint->edesc, pepext->physaddr, xfer->interval, xfer->max_packet_count, - (ecomp != NULL) ? (ecomp->bmAttributes & 3) + 1 : 1, + (ecomp != NULL) ? UE_GET_SS_ISO_MULT(ecomp->bmAttributes) + 1 : 1, usbd_xfer_get_fps_shift(xfer), xfer->max_packet_size, - xfer->max_frame_size)); + xfer->max_frame_size, xfer->endpoint->ep_mode)); } static usb_error_t @@ -2500,7 +2531,8 @@ xhci_get_endpoint_ext(struct usb_device pc = &sc->sc_hw.devs[index].endpoint_pc; - usbd_get_page(pc, (uintptr_t)&((struct xhci_dev_endpoint_trbs *)0)->trb[epno][0], &buf_ep); + usbd_get_page(pc, (uintptr_t)&((struct xhci_dev_endpoint_trbs *)0)-> + trb[epno][0], &buf_ep); pepext = &sc->sc_hw.devs[index].endp[epno]; pepext->page_cache = pc; @@ -2539,7 +2571,7 @@ xhci_transfer_remove(struct usb_xfer *xf pepext = xhci_get_endpoint_ext(xfer->xroot->udev, xfer->endpoint->edesc); - pepext->trb_used--; + pepext->trb_used[xfer->stream_id]--; pepext->xfer[xfer->qh_pos] = NULL; @@ -2557,12 +2589,15 @@ xhci_transfer_insert(struct usb_xfer *xf struct xhci_td *td_last; struct xhci_endpoint_ext *pepext; uint64_t addr; + usb_stream_t id; uint8_t i; uint8_t inext; uint8_t trb_limit; DPRINTFN(8, "\n"); + id = xfer->stream_id; + /* check if already inserted */ if (xfer->flags_int.bandwidth_reclaimed) { DPRINTFN(8, "Already in schedule\n"); @@ -2588,7 +2623,7 @@ xhci_transfer_insert(struct usb_xfer *xf break; } - if (pepext->trb_used >= trb_limit) { + if (pepext->trb_used[id] >= trb_limit) { DPRINTFN(8, "Too many TDs queued.\n"); return (USB_ERR_NOMEM); } @@ -2605,10 +2640,10 @@ xhci_transfer_insert(struct usb_xfer *xf return (0); } - pepext->trb_used++; + pepext->trb_used[id]++; /* get current TRB index */ - i = pepext->trb_index; + i = pepext->trb_index[id]; /* get next TRB index */ inext = (i + 1); @@ -2617,8 +2652,12 @@ xhci_transfer_insert(struct usb_xfer *xf if (inext >= (XHCI_MAX_TRANSFERS - 1)) inext = 0; + /* offset for stream */ + i += id * XHCI_MAX_TRANSFERS; + inext += id * XHCI_MAX_TRANSFERS; + /* compute terminating return address */ - addr += inext * sizeof(struct xhci_trb); + addr += (inext * sizeof(struct xhci_trb)); /* update next pointer of last link TRB */ td_last->td_trb[td_last->ntrb].qwTrb0 = htole64(addr); @@ -2662,7 +2701,7 @@ xhci_transfer_insert(struct usb_xfer *xf xfer->flags_int.bandwidth_reclaimed = 1; - pepext->trb_index = inext; + pepext->trb_index[id] = inext; xhci_endpoint_doorbell(xfer); @@ -2750,12 +2789,12 @@ xhci_device_generic_close(struct usb_xfe static void xhci_device_generic_multi_enter(struct usb_endpoint *ep, - struct usb_xfer *enter_xfer) + usb_stream_t stream_id, struct usb_xfer *enter_xfer) { struct usb_xfer *xfer; /* check if there is a current transfer */ - xfer = ep->endpoint_q.curr; + xfer = ep->endpoint_q[stream_id].curr; if (xfer == NULL) return; @@ -2767,7 +2806,7 @@ xhci_device_generic_multi_enter(struct u if (!xfer->flags_int.bandwidth_reclaimed) return; - xfer = TAILQ_FIRST(&ep->endpoint_q.head); + xfer = TAILQ_FIRST(&ep->endpoint_q[stream_id].head); if (xfer == NULL) { /* * In case of enter we have to consider that the @@ -2792,7 +2831,8 @@ xhci_device_generic_enter(struct usb_xfe /* setup TD's and QH */ xhci_setup_generic_chain(xfer); - xhci_device_generic_multi_enter(xfer->endpoint, xfer); + xhci_device_generic_multi_enter(xfer->endpoint, + xfer->stream_id, xfer); } static void @@ -2804,7 +2844,8 @@ xhci_device_generic_start(struct usb_xfe xhci_transfer_insert(xfer); /* try to multi buffer */ - xhci_device_generic_multi_enter(xfer->endpoint, NULL); + xhci_device_generic_multi_enter(xfer->endpoint, + xfer->stream_id, NULL); /* add transfer last on interrupt queue */ usbd_transfer_enqueue(&xfer->xroot->bus->intr_q, xfer); @@ -3465,6 +3506,7 @@ xhci_configure_reset_endpoint(struct usb struct usb_endpoint_descriptor *edesc; struct usb_page_cache *pcinp; usb_error_t err; + usb_stream_t stream_id; uint8_t index; uint8_t epno; @@ -3481,6 +3523,7 @@ xhci_configure_reset_endpoint(struct usb edesc = xfer->endpoint->edesc; epno = edesc->bEndpointAddress; + stream_id = xfer->stream_id; if ((edesc->bmAttributes & UE_XFERTYPE) == UE_CONTROL) epno |= UE_DIR_IN; @@ -3516,8 +3559,10 @@ xhci_configure_reset_endpoint(struct usb if (err != 0) DPRINTF("Could not reset endpoint %u\n", epno); - err = xhci_cmd_set_tr_dequeue_ptr(sc, pepext->physaddr | - XHCI_EPCTX_2_DCS_SET(1), 0, epno, index); + err = xhci_cmd_set_tr_dequeue_ptr(sc, + (pepext->physaddr + (stream_id * sizeof(struct xhci_trb) * + XHCI_MAX_TRANSFERS)) | XHCI_EPCTX_2_DCS_SET(1), + stream_id, epno, index); if (err != 0) DPRINTF("Could not set dequeue ptr for endpoint %u\n", epno); @@ -3615,7 +3660,8 @@ restart: /* check if halted is still cleared */ if (pepext->trb_halted == 0) { pepext->trb_running = 1; - pepext->trb_index = 0; + memset(pepext->trb_index, 0, + sizeof(pepext->trb_index)); } goto restart; } @@ -3639,7 +3685,8 @@ restart: xhci_transfer_insert(xfer); /* try to multi buffer */ - xhci_device_generic_multi_enter(xfer->endpoint, NULL); + xhci_device_generic_multi_enter(xfer->endpoint, + xfer->stream_id, NULL); } } @@ -3946,6 +3993,23 @@ xhci_device_state_change(struct usb_devi XHCI_CMD_UNLOCK(sc); } +static usb_error_t +xhci_set_endpoint_mode(struct usb_device *udev, struct usb_endpoint *ep, + uint8_t ep_mode) +{ + switch (ep_mode) { + case USB_EP_MODE_DEFAULT: + return (0); + case USB_EP_MODE_STREAMS: + if ((ep->edesc->bmAttributes & UE_XFERTYPE) != UE_BULK || + udev->speed != USB_SPEED_SUPER) + return (USB_ERR_INVAL); + return (0); + default: + return (USB_ERR_INVAL); + } +} + struct usb_bus_methods xhci_bus_methods = { .endpoint_init = xhci_ep_init, .endpoint_uninit = xhci_ep_uninit, @@ -3964,4 +4028,5 @@ struct usb_bus_methods xhci_bus_methods .clear_stall = xhci_ep_clear_stall, .device_state_change = xhci_device_state_change, .set_hw_power_sleep = xhci_set_hw_power_sleep, + .set_endpoint_mode = xhci_set_endpoint_mode, }; Modified: head/sys/dev/usb/controller/xhci.h ============================================================================== --- head/sys/dev/usb/controller/xhci.h Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/controller/xhci.h Sun Aug 12 17:53:06 2012 (r239214) @@ -35,7 +35,15 @@ #define XHCI_MAX_COMMANDS (16 * 1) #define XHCI_MAX_RSEG 1 #define XHCI_MAX_TRANSFERS 4 - +#if USB_MAX_EP_STREAMS == 8 +#define XHCI_MAX_STREAMS 8 +#define XHCI_MAX_STREAMS_LOG 3 +#elif USB_MAX_EP_STREAMS == 1 +#define XHCI_MAX_STREAMS 1 +#define XHCI_MAX_STREAMS_LOG 0 +#else +#error "The USB_MAX_EP_STREAMS value is not supported." +#endif #define XHCI_DEV_CTX_ADDR_ALIGN 64 /* bytes */ #define XHCI_DEV_CTX_ALIGN 64 /* bytes */ #define XHCI_INPUT_CTX_ALIGN 64 /* bytes */ @@ -307,7 +315,8 @@ struct xhci_trb { } __aligned(4); struct xhci_dev_endpoint_trbs { - struct xhci_trb trb[XHCI_MAX_ENDPOINTS][XHCI_MAX_TRANSFERS]; + struct xhci_trb trb[XHCI_MAX_ENDPOINTS] + [(XHCI_MAX_STREAMS * XHCI_MAX_TRANSFERS) + XHCI_MAX_STREAMS]; }; #define XHCI_TD_PAGE_NBUF 17 /* units, room enough for 64Kbytes */ @@ -353,11 +362,11 @@ struct xhci_hw_root { struct xhci_endpoint_ext { struct xhci_trb *trb; - struct usb_xfer *xfer[XHCI_MAX_TRANSFERS - 1]; + struct usb_xfer *xfer[XHCI_MAX_TRANSFERS * XHCI_MAX_STREAMS]; struct usb_page_cache *page_cache; uint64_t physaddr; - uint8_t trb_used; - uint8_t trb_index; + uint8_t trb_used[XHCI_MAX_STREAMS]; + uint8_t trb_index[XHCI_MAX_STREAMS]; uint8_t trb_halted; uint8_t trb_running; }; Modified: head/sys/dev/usb/usb.h ============================================================================== --- head/sys/dev/usb/usb.h Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/usb.h Sun Aug 12 17:53:06 2012 (r239214) @@ -546,6 +546,8 @@ struct usb_endpoint_ss_comp_descriptor { uByte bDescriptorType; uByte bMaxBurst; uByte bmAttributes; +#define UE_GET_BULK_STREAMS(x) ((x) & 0x0F) +#define UE_GET_SS_ISO_MULT(x) ((x) & 0x03) uWord wBytesPerInterval; } __packed; typedef struct usb_endpoint_ss_comp_descriptor @@ -744,7 +746,7 @@ enum usb_revision { #define USB_REV_MAX (USB_REV_3_0+1) /* - * Supported host contoller modes. + * Supported host controller modes. */ enum usb_hc_mode { USB_MODE_HOST, /* initiates transfers */ @@ -754,7 +756,7 @@ enum usb_hc_mode { #define USB_MODE_MAX (USB_MODE_DUAL+1) /* - * The "USB_MODE" macros defines all the supported device states. + * The "USB_STATE" enums define all the supported device states. */ enum usb_dev_state { USB_STATE_DETACHED, @@ -764,4 +766,18 @@ enum usb_dev_state { USB_STATE_CONFIGURED, }; #define USB_STATE_MAX (USB_STATE_CONFIGURED+1) + +/* + * The "USB_EP_MODE" macros define all the currently supported + * endpoint modes. + */ +enum usb_ep_mode { + USB_EP_MODE_DEFAULT, + USB_EP_MODE_STREAMS, /* USB3.0 specific */ + USB_EP_MODE_HW_MASS_STORAGE, + USB_EP_MODE_HW_SERIAL, + USB_EP_MODE_HW_ETHERNET_CDC, + USB_EP_MODE_HW_ETHERNET_NCM, + USB_EP_MODE_MAX +}; #endif /* _USB_STANDARD_H_ */ Modified: head/sys/dev/usb/usb_controller.h ============================================================================== --- head/sys/dev/usb/usb_controller.h Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/usb_controller.h Sun Aug 12 17:53:06 2012 (r239214) @@ -108,7 +108,8 @@ struct usb_bus_methods { /* USB Device mode only - Mandatory */ void (*get_hw_ep_profile) (struct usb_device *udev, const struct usb_hw_ep_profile **ppf, uint8_t ep_addr); - void (*set_stall) (struct usb_device *udev, struct usb_xfer *xfer, struct usb_endpoint *ep, uint8_t *did_stall); + void (*xfer_stall) (struct usb_xfer *xfer); + void (*set_stall) (struct usb_device *udev, struct usb_endpoint *ep, uint8_t *did_stall); /* USB Device mode mandatory. USB Host mode optional. */ @@ -143,6 +144,10 @@ struct usb_bus_methods { /* Optional for host mode */ usb_error_t (*set_address) (struct usb_device *, struct mtx *, uint16_t); + + /* Optional for device and host mode */ + + usb_error_t (*set_endpoint_mode) (struct usb_device *, struct usb_endpoint *, uint8_t); }; /* Modified: head/sys/dev/usb/usb_core.h ============================================================================== --- head/sys/dev/usb/usb_core.h Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/usb_core.h Sun Aug 12 17:53:06 2012 (r239214) @@ -151,6 +151,7 @@ struct usb_xfer { usb_frcount_t nframes; /* number of USB frames to transfer */ usb_frcount_t aframes; /* actual number of USB frames * transferred */ + usb_stream_t stream_id; /* USB3.0 specific field */ uint16_t max_packet_size; uint16_t max_frame_size; Modified: head/sys/dev/usb/usb_debug.c ============================================================================== --- head/sys/dev/usb/usb_debug.c Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/usb_debug.c Sun Aug 12 17:53:06 2012 (r239214) @@ -113,10 +113,12 @@ void usb_dump_queue(struct usb_endpoint *ep) { struct usb_xfer *xfer; + usb_stream_t x; printf("usb_dump_queue: endpoint=%p xfer: ", ep); - TAILQ_FOREACH(xfer, &ep->endpoint_q.head, wait_entry) { - printf(" %p", xfer); + for (x = 0; x != USB_MAX_EP_STREAMS; x++) { + TAILQ_FOREACH(xfer, &ep->endpoint_q[x].head, wait_entry) + printf(" %p", xfer); } printf("\n"); } Modified: head/sys/dev/usb/usb_device.c ============================================================================== --- head/sys/dev/usb/usb_device.c Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/usb_device.c Sun Aug 12 17:53:06 2012 (r239214) @@ -355,7 +355,6 @@ usbd_interface_count(struct usb_device * return (USB_ERR_NORMAL_COMPLETION); } - /*------------------------------------------------------------------------* * usb_init_endpoint * @@ -370,6 +369,7 @@ usb_init_endpoint(struct usb_device *ude struct usb_endpoint *ep) { struct usb_bus_methods *methods; + usb_stream_t x; methods = udev->bus->methods; @@ -379,13 +379,26 @@ usb_init_endpoint(struct usb_device *ude ep->edesc = edesc; ep->ecomp = ecomp; ep->iface_index = iface_index; - TAILQ_INIT(&ep->endpoint_q.head); - ep->endpoint_q.command = &usbd_pipe_start; + + /* setup USB stream queues */ + for (x = 0; x != USB_MAX_EP_STREAMS; x++) { + TAILQ_INIT(&ep->endpoint_q[x].head); + ep->endpoint_q[x].command = &usbd_pipe_start; + } /* the pipe is not supported by the hardware */ if (ep->methods == NULL) return; + /* check for SUPER-speed streams mode endpoint */ + if (udev->speed == USB_SPEED_SUPER && ecomp != NULL && + (edesc->bmAttributes & UE_XFERTYPE) == UE_BULK && + (UE_GET_BULK_STREAMS(ecomp->bmAttributes) != 0)) { + usbd_set_endpoint_mode(udev, ep, USB_EP_MODE_STREAMS); + } else { + usbd_set_endpoint_mode(udev, ep, USB_EP_MODE_DEFAULT); + } + /* clear stall, if any */ if (methods->clear_stall != NULL) { USB_BUS_LOCK(udev->bus); @@ -933,6 +946,7 @@ usbd_set_endpoint_stall(struct usb_devic uint8_t do_stall) { struct usb_xfer *xfer; + usb_stream_t x; uint8_t et; uint8_t was_stalled; @@ -975,18 +989,22 @@ usbd_set_endpoint_stall(struct usb_devic if (do_stall || (!was_stalled)) { if (!was_stalled) { - /* lookup the current USB transfer, if any */ - xfer = ep->endpoint_q.curr; - } else { - xfer = NULL; + for (x = 0; x != USB_MAX_EP_STREAMS; x++) { + /* lookup the current USB transfer, if any */ + xfer = ep->endpoint_q[x].curr; + if (xfer != NULL) { + /* + * The "xfer_stall" method + * will complete the USB + * transfer like in case of a + * timeout setting the error + * code "USB_ERR_STALLED". + */ + (udev->bus->methods->xfer_stall) (xfer); + } + } } - - /* - * If "xfer" is non-NULL the "set_stall" method will - * complete the USB transfer like in case of a timeout - * setting the error code "USB_ERR_STALLED". - */ - (udev->bus->methods->set_stall) (udev, xfer, ep, &do_stall); + (udev->bus->methods->set_stall) (udev, ep, &do_stall); } if (!do_stall) { ep->toggle_next = 0; /* reset data toggle */ @@ -994,8 +1012,11 @@ usbd_set_endpoint_stall(struct usb_devic (udev->bus->methods->clear_stall) (udev, ep); - /* start up the current or next transfer, if any */ - usb_command_wrapper(&ep->endpoint_q, ep->endpoint_q.curr); + /* start the current or next transfer, if any */ + for (x = 0; x != USB_MAX_EP_STREAMS; x++) { + usb_command_wrapper(&ep->endpoint_q[x], + ep->endpoint_q[x].curr); + } } USB_BUS_UNLOCK(udev->bus); return (0); @@ -2745,3 +2766,37 @@ usbd_add_dynamic_quirk(struct usb_device } return (USB_ERR_NOMEM); } + +/* + * The following function is used to select the endpoint mode. It + * should not be called outside enumeration context. + */ + +usb_error_t +usbd_set_endpoint_mode(struct usb_device *udev, struct usb_endpoint *ep, + uint8_t ep_mode) +{ + usb_error_t error; + + sx_assert(&udev->enum_sx, SA_LOCKED); + + if (udev->bus->methods->set_endpoint_mode != NULL) { + error = (udev->bus->methods->set_endpoint_mode) ( + udev, ep, ep_mode); + } else if (ep_mode != USB_EP_MODE_DEFAULT) { + error = USB_ERR_INVAL; + } else { + error = 0; + } + + /* only set new mode regardless of error */ + ep->ep_mode = ep_mode; + + return (error); +} + +uint8_t +usbd_get_endpoint_mode(struct usb_device *udev, struct usb_endpoint *ep) +{ + return (ep->ep_mode); +} Modified: head/sys/dev/usb/usb_freebsd.h ============================================================================== --- head/sys/dev/usb/usb_freebsd.h Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/usb_freebsd.h Sun Aug 12 17:53:06 2012 (r239214) @@ -60,6 +60,7 @@ #define USB_MAX_DEVICES 128 /* units */ #define USB_IFACE_MAX 32 /* units */ #define USB_FIFO_MAX 128 /* units */ +#define USB_MAX_EP_STREAMS 8 /* units */ #define USB_MAX_FS_ISOC_FRAMES_PER_XFER (120) /* units */ #define USB_MAX_HS_ISOC_FRAMES_PER_XFER (8*120) /* units */ @@ -76,5 +77,6 @@ typedef uint32_t usb_frcount_t; /* unit typedef uint32_t usb_size_t; /* bytes */ typedef uint32_t usb_ticks_t; /* system defined */ typedef uint16_t usb_power_mask_t; /* see "USB_HW_POWER_XXX" */ +typedef uint16_t usb_stream_t; /* stream ID */ #endif /* _USB_FREEBSD_H_ */ Modified: head/sys/dev/usb/usb_generic.c ============================================================================== --- head/sys/dev/usb/usb_generic.c Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/usb_generic.c Sun Aug 12 17:53:06 2012 (r239214) @@ -253,6 +253,7 @@ ugen_open_pipe_write(struct usb_fifo *f) usb_config[0].type = ed->bmAttributes & UE_XFERTYPE; usb_config[0].endpoint = ed->bEndpointAddress & UE_ADDR; + usb_config[0].stream_id = 0; /* XXX support more stream ID's */ usb_config[0].direction = UE_DIR_TX; usb_config[0].interval = USB_DEFAULT_INTERVAL; usb_config[0].flags.proxy_buffer = 1; @@ -321,6 +322,7 @@ ugen_open_pipe_read(struct usb_fifo *f) usb_config[0].type = ed->bmAttributes & UE_XFERTYPE; usb_config[0].endpoint = ed->bEndpointAddress & UE_ADDR; + usb_config[0].stream_id = 0; /* XXX support more stream ID's */ usb_config[0].direction = UE_DIR_RX; usb_config[0].interval = USB_DEFAULT_INTERVAL; usb_config[0].flags.proxy_buffer = 1; @@ -1391,6 +1393,7 @@ ugen_ioctl(struct usb_fifo *f, u_long cm struct usb_fs_start *pstart; struct usb_fs_stop *pstop; struct usb_fs_open *popen; + struct usb_fs_open_streams *popen_streams; struct usb_fs_close *pclose; struct usb_fs_clear_stall_sync *pstall; void *addr; @@ -1455,6 +1458,7 @@ ugen_ioctl(struct usb_fifo *f, u_long cm break; case USB_FS_OPEN: + case USB_FS_OPEN_STREAMS: if (u.popen->ep_index >= f->fs_ep_max) { error = EINVAL; break; @@ -1506,6 +1510,8 @@ ugen_ioctl(struct usb_fifo *f, u_long cm usb_config[0].frames = u.popen->max_frames; usb_config[0].bufsize = u.popen->max_bufsize; usb_config[0].usb_mode = USB_MODE_DUAL; /* both modes */ + if (cmd == USB_FS_OPEN_STREAMS) + usb_config[0].stream_id = u.popen_streams->stream_id; if (usb_config[0].type == UE_CONTROL) { if (f->udev->flags.usb_mode != USB_MODE_HOST) { Modified: head/sys/dev/usb/usb_hub.c ============================================================================== --- head/sys/dev/usb/usb_hub.c Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/usb_hub.c Sun Aug 12 17:53:06 2012 (r239214) @@ -1757,9 +1757,11 @@ usbd_fs_isoc_schedule_alloc_slot(struct data_len += len; } - /* check double buffered transfers */ - - TAILQ_FOREACH(pipe_xfer, &xfer->endpoint->endpoint_q.head, + /* + * Check double buffered transfers. Only stream ID + * equal to zero is valid here! + */ + TAILQ_FOREACH(pipe_xfer, &xfer->endpoint->endpoint_q[0].head, wait_entry) { /* skip self, if any */ Modified: head/sys/dev/usb/usb_ioctl.h ============================================================================== --- head/sys/dev/usb/usb_ioctl.h Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/usb_ioctl.h Sun Aug 12 17:53:06 2012 (r239214) @@ -206,6 +206,11 @@ struct usb_fs_open { uint8_t ep_no; /* bEndpointNumber */ }; +struct usb_fs_open_streams { + struct usb_fs_open fs_open; + uint16_t stream_id; +}; + struct usb_fs_close { uint8_t ep_index; }; @@ -302,6 +307,7 @@ struct usb_gen_quirk { #define USB_FS_OPEN _IOWR('U', 197, struct usb_fs_open) #define USB_FS_CLOSE _IOW ('U', 198, struct usb_fs_close) #define USB_FS_CLEAR_STALL_SYNC _IOW ('U', 199, struct usb_fs_clear_stall_sync) +#define USB_FS_OPEN_STREAMS _IOWR('U', 200, struct usb_fs_open_streams) /* USB quirk system interface */ #define USB_DEV_QUIRK_GET _IOWR('Q', 0, struct usb_gen_quirk) Modified: head/sys/dev/usb/usb_request.c ============================================================================== --- head/sys/dev/usb/usb_request.c Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/usb_request.c Sun Aug 12 17:53:06 2012 (r239214) @@ -228,6 +228,7 @@ usb_do_clear_stall_callback(struct usb_x struct usb_endpoint *ep; struct usb_endpoint *ep_end; struct usb_endpoint *ep_first; + usb_stream_t x; uint8_t to; udev = xfer->xroot->udev; @@ -255,9 +256,11 @@ tr_transferred: ep->is_stalled = 0; /* some hardware needs a callback to clear the data toggle */ usbd_clear_stall_locked(udev, ep); - /* start up the current or next transfer, if any */ - usb_command_wrapper(&ep->endpoint_q, - ep->endpoint_q.curr); + for (x = 0; x != USB_MAX_EP_STREAMS; x++) { + /* start the current or next transfer, if any */ + usb_command_wrapper(&ep->endpoint_q[x], + ep->endpoint_q[x].curr); + } } ep++; Modified: head/sys/dev/usb/usb_transfer.c ============================================================================== --- head/sys/dev/usb/usb_transfer.c Sun Aug 12 17:01:07 2012 (r239213) +++ head/sys/dev/usb/usb_transfer.c Sun Aug 12 17:53:06 2012 (r239214) @@ -358,7 +358,8 @@ usbd_transfer_setup_sub(struct usb_setup switch (type) { case UE_ISOCHRONOUS: case UE_INTERRUPT: - xfer->max_packet_count += (xfer->max_packet_size >> 11) & 3; + xfer->max_packet_count += + (xfer->max_packet_size >> 11) & 3; /* check for invalid max packet count */ if (xfer->max_packet_count > 3) @@ -387,7 +388,8 @@ usbd_transfer_setup_sub(struct usb_setup if (ecomp != NULL) { uint8_t mult; - mult = (ecomp->bmAttributes & 3) + 1; + mult = UE_GET_SS_ISO_MULT( + ecomp->bmAttributes) + 1; if (mult > 3) mult = 3; @@ -946,7 +948,20 @@ usbd_transfer_setup(struct usb_device *u ep = usbd_get_endpoint(udev, ifaces[setup->if_index], setup); - if ((ep == NULL) || (ep->methods == NULL)) { + /* + * Check that the USB PIPE is valid and that + * the endpoint mode is proper. + * + * Make sure we don't allocate a streams + * transfer when such a combination is not + * valid. + */ + if ((ep == NULL) || (ep->methods == NULL) || + ((ep->ep_mode != USB_EP_MODE_STREAMS) && + (ep->ep_mode != USB_EP_MODE_DEFAULT)) || + (setup->stream_id != 0 && + (setup->stream_id >= USB_MAX_EP_STREAMS || + (ep->ep_mode != USB_EP_MODE_STREAMS)))) { if (setup->flags.no_pipe_ok) continue; if ((setup->usb_mode != USB_MODE_DUAL) && @@ -990,6 +1005,9 @@ usbd_transfer_setup(struct usb_device *u /* set transfer endpoint pointer */ xfer->endpoint = ep; + /* set transfer stream ID */ + xfer->stream_id = setup->stream_id; + parm.size[0] += sizeof(xfer[0]); parm.methods = xfer->endpoint->methods; *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***