From owner-svn-src-all@freebsd.org Mon Jun 11 02:35:48 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F058C10132C8 for ; Mon, 11 Jun 2018 02:35:47 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from sonic314-22.consmr.mail.ne1.yahoo.com (sonic314-22.consmr.mail.ne1.yahoo.com [66.163.189.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8C02386A66 for ; Mon, 11 Jun 2018 02:35:47 +0000 (UTC) (envelope-from pfg@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1528684541; bh=fs5yY9pcinvKOIYE6tOUvw4H8aIcfA7Kd/y1px9f44Y=; h=Subject:To:References:From:Date:In-Reply-To:From:Subject; b=An/Sq9Ltf7UPdm4H7qTKVp7CJuXtrtABY69HP/iS4czLR3jaOg4f8oMV3PwZ5x+DPpPy8tLdzzfk1LgoXGl0CFv9eKaKMabk78OhwxOobUhZPPj7i+5vog9Ty36+kYltgkdnvYUQtc82RhBcnPNFn7hfwTuIsleQctmL/lCW53jI2ffkX43rA9m0MyVEJbZtf93p83CqsL0A/c3EKUxgfCLecac4IYtytFADTtvPA5Kat/7dPAgyeW3ZnKXL5engu7gUqcNfuXrlfJI2HuEkqifdQQcjSRXuHCGGuISEPp1d9af688qdCF0D/DdZ5mj9c3rozlo8l3E+riGJXgePEw== X-YMail-OSG: ZnJb4C0VM1kDc03XSMtsLLdQs0_kj1S0N2OorGVuW95gYmYiyWr.ONU7LJ2RH7E IPZRCvmGj9JmNzuUktb6I8QLTlcQQOyMC7QEPpD9z9jly9oNizR97pUOS1kFrDWavbkfhS4FSwei hMELowZfWfDh8LgQFWdL7k8YU7bby4574SpVwDEs1VHDTDjva1RoGhzkGP4MvzEV9e6hQN3t0COg daM89FSq8dNN8wJFqzC4Hl2r8JfeTV0osrtYQHkPxglobY5vGFR1x_rPk2JIKCBmhDI8hZG_FGkW ZTl8ym.F4mhPTyJca4AJ3T4gnOxdWnSssJQbt6QeQZGH6zV8jeUdvqYgAl3a9tM0.DGaGvpbPPRc L7s4tttqYc7_v3aKfUfSTUuYLrqBgrFrtX17AiVqWa81DVUlj.H9p0IBqubgn8wF5.oHtsZeP39A 8elCtNhF0ob8nXNlOQvtWq4XEiCTZIC3j4CrQl70Zl6l5q0mfcb7nB1d3Ggo1y3qCCzzBHx4MoMI b.7RM54Vkz1GA5TrUGV37go2k7ockKGooG_IWBNF5Dp.MH2LWitI9LDX9Ow8I9jTqufiEZOJTi2i JzQpC9F10dtvHHf0R_H557Pljx9ukOrcK0icSD10mYSQkicdKPm5XdTWefuo6tBKtbjvdksS9S1y wte8pujTZjXt7pJ4lfKQVobLqRzCrelHyxuKMVU5qKj74mRQ9hWikhzy44.EjxQLeYEiRMvHvPt4 jcMBBpWwz3XlYIQcLvTsJfwfxA3RKU_CaPwmJWk1G6E0RjbRqBxmgWvG2tZaGECFuGim3tAK8_mB uTsluiDornvhLEYyLpH8uqotLiVDVl_gEUX5yb9P3u9Awf08qzKrOVMmjz5ElL.22_kwgTRx_QV3 dG1HPAUZziXeoT0UUKEBmmTc88fo9T3fjFBhacNnhWA1gtDB9acb7G5dP5drLbjG2n8yqji6I4Aq 2ADSQxg-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.ne1.yahoo.com with HTTP; Mon, 11 Jun 2018 02:35:41 +0000 Received: from 181.52.72.201 (EHLO [192.168.0.6]) ([181.52.72.201]) by smtp422.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 4cf020aab94b4c5444456e4ba99bd765; Mon, 11 Jun 2018 02:25:30 +0000 (UTC) Subject: Re: svn commit: r334940 - head/usr.sbin/bhyve To: Marcelo Araujo , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201806110209.w5B29Lni010557@repo.freebsd.org> From: Pedro Giffuni Organization: FreeBSD Project Message-ID: Date: Sun, 10 Jun 2018 21:25:30 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <201806110209.w5B29Lni010557@repo.freebsd.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jun 2018 02:35:48 -0000 On 10/06/2018 21:09, Marcelo Araujo wrote: > Author: araujo > Date: Mon Jun 11 02:09:20 2018 > New Revision: 334940 > URL: https://svnweb.freebsd.org/changeset/base/334940 > > Log: > - Add bhyve virtio-scsi storage backend support. > > Example of configuration: > ctl.conf: > portal-group pg0 { > discovery-auth-group no-authentication > listen 0.0.0.0 > listen [::] > } > > target iqn.2012-06.com.example:target0 { > auth-group no-authentication > portal-group pg0 > port ioctl/5/3 > > lun 0 { > path /z/test.img > size 8G > } > lun 1 { > path /z/test1.img > size 8G > } > } > > bhyve <...> -s 4,virtio-scsi,/dev/cam/ctl5.3,iid=3 > > From inside guest: > root@:~ # zpool status test > pool: test > state: ONLINE > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > test ONLINE 0 0 0 > da0 ONLINE 0 0 0 > da1 ONLINE 0 0 0 > > dmesg: > da0 at vtscsi0 bus 0 scbus0 target 0 lun 0 > da0: Fixed Direct Access SPC-5 SCSI device > da0: Serial Number MYSERIAL0000 > da0: 300.000MB/s transfers > da0: Command Queueing enabled > da0: 8192MB (16777216 512 byte sectors) > da1 at vtscsi0 bus 0 scbus0 target 0 lun 1 > da1: Fixed Direct Access SPC-5 SCSI device > da1: Serial Number MYSERIAL0001 > da1: 300.000MB/s transfers > da1: Command Queueing enabled > da1: 8192MB (16777216 512 byte sectors) > > Discussed with: grehan > Reviewed by: mav > Obtained from: TrueOS > Relnotes: Yes > Sponsored by: iXsystems Inc. > Tested with: FreeBSD HEAD, Fedora 28 (Workstation) and > Ubuntu 18.04. > Differential Revision: https://reviews.freebsd.org/D15276 > > Added: > head/usr.sbin/bhyve/iov.c (contents, props changed) > head/usr.sbin/bhyve/iov.h (contents, props changed) > head/usr.sbin/bhyve/pci_virtio_scsi.c (contents, props changed) > Modified: > head/usr.sbin/bhyve/Makefile > head/usr.sbin/bhyve/bhyve.8 > head/usr.sbin/bhyve/virtio.h > > ... > Added: head/usr.sbin/bhyve/pci_virtio_scsi.c > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/usr.sbin/bhyve/pci_virtio_scsi.c Mon Jun 11 02:09:20 2018 (r334940) > @@ -0,0 +1,718 @@ > +/*- > + * SPDX-License-Identifier: BSD-2-Clause-FreeBSD > + * > + * Copyright (c) 2016 Jakub Klama . > + * Copyright (c) 2018 Marcelo Araujo . > + * All rights reserved. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer > + * in this position and unchanged. > + * 2. Redistributions in binary form must reproduce the above copyright > + * notice, this list of conditions and the following disclaimer in the > + * documentation and/or other materials provided with the distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND > + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE > + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE > + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL > + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT > + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY > + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > + * SUCH DAMAGE. > + */ > + > +#include > +__FBSDID("$FreeBSD$"); > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include "bhyverun.h" > +#include "pci_emul.h" > +#include "virtio.h" > +#include "iov.h" > + > +#define VTSCSI_RINGSZ 64 > +#define VTSCSI_REQUESTQ 1 > +#define VTSCSI_THR_PER_Q 16 > +#define VTSCSI_MAXQ (VTSCSI_REQUESTQ + 2) > +#define VTSCSI_MAXSEG 64 > + > +#define VTSCSI_IN_HEADER_LEN(_sc) \ > + (sizeof(struct pci_vtscsi_req_cmd_rd) + _sc->vss_config.cdb_size) > + > +#define VTSCSI_OUT_HEADER_LEN(_sc) \ > + (sizeof(struct pci_vtscsi_req_cmd_wr) + _sc->vss_config.sense_size) > + > +#define VIRTIO_SCSI_MAX_CHANNEL 0 > +#define VIRTIO_SCSI_MAX_TARGET 0 > +#define VIRTIO_SCSI_MAX_LUN 16383 > + > +#define VIRTIO_SCSI_F_INOUT (1 << 0) > +#define VIRTIO_SCSI_F_HOTPLUG (1 << 1) > +#define VIRTIO_SCSI_F_CHANGE (1 << 2) > + TABS missing inconsistently after #define above ... > +static int pci_vtscsi_debug = 0; > +#define DPRINTF(params) if (pci_vtscsi_debug) printf params > +#define WPRINTF(params) printf params > + > +struct pci_vtscsi_config { > + uint32_t num_queues; > + uint32_t seg_max; > + uint32_t max_sectors; > + uint32_t cmd_per_lun; > + uint32_t event_info_size; > + uint32_t sense_size; > + uint32_t cdb_size; > + uint16_t max_channel; > + uint16_t max_target; > + uint32_t max_lun; > +} __attribute__((packed)); > + > +struct pci_vtscsi_queue { > + struct pci_vtscsi_softc * vsq_sc; > + struct vqueue_info * vsq_vq; > + int vsq_ctl_fd; > + pthread_mutex_t vsq_mtx; > + pthread_mutex_t vsq_qmtx; > + pthread_cond_t vsq_cv; > + STAILQ_HEAD(, pci_vtscsi_request) vsq_requests; > + LIST_HEAD(, pci_vtscsi_worker) vsq_workers; > +}; > + > +struct pci_vtscsi_worker { > + struct pci_vtscsi_queue * vsw_queue; > + pthread_t vsw_thread; > + bool vsw_exiting; > + LIST_ENTRY(pci_vtscsi_worker) vsw_link; > +}; > + > +struct pci_vtscsi_request { > + struct pci_vtscsi_queue * vsr_queue; > + struct iovec vsr_iov_in[VTSCSI_MAXSEG]; > + int vsr_niov_in; > + struct iovec vsr_iov_out[VTSCSI_MAXSEG]; > + int vsr_niov_out; > + uint32_t vsr_idx; > + STAILQ_ENTRY(pci_vtscsi_request) vsr_link; > +}; > + > +/* > + * Per-device softc > + */ > +struct pci_vtscsi_softc { > + struct virtio_softc vss_vs; > + struct vqueue_info vss_vq[VTSCSI_MAXQ]; > + struct pci_vtscsi_queue vss_queues[VTSCSI_REQUESTQ]; > + pthread_mutex_t vss_mtx; > + int vss_iid; > + int vss_ctl_fd; > + uint32_t vss_features; > + struct pci_vtscsi_config vss_config; > +}; > + > +#define VIRTIO_SCSI_T_TMF 0 > +#define VIRTIO_SCSI_T_TMF_ABORT_TASK 0 > +#define VIRTIO_SCSI_T_TMF_ABORT_TASK_SET 1 > +#define VIRTIO_SCSI_T_TMF_CLEAR_ACA 2 > +#define VIRTIO_SCSI_T_TMF_CLEAR_TASK_SET 3 > +#define VIRTIO_SCSI_T_TMF_I_T_NEXUS_RESET 4 > +#define VIRTIO_SCSI_T_TMF_LOGICAL_UNIT_RESET 5 > +#define VIRTIO_SCSI_T_TMF_QUERY_TASK 6 > +#define VIRTIO_SCSI_T_TMF_QUERY_TASK_SET 7 > + > +/* command-specific response values */ > +#define VIRTIO_SCSI_S_FUNCTION_COMPLETE 0 > +#define VIRTIO_SCSI_S_FUNCTION_SUCCEEDED 10 > +#define VIRTIO_SCSI_S_FUNCTION_REJECTED 11 > + and here above ... > +struct pci_vtscsi_ctrl_tmf { > + uint32_t type; > + uint32_t subtype; > + uint8_t lun[8]; > + uint64_t id; > + uint8_t response; > +} __attribute__((packed)); > + > +#define VIRTIO_SCSI_T_AN_QUERY 1 > +#define VIRTIO_SCSI_EVT_ASYNC_OPERATIONAL_CHANGE 2 > +#define VIRTIO_SCSI_EVT_ASYNC_POWER_MGMT 4 > +#define VIRTIO_SCSI_EVT_ASYNC_EXTERNAL_REQUEST 8 > +#define VIRTIO_SCSI_EVT_ASYNC_MEDIA_CHANGE 16 > +#define VIRTIO_SCSI_EVT_ASYNC_MULTI_HOST 32 > +#define VIRTIO_SCSI_EVT_ASYNC_DEVICE_BUSY 64 > + > +struct pci_vtscsi_ctrl_an { > + uint32_t type; > + uint8_t lun[8]; > + uint32_t event_requested; > + uint32_t event_actual; > + uint8_t response; > +} __attribute__((packed)); > + > +/* command-specific response values */ > +#define VIRTIO_SCSI_S_OK 0 > +#define VIRTIO_SCSI_S_OVERRUN 1 > +#define VIRTIO_SCSI_S_ABORTED 2 > +#define VIRTIO_SCSI_S_BAD_TARGET 3 > +#define VIRTIO_SCSI_S_RESET 4 > +#define VIRTIO_SCSI_S_BUSY 5 > +#define VIRTIO_SCSI_S_TRANSPORT_FAILURE 6 > +#define VIRTIO_SCSI_S_TARGET_FAILURE 7 > +#define VIRTIO_SCSI_S_NEXUS_FAILURE 8 > +#define VIRTIO_SCSI_S_FAILURE 9 > +#define VIRTIO_SCSI_S_INCORRECT_LUN 12 > + > +/* task_attr */ > +#define VIRTIO_SCSI_S_SIMPLE 0 > +#define VIRTIO_SCSI_S_ORDERED 1 > +#define VIRTIO_SCSI_S_HEAD 2 > +#define VIRTIO_SCSI_S_ACA 3 > + And here ... Pedro.