From nobody Tue May 26 16:07:14 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4gPyLw2yGhz6g23f for ; Tue, 26 May 2026 16:07:20 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R13" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4gPyLw0g54z44wD for ; Tue, 26 May 2026 16:07:20 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1779811640; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=xxYTqboDFQXj1Ak2gtJQdoUnZREky0UO3BlNBNNQ6ro=; b=Evnxmph5iuY0Ics8MFLj0rIA6/6OqtkKSlj9ZDZM6RFid8w5grXL++oku2MkS2Fu0xLGkt odZM4tC1twaMJzC2okXQglTtP04/gQXrvHWbeUHpIxLgLc6+2/fH8yHPUnP4frumtX9DTx X9aqryYgAMrfkcU+b8bUzMovtj1cCqT+xawRg4Eq7LpG/ldQ8pViw6UwVcJbAkVSw5bQmR BNrk+CU+fphMLrzogD5lZ7D8zriNvkFN+Lukrv2OffBBu8yknk9fiAWTKBosOQ0VxbRoJ1 tc8WmfB7t1jp0ewrXUAouSwXund24IuXqyXZqNZTaRB2rjU+01PGxIf/Sz6hqg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1779811640; a=rsa-sha256; cv=none; b=Jsb8IVqGFnZsl1ywyzkbRtYiGB4pEaggY/DA8TjQ/4iPCiYfKlTdjXDM/d2QkKnUtsZWjU RC4h6h23IgXxHVKotHGNIIHmH9FoYnW/JrYvNap/g88O1DxIPPAwhDMHJ/JO8pgRDcWSiV MHaYIqDA9seFpuXXkudXdQxyyKm5s9i4gJl0GJEPDhu9bwgTn2bKNiHy4WMAbI41T0fkWo WGs7NoW5vnIh2ECTxaVsrzelE1ezvxjLCsZ7vBlIdkqLv0WGjus7jHJecd63veqs56qqHl zHW8lMKu2OmRNWmbbflzQ8DtCxtVASXyM79TJOopvssOdIGnEcSB11YsxoVv9A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1779811640; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=xxYTqboDFQXj1Ak2gtJQdoUnZREky0UO3BlNBNNQ6ro=; b=cTndyjMygBrmBWVaE0lU4Ws15Hq7LqSxJgOresYM34vmvw7mX294xd8C+ezdaihji8L7HF U8GBoZ5/u+W/UmwinHjoQrJXQHvfOpxLJvVKkeWzYAvjHT9zQvPwjbue2Yy794DMTEyBAz 6aylVf6+w5dMUKxVlLAUkZcd94nVWR76rLyzC2R1T1lfrbVrxikS6GfQdXLo8d2gD7wYZm AEorxD0pGH5zJIVpnNlMBwldhTzMyy4XI5O8xywo7UeDQJjPv+32jwuOnVjf/AXUG0SpAF PRMs0WVedWOwMfHJs09awILXisOzT6ErrCfYrLxnCXIwnhtklqp2rgCEmugZIg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4gPyLw04cpz41M for ; Tue, 26 May 2026 16:07:20 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 320fa by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Tue, 26 May 2026 16:07:14 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Cc: Hans Rosenfeld From: Mark Johnston Subject: git: 4151296fdcba - main - bhyve/virtio-scsi: Support for multiple targets List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 4151296fdcbad2706936ffa7d428d8621ad11823 Auto-Submitted: auto-generated Date: Tue, 26 May 2026 16:07:14 +0000 Message-Id: <6a15c532.320fa.65b8671f@gitrepo.freebsd.org> The branch main has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=4151296fdcbad2706936ffa7d428d8621ad11823 commit 4151296fdcbad2706936ffa7d428d8621ad11823 Author: Hans Rosenfeld AuthorDate: 2025-10-29 11:12:09 +0000 Commit: Mark Johnston CommitDate: 2026-05-26 16:03:12 +0000 bhyve/virtio-scsi: Support for multiple targets Currently, virtio-scsi supports only one target with 16383 LUNs, which fits nicely with what CTL provides. It would be useful to support more than that, multiple targets each with 16383 LUNs. While this can be useful with CTL by attaching each target to another CTL target port, this will be necessary to support SCSI passthrough. The new syntax for configuring targets will look like this: -s X,virtio-scsi,target=/dev/foo,target=/dev/bar,target=4:/dev/baz This will create the following configuration nodes: pci.0.X.0.device=virtio-scsi pci.0.X.0.target.0=/dev/foo pci.0.X.0.target.1=/dev/bar pci.0.X.0.target.4=/dev/baz The existing configuration syntax is still understood for compatibility: (1) -s X,virtio-scsi (2) -s X,virtio-scsi,/dev/foo (3) -s X,virtio-scsi,dev=/dev/foo This will create the following configuration nodes: pci.0.X.0.device=virtio-scsi (1, 2, 3) pci.0.X.0.target.0=/dev/cam/ctl (1) pci.0.X.0.target.0=/dev/foo (2, 3) Reviewed by: markj MFC after: 1 month Differential Revision: https://reviews.freebsd.org/D53221 --- usr.sbin/bhyve/bhyve.8 | 38 ++++- usr.sbin/bhyve/bhyve_config.5 | 28 ++- usr.sbin/bhyve/pci_virtio_scsi.c | 361 ++++++++++++++++++++++++++++++++++----- 3 files changed, 378 insertions(+), 49 deletions(-) diff --git a/usr.sbin/bhyve/bhyve.8 b/usr.sbin/bhyve/bhyve.8 index 5118974baace..b5b03dd9c211 100644 --- a/usr.sbin/bhyve/bhyve.8 +++ b/usr.sbin/bhyve/bhyve.8 @@ -2,6 +2,7 @@ .\" SPDX-License-Identifier: BSD-2-Clause .\" .\" Copyright (c) 2013 Peter Grehan +.\" Copyright (c) 2026 Hans Rosenfeld .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without @@ -680,10 +681,45 @@ In that case, this feature doesn't work as expected. .Bl -bullet .Sm off .It -.Pa /dev/cam/ctl Oo Ar pp Cm \&. Ar vp Oc Oo Cm \&, Ar scsi-device-options Oc +.Oo Cm target Ns = Ns Oo ID : Oc Ar /dev/cam/ctl Oo Ar pp Cm \&. Ar vp Oc Oc +.Oo Cm \&, Ar scsi-device-options Oc .Sm on .El .Pp +Multiple +.Pa target +parameters may be specified, each configuring a different +.Ar path +as a distinct SCSI target. +If the +.Pa target +.Ar ID +is not explicitly configured for a +.Pa target , +the +.Pa target +will be assigned the next sequential +.Ar ID +following the highest +.Pa target +.Ar ID +used at that point, or 0 if it is the first target configured. +All +.Pa target +.Ar ID Ns s +must be unique per instance. +The +.Ar path +must point to a valid CAM target layer +.Po CTL +.Pc +device node. +If no +.Pa target +is configured, a single default target backed by +.Sy /dev/cam/ctl +will be created. +.Pp The .Ar scsi-device-options are: diff --git a/usr.sbin/bhyve/bhyve_config.5 b/usr.sbin/bhyve/bhyve_config.5 index 429ce3e38138..ea3157cb54d6 100644 --- a/usr.sbin/bhyve/bhyve_config.5 +++ b/usr.sbin/bhyve/bhyve_config.5 @@ -1,6 +1,7 @@ .\" SPDX-License-Identifier: BSD-2-Clause .\" .\" Copyright (c) 2021 John H. Baldwin +.\" Copyright 2025 Hans Rosenfeld .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions @@ -761,14 +762,33 @@ If specified, it must be a unicast MAC address. The largest supported MTU advertised to the guest. .El .Ss VirtIO SCSI Settings -.Bl -column "Name" "integer" "Default" +.Bl -column "target" "integer" "/dev/cam/ctl" .It Sy Name Ta Sy Format Ta Sy Default Ta Sy Description -.It Va dev Ta path Ta Ta -The path of a CAM target layer (CTL) device to export: -.Pa /dev/cam/ctl Ns Oo Ar pp . Ns Ar vp Oc . .It Va iid Ta integer Ta 0 Ta Initiator ID to use when sending requests to the CTL port. +.It Va target Ta Oo Va ID : Oc Ns path Ta Sy /dev/cam/ctl Ta +The path of a CAM target layer (CTL) device to use: +.Pa /dev/cam/ctl Ns Oo Ar pp . Ns Ar vp Oc +Optionally, a numeric target +.Ar ID +in the range from 0 to 255 may be specified before the +.Ar path , +separated by a colon. .El +.Pp +The +.Va target +variable may be specified multiple times with different +.Ar path +arguments to configure multiple distinct SCSI targets. +If not explicitly configured, the target +.Ar ID Ns s +will be assigned sequentially beginning with the highest target +.Ar ID +configured so far, or 0 for the first target configured. +The target +.Ar ID Ns s +must be unique within each virtio-scsi instance. .Sh SEE ALSO .Xr expand_number 3 , .Xr getaddrinfo 3 , diff --git a/usr.sbin/bhyve/pci_virtio_scsi.c b/usr.sbin/bhyve/pci_virtio_scsi.c index 5fb867e5eae7..d7fd30f55209 100644 --- a/usr.sbin/bhyve/pci_virtio_scsi.c +++ b/usr.sbin/bhyve/pci_virtio_scsi.c @@ -78,7 +78,7 @@ (sizeof(struct pci_vtscsi_req_cmd_wr) + _sc->vss_config.sense_size) #define VIRTIO_SCSI_MAX_CHANNEL 0 -#define VIRTIO_SCSI_MAX_TARGET 0 +#define VIRTIO_SCSI_MAX_TARGET 255 #define VIRTIO_SCSI_MAX_LUN 16383 #define VIRTIO_SCSI_F_INOUT (1 << 0) @@ -170,6 +170,12 @@ struct pci_vtscsi_request { STAILQ_ENTRY(pci_vtscsi_request) vsr_link; }; +struct pci_vtscsi_target { + uint8_t vst_target; + int vst_fd; + int vst_max_sectors; +}; + /* * Per-device softc */ @@ -181,7 +187,9 @@ struct pci_vtscsi_softc { int vss_iid; int vss_ctl_fd; uint32_t vss_features; + size_t vss_num_target; struct pci_vtscsi_config vss_config; + struct pci_vtscsi_target *vss_targets; }; #define VIRTIO_SCSI_T_TMF 0 @@ -272,8 +280,12 @@ static void pci_vtscsi_neg_features(void *, uint64_t); static int pci_vtscsi_cfgread(void *, int, int, uint32_t *); static int pci_vtscsi_cfgwrite(void *, int, int, uint32_t); -static inline bool pci_vtscsi_check_lun(const uint8_t *); -static inline int pci_vtscsi_get_lun(const uint8_t *); +static inline bool pci_vtscsi_check_lun(struct pci_vtscsi_softc *, + const uint8_t *); +static inline uint8_t pci_vtscsi_get_target(struct pci_vtscsi_softc *, + const uint8_t *); +static inline uint16_t pci_vtscsi_get_lun(struct pci_vtscsi_softc *, + const uint8_t *); static void pci_vtscsi_control_handle(struct pci_vtscsi_softc *, void *, size_t); static void pci_vtscsi_tmf_handle(struct pci_vtscsi_softc *, @@ -292,13 +304,15 @@ static void pci_vtscsi_queue_request(struct pci_vtscsi_softc *, struct vqueue_info *); static void pci_vtscsi_return_request(struct pci_vtscsi_queue *, struct pci_vtscsi_request *, int); -static int pci_vtscsi_request_handle(struct pci_vtscsi_softc *, +static int pci_vtscsi_request_handle(struct pci_vtscsi_softc *, int, struct pci_vtscsi_request *); static void pci_vtscsi_controlq_notify(void *, struct vqueue_info *); static void pci_vtscsi_eventq_notify(void *, struct vqueue_info *); static void pci_vtscsi_requestq_notify(void *, struct vqueue_info *); -static int pci_vtscsi_init_queue(struct pci_vtscsi_softc *, + +static int pci_vtscsi_add_target_config(nvlist_t *, const char *, int); +static int pci_vtscsi_init_queue(struct pci_vtscsi_softc *, struct pci_vtscsi_queue *, int); static void pci_vtscsi_destroy_queue(struct pci_vtscsi_queue *); static int pci_vtscsi_init(struct pci_devinst *, nvlist_t *); @@ -320,10 +334,12 @@ pci_vtscsi_proc(void *arg) struct pci_vtscsi_worker *worker = (struct pci_vtscsi_worker *)arg; struct pci_vtscsi_queue *q = worker->vsw_queue; struct pci_vtscsi_softc *sc = q->vsq_sc; - int iolen; for (;;) { struct pci_vtscsi_request *req; + uint8_t target; + int iolen; + int fd; pthread_mutex_lock(&q->vsq_rmtx); @@ -338,11 +354,15 @@ pci_vtscsi_proc(void *arg) req = pci_vtscsi_get_request(&q->vsq_requests); pthread_mutex_unlock(&q->vsq_rmtx); - DPRINTF("I/O request lun %d, data_niov_in %zu, data_niov_out " - "%zu", pci_vtscsi_get_lun(req->vsr_cmd_rd->lun), + target = pci_vtscsi_get_target(sc, req->vsr_cmd_rd->lun); + fd = sc->vss_targets[target].vst_fd; + + DPRINTF("I/O request tgt %u, lun %d, data_niov_in %zu, " + "data_niov_out %zu", target, + pci_vtscsi_get_lun(sc, req->vsr_cmd_rd->lun), req->vsr_data_niov_in, req->vsr_data_niov_out); - iolen = pci_vtscsi_request_handle(sc, req); + iolen = pci_vtscsi_request_handle(sc, fd, req); pci_vtscsi_return_request(q, req, iolen); } @@ -370,7 +390,7 @@ pci_vtscsi_reset(void *vsc) .sense_size = 96, .cdb_size = 32, .max_channel = VIRTIO_SCSI_MAX_CHANNEL, - .max_target = VIRTIO_SCSI_MAX_TARGET, + .max_target = MAX(1, sc->vss_num_target) - 1, .max_lun = VIRTIO_SCSI_MAX_LUN }; } @@ -418,7 +438,6 @@ pci_vtscsi_cfgwrite(void *vsc __unused, int offset __unused, int size __unused, * or: Flat Space Addressing: LUN (0-16383) * Level 3 and 4: not used, MBZ * - * Currently, we only support Target 0. * * Alternatively, the first level may contain an extended LUN address to select * the REPORT_LUNS well-known logical unit: @@ -439,7 +458,7 @@ pci_vtscsi_cfgwrite(void *vsc __unused, int offset __unused, int size __unused, * well-known logical unit. */ static inline bool -pci_vtscsi_check_lun(const uint8_t *lun) +pci_vtscsi_check_lun(struct pci_vtscsi_softc *sc, const uint8_t *lun) { if (lun[0] == 0xC1) return (false); @@ -447,7 +466,13 @@ pci_vtscsi_check_lun(const uint8_t *lun) if (lun[0] != 0x01) return (false); - if (lun[1] != 0x00) + if (lun[1] >= sc->vss_num_target) + return (false); + + if (lun[1] != sc->vss_targets[lun[1]].vst_target) + return (false); + + if (sc->vss_targets[lun[1]].vst_fd < 0) return (false); if (lun[2] != 0x00 && (lun[2] & 0xc0) != 0x40) @@ -459,17 +484,37 @@ pci_vtscsi_check_lun(const uint8_t *lun) return (true); } +/* + * Get the target id from a LUN address. + * + * Every code path using this function must have called pci_vtscsi_check_lun() + * before to make sure the LUN address is valid. + */ +static inline uint8_t +pci_vtscsi_get_target(struct pci_vtscsi_softc *sc, const uint8_t *lun) +{ + assert(lun[0] == 0x01); + assert(lun[1] < sc->vss_num_target); + assert(lun[1] == sc->vss_targets[lun[1]].vst_target); + assert(sc->vss_targets[lun[1]].vst_fd >= 0); + assert(lun[2] == 0x00 || (lun[2] & 0xc0) == 0x40); + + return (lun[1]); +} + /* * Get the LUN id from a LUN address. * * Every code path using this function must have called pci_vtscsi_check_lun() * before to make sure the LUN address is valid. */ -static inline int -pci_vtscsi_get_lun(const uint8_t *lun) +static inline uint16_t +pci_vtscsi_get_lun(struct pci_vtscsi_softc *sc, const uint8_t *lun) { assert(lun[0] == 0x01); - assert(lun[1] == 0x00); + assert(lun[1] < sc->vss_num_target); + assert(lun[1] == sc->vss_targets[lun[1]].vst_target); + assert(sc->vss_targets[lun[1]].vst_fd >= 0); assert(lun[2] == 0x00 || (lun[2] & 0xc0) == 0x40); return (((lun[2] << 8) | lun[3]) & 0x3fff); @@ -512,9 +557,11 @@ pci_vtscsi_tmf_handle(struct pci_vtscsi_softc *sc, struct pci_vtscsi_ctrl_tmf *tmf) { union ctl_io *io; + uint8_t target; int err; + int fd; - if (pci_vtscsi_check_lun(tmf->lun) == false) { + if (pci_vtscsi_check_lun(sc, tmf->lun) == false) { DPRINTF("TMF request to invalid LUN %.2hhx%.2hhx-%.2hhx%.2hhx-" "%.2hhx%.2hhx-%.2hhx%.2hhx", tmf->lun[0], tmf->lun[1], tmf->lun[2], tmf->lun[3], tmf->lun[4], tmf->lun[5], @@ -524,6 +571,10 @@ pci_vtscsi_tmf_handle(struct pci_vtscsi_softc *sc, return; } + target = pci_vtscsi_get_target(sc, tmf->lun); + + fd = sc->vss_targets[target].vst_fd; + io = ctl_scsi_alloc_io(sc->vss_iid); if (io == NULL) { WPRINTF("failed to allocate ctl_io: err=%d (%s)", @@ -537,7 +588,7 @@ pci_vtscsi_tmf_handle(struct pci_vtscsi_softc *sc, io->io_hdr.io_type = CTL_IO_TASK; io->io_hdr.nexus.initid = sc->vss_iid; - io->io_hdr.nexus.targ_lun = pci_vtscsi_get_lun(tmf->lun); + io->io_hdr.nexus.targ_lun = pci_vtscsi_get_lun(sc, tmf->lun); io->taskio.tag_type = CTL_TAG_SIMPLE; io->taskio.tag_num = tmf->id; io->io_hdr.flags |= CTL_FLAG_USER_TAG; @@ -584,7 +635,7 @@ pci_vtscsi_tmf_handle(struct pci_vtscsi_softc *sc, sbuf_delete(sb); } - err = ioctl(sc->vss_ctl_fd, CTL_IO, io); + err = ioctl(fd, CTL_IO, io); if (err != 0) WPRINTF("CTL_IO: err=%d (%s)", errno, strerror(errno)); @@ -753,7 +804,7 @@ pci_vtscsi_queue_request(struct pci_vtscsi_softc *sc, struct vqueue_info *vq) assert(res == VTSCSI_IN_HEADER_LEN(q->vsq_sc)); /* Make sure this request addresses a valid LUN. */ - if (pci_vtscsi_check_lun(req->vsr_cmd_rd->lun) == false) { + if (pci_vtscsi_check_lun(sc, req->vsr_cmd_rd->lun) == false) { DPRINTF("I/O request to invalid LUN " "%.2hhx%.2hhx-%.2hhx%.2hhx-%.2hhx%.2hhx-%.2hhx%.2hhx", req->vsr_cmd_rd->lun[0], req->vsr_cmd_rd->lun[1], @@ -809,7 +860,7 @@ pci_vtscsi_return_request(struct pci_vtscsi_queue *q, } static int -pci_vtscsi_request_handle(struct pci_vtscsi_softc *sc, +pci_vtscsi_request_handle(struct pci_vtscsi_softc *sc, int fd, struct pci_vtscsi_request *req) { union ctl_io *io = req->vsr_ctl_io; @@ -818,7 +869,8 @@ pci_vtscsi_request_handle(struct pci_vtscsi_softc *sc, int err, nxferred; io->io_hdr.nexus.initid = sc->vss_iid; - io->io_hdr.nexus.targ_lun = pci_vtscsi_get_lun(req->vsr_cmd_rd->lun); + io->io_hdr.nexus.targ_lun = + pci_vtscsi_get_lun(sc, req->vsr_cmd_rd->lun); io->io_hdr.io_type = CTL_IO_SCSI; @@ -869,7 +921,7 @@ pci_vtscsi_request_handle(struct pci_vtscsi_softc *sc, sbuf_delete(sb); } - err = ioctl(sc->vss_ctl_fd, CTL_IO, io); + err = ioctl(fd, CTL_IO, io); if (err != 0) { WPRINTF("CTL_IO: err=%d (%s)", errno, strerror(errno)); req->vsr_cmd_wr->response = VIRTIO_SCSI_S_FAILURE; @@ -1006,32 +1058,222 @@ pci_vtscsi_destroy_queue(struct pci_vtscsi_queue *queue) pthread_mutex_destroy(&queue->vsq_rmtx); } +/* + * Create a target config node, return target id. If the target number isn't + * given as part of the path argument, use last_id + 1. + */ +static int +pci_vtscsi_add_target_config(nvlist_t *nvl, const char *path, int last_id) +{ + uint64_t target; + char *id; + char tmp[4]; + + if (path == NULL) { + EPRINTLN("target path must be specified"); + return (-1); + } + + if (path[0] != '/' && (id = strchr(path, ':')) != NULL) { + const char *errstr; + int len = id - path; + + id = strndup(path, len); + if (id == NULL) { + EPRINTLN("failed to get id string: %s", + strerror(errno)); + return (-1); + } + + target = strtonumx(id, 0, VIRTIO_SCSI_MAX_TARGET, &errstr, 0); + if (errstr != NULL) { + EPRINTLN("invalid target %s: target ID is %s", id, + errstr); + free(id); + return (-1); + } + + free(id); + path += len + 1; + } else { + target = last_id + 1; + + if (target > VIRTIO_SCSI_MAX_TARGET) { + EPRINTLN("max target (%d) reached, can't add another", + VIRTIO_SCSI_MAX_TARGET); + return (-1); + } + } + + snprintf(tmp, sizeof(tmp), "%lu", target); + + if (get_config_value_node(nvl, tmp) != NULL) { + EPRINTLN("cannot add '%s' as target %s: already exits as '%s'", + path, tmp, get_config_value_node(nvl, tmp)); + return (-1); + } + + set_config_value_node(nvl, tmp, path); + + return (target); +} + +/* + * The following forms are accepted for legacy config options to configure a + * single target: + * + * (0) -s B:D:F,virtio-scsi + * (1) -s B:D:F,virtio-scsi, + * (2) -s B:D:F,virtio-scsi,,,... + * (3) -s B:D:F,virtio-scsi,,... + * (4) -s B:D:F,virtio-scsi, + * + * To configure multiple targets, the following form is accepted: + * (5) -s B:D:F,virtio-scsi,[target=[id:],...] + */ static int pci_vtscsi_legacy_config(nvlist_t *nvl, const char *opts) { - char *cp, *devname; + int last_id = -1; + char *config, *tofree, *name, *value; + nvlist_t *targets; + size_t n; - if (opts == NULL) + /* Make sure no one accidentally sets "dev" anymore. */ + (void) create_relative_config_node(nvl, "dev"); + + targets = create_relative_config_node(nvl, "target"); + + /* Handle legacy form (0). */ + if (opts == NULL) { + pci_vtscsi_add_target_config(targets, "/dev/cam/ctl", 0); return (0); + } + + n = strcspn(opts, ",="); + + /* Handle legacy form (1) and (2). */ + if (opts[n] == ',' || opts[n] == '\0') { + char *tmp = strndup(opts, n); + + last_id = pci_vtscsi_add_target_config(targets, tmp, last_id); + free(tmp); + + if (last_id < 0) + return (-1); - cp = strchr(opts, ','); - if (cp == NULL) { - set_config_value_node(nvl, "dev", opts); + opts += n; + if (opts[0] == ',' && opts[1] != '\0') + opts++; + } + + /* If this was form (1), we're done. */ + if (opts[0] == '\0') return (0); + + /* + * For form (2), (3), (4), and (5), parse the remaining options. + * + * Contrary to other options, multiple target= options create a new + * target for each such option. + * + * For compatibility reasons we also accept dev= options for + * targets. + */ + config = tofree = strdup(opts); + while ((name = strsep(&config, ",")) != NULL) { + value = strchr(name, '='); + if (value != NULL) + *value++ = '\0'; + + if (strcmp(name, "dev") == 0 || strcmp(name, "target") == 0) { + int new_id = pci_vtscsi_add_target_config(targets, + value, last_id); + + if (new_id < 0) { + free(tofree); + return (-1); + } + + if (new_id > last_id) + last_id = new_id; + + } else if (value != NULL) { + set_config_value_node(nvl, name, value); + } else { + set_config_bool_node(nvl, name, true); + } } - devname = strndup(opts, cp - opts); - set_config_value_node(nvl, "dev", devname); - free(devname); - return (pci_parse_legacy_config(nvl, cp + 1)); + + free(tofree); + return (0); +} + +static int +pci_vtscsi_count_targets(const char *prefix __unused, + const nvlist_t *parent __unused, const char *name, int type, void *arg) +{ + struct pci_vtscsi_softc *sc = arg; + const char *errstr; + uint64_t target; + + if (type != NV_TYPE_STRING) { + EPRINTLN("invalid target \"%s\" type: not a string", name); + errno = EINVAL; + return (-1); + } + + target = strtonumx(name, 0, VIRTIO_SCSI_MAX_TARGET, &errstr, 0); + if (errstr != NULL) { + EPRINTLN("invalid target %s: target ID is %s", name, errstr); + return (-1); + } + + if (target >= sc->vss_num_target) + sc->vss_num_target = target + 1; + + return (0); +} + +static int +pci_vtscsi_init_target(const char *prefix __unused, const nvlist_t *parent, + const char *name, int type, void *arg) +{ + struct pci_vtscsi_softc *sc = arg; + const char *value; + const char *errstr; + uint64_t target; + + assert(type == NV_TYPE_STRING); + + /* + * Get the numeric value of the target id from 'name'. + */ + target = strtonumx(name, 0, sc->vss_num_target - 1, &errstr, 0); + assert(errstr == NULL); + sc->vss_targets[target].vst_target = target; + + /* + * 'value' contains the CTL device node path of this target. + */ + value = nvlist_get_string(parent, name); + sc->vss_targets[target].vst_fd = open(value, O_RDWR); + if (sc->vss_targets[target].vst_fd < 0) { + EPRINTLN("cannot open target %lu at %s: %s", target, value, + strerror(errno)); + return (-1); + } + + return (0); } static int pci_vtscsi_init(struct pci_devinst *pi, nvlist_t *nvl) { struct pci_vtscsi_softc *sc; - const char *devname, *value; + const char *value; + size_t i; int err; - int i; sc = calloc(1, sizeof(struct pci_vtscsi_softc)); if (sc == NULL) @@ -1045,19 +1287,42 @@ pci_vtscsi_init(struct pci_devinst *pi, nvlist_t *nvl) if (value != NULL) { if (pci_emul_add_boot_device(pi, atoi(value))) { EPRINTLN("Invalid bootindex %d", atoi(value)); + errno = EINVAL; goto fail; } } - devname = get_config_value_node(nvl, "dev"); - if (devname == NULL) - devname = "/dev/cam/ctl"; - sc->vss_ctl_fd = open(devname, O_RDWR); - if (sc->vss_ctl_fd < 0) { - WPRINTF("cannot open %s: %s", devname, strerror(errno)); - goto fail; + nvl = find_relative_config_node(nvl, "target"); + if (nvl != NULL) { + err = walk_config_nodes("", nvl, sc, pci_vtscsi_count_targets); + if (err != 0) + goto fail; + } + + if (sc->vss_num_target > 0) { + sc->vss_targets = malloc(sc->vss_num_target * + sizeof(struct pci_vtscsi_target)); + if (sc->vss_targets == NULL) { + EPRINTLN("can't allocate space for %lu targets", + sc->vss_num_target); + goto fail; + } + + memset(sc->vss_targets, -1, sc->vss_num_target * + sizeof(struct pci_vtscsi_target)); + + err = walk_config_nodes("", nvl, sc, pci_vtscsi_init_target); + if (err != 0) + goto fail; } + /* + * All targets should be open now and have a valid fd. + */ + for (i = 0; i < sc->vss_num_target; i++) + if (sc->vss_targets[i].vst_target == i) + assert(sc->vss_targets[i].vst_fd > 0); + pthread_mutex_init(&sc->vss_mtx, NULL); vi_softc_linkup(&sc->vss_vs, &vtscsi_vi_consts, sc, pi, sc->vss_vq); @@ -1101,7 +1366,8 @@ pci_vtscsi_init(struct pci_devinst *pi, nvlist_t *nvl) pci_set_cfgdata16(pi, PCIR_SUBDEV_0, VIRTIO_ID_SCSI); pci_set_cfgdata16(pi, PCIR_SUBVEND_0, VIRTIO_VENDOR); - if (vi_intr_init(&sc->vss_vs, 1, fbsdrun_virtio_msix())) + err = vi_intr_init(&sc->vss_vs, 1, fbsdrun_virtio_msix()); + if (err != 0) goto fail; vi_set_io_bar(&sc->vss_vs, 0); @@ -1112,9 +1378,16 @@ fail: for (i = 2; i < VTSCSI_MAXQ; i++) pci_vtscsi_destroy_queue(&sc->vss_queues[i - 2]); - if (sc->vss_ctl_fd > 0) - close(sc->vss_ctl_fd); + pthread_mutex_destroy(&sc->vss_mtx); + + for (i = 0; i < sc->vss_num_target; i++) { + if (sc->vss_targets[i].vst_target == i && + sc->vss_targets[i].vst_fd >= 0) { + close(sc->vss_targets[i].vst_fd); + } + } + free(sc->vss_targets); free(sc); return (-1); }