From nobody Fri May 3 00:15:55 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VVrtg58kpz5KK1y; Fri, 3 May 2024 00:15:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VVrtg2S89z4QrT; Fri, 3 May 2024 00:15:55 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1714695355; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=/flzVVeuHpdzV5qFfKmwGjh1foAlemKQXFuo+wQ9YDM=; b=wcsTkGa/Y0g5ERI2lgZes94MZYpCYpvleJOarunsosloTZXG3JtEDqx6x7jI5g9nZ1xqE+ SHpDtVXHlckvTOx24lP+ayniPVnOB1ZpLd9ZxdxfEjwhboPiLCNNw6eUvLQnn5dwCXri6Y gQSl5NH9xMcNFURCE66WqiNqKvi5MjvE1GWBHnVMYxqYL0ltzQ4kQYL6bbDOJ8P1FvN+RG y5z1HcOslEfYEM3SvVppfz8bULsi9l0fCfJ7vf3Fhl7jhnTwbEYlNeTWX+PlAlHGwNhpfx 9ElF1rjOfoHT+xJ2IrZ5Ktdw/QrvXVE6wTtxHWhB6kKFWwPp+iC8n03ycxvPhQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1714695355; a=rsa-sha256; cv=none; b=PJF7lCVD+JncYhfHP1PqFOWgCkwH505vyVBI4RmAgJSMb7Lc9q9SGmf3BjNOyKPhdjKWYK 27JsRI2OYA/mzqcvUOektoba8xLtR6KzQzha38wIy7cUopbo7rIANtyOyisDGD8nwmoeBY 92UGE5haeYYtpv2iGXgc0qP9Z0CgfDiEDyuEVVj04SliUSWjjY/nf6WCkfA/piDWPrpPDz ADB09xgyrV45ToUhPaMyKnL+JFDzbNe1ygeoGkXOY4cfvCAJ9xj5WMFxjCs7092IPDvH+a rUT2M9IYVnbJtnIyl9IhCMZyrUKaJqhvivbSbd1C1y9XPKBqXIqmb7HAprKVeQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1714695355; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=/flzVVeuHpdzV5qFfKmwGjh1foAlemKQXFuo+wQ9YDM=; b=gGAmbItDhJDLraFWcSansgtVK4nBJLKSqzmb0Tyig1+uru+aSOB2QJzcC1p7+GpfSb0uuY BQCLvV6BKkwpgfsr26UZl17sz+k2UjV+iZOvI9ZZ4xytc/ooISGqlt61/WTcoYAC5Nvhsl LVx9orWCAM67XZFA7hCMNabgF8X0gCzgjjj49z2Hr4D+4QGHNyAM5Hig8XqQ03e9pYobBm ijYWi0KPMcoaYqrYV1Fe8m4EPdkGOMJOsyJXRd8b515RgUswP2+U0lFLYpwqKZFK78QfBC 1r4r5zt5ewd0UMBi1sJvrv6Ft6dLjGU8DLu/Fi5OzwrG7NXbWgDY2j67QL3DaQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4VVrtg1zr3zNct; Fri, 3 May 2024 00:15:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 4430Ft4d079479; Fri, 3 May 2024 00:15:55 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 4430FtWo079476; Fri, 3 May 2024 00:15:55 GMT (envelope-from git) Date: Fri, 3 May 2024 00:15:55 GMT Message-Id: <202405030015.4430FtWo079476@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: 59657816b9e8 - main - ctl: Add structure and related constants for NVMe commands List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 59657816b9e8bb7e51c433d38bbf9fc3e474068e Auto-Submitted: auto-generated The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=59657816b9e8bb7e51c433d38bbf9fc3e474068e commit 59657816b9e8bb7e51c433d38bbf9fc3e474068e Author: John Baldwin AuthorDate: 2024-05-02 23:31:11 +0000 Commit: John Baldwin CommitDate: 2024-05-02 23:31:11 +0000 ctl: Add structure and related constants for NVMe commands This includes static inline functions to serve as getters/setters for fields shared between SCSI and NVMe I/O requests to manage data buffers. Reviewed by: ken, imp Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D44717 --- sys/cam/ctl/ctl_io.h | 337 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 335 insertions(+), 2 deletions(-) diff --git a/sys/cam/ctl/ctl_io.h b/sys/cam/ctl/ctl_io.h index 62ac05713b47..6ce3970a0500 100644 --- a/sys/cam/ctl/ctl_io.h +++ b/sys/cam/ctl/ctl_io.h @@ -47,6 +47,7 @@ #include #include +#include #define CTL_MAX_CDBLEN 32 /* @@ -74,6 +75,7 @@ typedef enum { CTL_SEL_TIMEOUT, /* Selection timeout, shouldn't happen here */ CTL_ERROR, /* General CTL error XXX expand on this? */ CTL_SCSI_ERROR, /* SCSI error, look at status byte/sense data */ + CTL_NVME_ERROR, /* NVMe error, look at NVMe completion */ CTL_CMD_ABORTED, /* Command aborted, don't return status */ CTL_STATUS_MASK = 0xfff,/* Mask off any status flags */ CTL_AUTOSENSE = 0x1000 /* Autosense performed */ @@ -192,6 +194,8 @@ typedef enum { CTL_IO_NONE, CTL_IO_SCSI, CTL_IO_TASK, + CTL_IO_NVME, + CTL_IO_NVME_ADMIN, } ctl_io_type; struct ctl_nexus { @@ -263,6 +267,8 @@ typedef enum { union ctl_io; typedef void (*ctl_ref)(void *arg, int diff); +typedef int (*ctl_be_move_done_t)(union ctl_io *io, bool samethr); +typedef int (*ctl_io_cont)(union ctl_io *io); /* * SCSI passthrough I/O structure for the CAM Target Layer. Note @@ -334,8 +340,8 @@ struct ctl_scsiio { ctl_tag_type tag_type; /* simple, ordered, head of queue,etc.*/ uint8_t cdb_len; /* CDB length */ uint8_t cdb[CTL_MAX_CDBLEN]; /* CDB */ - int (*be_move_done)(union ctl_io *io, bool samethr); /* called by fe */ - int (*io_cont)(union ctl_io *io); /* to continue processing */ + ctl_be_move_done_t be_move_done; /* called by fe */ + ctl_io_cont io_cont; /* to continue processing */ ctl_ref kern_data_ref; /* Method to reference/release data */ void *kern_data_arg; /* Opaque argument for kern_data_ref() */ }; @@ -380,6 +386,72 @@ struct ctl_taskio { uint8_t task_resp[3];/* Response information */ }; +/* + * NVME passthrough I/O structure for the CAM Target Layer. Note that + * this structure is used for both I/O and admin commands. + * + * Note: Make sure the io_hdr is *always* the first element in this + * structure. + */ +struct ctl_nvmeio { + struct ctl_io_hdr io_hdr; /* common to all I/O types */ + + /* + * The ext_* fields are generally intended for frontend use; CTL itself + * doesn't modify or use them. + */ + uint32_t ext_sg_entries; /* 0 = no S/G list, > 0 = num entries */ + uint8_t *ext_data_ptr; /* data buffer or S/G list */ + uint32_t ext_data_len; /* Data transfer length */ + uint32_t ext_data_filled; /* Amount of data filled so far */ + + /* + * The number of scatter/gather entries in the list pointed to + * by kern_data_ptr. 0 means there is no list, just a data pointer. + */ + uint32_t kern_sg_entries; + + /* + * The data pointer or a pointer to the scatter/gather list. + */ + uint8_t *kern_data_ptr; + + /* + * Length of the data buffer or scatter/gather list. It's also + * the length of this particular piece of the data transfer, + * ie. number of bytes expected to be transferred by the current + * invocation of frontend's datamove() callback. It's always + * less than or equal to kern_total_len. + */ + uint32_t kern_data_len; + + /* + * Total length of data to be transferred during this particular + * NVMe command, as decoded from the NVMe SQE. + */ + uint32_t kern_total_len; + + /* + * Amount of data left after the current data transfer. + */ + uint32_t kern_data_resid; + + /* + * Byte offset of this transfer, equal to the amount of data + * already transferred for this NVMe command during previous + * datamove() invocations. + */ + uint32_t kern_rel_offset; + + struct nvme_command cmd; /* SQE */ + struct nvme_completion cpl; /* CQE */ + bool success_sent; /* datamove already sent CQE */ + ctl_be_move_done_t be_move_done; /* called by fe */ + ctl_io_cont io_cont; /* to continue processing */ + ctl_ref kern_data_ref; /* Method to reference/release data */ + void *kern_data_arg; /* Opaque argument for kern_data_ref() */ +}; + /* * HA link messages. */ @@ -588,6 +660,7 @@ union ctl_io { struct ctl_io_hdr io_hdr; /* common to all I/O types */ struct ctl_scsiio scsiio; /* Normal SCSI commands */ struct ctl_taskio taskio; /* SCSI task management/reset */ + struct ctl_nvmeio nvmeio; /* Normal and admin NVMe commands */ struct ctl_prio presio; /* update per. res info on other SC */ }; @@ -608,6 +681,266 @@ union ctl_io { _CTL_IO_ASSERT_MACRO(__VA_ARGS__, _CTL_IO_ASSERT_2, \ _CTL_IO_ASSERT_1)(__VA_ARGS__) +static __inline uint32_t +ctl_kern_sg_entries(union ctl_io *io) +{ + switch (io->io_hdr.io_type) { + case CTL_IO_SCSI: + return (io->scsiio.kern_sg_entries); + case CTL_IO_NVME: + case CTL_IO_NVME_ADMIN: + return (io->nvmeio.kern_sg_entries); + default: + __assert_unreachable(); + } +} + +static __inline uint8_t * +ctl_kern_data_ptr(union ctl_io *io) +{ + switch (io->io_hdr.io_type) { + case CTL_IO_SCSI: + return (io->scsiio.kern_data_ptr); + case CTL_IO_NVME: + case CTL_IO_NVME_ADMIN: + return (io->nvmeio.kern_data_ptr); + default: + __assert_unreachable(); + } +} + +static __inline uint32_t +ctl_kern_data_len(union ctl_io *io) +{ + switch (io->io_hdr.io_type) { + case CTL_IO_SCSI: + return (io->scsiio.kern_data_len); + case CTL_IO_NVME: + case CTL_IO_NVME_ADMIN: + return (io->nvmeio.kern_data_len); + default: + __assert_unreachable(); + } +} + +static __inline uint32_t +ctl_kern_total_len(union ctl_io *io) +{ + switch (io->io_hdr.io_type) { + case CTL_IO_SCSI: + return (io->scsiio.kern_total_len); + case CTL_IO_NVME: + case CTL_IO_NVME_ADMIN: + return (io->nvmeio.kern_total_len); + default: + __assert_unreachable(); + } +} + +static __inline uint32_t +ctl_kern_data_resid(union ctl_io *io) +{ + switch (io->io_hdr.io_type) { + case CTL_IO_SCSI: + return (io->scsiio.kern_data_resid); + case CTL_IO_NVME: + case CTL_IO_NVME_ADMIN: + return (io->nvmeio.kern_data_resid); + default: + __assert_unreachable(); + } +} + +static __inline uint32_t +ctl_kern_rel_offset(union ctl_io *io) +{ + switch (io->io_hdr.io_type) { + case CTL_IO_SCSI: + return (io->scsiio.kern_rel_offset); + case CTL_IO_NVME: + case CTL_IO_NVME_ADMIN: + return (io->nvmeio.kern_rel_offset); + default: + __assert_unreachable(); + } +} + +static __inline void +ctl_add_kern_rel_offset(union ctl_io *io, uint32_t offset) +{ + switch (io->io_hdr.io_type) { + case CTL_IO_SCSI: + io->scsiio.kern_rel_offset += offset; + break; + case CTL_IO_NVME: + case CTL_IO_NVME_ADMIN: + io->nvmeio.kern_rel_offset += offset; + break; + default: + __assert_unreachable(); + } +} + +static __inline void +ctl_set_kern_sg_entries(union ctl_io *io, uint32_t kern_sg_entries) +{ + switch (io->io_hdr.io_type) { + case CTL_IO_SCSI: + io->scsiio.kern_sg_entries = kern_sg_entries; + break; + case CTL_IO_NVME: + case CTL_IO_NVME_ADMIN: + io->nvmeio.kern_sg_entries = kern_sg_entries; + break; + default: + __assert_unreachable(); + } +} + +static __inline void +ctl_set_kern_data_ptr(union ctl_io *io, void *kern_data_ptr) +{ + switch (io->io_hdr.io_type) { + case CTL_IO_SCSI: + io->scsiio.kern_data_ptr = kern_data_ptr; + break; + case CTL_IO_NVME: + case CTL_IO_NVME_ADMIN: + io->nvmeio.kern_data_ptr = kern_data_ptr; + break; + default: + __assert_unreachable(); + } +} + +static __inline void +ctl_set_kern_data_len(union ctl_io *io, uint32_t kern_data_len) +{ + switch (io->io_hdr.io_type) { + case CTL_IO_SCSI: + io->scsiio.kern_data_len = kern_data_len; + break; + case CTL_IO_NVME: + case CTL_IO_NVME_ADMIN: + io->nvmeio.kern_data_len = kern_data_len; + break; + default: + __assert_unreachable(); + } +} + +static __inline void +ctl_set_kern_total_len(union ctl_io *io, uint32_t kern_total_len) +{ + switch (io->io_hdr.io_type) { + case CTL_IO_SCSI: + io->scsiio.kern_total_len = kern_total_len; + break; + case CTL_IO_NVME: + case CTL_IO_NVME_ADMIN: + io->nvmeio.kern_total_len = kern_total_len; + break; + default: + __assert_unreachable(); + } +} + +static __inline void +ctl_set_kern_data_resid(union ctl_io *io, uint32_t kern_data_resid) +{ + switch (io->io_hdr.io_type) { + case CTL_IO_SCSI: + io->scsiio.kern_data_resid = kern_data_resid; + break; + case CTL_IO_NVME: + case CTL_IO_NVME_ADMIN: + io->nvmeio.kern_data_resid = kern_data_resid; + break; + default: + __assert_unreachable(); + } +} + +static __inline void +ctl_set_kern_rel_offset(union ctl_io *io, uint32_t kern_rel_offset) +{ + switch (io->io_hdr.io_type) { + case CTL_IO_SCSI: + io->scsiio.kern_rel_offset = kern_rel_offset; + break; + case CTL_IO_NVME: + case CTL_IO_NVME_ADMIN: + io->nvmeio.kern_rel_offset = kern_rel_offset; + break; + default: + __assert_unreachable(); + } +} + +static __inline void +ctl_set_be_move_done(union ctl_io *io, ctl_be_move_done_t be_move_done) +{ + switch (io->io_hdr.io_type) { + case CTL_IO_SCSI: + io->scsiio.be_move_done = be_move_done; + break; + case CTL_IO_NVME: + case CTL_IO_NVME_ADMIN: + io->nvmeio.be_move_done = be_move_done; + break; + default: + __assert_unreachable(); + } +} + +static __inline void +ctl_set_io_cont(union ctl_io *io, ctl_io_cont io_cont) +{ + switch (io->io_hdr.io_type) { + case CTL_IO_SCSI: + io->scsiio.io_cont = io_cont; + break; + case CTL_IO_NVME: + case CTL_IO_NVME_ADMIN: + io->nvmeio.io_cont = io_cont; + break; + default: + __assert_unreachable(); + } +} + +static __inline void +ctl_set_kern_data_ref(union ctl_io *io, ctl_ref kern_data_ref) +{ + switch (io->io_hdr.io_type) { + case CTL_IO_SCSI: + io->scsiio.kern_data_ref = kern_data_ref; + break; + case CTL_IO_NVME: + case CTL_IO_NVME_ADMIN: + io->nvmeio.kern_data_ref = kern_data_ref; + break; + default: + __assert_unreachable(); + } +} + +static __inline void +ctl_set_kern_data_arg(union ctl_io *io, void *kern_data_arg) +{ + switch (io->io_hdr.io_type) { + case CTL_IO_SCSI: + io->scsiio.kern_data_arg = kern_data_arg; + break; + case CTL_IO_NVME: + case CTL_IO_NVME_ADMIN: + io->nvmeio.kern_data_arg = kern_data_arg; + break; + default: + __assert_unreachable(); + } +} + union ctl_io *ctl_alloc_io(void *pool_ref); union ctl_io *ctl_alloc_io_nowait(void *pool_ref); void ctl_free_io(union ctl_io *io);