From owner-svn-src-head@freebsd.org Thu Aug 25 05:22:56 2016 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2CB0DBC3E27; Thu, 25 Aug 2016 05:22:56 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E943E19AF; Thu, 25 Aug 2016 05:22:55 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u7P5MtLG003351; Thu, 25 Aug 2016 05:22:55 GMT (envelope-from np@FreeBSD.org) Received: (from np@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u7P5MrEL003331; Thu, 25 Aug 2016 05:22:53 GMT (envelope-from np@FreeBSD.org) Message-Id: <201608250522.u7P5MrEL003331@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: np set sender to np@FreeBSD.org using -f From: Navdeep Parhar Date: Thu, 25 Aug 2016 05:22:53 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r304787 - in head: sys/cam/ctl sys/dev/cxgbe/cxgbei sys/dev/iscsi sys/dev/iser usr.bin/iscsictl usr.sbin/ctladm usr.sbin/ctld usr.sbin/iscsid X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2016 05:22:56 -0000 Author: np Date: Thu Aug 25 05:22:53 2016 New Revision: 304787 URL: https://svnweb.freebsd.org/changeset/base/304787 Log: Make the iSCSI parameter negotiation more flexible. Decouple the send and receive limits on the amount of data in a single iSCSI PDU. MaxRecvDataSegmentLength is declarative, not negotiated, and is direction-specific so there is no reason for both ends to limit themselves to the same min(initiator, target) value in both directions. Allow iSCSI drivers to report their send, receive, first burst, and max burst limits explicitly instead of using hardcoded values or trying to derive all of them from the receive limit (which was the only limit reported by the drivers prior to this change). Display the send and receive limits separately in the userspace iSCSI utilities. Reviewed by: jpaetzel@ (earlier version), trasz@ Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D7279 Modified: head/sys/cam/ctl/ctl_frontend_iscsi.c head/sys/cam/ctl/ctl_frontend_iscsi.h head/sys/cam/ctl/ctl_ioctl.h head/sys/dev/cxgbe/cxgbei/icl_cxgbei.c head/sys/dev/iscsi/icl.c head/sys/dev/iscsi/icl.h head/sys/dev/iscsi/icl_soft.c head/sys/dev/iscsi/iscsi.c head/sys/dev/iscsi/iscsi.h head/sys/dev/iscsi/iscsi_ioctl.h head/sys/dev/iser/icl_iser.c head/usr.bin/iscsictl/iscsictl.c head/usr.sbin/ctladm/ctladm.c head/usr.sbin/ctld/ctld.c head/usr.sbin/ctld/ctld.h head/usr.sbin/ctld/kernel.c head/usr.sbin/ctld/login.c head/usr.sbin/ctld/pdu.c head/usr.sbin/iscsid/iscsid.c head/usr.sbin/iscsid/iscsid.h head/usr.sbin/iscsid/login.c Modified: head/sys/cam/ctl/ctl_frontend_iscsi.c ============================================================================== --- head/sys/cam/ctl/ctl_frontend_iscsi.c Thu Aug 25 05:11:04 2016 (r304786) +++ head/sys/cam/ctl/ctl_frontend_iscsi.c Thu Aug 25 05:22:53 2016 (r304787) @@ -1512,7 +1512,8 @@ cfiscsi_ioctl_handoff(struct ctl_iscsi * */ cs->cs_cmdsn = cihp->cmdsn; cs->cs_statsn = cihp->statsn; - cs->cs_max_data_segment_length = cihp->max_recv_data_segment_length; + cs->cs_max_recv_data_segment_length = cihp->max_recv_data_segment_length; + cs->cs_max_send_data_segment_length = cihp->max_send_data_segment_length; cs->cs_max_burst_length = cihp->max_burst_length; cs->cs_first_burst_length = cihp->first_burst_length; cs->cs_immediate_data = !!cihp->immediate_data; @@ -1652,9 +1653,10 @@ cfiscsi_ioctl_list(struct ctl_iscsi *ci) "%u" "%s" "%s" - "%zd" - "%zd" - "%zd" + "%d" + "%d" + "%d" + "%d" "%d" "%d" "%s" @@ -1665,7 +1667,8 @@ cfiscsi_ioctl_list(struct ctl_iscsi *ci) cs->cs_target->ct_tag, cs->cs_conn->ic_header_crc32c ? "CRC32C" : "None", cs->cs_conn->ic_data_crc32c ? "CRC32C" : "None", - cs->cs_max_data_segment_length, + cs->cs_max_recv_data_segment_length, + cs->cs_max_send_data_segment_length, cs->cs_max_burst_length, cs->cs_first_burst_length, cs->cs_immediate_data, @@ -1794,12 +1797,12 @@ static void cfiscsi_ioctl_limits(struct ctl_iscsi *ci) { struct ctl_iscsi_limits_params *cilp; + struct icl_drv_limits idl; int error; cilp = (struct ctl_iscsi_limits_params *)&(ci->data); - error = icl_limits(cilp->offload, false, - &cilp->data_segment_limit); + error = icl_limits(cilp->offload, false, &idl); if (error != 0) { ci->status = CTL_ISCSI_ERROR; snprintf(ci->error_str, sizeof(ci->error_str), @@ -1808,6 +1811,13 @@ cfiscsi_ioctl_limits(struct ctl_iscsi *c return; } + cilp->max_recv_data_segment_length = + idl.idl_max_recv_data_segment_length; + cilp->max_send_data_segment_length = + idl.idl_max_send_data_segment_length; + cilp->max_burst_length = idl.idl_max_burst_length; + cilp->first_burst_length = idl.idl_first_burst_length; + ci->status = CTL_ISCSI_OK; } @@ -2466,12 +2476,12 @@ cfiscsi_datamove_in(union ctl_io *io) /* * Truncate to maximum data segment length. */ - KASSERT(response->ip_data_len < cs->cs_max_data_segment_length, - ("ip_data_len %zd >= max_data_segment_length %zd", - response->ip_data_len, cs->cs_max_data_segment_length)); + KASSERT(response->ip_data_len < cs->cs_max_send_data_segment_length, + ("ip_data_len %zd >= max_send_data_segment_length %d", + response->ip_data_len, cs->cs_max_send_data_segment_length)); if (response->ip_data_len + len > - cs->cs_max_data_segment_length) { - len = cs->cs_max_data_segment_length - + cs->cs_max_send_data_segment_length) { + len = cs->cs_max_send_data_segment_length - response->ip_data_len; KASSERT(len <= sg_len, ("len %zd > sg_len %zd", len, sg_len)); @@ -2529,7 +2539,7 @@ cfiscsi_datamove_in(union ctl_io *io) i++; } - if (response->ip_data_len == cs->cs_max_data_segment_length) { + if (response->ip_data_len == cs->cs_max_send_data_segment_length) { /* * Can't stuff more data into the current PDU; * queue it. Note that's not enough to check Modified: head/sys/cam/ctl/ctl_frontend_iscsi.h ============================================================================== --- head/sys/cam/ctl/ctl_frontend_iscsi.h Thu Aug 25 05:11:04 2016 (r304786) +++ head/sys/cam/ctl/ctl_frontend_iscsi.h Thu Aug 25 05:22:53 2016 (r304787) @@ -84,9 +84,10 @@ struct cfiscsi_session { struct cv cs_maintenance_cv; bool cs_terminating; bool cs_tasks_aborted; - size_t cs_max_data_segment_length; - size_t cs_max_burst_length; - size_t cs_first_burst_length; + int cs_max_recv_data_segment_length; + int cs_max_send_data_segment_length; + int cs_max_burst_length; + int cs_first_burst_length; bool cs_immediate_data; char cs_initiator_name[CTL_ISCSI_NAME_LEN]; char cs_initiator_addr[CTL_ISCSI_ADDR_LEN]; Modified: head/sys/cam/ctl/ctl_ioctl.h ============================================================================== --- head/sys/cam/ctl/ctl_ioctl.h Thu Aug 25 05:11:04 2016 (r304786) +++ head/sys/cam/ctl/ctl_ioctl.h Thu Aug 25 05:22:53 2016 (r304787) @@ -622,7 +622,7 @@ struct ctl_iscsi_handoff_params { char target_name[CTL_ISCSI_NAME_LEN]; int socket; int portal_group_tag; - + /* * Connection parameters negotiated by ctld(8). */ @@ -630,17 +630,17 @@ struct ctl_iscsi_handoff_params { ctl_iscsi_digest data_digest; uint32_t cmdsn; uint32_t statsn; - uint32_t max_recv_data_segment_length; - uint32_t max_burst_length; - uint32_t first_burst_length; + int max_recv_data_segment_length; + int max_burst_length; + int first_burst_length; uint32_t immediate_data; char offload[CTL_ISCSI_OFFLOAD_LEN]; #ifdef ICL_KERNEL_PROXY int connection_id; - int spare[1]; #else - int spare[2]; + int spare; #endif + int max_send_data_segment_length; }; struct ctl_iscsi_list_params { @@ -671,11 +671,15 @@ struct ctl_iscsi_terminate_params { }; struct ctl_iscsi_limits_params { + /* passed to kernel */ char offload[CTL_ISCSI_OFFLOAD_LEN]; - /* passed to kernel */ - size_t data_segment_limit; - /* passed to userland */ - int spare[4]; + + /* passed to userland */ + size_t spare; + int max_recv_data_segment_length; + int max_send_data_segment_length; + int max_burst_length; + int first_burst_length; }; #ifdef ICL_KERNEL_PROXY Modified: head/sys/dev/cxgbe/cxgbei/icl_cxgbei.c ============================================================================== --- head/sys/dev/cxgbe/cxgbei/icl_cxgbei.c Thu Aug 25 05:11:04 2016 (r304786) +++ head/sys/dev/cxgbe/cxgbei/icl_cxgbei.c Thu Aug 25 05:22:53 2016 (r304787) @@ -832,10 +832,13 @@ icl_cxgbei_conn_transfer_done(struct icl } static int -icl_cxgbei_limits(size_t *limitp) +icl_cxgbei_limits(struct icl_drv_limits *idl) { - *limitp = CXGBEI_MAX_DSL; + idl->idl_max_recv_data_segment_length = CXGBEI_MAX_DSL; + idl->idl_max_send_data_segment_length = CXGBEI_MAX_DSL; + idl->idl_max_burst_length = 2 * 1024 * 1024; + idl->idl_first_burst_length = CXGBEI_MAX_DSL; return (0); } Modified: head/sys/dev/iscsi/icl.c ============================================================================== --- head/sys/dev/iscsi/icl.c Thu Aug 25 05:11:04 2016 (r304786) +++ head/sys/dev/iscsi/icl.c Thu Aug 25 05:22:53 2016 (r304787) @@ -59,7 +59,7 @@ struct icl_module { char *im_name; bool im_iser; int im_priority; - int (*im_limits)(size_t *limitp); + int (*im_limits)(struct icl_drv_limits *idl); struct icl_conn *(*im_new_conn)(const char *name, struct mtx *lock); }; @@ -182,11 +182,12 @@ icl_new_conn(const char *offload, bool i } int -icl_limits(const char *offload, bool iser, size_t *limitp) +icl_limits(const char *offload, bool iser, struct icl_drv_limits *idl) { struct icl_module *im; int error; + bzero(idl, sizeof(*idl)); sx_slock(&sc->sc_lock); im = icl_find(offload, iser, false); if (im == NULL) { @@ -194,14 +195,42 @@ icl_limits(const char *offload, bool ise return (ENXIO); } - error = im->im_limits(limitp); + error = im->im_limits(idl); sx_sunlock(&sc->sc_lock); + /* + * Validate the limits provided by the driver against values allowed by + * the iSCSI RFC. 0 means iscsid/ctld should pick a reasonable value. + * + * Note that max_send_dsl is an internal implementation detail and not + * part of the RFC. + */ +#define OUT_OF_RANGE(x, lo, hi) ((x) != 0 && ((x) < (lo) || (x) > (hi))) + if (error == 0 && + (OUT_OF_RANGE(idl->idl_max_recv_data_segment_length, 512, 16777215) || + OUT_OF_RANGE(idl->idl_max_send_data_segment_length, 512, 16777215) || + OUT_OF_RANGE(idl->idl_max_burst_length, 512, 16777215) || + OUT_OF_RANGE(idl->idl_first_burst_length, 512, 16777215))) { + error = EINVAL; + } +#undef OUT_OF_RANGE + + /* + * If both first_burst and max_burst are provided then first_burst must + * not exceed max_burst. + */ + if (error == 0 && idl->idl_first_burst_length > 0 && + idl->idl_max_burst_length > 0 && + idl->idl_first_burst_length > idl->idl_max_burst_length) { + error = EINVAL; + } + return (error); } int -icl_register(const char *offload, bool iser, int priority, int (*limits)(size_t *), +icl_register(const char *offload, bool iser, int priority, + int (*limits)(struct icl_drv_limits *), struct icl_conn *(*new_conn)(const char *, struct mtx *)) { struct icl_module *im; Modified: head/sys/dev/iscsi/icl.h ============================================================================== --- head/sys/dev/iscsi/icl.h Thu Aug 25 05:11:04 2016 (r304786) +++ head/sys/dev/iscsi/icl.h Thu Aug 25 05:22:53 2016 (r304787) @@ -126,12 +126,20 @@ struct icl_conn { void *ic_prv0; }; +struct icl_drv_limits { + int idl_max_recv_data_segment_length; + int idl_max_send_data_segment_length; + int idl_max_burst_length; + int idl_first_burst_length; + int spare[4]; +}; + struct icl_conn *icl_new_conn(const char *offload, bool iser, const char *name, struct mtx *lock); -int icl_limits(const char *offload, bool iser, size_t *limitp); - +int icl_limits(const char *offload, bool iser, + struct icl_drv_limits *idl); int icl_register(const char *offload, bool iser, int priority, - int (*limits)(size_t *), + int (*limits)(struct icl_drv_limits *), struct icl_conn *(*new_conn)(const char *, struct mtx *)); int icl_unregister(const char *offload, bool rdma); Modified: head/sys/dev/iscsi/icl_soft.c ============================================================================== --- head/sys/dev/iscsi/icl_soft.c Thu Aug 25 05:11:04 2016 (r304786) +++ head/sys/dev/iscsi/icl_soft.c Thu Aug 25 05:22:53 2016 (r304787) @@ -1474,10 +1474,10 @@ icl_soft_conn_transfer_done(struct icl_c } static int -icl_soft_limits(size_t *limitp) +icl_soft_limits(struct icl_drv_limits *idl) { - *limitp = 128 * 1024; + idl->idl_max_recv_data_segment_length = 128 * 1024; return (0); } Modified: head/sys/dev/iscsi/iscsi.c ============================================================================== --- head/sys/dev/iscsi/iscsi.c Thu Aug 25 05:11:04 2016 (r304786) +++ head/sys/dev/iscsi/iscsi.c Thu Aug 25 05:22:53 2016 (r304787) @@ -1204,8 +1204,8 @@ iscsi_pdu_handle_r2t(struct icl_pdu *res for (;;) { len = total_len; - if (len > is->is_max_data_segment_length) - len = is->is_max_data_segment_length; + if (len > is->is_max_send_data_segment_length) + len = is->is_max_send_data_segment_length; if (off + len > csio->dxfer_len) { ISCSI_SESSION_WARN(is, "target requested invalid " @@ -1313,6 +1313,7 @@ iscsi_ioctl_daemon_wait(struct iscsi_sof struct iscsi_daemon_request *request) { struct iscsi_session *is; + struct icl_drv_limits idl; int error; sx_slock(&sc->sc_lock); @@ -1352,10 +1353,9 @@ iscsi_ioctl_daemon_wait(struct iscsi_sof request->idr_tsih = 0; /* New or reinstated session. */ memcpy(&request->idr_conf, &is->is_conf, sizeof(request->idr_conf)); - + error = icl_limits(is->is_conf.isc_offload, - is->is_conf.isc_iser, - &request->idr_limits.isl_max_data_segment_length); + is->is_conf.isc_iser, &idl); if (error != 0) { ISCSI_SESSION_WARN(is, "icl_limits for offload \"%s\" " "failed with error %d", is->is_conf.isc_offload, @@ -1363,6 +1363,14 @@ iscsi_ioctl_daemon_wait(struct iscsi_sof sx_sunlock(&sc->sc_lock); return (error); } + request->idr_limits.isl_max_recv_data_segment_length = + idl.idl_max_recv_data_segment_length; + request->idr_limits.isl_max_send_data_segment_length = + idl.idl_max_recv_data_segment_length; + request->idr_limits.isl_max_burst_length = + idl.idl_max_burst_length; + request->idr_limits.isl_first_burst_length = + idl.idl_first_burst_length; sx_sunlock(&sc->sc_lock); return (0); @@ -1417,12 +1425,10 @@ iscsi_ioctl_daemon_handoff(struct iscsi_ is->is_initial_r2t = handoff->idh_initial_r2t; is->is_immediate_data = handoff->idh_immediate_data; - /* - * Cap MaxRecvDataSegmentLength obtained from the target to the maximum - * size supported by our ICL module. - */ - is->is_max_data_segment_length = min(ic->ic_max_data_segment_length, - handoff->idh_max_data_segment_length); + is->is_max_recv_data_segment_length = + handoff->idh_max_recv_data_segment_length; + is->is_max_send_data_segment_length = + handoff->idh_max_send_data_segment_length; is->is_max_burst_length = handoff->idh_max_burst_length; is->is_first_burst_length = handoff->idh_first_burst_length; @@ -1634,7 +1640,7 @@ iscsi_ioctl_daemon_send(struct iscsi_sof return (EIO); datalen = ids->ids_data_segment_len; - if (datalen > ISCSI_MAX_DATA_SEGMENT_LENGTH) + if (datalen > is->is_max_send_data_segment_length) return (EINVAL); if (datalen > 0) { data = malloc(datalen, M_ISCSI, M_WAITOK); @@ -1933,12 +1939,15 @@ iscsi_ioctl_session_list(struct iscsi_so else iss.iss_data_digest = ISCSI_DIGEST_NONE; - iss.iss_max_data_segment_length = is->is_max_data_segment_length; + iss.iss_max_send_data_segment_length = + is->is_max_send_data_segment_length; + iss.iss_max_recv_data_segment_length = + is->is_max_recv_data_segment_length; iss.iss_max_burst_length = is->is_max_burst_length; iss.iss_first_burst_length = is->is_first_burst_length; iss.iss_immediate_data = is->is_immediate_data; iss.iss_connected = is->is_connected; - + error = copyout(&iss, isl->isl_pstates + i, sizeof(iss)); if (error != 0) { sx_sunlock(&sc->sc_lock); @@ -2259,12 +2268,13 @@ iscsi_action_scsiio(struct iscsi_session len = csio->dxfer_len; //ISCSI_SESSION_DEBUG(is, "adding %zd of immediate data", len); if (len > is->is_first_burst_length) { - ISCSI_SESSION_DEBUG(is, "len %zd -> %zd", len, is->is_first_burst_length); + ISCSI_SESSION_DEBUG(is, "len %zd -> %d", len, is->is_first_burst_length); len = is->is_first_burst_length; } - if (len > is->is_max_data_segment_length) { - ISCSI_SESSION_DEBUG(is, "len %zd -> %zd", len, is->is_max_data_segment_length); - len = is->is_max_data_segment_length; + if (len > is->is_max_send_data_segment_length) { + ISCSI_SESSION_DEBUG(is, "len %zd -> %d", len, + is->is_max_send_data_segment_length); + len = is->is_max_send_data_segment_length; } error = icl_pdu_append_data(request, csio->data_ptr, len, M_NOWAIT); Modified: head/sys/dev/iscsi/iscsi.h ============================================================================== --- head/sys/dev/iscsi/iscsi.h Thu Aug 25 05:11:04 2016 (r304786) +++ head/sys/dev/iscsi/iscsi.h Thu Aug 25 05:22:53 2016 (r304787) @@ -62,12 +62,13 @@ struct iscsi_session { int is_header_digest; int is_data_digest; int is_initial_r2t; - size_t is_max_burst_length; - size_t is_first_burst_length; + int is_max_burst_length; + int is_first_burst_length; uint8_t is_isid[6]; uint16_t is_tsih; bool is_immediate_data; - size_t is_max_data_segment_length; + int is_max_recv_data_segment_length; + int is_max_send_data_segment_length; char is_target_alias[ISCSI_ALIAS_LEN]; TAILQ_HEAD(, iscsi_outstanding) is_outstanding; Modified: head/sys/dev/iscsi/iscsi_ioctl.h ============================================================================== --- head/sys/dev/iscsi/iscsi_ioctl.h Thu Aug 25 05:11:04 2016 (r304786) +++ head/sys/dev/iscsi/iscsi_ioctl.h Thu Aug 25 05:22:53 2016 (r304787) @@ -76,8 +76,12 @@ struct iscsi_session_conf { * iscsid(8) must obey those when negotiating operational parameters. */ struct iscsi_session_limits { - size_t isl_max_data_segment_length; - int isl_spare[8]; + size_t isl_spare0; + int isl_max_recv_data_segment_length; + int isl_max_send_data_segment_length; + int isl_max_burst_length; + int isl_first_burst_length; + int isl_spare[4]; }; /* @@ -89,14 +93,15 @@ struct iscsi_session_state { char iss_target_alias[ISCSI_ALIAS_LEN]; int iss_header_digest; int iss_data_digest; - int iss_max_data_segment_length; + int iss_max_recv_data_segment_length; int iss_max_burst_length; int iss_first_burst_length; int iss_immediate_data; int iss_connected; char iss_reason[ISCSI_REASON_LEN]; char iss_offload[ISCSI_OFFLOAD_LEN]; - int iss_spare[4]; + int iss_max_send_data_segment_length; + int iss_spare[3]; }; /* @@ -122,12 +127,13 @@ struct iscsi_daemon_handoff { uint32_t idh_statsn; int idh_header_digest; int idh_data_digest; - size_t idh_max_data_segment_length; - size_t idh_max_burst_length; - size_t idh_first_burst_length; + size_t spare[3]; int idh_immediate_data; int idh_initial_r2t; - int idh_spare[4]; + int idh_max_recv_data_segment_length; + int idh_max_send_data_segment_length; + int idh_max_burst_length; + int idh_first_burst_length; }; struct iscsi_daemon_fail { Modified: head/sys/dev/iser/icl_iser.c ============================================================================== --- head/sys/dev/iser/icl_iser.c Thu Aug 25 05:11:04 2016 (r304786) +++ head/sys/dev/iser/icl_iser.c Thu Aug 25 05:22:53 2016 (r304787) @@ -483,9 +483,9 @@ iser_conn_task_done(struct icl_conn *ic, } static int -iser_limits(size_t *limitp) +iser_limits(struct icl_drv_limits *idl) { - *limitp = 128 * 1024; + idl->idl_max_recv_data_segment_length = 128 * 1024; return (0); } Modified: head/usr.bin/iscsictl/iscsictl.c ============================================================================== --- head/usr.bin/iscsictl/iscsictl.c Thu Aug 25 05:11:04 2016 (r304786) +++ head/usr.bin/iscsictl/iscsictl.c Thu Aug 25 05:22:53 2016 (r304787) @@ -514,70 +514,74 @@ kernel_list(int iscsi_fd, const struct t * Display-only modifier as this information * is also present within the 'session' container */ - xo_emit("{L:/%-18s}{V:sessionId/%u}\n", + xo_emit("{L:/%-25s}{V:sessionId/%u}\n", "Session ID:", state->iss_id); xo_open_container("initiator"); - xo_emit("{L:/%-18s}{V:name/%s}\n", + xo_emit("{L:/%-25s}{V:name/%s}\n", "Initiator name:", conf->isc_initiator); - xo_emit("{L:/%-18s}{V:portal/%s}\n", + xo_emit("{L:/%-25s}{V:portal/%s}\n", "Initiator portal:", conf->isc_initiator_addr); - xo_emit("{L:/%-18s}{V:alias/%s}\n", + xo_emit("{L:/%-25s}{V:alias/%s}\n", "Initiator alias:", conf->isc_initiator_alias); xo_close_container("initiator"); xo_open_container("target"); - xo_emit("{L:/%-18s}{V:name/%s}\n", + xo_emit("{L:/%-25s}{V:name/%s}\n", "Target name:", conf->isc_target); - xo_emit("{L:/%-18s}{V:portal/%s}\n", + xo_emit("{L:/%-25s}{V:portal/%s}\n", "Target portal:", conf->isc_target_addr); - xo_emit("{L:/%-18s}{V:alias/%s}\n", + xo_emit("{L:/%-25s}{V:alias/%s}\n", "Target alias:", state->iss_target_alias); xo_close_container("target"); xo_open_container("auth"); - xo_emit("{L:/%-18s}{V:user/%s}\n", + xo_emit("{L:/%-25s}{V:user/%s}\n", "User:", conf->isc_user); - xo_emit("{L:/%-18s}{V:secret/%s}\n", + xo_emit("{L:/%-25s}{V:secret/%s}\n", "Secret:", conf->isc_secret); - xo_emit("{L:/%-18s}{V:mutualUser/%s}\n", + xo_emit("{L:/%-25s}{V:mutualUser/%s}\n", "Mutual user:", conf->isc_mutual_user); - xo_emit("{L:/%-18s}{V:mutualSecret/%s}\n", + xo_emit("{L:/%-25s}{V:mutualSecret/%s}\n", "Mutual secret:", conf->isc_mutual_secret); xo_close_container("auth"); - xo_emit("{L:/%-18s}{V:type/%s}\n", + xo_emit("{L:/%-25s}{V:type/%s}\n", "Session type:", conf->isc_discovery ? "Discovery" : "Normal"); - xo_emit("{L:/%-18s}{V:enable/%s}\n", + xo_emit("{L:/%-25s}{V:enable/%s}\n", "Enable:", conf->isc_enable ? "Yes" : "No"); - xo_emit("{L:/%-18s}{V:state/%s}\n", + xo_emit("{L:/%-25s}{V:state/%s}\n", "Session state:", state->iss_connected ? "Connected" : "Disconnected"); - xo_emit("{L:/%-18s}{V:failureReason/%s}\n", + xo_emit("{L:/%-25s}{V:failureReason/%s}\n", "Failure reason:", state->iss_reason); - xo_emit("{L:/%-18s}{V:headerDigest/%s}\n", + xo_emit("{L:/%-25s}{V:headerDigest/%s}\n", "Header digest:", state->iss_header_digest == ISCSI_DIGEST_CRC32C ? "CRC32C" : "None"); - xo_emit("{L:/%-18s}{V:dataDigest/%s}\n", + xo_emit("{L:/%-25s}{V:dataDigest/%s}\n", "Data digest:", state->iss_data_digest == ISCSI_DIGEST_CRC32C ? "CRC32C" : "None"); - xo_emit("{L:/%-18s}{V:dataSegmentLen/%d}\n", - "DataSegmentLen:", state->iss_max_data_segment_length); - xo_emit("{L:/%-18s}{V:maxBurstLen/%d}\n", + xo_emit("{L:/%-25s}{V:recvDataSegmentLen/%d}\n", + "MaxRecvDataSegmentLength:", + state->iss_max_recv_data_segment_length); + xo_emit("{L:/%-25s}{V:sendDataSegmentLen/%d}\n", + "MaxSendDataSegmentLength:", + state->iss_max_send_data_segment_length); + xo_emit("{L:/%-25s}{V:maxBurstLen/%d}\n", "MaxBurstLen:", state->iss_max_burst_length); - xo_emit("{L:/%-18s}{V:firstBurstLen/%d}\n", + xo_emit("{L:/%-25s}{V:firstBurstLen/%d}\n", "FirstBurstLen:", state->iss_first_burst_length); - xo_emit("{L:/%-18s}{V:immediateData/%s}\n", + xo_emit("{L:/%-25s}{V:immediateData/%s}\n", "ImmediateData:", state->iss_immediate_data ? "Yes" : "No"); - xo_emit("{L:/%-18s}{V:iSER/%s}\n", + xo_emit("{L:/%-25s}{V:iSER/%s}\n", "iSER (RDMA):", conf->isc_iser ? "Yes" : "No"); - xo_emit("{L:/%-18s}{V:offloadDriver/%s}\n", + xo_emit("{L:/%-25s}{V:offloadDriver/%s}\n", "Offload driver:", state->iss_offload); - xo_emit("{L:/%-18s}", + xo_emit("{L:/%-25s}", "Device nodes:"); print_periphs(state->iss_id); xo_emit("\n\n"); Modified: head/usr.sbin/ctladm/ctladm.c ============================================================================== --- head/usr.sbin/ctladm/ctladm.c Thu Aug 25 05:11:04 2016 (r304786) +++ head/usr.sbin/ctladm/ctladm.c Thu Aug 25 05:22:53 2016 (r304787) @@ -2794,7 +2794,8 @@ struct cctl_islist_conn { char *target_alias; char *header_digest; char *data_digest; - char *max_data_segment_length; + char *max_recv_data_segment_length; + char *max_send_data_segment_length; char *max_burst_length; char *first_burst_length; char *offload; @@ -2908,8 +2909,11 @@ cctl_islist_end_element(void *user_data, } else if (strcmp(name, "data_digest") == 0) { cur_conn->data_digest = str; str = NULL; - } else if (strcmp(name, "max_data_segment_length") == 0) { - cur_conn->max_data_segment_length = str; + } else if (strcmp(name, "max_recv_data_segment_length") == 0) { + cur_conn->max_recv_data_segment_length = str; + str = NULL; + } else if (strcmp(name, "max_send_data_segment_length") == 0) { + cur_conn->max_send_data_segment_length = str; str = NULL; } else if (strcmp(name, "max_burst_length") == 0) { cur_conn->max_burst_length = str; @@ -3030,20 +3034,21 @@ retry: if (verbose != 0) { STAILQ_FOREACH(conn, &islist.conn_list, links) { - printf("Session ID: %d\n", conn->connection_id); - printf("Initiator name: %s\n", conn->initiator); - printf("Initiator portal: %s\n", conn->initiator_addr); - printf("Initiator alias: %s\n", conn->initiator_alias); - printf("Target name: %s\n", conn->target); - printf("Target alias: %s\n", conn->target_alias); - printf("Header digest: %s\n", conn->header_digest); - printf("Data digest: %s\n", conn->data_digest); - printf("DataSegmentLen: %s\n", conn->max_data_segment_length); - printf("MaxBurstLen: %s\n", conn->max_burst_length); - printf("FirstBurstLen: %s\n", conn->first_burst_length); - printf("ImmediateData: %s\n", conn->immediate_data ? "Yes" : "No"); - printf("iSER (RDMA): %s\n", conn->iser ? "Yes" : "No"); - printf("Offload driver: %s\n", conn->offload); + printf("%-25s %d\n", "Session ID:", conn->connection_id); + printf("%-25s %s\n", "Initiator name:", conn->initiator); + printf("%-25s %s\n", "Initiator portal:", conn->initiator_addr); + printf("%-25s %s\n", "Initiator alias:", conn->initiator_alias); + printf("%-25s %s\n", "Target name:", conn->target); + printf("%-25s %s\n", "Target alias:", conn->target_alias); + printf("%-25s %s\n", "Header digest:", conn->header_digest); + printf("%-25s %s\n", "Data digest:", conn->data_digest); + printf("%-25s %s\n", "MaxRecvDataSegmentLength:", conn->max_recv_data_segment_length); + printf("%-25s %s\n", "MaxSendDataSegmentLength:", conn->max_send_data_segment_length); + printf("%-25s %s\n", "MaxBurstLen:", conn->max_burst_length); + printf("%-25s %s\n", "FirstBurstLen:", conn->first_burst_length); + printf("%-25s %s\n", "ImmediateData:", conn->immediate_data ? "Yes" : "No"); + printf("%-25s %s\n", "iSER (RDMA):", conn->iser ? "Yes" : "No"); + printf("%-25s %s\n", "Offload driver:", conn->offload); printf("\n"); } } else { Modified: head/usr.sbin/ctld/ctld.c ============================================================================== --- head/usr.sbin/ctld/ctld.c Thu Aug 25 05:11:04 2016 (r304786) +++ head/usr.sbin/ctld/ctld.c Thu Aug 25 05:22:53 2016 (r304787) @@ -1578,8 +1578,9 @@ connection_new(struct portal *portal, in /* * Default values, from RFC 3720, section 12. */ - conn->conn_max_data_segment_length = 8192; + conn->conn_max_recv_data_segment_length = 8192; conn->conn_max_burst_length = 262144; + conn->conn_first_burst_length = 65536; conn->conn_immediate_data = true; return (conn); Modified: head/usr.sbin/ctld/ctld.h ============================================================================== --- head/usr.sbin/ctld/ctld.h Thu Aug 25 05:11:04 2016 (r304786) +++ head/usr.sbin/ctld/ctld.h Thu Aug 25 05:22:53 2016 (r304787) @@ -242,10 +242,10 @@ struct connection { struct sockaddr_storage conn_initiator_sa; uint32_t conn_cmdsn; uint32_t conn_statsn; - size_t conn_data_segment_limit; - size_t conn_max_data_segment_length; - size_t conn_max_burst_length; - size_t conn_first_burst_length; + int conn_max_recv_data_segment_length; + int conn_max_send_data_segment_length; + int conn_max_burst_length; + int conn_first_burst_length; int conn_immediate_data; int conn_header_digest; int conn_data_digest; @@ -404,7 +404,10 @@ int kernel_lun_modify(struct lun *lun) int kernel_lun_remove(struct lun *lun); void kernel_handoff(struct connection *conn); void kernel_limits(const char *offload, - size_t *max_data_segment_length); + int *max_recv_data_segment_length, + int *max_send_data_segment_length, + int *max_burst_length, + int *first_burst_length); int kernel_port_add(struct port *port); int kernel_port_update(struct port *port, struct port *old); int kernel_port_remove(struct port *port); Modified: head/usr.sbin/ctld/kernel.c ============================================================================== --- head/usr.sbin/ctld/kernel.c Thu Aug 25 05:11:04 2016 (r304786) +++ head/usr.sbin/ctld/kernel.c Thu Aug 25 05:22:53 2016 (r304787) @@ -898,7 +898,9 @@ kernel_handoff(struct connection *conn) req.data.handoff.cmdsn = conn->conn_cmdsn; req.data.handoff.statsn = conn->conn_statsn; req.data.handoff.max_recv_data_segment_length = - conn->conn_max_data_segment_length; + conn->conn_max_recv_data_segment_length; + req.data.handoff.max_send_data_segment_length = + conn->conn_max_send_data_segment_length; req.data.handoff.max_burst_length = conn->conn_max_burst_length; req.data.handoff.first_burst_length = conn->conn_first_burst_length; req.data.handoff.immediate_data = conn->conn_immediate_data; @@ -915,16 +917,18 @@ kernel_handoff(struct connection *conn) } void -kernel_limits(const char *offload, size_t *max_data_segment_length) +kernel_limits(const char *offload, int *max_recv_dsl, int *max_send_dsl, + int *max_burst_length, int *first_burst_length) { struct ctl_iscsi req; + struct ctl_iscsi_limits_params *cilp; bzero(&req, sizeof(req)); req.type = CTL_ISCSI_LIMITS; + cilp = (struct ctl_iscsi_limits_params *)&(req.data.limits); if (offload != NULL) { - strlcpy(req.data.limits.offload, offload, - sizeof(req.data.limits.offload)); + strlcpy(cilp->offload, offload, sizeof(cilp->offload)); } if (ioctl(ctl_fd, CTL_ISCSI, &req) == -1) { @@ -937,13 +941,31 @@ kernel_limits(const char *offload, size_ "%s; dropping connection", req.error_str); } - *max_data_segment_length = req.data.limits.data_segment_limit; + if (cilp->max_recv_data_segment_length != 0) { + *max_recv_dsl = cilp->max_recv_data_segment_length; + *max_send_dsl = cilp->max_recv_data_segment_length; + } + if (cilp->max_send_data_segment_length != 0) + *max_send_dsl = cilp->max_send_data_segment_length; + if (cilp->max_burst_length != 0) + *max_burst_length = cilp->max_burst_length; + if (cilp->first_burst_length != 0) + *first_burst_length = cilp->first_burst_length; + if (*max_burst_length < *first_burst_length) + *first_burst_length = *max_burst_length; + if (offload != NULL) { - log_debugx("MaxRecvDataSegment kernel limit for offload " - "\"%s\" is %zd", offload, *max_data_segment_length); + log_debugx("Kernel limits for offload \"%s\" are " + "MaxRecvDataSegment=%d, max_send_dsl=%d, " + "MaxBurstLength=%d, FirstBurstLength=%d", + offload, *max_recv_dsl, *max_send_dsl, *max_burst_length, + *first_burst_length); } else { - log_debugx("MaxRecvDataSegment kernel limit is %zd", - *max_data_segment_length); + log_debugx("Kernel limits are " + "MaxRecvDataSegment=%d, max_send_dsl=%d, " + "MaxBurstLength=%d, FirstBurstLength=%d", + *max_recv_dsl, *max_send_dsl, *max_burst_length, + *first_burst_length); } } @@ -1217,18 +1239,21 @@ kernel_send(struct pdu *pdu) void kernel_receive(struct pdu *pdu) { + struct connection *conn; struct ctl_iscsi req; - pdu->pdu_data = malloc(MAX_DATA_SEGMENT_LENGTH); + conn = pdu->pdu_connection; + pdu->pdu_data = malloc(conn->conn_max_recv_data_segment_length); if (pdu->pdu_data == NULL) log_err(1, "malloc"); bzero(&req, sizeof(req)); req.type = CTL_ISCSI_RECEIVE; - req.data.receive.connection_id = pdu->pdu_connection->conn_socket; + req.data.receive.connection_id = conn->conn_socket; req.data.receive.bhs = pdu->pdu_bhs; - req.data.receive.data_segment_len = MAX_DATA_SEGMENT_LENGTH; + req.data.receive.data_segment_len = + conn->conn_max_recv_data_segment_length; req.data.receive.data_segment = pdu->pdu_data; if (ioctl(ctl_fd, CTL_ISCSI, &req) == -1) { Modified: head/usr.sbin/ctld/login.c ============================================================================== --- head/usr.sbin/ctld/login.c Thu Aug 25 05:11:04 2016 (r304786) +++ head/usr.sbin/ctld/login.c Thu Aug 25 05:22:53 2016 (r304787) @@ -550,23 +550,32 @@ login_negotiate_key(struct pdu *request, log_errx(1, "received invalid " "MaxRecvDataSegmentLength"); } - if (tmp > conn->conn_data_segment_limit) { - log_debugx("capping MaxRecvDataSegmentLength " - "from %zd to %zd", tmp, conn->conn_data_segment_limit); - tmp = conn->conn_data_segment_limit; - } - conn->conn_max_data_segment_length = tmp; - keys_add_int(response_keys, name, conn->conn_data_segment_limit); + + /* + * MaxRecvDataSegmentLength is a direction-specific parameter. + * We'll limit our _send_ to what the initiator can handle but + * our MaxRecvDataSegmentLength is not influenced by the + * initiator in any way. + */ + if ((int)tmp > conn->conn_max_send_data_segment_length) { + log_debugx("capping max_send_data_segment_length " + "from %zd to %d", tmp, + conn->conn_max_send_data_segment_length); + tmp = conn->conn_max_send_data_segment_length; + } + conn->conn_max_send_data_segment_length = tmp; + keys_add_int(response_keys, name, + conn->conn_max_recv_data_segment_length); } else if (strcmp(name, "MaxBurstLength") == 0) { tmp = strtoul(value, NULL, 10); if (tmp <= 0) { login_send_error(request, 0x02, 0x00); log_errx(1, "received invalid MaxBurstLength"); } - if (tmp > MAX_BURST_LENGTH) { + if ((int)tmp > conn->conn_max_burst_length) { log_debugx("capping MaxBurstLength from %zd to %d", - tmp, MAX_BURST_LENGTH); - tmp = MAX_BURST_LENGTH; + tmp, conn->conn_max_burst_length); + tmp = conn->conn_max_burst_length; } conn->conn_max_burst_length = tmp; keys_add_int(response_keys, name, tmp); @@ -576,10 +585,10 @@ login_negotiate_key(struct pdu *request, login_send_error(request, 0x02, 0x00); log_errx(1, "received invalid FirstBurstLength"); } - if (tmp > FIRST_BURST_LENGTH) { + if ((int)tmp > conn->conn_first_burst_length) { log_debugx("capping FirstBurstLength from %zd to %d", - tmp, FIRST_BURST_LENGTH); - tmp = FIRST_BURST_LENGTH; + tmp, conn->conn_first_burst_length); + tmp = conn->conn_first_burst_length; } conn->conn_first_burst_length = tmp; keys_add_int(response_keys, name, tmp); @@ -681,14 +690,30 @@ login_negotiate(struct connection *conn, if (conn->conn_session_type == CONN_SESSION_TYPE_NORMAL) { /* - * Query the kernel for MaxDataSegmentLength it can handle. - * In case of offload, it depends on hardware capabilities. + * Query the kernel for various size limits. In case of + * offload, it depends on hardware capabilities. */ assert(conn->conn_target != NULL); kernel_limits(conn->conn_portal->p_portal_group->pg_offload, - &conn->conn_data_segment_limit); + &conn->conn_max_recv_data_segment_length, + &conn->conn_max_send_data_segment_length, + &conn->conn_max_burst_length, + &conn->conn_first_burst_length); + + /* We expect legal, usable values at this point. */ + assert(conn->conn_max_recv_data_segment_length >= 512); + assert(conn->conn_max_recv_data_segment_length < (1 << 24)); + assert(conn->conn_max_burst_length >= 512); + assert(conn->conn_max_burst_length < (1 << 24)); + assert(conn->conn_first_burst_length >= 512); + assert(conn->conn_first_burst_length < (1 << 24)); + assert(conn->conn_first_burst_length <= + conn->conn_max_burst_length); } else { - conn->conn_data_segment_limit = MAX_DATA_SEGMENT_LENGTH; + conn->conn_max_recv_data_segment_length = + MAX_DATA_SEGMENT_LENGTH; + conn->conn_max_send_data_segment_length = + MAX_DATA_SEGMENT_LENGTH; } if (request == NULL) { @@ -739,6 +764,18 @@ login_negotiate(struct connection *conn, response_keys); } + /* + * We'd started with usable values at our end. But a bad initiator + * could have presented a large FirstBurstLength and then a smaller + * MaxBurstLength (in that order) and because we process the key/value + * pairs in the order they are in the request we might have ended up + * with illegal values here. + */ + if (conn->conn_session_type == CONN_SESSION_TYPE_NORMAL && + conn->conn_first_burst_length > conn->conn_max_burst_length) { + log_errx(1, "initiator sent FirstBurstLength > MaxBurstLength"); + } + log_debugx("operational parameter negotiation done; " "transitioning to Full Feature Phase"); Modified: head/usr.sbin/ctld/pdu.c ============================================================================== --- head/usr.sbin/ctld/pdu.c Thu Aug 25 05:11:04 2016 (r304786) +++ head/usr.sbin/ctld/pdu.c Thu Aug 25 05:22:53 2016 (r304787) @@ -117,7 +117,7 @@ pdu_receive_proxy(struct pdu *pdu) log_errx(1, "protocol error: non-empty AHS"); len = pdu_data_segment_length(pdu); - assert(len <= MAX_DATA_SEGMENT_LENGTH); + assert(len <= pdu->pdu_connection->conn_max_recv_data_segment_length); pdu->pdu_data_len = len; } @@ -164,6 +164,7 @@ pdu_read(int fd, char *data, size_t len) void pdu_receive(struct pdu *pdu) { + struct connection *conn; size_t len, padding; char dummy[4]; @@ -173,9 +174,10 @@ pdu_receive(struct pdu *pdu) #endif assert(proxy_mode == false); + conn = pdu->pdu_connection; - pdu_read(pdu->pdu_connection->conn_socket, - (char *)pdu->pdu_bhs, sizeof(*pdu->pdu_bhs)); + pdu_read(conn->conn_socket, (char *)pdu->pdu_bhs, + sizeof(*pdu->pdu_bhs)); len = pdu_ahs_length(pdu); if (len > 0) @@ -183,10 +185,10 @@ pdu_receive(struct pdu *pdu) len = pdu_data_segment_length(pdu); if (len > 0) { - if (len > MAX_DATA_SEGMENT_LENGTH) { + if ((int)len > conn->conn_max_recv_data_segment_length) { log_errx(1, "protocol error: received PDU " "with DataSegmentLength exceeding %d", - MAX_DATA_SEGMENT_LENGTH); + conn->conn_max_recv_data_segment_length); } pdu->pdu_data_len = len; @@ -194,14 +196,13 @@ pdu_receive(struct pdu *pdu) if (pdu->pdu_data == NULL) log_err(1, "malloc"); - pdu_read(pdu->pdu_connection->conn_socket, - (char *)pdu->pdu_data, pdu->pdu_data_len); + pdu_read(conn->conn_socket, (char *)pdu->pdu_data, + pdu->pdu_data_len); padding = pdu_padding(pdu); if (padding != 0) { assert(padding < sizeof(dummy)); - pdu_read(pdu->pdu_connection->conn_socket, - (char *)dummy, padding); + pdu_read(conn->conn_socket, (char *)dummy, padding); } } } Modified: head/usr.sbin/iscsid/iscsid.c ============================================================================== --- head/usr.sbin/iscsid/iscsid.c Thu Aug 25 05:11:04 2016 (r304786) +++ head/usr.sbin/iscsid/iscsid.c Thu Aug 25 05:22:53 2016 (r304787) @@ -153,6 +153,7 @@ static struct connection * connection_new(int iscsi_fd, const struct iscsi_daemon_request *request) { struct connection *conn; + struct iscsi_session_limits *isl; struct addrinfo *from_ai, *to_ai; const char *from_addr, *to_addr; #ifdef ICL_KERNEL_PROXY @@ -171,16 +172,49 @@ connection_new(int iscsi_fd, const struc conn->conn_data_digest = CONN_DIGEST_NONE; conn->conn_initial_r2t = true; conn->conn_immediate_data = true; - conn->conn_max_data_segment_length = 8192; - conn->conn_max_burst_length = 262144; - conn->conn_first_burst_length = 65536; + conn->conn_max_burst_length = MAX_BURST_LENGTH; + conn->conn_first_burst_length = FIRST_BURST_LENGTH; conn->conn_iscsi_fd = iscsi_fd; conn->conn_session_id = request->idr_session_id; memcpy(&conn->conn_conf, &request->idr_conf, sizeof(conn->conn_conf)); memcpy(&conn->conn_isid, &request->idr_isid, sizeof(conn->conn_isid)); conn->conn_tsih = request->idr_tsih; - memcpy(&conn->conn_limits, &request->idr_limits, sizeof(conn->conn_limits)); + + /* + * Read the driver limits and provide reasonable defaults for the ones + * the driver doesn't care about. If a max_snd_dsl is not explicitly + * provided by the driver then we'll make sure both conn->max_snd_dsl + * and isl->max_snd_dsl are set to the rcv_dsl. This preserves historic + * behavior. *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***