From owner-svn-src-all@freebsd.org Wed Nov 11 18:55:36 2015 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58C92A2C9D7; Wed, 11 Nov 2015 18:55:36 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 302581BB5; Wed, 11 Nov 2015 18:55:36 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id tABItZGx008037; Wed, 11 Nov 2015 18:55:35 GMT (envelope-from cem@FreeBSD.org) Received: (from cem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id tABItZZf008035; Wed, 11 Nov 2015 18:55:35 GMT (envelope-from cem@FreeBSD.org) Message-Id: <201511111855.tABItZZf008035@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: cem set sender to cem@FreeBSD.org using -f From: "Conrad E. Meyer" Date: Wed, 11 Nov 2015 18:55:35 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r290683 - in head/sys/dev/ntb: if_ntb ntb_hw X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Nov 2015 18:55:36 -0000 Author: cem Date: Wed Nov 11 18:55:34 2015 New Revision: 290683 URL: https://svnweb.freebsd.org/changeset/base/290683 Log: if_ntb: Transport link cleanup needs to be on a taskqueue Because it can sleep drainking link work callout(s). Linux (dual BSD/GPL driver) does something very similar. At the same time, switch the NTB CTX lock to a non-spin mutex, because the taskqueue_swi lock can't be taken after a spin mutex. Suggested by: Witness Sponsored by: EMC / Isilon Storage Division Modified: head/sys/dev/ntb/if_ntb/if_ntb.c head/sys/dev/ntb/ntb_hw/ntb_hw.c Modified: head/sys/dev/ntb/if_ntb/if_ntb.c ============================================================================== --- head/sys/dev/ntb/if_ntb/if_ntb.c Wed Nov 11 18:55:25 2015 (r290682) +++ head/sys/dev/ntb/if_ntb/if_ntb.c Wed Nov 11 18:55:34 2015 (r290683) @@ -216,6 +216,7 @@ struct ntb_transport_ctx { unsigned qp_count; enum ntb_link_event link_is_up; struct callout link_work; + struct task link_cleanup; uint64_t bufsize; u_char eaddr[ETHER_ADDR_LEN]; struct mtx tx_lock; @@ -304,6 +305,7 @@ static int ntb_transport_setup_qp_mw(str unsigned int qp_num); static void ntb_qp_link_work(void *arg); static void ntb_transport_link_cleanup(struct ntb_transport_ctx *nt); +static void ntb_transport_link_cleanup_work(void *, int); static void ntb_qp_link_down(struct ntb_transport_qp *qp); static void ntb_qp_link_down_reset(struct ntb_transport_qp *qp); static void ntb_qp_link_cleanup(struct ntb_transport_qp *qp); @@ -588,6 +590,7 @@ ntb_transport_probe(struct ntb_softc *nt } callout_init(&nt->link_work, 0); + TASK_INIT(&nt->link_cleanup, 0, ntb_transport_link_cleanup_work, nt); rc = ntb_set_ctx(ntb, nt, &ntb_transport_ops); if (rc != 0) @@ -614,7 +617,7 @@ ntb_transport_free(struct ntb_transport_ uint8_t i; ntb_transport_link_cleanup(nt); - + taskqueue_drain(taskqueue_swi, &nt->link_cleanup); callout_drain(&nt->link_work); BIT_COPY(QP_SETSIZE, &nt->qp_bitmap, &qp_bitmap_alloc); @@ -1178,7 +1181,7 @@ ntb_transport_event_callback(void *data) } else { if (bootverbose) if_printf(nt->ifp, "HW link down\n"); - ntb_transport_link_cleanup(nt); + taskqueue_enqueue(taskqueue_swi, &nt->link_cleanup); } } @@ -1448,6 +1451,12 @@ ntb_transport_link_cleanup(struct ntb_tr ntb_spad_write(nt->ntb, i, 0); } +static void +ntb_transport_link_cleanup_work(void *arg, int pending __unused) +{ + + ntb_transport_link_cleanup(arg); +} static void ntb_qp_link_down(struct ntb_transport_qp *qp) Modified: head/sys/dev/ntb/ntb_hw/ntb_hw.c ============================================================================== --- head/sys/dev/ntb/ntb_hw/ntb_hw.c Wed Nov 11 18:55:25 2015 (r290682) +++ head/sys/dev/ntb/ntb_hw/ntb_hw.c Wed Nov 11 18:55:34 2015 (r290683) @@ -182,8 +182,8 @@ struct ntb_softc { void *ntb_ctx; const struct ntb_ctx_ops *ctx_ops; struct ntb_vec *msix_vec; -#define CTX_LOCK(sc) mtx_lock_spin(&(sc)->ctx_lock) -#define CTX_UNLOCK(sc) mtx_unlock_spin(&(sc)->ctx_lock) +#define CTX_LOCK(sc) mtx_lock(&(sc)->ctx_lock) +#define CTX_UNLOCK(sc) mtx_unlock(&(sc)->ctx_lock) #define CTX_ASSERT(sc,f) mtx_assert(&(sc)->ctx_lock, (f)) struct mtx ctx_lock; @@ -512,7 +512,7 @@ ntb_attach(device_t device) callout_init(&ntb->heartbeat_timer, 1); callout_init(&ntb->lr_timer, 1); mtx_init(&ntb->db_mask_lock, "ntb hw bits", NULL, MTX_SPIN); - mtx_init(&ntb->ctx_lock, "ntb ctx", NULL, MTX_SPIN); + mtx_init(&ntb->ctx_lock, "ntb ctx", NULL, MTX_DEF); if (ntb->type == NTB_ATOM) error = ntb_detect_atom(ntb);