From owner-svn-src-all@freebsd.org Wed Oct 9 16:48:48 2019 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E487F128001; Wed, 9 Oct 2019 16:48:48 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46pKt85hqQz4dwx; Wed, 9 Oct 2019 16:48:48 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8E9551B64D; Wed, 9 Oct 2019 16:48:48 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x99GmmBk080501; Wed, 9 Oct 2019 16:48:48 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x99GmmWp080500; Wed, 9 Oct 2019 16:48:48 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201910091648.x99GmmWp080500@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Wed, 9 Oct 2019 16:48:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r353353 - head/sys/netinet X-SVN-Group: head X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: head/sys/netinet X-SVN-Commit-Revision: 353353 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Oct 2019 16:48:49 -0000 Author: hselasky Date: Wed Oct 9 16:48:48 2019 New Revision: 353353 URL: https://svnweb.freebsd.org/changeset/base/353353 Log: Fix locking order reversal in the TCP ratelimit code by moving destructors outside the rsmtx mutex. Witness message: lock order reversal: (sleepable after non-sleepable) 1st tcp_rs_mtx (rsmtx) @ sys/netinet/tcp_ratelimit.c:242 2nd sysctl lock (sysctl lock) @ sys/kern/kern_sysctl.c:607 Backtrace: witness_debugger witness_checkorder _rm_wlock_debug sysctl_ctx_free rs_destroy epoch_call_task gtaskqueue_run_locked gtaskqueue_thread_loop Discussed with: rrs@ Sponsored by: Mellanox Technologies Modified: head/sys/netinet/tcp_ratelimit.c Modified: head/sys/netinet/tcp_ratelimit.c ============================================================================== --- head/sys/netinet/tcp_ratelimit.c Wed Oct 9 16:42:51 2019 (r353352) +++ head/sys/netinet/tcp_ratelimit.c Wed Oct 9 16:48:48 2019 (r353353) @@ -237,34 +237,37 @@ static void rs_destroy(epoch_context_t ctx) { struct tcp_rate_set *rs; + bool do_free_rs; rs = __containerof(ctx, struct tcp_rate_set, rs_epoch_ctx); + mtx_lock(&rs_mtx); rs->rs_flags &= ~RS_FUNERAL_SCHD; - if (rs->rs_flows_using == 0) { - /* - * In theory its possible (but unlikely) - * that while the delete was occuring - * and we were applying the DEAD flag - * someone slipped in and found the - * interface in a lookup. While we - * decided rs_flows_using were 0 and - * scheduling the epoch_call, the other - * thread incremented rs_flow_using. This - * is because users have a pointer and - * we only use the rs_flows_using in an - * atomic fashion, i.e. the other entities - * are not protected. To assure this did - * not occur, we check rs_flows_using here - * before deleteing. - */ + /* + * In theory its possible (but unlikely) + * that while the delete was occuring + * and we were applying the DEAD flag + * someone slipped in and found the + * interface in a lookup. While we + * decided rs_flows_using were 0 and + * scheduling the epoch_call, the other + * thread incremented rs_flow_using. This + * is because users have a pointer and + * we only use the rs_flows_using in an + * atomic fashion, i.e. the other entities + * are not protected. To assure this did + * not occur, we check rs_flows_using here + * before deleting. + */ + do_free_rs = (rs->rs_flows_using == 0); + rs_number_dead--; + mtx_unlock(&rs_mtx); + + if (do_free_rs) { sysctl_ctx_free(&rs->sysctl_ctx); free(rs->rs_rlt, M_TCPPACE); free(rs, M_TCPPACE); - rs_number_dead--; } - mtx_unlock(&rs_mtx); - } #ifdef INET