From owner-svn-src-all@freebsd.org Wed Mar 7 14:35:28 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B5010F31AA4; Wed, 7 Mar 2018 14:35:28 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 63B7D770CA; Wed, 7 Mar 2018 14:35:28 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5A63F22A08; Wed, 7 Mar 2018 14:35:28 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w27EZSj5048069; Wed, 7 Mar 2018 14:35:28 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w27EZSvr048068; Wed, 7 Mar 2018 14:35:28 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201803071435.w27EZSvr048068@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Wed, 7 Mar 2018 14:35:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r330599 - head/sys/dev/mlx5/mlx5_core X-SVN-Group: head X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: head/sys/dev/mlx5/mlx5_core X-SVN-Commit-Revision: 330599 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Mar 2018 14:35:29 -0000 Author: hselasky Date: Wed Mar 7 14:35:28 2018 New Revision: 330599 URL: https://svnweb.freebsd.org/changeset/base/330599 Log: Fix potential deadlock in command mode change in mlx5core. Call command completion handler in case of timeout when working in interrupts mode. Avoid flushing the commands workqueue after acquiring the semaphores to prevent a potential deadlock. linux commit commit 9cba4ebcf374c3772f6eb61f2d065294b2451b49 MFC after: 1 week Sponsored by: Mellanox Technologies Modified: head/sys/dev/mlx5/mlx5_core/mlx5_cmd.c Modified: head/sys/dev/mlx5/mlx5_core/mlx5_cmd.c ============================================================================== --- head/sys/dev/mlx5/mlx5_core/mlx5_cmd.c Wed Mar 7 14:29:30 2018 (r330598) +++ head/sys/dev/mlx5/mlx5_core/mlx5_cmd.c Wed Mar 7 14:35:28 2018 (r330599) @@ -735,11 +735,6 @@ static void cmd_work_handler(struct work_struct *work) struct semaphore *sem; sem = ent->page_queue ? &cmd->pages_sem : &cmd->sem; - if (cmd->moving_to_polling) { - mlx5_core_warn(dev, "not expecting command execution, ignoring...\n"); - return; - } - down(sem); if (alloc_ent(ent) < 0) { @@ -826,13 +821,13 @@ static int wait_func(struct mlx5_core_dev *dev, struct if (cmd->mode == CMD_MODE_POLLING) { wait_for_completion(&ent->done); err = ent->ret; - } else { - if (!wait_for_completion_timeout(&ent->done, timeout)) - err = -ETIMEDOUT; - else - err = 0; - } + } else if (!wait_for_completion_timeout(&ent->done, timeout)) { + ent->ret = -ETIMEDOUT; + mlx5_cmd_comp_handler(dev, 1UL << ent->idx); + } + err = ent->ret; + if (err == -ETIMEDOUT) { mlx5_core_warn(dev, "%s(0x%x) timeout. Will cause a leak of a command resource\n", mlx5_command_str(msg_to_opcode(ent->in)), @@ -881,27 +876,28 @@ static int mlx5_cmd_invoke(struct mlx5_core_dev *dev, goto out_free; } - if (!callback) { - err = wait_func(dev, ent); - if (err == -ETIMEDOUT) - goto out; + if (callback) + goto out; - ds = ent->ts2 - ent->ts1; - op = be16_to_cpu(((struct mlx5_inbox_hdr *)in->first.data)->opcode); - if (op < ARRAY_SIZE(cmd->stats)) { - stats = &cmd->stats[op]; - spin_lock_irq(&stats->lock); - stats->sum += ds; - ++stats->n; - spin_unlock_irq(&stats->lock); - } - mlx5_core_dbg_mask(dev, 1 << MLX5_CMD_TIME, - "fw exec time for %s is %lld nsec\n", - mlx5_command_str(op), (long long)ds); - *status = ent->status; - free_cmd(ent); - } + err = wait_func(dev, ent); + if (err == -ETIMEDOUT) + goto out; + ds = ent->ts2 - ent->ts1; + op = be16_to_cpu(((struct mlx5_inbox_hdr *)in->first.data)->opcode); + if (op < ARRAY_SIZE(cmd->stats)) { + stats = &cmd->stats[op]; + spin_lock_irq(&stats->lock); + stats->sum += ds; + ++stats->n; + spin_unlock_irq(&stats->lock); + } + mlx5_core_dbg_mask(dev, 1 << MLX5_CMD_TIME, + "fw exec time for %s is %lld nsec\n", + mlx5_command_str(op), (long long)ds); + *status = ent->status; + free_cmd(ent); + return err; out_free: @@ -1017,7 +1013,7 @@ static void clean_debug_files(struct mlx5_core_dev *de } -void mlx5_cmd_use_events(struct mlx5_core_dev *dev) +static void mlx5_cmd_change_mod(struct mlx5_core_dev *dev, int mode) { struct mlx5_cmd *cmd = &dev->cmd; int i; @@ -1026,26 +1022,21 @@ void mlx5_cmd_use_events(struct mlx5_core_dev *dev) down(&cmd->sem); down(&cmd->pages_sem); + cmd->mode = mode; - flush_workqueue(cmd->wq); - - cmd->mode = CMD_MODE_EVENTS; - up(&cmd->pages_sem); for (i = 0; i < cmd->max_reg_cmds; i++) up(&cmd->sem); } -void mlx5_cmd_use_polling(struct mlx5_core_dev *dev) +void mlx5_cmd_use_events(struct mlx5_core_dev *dev) { - struct mlx5_cmd *cmd = &dev->cmd; + mlx5_cmd_change_mod(dev, CMD_MODE_EVENTS); +} - synchronize_irq(dev->priv.eq_table.pages_eq.irqn); - flush_workqueue(dev->priv.pg_wq); - cmd->moving_to_polling = 1; - flush_workqueue(cmd->wq); - cmd->mode = CMD_MODE_POLLING; - cmd->moving_to_polling = 0; +void mlx5_cmd_use_polling(struct mlx5_core_dev *dev) +{ + mlx5_cmd_change_mod(dev, CMD_MODE_POLLING); } static void free_msg(struct mlx5_core_dev *dev, struct mlx5_cmd_msg *msg)