From owner-svn-src-head@freebsd.org Wed May 8 11:09:10 2019 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 11C9215A6FC5; Wed, 8 May 2019 11:09:10 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ABBFB6DFEC; Wed, 8 May 2019 11:09:09 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 867C04F6A; Wed, 8 May 2019 11:09:09 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x48B997n064211; Wed, 8 May 2019 11:09:09 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x48B98co064208; Wed, 8 May 2019 11:09:08 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201905081109.x48B98co064208@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Wed, 8 May 2019 11:09:08 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r347324 - in head/sys/dev/mlx5: . mlx5_core X-SVN-Group: head X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in head/sys/dev/mlx5: . mlx5_core X-SVN-Commit-Revision: 347324 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: ABBFB6DFEC X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.98)[-0.975,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 May 2019 11:09:10 -0000 Author: hselasky Date: Wed May 8 11:09:08 2019 New Revision: 347324 URL: https://svnweb.freebsd.org/changeset/base/347324 Log: Make command workqueue persistant in mlx5core. There is no reason to re-create the command workqueue during healthcare. This also fixes an issue where a previous work struct may refer to a destroyed workqueue. MFC after: 3 days Sponsored by: Mellanox Technologies Modified: head/sys/dev/mlx5/driver.h head/sys/dev/mlx5/mlx5_core/mlx5_cmd.c head/sys/dev/mlx5/mlx5_core/mlx5_health.c Modified: head/sys/dev/mlx5/driver.h ============================================================================== --- head/sys/dev/mlx5/driver.h Wed May 8 11:08:48 2019 (r347323) +++ head/sys/dev/mlx5/driver.h Wed May 8 11:09:08 2019 (r347324) @@ -56,7 +56,6 @@ enum { enum { MLX5_CMD_TIMEOUT_MSEC = 60 * 1000, - MLX5_CMD_WQ_MAX_NAME = 32, }; enum { @@ -353,8 +352,6 @@ struct mlx5_cmd { spinlock_t token_lock; u8 token; unsigned long bitmask; - char wq_name[MLX5_CMD_WQ_MAX_NAME]; - struct workqueue_struct *wq; struct semaphore sem; struct semaphore pages_sem; enum mlx5_cmd_mode mode; @@ -516,6 +513,7 @@ struct mlx5_core_health { struct delayed_work recover_work; unsigned int last_reset_req; struct work_struct work_cmd_completion; + struct workqueue_struct *wq_cmd; }; #ifdef RATELIMIT Modified: head/sys/dev/mlx5/mlx5_core/mlx5_cmd.c ============================================================================== --- head/sys/dev/mlx5/mlx5_core/mlx5_cmd.c Wed May 8 11:08:48 2019 (r347323) +++ head/sys/dev/mlx5/mlx5_core/mlx5_cmd.c Wed May 8 11:09:08 2019 (r347324) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2019, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions @@ -997,7 +997,7 @@ static int mlx5_cmd_invoke(struct mlx5_core_dev *dev, INIT_WORK(&ent->work, cmd_work_handler); if (page_queue) { cmd_work_handler(&ent->work); - } else if (!queue_work(cmd->wq, &ent->work)) { + } else if (!queue_work(dev->priv.health.wq_cmd, &ent->work)) { mlx5_core_warn(dev, "failed to queue work\n"); err = -ENOMEM; goto out_free; @@ -1127,14 +1127,6 @@ mlx5_free_cmd_msg(struct mlx5_core_dev *dev, struct ml mlx5_fwp_free(msg); } -static void set_wqname(struct mlx5_core_dev *dev) -{ - struct mlx5_cmd *cmd = &dev->cmd; - - snprintf(cmd->wq_name, sizeof(cmd->wq_name), "mlx5_cmd_%s", - dev_name(&dev->pdev->dev)); -} - static void clean_debug_files(struct mlx5_core_dev *dev) { } @@ -1562,20 +1554,8 @@ int mlx5_cmd_init(struct mlx5_core_dev *dev) device_printf((&dev->pdev->dev)->bsddev, "ERR: ""failed to create command cache\n"); goto err_free_page; } - - set_wqname(dev); - cmd->wq = create_singlethread_workqueue(cmd->wq_name); - if (!cmd->wq) { - device_printf((&dev->pdev->dev)->bsddev, "ERR: ""failed to create command workqueue\n"); - err = -ENOMEM; - goto err_cache; - } - return 0; -err_cache: - destroy_msg_cache(dev); - err_free_page: free_cmd_page(dev, cmd); @@ -1589,7 +1569,7 @@ void mlx5_cmd_cleanup(struct mlx5_core_dev *dev) struct mlx5_cmd *cmd = &dev->cmd; clean_debug_files(dev); - destroy_workqueue(cmd->wq); + flush_workqueue(dev->priv.health.wq_cmd); destroy_msg_cache(dev); free_cmd_page(dev, cmd); } Modified: head/sys/dev/mlx5/mlx5_core/mlx5_health.c ============================================================================== --- head/sys/dev/mlx5/mlx5_core/mlx5_health.c Wed May 8 11:08:48 2019 (r347323) +++ head/sys/dev/mlx5/mlx5_core/mlx5_health.c Wed May 8 11:09:08 2019 (r347324) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2019, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions @@ -280,8 +280,8 @@ void mlx5_enter_error_state(struct mlx5_core_dev *dev, * completion handler and then wait for it to * complete: */ - queue_work(dev->cmd.wq, &dev->priv.health.work_cmd_completion); - flush_workqueue(dev->cmd.wq); + queue_work(dev->priv.health.wq_cmd, &dev->priv.health.work_cmd_completion); + flush_workqueue(dev->priv.health.wq_cmd); } mutex_lock(&dev->intf_state_mutex); @@ -679,6 +679,7 @@ void mlx5_health_cleanup(struct mlx5_core_dev *dev) destroy_workqueue(health->wq); destroy_workqueue(health->wq_watchdog); + destroy_workqueue(health->wq_cmd); } int mlx5_health_init(struct mlx5_core_dev *dev) @@ -691,15 +692,18 @@ int mlx5_health_init(struct mlx5_core_dev *dev) snprintf(name, sizeof(name), "%s-rec", dev_name(&dev->pdev->dev)); health->wq = create_singlethread_workqueue(name); if (!health->wq) - return -ENOMEM; + goto err_recovery; snprintf(name, sizeof(name), "%s-wdg", dev_name(&dev->pdev->dev)); health->wq_watchdog = create_singlethread_workqueue(name); - if (!health->wq_watchdog) { - destroy_workqueue(health->wq); - return -ENOMEM; - } + if (!health->wq_watchdog) + goto err_watchdog; + snprintf(name, sizeof(name), "%s-cmd", dev_name(&dev->pdev->dev)); + health->wq_cmd = create_singlethread_workqueue(name); + if (!health->wq_cmd) + goto err_cmd; + spin_lock_init(&health->wq_lock); INIT_WORK(&health->work, health_care); INIT_WORK(&health->work_watchdog, health_watchdog); @@ -707,4 +711,11 @@ int mlx5_health_init(struct mlx5_core_dev *dev) INIT_DELAYED_WORK(&health->recover_work, health_recover); return 0; + +err_cmd: + destroy_workqueue(health->wq_watchdog); +err_watchdog: + destroy_workqueue(health->wq); +err_recovery: + return -ENOMEM; }