From owner-svn-src-all@freebsd.org Thu May 16 18:29:01 2019 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5679F15A3F9F; Thu, 16 May 2019 18:29:01 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EC2316F1CF; Thu, 16 May 2019 18:29:00 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D49D524A22; Thu, 16 May 2019 18:29:00 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x4GIT0Rj041191; Thu, 16 May 2019 18:29:00 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x4GIT0Ks041186; Thu, 16 May 2019 18:29:00 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201905161829.x4GIT0Ks041186@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 16 May 2019 18:29:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r347881 - in stable/11/sys/dev/mlx5: . mlx5_core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in stable/11/sys/dev/mlx5: . mlx5_core X-SVN-Commit-Revision: 347881 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: EC2316F1CF X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_SHORT(-0.99)[-0.988,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 May 2019 18:29:01 -0000 Author: hselasky Date: Thu May 16 18:28:59 2019 New Revision: 347881 URL: https://svnweb.freebsd.org/changeset/base/347881 Log: MFC r347324: Make command workqueue persistant in mlx5core. There is no reason to re-create the command workqueue during healthcare. This also fixes an issue where a previous work struct may refer to a destroyed workqueue. Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx5/driver.h stable/11/sys/dev/mlx5/mlx5_core/mlx5_cmd.c stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx5/driver.h ============================================================================== --- stable/11/sys/dev/mlx5/driver.h Thu May 16 18:28:12 2019 (r347880) +++ stable/11/sys/dev/mlx5/driver.h Thu May 16 18:28:59 2019 (r347881) @@ -54,7 +54,6 @@ enum { enum { MLX5_CMD_TIMEOUT_MSEC = 60 * 1000, - MLX5_CMD_WQ_MAX_NAME = 32, }; enum { @@ -351,8 +350,6 @@ struct mlx5_cmd { spinlock_t token_lock; u8 token; unsigned long bitmask; - char wq_name[MLX5_CMD_WQ_MAX_NAME]; - struct workqueue_struct *wq; struct semaphore sem; struct semaphore pages_sem; enum mlx5_cmd_mode mode; @@ -514,6 +511,7 @@ struct mlx5_core_health { struct delayed_work recover_work; unsigned int last_reset_req; struct work_struct work_cmd_completion; + struct workqueue_struct *wq_cmd; }; #define MLX5_CQ_LINEAR_ARRAY_SIZE 1024 Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_cmd.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_cmd.c Thu May 16 18:28:12 2019 (r347880) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_cmd.c Thu May 16 18:28:59 2019 (r347881) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2019, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions @@ -997,7 +997,7 @@ static int mlx5_cmd_invoke(struct mlx5_core_dev *dev, INIT_WORK(&ent->work, cmd_work_handler); if (page_queue) { cmd_work_handler(&ent->work); - } else if (!queue_work(cmd->wq, &ent->work)) { + } else if (!queue_work(dev->priv.health.wq_cmd, &ent->work)) { mlx5_core_warn(dev, "failed to queue work\n"); err = -ENOMEM; goto out_free; @@ -1127,14 +1127,6 @@ mlx5_free_cmd_msg(struct mlx5_core_dev *dev, struct ml mlx5_fwp_free(msg); } -static void set_wqname(struct mlx5_core_dev *dev) -{ - struct mlx5_cmd *cmd = &dev->cmd; - - snprintf(cmd->wq_name, sizeof(cmd->wq_name), "mlx5_cmd_%s", - dev_name(&dev->pdev->dev)); -} - static void clean_debug_files(struct mlx5_core_dev *dev) { } @@ -1562,20 +1554,8 @@ int mlx5_cmd_init(struct mlx5_core_dev *dev) device_printf((&dev->pdev->dev)->bsddev, "ERR: ""failed to create command cache\n"); goto err_free_page; } - - set_wqname(dev); - cmd->wq = create_singlethread_workqueue(cmd->wq_name); - if (!cmd->wq) { - device_printf((&dev->pdev->dev)->bsddev, "ERR: ""failed to create command workqueue\n"); - err = -ENOMEM; - goto err_cache; - } - return 0; -err_cache: - destroy_msg_cache(dev); - err_free_page: free_cmd_page(dev, cmd); @@ -1589,7 +1569,7 @@ void mlx5_cmd_cleanup(struct mlx5_core_dev *dev) struct mlx5_cmd *cmd = &dev->cmd; clean_debug_files(dev); - destroy_workqueue(cmd->wq); + flush_workqueue(dev->priv.health.wq_cmd); destroy_msg_cache(dev); free_cmd_page(dev, cmd); } Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c Thu May 16 18:28:12 2019 (r347880) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c Thu May 16 18:28:59 2019 (r347881) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2019, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions @@ -280,8 +280,8 @@ void mlx5_enter_error_state(struct mlx5_core_dev *dev, * completion handler and then wait for it to * complete: */ - queue_work(dev->cmd.wq, &dev->priv.health.work_cmd_completion); - flush_workqueue(dev->cmd.wq); + queue_work(dev->priv.health.wq_cmd, &dev->priv.health.work_cmd_completion); + flush_workqueue(dev->priv.health.wq_cmd); } mutex_lock(&dev->intf_state_mutex); @@ -679,6 +679,7 @@ void mlx5_health_cleanup(struct mlx5_core_dev *dev) destroy_workqueue(health->wq); destroy_workqueue(health->wq_watchdog); + destroy_workqueue(health->wq_cmd); } int mlx5_health_init(struct mlx5_core_dev *dev) @@ -691,15 +692,18 @@ int mlx5_health_init(struct mlx5_core_dev *dev) snprintf(name, sizeof(name), "%s-rec", dev_name(&dev->pdev->dev)); health->wq = create_singlethread_workqueue(name); if (!health->wq) - return -ENOMEM; + goto err_recovery; snprintf(name, sizeof(name), "%s-wdg", dev_name(&dev->pdev->dev)); health->wq_watchdog = create_singlethread_workqueue(name); - if (!health->wq_watchdog) { - destroy_workqueue(health->wq); - return -ENOMEM; - } + if (!health->wq_watchdog) + goto err_watchdog; + snprintf(name, sizeof(name), "%s-cmd", dev_name(&dev->pdev->dev)); + health->wq_cmd = create_singlethread_workqueue(name); + if (!health->wq_cmd) + goto err_cmd; + spin_lock_init(&health->wq_lock); INIT_WORK(&health->work, health_care); INIT_WORK(&health->work_watchdog, health_watchdog); @@ -707,4 +711,11 @@ int mlx5_health_init(struct mlx5_core_dev *dev) INIT_DELAYED_WORK(&health->recover_work, health_recover); return 0; + +err_cmd: + destroy_workqueue(health->wq_watchdog); +err_watchdog: + destroy_workqueue(health->wq); +err_recovery: + return -ENOMEM; }