From owner-svn-src-head@freebsd.org Fri Nov 10 09:49:10 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C00F7E69916; Fri, 10 Nov 2017 09:49:10 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3E0036D2F5; Fri, 10 Nov 2017 09:49:10 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id vAA9n9X3008435; Fri, 10 Nov 2017 09:49:09 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id vAA9n8Sw008427; Fri, 10 Nov 2017 09:49:08 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201711100949.vAA9n8Sw008427@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Fri, 10 Nov 2017 09:49:08 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r325638 - in head/sys: conf dev/mlx5 dev/mlx5/mlx5_core dev/mlx5/mlx5_en modules/mlx5 X-SVN-Group: head X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in head/sys: conf dev/mlx5 dev/mlx5/mlx5_core dev/mlx5/mlx5_en modules/mlx5 X-SVN-Commit-Revision: 325638 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Nov 2017 09:49:10 -0000 Author: hselasky Date: Fri Nov 10 09:49:08 2017 New Revision: 325638 URL: https://svnweb.freebsd.org/changeset/base/325638 Log: Refactor the flowsteering APIs used by mlx5en(4). This change is needed by the coming ibcore and mlx5ib updates in order to support traffic redirection to so-called raw ethernet QPs. Remove unused E-switch related routines and files while at it. Sponsored by: Mellanox Technologies MFC after: 1 week Added: head/sys/dev/mlx5/fs.h (contents, props changed) head/sys/dev/mlx5/mlx5_core/fs_core.h (contents, props changed) head/sys/dev/mlx5/mlx5_core/mlx5_fs_cmd.c (contents, props changed) head/sys/dev/mlx5/mlx5_core/mlx5_fs_tree.c (contents, props changed) Deleted: head/sys/dev/mlx5/eswitch_vacl.h head/sys/dev/mlx5/flow_table.h head/sys/dev/mlx5/mlx5_core/mlx5_eswitch_vacl.c head/sys/dev/mlx5/mlx5_core/mlx5_flow_table.c Modified: head/sys/conf/files head/sys/dev/mlx5/device.h head/sys/dev/mlx5/driver.h head/sys/dev/mlx5/mlx5_core/mlx5_main.c head/sys/dev/mlx5/mlx5_en/en.h head/sys/dev/mlx5/mlx5_en/mlx5_en_flow_table.c head/sys/dev/mlx5/mlx5_ifc.h head/sys/modules/mlx5/Makefile Modified: head/sys/conf/files ============================================================================== --- head/sys/conf/files Fri Nov 10 08:58:42 2017 (r325637) +++ head/sys/conf/files Fri Nov 10 09:49:08 2017 (r325638) @@ -4644,7 +4644,9 @@ dev/mlx5/mlx5_core/mlx5_diagnostics.c optional mlx5 p compile-with "${OFED_C}" dev/mlx5/mlx5_core/mlx5_eq.c optional mlx5 pci \ compile-with "${OFED_C}" -dev/mlx5/mlx5_core/mlx5_flow_table.c optional mlx5 pci \ +dev/mlx5/mlx5_core/mlx5_fs_cmd.c optional mlx5 pci \ + compile-with "${OFED_C}" +dev/mlx5/mlx5_core/mlx5_fs_tree.c optional mlx5 pci \ compile-with "${OFED_C}" dev/mlx5/mlx5_core/mlx5_fw.c optional mlx5 pci \ compile-with "${OFED_C}" Modified: head/sys/dev/mlx5/device.h ============================================================================== --- head/sys/dev/mlx5/device.h Fri Nov 10 08:58:42 2017 (r325637) +++ head/sys/dev/mlx5/device.h Fri Nov 10 09:49:08 2017 (r325638) @@ -1085,6 +1085,7 @@ enum { MLX5_FLOW_TABLE_TYPE_ESWITCH = 4, MLX5_FLOW_TABLE_TYPE_SNIFFER_RX = 5, MLX5_FLOW_TABLE_TYPE_SNIFFER_TX = 6, + MLX5_FLOW_TABLE_TYPE_NIC_RX_RDMA = 7, }; enum { Modified: head/sys/dev/mlx5/driver.h ============================================================================== --- head/sys/dev/mlx5/driver.h Fri Nov 10 08:58:42 2017 (r325637) +++ head/sys/dev/mlx5/driver.h Fri Nov 10 09:49:08 2017 (r325638) @@ -582,6 +582,7 @@ struct mlx5_special_contexts { int resd_lkey; }; +struct mlx5_flow_root_namespace; struct mlx5_core_dev { struct pci_dev *pdev; char board_id[MLX5_BOARD_ID_LEN]; @@ -600,6 +601,12 @@ struct mlx5_core_dev { u32 issi; struct mlx5_special_contexts special_contexts; unsigned int module_status[MLX5_MAX_PORTS]; + struct mlx5_flow_root_namespace *root_ns; + struct mlx5_flow_root_namespace *fdb_root_ns; + struct mlx5_flow_root_namespace *esw_egress_root_ns; + struct mlx5_flow_root_namespace *esw_ingress_root_ns; + struct mlx5_flow_root_namespace *sniffer_rx_root_ns; + struct mlx5_flow_root_namespace *sniffer_tx_root_ns; u32 num_q_counter_allocated[MLX5_INTERFACE_NUMBER]; }; Added: head/sys/dev/mlx5/fs.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/dev/mlx5/fs.h Fri Nov 10 09:49:08 2017 (r325638) @@ -0,0 +1,232 @@ +/*- + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS `AS IS' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef _MLX5_FS_ +#define _MLX5_FS_ + +#include + +#include +#include +#include + +enum { + MLX5_FLOW_CONTEXT_ACTION_FWD_NEXT_PRIO = 1 << 16, +}; + +/*Flow tag*/ +enum { + MLX5_FS_DEFAULT_FLOW_TAG = 0xFFFFFF, + MLX5_FS_ETH_FLOW_TAG = 0xFFFFFE, + MLX5_FS_SNIFFER_FLOW_TAG = 0xFFFFFD, +}; + +enum { + MLX5_FS_FLOW_TAG_MASK = 0xFFFFFF, +}; + +#define FS_MAX_TYPES 10 +#define FS_MAX_ENTRIES 32000U + +enum mlx5_flow_namespace_type { + MLX5_FLOW_NAMESPACE_BYPASS, + MLX5_FLOW_NAMESPACE_KERNEL, + MLX5_FLOW_NAMESPACE_LEFTOVERS, + MLX5_FLOW_NAMESPACE_SNIFFER_RX, + MLX5_FLOW_NAMESPACE_SNIFFER_TX, + MLX5_FLOW_NAMESPACE_FDB, + MLX5_FLOW_NAMESPACE_ESW_EGRESS, + MLX5_FLOW_NAMESPACE_ESW_INGRESS, +}; + +struct mlx5_flow_table; +struct mlx5_flow_group; +struct mlx5_flow_rule; +struct mlx5_flow_namespace; + +struct mlx5_flow_spec { + u8 match_criteria_enable; + u32 match_criteria[MLX5_ST_SZ_DW(fte_match_param)]; + u32 match_value[MLX5_ST_SZ_DW(fte_match_param)]; +}; + +struct mlx5_flow_destination { + u32 type; + union { + u32 tir_num; + struct mlx5_flow_table *ft; + u32 vport_num; + }; +}; + +#define FT_NAME_STR_SZ 20 +#define LEFTOVERS_RULE_NUM 2 +static inline void build_leftovers_ft_param(char *name, + unsigned int *priority, + int *n_ent, + int *n_grp) +{ + snprintf(name, FT_NAME_STR_SZ, "leftovers"); + *priority = 0; /*Priority of leftovers_prio-0*/ + *n_ent = LEFTOVERS_RULE_NUM + 1; /*1: star rules*/ + *n_grp = LEFTOVERS_RULE_NUM; +} + +static inline bool outer_header_zero(u32 *match_criteria) +{ + int size = MLX5_ST_SZ_BYTES(fte_match_param); + char *outer_headers_c = MLX5_ADDR_OF(fte_match_param, match_criteria, + outer_headers); + + return outer_headers_c[0] == 0 && !memcmp(outer_headers_c, + outer_headers_c + 1, + size - 1); +} + +struct mlx5_flow_namespace * +mlx5_get_flow_namespace(struct mlx5_core_dev *dev, + enum mlx5_flow_namespace_type type); + +/* The underlying implementation create two more entries for + * chaining flow tables. the user should be aware that if he pass + * max_num_ftes as 2^N it will result in doubled size flow table + */ +struct mlx5_flow_table * +mlx5_create_auto_grouped_flow_table(struct mlx5_flow_namespace *ns, + int prio, + const char *name, + int num_flow_table_entries, + int max_num_groups); + +struct mlx5_flow_table * +mlx5_create_vport_flow_table(struct mlx5_flow_namespace *ns, + u16 vport, + int prio, + const char *name, + int num_flow_table_entries); + +struct mlx5_flow_table * +mlx5_create_flow_table(struct mlx5_flow_namespace *ns, + int prio, + const char *name, + int num_flow_table_entries); +int mlx5_destroy_flow_table(struct mlx5_flow_table *ft); + +/* inbox should be set with the following values: + * start_flow_index + * end_flow_index + * match_criteria_enable + * match_criteria + */ +struct mlx5_flow_group * +mlx5_create_flow_group(struct mlx5_flow_table *ft, u32 *in); +void mlx5_destroy_flow_group(struct mlx5_flow_group *fg); + +/* Single destination per rule. + * Group ID is implied by the match criteria. + */ +struct mlx5_flow_rule * +mlx5_add_flow_rule(struct mlx5_flow_table *ft, + u8 match_criteria_enable, + u32 *match_criteria, + u32 *match_value, + u32 action, + u32 flow_tag, + struct mlx5_flow_destination *dest); +void mlx5_del_flow_rule(struct mlx5_flow_rule *fr); + +/*The following API is for sniffer*/ +typedef int (*rule_event_fn)(struct mlx5_flow_rule *rule, + bool ctx_changed, + void *client_data, + void *context); + +struct mlx5_flow_handler; + +struct flow_client_priv_data; + +void mlx5e_sniffer_roce_mode_notify( + struct mlx5_core_dev *mdev, + int action); + +int mlx5_set_rule_private_data(struct mlx5_flow_rule *rule, struct + mlx5_flow_handler *handler, void + *client_data); + +struct mlx5_flow_handler *mlx5_register_rule_notifier(struct mlx5_core_dev *dev, + enum mlx5_flow_namespace_type ns_type, + rule_event_fn add_cb, + rule_event_fn del_cb, + void *context); + +void mlx5_unregister_rule_notifier(struct mlx5_flow_handler *handler); + +void mlx5_flow_iterate_existing_rules(struct mlx5_flow_namespace *ns, + rule_event_fn cb, + void *context); + +void mlx5_get_match_criteria(u32 *match_criteria, + struct mlx5_flow_rule *rule); + +void mlx5_get_match_value(u32 *match_value, + struct mlx5_flow_rule *rule); + +u8 mlx5_get_match_criteria_enable(struct mlx5_flow_rule *rule); + +struct mlx5_flow_rules_list *get_roce_flow_rules(u8 roce_mode); + +void mlx5_del_flow_rules_list(struct mlx5_flow_rules_list *rules_list); + +struct mlx5_flow_rules_list { + struct list_head head; +}; + +struct mlx5_flow_rule_node { + struct list_head list; + u32 match_criteria[MLX5_ST_SZ_DW(fte_match_param)]; + u32 match_value[MLX5_ST_SZ_DW(fte_match_param)]; + u8 match_criteria_enable; +}; + +struct mlx5_core_fs_mask { + u8 match_criteria_enable; + u32 match_criteria[MLX5_ST_SZ_DW(fte_match_param)]; +}; + +bool fs_match_exact_val( + struct mlx5_core_fs_mask *mask, + void *val1, + void *val2); + +bool fs_match_exact_mask( + u8 match_criteria_enable1, + u8 match_criteria_enable2, + void *mask1, + void *mask2); +/**********end API for sniffer**********/ + +#endif Added: head/sys/dev/mlx5/mlx5_core/fs_core.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/dev/mlx5/mlx5_core/fs_core.h Fri Nov 10 09:49:08 2017 (r325638) @@ -0,0 +1,300 @@ +/*- + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS `AS IS' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef _MLX5_FS_CORE_ +#define _MLX5_FS_CORE_ + +#include +#include +#include +#include + +enum fs_type { + FS_TYPE_NAMESPACE, + FS_TYPE_PRIO, + FS_TYPE_FLOW_TABLE, + FS_TYPE_FLOW_GROUP, + FS_TYPE_FLOW_ENTRY, + FS_TYPE_FLOW_DEST +}; + +enum fs_ft_type { + FS_FT_NIC_RX = 0x0, + FS_FT_ESW_EGRESS_ACL = 0x2, + FS_FT_ESW_INGRESS_ACL = 0x3, + FS_FT_FDB = 0X4, + FS_FT_SNIFFER_RX = 0x5, + FS_FT_SNIFFER_TX = 0x6 +}; + +enum fs_fte_status { + FS_FTE_STATUS_EXISTING = 1UL << 0, +}; + +/* Should always be the first variable in the struct */ +struct fs_base { + struct list_head list; + struct fs_base *parent; + enum fs_type type; + struct kref refcount; + /* lock the node for writing and traversing */ + struct mutex lock; + struct completion complete; + atomic_t users_refcount; + const char *name; +}; + +struct mlx5_flow_rule { + struct fs_base base; + struct mlx5_flow_destination dest_attr; + struct list_head clients_data; + /*protect clients lits*/ + struct mutex clients_lock; +}; + +struct fs_fte { + struct fs_base base; + u32 val[MLX5_ST_SZ_DW(fte_match_param)]; + uint32_t dests_size; + uint32_t flow_tag; + struct list_head dests; + uint32_t index; /* index in ft */ + u8 action; /* MLX5_FLOW_CONTEXT_ACTION */ + enum fs_fte_status status; +}; + +struct fs_star_rule { + struct mlx5_flow_group *fg; + struct fs_fte *fte; +}; + +struct mlx5_flow_table { + struct fs_base base; + /* sorted list by start_index */ + struct list_head fgs; + struct { + bool active; + unsigned int max_types; + unsigned int num_types; + } autogroup; + unsigned int max_fte; + unsigned int level; + uint32_t id; + u16 vport; + enum fs_ft_type type; + struct fs_star_rule star_rule; + unsigned int shared_refcount; +}; + +enum fs_prio_flags { + MLX5_CORE_FS_PRIO_SHARED = 1 +}; + +struct fs_prio { + struct fs_base base; + struct list_head objs; /* each object is a namespace or ft */ + unsigned int max_ft; + unsigned int num_ft; + unsigned int max_ns; + unsigned int prio; + /*When create shared flow table, this lock should be taken*/ + struct mutex shared_lock; + u8 flags; +}; + +struct mlx5_flow_namespace { + /* parent == NULL => root ns */ + struct fs_base base; + /* sorted by priority number */ + struct list_head prios; /* list of fs_prios */ + struct list_head list_notifiers; + struct rw_semaphore notifiers_rw_sem; + struct rw_semaphore dests_rw_sem; +}; + +struct mlx5_flow_root_namespace { + struct mlx5_flow_namespace ns; + struct mlx5_flow_table *ft_level_0; + enum fs_ft_type table_type; + struct mlx5_core_dev *dev; + struct mlx5_flow_table *root_ft; + /* When chaining flow-tables, this lock should be taken */ + struct mutex fs_chain_lock; +}; + +struct mlx5_flow_group { + struct fs_base base; + struct list_head ftes; + struct mlx5_core_fs_mask mask; + uint32_t start_index; + uint32_t max_ftes; + uint32_t num_ftes; + uint32_t id; +}; + +struct mlx5_flow_handler { + struct list_head list; + rule_event_fn add_dst_cb; + rule_event_fn del_dst_cb; + void *client_context; + struct mlx5_flow_namespace *ns; +}; + +struct fs_client_priv_data { + struct mlx5_flow_handler *fs_handler; + struct list_head list; + void *client_dst_data; +}; + +void _fs_remove_node(struct kref *kref); +#define fs_get_obj(v, _base) {v = container_of((_base), typeof(*v), base); } +#define fs_get_parent(v, child) {v = (child)->base.parent ? \ + container_of((child)->base.parent, \ + typeof(*v), base) : NULL; } + +#define fs_list_for_each_entry(pos, cond, root) \ + list_for_each_entry(pos, root, base.list) \ + if (!(cond)) {} else + +#define fs_list_for_each_entry_continue(pos, cond, root) \ + list_for_each_entry_continue(pos, root, base.list) \ + if (!(cond)) {} else + +#define fs_list_for_each_entry_reverse(pos, cond, root) \ + list_for_each_entry_reverse(pos, root, base.list) \ + if (!(cond)) {} else + +#define fs_list_for_each_entry_continue_reverse(pos, cond, root) \ + list_for_each_entry_continue_reverse(pos, root, base.list) \ + if (!(cond)) {} else + +#define fs_for_each_ft(pos, prio) \ + fs_list_for_each_entry(pos, (pos)->base.type == FS_TYPE_FLOW_TABLE, \ + &(prio)->objs) + +#define fs_for_each_ft_reverse(pos, prio) \ + fs_list_for_each_entry_reverse(pos, \ + (pos)->base.type == FS_TYPE_FLOW_TABLE, \ + &(prio)->objs) + +#define fs_for_each_ns(pos, prio) \ + fs_list_for_each_entry(pos, \ + (pos)->base.type == FS_TYPE_NAMESPACE, \ + &(prio)->objs) + +#define fs_for_each_ns_or_ft_reverse(pos, prio) \ + list_for_each_entry_reverse(pos, &(prio)->objs, list) \ + if (!((pos)->type == FS_TYPE_NAMESPACE || \ + (pos)->type == FS_TYPE_FLOW_TABLE)) {} else + +#define fs_for_each_ns_or_ft(pos, prio) \ + list_for_each_entry(pos, &(prio)->objs, list) \ + if (!((pos)->type == FS_TYPE_NAMESPACE || \ + (pos)->type == FS_TYPE_FLOW_TABLE)) {} else + +#define fs_for_each_ns_or_ft_continue_reverse(pos, prio) \ + list_for_each_entry_continue_reverse(pos, &(prio)->objs, list) \ + if (!((pos)->type == FS_TYPE_NAMESPACE || \ + (pos)->type == FS_TYPE_FLOW_TABLE)) {} else + +#define fs_for_each_ns_or_ft_continue(pos, prio) \ + list_for_each_entry_continue(pos, &(prio)->objs, list) \ + if (!((pos)->type == FS_TYPE_NAMESPACE || \ + (pos)->type == FS_TYPE_FLOW_TABLE)) {} else + +#define fs_for_each_prio(pos, ns) \ + fs_list_for_each_entry(pos, (pos)->base.type == FS_TYPE_PRIO, \ + &(ns)->prios) + +#define fs_for_each_prio_reverse(pos, ns) \ + fs_list_for_each_entry_reverse(pos, (pos)->base.type == FS_TYPE_PRIO, \ + &(ns)->prios) + +#define fs_for_each_prio_continue(pos, ns) \ + fs_list_for_each_entry_continue(pos, (pos)->base.type == FS_TYPE_PRIO, \ + &(ns)->prios) + +#define fs_for_each_prio_continue_reverse(pos, ns) \ + fs_list_for_each_entry_continue_reverse(pos, \ + (pos)->base.type == FS_TYPE_PRIO, \ + &(ns)->prios) + +#define fs_for_each_fg(pos, ft) \ + fs_list_for_each_entry(pos, (pos)->base.type == FS_TYPE_FLOW_GROUP, \ + &(ft)->fgs) + +#define fs_for_each_fte(pos, fg) \ + fs_list_for_each_entry(pos, (pos)->base.type == FS_TYPE_FLOW_ENTRY, \ + &(fg)->ftes) +#define fs_for_each_dst(pos, fte) \ + fs_list_for_each_entry(pos, (pos)->base.type == FS_TYPE_FLOW_DEST, \ + &(fte)->dests) + +int mlx5_cmd_fs_create_ft(struct mlx5_core_dev *dev, + u16 vport, + enum fs_ft_type type, unsigned int level, + unsigned int log_size, unsigned int *table_id); + +int mlx5_cmd_fs_destroy_ft(struct mlx5_core_dev *dev, + u16 vport, + enum fs_ft_type type, unsigned int table_id); + +int mlx5_cmd_fs_create_fg(struct mlx5_core_dev *dev, + u32 *in, + u16 vport, + enum fs_ft_type type, unsigned int table_id, + unsigned int *group_id); + +int mlx5_cmd_fs_destroy_fg(struct mlx5_core_dev *dev, + u16 vport, + enum fs_ft_type type, unsigned int table_id, + unsigned int group_id); + + +int mlx5_cmd_fs_set_fte(struct mlx5_core_dev *dev, + u16 vport, + enum fs_fte_status *fte_status, + u32 *match_val, + enum fs_ft_type type, unsigned int table_id, + unsigned int index, unsigned int group_id, + unsigned int flow_tag, + unsigned short action, int dest_size, + struct list_head *dests); /* mlx5_flow_desination */ + +int mlx5_cmd_fs_delete_fte(struct mlx5_core_dev *dev, + u16 vport, + enum fs_fte_status *fte_status, + enum fs_ft_type type, unsigned int table_id, + unsigned int index); + +int mlx5_cmd_update_root_ft(struct mlx5_core_dev *dev, + enum fs_ft_type type, + unsigned int id); + +int mlx5_init_fs(struct mlx5_core_dev *dev); +void mlx5_cleanup_fs(struct mlx5_core_dev *dev); +#endif Added: head/sys/dev/mlx5/mlx5_core/mlx5_fs_cmd.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/dev/mlx5/mlx5_core/mlx5_fs_cmd.c Fri Nov 10 09:49:08 2017 (r325638) @@ -0,0 +1,302 @@ +/*- + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS `AS IS' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include +#include +#include +#include +#include + +#include "fs_core.h" +#include "mlx5_core.h" + +int mlx5_cmd_update_root_ft(struct mlx5_core_dev *dev, + enum fs_ft_type type, + unsigned int id) +{ + u32 in[MLX5_ST_SZ_DW(set_flow_table_root_in)]; + u32 out[MLX5_ST_SZ_DW(set_flow_table_root_out)]; + + if (!dev) + return -EINVAL; + memset(in, 0, sizeof(in)); + + MLX5_SET(set_flow_table_root_in, in, opcode, + MLX5_CMD_OP_SET_FLOW_TABLE_ROOT); + MLX5_SET(set_flow_table_root_in, in, table_type, type); + MLX5_SET(set_flow_table_root_in, in, table_id, id); + + memset(out, 0, sizeof(out)); + return mlx5_cmd_exec_check_status(dev, in, sizeof(in), out, + sizeof(out)); +} + +int mlx5_cmd_fs_create_ft(struct mlx5_core_dev *dev, + u16 vport, + enum fs_ft_type type, unsigned int level, + unsigned int log_size, unsigned int *table_id) +{ + u32 in[MLX5_ST_SZ_DW(create_flow_table_in)]; + u32 out[MLX5_ST_SZ_DW(create_flow_table_out)]; + int err; + + if (!dev) + return -EINVAL; + memset(in, 0, sizeof(in)); + + MLX5_SET(create_flow_table_in, in, opcode, + MLX5_CMD_OP_CREATE_FLOW_TABLE); + + MLX5_SET(create_flow_table_in, in, table_type, type); + MLX5_SET(create_flow_table_in, in, flow_table_context.level, level); + MLX5_SET(create_flow_table_in, in, flow_table_context.log_size, + log_size); + if (vport) { + MLX5_SET(create_flow_table_in, in, vport_number, vport); + MLX5_SET(create_flow_table_in, in, other_vport, 1); + } + + memset(out, 0, sizeof(out)); + err = mlx5_cmd_exec_check_status(dev, in, sizeof(in), out, + sizeof(out)); + if (err) + return err; + + *table_id = MLX5_GET(create_flow_table_out, out, table_id); + + return 0; +} + +int mlx5_cmd_fs_destroy_ft(struct mlx5_core_dev *dev, + u16 vport, + enum fs_ft_type type, unsigned int table_id) +{ + u32 in[MLX5_ST_SZ_DW(destroy_flow_table_in)]; + u32 out[MLX5_ST_SZ_DW(destroy_flow_table_out)]; + + if (!dev) + return -EINVAL; + memset(in, 0, sizeof(in)); + memset(out, 0, sizeof(out)); + + MLX5_SET(destroy_flow_table_in, in, opcode, + MLX5_CMD_OP_DESTROY_FLOW_TABLE); + MLX5_SET(destroy_flow_table_in, in, table_type, type); + MLX5_SET(destroy_flow_table_in, in, table_id, table_id); + if (vport) { + MLX5_SET(destroy_flow_table_in, in, vport_number, vport); + MLX5_SET(destroy_flow_table_in, in, other_vport, 1); + } + + return mlx5_cmd_exec_check_status(dev, in, sizeof(in), out, sizeof(out)); +} + +int mlx5_cmd_fs_create_fg(struct mlx5_core_dev *dev, + u32 *in, + u16 vport, + enum fs_ft_type type, unsigned int table_id, + unsigned int *group_id) +{ + u32 out[MLX5_ST_SZ_DW(create_flow_group_out)]; + int err; + int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in); + if (!dev) + return -EINVAL; + memset(out, 0, sizeof(out)); + + MLX5_SET(create_flow_group_in, in, opcode, + MLX5_CMD_OP_CREATE_FLOW_GROUP); + MLX5_SET(create_flow_group_in, in, table_type, type); + MLX5_SET(create_flow_group_in, in, table_id, table_id); + if (vport) { + MLX5_SET(create_flow_group_in, in, vport_number, vport); + MLX5_SET(create_flow_group_in, in, other_vport, 1); + } + + err = mlx5_cmd_exec_check_status(dev, in, + inlen, out, + sizeof(out)); + if (!err) + *group_id = MLX5_GET(create_flow_group_out, out, group_id); + + return err; +} + +int mlx5_cmd_fs_destroy_fg(struct mlx5_core_dev *dev, + u16 vport, + enum fs_ft_type type, unsigned int table_id, + unsigned int group_id) +{ + u32 in[MLX5_ST_SZ_DW(destroy_flow_group_in)]; + u32 out[MLX5_ST_SZ_DW(destroy_flow_group_out)]; + + if (!dev) + return -EINVAL; + memset(in, 0, sizeof(in)); + memset(out, 0, sizeof(out)); + + MLX5_SET(destroy_flow_group_in, in, opcode, + MLX5_CMD_OP_DESTROY_FLOW_GROUP); + MLX5_SET(destroy_flow_group_in, in, table_type, type); + MLX5_SET(destroy_flow_group_in, in, table_id, table_id); + MLX5_SET(destroy_flow_group_in, in, group_id, group_id); + if (vport) { + MLX5_SET(destroy_flow_group_in, in, vport_number, vport); + MLX5_SET(destroy_flow_group_in, in, other_vport, 1); + } + + return mlx5_cmd_exec_check_status(dev, in, sizeof(in), out, sizeof(out)); +} + +int mlx5_cmd_fs_set_fte(struct mlx5_core_dev *dev, + u16 vport, + enum fs_fte_status *fte_status, + u32 *match_val, + enum fs_ft_type type, unsigned int table_id, + unsigned int index, unsigned int group_id, + unsigned int flow_tag, + unsigned short action, int dest_size, + struct list_head *dests) /* mlx5_flow_desination */ +{ + u32 out[MLX5_ST_SZ_DW(set_fte_out)]; + u32 *in; + unsigned int inlen; + struct mlx5_flow_rule *dst; + void *in_flow_context; + void *in_match_value; + void *in_dests; + int err; + int opmod = 0; + int modify_mask = 0; + int atomic_mod_cap; + + if (action != MLX5_FLOW_CONTEXT_ACTION_FWD_DEST) + dest_size = 0; + + inlen = MLX5_ST_SZ_BYTES(set_fte_in) + + dest_size * MLX5_ST_SZ_BYTES(dest_format_struct); + + if (!dev) + return -EINVAL; + + if (*fte_status & FS_FTE_STATUS_EXISTING) { + atomic_mod_cap = MLX5_CAP_FLOWTABLE(dev, + flow_table_properties_nic_receive. + flow_modify_en); + if (!atomic_mod_cap) + return -ENOTSUPP; + opmod = 1; + modify_mask = 1 << + MLX5_SET_FTE_MODIFY_ENABLE_MASK_DESTINATION_LIST; + } + + in = mlx5_vzalloc(inlen); + if (!in) { + mlx5_core_warn(dev, "failed to allocate inbox\n"); + return -ENOMEM; + } + + MLX5_SET(set_fte_in, in, opcode, MLX5_CMD_OP_SET_FLOW_TABLE_ENTRY); + MLX5_SET(set_fte_in, in, op_mod, opmod); + MLX5_SET(set_fte_in, in, modify_enable_mask, modify_mask); + MLX5_SET(set_fte_in, in, table_type, type); + MLX5_SET(set_fte_in, in, table_id, table_id); + MLX5_SET(set_fte_in, in, flow_index, index); + if (vport) { + MLX5_SET(set_fte_in, in, vport_number, vport); + MLX5_SET(set_fte_in, in, other_vport, 1); + } + + in_flow_context = MLX5_ADDR_OF(set_fte_in, in, flow_context); + MLX5_SET(flow_context, in_flow_context, group_id, group_id); + MLX5_SET(flow_context, in_flow_context, flow_tag, flow_tag); + MLX5_SET(flow_context, in_flow_context, action, action); + MLX5_SET(flow_context, in_flow_context, destination_list_size, + dest_size); + in_match_value = MLX5_ADDR_OF(flow_context, in_flow_context, + match_value); + memcpy(in_match_value, match_val, MLX5_ST_SZ_BYTES(fte_match_param)); + if (dest_size) { + in_dests = MLX5_ADDR_OF(flow_context, in_flow_context, destination); + list_for_each_entry(dst, dests, base.list) { + unsigned int id; + + MLX5_SET(dest_format_struct, in_dests, destination_type, + dst->dest_attr.type); + if (dst->dest_attr.type == + MLX5_FLOW_CONTEXT_DEST_TYPE_FLOW_TABLE) + id = dst->dest_attr.ft->id; + else + id = dst->dest_attr.tir_num; + MLX5_SET(dest_format_struct, in_dests, destination_id, id); + in_dests += MLX5_ST_SZ_BYTES(dest_format_struct); + } + } + memset(out, 0, sizeof(out)); + err = mlx5_cmd_exec_check_status(dev, in, inlen, out, + sizeof(out)); + if (!err) + *fte_status |= FS_FTE_STATUS_EXISTING; + + kvfree(in); + + return err; +} + +int mlx5_cmd_fs_delete_fte(struct mlx5_core_dev *dev, + u16 vport, + enum fs_fte_status *fte_status, + enum fs_ft_type type, unsigned int table_id, + unsigned int index) +{ + u32 in[MLX5_ST_SZ_DW(delete_fte_in)]; + u32 out[MLX5_ST_SZ_DW(delete_fte_out)]; + int err; + + if (!(*fte_status & FS_FTE_STATUS_EXISTING)) + return 0; + + if (!dev) + return -EINVAL; + memset(in, 0, sizeof(in)); + memset(out, 0, sizeof(out)); + + MLX5_SET(delete_fte_in, in, opcode, MLX5_CMD_OP_DELETE_FLOW_TABLE_ENTRY); + MLX5_SET(delete_fte_in, in, table_type, type); + MLX5_SET(delete_fte_in, in, table_id, table_id); + MLX5_SET(delete_fte_in, in, flow_index, index); + if (vport) { + MLX5_SET(delete_fte_in, in, vport_number, vport); + MLX5_SET(delete_fte_in, in, other_vport, 1); + } + + err = mlx5_cmd_exec_check_status(dev, in, sizeof(in), out, sizeof(out)); + if (!err) + *fte_status = 0; + + return err; +} Added: head/sys/dev/mlx5/mlx5_core/mlx5_fs_tree.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/dev/mlx5/mlx5_core/mlx5_fs_tree.c Fri Nov 10 09:49:08 2017 (r325638) @@ -0,0 +1,2721 @@ +/*- + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS `AS IS' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include +#include +#include "mlx5_core.h" +#include "fs_core.h" +#include +#include + +#define INIT_TREE_NODE_ARRAY_SIZE(...) (sizeof((struct init_tree_node[]){__VA_ARGS__}) /\ + sizeof(struct init_tree_node)) + +#define ADD_PRIO(name_val, flags_val, min_level_val, max_ft_val, caps_val, \ + ...) {.type = FS_TYPE_PRIO,\ + .name = name_val,\ + .min_ft_level = min_level_val,\ + .flags = flags_val,\ + .max_ft = max_ft_val,\ + .caps = caps_val,\ + .children = (struct init_tree_node[]) {__VA_ARGS__},\ + .ar_size = INIT_TREE_NODE_ARRAY_SIZE(__VA_ARGS__) \ +} + +#define ADD_FT_PRIO(name_val, flags_val, max_ft_val, ...)\ + ADD_PRIO(name_val, flags_val, 0, max_ft_val, {},\ + __VA_ARGS__)\ + +#define ADD_NS(name_val, ...) {.type = FS_TYPE_NAMESPACE,\ + .name = name_val,\ + .children = (struct init_tree_node[]) {__VA_ARGS__},\ + .ar_size = INIT_TREE_NODE_ARRAY_SIZE(__VA_ARGS__) \ +} + +#define INIT_CAPS_ARRAY_SIZE(...) (sizeof((long[]){__VA_ARGS__}) /\ + sizeof(long)) + +#define FS_CAP(cap) (__mlx5_bit_off(flow_table_nic_cap, cap)) + +#define FS_REQUIRED_CAPS(...) {.arr_sz = INIT_CAPS_ARRAY_SIZE(__VA_ARGS__), \ + .caps = (long[]) {__VA_ARGS__}} + +#define BYPASS_MAX_FT 5 +#define BYPASS_PRIO_MAX_FT 1 +#define KERNEL_MAX_FT 3 +#define LEFTOVER_MAX_FT 1 +#define KENREL_MIN_LEVEL 3 +#define LEFTOVER_MIN_LEVEL KENREL_MIN_LEVEL + 1 +#define BYPASS_MIN_LEVEL MLX5_NUM_BYPASS_FTS + LEFTOVER_MIN_LEVEL +struct node_caps { + size_t arr_sz; + long *caps; +}; + +struct init_tree_node { + enum fs_type type; + const char *name; + struct init_tree_node *children; + int ar_size; + struct node_caps caps; + u8 flags; + int min_ft_level; + int prio; + int max_ft; +} root_fs = { + .type = FS_TYPE_NAMESPACE, + .name = "root", + .ar_size = 3, + .children = (struct init_tree_node[]) { + ADD_PRIO("by_pass_prio", 0, BYPASS_MIN_LEVEL, 0, + FS_REQUIRED_CAPS(FS_CAP(flow_table_properties_nic_receive.flow_modify_en), + FS_CAP(flow_table_properties_nic_receive.modify_root)), + ADD_NS("by_pass_ns", + ADD_FT_PRIO("prio0", 0, + BYPASS_PRIO_MAX_FT), + ADD_FT_PRIO("prio1", 0, + BYPASS_PRIO_MAX_FT), + ADD_FT_PRIO("prio2", 0, + BYPASS_PRIO_MAX_FT), + ADD_FT_PRIO("prio3", 0, + BYPASS_PRIO_MAX_FT), + ADD_FT_PRIO("prio4", 0, + BYPASS_PRIO_MAX_FT), + ADD_FT_PRIO("prio5", 0, *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***