From owner-freebsd-ipfw@freebsd.org Sun Apr 5 21:01:13 2020 Return-Path: Delivered-To: freebsd-ipfw@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6BFF726B8AC for ; Sun, 5 Apr 2020 21:01:13 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 48wR0m09xLz3RFs for ; Sun, 5 Apr 2020 21:01:12 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.nyi.freebsd.org (Postfix) id DF20826B8A1; Sun, 5 Apr 2020 21:01:11 +0000 (UTC) Delivered-To: ipfw@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CE95226B89C for ; Sun, 5 Apr 2020 21:01:11 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48wR0h3y6Gz3RF1 for ; Sun, 5 Apr 2020 21:01:08 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D41491BF95 for ; Sun, 5 Apr 2020 21:00:22 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 035L0M4q058684 for ; Sun, 5 Apr 2020 21:00:22 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 035L0MKR058683 for ipfw@FreeBSD.org; Sun, 5 Apr 2020 21:00:22 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202004052100.035L0MKR058683@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: ipfw@FreeBSD.org Subject: Problem reports for ipfw@FreeBSD.org that need special attention Date: Sun, 5 Apr 2020 21:00:22 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Apr 2020 21:01:13 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 215875 | [ipfw] ipfw lookup tables do not support mbuf_tag New | 232764 | [ipfw] share/examples/ipfw/change_rules.sh: Suppo 2 problems total for which you should take action. From owner-freebsd-ipfw@freebsd.org Thu Apr 9 09:17:50 2020 Return-Path: Delivered-To: freebsd-ipfw@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3FC2A2B4242; Thu, 9 Apr 2020 09:17:50 +0000 (UTC) (envelope-from ozkan.kirik@gmail.com) Received: from mail-vs1-xe34.google.com (mail-vs1-xe34.google.com [IPv6:2607:f8b0:4864:20::e34]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48ybCK1zhMz4MKn; Thu, 9 Apr 2020 09:17:49 +0000 (UTC) (envelope-from ozkan.kirik@gmail.com) Received: by mail-vs1-xe34.google.com with SMTP id s25so4546144vsp.11; Thu, 09 Apr 2020 02:17:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=FgjZI4jL1xLNR5LM153d7T1DCZM6A4aTeSpQ3y6uNwo=; b=L6NExeWbyYj+Xv91sPaExtoi4RWR+en1Xj+te3KDo+Yab++q4Z6T1OF8cjKjU8o2Zb 9S5t9yaBr3I45Z6zqcm4o3t3Vw+dalV2KQx2vqw9Kx8C/R9YJS5s9sj+jGj9C0oX4oPk vWJpsi+j/Fu4uXuli+UH9nW5Zk2azpGRSISPT5kvuh0s7j7MQ8gggqEcBhWaIvNUPiw9 lEmVmoy/KbDajz0bSeUdNV1QuW0BrWGXJhYx0wsknVlkpSvpDC80meuhhJV/RvLAY+Th UACNX02lnk+ODID1Jizn8rIYZEqTycHrvTN3nFSw0qL2tVo3523ygO/9zWP8Hm/Kke7/ B6XQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=FgjZI4jL1xLNR5LM153d7T1DCZM6A4aTeSpQ3y6uNwo=; b=gmUCEiSjcwGzEqUnx36EEYAff9TN/ZFGyZ/xB0+zK1dM7Oj870oSTOinZ6WDp0mSA2 WxbEtMzKOUCoFtIJ14vZJHFRyYHDq1X8fZIE9MU9+rADPrX3en+AXI17Ug2k9ppsVwD3 lc3vIxKrtCg/uQCA5FGUorBYsN08Uca857Ys+NEVbl3eb4nLKo0CZmb24//rCfgi6QpB 7cWrxe0ImzjpPG7YocZ5TcxrvEXSs8tzGNQdAPvrLPpLG3ncyfB1R3qiHL0Od1FIolzt ywhwkNBTmjX51fJYrl6EGZyNax6UkpYQbrLXFM/Wscttlf+oscD1r5a7gKJxN5LeMxfR wiYA== X-Gm-Message-State: AGi0PuaugVO0YhAm0d7+Ug6wQZ43aX14MPj54wNX0VjkRvSKYsp/Tl5q nqkPRylLrjeY650u6FjgqEDE3gHnlroa3RAsEG2BNE1f X-Google-Smtp-Source: APiQypJWKgINDnWNqi4Qk7Nvs3nbzrdzaTLLtB989bxtzym2H3uuyQRwdjUYJfy+PvkWq/WYts6mqaRbm8IF9bcozOY= X-Received: by 2002:a67:80d1:: with SMTP id b200mr7285256vsd.76.1586423867796; Thu, 09 Apr 2020 02:17:47 -0700 (PDT) MIME-Version: 1.0 From: =?UTF-8?B?w5Z6a2FuIEtJUklL?= Date: Thu, 9 Apr 2020 12:17:35 +0300 Message-ID: Subject: dummynet performance To: freebsd-net@freebsd.org, freebsd-ipfw Cc: Luigi Rizzo , "Andrey V. Elsukov" X-Rspamd-Queue-Id: 48ybCK1zhMz4MKn X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=L6NExeWb; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of ozkankirik@gmail.com designates 2607:f8b0:4864:20::e34 as permitted sender) smtp.mailfrom=ozkankirik@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; IP_SCORE(0.00)[ip: (-9.63), ipnet: 2607:f8b0::/32(-0.33), asn: 15169(-0.43), country: US(-0.05)]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; URI_COUNT_ODD(1.00)[3]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[4.3.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Apr 2020 09:17:50 -0000 Hello, I wonder if it is possible to update dummynet code multi-threading capable practically? My idea is below: - A new sysctl tunable will be defined as "net.inet.ip.dummynet.thread_count" (default 1) - To distribute tasks along threads, each sched instance can be assigned to different thread. - By default, all schedulers assigned to thread 0 if no thread_id is set. With a new option named "thr_id" affinity can be set. as below: ipfw sched 1 config type QFQ (by defaut thr_id = 0) ipfw sched 2 config type QFQ thr_id 1 ipfw sched 3 config type QFQ thr_id 2 ipfw sched 4 config type FIFO thr_id 3 ipfw sched 5 config type FIFO thr_id 3 ( sched 4 & 5 assigned to same thread 3 ) I think this approach is more easy then completely rewrite or other solutions. What is the right way to do this? Where to start? I read the explanations in /usr/src/sys/netpfil/ipfw/dummynet.txt On this line below, multiple threads (count can be get from sysctl "net.inet.ip.dummynet. thread_count" tunable) https://github.com/freebsd/freebsd/blob/master/sys/netpfil/ipfw/ip_dummynet.c#L2556 Thanks From owner-freebsd-ipfw@freebsd.org Thu Apr 9 09:44:00 2020 Return-Path: Delivered-To: freebsd-ipfw@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C6DE82B4F5C; Thu, 9 Apr 2020 09:44:00 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-vs1-f67.google.com (mail-vs1-f67.google.com [209.85.217.67]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48ybnW5Vhhz4P08; Thu, 9 Apr 2020 09:43:59 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: by mail-vs1-f67.google.com with SMTP id u11so4423888vsu.10; Thu, 09 Apr 2020 02:43:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=FHvxlFpLbd8BKskT+sasAAw6iO0ZGNWtHlC+vKInvWc=; b=D5IBnk2UZ08QqPk93+jVRPOfyy/nwHrEt83KCHPlZnWJ/AYwROEIjEbJ73arX74I57 Miq1Ynw8acKP1zLncprqzZ9pBiOykDrRYEsoepDCc36FTS9lf73h+zYE8sosfWW8q1Ff NjpfHPvRGsf5G1TbkFs+HZBWAgfrc3UzHqornZNov45VRdBBH3PBLo4Bw1rPTqCyDSJl Y7ttYjrll4ZWBPWgqtukRA1u5vrDyquyeZPmSPP2NLX9IlLZo5dXZzB5jl4VFBMyZt4S DoHpjjOJYLfKssiYyiutgfve2xwmYfTKrGOotqY0kBZrv4iD/qanDLD2fB294fPmRgcO zdyA== X-Gm-Message-State: AGi0Puan8h/XXURflOoewLu+nmFSQEsRH49yAFEZY5Az+TR5iTwZnJ1H 2OxMF0zT/I1upRmAxwJ5oUJPKTCd7ZMwfWU1kRA= X-Google-Smtp-Source: APiQypK1U5PJoAFO2uZBqN212h2ebOqfKteej8/etxmH0z45fihjtk9zxVAeaFKD6eqXTBZcQ6McgYlYWWDnLUxRkLQ= X-Received: by 2002:a67:e308:: with SMTP id j8mr3978629vsf.74.1586425438195; Thu, 09 Apr 2020 02:43:58 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Luigi Rizzo Date: Thu, 9 Apr 2020 02:43:47 -0700 Message-ID: Subject: Re: dummynet performance To: =?UTF-8?B?w5Z6a2FuIEtJUklL?= Cc: "freebsd-net@freebsd.org" , freebsd-ipfw , "Andrey V. Elsukov" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 48ybnW5Vhhz4P08 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of rizzounipi@gmail.com designates 209.85.217.67 as permitted sender) smtp.mailfrom=rizzounipi@gmail.com X-Spamd-Result: default: False [-0.50 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; FORGED_SENDER(0.30)[rizzo@iet.unipi.it,rizzounipi@gmail.com]; FREEMAIL_TO(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FROM_NEQ_ENVFROM(0.00)[rizzo@iet.unipi.it,rizzounipi@gmail.com]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.84)[-0.844,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-0.99)[-0.986,0]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[unipi.it]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[67.217.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-0.17)[ip: (0.02), ipnet: 209.85.128.0/17(-0.40), asn: 15169(-0.43), country: US(-0.05)]; RWL_MAILSPIKE_POSSIBLE(0.00)[67.217.85.209.rep.mailspike.net : 127.0.0.17]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; SUSPICIOUS_RECIPS(1.50)[] X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Apr 2020 09:44:00 -0000 On Thu, Apr 9, 2020 at 2:18 AM =C3=96zkan KIRIK wro= te: > > Hello, > > I wonder if it is possible to update dummynet code multi-threading capabl= e > practically? > My idea is below: > - A new sysctl tunable will be defined as > "net.inet.ip.dummynet.thread_count" (default 1) > - To distribute tasks along threads, each sched instance can be assigned = to > different thread. > ... What are you trying to achieve ? In general, the scheduler needs to know the state of all flows to perform correct scheduling of traffic. You could in principle split the load on multiple threads under specific, very restrictive conditions (each thread runs a different scheduler, they cannot share the excess bandwidth, and the total bw does not exceed the link's capacity, each scheduler has a different queue on the NIC). cheers luigi From owner-freebsd-ipfw@freebsd.org Thu Apr 9 10:00:36 2020 Return-Path: Delivered-To: freebsd-ipfw@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7096D2B544A; Thu, 9 Apr 2020 10:00:36 +0000 (UTC) (envelope-from ozkan.kirik@gmail.com) Received: from mail-ua1-x92a.google.com (mail-ua1-x92a.google.com [IPv6:2607:f8b0:4864:20::92a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48yc8g3X10z4Pjv; Thu, 9 Apr 2020 10:00:35 +0000 (UTC) (envelope-from ozkan.kirik@gmail.com) Received: by mail-ua1-x92a.google.com with SMTP id a6so3741773uao.2; Thu, 09 Apr 2020 03:00:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=PCNbLgVyXxAXTIz1/iOVQxWmSnoXMJB9VH6o4RqZxkU=; b=GAWFEClTm6N8qQNCRSo90CdPI10JuKRNffZJhClveOCXIjuTzSaCVtJLd0KRkYbPH/ LMcLoseCLqHtpugODuA8g9gLcQ82/S0hETbdtWmx2tBy4v8xTlGeZYEbRfeM/F6k48wL M1h1hL6mWXVu2kkahNizGwKJ3+6G+px7GtGyDci5XNVciCdSz5T5dJ1fQetFX1QI/aVr 3wwvvtqBjxi6tDTaytadYEoEZVSBH7Yeq8n4pGezFAZ+MN8pg4tdbqkunfUc151Kyu3e c5ldMguu60xYmZYkQzCheTmjGfcYZGW5kwaGv32NnjZ+ZjYBse0i+J5zQV9gZNzQDVmr Vlng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=PCNbLgVyXxAXTIz1/iOVQxWmSnoXMJB9VH6o4RqZxkU=; b=G2MzHhCnN1G10Zj1SbzG+4cKKeZ/z69V6v8R7G+nESX3hUAU2TSyU/k46UR82k9tkB 41GDTlhFQOp8+D/0F8iMnaswYekdUKklGH8uQnZI64CW4JXfG5HUSpe2mNMtXEAmmHLz VOVX9h85cwI0QdbUVep/HFn4/S655HcMG4VI98T7zFIgJrVCZCNAJM0Az5kQ+YDeBrGI weeHKubmOr7awLkkAXT+a0M1GNQLQV9K6m5lmwo8sZ/rHt14EpHeWIbUDKsF3eMgd4v9 tsXmdw6/l1tuBhBeHjhIE3HFMhmgEgJNqQY3Dpov/xAJ09nquJPxub0XMCEKDbs5JsE8 pw0w== X-Gm-Message-State: AGi0PuY9lb/+N7tn+hcayZ4VZ4uuqoswbM82CckKB1E1gshHrAnXvZZf CBYAjH12BRAxITw9tq60QSHyy4NFgLi48U+iJwA= X-Google-Smtp-Source: APiQypLfFBo2rGiF8q5QCEMNY8tjGo89HrM+kP9iSItRigJiX9xbImMGvE/QdMDRBZXEqb/WPXf8ebDtRCwytZAuU6M= X-Received: by 2002:ab0:5550:: with SMTP id u16mr3626438uaa.62.1586426434505; Thu, 09 Apr 2020 03:00:34 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: =?UTF-8?B?w5Z6a2FuIEtJUklL?= Date: Thu, 9 Apr 2020 13:00:22 +0300 Message-ID: Subject: Re: dummynet performance To: Luigi Rizzo Cc: "freebsd-net@freebsd.org" , freebsd-ipfw , "Andrey V. Elsukov" X-Rspamd-Queue-Id: 48yc8g3X10z4Pjv X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=GAWFEClT; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of ozkankirik@gmail.com designates 2607:f8b0:4864:20::92a as permitted sender) smtp.mailfrom=ozkankirik@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; IP_SCORE(0.00)[ip: (-9.73), ipnet: 2607:f8b0::/32(-0.33), asn: 15169(-0.43), country: US(-0.05)]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[a.2.9.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Apr 2020 10:00:36 -0000 In my production environment there are many schedulers and pipes & queues belongs to this scheds. dummynet uses %90-100 percent of single cpu. (ie. 20 scheds, 30 pipes, 35 queues total. Each pipe has max 30-100Mbps) In my idea, by distributing schedulers to different threads, all pipes that belongs to same scheduler also should run on same thread. By this way, dummynet can manage much higher total bandwidths. Thanks for greate explanation. In another way, for each scheduler we can create a new taskqueue_thread, Is it possible if dummnet_io() function will stay as is (in main thread), and it should call the right schedulers enqueue function that runs different thread? cheers Ozkan On Thu, Apr 9, 2020 at 12:43 PM Luigi Rizzo wrote: > On Thu, Apr 9, 2020 at 2:18 AM =C3=96zkan KIRIK w= rote: > > > > Hello, > > > > I wonder if it is possible to update dummynet code multi-threading > capable > > practically? > > My idea is below: > > - A new sysctl tunable will be defined as > > "net.inet.ip.dummynet.thread_count" (default 1) > > - To distribute tasks along threads, each sched instance can be assigne= d > to > > different thread. > > ... > > What are you trying to achieve ? In general, the scheduler needs to > know the state of all flows to perform correct scheduling of traffic. > You could in principle split the load on multiple threads under specific, > very restrictive conditions (each thread runs a different scheduler, they > cannot share the excess bandwidth, and the total bw does not exceed the > link's capacity, each scheduler has a different queue on the NIC). > > cheers > luigi > From owner-freebsd-ipfw@freebsd.org Thu Apr 9 15:53:39 2020 Return-Path: Delivered-To: freebsd-ipfw@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 44F762BDE6C; Thu, 9 Apr 2020 15:53:39 +0000 (UTC) (envelope-from lutz@donnerhacke.de) Received: from annwfn.iks-jena.de (annwfn.iks-jena.de [IPv6:2001:4bd8::19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48ym0206Ypz3HQk; Thu, 9 Apr 2020 15:53:37 +0000 (UTC) (envelope-from lutz@donnerhacke.de) X-SMTP-Sender: IPv6:2001:4bd8:59:1:172:27:107:102 Received: from lyoness (lyoness.intern.iks-service.de [IPv6:2001:4bd8:59:1:172:27:107:102]) by annwfn.iks-jena.de (8.15.2/8.15.2) with ESMTP id 039FrVE7014556; Thu, 9 Apr 2020 17:53:32 +0200 From: "Lutz Donnerhacke" To: "=?UTF-8?Q?'=C3=96zkan_KIRIK'?=" , "'Luigi Rizzo'" Cc: , "'Andrey V. Elsukov'" , "'freebsd-ipfw'" References: In-Reply-To: Subject: AW: dummynet performance Date: Thu, 9 Apr 2020 17:53:31 +0200 Message-ID: <001501d60e87$05345d60$0f9d1820$@donnerhacke.de> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 16.0 Content-Language: de Thread-Index: AQHgQLB+JflcKPVCqeLysoA/Iqzj0AIoT9GQAbiA3uOoPYW0gA== X-Rspamd-Queue-Id: 48ym0206Ypz3HQk X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of lutz@donnerhacke.de has no SPF policy when checking 2001:4bd8::19) smtp.mailfrom=lutz@donnerhacke.de X-Spamd-Result: default: False [4.02 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[donnerhacke.de]; AUTH_NA(1.00)[]; RCPT_COUNT_FIVE(0.00)[5]; NEURAL_SPAM_MEDIUM(0.91)[0.908,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; IP_SCORE(-0.00)[country: DE(-0.02)]; NEURAL_SPAM_LONG(0.71)[0.713,0]; RCVD_IN_DNSWL_NONE(0.00)[9.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.8.d.b.4.1.0.0.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15725, ipnet:2001:4bd8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; SUSPICIOUS_RECIPS(1.50)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Apr 2020 15:53:39 -0000 > In my production environment there are many schedulers and pipes & > queues belongs to this scheds. > dummynet uses %90-100 percent of single cpu. (ie. 20 scheds, 30 pipes, > 35 queues total. Each pipe has max 30-100Mbps) If the perfomance issue is urgent, you may give netgraph (ng_car, = ng_pipe) a try. From owner-freebsd-ipfw@freebsd.org Thu Apr 9 17:29:14 2020 Return-Path: Delivered-To: freebsd-ipfw@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 70C97278270; Thu, 9 Apr 2020 17:29:14 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-vs1-f65.google.com (mail-vs1-f65.google.com [209.85.217.65]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48yp6K3wqCz3QcV; Thu, 9 Apr 2020 17:29:13 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: by mail-vs1-f65.google.com with SMTP id u9so7432535vsp.6; Thu, 09 Apr 2020 10:29:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=+FiWlM0JvZZqa9MBjftqXtQ47GZYrye0Dadae5IeD9A=; b=VrcrF67ujkrKA4xJYBYPOUWkdLPdq+Q0vics3EjvNZ48t0Mzom9vMgUT6GF+n54Xsu 8W2bvYqEOZDqzaFvCg+sadFmhVCmHM7zN0jb78N5DpqgimQdnUxx9aC0NbzVEQP83ID7 l8wZFDlybBCCMUZjRwsjToIE6Ah0UHrLwYGtY297v389EJQIH8qdq93qOBfwznviJLEd EbFvd/vZDOClxzg72MVMI1olM+W03qUhfvEPpbgcjbtCKSyLY9Bp2bqfC5M06retFwXt 2oxM4eH6lUc/FKt32b2R0yFKmF8ipYhgaPp37FERUaL7my7lk3DOb8RZ+hWR7FF/mc4F 49KQ== X-Gm-Message-State: AGi0PuaTCJoRZ9Nxif4W1djYb8wA/QrNIC/lvWUtqKjeRy9puEwLSrod SfHBCUKAJZHAY2aRjfxclTivE3kfZScaIFk722A= X-Google-Smtp-Source: APiQypJJcaJARD6KtFR7oFpCExpZLtjvDgbCL89zQbl2u8LzumyvXWN5YuFLqePfsF+J9TZzqzZH0KExcZsP9qfSyjM= X-Received: by 2002:a67:e308:: with SMTP id j8mr922317vsf.74.1586453352440; Thu, 09 Apr 2020 10:29:12 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Luigi Rizzo Date: Thu, 9 Apr 2020 10:28:56 -0700 Message-ID: Subject: Re: dummynet performance To: =?UTF-8?B?w5Z6a2FuIEtJUklL?= Cc: "freebsd-net@freebsd.org" , freebsd-ipfw , "Andrey V. Elsukov" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 48yp6K3wqCz3QcV X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of rizzounipi@gmail.com designates 209.85.217.65 as permitted sender) smtp.mailfrom=rizzounipi@gmail.com X-Spamd-Result: default: False [-0.49 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; FORGED_SENDER(0.30)[rizzo@iet.unipi.it,rizzounipi@gmail.com]; FREEMAIL_TO(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[rizzo@iet.unipi.it,rizzounipi@gmail.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.84)[-0.839,0]; TAGGED_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-0.99)[-0.986,0]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[unipi.it]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[65.217.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-0.16)[ip: (0.05), ipnet: 209.85.128.0/17(-0.40), asn: 15169(-0.43), country: US(-0.05)]; RWL_MAILSPIKE_POSSIBLE(0.00)[65.217.85.209.rep.mailspike.net : 127.0.0.17]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; SUSPICIOUS_RECIPS(1.50)[] X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Apr 2020 17:29:14 -0000 On Thu, Apr 9, 2020 at 2:18 AM =C3=96zkan KIRIK wro= te: > > Hello, > > I wonder if it is possible to update dummynet code multi-threading capabl= e > practically? > My idea is below: > - A new sysctl tunable will be defined as > "net.inet.ip.dummynet.thread_count" (default 1) > - To distribute tasks along threads, each sched instance can be assigned = to > different thread. > - By default, all schedulers assigned to thread 0 if no thread_id is set. > With a new option named "thr_id" affinity can be set. as below: > ipfw sched 1 config type QFQ (by defaut thr_id =3D 0) > ipfw sched 2 config type QFQ thr_id 1 > ipfw sched 3 config type QFQ thr_id 2 > ipfw sched 4 config type FIFO thr_id 3 > ipfw sched 5 config type FIFO thr_id 3 ( sched 4 & 5 assigned to same > thread 3 ) > > I think this approach is more easy then completely rewrite or other > solutions. > This change may not be completely trivial given that dummynet was designed = as single threaded and protected by a single lock. If you want to proceed, you should do as follows: - some members of struct dn_parms need to be duplicated for each thread, including prev_t, evheap, curr_time, the locks, and perhaps others. Probably may be useful to create a new struct for them, with a back pointer to the parent dn_parms, and include a refcount. - add another rwlock to protect access to dn_cfg - dummynet_io() does the initial demultiplexing calling dn_ht_find(), currently protected DN_BH_WLOCK(). This could be changed to an RWLOCK and I believe this initial step could be protected by a read lock (maybe there are corner cases where you need to create a new entry so you need to drop the lock, acquire in write mode and retry...) - once the demux has returned a dn_fsk, maybe you can use a refcount to avoid the object being destroyed, release the read lock on dn_cfg.fshash, acquire an exclusive lock on the thread handling the dn_fsk, and proceed from there. - the kernel thread that process the heap should now run each on one instance of the evheap cheers luigi From owner-freebsd-ipfw@freebsd.org Fri Apr 10 06:50:07 2020 Return-Path: Delivered-To: freebsd-ipfw@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 85CA52B425E; Fri, 10 Apr 2020 06:50:07 +0000 (UTC) (envelope-from ozkan.kirik@gmail.com) Received: from mail-vs1-xe2f.google.com (mail-vs1-xe2f.google.com [IPv6:2607:f8b0:4864:20::e2f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48z7tQ61WPz4sd6; Fri, 10 Apr 2020 06:50:06 +0000 (UTC) (envelope-from ozkan.kirik@gmail.com) Received: by mail-vs1-xe2f.google.com with SMTP id z63so830859vsz.2; Thu, 09 Apr 2020 23:50:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=NTwHza1u6b+yBmZ78FAaw25N6tPR3oz5c1qG88t/lcs=; b=uAnOQeqvVjaYv2qwMTrmJmIWz8EwiWrZCV1xTDSjtXem6R4R3n7G+X1Aa/cOqFMwhJ 4WZPNfTTsYeSXMDM4dg5CpiOX3+KrcpKN2c9/R2ppntRzW9lFG8u8G5S4tUd+E48YxQk V2ilA4eKilyUP1sRE8w6m0Hw8YZbzKY0pPhxvgqplhTtFvOYH1McvNtLSSRdf7oIRCxC qqWO2PHeSUw17YekJ4wf9wpwXE+5+7Yok2YE1O1ufziTifb7+bEZEoodig10Hww6YLbh okaV2dCiKRlCqqbNx0HGvbis9vDGTB6EMCwi+SSgcs7OXTuOlHiahX6T382YW26iGLGV zPdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=NTwHza1u6b+yBmZ78FAaw25N6tPR3oz5c1qG88t/lcs=; b=UDDcKnrp/yrHTlrDnWYq2dbYCBVPgYyrXj0LF7k7kpXYhOrZCGFGoSBrEXPJyahLWS DJTHbZKwq8V2v5y2BIKHOkrqYspr+6a+jpyrzjvdxR+OSYV246amiNo/zOrcUgbuxAmA l6dZucocPOsJctUFI59fq94ZGcaOsCvHWT+XY4x6+eHuF77weR47MK14iU9X9JJVULxo LiNQHU10l4KD4wb122DVjwpDp/WdZ1sY26PIN5uPVEh2izo4xiunKPjWmFrnpyVWeui6 l6SDs8eX/bSB74j4Q1P/abjrThPkIR94KD3Y7jnGJbx9q1g8NMphqgijKr+NKsCwZm4T Swag== X-Gm-Message-State: AGi0PuYCRGow2Y+9PpiczZDFCMgHytAFcIAc75qO/YYoi0pGGN1rX53O Rl/qOQPJL3ENr7IAaodCYZliLVsO87hBo1JiAro= X-Google-Smtp-Source: APiQypLJVtGw/w3u+DsawnbE+BVr+JMm2BUGyitChGct2zLSr8Cirvrir52zHTjAPCW/WaYoZw1YwpjrFCDpXHevh58= X-Received: by 2002:a05:6102:10c9:: with SMTP id t9mr2654873vsr.41.1586501405689; Thu, 09 Apr 2020 23:50:05 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: =?UTF-8?B?w5Z6a2FuIEtJUklL?= Date: Fri, 10 Apr 2020 09:49:51 +0300 Message-ID: Subject: Re: dummynet performance To: Luigi Rizzo Cc: "freebsd-net@freebsd.org" , freebsd-ipfw , "Andrey V. Elsukov" X-Rspamd-Queue-Id: 48z7tQ61WPz4sd6 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=uAnOQeqv; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of ozkankirik@gmail.com designates 2607:f8b0:4864:20::e2f as permitted sender) smtp.mailfrom=ozkankirik@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; IP_SCORE(0.00)[ip: (-9.54), ipnet: 2607:f8b0::/32(-0.33), asn: 15169(-0.43), country: US(-0.05)]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[f.2.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2020 06:50:07 -0000 Thanks! On Thu, Apr 9, 2020 at 8:29 PM Luigi Rizzo wrote: > On Thu, Apr 9, 2020 at 2:18 AM =C3=96zkan KIRIK w= rote: > > > > Hello, > > > > I wonder if it is possible to update dummynet code multi-threading > capable > > practically? > > My idea is below: > > - A new sysctl tunable will be defined as > > "net.inet.ip.dummynet.thread_count" (default 1) > > - To distribute tasks along threads, each sched instance can be assigne= d > to > > different thread. > > - By default, all schedulers assigned to thread 0 if no thread_id is se= t. > > With a new option named "thr_id" affinity can be set. as below: > > ipfw sched 1 config type QFQ (by defaut thr_id =3D 0) > > ipfw sched 2 config type QFQ thr_id 1 > > ipfw sched 3 config type QFQ thr_id 2 > > ipfw sched 4 config type FIFO thr_id 3 > > ipfw sched 5 config type FIFO thr_id 3 ( sched 4 & 5 assigned to sam= e > > thread 3 ) > > > > I think this approach is more easy then completely rewrite or other > > solutions. > > > > This change may not be completely trivial given that dummynet was designe= d > as > single threaded and protected by a single lock. > > If you want to proceed, you should do as follows: > - some members of struct dn_parms need to be duplicated for each thread, > including prev_t, evheap, curr_time, the locks, and perhaps others. > Probably may be useful to create a new struct for them, with a back > pointer to the parent dn_parms, and include a refcount. > > - add another rwlock to protect access to dn_cfg > > - dummynet_io() does the initial demultiplexing calling dn_ht_find(), > currently protected DN_BH_WLOCK(). This could be changed to an RWLOCK > and I believe this initial step could be protected by a read lock > (maybe there are corner cases where you need to create a new entry > so you need to drop the lock, acquire in write mode and retry...) > > - once the demux has returned a dn_fsk, maybe you can use a refcount > to avoid the object being destroyed, release the read lock on > dn_cfg.fshash, acquire an exclusive lock on the thread handling the > dn_fsk, and proceed from there. > > - the kernel thread that process the heap should now run each on > one instance of the evheap > > cheers > luigi >