From owner-freebsd-pf@freebsd.org Mon Apr 30 20:07:41 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B4B25FB5755 for ; Mon, 30 Apr 2018 20:07:41 +0000 (UTC) (envelope-from jason@singular.net) Received: from mail-ot0-x234.google.com (mail-ot0-x234.google.com [IPv6:2607:f8b0:4003:c0f::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DB6478CCC for ; Mon, 30 Apr 2018 20:07:41 +0000 (UTC) (envelope-from jason@singular.net) Received: by mail-ot0-x234.google.com with SMTP id g7-v6so10861230otj.11 for ; Mon, 30 Apr 2018 13:07:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=singular-net.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=4H5W/l1EqpWIOUBlHDzIsEEvDPjrTkFPGKR/q49XEIk=; b=yOafdEnpP21ZS0W3EdEdOjT4qA2qhrbj7AX25HVB7CSt/hsRtLwxu2Jih+hGIhH3Zn lQUavbFyUNwJlvF2Z1wDC8+BAi8elwYe8Bs3DPD5qbd+T2NzcWuIlMKXNTQeEbkMlb82 X+bfh2z1U4JD1UTB8pG8dKVekXcOq5Ny3gtldILBZi29xO6gmv7/yvOCqBHTEEGfWAMe QCQXHbAOyClxBc9zS9YsUcpPltPowBB+C0v/mh8aStO8G5PhWQt31lOoTI5IW/hkeae4 1Hk3zVV6V19MSf29M2T/hB5XIgF7RIge/mztNcNDn/MEjP0N/Lprdr9SmOLKWOPEkK0P ga7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=4H5W/l1EqpWIOUBlHDzIsEEvDPjrTkFPGKR/q49XEIk=; b=noMHrt+8Xwe04FpuR1OIZ20uIkWESvDOd93md9StS5NiLp4/RwB2H8pyuHhA9TGnEo 901VPW6cMDvIe6T09jZmYNVKkDEeXhacoPPp9uT74m6irpEM2VtjcAa3PzW9l9RHWWb1 uHBAP+vgDu0bUzJXhOKicHK+I/jUJ00emHJOt2Q2AjPuomVeiPOB0MiY6HTyfqKdeg5f J1k0SlQOcSx6loIrohe1ANMIV+A9i9OP7toAL2FwvaQImATdQhrjB1EZIG9CE6FQENZc ECJsCPSeg/FWgnlhCtU1a+5+854e996/REP7k+woMOxv7/XYFHUczCEt4Fi4n+rCcvQz J2fw== X-Gm-Message-State: ALQs6tCxdog9T3cJjcnRWSLuRwmY8Xw9UCebMcVQS3SxCz3sNjrwAZU5 IiyyzI9IZSVHNtNRQw9hSKwsWYW2W26OwvL76xtkpOj/b0g= X-Google-Smtp-Source: AB8JxZoSsn4nMs2gBjeeqm9vjE0xUodh7FomnZ4EFBvxdQ0uphRjPcgPdP2/odT9MfKKQK8M0oaNUDUTCkH+dG3/NP4= X-Received: by 2002:a9d:2456:: with SMTP id p80-v6mr8736110ota.231.1525118860372; Mon, 30 Apr 2018 13:07:40 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a9d:1dd2:0:0:0:0:0 with HTTP; Mon, 30 Apr 2018 13:07:39 -0700 (PDT) From: Jason Matthews Date: Mon, 30 Apr 2018 13:07:39 -0700 Message-ID: Subject: high cpu utilization due to interrupts To: freebsd-pf@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Apr 2018 20:07:42 -0000 I have a pair systems running relayd in DSR mode. The load balancer works well, to a point, but suffers from high interrupts and cpu utilization on eight cores. The monitoring systems show increased latency until it falls over dead at round 330k requests/sec where the CARP seems to stop responding and the partner jumps in only to wash-rinse-repeat the cycle. top -aSCHIP shows 85-90% CPU utilization due to interrupts when running near peak. What can be done to lower the interrupt load? Or spread it across more CPUs? Is the design such that one core is dedicated to a queue on the card? Or can the load be spread across more CPUs? The systems are Intel S5520UR boards with a X540-AT2 card on X5650 CPUs. Any suggestions are appreciated. thanks, j. ---------------- = Internet = ---------------- | ix0 --------------- = relayd = ---------------- | ix1 | -------------------- - web servers - ---------------------