From owner-freebsd-net@freebsd.org Mon Mar 23 04:50:23 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D140B27848E for ; Mon, 23 Mar 2020 04:50:23 +0000 (UTC) (envelope-from pkelsey@gmail.com) Received: from mail-io1-f54.google.com (mail-io1-f54.google.com [209.85.166.54]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48m24Z3DVvz4Lr5 for ; Mon, 23 Mar 2020 04:50:22 +0000 (UTC) (envelope-from pkelsey@gmail.com) Received: by mail-io1-f54.google.com with SMTP id h131so12857982iof.1 for ; Sun, 22 Mar 2020 21:50:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=iixK0l1EVdyhjj/FQyb0YkG8SnpaaTCcrU6sy1F7CdM=; b=FK9kSF+arncjvBb1hnvCxxbQOf2ete310JMutjNWLu8PIho0uPHHDjo8qJo0HqTymo l9XjbscUZBEKLndhsOL5oj5HyM85V+9T88Ub9GIARzcYfxkSg0/DZkoQeCvK86xaU4jv CNxIK7zDQo+ujnJKKPDDRsIt+DSBL80r2D1imgZFPIa9NUrusoH99ALidTTS59rzdXts gK2+cnU3PCdDM4oX3omaXktN6VZeglFXt78ixjuYr5X0o9+izjh3G2bXJ0McXqJPy6AJ EtMzMKp6HBpacJJquYSapdegcVLMYhJjawd5545ba/vvABQ4D16InVqUkbqVzc2jWzbV ilHw== X-Gm-Message-State: ANhLgQ1UYf+en4krNUPalObncNLfWhGhg6UYwMUgM6yPC8xn25XJA6eT y0HXe0dImewJBLjnkwKCU2o8sVWmehkBcRg+CwM= X-Google-Smtp-Source: ADFU+vumZadE6Q/28S1fG61pYcI+e92PlcQfrF2UDW5QtFd2WAHGGtVfJAjrr1Q26OE/e+/GQmzSDEB3TlKmc7sZkXI= X-Received: by 2002:a02:cc6f:: with SMTP id j15mr19145040jaq.80.1584939020855; Sun, 22 Mar 2020 21:50:20 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Patrick Kelsey Date: Mon, 23 Mar 2020 00:50:08 -0400 Message-ID: Subject: Re: Can net.iflib.min_tx_latency=0 introduce an unbounded amount of latency? To: Ryan Stone Cc: freebsd-net X-Rspamd-Queue-Id: 48m24Z3DVvz4Lr5 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of pkelsey@gmail.com designates 209.85.166.54 as permitted sender) smtp.mailfrom=pkelsey@gmail.com X-Spamd-Result: default: False [-2.63 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-net@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[54.166.85.209.list.dnswl.org : 127.0.5.0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; IP_SCORE(-1.63)[ip: (-4.90), ipnet: 209.85.128.0/17(-2.60), asn: 15169(-0.60), country: US(-0.05)]; FORGED_SENDER(0.30)[pkelsey@freebsd.org,pkelsey@gmail.com]; FREEMAIL_TO(0.00)[gmail.com]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[pkelsey@freebsd.org,pkelsey@gmail.com]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2020 04:50:23 -0000 On Sun, Mar 22, 2020 at 2:31 PM Ryan Stone wrote: > I've been tracking down a bug at work that appears to be due to > excessive latency introduced on the TX side of a TCP connection. In > looking through the iflib code, I noticed the tunable > net.iflib.min_tx_latency. My reading of the iflib code is that if a > packet is enqueued to a tx ring but not posted (when > min_tx_latency=0), then iflib can introduce an unbounded amount of > latency if the ring is idle. Is my reading of the code correct? > > Let's see...in the case described above, the next iflib_timer() invocation (max 500ms later) should find that txq->ift_db_pending is non-zero, which will cause it to enqueue the TX task, which will invoke _task_fn_tx(), which will then enqueue the special marker (void **)&txq, and one way or another, iflib_txq_drain() will be called. Looking at iflib_txq_drain() it appears that the first call to iflib_txd_db_check() should post the pending packets either because a non-zero number of slots were just reclaimed, or the check for txq->ift_db_pending >= TXQ_MAX_DB_DEFERRED(...) is satisfied, unless there are too many posted packets in the hardware queue that aren't being drained by the hardware (this would result in the in_use parameter remaining elevated such that a small number of pending packets would not exceed the threshold, and also no slot reclaim occurring). -Patrick