From owner-svn-src-all@freebsd.org Fri Jan 5 15:42:03 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D66CCEB01A4 for ; Fri, 5 Jan 2018 15:42:03 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: from mail-wm0-x230.google.com (mail-wm0-x230.google.com [IPv6:2a00:1450:400c:c09::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 618386E7DE for ; Fri, 5 Jan 2018 15:42:03 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: by mail-wm0-x230.google.com with SMTP id b76so3218067wmg.1 for ; Fri, 05 Jan 2018 07:42:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language; bh=rWaA0HvonxHzf3TmqYc2xGFqaRi9I03ZLfRDpX5DolU=; b=RiqbTkEpXytsa0rCfIqe5q+8GQdiaqyiU6muzgjU+Sx7oIc9cVQ4OrgBUuBvrVntcT u+di/CZ5rQd4ewkcC0uNuKwg4S+N9OAe/PflaGhtWFvTALzcc/1NCrihhoEDrO0X4Nih TQ0lhntMr55hQNwnRKRAxIUMyq2Q1FhUn1ZjTZId36mkN9inILDiw8SEXfC9gTBoF1FP DGd8yNXz6kMK8+ztLNsV+tZINm+2O1R6Js3HFR1jlvT8RZp7Mv7aSn7rTYUEyL7loPdw 7MoV6b9TEW8j2Kq0uIMgl+rh9ylYn4wtt2LaNT/ORcGv5K3xqotxV/4PzFzoVjiDt545 QYWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=rWaA0HvonxHzf3TmqYc2xGFqaRi9I03ZLfRDpX5DolU=; b=NfqO6nR1OL0a0r+R9sZneDqfYiwGgwevfiqA0qmiUPg5WtTqpZW/EQGRdWGBKNIHBf TVEsJGhk2mdCGopLxjn+Vm1cSJioqrZmASWq/QiKwy5RIc55npeZtKZl4bLN9/HtOxZP AfaBWJKsVT8Etka8xIv+NGWzYWrec+WuKmNFJ8zrqVXlY9O9Xd4VRovrJVxuG3wMNUyY 3rVhUdT5Qz1M2NMd4STZzhckJPbiMEmQna7sZhMZNFJFQQt5/vm3VvWUJwYgLYi+A+Gk 3AlyoXT7tV7M16nnM2RGx+TcErqSSKHCQUPn9Z8ogyKpth98vRgUIb2mD6YJw1J8lGpm AATw== X-Gm-Message-State: AKGB3mIeEZ3QvVWmffLxdPSoRgsPLkY+0O5JTSqR+dYP3dvnSMOrSLy8 Dh08zQtyTaRySdkG3OvrcQz88A== X-Google-Smtp-Source: ACJfBouBu5mg3x3U/3VXTKXvAhaQyKfjIlOP5ctZQ4zMjsErtFHFuVBj+FMIEOyAXhn/leVB6od07w== X-Received: by 10.80.206.11 with SMTP id y11mr4774219edi.137.1515166921738; Fri, 05 Jan 2018 07:42:01 -0800 (PST) Received: from [10.10.1.111] ([185.97.61.1]) by smtp.gmail.com with ESMTPSA id a52sm4323015eda.92.2018.01.05.07.41.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Jan 2018 07:42:00 -0800 (PST) Subject: Re: svn commit: r327559 - in head: . sys/net To: Eugene Grosbein , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201801042005.w04K5liB049411@repo.freebsd.org> <5A4E9397.9000308@grosbein.net> <5A4EDC62.50508@grosbein.net> <5A4F824C.1060405@grosbein.net> From: Steven Hartland Message-ID: <97d173fb-4f47-609d-8319-07282a283473@multiplay.co.uk> Date: Fri, 5 Jan 2018 15:42:01 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: <5A4F824C.1060405@grosbein.net> Content-Language: en-US Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 15:42:03 -0000 On 05/01/2018 13:49, Eugene Grosbein wrote: > 05.01.2018 16:26, Steven Hartland пишет: >> On 05/01/2018 02:01, Eugene Grosbein wrote: >>> 05.01.2018 4:52, Steven Hartland wrote: >>> >>>>> RSS by definition has meaning to received stream. What is "outbound" stream >>>>> in this context, why can the hash calculatiom method change and what exactly >>>>> does it mean "a stream being incorrectly split"? >>>> Yes RSS is indeed a received stream but that is used by lagg for lacp and loadbalance protocols >>>> to decide which port of the lagg to "send" the packet out of. >>>> As the flowid is not known when a new "output" stream is instigated the current code >>>> falls back to manual hash calculation to determine which port to send the initial packet from. >>>> Once a response is received a tx then uses the flowid. >>>> This change of hash calculation method can result in the initial packet being sent >>>> from a different port than the rest of the stream; this is what I meant by "incorrectly split". >>>> >>>> See the following: >>>> https://github.com/freebsd/freebsd/blob/master/sys/net/if_lagg.c#L2066 >>>> https://github.com/freebsd/freebsd/blob/master/sys/net/ieee8023ad_lacp.c#L846 >>> I still do not get what is "output stream" for you. >>> >>> If you are talking on forwarding (routing) transit packets at IP layer, >>> they all have flowid from the beginning and first packet does not differ from others at all. >> At the simplest level its a tcp stream that is started from the host. So given we have hostA (src) and hostB (dest), the output stream is one started by hostA with a destination of hostB where hostA is configured with lagg. >> >> In this case with use_flowid we've confirmed we get the following (the interfaces used vary per flow of cause): >> hostA - SYN (ix0) -> hostB # Manual hash calculated >> hostB - SYN,ACK (ix0) -> hostA# flowid used >> hostA - ACK (ix1) -> hostB # flowid used >> hostA - Data(ix1) -> hostB # flowid used >> hostB - ACK (ix0) -> hostA # flowid used >> ... >> >> Here hostA and hostB both had lagg0 comprising of ix0 and ix1. > It should be: > > hostA - SYN (ix0) -> hostB # Manual hash (1) calculated > hostB - SYN,ACK (ix0) -> hostA# hardware flowid (2) received > hostA - ACK (ix1) -> hostB # Manual hash (1) calculated > hostA - Data(ix1) -> hostB # hardware flowid (2 or 3) received > hostB - ACK (ix0) -> hostA # Manual hash (1) calculated > > That is, there is no guarantee of persistance of flowid of incoming packets > as they can be received with distinct ports of lagg being distinct hardware > computing flowid differently. Some ports may not support RSS at all. > We should not use incoming hardware flowid for anything by default in case of TCP. I don't believe your statement about persistence of flowid due to the use of variant ports is correct as LACP states that packets from the same flow "should" under normal conditions (no failure) be received on the same port. In the case where the HW doesn't support RSS, then flowid should either always be unset or be set by OS to consistent value hence that should function as expected. That said I don't disagree that all hostA -> hostB should use Manual hash, as I can't see anyway to use to HW hash, however the ports in your example are wrong, all hostA -> hostB should be sent from the same ixY and all hostB -> hostA should be sent from the same ixZ (under normal circumstances) of course. >>> If you are talking on locally originated (not transit) data streem from local TCP socket >>> being sent in response to corresponding incoming TCP segments, then these outgoing >>> packets should have their own fixed flow id by default in case of LACP >>> and thhis flow id should not depend on (possibly ever changing) flow id of incoming TCP segments. >> Nope in this case we have all the information needed, but I don't believe we can't tell that's the case. >>> If you insist that flow id of outgoing packets does depend on ever changing incoming packet's flow id, >>> then this is the bug that should be fixed and not lagg's defaults. >> As detailed above once the session is established then the flowid remains fixed. > Why do you mix flowid of incoming stream with flowid of outgoing stream? > I expect this was done so we don't have the overhead of calculating a packet hash for every outgoing packet i.e. its an optimization, however I believe this is only possible for the destination host which always has a valid flowid, and not for the source host. My current thinking is that flowid shouldn't be used for either LACP or loadbalance protocols as doing so will almost certainly lead to unexpected behavior (the stated lagghash may not be valid).     Regards     Steve