From owner-freebsd-stable@FreeBSD.ORG Sun Jul 7 17:28:09 2013 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 7D54E772 for ; Sun, 7 Jul 2013 17:28:09 +0000 (UTC) (envelope-from kaushalgoa@gmail.com) Received: from mail-ee0-x22d.google.com (mail-ee0-x22d.google.com [IPv6:2a00:1450:4013:c00::22d]) by mx1.freebsd.org (Postfix) with ESMTP id 154D21D0B for ; Sun, 7 Jul 2013 17:28:08 +0000 (UTC) Received: by mail-ee0-f45.google.com with SMTP id c1so2362358eek.18 for ; Sun, 07 Jul 2013 10:28:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=uDSSC48pvpQLp8YWuiKQGnVAoEEHiUAd36yM87O9axk=; b=EIAADfzQt9HbcagH1c9WRkl770dw6uvEyzpRS8RJoETsCm5UsghwqXMER86+H10rPd g4wU0NNHDh7fNBJzpwrNRBgTSox/fkxF+jvytuEKMurT65ZGI3JaU3qkNjy3LrV7H1vL z/LkRd0k1SZZ0je4MXB6bU3E0torGbFrLpRVtogfBe7CC4frJiG52Ro7XAFYgRl78Ojr mx027HZdSv+iKOBYSwpOkPDAqRlgCl9LBSUQAal4y85bpNMIormOct244UuzTWzYdP19 JvQtfGfhSekAw5AFkORGbyut0y82cF8aTzPfJXhayjeLDM0QKHVc4WrSnhXxGxkuUEEU XmaA== X-Received: by 10.14.7.133 with SMTP id 5mr21408455eep.115.1373218088077; Sun, 07 Jul 2013 10:28:08 -0700 (PDT) MIME-Version: 1.0 Received: by 10.14.138.141 with HTTP; Sun, 7 Jul 2013 10:27:47 -0700 (PDT) In-Reply-To: References: From: Kaushal Bhandankar Date: Sun, 7 Jul 2013 22:57:47 +0530 Message-ID: Subject: Re: ixgbe Jumbo race condition leading to Deadlock To: Jack Vogel Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD stable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Jul 2013 17:28:09 -0000 The solution is: I have a function which pre-calculates the buffers required for processing the packet. This is to eliminate any lack-of-memory errors during processing. In this function I loop over descriptors from next_to_check onwards. If I loop over some descriptors with DD set and do not see the EOP set till I reach 5 descriptors, I will return a failure and not change next_to_check etc. So that on the next read, or later hopefully descriptor with EOP would have been written back. This will ensure that race condition does not happen. Let me know if it sounds good. Regards, Kaushal On Sun, Jul 7, 2013 at 10:43 PM, Kaushal Bhandankar wrote: > Hi Jack, > Thanks for the explanation. Do you suggest that I keep reading rx > descriptor with DD bit and keep them pending till I get the descriptor with > EOP set ? How much max delay can be expected for the EOP descriptor to be > written back ? > > Regards, > Kaushal > > > On Sun, Jul 7, 2013 at 10:40 PM, Jack Vogel wrote: > >> The "potential race condition" as the data sheet puts it, is only when >> you are >> trying to manage your RX ring by reading the RDH register, this is a bad >> idea >> anyway, none of our (Intel) drivers do this. Using the DD bit is what you >> want >> to do. The DD bit is set when the descriptor is written back, and that >> happens >> when the DMA is complete. >> >> The packet is incomplete until the descriptor with EOP set, in my code an >> mbuf chain is created, and as each new descriptor is processed the pointer >> to the head of the whole chain is kept in rxbuf->fmp, thus when you get to >> the EOP descriptor you will be ready to send the whole chain to the stack. >> >> Its good that you are using ONEBUF since packet split has hardware issues >> on 82599. >> >> Are you developing a new driver, or simply having issues using mine? >> >> Regards, >> >> Jack >> >> >> >> On Sun, Jul 7, 2013 at 2:24 AM, Kaushal Bhandankar wrote: >> >>> In 82599, for a Jumbo packet of 9.5 K ( which consumes 5 descriptors of >>> 2048 bytes each ), when does the Descriptor write back happen ? Does it >>> happen per Descriptor or once per aggregated Descriptors ? Is it possible >>> that all descriptors except last one to be written back and when you read >>> RDH register, I get the last pending descriptor waiting inside 82599. >>> We are using srrctl |= IXGBE_SRRCTL_DESCTYPE_ADV_ONEBUF; >>> >>> In my setup, I am seeing that, I don't see EOP set even when I read 5 >>> descriptors. Checking DD will return me an incomplete packet. What >>> should I >>> do in such a case ? >>> >>> References from Data sheet: >>> -> Checking through DD bits eliminates a potential race condition: all >>> descriptor data is posted internally prior to incrementing the head >>> register and a read of the head register could potentially pass the >>> descriptor waiting inside the 82599. >>> >>> Regards, >>> Kaushal >>> _______________________________________________ >>> freebsd-stable@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >>> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org >>> " >>> >> >> >