From owner-freebsd-current@FreeBSD.ORG Thu Sep 23 23:21:27 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3B86816A4CE for ; Thu, 23 Sep 2004 23:21:27 +0000 (GMT) Received: from mail6.speakeasy.net (mail6.speakeasy.net [216.254.0.206]) by mx1.FreeBSD.org (Postfix) with ESMTP id E023E43D4C for ; Thu, 23 Sep 2004 23:21:26 +0000 (GMT) (envelope-from jmg@hydrogen.funkthat.com) Received: (qmail 11912 invoked from network); 23 Sep 2004 23:21:26 -0000 Received: from gate.funkthat.com (HELO hydrogen.funkthat.com) ([69.17.45.168]) (envelope-sender ) by mail6.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 23 Sep 2004 23:21:26 -0000 Received: from hydrogen.funkthat.com (rahsrq@localhost.funkthat.com [127.0.0.1])i8NNLPuU040680; Thu, 23 Sep 2004 16:21:26 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: (from jmg@localhost) by hydrogen.funkthat.com (8.12.10/8.12.10/Submit) id i8NNLPoq040679; Thu, 23 Sep 2004 16:21:25 -0700 (PDT) Date: Thu, 23 Sep 2004 16:21:25 -0700 From: John-Mark Gurney To: Sean McNeil Message-ID: <20040923232124.GM72089@funkthat.com> Mail-Followup-To: Sean McNeil , freebsd-current@freebsd.org References: <1095978035.59583.4.camel@server.mcneil.com> <20040923225234.GL72089@funkthat.com> <1095980931.59840.5.camel@server.mcneil.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1095980931.59840.5.camel@server.mcneil.com> User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD 4.2-RELEASE i386 X-PGP-Fingerprint: B7 EC EF F8 AE ED A7 31 96 7A 22 B3 D8 56 36 F4 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html cc: freebsd-current@freebsd.org Subject: Re: re0 device txcsum issue X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: John-Mark Gurney List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Sep 2004 23:21:27 -0000 Sean McNeil wrote this message on Thu, Sep 23, 2004 at 16:08 -0700: > On Thu, 2004-09-23 at 15:52, John-Mark Gurney wrote: > > Sean McNeil wrote this message on Thu, Sep 23, 2004 at 15:20 -0700: > > > Is anyone willing to work with me to help trace down this problem? It > > > has been outstanding for a long time and I would dearly like it fixed. > > > I'm willing to help in any capacity to trace down the culprit. > > > > > > To recap, on the re0 device (possibly others) running -current on an > > > amd64 processor there are times when udp packets get improper checksum > > > calculations with txcsum set for the interface. This causes a deadlock > > > in nfs as the client just contunuously requests this packet and it is > > > rejected because of the checksum being bad. > > > > I have recently been working with the re driver, so I'm somewhat familar > > with the driver. I haven't seen any issues, but I also don't have an > > AMD64 system to test with. > > > > Have you tried to find out if it is packet size related? are you > > trying to use jumbo frames? rwatson committed netsend to the src/tools > > tree that could help this, and I have attached udpcheck.py which is > > a client/server script to test/verify packet sizes of difference > > sizes. > > My method of testing has been to just do an "ls -lR" from a large > directory structure. With txcsum set, it consistently locks up. This > is on clients ranging from x86 linux, BSD, a sparc solaris2, and an hppa > HPUX machine. If I turn off txcsum (i.e. ifconfig re0 -txcsum) I have > never had any problems. > > I tried your program with txcsum and it just hangs. Without txcsum, You can provide a -v to get more detailed information on what is going on... Yes, the naming (in the python script) is a bit confusing since it is from the server's poing of view.. I'm assuming you run the server side (-s 1234) on another box and the client (server 1234) on the box w/ the re driver? so, the errors below means that the client sent a udp packet and it didn't match... > I'm not sure what the output here means, but this is what I got: w/o txcsum you got the following errors? This is very worrysome as it means that you're getting udp packet corruption even w/o checksuming.. Ok, what you can do is add the line: open('r.%d' % i, 'w').write(rdata) just before the line: print 'packet send mismatch at:', i, 'got:', rlen and then run the client as: python udpcheck.py server 1234 -s 1792 -e 1795 You will then have a set of r.179[2-5] files... They are the contents of the packet that was received by the server... if you could email them to me in private mail, it might shed light on the problem.. Thanks. > [sean@office mahi]$ python ~/Desktop/Downloads/udpcheck.py server 1234 > packet send mismatch at: 1792 > packet send mismatch at: 1793 > packet send mismatch at: 1794 > packet send mismatch at: 1795 > packet send mismatch at: 1796 > packet send mismatch at: 1797 > packet send mismatch at: 1798 > packet send mismatch at: 1799 > packet send mismatch at: 1800 > packet send mismatch at: 1801 > packet send mismatch at: 1802 > packet send mismatch at: 1803 > packet send mismatch at: 1804 > packet send mismatch at: 1805 > packet send mismatch at: 1806 > packet send mismatch at: 1807 > packet send mismatch at: 1808 > packet send mismatch at: 1809 > packet send mismatch at: 1810 > packet send mismatch at: 1811 > packet send mismatch at: 1812 > packet send mismatch at: 1813 > packet send mismatch at: 1814 > packet send mismatch at: 1815 > packet send mismatch at: 1816 > packet send mismatch at: 1817 > packet send mismatch at: 1818 > packet send mismatch at: 1819 > packet send mismatch at: 1820 > packet send mismatch at: 1821 > packet send mismatch at: 1822 > packet send mismatch at: 1823 > packet send mismatch at: 1824 > packet send mismatch at: 1825 > packet send mismatch at: 1826 > packet send mismatch at: 1827 > packet send mismatch at: 1828 > packet send mismatch at: 1829 > packet send mismatch at: 1830 > packet send mismatch at: 1831 > packet send mismatch at: 1832 > packet send mismatch at: 1833 > packet send mismatch at: 1834 > packet send mismatch at: 1835 > packet send mismatch at: 1836 > packet send mismatch at: 1837 > packet send mismatch at: 1838 > packet send mismatch at: 1839 > packet send mismatch at: 1840 > packet send mismatch at: 1841 > packet send mismatch at: 1842 > packet send mismatch at: 1843 > packet send mismatch at: 1844 > packet send mismatch at: 1845 > packet send mismatch at: 1846 > packet send mismatch at: 1847 > packet send mismatch at: 1848 > packet send mismatch at: 1849 > packet send mismatch at: 1850 > packet send mismatch at: 1851 > packet send mismatch at: 1852 > packet send mismatch at: 1853 > packet send mismatch at: 1854 > packet send mismatch at: 1855 > packet send mismatch at: 1856 > packet send mismatch at: 1857 > packet send mismatch at: 1858 > packet send mismatch at: 1859 > packet send mismatch at: 1860 > packet send mismatch at: 1861 > packet send mismatch at: 1862 > packet send mismatch at: 1863 > packet send mismatch at: 1864 > packet send mismatch at: 1865 > packet send mismatch at: 1866 > packet send mismatch at: 1867 > packet send mismatch at: 1868 > packet send mismatch at: 1869 > packet send mismatch at: 1870 > packet send mismatch at: 1871 > packet send mismatch at: 1872 > packet send mismatch at: 1873 > packet send mismatch at: 1874 > packet send mismatch at: 1875 > packet send mismatch at: 1876 > packet send mismatch at: 1877 > packet send mismatch at: 1878 > packet send mismatch at: 1879 > packet send mismatch at: 1880 > packet send mismatch at: 1881 > packet send mismatch at: 1882 > packet send mismatch at: 1883 > packet send mismatch at: 1884 > packet send mismatch at: 1885 > packet send mismatch at: 1886 > packet send mismatch at: 1887 > packet send mismatch at: 1888 > packet send mismatch at: 1889 > packet send mismatch at: 1890 > packet send mismatch at: 1891 > packet send mismatch at: 1892 > packet send mismatch at: 1893 > packet send mismatch at: 1894 > packet send mismatch at: 1895 > packet send mismatch at: 1896 > packet send mismatch at: 1897 > packet send mismatch at: 1898 > packet send mismatch at: 1899 > packet send mismatch at: 1900 > packet send mismatch at: 1901 > packet send mismatch at: 1902 > packet send mismatch at: 1903 > packet send mismatch at: 1904 > packet send mismatch at: 1905 > packet send mismatch at: 1906 > packet send mismatch at: 1907 > packet send mismatch at: 1908 > packet send mismatch at: 1909 > packet send mismatch at: 1910 > packet send mismatch at: 1911 > packet send mismatch at: 1912 > packet send mismatch at: 1913 > packet send mismatch at: 1914 > packet send mismatch at: 1915 > packet send mismatch at: 1916 > packet send mismatch at: 1917 > packet send mismatch at: 1918 > packet send mismatch at: 1919 > packet send mismatch at: 1920 > packet send mismatch at: 1921 > packet send mismatch at: 1922 > packet send mismatch at: 1923 > packet send mismatch at: 1924 > packet send mismatch at: 1925 > packet send mismatch at: 1926 > packet send mismatch at: 1927 > packet send mismatch at: 1928 > packet send mismatch at: 1929 > packet send mismatch at: 1930 > packet send mismatch at: 1931 > packet send mismatch at: 1932 > packet send mismatch at: 1933 > packet send mismatch at: 1934 > packet send mismatch at: 1935 > packet send mismatch at: 1936 > packet send mismatch at: 1937 > packet send mismatch at: 1938 > packet send mismatch at: 1939 > packet send mismatch at: 1940 > packet send mismatch at: 1941 > packet send mismatch at: 1942 > packet send mismatch at: 1943 > packet send mismatch at: 1944 > packet send mismatch at: 1945 > packet send mismatch at: 1946 > packet send mismatch at: 1947 > packet send mismatch at: 1948 > packet send mismatch at: 1949 > packet send mismatch at: 1950 > packet send mismatch at: 1951 > packet send mismatch at: 1952 > packet send mismatch at: 1953 > packet send mismatch at: 1954 > packet send mismatch at: 1955 > packet send mismatch at: 1956 > packet send mismatch at: 1957 > packet send mismatch at: 1958 > packet send mismatch at: 1959 > packet send mismatch at: 1960 > packet send mismatch at: 1961 > packet send mismatch at: 1962 > packet send mismatch at: 1963 > packet send mismatch at: 1964 > packet send mismatch at: 1965 > packet send mismatch at: 1966 > packet send mismatch at: 1967 > packet send mismatch at: 1968 > packet send mismatch at: 1969 > packet send mismatch at: 1970 > packet send mismatch at: 1971 > packet send mismatch at: 1972 > packet send mismatch at: 1973 > packet send mismatch at: 1974 > packet send mismatch at: 1975 > packet send mismatch at: 1976 > packet send mismatch at: 1977 > packet send mismatch at: 1978 > packet send mismatch at: 1979 > packet send mismatch at: 1980 > packet send mismatch at: 1981 > packet send mismatch at: 1982 > packet send mismatch at: 1983 > packet send mismatch at: 1984 > packet send mismatch at: 1985 > packet send mismatch at: 1986 > packet send mismatch at: 1987 > packet send mismatch at: 1988 > packet send mismatch at: 1989 > packet send mismatch at: 1990 > packet send mismatch at: 1991 > packet send mismatch at: 1992 > packet send mismatch at: 1993 > packet send mismatch at: 1994 > packet send mismatch at: 1995 > packet send mismatch at: 1996 > packet send mismatch at: 1997 > packet send mismatch at: 1998 > packet send mismatch at: 1999 > packet send mismatch at: 2000 > packet send mismatch at: 2001 > packet send mismatch at: 2002 > packet send mismatch at: 2003 > packet send mismatch at: 2004 > packet send mismatch at: 2005 > packet send mismatch at: 2006 > packet send mismatch at: 2007 > packet send mismatch at: 2008 > packet send mismatch at: 2009 > packet send mismatch at: 2010 > packet send mismatch at: 2011 > packet send mismatch at: 2012 > packet send mismatch at: 2013 > packet send mismatch at: 2014 > packet send mismatch at: 2015 > packet send mismatch at: 2016 > packet send mismatch at: 2017 > packet send mismatch at: 2018 > packet send mismatch at: 2019 > packet send mismatch at: 2020 > packet send mismatch at: 2021 > packet send mismatch at: 2022 > packet send mismatch at: 2023 > packet send mismatch at: 2024 > packet send mismatch at: 2025 > packet send mismatch at: 2026 > packet send mismatch at: 2027 > packet send mismatch at: 2028 > packet send mismatch at: 2029 > packet send mismatch at: 2030 > packet send mismatch at: 2031 > packet send mismatch at: 2032 > packet send mismatch at: 2033 > packet send mismatch at: 2034 > packet send mismatch at: 2035 > packet send mismatch at: 2036 > packet send mismatch at: 2037 > packet send mismatch at: 2038 > packet send mismatch at: 2039 > packet send mismatch at: 2040 > packet send mismatch at: 2041 > packet send mismatch at: 2042 > packet send mismatch at: 2043 > packet send mismatch at: 2044 > packet send mismatch at: 2045 > packet send mismatch at: 2046 > packet send mismatch at: 2047 > [('r', 1792), ('r', 1793), ('r', 1794), ('r', 1795), ('r', 1796), ('r', > 1797), ('r', 1798), ('r', 1799), ('r', 1800), ('r', 1801), ('r', 1802), > ('r', 1803), ('r', 1804), ('r', 1805), ('r', 1806), ('r', 1807), ('r', > 1808), ('r', 1809), ('r', 1810), ('r', 1811), ('r', 1812), ('r', 1813), > ('r', 1814), ('r', 1815), ('r', 1816), ('r', 1817), ('r', 1818), ('r', > 1819), ('r', 1820), ('r', 1821), ('r', 1822), ('r', 1823), ('r', 1824), > ('r', 1825), ('r', 1826), ('r', 1827), ('r', 1828), ('r', 1829), ('r', > 1830), ('r', 1831), ('r', 1832), ('r', 1833), ('r', 1834), ('r', 1835), > ('r', 1836), ('r', 1837), ('r', 1838), ('r', 1839), ('r', 1840), ('r', > 1841), ('r', 1842), ('r', 1843), ('r', 1844), ('r', 1845), ('r', 1846), > ('r', 1847), ('r', 1848), ('r', 1849), ('r', 1850), ('r', 1851), ('r', > 1852), ('r', 1853), ('r', 1854), ('r', 1855), ('r', 1856), ('r', 1857), > ('r', 1858), ('r', 1859), ('r', 1860), ('r', 1861), ('r', 1862), ('r', > 1863), ('r', 1864), ('r', 1865), ('r', 1866), ('r', 1867), ('r', 1868), > ('r', 1869), ('r', 1870), ('r', 1871), ('r', 1872), ('r', 1873), ('r', > 1874), ('r', 1875), ('r', 1876), ('r', 1877), ('r', 1878), ('r', 1879), > ('r', 1880), ('r', 1881), ('r', 1882), ('r', 1883), ('r', 1884), ('r', > 1885), ('r', 1886), ('r', 1887), ('r', 1888), ('r', 1889), ('r', 1890), > ('r', 1891), ('r', 1892), ('r', 1893), ('r', 1894), ('r', 1895), ('r', > 1896), ('r', 1897), ('r', 1898), ('r', 1899), ('r', 1900), ('r', 1901), > ('r', 1902), ('r', 1903), ('r', 1904), ('r', 1905), ('r', 1906), ('r', > 1907), ('r', 1908), ('r', 1909), ('r', 1910), ('r', 1911), ('r', 1912), > ('r', 1913), ('r', 1914), ('r', 1915), ('r', 1916), ('r', 1917), ('r', > 1918), ('r', 1919), ('r', 1920), ('r', 1921), ('r', 1922), ('r', 1923), > ('r', 1924), ('r', 1925), ('r', 1926), ('r', 1927), ('r', 1928), ('r', > 1929), ('r', 1930), ('r', 1931), ('r', 1932), ('r', 1933), ('r', 1934), > ('r', 1935), ('r', 1936), ('r', 1937), ('r', 1938), ('r', 1939), ('r', > 1940), ('r', 1941), ('r', 1942), ('r', 1943), ('r', 1944), ('r', 1945), > ('r', 1946), ('r', 1947), ('r', 1948), ('r', 1949), ('r', 1950), ('r', > 1951), ('r', 1952), ('r', 1953), ('r', 1954), ('r', 1955), ('r', 1956), > ('r', 1957), ('r', 1958), ('r', 1959), ('r', 1960), ('r', 1961), ('r', > 1962), ('r', 1963), ('r', 1964), ('r', 1965), ('r', 1966), ('r', 1967), > ('r', 1968), ('r', 1969), ('r', 1970), ('r', 1971), ('r', 1972), ('r', > 1973), ('r', 1974), ('r', 1975), ('r', 1976), ('r', 1977), ('r', 1978), > ('r', 1979), ('r', 1980), ('r', 1981), ('r', 1982), ('r', 1983), ('r', > 1984), ('r', 1985), ('r', 1986), ('r', 1987), ('r', 1988), ('r', 1989), > ('r', 1990), ('r', 1991), ('r', 1992), ('r', 1993), ('r', 1994), ('r', > 1995), ('r', 1996), ('r', 1997), ('r', 1998), ('r', 1999), ('r', 2000), > ('r', 2001), ('r', 2002), ('r', 2003), ('r', 2004), ('r', 2005), ('r', > 2006), ('r', 2007), ('r', 2008), ('r', 2009), ('r', 2010), ('r', 2011), > ('r', 2012), ('r', 2013), ('r', 2014), ('r', 2015), ('r', 2016), ('r', > 2017), ('r', 2018), ('r', 2019), ('r', 2020), ('r', 2021), ('r', 2022), > ('r', 2023), ('r', 2024), ('r', 2025), ('r', 2026), ('r', 2027), ('r', > 2028), ('r', 2029), ('r', 2030), ('r', 2031), ('r', 2032), ('r', 2033), > ('r', 2034), ('r', 2035), ('r', 2036), ('r', 2037), ('r', 2038), ('r', > 2039), ('r', 2040), ('r', 2041), ('r', 2042), ('r', 2043), ('r', 2044), > ('r', 2045), ('r', 2046), ('r', 2047)] > -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."