Date: Fri, 7 Sep 2012 11:01:34 +0100 From: Simon Dick <simond@irrelevant.org> To: Giulio Ferro <auryn@zirakzigil.org> Cc: freebsd-net@freebsd.org, "freebsd-stable@freebsd.org" <freebsd-stable@freebsd.org> Subject: Re: Problem with link aggregation + sshd Message-ID: <CAPyG9gMaNnLJOE00bH66qpAoJfdU=YtpUvz24fj%2B1nyvjDDCAg@mail.gmail.com> In-Reply-To: <5044F62E.8030001@zirakzigil.org> References: <5033FB17.7020600@zirakzigil.org> <503884A0.50708@zirakzigil.org> <FF06542A-9507-4C8C-99EC-8275B04D4CF1@my.gd> <E183609A-19E1-4EF4-B08D-FAA55779E193@my.gd> <503BC8F5.3040208@zirakzigil.org> <CAE63ME6oi_5Yam5wXuJzYBhhv%2BN6MnQPOXReXo2Ugo1hjvv25Q@mail.gmail.com> <503E7A16.6030600@zirakzigil.org> <5044F62E.8030001@zirakzigil.org>
next in thread | previous in thread | raw e-mail | index | archive | help
We've had similar problems with lagg at work, each lagg is made up of one igb and one em port, sometimes for no apparent reason they seem to stop passing through traffic. The easiest way we've found to get it working again is ifconfig down and up on one of the physical interfaces. This is on 8.1 On 3 September 2012 19:25, Giulio Ferro <auryn@zirakzigil.org> wrote: > No idea anybody why this bug happens? Patches? > > > > On 08/29/2012 10:22 PM, Giulio Ferro wrote: >> >> On 08/28/2012 11:12 AM, Damien Fleuriot wrote: >>> >>> Hi Giulio, >>> >>> >>> >>> Just to clear things up: >>> igb0: 192.168.9.60/24 >>> lagg0: 192.168.12.21/24 >>> >> >> Yes. >> Actually I notice now that the lagg0 address is different from what >> I wrote below in my rc.conf (192.168.12.7). I've just made many test >> with different configuration, but no matter, it just doesn't work... >> >> >>> >>> What's the IP of the host you're trying ssh connections from ? >> >> >> I'm just trying to connect to and from management interface igb0 >> (192.168.9.60). >> From external pc I do : ssh myuser@192.168.9.60 >> From that server I do : ssh myuser@pcaddress >> >> Just to be more precise, the consequences are: >> 1) daemon sshd on the server gets stuck and becomes unkillable >> 2) the first connection may work, but then the program ssh on the >> server becomes unresponsive and unkillable >> >> If I don't create a lagg0 interface and just connect (say) igb1 to >> the data switch, I've no problem and everything works. >> >> Just to answer others' question, I connect igb1, igb2 and igb3 to the >> same data switch in ports configured for aggregation. >> I connect igb0 to another management switch (of course not configured >> for aggregation) >> >> >>> >>> Also, just in case, did you enable any firewall ? (PF, ipfw) >> >> >> As I already said, no. Nothing is working/active on this server, just >> sshd. >> >> Thank you. >> >> >>> >>> >>> >>> On 27 August 2012 21:22, Giulio Ferro <auryn@zirakzigil.org> wrote: >>>> >>>> Hi, thanks for the answer >>>> >>>> Here is what you asked for: >>>> >>>> # ifconfig igb0 >>>> igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu >>>> 1500 >>>> >>>> >>>> options=4401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO> >>>> >>>> ether ... >>>> inet 192.168.9.60 netmask 0xffffff00 broadcast 192.168.9.255 >>>> inet6 .... prefixlen 64 scopeid 0x1 >>>> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> >>>> media: Ethernet autoselect (1000baseT <full-duplex>) >>>> status: active >>>> >>>> >>>> >>>> # netstat -rn >>>> Routing tables >>>> >>>> Internet: >>>> Destination Gateway Flags Refs Use Netif >>>> Expire >>>> default 192.168.9.1 UGS 0 0 igb0 >>>> 127.0.0.1 link#12 UH 0 0 lo0 >>>> 192.168.9.0/24 link#1 U 0 14 igb0 >>>> 192.168.9.60 link#1 UHS 0 0 lo0 >>>> 192.168.12.0/24 link#13 U 0 109 lagg0 >>>> 192.168.12.21 link#13 UHS 0 0 lo0 >>>> >>>> Internet6: >>>> Destination Gateway Flags >>>> Netif Expire >>>> ::/96 ::1 >>>> UGRS lo0 >>>> ::1 link#12 >>>> UH lo0 >>>> ::ffff:0.0.0.0/96 ::1 >>>> UGRS lo0 >>>> fe80::/10 ::1 >>>> UGRS lo0 >>>> fe80::%igb0/64 link#1 U >>>> igb0 >>>> fe80::ea39:35ff:feb6:a0d4%igb0 link#1 >>>> UHS lo0 >>>> fe80::%igb1/64 link#2 U >>>> igb1 >>>> fe80::ea39:35ff:feb6:a0d5%igb1 link#2 >>>> UHS lo0 >>>> fe80::%igb2/64 link#3 U >>>> igb2 >>>> fe80::ea39:35ff:feb6:a0d6%igb2 link#3 >>>> UHS lo0 >>>> fe80::%igb3/64 link#4 U >>>> igb3 >>>> fe80::ea39:35ff:feb6:a0d7%igb3 link#4 >>>> UHS lo0 >>>> fe80::%lo0/64 link#12 U >>>> lo0 >>>> fe80::1%lo0 link#12 >>>> UHS lo0 >>>> fe80::%lagg0/64 link#13 U >>>> lagg0 >>>> fe80::ea39:35ff:feb6:a0d5%lagg0 link#13 >>>> UHS lo0 >>>> ff01::%igb0/32 fe80::ea39:35ff:feb6:a0d4%igb0 >>>> U igb0 >>>> ff01::%igb1/32 fe80::ea39:35ff:feb6:a0d5%igb1 >>>> U igb1 >>>> ff01::%igb2/32 fe80::ea39:35ff:feb6:a0d6%igb2 >>>> U igb2 >>>> ff01::%igb3/32 fe80::ea39:35ff:feb6:a0d7%igb3 >>>> U igb3 >>>> ff01::%lo0/32 ::1 U >>>> lo0 >>>> ff01::%lagg0/32 fe80::ea39:35ff:feb6:a0d5%lagg0 U >>>> lagg0 >>>> ff02::/16 ::1 >>>> UGRS lo0 >>>> ff02::%igb0/32 fe80::ea39:35ff:feb6:a0d4%igb0 >>>> U igb0 >>>> ff02::%igb1/32 fe80::ea39:35ff:feb6:a0d5%igb1 >>>> U igb1 >>>> ff02::%igb2/32 fe80::ea39:35ff:feb6:a0d6%igb2 >>>> U igb2 >>>> ff02::%igb3/32 fe80::ea39:35ff:feb6:a0d7%igb3 >>>> U igb3 >>>> ff02::%lo0/32 ::1 U >>>> lo0 >>>> ff02::%lagg0/32 fe80::ea39:35ff:feb6:a0d5%lagg0 U >>>> lagg0 >>>> >>>> >>>> >>>> # netstat -aln | grep 22 >>>> tcp4 0 0 *.22 *.* LISTEN >>>> tcp6 0 0 *.22 *.* LISTEN >>>> >>>> Note that I already tried to only listen on igb0 interface >>>> (192.168.9.60) in >>>> sshd_config, but the results are exactly >>>> the same described below. >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> On 08/25/2012 01:22 PM, Damien Fleuriot wrote: >>>>> >>>>> >>>>> In the meantime kindly post: >>>>> >>>>> >>>>> Ifconfig for your igb0 >>>>> Netstat -rn >>>>> Netstat -aln | grep 22 >>>>> >>>>> >>>>> >>>>> On 25 Aug 2012, at 13:18, Damien Fleuriot <ml@my.gd> wrote: >>>>> >>>>>> I'll get back to you regarding link aggregation when I'm done with >>>>>> groceries. >>>>>> >>>>>> We use it here in production and it works flawlessly. >>>>>> >>>>>> >>>>>> >>>>>> On 25 Aug 2012, at 09:54, Giulio Ferro <auryn@zirakzigil.org> wrote: >>>>>> >>>>>>> No answer, so it seems that link aggregation doesn't really work in >>>>>>> freebsd, >>>>>>> this may help others with the same problem... >>>>>>> >>>>>>> I reverted back to one link for management and one for service, >>>>>>> and ssh >>>>>>> works as it should... >>>>>>> >>>>>>> >>>>>>> On 08/21/2012 11:18 PM, Giulio Ferro wrote: >>>>>>>> >>>>>>>> >>>>>>>> Scenario : freebsd 9 stable (yesterday) amd64 on HP server with 4 >>>>>>>> nic >>>>>>>> (igb) >>>>>>>> >>>>>>>> 1 nic is connected standalone to the management switch, the 3 other >>>>>>>> nics >>>>>>>> are connected to a switch configured for aggregation. >>>>>>>> >>>>>>>> If I configure the first nic (igb0) there is no problem, I can >>>>>>>> operate >>>>>>>> as I normally do and sshd functions normally. >>>>>>>> >>>>>>>> The problems start when I configure the 3 other nics for >>>>>>>> aggregation: >>>>>>>> >>>>>>>> in /etc/rc.conf >>>>>>>> ... >>>>>>>> ifconfig_igb1="up" >>>>>>>> ifconfig_igb2="up" >>>>>>>> ifconfig_igb3="up" >>>>>>>> >>>>>>>> cloned_interfaces=lagg0 >>>>>>>> ifconfig_lagg0="laggproto lacp laggport igb1 laggport igb2 laggport >>>>>>>> igb3 192.168.12.7/24" >>>>>>>> ... >>>>>>>> >>>>>>>> I restart the server and the aggregation seems to work correctly, in >>>>>>>> fact ifconfig returns the correct lagg0 interface with the >>>>>>>> aggregated >>>>>>>> links, the correct protocol (lacp) and the correct ip address and >>>>>>>> the >>>>>>>> status is active. I can ping other IPs on the aggregated link. >>>>>>>> >>>>>>>> Also the other (standalone) link seems to work correctly. I can ping >>>>>>>> that address from other machines, and I can ping other IPs from that >>>>>>>> server. >>>>>>>> >>>>>>>> DNS lookups work ok too I can also use telnet to connect to pop3 >>>>>>>> servers so there seems to be no problem on the network stack. >>>>>>>> >>>>>>>> But if I try to connect to the sshd service on that server, it hangs >>>>>>>> indefinitely. On the server I find two sshd processes: >>>>>>>> /usr/sbin/sshd >>>>>>>> /usr/sbin/sshd -R >>>>>>>> >>>>>>>> There is no message in the logs. >>>>>>>> >>>>>>>> If I try to kill sshd (/etc/rc.d/sshd stop) I can't. it just stays >>>>>>>> there >>>>>>>> forever waiting for the pid to die (it never does) >>>>>>>> >>>>>>>> Even ssh client doesn't seem to work. In fact, if I try to >>>>>>>> connect to >>>>>>>> another server, the ssh client may start to work correctly, then >>>>>>>> soon >>>>>>>> or later it just hangs there forever, and I can't kill it with >>>>>>>> ctrl-c. >>>>>>>> >>>>>>>> No firewall is configured, there is nothing else working on this >>>>>>>> server. >>>>>>>> >>>>>>>> Thanks for any suggestions... >>>>>>>> _______________________________________________ >>>>>>>> freebsd-stable@freebsd.org mailing list >>>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >>>>>>>> To unsubscribe, send any mail to >>>>>>>> "freebsd-stable-unsubscribe@freebsd.org" >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> freebsd-stable@freebsd.org mailing list >>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >>>>>>> To unsubscribe, send any mail to >>>>>>> "freebsd-stable-unsubscribe@freebsd.org" >>>> >>>> >>>> >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAPyG9gMaNnLJOE00bH66qpAoJfdU=YtpUvz24fj%2B1nyvjDDCAg>