Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 4 Oct 2012 09:41:13 -0400
From:      Dominic Blais <dblais@interplex.ca>
To:        "Alexander V. Chernikov" <melifaro@FreeBSD.org>
Cc:        "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>
Subject:   RE: Default route destination changing without warning follow-up
Message-ID:  <2DE61B0869B7484997BCA012845482C7EBE8E28127@WIN2008.Domnt.abi.ca>
In-Reply-To: <5068B48E.2070303@FreeBSD.org>
References:  <2DE61B0869B7484997BCA012845482C7EBE8E280DB@WIN2008.Domnt.abi.ca> <5068AC17.8020704@FreeBSD.org> <2DE61B0869B7484997BCA012845482C7EBE8E280DC@WIN2008.Domnt.abi.ca> <5068ADCC.5030105@FreeBSD.org> <2DE61B0869B7484997BCA012845482C7EBE8E280DD@WIN2008.Domnt.abi.ca> <5068B48E.2070303@FreeBSD.org>

next in thread | previous in thread | raw e-mail | index | archive | help
Hi,

The server that actually has the problem is:

- FreeBSD 9.0-RELEASE-p3
- Uses IPFW & dummynet for traffic shaping
- Uses PF for firewalling and NAT
- Uses MPD 5.6 for PPPoE server
- Uses freeradius and PostgreSQL for authentication.

I wrote a script some weeks ago that does the following:

=3D=3D=3D=3D=3D
#!/bin/sh

RIGHT_GATEWAY=3D'1.1.1.1'
NOC_EMAIL=3D'blah@test.com'

i=3D5
while [ $i -gt 0 ]
do
    if [ `/usr/bin/netstat -rn |grep default |grep $RIGHT_GATEWAY |wc -l` -=
ne 1 ]
    then
        /sbin/route del -net default
        /sbin/route add -net default $RIGHT_GATEWAY
        tail -n 100 /tmp/route_monitor.log | mail -s "Default route bug hap=
pened" $NOC_EMAIL
    fi
    i=3D`expr $i - 1`
    sleep 10
done
=3D=3D=3D=3D=3D

I've changed the variables for privacy purposes. This script is executed ea=
ch minutes in crontab. So if the gateway changes, it is fixed within 10 sec=
onds (a mean of 5 seconds).

I have seen multiple days where it doesn't happen at all... Today was a rec=
ord!! 43 times between 22:43 and 23:56. It means, 43 times the script detec=
ted the gateway wasn't the good one and had to change it. It looks like it =
happens when network usage is higher than usual.. there's obviously a link =
with usage...The server isn't using all its resources. I usually see 25% CP=
U usage at higher loads...  All loads value are almost always under 0.4 and=
 are mostly around 0.35 for past 15 minutes.

I'm running a route monitor in a screen and I'm sending the output to a tex=
tfile. As you see in the script, I'm sending me the last 100 lines of it an=
d I only see my script changing the gateway... nothing before is changing i=
t. The other people experiencing that bug seems to have the same behaviour.




-----Message d'origine-----
De=A0: Alexander V. Chernikov [mailto:melifaro@FreeBSD.org]=20
Envoy=E9=A0: 30 septembre 2012 17:07
=C0=A0: Dominic Blais
Cc=A0: freebsd-net@freebsd.org
Objet=A0: Re: Default route destination changing without warning follow-up

On 01.10.2012 00:59, Dominic Blais wrote:
> It's all about IPv4 in my case.

It will be great to supply some more details (e.g. like FreeBSD version, in=
terfaces configuration, netstat -rn output).

How often does this happen ?
(e.g. while true; do echo -n `date` ; route -n get default | grep gate; sle=
ep 1; done can help)

If this is reproducible, what actions precedes this change?
Maybe some ARP traffic on that interface, or interface creation/deletion, o=
r.. ?

Is route monitor completely silent when the change happens?

>
>
> --
>
>
>
> -----Message d'origine-----
> De : Alexander V. Chernikov [mailto:melifaro@FreeBSD.org] Envoy=E9 : 30=20
> septembre 2012 16:39 =C0 : Dominic Blais Cc : freebsd-net@freebsd.org=20
> Objet : Re: Default route destination changing without warning=20
> follow-up
>
> On 01.10.2012 00:33, Dominic Blais wrote:
>> Yes, I'm very sure of it! A "route monitor" show no changes when it happ=
ens. I've heard of another person with the same bug that has no traces in "=
route monitor" either... We have in common that we're using IPFW and PF.
> So, are we talking about IPv4 or IPv6?
>>
>>
>>
>> --
>>
>>
>>
>> -----Message d'origine-----
>> De : Alexander V. Chernikov [mailto:melifaro@FreeBSD.org] Envoy=E9 : 30=
=20
>> septembre 2012 16:31 =C0 : Dominic Blais Cc : freebsd-net@freebsd.org=20
>> Objet : Re: Default route destination changing without warning=20
>> follow-up
>>
>> On 01.10.2012 00:00, Dominic Blais wrote:
>>> Hi,
>> Hello!
>>>
>>> I was just wondering if there was anything new about the bug of default=
 route changing without warning...  Is there any test I can do to help fixi=
ng it?
>> Can you be a bit more precise and specify FreeBSD version and address fa=
mily?
>>
>> Are you sure that it is not changed by some other userland process (e.g.
>> did you do some `route monitor` checks) ?
>>>
>>> --
>>> [cid:image001.gif@01CD9F24.B366CBF0]
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> freebsd-net@freebsd.org mailing list=20
>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
>>
>>
>
>




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2DE61B0869B7484997BCA012845482C7EBE8E28127>