Date: Sat, 25 Oct 2014 00:56:28 +0800 From: Julian Elischer <julian@freebsd.org> To: javocado <javocado@gmail.com>, freebsd-net@freebsd.org Subject: Re: ipfw command freezes system Message-ID: <544A84BC.5000602@freebsd.org> In-Reply-To: <CAP1HOmSBcvTcJs4FHGk%2BO2dVmyn=UN6URnre74bQOcyJx8hWog@mail.gmail.com> References: <CAP1HOmSBcvTcJs4FHGk%2BO2dVmyn=UN6URnre74bQOcyJx8hWog@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 10/23/14, 7:41 AM, javocado wrote: > I'm seeing an occasional, recurring problem on my 8.3-RELEASE amd64 systems > where when I enter an ipfw rule, the system becomes locked up. > > For example, when entering a command like this: > > ipfw add 1 allow ip from x.x.x.x to me > > or other times with a command like: > > ipfw add xxx skipto ........ be careful of making packet loops. especially with skiptos , state and pipes. depending on the settings of the sysctls that control reinjection behaviour. > the server becomes unreachable via the network. I am however still able to > get to a shell via console where I ran top: > > last pid: 25518; load averages: 0.75, 1.12, 0.93 up 4+00:13:02 > 13:55:34 > 221 processes: 1 running, 215 sleeping, 5 lock > CPU: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle > Mem: 486M Active, 40G Inact, 174G Wired, 24M Cache, 24G Buf, 18G Free > Swap: 32G Total, 32G Free > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND > 25518 root 1 44 0 9372K 2768K CPU12 12 0:00 0.10% top > 79561 xxxx 1 46 0 35284K 13424K *IPFW 12 271:00 0.00% sshd > 79564 xxxx 1 45 0 156M 16808K select 0 65:08 0.00% rsync > 22311 xxxx 1 44 0 27092K 4676K select 4 13:23 0.00% sshd > 13056 xxxx 1 44 0 27092K 4720K select 0 12:13 0.00% sshd > 43625 xxxx 1 45 0 29140K 6924K select 5 10:25 0.00% sshd > 18925 xxxx 1 44 0 27092K 4640K select 8 10:08 0.00% sshd > 14423 xxxx 1 44 0 29140K 6328K select 3 7:59 0.00% sshd > 11356 xxxx 1 44 0 12992K 2544K select 2 7:52 0.00% > sftp-server > 20619 xxxx 1 44 0 116M 101M select 5 6:25 0.00% rsync > 98406 xxxx 1 44 0 29140K 7136K select 2 6:09 0.00% sshd > 20617 xxxx 1 44 0 35284K 12992K *IPFW 15 6:03 0.00% sshd > 54146 xxxx 1 44 0 27092K 4556K select 3 5:04 0.00% sshd > 63688 xxxx 1 44 0 27092K 5728K select 16 4:07 0.00% sshd > 20624 xxxx 1 44 0 156M 102M select 0 3:45 0.00% rsync > 43629 xxxx 1 44 0 5832K 2084K select 0 3:41 0.00% rsync > > Note those "*IPFW" state processes are long running child sshd processes, > not master sshd daemon itself. > > I've tried to do an ipfw flush in these situations before but those > commands never return and the system just stays locked up and unreachable. > > I was able to issue a reboot from the console, but even that did not > complete: > > Oct 21 13:56:53 xxxx reboot: rebooted by root > Oct 21 13:56:54 xxxx syslogd: exiting on signal 15 > Waiting (max 60 seconds) for system process `vnlru' to stop...done > Waiting (max 60 seconds) for system process `bufdaemon' to stop... > > and I had to reset. > > Here's the ipfw ruleset in place: > > 00110 count ip from any to any via igb0 in > 00111 count ip from any to any via igb0 out > 00210 skipto 410 ip from x.x.x.x/27,x.x.x.x/29,x.x.x.x/24,x.x.x.x/28 to me > 00210 skipto 410 ip from me to > x.x.x.x/27,x.x.x.x/29,x.x.x.x/24,x.x.x.x/28 > 00211 skipto 410 ip from x.x.x.x/24 to me > 00211 skipto 410 ip from me to x.x.x.x/24 > 00212 skipto 410 ip from x.x.x.x/24 to me > 00212 skipto 410 ip from me to x.x.x.x/24 > 00214 skipto 410 ip from x.x.x.x,x.x.x.x to me > 00214 skipto 410 ip from me to x.x.x.x,x.x.x.x > 00218 skipto 410 ip from x.x.x.x to me > 00218 skipto 410 ip from me to x.x.x.x > 00219 skipto 410 ip from x.x.x.x to me > 00219 skipto 410 ip from me to x.x.x.x > 00225 skipto 410 ip from x.x.x.x to me > 00225 skipto 410 ip from me to x.x.x.x > 00226 skipto 410 ip from x.x.x.x to me > 00226 skipto 410 ip from me to x.x.x.x > 00227 skipto 410 ip from x.x.x.x to me skipto tablearg maybe? instead of N rules also, separate all incoming and outgoing packets to diffent sets of rules.. e.g. send all incoming to 10000 and outgoing to 20000 the way you are doing it all packets look at all rules. > 00227 skipto 410 ip from me to x.x.x.x > 00310 pipe 5 ip from me to x.x.x.x > 00310 pipe 5 ip from x.x.x.x to me > 00311 skipto 410 ip from me to x.x.x.x > 00311 skipto 410 ip from x.x.x.x to me > 00312 pipe 1 ip from any to me > 00313 pipe 2 ip from any to me ummm 2 pipes for the same packets? > 00314 pipe 3 ip from me to any > 00315 pipe 4 ip from me to any once again... in and out should be separated. > 00410 allow tcp from any to any established > 00411 allow icmp from any to any icmptypes 0,3,8,11 > 00420 deny icmp from any to any > 00430 allow ip from any to any via lo0 > 00510 deny ip from any to 127.0.0.0/8 > 00511 deny ip from 127.0.0.0/8 to any > 00512 deny ip from 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16 to any table > 00513 deny ip from any to 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16 table > 00514 deny ip from > 0.0.0.0/8,169.254.0.0/16,192.0.2.0/24,224.0.0.0/4,240.0.0.0/4 to > any > 00515 deny ip from any to > 0.0.0.0/8,169.254.0.0/16,192.0.2.0/24,224.0.0.0/4,240.0.0.0/4 > 00610 deny tcp from any to any tcpflags syn tcpoptions !mss table > 00611 deny tcp from any to any tcpflags syn,fin > 00612 deny tcp from any to any tcpflags fin,psh,rst,urg > 00613 deny tcp from any to any tcpflags fin,psh,urg > 00614 deny tcp from any to any tcpflags syn,fin,ack,rst > 00615 deny tcp from any to any tcpflags !syn,!fin,!ack > 01010 allow udp from me to any dst-port 53 > 01011 allow udp from x.x.x.x,x.x.x.x 53 to me > 01012 allow udp from any to me dst-port 33433-33499 > 01020 allow tcp from any to x.x.x.x dst-port 21,22,62000-64000 setup > 65000 deny log logamount 1000 ip from any to me > 65001 deny log logamount 1000 ip from any to me6 > 65535 allow ip from any to any > > # ipfw pipe list > 00001: 200.000 Mbit/s 0 ms burst 0 > q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 > droptail > sched 65537 type FIFO flags 0x0 0 buckets 1 active > 0 ip 0.0.0.0/0 0.0.0.0/0 10 15000 0 0 0 > > 00002: 32.000 Mbit/s 0 ms burst 0 > q131074 50 sl. 0 flows (1 buckets) sched 65538 weight 0 lmax 0 pri 0 > droptail > sched 65538 type FIFO flags 0x1 64 buckets 10 active > mask: 0x00 0xffffffff/0x0000 -> 0x00000000/0x0000 > 0 > 00003: 100.000 Mbit/s 0 ms burst 0 > q131075 50 sl. 0 flows (1 buckets) sched 65539 weight 0 lmax 0 pri 0 > droptail > sched 65539 type FIFO flags 0x0 0 buckets 1 active > 0 ip 0.0.0.0/0 0.0.0.0/0 6 312 1 52 0 > > 00004: 24.000 Mbit/s 0 ms burst 0 > q131076 50 sl. 0 flows (1 buckets) sched 65540 weight 0 lmax 0 pri 0 > droptail > sched 65540 type FIFO flags 0x1 64 buckets 9 active > mask: 0x00 0x00000000/0x0000 -> 0xffffffff/0x0000 > 0 > > 00005: 40.000 Mbit/s 0 ms burst 0 > q131077 50 sl. 0 flows (1 buckets) sched 65541 weight 0 lmax 0 pri 0 > droptail > sched 65541 type FIFO flags 0x0 0 buckets 0 active > > > What other troubleshooting or remedy should I do via console when this > happens, or perhaps is there a problem with the way we've setup our ruleset? > > Thanks for your help! > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?544A84BC.5000602>