Date: Wed, 15 Nov 2017 12:30:40 +0100 From: Cos Chan <rosettas@gmail.com> To: Ian Smith <smithi@nimnet.asn.au> Cc: freebsd-questions <freebsd-questions@freebsd.org>, Michael Ross <gmx@ross.cx>, Kurt Lidl <lidl@freebsd.org> Subject: Re: How to setup IPFW working with blacklistd Message-ID: <CAKV%2BxLC=ABe2i3TN8bo4XaVg3KfUbKsS96=6iyVDnsmWw-e8ag@mail.gmail.com> In-Reply-To: <20171115185528.V72828@sola.nimnet.asn.au> References: <mailman.87.1509969603.28633.freebsd-questions@freebsd.org> <20171106235944.U9710@sola.nimnet.asn.au> <CAKV%2BxLCizjt5M%2BmJmTZj-cr=D6rhXRwDjCkE=6Q-VQX73iY%2B4A@mail.gmail.com> <20171107033226.M9710@sola.nimnet.asn.au> <CAKV%2BxLBWgU6zmc7tQNA=0%2B=2aF23C1QfJ2i3q1gKYDttwsCTkg@mail.gmail.com> <20171107162914.G9710@sola.nimnet.asn.au> <CAKV%2BxLDQQcG3bvo1b2nUAu7oOVhdNzDDrPWTVp2qOmkWVV89BQ@mail.gmail.com> <20171108012948.A9710@sola.nimnet.asn.au> <CAKV%2BxLCQ9NE6%2BEg6NvHZuEED8Cf6ZX74unvk9ajfLyG-yA2rXA@mail.gmail.com> <CAKV%2BxLAkfiQCLXfgZOtQGUXOW8gYN7sjOD5uWezv-N%2BTBjybMQ@mail.gmail.com> <20171111213759.I72828@sola.nimnet.asn.au> <CAKV%2BxLDicLze3Dvd2i7HGWJUxCdSLjvhuWWZUJ65pMi%2Bx483=A@mail.gmail.com> <20171115185528.V72828@sola.nimnet.asn.au>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Nov 15, 2017 at 9:25 AM, Ian Smith <smithi@nimnet.asn.au> wrote:
> On Mon, 13 Nov 2017 15:17:20 +0100, Cos Chan wrote:
>
> > On Sat, Nov 11, 2017 at 1:42 PM, Ian Smith <smithi@nimnet.asn.au>
> wrote:
> > > On Thu, 9 Nov 2017 14:25:52 +0100, Cos Chan wrote:
>
> I'll have to cut mercilessly, trying to keep to newest issues ..
>
> > > When ipfw is running, issuing this will show you the addresses
> blocked:
> > >
> > > # ipfw table port22 list
> >
> > until now it seems working on list updating. but I am not sure if it is
> > really working fine.
> >
> > here is one strange record:
> >
> > $ sudo blacklistctl dump -b | grep 1662
> > 193.201.224.218/32:22 OK 1662/1 2017/11/13 00:31:04
> >
> > This IP was blocked in ipfw from last week. while I checked it last week
> > Friday it was 800+/1 in blacklist and until today it become 1662.
> >
> > To my knowledge the ipfw should block the connection, the times of
> banned
> > IP should be not increased?
> >
> > I could see more entries with more than 3/1, for example:
> >
> > 89.160.221.132/32:22 OK 18/1 2017/11/13 00:01:21
> > 60.125.42.119/32:22 OK 3/1 2017/11/12 16:13:53
> > 166.62.35.180/32:22 OK 3/1 2017/11/10 06:36:25
> > 202.162.221.51/32:22 OK 6/1 2017/11/10 00:42:14
> > 168.0.114.130/32:22 OK 3/1 2017/11/10 23:40:30
> > 95.145.71.165/32:22 OK 3/1 2017/11/11 07:07:07
> > 123.161.206.210/32:22 OK 3/1 2017/11/12 18:14:00
> > 203.146.208.208/32:22 OK 6/1 2017/11/10 10:16:21
> > 149.56.223.241/32:22 OK 1/1 2017/11/12 06:09:16
> > 121.169.217.98/32:22 OK 9/1 2017/11/12 21:59:57
> > 211.251.237.162/32:22 OK 2/1 2017/11/13 12:08:07
> > 103.99.0.116/32:22 OK 30/1 2017/11/10 14:56:07
> >
> > These records I am not sure if they were not increased after added to
> ipfw
> > list. but the 1662 times one, I am sure it was increased after ipfw had
> the
> > ip in list.
>
> That one does seem strange, though Kurt explained how this can happen.
> Without seeing synchronised logs from blacklistd and blacklistd-helper
> and ipfw, with clearly stated current configuration and switches, it's
> very difficult to know what might be happening ..
>
> > > You might instead try MaxAuthTries 4 .. sshd_config(5) says:
> > >
> > > MaxAuthTries
> > > Specifies the maximum number of authentication attempts
> > > permitted
> > > per connection. Once the number of failures reaches
> half this
> > > value, additional failures are logged. The default is 6.
> > >
> > > Half of 3 as an integer is only 1, but half of 4 is 2. See if it
> helps?
>
> > I didnt change the MaxAuthTries, since I found something interesting
> from
> > the different logs concerning that issue:
> >
> > >From blacklistctl dump:
> >
> > $ sudo blacklistctl dump
> > address/ma:port id nfail last access
> > 78.203.146.34/32:22 0/1 1970/01/01 01:00:00
> > 195.225.116.21/32:22 0/1 1970/01/01 01:00:00
> > 123.31.26.123/32:22 0/1 1970/01/01 01:00:00
> > 112.148.101.13/32:22 0/1 1970/01/01 01:00:00
> > 93.23.6.18/32:22 0/1 1970/01/01 01:00:00
> > 5.102.197.124/32:22 0/1 1970/01/01 01:00:00
> > 193.154.127.32/32:22 0/1 1970/01/01 01:00:00
> > 113.232.216.41/32:22 0/1 1970/01/01 01:00:00
> >
> > >From sshd log:
> >
> > Nov 10 17:57:41 res sshd[49839]: Invalid user pi from 193.154.127.32
> > Nov 10 17:57:41 res sshd[49840]: Invalid user pi from 193.154.127.32
> > Nov 10 17:57:41 res sshd[49840]: input_userauth_request: invalid user pi
> > [preauth]
> > Nov 10 17:57:41 res sshd[49839]: input_userauth_request: invalid user pi
> > [preauth]
>
> Note the two different PIDs on these, indicating sshd handling two
> separate connections. From above, MaxAuthTries limits the maximum
> number of attempts _per_connection_. So each of these indicate only one
> (or possibly two, as again from above, only those greater than half of
> the maximum (here 3/2 = 1) are supposedly logged by sshd).
>
> I don't know just what sshd reports to blacklistd in what circumstances,
> nor how those are reflected in blacklistd's logging .. Kurt likely does.
>
> > Nov 11 03:50:47 res sshd[57896]: Invalid user support from 123.31.26.123
> > Nov 11 03:50:47 res sshd[57896]: input_userauth_request: invalid user
> > support [preauth]
> > Nov 11 03:50:47 res sshd[57896]: error: Received disconnect from
> > 123.31.26.123 port 55811:3: com.jcraft.jsch.JSchException: Auth fail
> > [preauth]
>
> That's on one PID, ie one connection. Less than three failures on it.
>
> > Nov 11 03:50:49 res sshd[57898]: Invalid user admin from 123.31.26.123
> > Nov 11 03:50:49 res sshd[57898]: input_userauth_request: invalid user
> admin
> > [preauth]
> > Nov 11 03:50:49 res sshd[57898]: error: Received disconnect from
> > 123.31.26.123 port 57823:3: com.jcraft.jsch.JSchException: Auth fail
> > [preauth]
>
> Ditto.
>
> > Nov 11 03:50:51 res sshd[57900]: Invalid user admin from 123.31.26.123
> > Nov 11 03:50:51 res sshd[57900]: input_userauth_request: invalid user
> admin
> > [preauth]
> > Nov 11 03:50:51 res sshd[57900]: error: Received disconnect from
> > 123.31.26.123 port 59819:3: com.jcraft.jsch.JSchException: Auth fail
> > [preauth]
>
> Another.
>
> > Nov 11 03:50:53 res sshd[57902]: Invalid user ubnt from 123.31.26.123
> > Nov 11 03:50:53 res sshd[57902]: input_userauth_request: invalid user
> ubnt
> > [preauth]
> > Nov 11 03:50:53 res sshd[57902]: error: Received disconnect from
> > 123.31.26.123 port 61795:3: com.jcraft.jsch.JSchException: Auth fail
> > [preauth]
>
> Again.
>
> > Nov 11 03:50:55 res sshd[57904]: Invalid user PlcmSpIp from
> 123.31.26.123
> > Nov 11 03:50:55 res sshd[57904]: input_userauth_request: invalid user
> > PlcmSpIp [preauth]
> > Nov 11 03:50:55 res sshd[57904]: error: Received disconnect from
> > 123.31.26.123 port 61920:3: com.jcraft.jsch.JSchException: Auth fail
> > [preauth]
>
> Again.
>
> > Nov 11 03:50:57 res sshd[57906]: Invalid user admin from 123.31.26.123
> > Nov 11 03:50:57 res sshd[57906]: input_userauth_request: invalid user
> admin
> > [preauth]
> > Nov 11 03:50:57 res sshd[57906]: error: Received disconnect from
> > 123.31.26.123 port 61949:3: com.jcraft.jsch.JSchException: Auth fail
> > [preauth]
>
> And yet another. There's no indication that sshd is - or is supposed to
> be - keeping track of separate connections from the same IP address.
>
I agree that sshd should not keep track the IP, but blacklistd should do.
>
> > I see 2 problems:
> >
> > Problem 1:
> > The IP 193.154.127.32 didn't reach sshd maximum authentication (=3), it
> > tried only 2 times.
>
> Perhaps rather, only once or twice on each of two separate connections?
>
> > But in my opinion it should be recorded to blacklistd as 2/1 instead of
> 0/1.
>
> I gather that it would take 3 failed logins on any _one_ connection to
> report it as _one_ failure to blacklistd.
>
is this reasonable? in case one IP was using thousands connections which
failed once per connection, then it will never be banned by blacklistd
(unless the maxauth of sshd is 1)?
>
> > Problem 2:
> > The IP 123.31.26.123 was trying to use different user name to login more
> > than 3 times. it was also recorded in blacklistd as 0/1.
> >
> > In my opinion the above 2 all should be banned by blacklistd.
>
> Again, no single one of those connections failed 3 times. In other
> words, I don't think this works the way you're expecting.
> > > Earlier you said you'd run it without /etc/ipfw-blacklist.rc existing.
> > > In that case - UNLESS you had either /etc/pf.conf or /etc/ipf.conf
> lying
> > > around from before? it should have failed with 'exit 1' .. though it's
> > > not clear from browsing the code that even that would cause it to
> quit.
> > >
> >
> > No, there are not /etc/pf.conf and /etc/ipf.conf.
>
> So it looks like you maybe just didn't see any failure message at the
> time, likely to stderr, and you weren't logging blacxklistd at that
> time. It would be good to know what happens if blacklistd-helper fails.
>
I did it again. to make a little clear to Kurt, I will explain the problem
and configurations.
here is the log to show "problem n-1/n", the blacklistd could not never
reach maximum nfail and ban the IP.
To produce the problem, I only need to remove /etc/ipfw-blacklist.rc and
there is no /etc/pf.conf or /etc/ipf.conf either.
I run blacklistd by "service blacklistd start", here is the rc.conf:
blacklistd_enable="YES"
blacklistd_flags="-r"
here is sshd_config:
AuthenticationMethods publickey
MaxAuthTries 4
UseBlacklist yes
here is ipfw in rc.conf:
#ipfw
firewall_enable="YES"
firewall_quiet="YES"
firewall_type="open"
firewall_script="/usr/local/etc/firewall.rules"
firewall_logging="YES"
modification to /usr/libexec/blacklistd-helper is to add one line for log:
# $7 id
echo "`date` $0 run $@" >>/var/log/blacklistd-helper.log
pf=
the ipfw list:
$ sudo ipfw list
00100 allow ip from any to any via lo0
00200 deny ip from any to 127.0.0.0/8
00300 deny ip from 127.0.0.0/8 to any
00400 deny ip from any to ::1
00500 deny ip from ::1 to any
00600 allow ipv6-icmp from :: to ff02::/16
00700 allow ipv6-icmp from fe80::/10 to fe80::/10
00800 allow ipv6-icmp from fe80::/10 to ff02::/16
00900 allow ipv6-icmp from any to any ip6 icmp6types 1
01000 allow ipv6-icmp from any to any ip6 icmp6types 2,135,136
02022 deny log tcp from table(port22) to any dst-port 22
65000 allow ip from any to any
65535 deny ip from any to any
the rule "02022 deny log tcp from table(port22) to any dst-port 22" was
added by myself to have log from ipfw
syslog.conf:
!blacklistd
*.* /var/log/blacklistd.log
I did sshd MaxAuthTries =3 and 4.
maxauth =3, the blacklistd-helper.log:
--start sshd maxauth=3; blacklist nfail=2, disable=*; ipfw enabled, removed
/etc/ipfw-blacklist.rc--
Wed Nov 15 09:53:40 CET 2017 /usr/libexec/blacklistd-helper run flush
blacklistd
Wed Nov 15 09:55:47 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 59.120.35.74 32 22
Wed Nov 15 09:55:47 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 59.120.35.74 32 22
Wed Nov 15 09:59:21 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 193.201.224.218 32 22
Wed Nov 15 09:59:21 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 193.201.224.218 32 22
Wed Nov 15 09:59:25 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 193.201.224.218 32 22
Wed Nov 15 09:59:26 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 193.201.224.218 32 22
Wed Nov 15 09:59:26 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 193.201.224.218 32 22
....
blacklistd.log:
Nov 15 09:55:09 res blacklistd[18044]: Connected to blacklist server
Nov 15 10:14:14 res blacklistd[18045]: message too short 144
Nov 15 10:14:14 res blacklistd[18045]: no message (Connection refused)
Nov 15 10:17:33 res blacklistd[18045]: message too short 144
Nov 15 10:17:33 res blacklistd[18045]: no message (Connection refused)
Nov 15 10:17:34 res blacklistd[18045]: message too short 144
Nov 15 10:17:34 res blacklistd[18045]: no message (Connection refused)
Nov 15 10:17:44 res blacklistd[18045]: message too short 144
Nov 15 10:17:44 res blacklistd[18045]: no message (Connection refused)
Nov 15 10:17:54 res blacklistd[18045]: message too short 144
Nov 15 10:17:54 res blacklistd[18045]: no message (Connection refused)
Nov 15 10:18:20 res blacklistd[18045]: message too short 144
Nov 15 10:18:20 res blacklistd[18045]: no message (Connection refused)
Nov 15 10:18:30 res blacklistd[18045]: message too short 144
Nov 15 10:18:30 res blacklistd[18045]: no message (Connection refused)
dump:
$ sudo blacklistctl dump
address/ma:port id nfail last access
59.120.35.74/32:22 1/2 2017/11/15 09:55:47
89.135.123.209/32:22 1/2 2017/11/15 10:32:53
193.201.224.218/32:22 1/2 2017/11/15 09:59:20
118.123.245.239/32:22 1/2 2017/11/15 10:15:10
$ sudo blacklistctl dump -b
address/ma:port id nfail last access
maxauth=4, the logs
$ cat blacklistd-helper.log
--start sshd maxauth=4; blacklist nfail=2, disable=*; ipfw enabled, removed
/etc/ipfw-blacklist.rc--
Wed Nov 15 10:53:39 CET 2017 /usr/libexec/blacklistd-helper run flush
blacklistd
Wed Nov 15 10:56:45 CET 2017 /usr/libexec/blacklistd-helper run flush
blacklistd
Wed Nov 15 10:58:44 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 41.73.194.139 32 22
Wed Nov 15 10:58:44 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 41.73.194.139 32 22
Wed Nov 15 11:01:04 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 83.246.164.83 32 22
Wed Nov 15 11:01:04 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 83.246.164.83 32 22
$ tail blacklistd.log
Nov 15 10:53:39 res blacklistd[21125]: Connected to blacklist server
Nov 15 10:53:53 res blacklistd[21161]: Connected to blacklist server
Nov 15 10:56:45 res blacklistd[21264]: Connected to blacklist server
Nov 15 10:56:57 res blacklistd[21312]: Connected to blacklist server
$ sudo blacklistctl dump
address/ma:port id nfail last access
41.73.194.139/32:22 1/2 2017/11/15 10:58:44
83.246.164.83/32:22 1/2 2017/11/15 11:01:04
$ sudo blacklistctl dump -b
address/ma:port id nfail last access
>
> Moving on ..
>
> cheers, Ian
>
--
with kind regards
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAKV%2BxLC=ABe2i3TN8bo4XaVg3KfUbKsS96=6iyVDnsmWw-e8ag>