From owner-freebsd-questions@freebsd.org Tue Nov 14 08:31:40 2017 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 17737D7FBAA for ; Tue, 14 Nov 2017 08:31:40 +0000 (UTC) (envelope-from rosettas@gmail.com) Received: from mail-wr0-x22a.google.com (mail-wr0-x22a.google.com [IPv6:2a00:1450:400c:c0c::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6E3AC75149; Tue, 14 Nov 2017 08:31:39 +0000 (UTC) (envelope-from rosettas@gmail.com) Received: by mail-wr0-x22a.google.com with SMTP id o88so16708275wrb.6; Tue, 14 Nov 2017 00:31:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=K/2JXUtFF80qGrttxqHsHd6LziuFSQdAOYv736hqLK4=; b=PfPeQeM8IbS7PhP2nv6H+wlCdwsv7oNI29KYoHigUXPwouT5+SzcqUIUtWf4lh3AN/ NflrrMUyp88BLg9/DaOmbQkqLeaO7jXyIFjEb4f0PJaUDGU0ELC/zCO3bTsrSsJviXLD 07HUbnYvk4O8u6a0ByirSntthQy1lYbklf7U2K/we2pOK6crvS0dpL1alqIw63al/MA7 eQ2E3cVTNdSv8qGab9u1ouI8xLgv+ZDa5xL0JQaKH3bMwEPcapxTklTK6t8yXTZySNKc Z9u025witGDZM0SunmNtyqHEnNFVfyqVT/plIUF0mkI65xqrL558ZYP69NGVazIf+6fW ZkAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=K/2JXUtFF80qGrttxqHsHd6LziuFSQdAOYv736hqLK4=; b=oKrPRb2CV7+iouub1WCamWLxzntbFziNJ4EajASBU37Myh8ZrKptajpmk9wmnzleV2 y5D1LGmjkZmxBT1gLyR85XLUYakmmi1VoBcbO8i35k8Rvs0Hnus+alWm5UbdwaPYXbaC w2AHOpn0bofD7GtsL67OvxrQI5dWkYZsga5i1E4P3PxlxG/K93wnmkEaF5jPUafHulNx VSLPRqqnOajRwweUd8sKQH0cquOKE3zKP8lOi3HWTbP2hhwHSVOTi4DTKiPEQKnOsRC7 iuPPi2jBl7Znt/82zCHSWejiN/FwV5o48UTLyuS0z19ypNjcnBMlob7hznhYIz3FqJoc L/5g== X-Gm-Message-State: AJaThX5nl52kVv82kjonz487VlaxOR5GvdHJk4TbLH5tlJs77/f9iRfn /jVh2cjMtHexOmapXCWsclk3WYSniIL/dmKOhIzywm0l X-Google-Smtp-Source: AGs4zMZjoz8rjOu+uedoIkPV9wQ+yvDoDB07aJyjDIAK4KTK8PlZrnBjUG5sF8Zx79llahWnam0H5RJAGMiGYKO3Ysw= X-Received: by 10.223.132.129 with SMTP id 1mr8769930wrg.136.1510648297048; Tue, 14 Nov 2017 00:31:37 -0800 (PST) MIME-Version: 1.0 Received: by 10.28.125.8 with HTTP; Tue, 14 Nov 2017 00:31:36 -0800 (PST) In-Reply-To: References: <20171106235944.U9710@sola.nimnet.asn.au> <20171107033226.M9710@sola.nimnet.asn.au> <20171107162914.G9710@sola.nimnet.asn.au> <20171108012948.A9710@sola.nimnet.asn.au> <20171111213759.I72828@sola.nimnet.asn.au> From: Cos Chan Date: Tue, 14 Nov 2017 09:31:36 +0100 Message-ID: Subject: Re: How to setup IPFW working with blacklistd To: Ian Smith Cc: freebsd-questions , Michael Ross , Kurt Lidl Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Nov 2017 08:31:40 -0000 On Mon, Nov 13, 2017 at 3:17 PM, Cos Chan wrote: > > > On Sat, Nov 11, 2017 at 1:42 PM, Ian Smith wrote: > >> On Thu, 9 Nov 2017 14:25:52 +0100, Cos Chan wrote: >> >> > Dear All >> > >> > Thanks Ian's great help, I have solved problem to post banned entries >> from >> > blacklistd to ipfw. >> >> Well, we're some of the way there :) We really need Kurt Lidl's eyes on >> this to make real progress, and indications are that my and your emails >> cc'ing him were still being deferred for some reason - maybe he's away? >> >> > The original message was received at Tue, 7 Nov 2017 10:12:05 -0500 >> (EST) >> > from mx2.freebsd.org [8.8.178.116] >> > >> > ----- Transcript of session follows ----- >> > ... Deferred: Operation timed out with hydra.pix.net. >> > Warning: message still undelivered after 4 hours >> > Will keep trying until message is 1 week, 3 days old >> >> >> > To my knowledge the problem is: >> > >> > I setup sshd+blacklistd without ipfw at first. Then I got problem the >> entry >> > was never reached nfail number (is it a bug?). >> >> The first issue was because of a severe deficiency in blacklistd-helper, >> in that it doesn't actually check that the chosen firewall is running, >> and it then fails to detect commands for that firewall that do not (can >> not) succeed as any sort of error! More about that below. >> >> The second, however, was mainly that you missed that nfail set to '*' >> means that the host is NOT to be blocked, no matter how many auth or >> other failures that (in this case) sshd reports. >> >> That also answers another question you had .. "nnn/-1" indicates that >> nfail=* ie never to be blocked. These still get accumulated in the >> database, but are not applied as ipfw block rule table entries. >> >> >> > so I have to change the nfail to * to get the entry into banned list. >> >> In combination with other factors - like whether ipfw was running at the >> time - that got blacklistd to record reported failures to its database, >> but not to execute the 'add' commands to blacklistd-helper, so that >> address was not in fact blocked, and subsequent attempts kept trying. >> >> > But while I setup ipfw, the nfail=* would not activate >> blacklistd-helper so >> > no entry in blacklist banned list were added to ipfw. >> >> Yes, nfail=* means NEVER block these failed addreses. blacklistd.conf(5) >> >> > I have modify the blacklistd nfail to 2, sshd MaxAuthTries to 3. The >> > blacklist entries working fine. >> >> With ipfw running, yes :) But it should have failed - noisily - sooner. >> >> When ipfw is running, issuing this will show you the addresses blocked: >> >> # ipfw table port22 list >> > > until now it seems working on list updating. but I am not sure if it is > really working fine. > > here is one strange record: > > $ sudo blacklistctl dump -b | grep 1662 > 193.201.224.218/32:22 OK 1662/1 2017/11/13 00:31:04 > > This IP was blocked in ipfw from last week. while I checked it last week > Friday it was 800+/1 in blacklist and until today it become 1662. > > To my knowledge the ipfw should block the connection, the times of banned > IP should be not increased? > > I could see more entries with more than 3/1, for example: > > 89.160.221.132/32:22 OK 18/1 2017/11/13 00:01:21 > 60.125.42.119/32:22 OK 3/1 2017/11/12 16:13:53 > 166.62.35.180/32:22 OK 3/1 2017/11/10 06:36:25 > 202.162.221.51/32:22 OK 6/1 2017/11/10 00:42:14 > 168.0.114.130/32:22 OK 3/1 2017/11/10 23:40:30 > 95.145.71.165/32:22 OK 3/1 2017/11/11 07:07:07 > 123.161.206.210/32:22 OK 3/1 2017/11/12 18:14:00 > 203.146.208.208/32:22 OK 6/1 2017/11/10 10:16:21 > 149.56.223.241/32:22 OK 1/1 2017/11/12 06:09:16 > 121.169.217.98/32:22 OK 9/1 2017/11/12 21:59:57 > 211.251.237.162/32:22 OK 2/1 2017/11/13 12:08:07 > 103.99.0.116/32:22 OK 30/1 2017/11/10 14:56:07 > > These records I am not sure if they were not increased after added to ipfw > list. but the 1662 times one, I am sure it was increased after ipfw had the > ip in list. > add the ipfw rules: $ sudo ipfw list 00100 allow ip from any to any via lo0 00200 deny ip from any to 127.0.0.0/8 00300 deny ip from 127.0.0.0/8 to any 00400 deny ip from any to ::1 00500 deny ip from ::1 to any 00600 allow ipv6-icmp from :: to ff02::/16 00700 allow ipv6-icmp from fe80::/10 to fe80::/10 00800 allow ipv6-icmp from fe80::/10 to ff02::/16 00900 allow ipv6-icmp from any to any ip6 icmp6types 1 01000 allow ipv6-icmp from any to any ip6 icmp6types 2,135,136 02022 deny tcp from table(port22) to any dst-port 22 65000 allow ip from any to any 65535 deny ip from any to any > > >> > BUT I found another problem. >> > >> > The output of blacklist dump is strange: >> > >> > $ sudo blacklistctl dump >> > address/ma:port id nfail last access >> > 96.227.104.132/32:22 0/2 1970/01/01 01:00:00 >> > 89.245.78.187/32:22 0/2 1970/01/01 01:00:00 >> > 116.193.162.203/32:22 1/2 2017/11/09 11:48:05 >> > >> > Since the blacklistd accepts instruction from sshd. how could be 0/2 >> > entries presented there? I am sure my successful logins were not added >> to >> > blacklistd. >> >> 1970/01/01 01:00:00 is just the UNIX '0' timestamp, in this case plus >> one hour (your TZ offset). It here means 'no previous entry'. Not sure >> about that 0/2, but there are several different codes returned by sshd >> including success, failed auth and 'abusive behaviour' .. I'm not sure >> which ones your reports (including in off-list mail) indicate. >> >> As for the mysterious 'n-1' behaviour you mentioned offlist for nfail, >> in /usr/src/contrib/blacklist/bin/blacklistd.c there's this: >> >> switch (bi->bi_type) { >> case BL_ABUSE: >> /* >> * If the application has signaled abusive behavior, >> * set the number of fails to be one less than the >> * configured limit. Fallthrough to the normal BL_ADD >> * processing, which will increment the failure count >> * to the threshhold, and block the abusive address. >> */ >> if (c.c_nfail != -1) >> dbi.count = c.c_nfail - 1; >> /*FALLTHROUGH*/ >> case BL_ADD: >> dbi.count++; >> dbi.last = ts.tv_sec; >> if (dbi.id[0]) { >> /* >> * We should not be getting this since the rule >> * should have blocked the address. A possible >> * explanation is that someone removed that rule, >> * and another would be that we got another >> attempt >> * before we added the rule. In anycase, we remove >> * and re-add the rule because we don't want to >> add >> * it twice, because then we'd lose track of it. >> */ >> (*lfun)(LOG_DEBUG, "rule exists %s", dbi.id); >> (void)run_change("rem", &c, dbi.id, 0); >> dbi.id[0] = '\0'; >> } >> if (c.c_nfail != -1 && dbi.count >= c.c_nfail) { >> int res = run_change("add", &c, dbi.id, sizeof( >> dbi.id)); >> if (res == -1) >> goto out; >> sockaddr_snprintf(rbuf, sizeof(rbuf), "%a", >> (void *)&rss); >> (*lfun)(LOG_INFO, >> "blocked %s/%d:%d for %d seconds", >> rbuf, c.c_lmask, c.c_port, c.c_duration); >> >> } >> break; >> >> But if the 'add' command via blacklistd-helper fails, it will never add >> the 1 .. I'm not certain about this, but it could explain what you see, >> although I can't discern whether sshd is reporting BL_ADD or BL_ABUSE. >> >> You might instead try MaxAuthTries 4 .. sshd_config(5) says: >> >> MaxAuthTries >> Specifies the maximum number of authentication attempts >> permitted >> per connection. Once the number of failures reaches half >> this >> value, additional failures are logged. The default is 6. >> >> Half of 3 as an integer is only 1, but half of 4 is 2. See if it helps? >> > > I didnt change the MaxAuthTries, since I found something interesting from > the different logs concerning that issue: > > From blacklistctl dump: > > $ sudo blacklistctl dump > address/ma:port id nfail last access > 78.203.146.34/32:22 0/1 1970/01/01 01:00:00 > 195.225.116.21/32:22 0/1 1970/01/01 01:00:00 > 123.31.26.123/32:22 0/1 1970/01/01 01:00:00 > 112.148.101.13/32:22 0/1 1970/01/01 01:00:00 > 93.23.6.18/32:22 0/1 1970/01/01 01:00:00 > 5.102.197.124/32:22 0/1 1970/01/01 01:00:00 > 193.154.127.32/32:22 0/1 1970/01/01 01:00:00 > 113.232.216.41/32:22 0/1 1970/01/01 01:00:00 > > From sshd log: > > Nov 10 17:57:41 res sshd[49839]: Invalid user pi from 193.154.127.32 > Nov 10 17:57:41 res sshd[49840]: Invalid user pi from 193.154.127.32 > Nov 10 17:57:41 res sshd[49840]: input_userauth_request: invalid user pi > [preauth] > Nov 10 17:57:41 res sshd[49839]: input_userauth_request: invalid user pi > [preauth] > ... > Nov 11 03:50:47 res sshd[57896]: Invalid user support from 123.31.26.123 > Nov 11 03:50:47 res sshd[57896]: input_userauth_request: invalid user > support [preauth] > Nov 11 03:50:47 res sshd[57896]: error: Received disconnect from > 123.31.26.123 port 55811:3: com.jcraft.jsch.JSchException: Auth fail > [preauth] > Nov 11 03:50:49 res sshd[57898]: Invalid user admin from 123.31.26.123 > Nov 11 03:50:49 res sshd[57898]: input_userauth_request: invalid user > admin [preauth] > Nov 11 03:50:49 res sshd[57898]: error: Received disconnect from > 123.31.26.123 port 57823:3: com.jcraft.jsch.JSchException: Auth fail > [preauth] > Nov 11 03:50:51 res sshd[57900]: Invalid user admin from 123.31.26.123 > Nov 11 03:50:51 res sshd[57900]: input_userauth_request: invalid user > admin [preauth] > Nov 11 03:50:51 res sshd[57900]: error: Received disconnect from > 123.31.26.123 port 59819:3: com.jcraft.jsch.JSchException: Auth fail > [preauth] > Nov 11 03:50:53 res sshd[57902]: Invalid user ubnt from 123.31.26.123 > Nov 11 03:50:53 res sshd[57902]: input_userauth_request: invalid user ubnt > [preauth] > Nov 11 03:50:53 res sshd[57902]: error: Received disconnect from > 123.31.26.123 port 61795:3: com.jcraft.jsch.JSchException: Auth fail > [preauth] > Nov 11 03:50:55 res sshd[57904]: Invalid user PlcmSpIp from 123.31.26.123 > Nov 11 03:50:55 res sshd[57904]: input_userauth_request: invalid user > PlcmSpIp [preauth] > Nov 11 03:50:55 res sshd[57904]: error: Received disconnect from > 123.31.26.123 port 61920:3: com.jcraft.jsch.JSchException: Auth fail > [preauth] > Nov 11 03:50:57 res sshd[57906]: Invalid user admin from 123.31.26.123 > Nov 11 03:50:57 res sshd[57906]: input_userauth_request: invalid user > admin [preauth] > Nov 11 03:50:57 res sshd[57906]: error: Received disconnect from > 123.31.26.123 port 61949:3: com.jcraft.jsch.JSchException: Auth fail > [preauth] > > I see 2 problems: > > Problem 1: > The IP 193.154.127.32 didn't reach sshd maximum authentication (=3), it > tried only 2 times. > But in my opinion it should be recorded to blacklistd as 2/1 instead of > 0/1. > > Problem 2: > The IP 123.31.26.123 was trying to use different user name to login more > than 3 times. it was also recorded in blacklistd as 0/1. > > In my opinion the above 2 all should be banned by blacklistd. > > >> >> > I am trying to find out the reason from log but I dont know how to see >> > blacklistd log. man page said that is to syslogd but what the facility >> it >> > is? or some other ways to get out log? >> >> Not sure of the facility but when using the -v switch, as you have been, >> logging goes to stderr instead of syslog. Without -v you should see it >> logging to /var/log/messages. If not, try adding to /etc/syslog.conf: >> >> !blacklistd >> *.* /var/log/myblacklistd.log >> >> then '# touch /var/log/myblacklistd.log && service syslogd restart' >> > > Unfortunately I started the logging later than Nov 11 03:50:57, so I didnt > get the log of "0/1" records yet. > got the log for one new "0/1" entry: $ sudo blacklistctl dump address/ma:port id nfail last access 24.7.90.146/32:22 0/1 1970/01/01 01:00:00 ... $ sudo cat auth.log | grep 24.7.90.146 Nov 14 02:13:58 res sshd[6212]: Invalid user pi from 24.7.90.146 Nov 14 02:13:58 res sshd[6215]: Invalid user pi from 24.7.90.146 Nov 14 02:13:59 res sshd[6215]: Connection closed by 24.7.90.146 port 34746 [preauth] Nov 14 02:13:59 res sshd[6212]: Connection closed by 24.7.90.146 port 34742 [preauth] $ cat myblacklistd.log | grep 'Nov 14' ... Nov 14 02:09:11 res blacklistd[5590]: blocked 202.51.74.55/32:22 for -1 seconds Nov 14 02:11:06 res blacklistd[5590]: rule exists OK Nov 14 02:11:06 res blacklistd[5590]: blocked 202.51.74.55/32:22 for -1 seconds Nov 14 02:14:43 res blacklistd[5590]: blocked 66.232.147.46/32:22 for -1 seconds Nov 14 02:16:40 res blacklistd[5590]: rule exists OK could not see operation against that IP from blacklistd.log > > >> >> Ok, problems with blacklistd-helper; the first bit verbatim, tabs lost: >> >> #!/bin/sh >> #echo "run $@" 1>&2 >> #set -x >> # $1 command >> # $2 rulename >> # $3 protocol >> # $4 address >> # $5 mask >> # $6 port >> # $7 id >> >> pf= >> if [ -f "/etc/ipfw-blacklist.rc" ]; then >> pf="ipfw" >> . /etc/ipfw-blacklist.rc >> ipfw_offset=${ipfw_offset:-2000} >> fi >> >> if [ -z "$pf" ]; then >> for f in npf pf ipf; do >> if [ -f "/etc/$f.conf" ]; then >> pf="$f" >> break >> fi >> done >> fi >> >> if [ -z "$pf" ]; then >> echo "$0: Unsupported packet filter" 1>&2 >> exit 1 >> fi >> >> Earlier you said you'd run it without /etc/ipfw-blacklist.rc existing. >> In that case - UNLESS you had either /etc/pf.conf or /etc/ipf.conf lying >> around from before? it should have failed with 'exit 1' .. though it's >> not clear from browsing the code that even that would cause it to quit. >> > > No, there are not /etc/pf.conf and /etc/ipf.conf. > > >> >> So once /etc/ipfw-blacklist.rc exists, that's a flag indicating you >> intend using ipfw, however there's NO check that ipfw is running .. >> >> Then - ignoring the pf) and ipf) sections - though I suspect they'd have >> the same issue unless really running - here's the ipfw add bit, no tabs: >> >> add) >> case "$pf" in >> [..] >> ipfw) >> # use $ipfw_offset+$port for rule number >> rule=$(($ipfw_offset + $6)) >> tname="port$6" >> /sbin/ipfw table $tname create type addr 2>/dev/null >> >> Unless ipfw is running, enabled, that will fail - silently. >> >> /sbin/ipfw -q table $tname add "$addr/$mask" >> >> Ditto, perhaps with a message to stderr - that's simply ignored. >> >> # if rule number $rule does not already exist, create it >> /sbin/ipfw show $rule >/dev/null 2>&1 || \ >> /sbin/ipfw add $rule drop $3 from \ >> table"("$tname")" to any dst-port $6 >/dev/null >> && \ >> echo OK >> ;; >> >> When both of these ipfw commands also fail, it'll only fail to echo OK. >> >> Not that failing to echo OK seems to matter to the calling code, but >> the OK is kept as 'id' which is passed to the rem)ove code, but is >> unused except by the npf firewall .. 'netbsd packet filter' I guess. >> >> I can certainly suggest patches for at least the ipfw sections - and >> really, if the introductory code checks ipfw is working that should be >> enough - but I'm unsure whether 'exit 1' after an error message is all >> that's needed to get blacklistd to whinge loudly and refuse to continue? >> >> This should be turned into a PR via bugzilla, but since I'm not running >> 11.x here, I can only really contribute if you do so and add me as a cc. >> > > Sorry I dont know how to describe the problem in bugzilla since I dont > really understand what you said. > I have to learn more about the script :) > > >> >> Please try to avoid top-posting on replies, thanks. > > > Sure, I will. > > >> >> cheers, Ian >> > > > > -- > with kind regards > -- with kind regards