From owner-freebsd-questions@freebsd.org Mon Nov 13 14:17:24 2017 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 00A68DBB870 for ; Mon, 13 Nov 2017 14:17:24 +0000 (UTC) (envelope-from rosettas@gmail.com) Received: from mail-wr0-x235.google.com (mail-wr0-x235.google.com [IPv6:2a00:1450:400c:c0c::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5B6846FC6B; Mon, 13 Nov 2017 14:17:23 +0000 (UTC) (envelope-from rosettas@gmail.com) Received: by mail-wr0-x235.google.com with SMTP id l22so14536783wrc.11; Mon, 13 Nov 2017 06:17:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=63TfCkuX1V0ZHH+lv9OZ/MTPcJFII8jYdQZX5nOMW7w=; b=Z7pzBBB2695FBz6Ut1XePkRZyH/w+CB0n5hUPGWbfP7UDsblZnoWhgIViSTZkevxMh mrGTs43/v4ZbCCE33/OgtXHFck7+5TjhNeMiH47uPkW+MnZruZC1z2Tmqff1i3EaXVxH OR9De7iHTSs9oMx1v0PWmXnzxp6p23kFMav/9tmNssFNLmXyWbbXGY6hzssMpq9jKXrN vndgFyUNaS0bcTNwHpu9FmWBXAyenwVv3RdWP7q0oSlBpPVmaYAshsHeQYqyXe7C9iyr hUvS2AkjS85jgeP/2iPV+NeAtQ1I+26PRFd4V0XUSRw76HoRURemXoHBipvOkEOyTwN5 SHFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=63TfCkuX1V0ZHH+lv9OZ/MTPcJFII8jYdQZX5nOMW7w=; b=U5oKz2fnTzCn5nE4Eqv2sVe6B96yI0d5S7mE/xR469cgTsmYEyZRA5rFmHz4hRvDgm sTaOopqzCOIzrk/WLPnib++Iu9EYckcpT0XxoVdpJLkwwUeLRuXb9wKtSZxqTlwT9Xz6 2ifF3B9Ol257mopHe4HOp08ydtn2W3V+9yat2psvHDtdzCRvZJ7wbHdQkYkxnaXLcNg3 HoEnvLHFlEtdQMDIMxrXztPgGoN8JJiinfgWCKpAwr/A0Mz97w8XoErpbRg9A2rv5RaO pvqSrd46QnbXochyJbZtZroCeLmbJfNbHXmMyS9NrZJqVfKMM7kRTn9AAnQOZ3XAJAWH HKtQ== X-Gm-Message-State: AJaThX6K6ID7+8gTgsFIALL1NHmQsaqIBHvTzN4IBZKOR5N6x4ahRZ/Y i27xKNfbB5aj3LGUox1CkhURdc0kBgLF3a1Ugks= X-Google-Smtp-Source: AGs4zMathU9npBKFD7LDWu1QjPxfwyk9+7HFhEQkaeF+7bC3WPAcqZgiLX6MhJjQ1qPv0aLNShJc3JvJCFsYMGWmIXg= X-Received: by 10.223.154.202 with SMTP id a68mr7084966wrc.8.1510582641090; Mon, 13 Nov 2017 06:17:21 -0800 (PST) MIME-Version: 1.0 Received: by 10.28.125.8 with HTTP; Mon, 13 Nov 2017 06:17:20 -0800 (PST) In-Reply-To: <20171111213759.I72828@sola.nimnet.asn.au> References: <20171106235944.U9710@sola.nimnet.asn.au> <20171107033226.M9710@sola.nimnet.asn.au> <20171107162914.G9710@sola.nimnet.asn.au> <20171108012948.A9710@sola.nimnet.asn.au> <20171111213759.I72828@sola.nimnet.asn.au> From: Cos Chan Date: Mon, 13 Nov 2017 15:17:20 +0100 Message-ID: Subject: Re: How to setup IPFW working with blacklistd To: Ian Smith Cc: freebsd-questions , Michael Ross , Kurt Lidl Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Nov 2017 14:17:24 -0000 On Sat, Nov 11, 2017 at 1:42 PM, Ian Smith wrote: > On Thu, 9 Nov 2017 14:25:52 +0100, Cos Chan wrote: > > > Dear All > > > > Thanks Ian's great help, I have solved problem to post banned entries > from > > blacklistd to ipfw. > > Well, we're some of the way there :) We really need Kurt Lidl's eyes on > this to make real progress, and indications are that my and your emails > cc'ing him were still being deferred for some reason - maybe he's away? > > > The original message was received at Tue, 7 Nov 2017 10:12:05 -0500 (EST) > > from mx2.freebsd.org [8.8.178.116] > > > > ----- Transcript of session follows ----- > > ... Deferred: Operation timed out with hydra.pix.net. > > Warning: message still undelivered after 4 hours > > Will keep trying until message is 1 week, 3 days old > > > > To my knowledge the problem is: > > > > I setup sshd+blacklistd without ipfw at first. Then I got problem the > entry > > was never reached nfail number (is it a bug?). > > The first issue was because of a severe deficiency in blacklistd-helper, > in that it doesn't actually check that the chosen firewall is running, > and it then fails to detect commands for that firewall that do not (can > not) succeed as any sort of error! More about that below. > > The second, however, was mainly that you missed that nfail set to '*' > means that the host is NOT to be blocked, no matter how many auth or > other failures that (in this case) sshd reports. > > That also answers another question you had .. "nnn/-1" indicates that > nfail=* ie never to be blocked. These still get accumulated in the > database, but are not applied as ipfw block rule table entries. > > > > so I have to change the nfail to * to get the entry into banned list. > > In combination with other factors - like whether ipfw was running at the > time - that got blacklistd to record reported failures to its database, > but not to execute the 'add' commands to blacklistd-helper, so that > address was not in fact blocked, and subsequent attempts kept trying. > > > But while I setup ipfw, the nfail=* would not activate > blacklistd-helper so > > no entry in blacklist banned list were added to ipfw. > > Yes, nfail=* means NEVER block these failed addreses. blacklistd.conf(5) > > > I have modify the blacklistd nfail to 2, sshd MaxAuthTries to 3. The > > blacklist entries working fine. > > With ipfw running, yes :) But it should have failed - noisily - sooner. > > When ipfw is running, issuing this will show you the addresses blocked: > > # ipfw table port22 list > until now it seems working on list updating. but I am not sure if it is really working fine. here is one strange record: $ sudo blacklistctl dump -b | grep 1662 193.201.224.218/32:22 OK 1662/1 2017/11/13 00:31:04 This IP was blocked in ipfw from last week. while I checked it last week Friday it was 800+/1 in blacklist and until today it become 1662. To my knowledge the ipfw should block the connection, the times of banned IP should be not increased? I could see more entries with more than 3/1, for example: 89.160.221.132/32:22 OK 18/1 2017/11/13 00:01:21 60.125.42.119/32:22 OK 3/1 2017/11/12 16:13:53 166.62.35.180/32:22 OK 3/1 2017/11/10 06:36:25 202.162.221.51/32:22 OK 6/1 2017/11/10 00:42:14 168.0.114.130/32:22 OK 3/1 2017/11/10 23:40:30 95.145.71.165/32:22 OK 3/1 2017/11/11 07:07:07 123.161.206.210/32:22 OK 3/1 2017/11/12 18:14:00 203.146.208.208/32:22 OK 6/1 2017/11/10 10:16:21 149.56.223.241/32:22 OK 1/1 2017/11/12 06:09:16 121.169.217.98/32:22 OK 9/1 2017/11/12 21:59:57 211.251.237.162/32:22 OK 2/1 2017/11/13 12:08:07 103.99.0.116/32:22 OK 30/1 2017/11/10 14:56:07 These records I am not sure if they were not increased after added to ipfw list. but the 1662 times one, I am sure it was increased after ipfw had the ip in list. > > BUT I found another problem. > > > > The output of blacklist dump is strange: > > > > $ sudo blacklistctl dump > > address/ma:port id nfail last access > > 96.227.104.132/32:22 0/2 1970/01/01 01:00:00 > > 89.245.78.187/32:22 0/2 1970/01/01 01:00:00 > > 116.193.162.203/32:22 1/2 2017/11/09 11:48:05 > > > > Since the blacklistd accepts instruction from sshd. how could be 0/2 > > entries presented there? I am sure my successful logins were not added > to > > blacklistd. > > 1970/01/01 01:00:00 is just the UNIX '0' timestamp, in this case plus > one hour (your TZ offset). It here means 'no previous entry'. Not sure > about that 0/2, but there are several different codes returned by sshd > including success, failed auth and 'abusive behaviour' .. I'm not sure > which ones your reports (including in off-list mail) indicate. > > As for the mysterious 'n-1' behaviour you mentioned offlist for nfail, > in /usr/src/contrib/blacklist/bin/blacklistd.c there's this: > > switch (bi->bi_type) { > case BL_ABUSE: > /* > * If the application has signaled abusive behavior, > * set the number of fails to be one less than the > * configured limit. Fallthrough to the normal BL_ADD > * processing, which will increment the failure count > * to the threshhold, and block the abusive address. > */ > if (c.c_nfail != -1) > dbi.count = c.c_nfail - 1; > /*FALLTHROUGH*/ > case BL_ADD: > dbi.count++; > dbi.last = ts.tv_sec; > if (dbi.id[0]) { > /* > * We should not be getting this since the rule > * should have blocked the address. A possible > * explanation is that someone removed that rule, > * and another would be that we got another attempt > * before we added the rule. In anycase, we remove > * and re-add the rule because we don't want to add > * it twice, because then we'd lose track of it. > */ > (*lfun)(LOG_DEBUG, "rule exists %s", dbi.id); > (void)run_change("rem", &c, dbi.id, 0); > dbi.id[0] = '\0'; > } > if (c.c_nfail != -1 && dbi.count >= c.c_nfail) { > int res = run_change("add", &c, dbi.id, sizeof( > dbi.id)); > if (res == -1) > goto out; > sockaddr_snprintf(rbuf, sizeof(rbuf), "%a", > (void *)&rss); > (*lfun)(LOG_INFO, > "blocked %s/%d:%d for %d seconds", > rbuf, c.c_lmask, c.c_port, c.c_duration); > > } > break; > > But if the 'add' command via blacklistd-helper fails, it will never add > the 1 .. I'm not certain about this, but it could explain what you see, > although I can't discern whether sshd is reporting BL_ADD or BL_ABUSE. > > You might instead try MaxAuthTries 4 .. sshd_config(5) says: > > MaxAuthTries > Specifies the maximum number of authentication attempts > permitted > per connection. Once the number of failures reaches half this > value, additional failures are logged. The default is 6. > > Half of 3 as an integer is only 1, but half of 4 is 2. See if it helps? > I didnt change the MaxAuthTries, since I found something interesting from the different logs concerning that issue: >From blacklistctl dump: $ sudo blacklistctl dump address/ma:port id nfail last access 78.203.146.34/32:22 0/1 1970/01/01 01:00:00 195.225.116.21/32:22 0/1 1970/01/01 01:00:00 123.31.26.123/32:22 0/1 1970/01/01 01:00:00 112.148.101.13/32:22 0/1 1970/01/01 01:00:00 93.23.6.18/32:22 0/1 1970/01/01 01:00:00 5.102.197.124/32:22 0/1 1970/01/01 01:00:00 193.154.127.32/32:22 0/1 1970/01/01 01:00:00 113.232.216.41/32:22 0/1 1970/01/01 01:00:00 >From sshd log: Nov 10 17:57:41 res sshd[49839]: Invalid user pi from 193.154.127.32 Nov 10 17:57:41 res sshd[49840]: Invalid user pi from 193.154.127.32 Nov 10 17:57:41 res sshd[49840]: input_userauth_request: invalid user pi [preauth] Nov 10 17:57:41 res sshd[49839]: input_userauth_request: invalid user pi [preauth] ... Nov 11 03:50:47 res sshd[57896]: Invalid user support from 123.31.26.123 Nov 11 03:50:47 res sshd[57896]: input_userauth_request: invalid user support [preauth] Nov 11 03:50:47 res sshd[57896]: error: Received disconnect from 123.31.26.123 port 55811:3: com.jcraft.jsch.JSchException: Auth fail [preauth] Nov 11 03:50:49 res sshd[57898]: Invalid user admin from 123.31.26.123 Nov 11 03:50:49 res sshd[57898]: input_userauth_request: invalid user admin [preauth] Nov 11 03:50:49 res sshd[57898]: error: Received disconnect from 123.31.26.123 port 57823:3: com.jcraft.jsch.JSchException: Auth fail [preauth] Nov 11 03:50:51 res sshd[57900]: Invalid user admin from 123.31.26.123 Nov 11 03:50:51 res sshd[57900]: input_userauth_request: invalid user admin [preauth] Nov 11 03:50:51 res sshd[57900]: error: Received disconnect from 123.31.26.123 port 59819:3: com.jcraft.jsch.JSchException: Auth fail [preauth] Nov 11 03:50:53 res sshd[57902]: Invalid user ubnt from 123.31.26.123 Nov 11 03:50:53 res sshd[57902]: input_userauth_request: invalid user ubnt [preauth] Nov 11 03:50:53 res sshd[57902]: error: Received disconnect from 123.31.26.123 port 61795:3: com.jcraft.jsch.JSchException: Auth fail [preauth] Nov 11 03:50:55 res sshd[57904]: Invalid user PlcmSpIp from 123.31.26.123 Nov 11 03:50:55 res sshd[57904]: input_userauth_request: invalid user PlcmSpIp [preauth] Nov 11 03:50:55 res sshd[57904]: error: Received disconnect from 123.31.26.123 port 61920:3: com.jcraft.jsch.JSchException: Auth fail [preauth] Nov 11 03:50:57 res sshd[57906]: Invalid user admin from 123.31.26.123 Nov 11 03:50:57 res sshd[57906]: input_userauth_request: invalid user admin [preauth] Nov 11 03:50:57 res sshd[57906]: error: Received disconnect from 123.31.26.123 port 61949:3: com.jcraft.jsch.JSchException: Auth fail [preauth] I see 2 problems: Problem 1: The IP 193.154.127.32 didn't reach sshd maximum authentication (=3), it tried only 2 times. But in my opinion it should be recorded to blacklistd as 2/1 instead of 0/1. Problem 2: The IP 123.31.26.123 was trying to use different user name to login more than 3 times. it was also recorded in blacklistd as 0/1. In my opinion the above 2 all should be banned by blacklistd. > > > I am trying to find out the reason from log but I dont know how to see > > blacklistd log. man page said that is to syslogd but what the facility > it > > is? or some other ways to get out log? > > Not sure of the facility but when using the -v switch, as you have been, > logging goes to stderr instead of syslog. Without -v you should see it > logging to /var/log/messages. If not, try adding to /etc/syslog.conf: > > !blacklistd > *.* /var/log/myblacklistd.log > > then '# touch /var/log/myblacklistd.log && service syslogd restart' > Unfortunately I started the logging later than Nov 11 03:50:57, so I didnt get the log of "0/1" records yet. > > Ok, problems with blacklistd-helper; the first bit verbatim, tabs lost: > > #!/bin/sh > #echo "run $@" 1>&2 > #set -x > # $1 command > # $2 rulename > # $3 protocol > # $4 address > # $5 mask > # $6 port > # $7 id > > pf= > if [ -f "/etc/ipfw-blacklist.rc" ]; then > pf="ipfw" > . /etc/ipfw-blacklist.rc > ipfw_offset=${ipfw_offset:-2000} > fi > > if [ -z "$pf" ]; then > for f in npf pf ipf; do > if [ -f "/etc/$f.conf" ]; then > pf="$f" > break > fi > done > fi > > if [ -z "$pf" ]; then > echo "$0: Unsupported packet filter" 1>&2 > exit 1 > fi > > Earlier you said you'd run it without /etc/ipfw-blacklist.rc existing. > In that case - UNLESS you had either /etc/pf.conf or /etc/ipf.conf lying > around from before? it should have failed with 'exit 1' .. though it's > not clear from browsing the code that even that would cause it to quit. > No, there are not /etc/pf.conf and /etc/ipf.conf. > > So once /etc/ipfw-blacklist.rc exists, that's a flag indicating you > intend using ipfw, however there's NO check that ipfw is running .. > > Then - ignoring the pf) and ipf) sections - though I suspect they'd have > the same issue unless really running - here's the ipfw add bit, no tabs: > > add) > case "$pf" in > [..] > ipfw) > # use $ipfw_offset+$port for rule number > rule=$(($ipfw_offset + $6)) > tname="port$6" > /sbin/ipfw table $tname create type addr 2>/dev/null > > Unless ipfw is running, enabled, that will fail - silently. > > /sbin/ipfw -q table $tname add "$addr/$mask" > > Ditto, perhaps with a message to stderr - that's simply ignored. > > # if rule number $rule does not already exist, create it > /sbin/ipfw show $rule >/dev/null 2>&1 || \ > /sbin/ipfw add $rule drop $3 from \ > table"("$tname")" to any dst-port $6 >/dev/null && > \ > echo OK > ;; > > When both of these ipfw commands also fail, it'll only fail to echo OK. > > Not that failing to echo OK seems to matter to the calling code, but > the OK is kept as 'id' which is passed to the rem)ove code, but is > unused except by the npf firewall .. 'netbsd packet filter' I guess. > > I can certainly suggest patches for at least the ipfw sections - and > really, if the introductory code checks ipfw is working that should be > enough - but I'm unsure whether 'exit 1' after an error message is all > that's needed to get blacklistd to whinge loudly and refuse to continue? > > This should be turned into a PR via bugzilla, but since I'm not running > 11.x here, I can only really contribute if you do so and add me as a cc. > Sorry I dont know how to describe the problem in bugzilla since I dont really understand what you said. I have to learn more about the script :) > > Please try to avoid top-posting on replies, thanks. Sure, I will. > > cheers, Ian > -- with kind regards