From owner-freebsd-questions@freebsd.org Sat Nov 11 12:43:12 2017 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BAEA4E6A346 for ; Sat, 11 Nov 2017 12:43:12 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AC4548019F; Sat, 11 Nov 2017 12:43:10 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id vABCgwpH067824; Sat, 11 Nov 2017 23:42:58 +1100 (EST) (envelope-from smithi@nimnet.asn.au) Date: Sat, 11 Nov 2017 23:42:58 +1100 (EST) From: Ian Smith To: Cos Chan cc: freebsd-questions , Michael Ross , Kurt Lidl Subject: Re: How to setup IPFW working with blacklistd In-Reply-To: Message-ID: <20171111213759.I72828@sola.nimnet.asn.au> References: <20171106235944.U9710@sola.nimnet.asn.au> <20171107033226.M9710@sola.nimnet.asn.au> <20171107162914.G9710@sola.nimnet.asn.au> <20171108012948.A9710@sola.nimnet.asn.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Nov 2017 12:43:12 -0000 On Thu, 9 Nov 2017 14:25:52 +0100, Cos Chan wrote: > Dear All > > Thanks Ian's great help, I have solved problem to post banned entries from > blacklistd to ipfw. Well, we're some of the way there :) We really need Kurt Lidl's eyes on this to make real progress, and indications are that my and your emails cc'ing him were still being deferred for some reason - maybe he's away? > The original message was received at Tue, 7 Nov 2017 10:12:05 -0500 (EST) > from mx2.freebsd.org [8.8.178.116] > > ----- Transcript of session follows ----- > ... Deferred: Operation timed out with hydra.pix.net. > Warning: message still undelivered after 4 hours > Will keep trying until message is 1 week, 3 days old > To my knowledge the problem is: > > I setup sshd+blacklistd without ipfw at first. Then I got problem the entry > was never reached nfail number (is it a bug?). The first issue was because of a severe deficiency in blacklistd-helper, in that it doesn't actually check that the chosen firewall is running, and it then fails to detect commands for that firewall that do not (can not) succeed as any sort of error! More about that below. The second, however, was mainly that you missed that nfail set to '*' means that the host is NOT to be blocked, no matter how many auth or other failures that (in this case) sshd reports. That also answers another question you had .. "nnn/-1" indicates that nfail=* ie never to be blocked. These still get accumulated in the database, but are not applied as ipfw block rule table entries. > so I have to change the nfail to * to get the entry into banned list. In combination with other factors - like whether ipfw was running at the time - that got blacklistd to record reported failures to its database, but not to execute the 'add' commands to blacklistd-helper, so that address was not in fact blocked, and subsequent attempts kept trying. > But while I setup ipfw, the nfail=* would not activate blacklistd-helper so > no entry in blacklist banned list were added to ipfw. Yes, nfail=* means NEVER block these failed addreses. blacklistd.conf(5) > I have modify the blacklistd nfail to 2, sshd MaxAuthTries to 3. The > blacklist entries working fine. With ipfw running, yes :) But it should have failed - noisily - sooner. When ipfw is running, issuing this will show you the addresses blocked: # ipfw table port22 list > BUT I found another problem. > > The output of blacklist dump is strange: > > $ sudo blacklistctl dump > address/ma:port id nfail last access > 96.227.104.132/32:22 0/2 1970/01/01 01:00:00 > 89.245.78.187/32:22 0/2 1970/01/01 01:00:00 > 116.193.162.203/32:22 1/2 2017/11/09 11:48:05 > > Since the blacklistd accepts instruction from sshd. how could be 0/2 > entries presented there? I am sure my successful logins were not added to > blacklistd. 1970/01/01 01:00:00 is just the UNIX '0' timestamp, in this case plus one hour (your TZ offset). It here means 'no previous entry'. Not sure about that 0/2, but there are several different codes returned by sshd including success, failed auth and 'abusive behaviour' .. I'm not sure which ones your reports (including in off-list mail) indicate. As for the mysterious 'n-1' behaviour you mentioned offlist for nfail, in /usr/src/contrib/blacklist/bin/blacklistd.c there's this: switch (bi->bi_type) { case BL_ABUSE: /* * If the application has signaled abusive behavior, * set the number of fails to be one less than the * configured limit. Fallthrough to the normal BL_ADD * processing, which will increment the failure count * to the threshhold, and block the abusive address. */ if (c.c_nfail != -1) dbi.count = c.c_nfail - 1; /*FALLTHROUGH*/ case BL_ADD: dbi.count++; dbi.last = ts.tv_sec; if (dbi.id[0]) { /* * We should not be getting this since the rule * should have blocked the address. A possible * explanation is that someone removed that rule, * and another would be that we got another attempt * before we added the rule. In anycase, we remove * and re-add the rule because we don't want to add * it twice, because then we'd lose track of it. */ (*lfun)(LOG_DEBUG, "rule exists %s", dbi.id); (void)run_change("rem", &c, dbi.id, 0); dbi.id[0] = '\0'; } if (c.c_nfail != -1 && dbi.count >= c.c_nfail) { int res = run_change("add", &c, dbi.id, sizeof(dbi.id)); if (res == -1) goto out; sockaddr_snprintf(rbuf, sizeof(rbuf), "%a", (void *)&rss); (*lfun)(LOG_INFO, "blocked %s/%d:%d for %d seconds", rbuf, c.c_lmask, c.c_port, c.c_duration); } break; But if the 'add' command via blacklistd-helper fails, it will never add the 1 .. I'm not certain about this, but it could explain what you see, although I can't discern whether sshd is reporting BL_ADD or BL_ABUSE. You might instead try MaxAuthTries 4 .. sshd_config(5) says: MaxAuthTries Specifies the maximum number of authentication attempts permitted per connection. Once the number of failures reaches half this value, additional failures are logged. The default is 6. Half of 3 as an integer is only 1, but half of 4 is 2. See if it helps? > I am trying to find out the reason from log but I dont know how to see > blacklistd log. man page said that is to syslogd but what the facility it > is? or some other ways to get out log? Not sure of the facility but when using the -v switch, as you have been, logging goes to stderr instead of syslog. Without -v you should see it logging to /var/log/messages. If not, try adding to /etc/syslog.conf: !blacklistd *.* /var/log/myblacklistd.log then '# touch /var/log/myblacklistd.log && service syslogd restart' Ok, problems with blacklistd-helper; the first bit verbatim, tabs lost: #!/bin/sh #echo "run $@" 1>&2 #set -x # $1 command # $2 rulename # $3 protocol # $4 address # $5 mask # $6 port # $7 id pf= if [ -f "/etc/ipfw-blacklist.rc" ]; then pf="ipfw" . /etc/ipfw-blacklist.rc ipfw_offset=${ipfw_offset:-2000} fi if [ -z "$pf" ]; then for f in npf pf ipf; do if [ -f "/etc/$f.conf" ]; then pf="$f" break fi done fi if [ -z "$pf" ]; then echo "$0: Unsupported packet filter" 1>&2 exit 1 fi Earlier you said you'd run it without /etc/ipfw-blacklist.rc existing. In that case - UNLESS you had either /etc/pf.conf or /etc/ipf.conf lying around from before? it should have failed with 'exit 1' .. though it's not clear from browsing the code that even that would cause it to quit. So once /etc/ipfw-blacklist.rc exists, that's a flag indicating you intend using ipfw, however there's NO check that ipfw is running .. Then - ignoring the pf) and ipf) sections - though I suspect they'd have the same issue unless really running - here's the ipfw add bit, no tabs: add) case "$pf" in [..] ipfw) # use $ipfw_offset+$port for rule number rule=$(($ipfw_offset + $6)) tname="port$6" /sbin/ipfw table $tname create type addr 2>/dev/null Unless ipfw is running, enabled, that will fail - silently. /sbin/ipfw -q table $tname add "$addr/$mask" Ditto, perhaps with a message to stderr - that's simply ignored. # if rule number $rule does not already exist, create it /sbin/ipfw show $rule >/dev/null 2>&1 || \ /sbin/ipfw add $rule drop $3 from \ table"("$tname")" to any dst-port $6 >/dev/null && \ echo OK ;; When both of these ipfw commands also fail, it'll only fail to echo OK. Not that failing to echo OK seems to matter to the calling code, but the OK is kept as 'id' which is passed to the rem)ove code, but is unused except by the npf firewall .. 'netbsd packet filter' I guess. I can certainly suggest patches for at least the ipfw sections - and really, if the introductory code checks ipfw is working that should be enough - but I'm unsure whether 'exit 1' after an error message is all that's needed to get blacklistd to whinge loudly and refuse to continue? This should be turned into a PR via bugzilla, but since I'm not running 11.x here, I can only really contribute if you do so and add me as a cc. Please try to avoid top-posting on replies, thanks. cheers, Ian