Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 15 Nov 2017 12:30:40 +0100
From:      Cos Chan <rosettas@gmail.com>
To:        Ian Smith <smithi@nimnet.asn.au>
Cc:        freebsd-questions <freebsd-questions@freebsd.org>, Michael Ross <gmx@ross.cx>, Kurt Lidl <lidl@freebsd.org>
Subject:   Re: How to setup IPFW working with blacklistd
Message-ID:  <CAKV%2BxLC=ABe2i3TN8bo4XaVg3KfUbKsS96=6iyVDnsmWw-e8ag@mail.gmail.com>
In-Reply-To: <20171115185528.V72828@sola.nimnet.asn.au>
References:  <mailman.87.1509969603.28633.freebsd-questions@freebsd.org> <20171106235944.U9710@sola.nimnet.asn.au> <CAKV%2BxLCizjt5M%2BmJmTZj-cr=D6rhXRwDjCkE=6Q-VQX73iY%2B4A@mail.gmail.com> <20171107033226.M9710@sola.nimnet.asn.au> <CAKV%2BxLBWgU6zmc7tQNA=0%2B=2aF23C1QfJ2i3q1gKYDttwsCTkg@mail.gmail.com> <20171107162914.G9710@sola.nimnet.asn.au> <CAKV%2BxLDQQcG3bvo1b2nUAu7oOVhdNzDDrPWTVp2qOmkWVV89BQ@mail.gmail.com> <20171108012948.A9710@sola.nimnet.asn.au> <CAKV%2BxLCQ9NE6%2BEg6NvHZuEED8Cf6ZX74unvk9ajfLyG-yA2rXA@mail.gmail.com> <CAKV%2BxLAkfiQCLXfgZOtQGUXOW8gYN7sjOD5uWezv-N%2BTBjybMQ@mail.gmail.com> <20171111213759.I72828@sola.nimnet.asn.au> <CAKV%2BxLDicLze3Dvd2i7HGWJUxCdSLjvhuWWZUJ65pMi%2Bx483=A@mail.gmail.com> <20171115185528.V72828@sola.nimnet.asn.au>

next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Nov 15, 2017 at 9:25 AM, Ian Smith <smithi@nimnet.asn.au> wrote:

> On Mon, 13 Nov 2017 15:17:20 +0100, Cos Chan wrote:
>
>  > On Sat, Nov 11, 2017 at 1:42 PM, Ian Smith <smithi@nimnet.asn.au>
> wrote:
>  > > On Thu, 9 Nov 2017 14:25:52 +0100, Cos Chan wrote:
>
> I'll have to cut mercilessly, trying to keep to newest issues ..
>
>  > > When ipfw is running, issuing this will show you the addresses
> blocked:
>  > >
>  > >  # ipfw table port22 list
>  >
>  > until now it seems working on list updating. but I am not sure if it is
>  > really working fine.
>  >
>  > here is one strange record:
>  >
>  > $ sudo blacklistctl dump -b | grep 1662
>  > 193.201.224.218/32:22   OK      1662/1  2017/11/13 00:31:04
>  >
>  > This IP was blocked in ipfw from last week. while I checked it last week
>  > Friday it was 800+/1 in blacklist and until today it become 1662.
>  >
>  > To my knowledge the ipfw should block the connection, the times of
> banned
>  > IP should be not increased?
>  >
>  > I could see more entries with more than 3/1, for example:
>  >
>  >  89.160.221.132/32:22   OK      18/1    2017/11/13 00:01:21
>  >   60.125.42.119/32:22   OK      3/1     2017/11/12 16:13:53
>  >   166.62.35.180/32:22   OK      3/1     2017/11/10 06:36:25
>  >  202.162.221.51/32:22   OK      6/1     2017/11/10 00:42:14
>  >   168.0.114.130/32:22   OK      3/1     2017/11/10 23:40:30
>  >   95.145.71.165/32:22   OK      3/1     2017/11/11 07:07:07
>  > 123.161.206.210/32:22   OK      3/1     2017/11/12 18:14:00
>  > 203.146.208.208/32:22   OK      6/1     2017/11/10 10:16:21
>  >  149.56.223.241/32:22   OK      1/1     2017/11/12 06:09:16
>  >  121.169.217.98/32:22   OK      9/1     2017/11/12 21:59:57
>  > 211.251.237.162/32:22   OK      2/1     2017/11/13 12:08:07
>  >    103.99.0.116/32:22   OK      30/1    2017/11/10 14:56:07
>  >
>  > These records I am not sure if they were not increased after added to
> ipfw
>  > list. but the 1662 times one, I am sure it was increased after ipfw had
> the
>  > ip in list.
>
> That one does seem strange, though Kurt explained how this can happen.
> Without seeing synchronised logs from blacklistd and blacklistd-helper
> and ipfw, with clearly stated current configuration and switches, it's
> very difficult to know what might be happening ..
>
>  > > You might instead try MaxAuthTries 4 .. sshd_config(5) says:
>  > >
>  > >      MaxAuthTries
>  > >              Specifies the maximum number of authentication attempts
>  > > permitted
>  > >              per connection.  Once the number of failures reaches
> half this
>  > >              value, additional failures are logged.  The default is 6.
>  > >
>  > > Half of 3 as an integer is only 1, but half of 4 is 2.  See if it
> helps?
>
>  > I didnt change the MaxAuthTries, since I found something interesting
> from
>  > the different logs concerning that issue:
>  >
>  > >From blacklistctl dump:
>  >
>  > $ sudo blacklistctl dump
>  >         address/ma:port id      nfail   last access
>  >   78.203.146.34/32:22           0/1     1970/01/01 01:00:00
>  >  195.225.116.21/32:22           0/1     1970/01/01 01:00:00
>  >   123.31.26.123/32:22           0/1     1970/01/01 01:00:00
>  >  112.148.101.13/32:22           0/1     1970/01/01 01:00:00
>  >      93.23.6.18/32:22           0/1     1970/01/01 01:00:00
>  >   5.102.197.124/32:22           0/1     1970/01/01 01:00:00
>  >  193.154.127.32/32:22           0/1     1970/01/01 01:00:00
>  >  113.232.216.41/32:22           0/1     1970/01/01 01:00:00
>  >
>  > >From sshd log:
>  >
>  > Nov 10 17:57:41 res sshd[49839]: Invalid user pi from 193.154.127.32
>  > Nov 10 17:57:41 res sshd[49840]: Invalid user pi from 193.154.127.32
>  > Nov 10 17:57:41 res sshd[49840]: input_userauth_request: invalid user pi
>  > [preauth]
>  > Nov 10 17:57:41 res sshd[49839]: input_userauth_request: invalid user pi
>  > [preauth]
>
> Note the two different PIDs on these, indicating sshd handling two
> separate connections.  From above, MaxAuthTries limits the maximum
> number of attempts _per_connection_.  So each of these indicate only one
> (or possibly two, as again from above, only those greater than half of
> the maximum (here 3/2 = 1) are supposedly logged by sshd).
>
> I don't know just what sshd reports to blacklistd in what circumstances,
> nor how those are reflected in blacklistd's logging .. Kurt likely does.
>
>  > Nov 11 03:50:47 res sshd[57896]: Invalid user support from 123.31.26.123
>  > Nov 11 03:50:47 res sshd[57896]: input_userauth_request: invalid user
>  > support [preauth]
>  > Nov 11 03:50:47 res sshd[57896]: error: Received disconnect from
>  > 123.31.26.123 port 55811:3: com.jcraft.jsch.JSchException: Auth fail
>  > [preauth]
>
> That's on one PID, ie one connection.  Less than three failures on it.
>
>  > Nov 11 03:50:49 res sshd[57898]: Invalid user admin from 123.31.26.123
>  > Nov 11 03:50:49 res sshd[57898]: input_userauth_request: invalid user
> admin
>  > [preauth]
>  > Nov 11 03:50:49 res sshd[57898]: error: Received disconnect from
>  > 123.31.26.123 port 57823:3: com.jcraft.jsch.JSchException: Auth fail
>  > [preauth]
>
> Ditto.
>
>  > Nov 11 03:50:51 res sshd[57900]: Invalid user admin from 123.31.26.123
>  > Nov 11 03:50:51 res sshd[57900]: input_userauth_request: invalid user
> admin
>  > [preauth]
>  > Nov 11 03:50:51 res sshd[57900]: error: Received disconnect from
>  > 123.31.26.123 port 59819:3: com.jcraft.jsch.JSchException: Auth fail
>  > [preauth]
>
> Another.
>
>  > Nov 11 03:50:53 res sshd[57902]: Invalid user ubnt from 123.31.26.123
>  > Nov 11 03:50:53 res sshd[57902]: input_userauth_request: invalid user
> ubnt
>  > [preauth]
>  > Nov 11 03:50:53 res sshd[57902]: error: Received disconnect from
>  > 123.31.26.123 port 61795:3: com.jcraft.jsch.JSchException: Auth fail
>  > [preauth]
>
> Again.
>
>  > Nov 11 03:50:55 res sshd[57904]: Invalid user PlcmSpIp from
> 123.31.26.123
>  > Nov 11 03:50:55 res sshd[57904]: input_userauth_request: invalid user
>  > PlcmSpIp [preauth]
>  > Nov 11 03:50:55 res sshd[57904]: error: Received disconnect from
>  > 123.31.26.123 port 61920:3: com.jcraft.jsch.JSchException: Auth fail
>  > [preauth]
>
> Again.
>
>  > Nov 11 03:50:57 res sshd[57906]: Invalid user admin from 123.31.26.123
>  > Nov 11 03:50:57 res sshd[57906]: input_userauth_request: invalid user
> admin
>  > [preauth]
>  > Nov 11 03:50:57 res sshd[57906]: error: Received disconnect from
>  > 123.31.26.123 port 61949:3: com.jcraft.jsch.JSchException: Auth fail
>  > [preauth]
>
> And yet another.  There's no indication that sshd is - or is supposed to
> be - keeping track of separate connections from the same IP address.
>

I agree that sshd should not keep track the IP, but blacklistd should do.


>
>  > I see 2 problems:
>  >
>  > Problem 1:
>  > The IP 193.154.127.32 didn't reach sshd maximum authentication (=3), it
>  > tried only 2 times.
>
> Perhaps rather, only once or twice on each of two separate connections?
>
>  > But in my opinion it should be recorded to blacklistd as 2/1 instead of
> 0/1.
>
> I gather that it would take 3 failed logins on any _one_ connection to
> report it as _one_ failure to blacklistd.
>

is this reasonable? in case one IP was using thousands connections which
failed once per connection, then it will never be banned by blacklistd
(unless the maxauth of sshd is 1)?


>
>  > Problem 2:
>  > The IP 123.31.26.123 was trying to use different user name to login more
>  > than 3 times. it was also recorded in blacklistd as 0/1.
>  >
>  > In my opinion the above 2 all should be banned by blacklistd.
>
> Again, no single one of those connections failed 3 times.  In other
> words, I don't think this works the way you're expecting.


>  > > Earlier you said you'd run it without /etc/ipfw-blacklist.rc existing.
>  > > In that case - UNLESS you had either /etc/pf.conf or /etc/ipf.conf
> lying
>  > > around from before? it should have failed with 'exit 1' .. though it's
>  > > not clear from browsing the code that even that would cause it to
> quit.
>  > >
>  >
>  > No, there are not /etc/pf.conf and /etc/ipf.conf.
>
> So it looks like you maybe just didn't see any failure message at the
> time, likely to stderr, and you weren't logging blacxklistd at that
> time.  It would be good to know what happens if blacklistd-helper fails.
>

I did it again. to make a little clear to Kurt, I will explain the problem
and configurations.

here is the log to show "problem n-1/n", the blacklistd could not never
reach maximum nfail and ban the IP.

To produce the problem, I only need to remove /etc/ipfw-blacklist.rc and
there is no /etc/pf.conf or /etc/ipf.conf  either.

I run blacklistd by "service blacklistd start", here is the rc.conf:

blacklistd_enable="YES"
blacklistd_flags="-r"

here is sshd_config:

AuthenticationMethods publickey
MaxAuthTries 4
UseBlacklist yes

here is ipfw in rc.conf:
#ipfw
firewall_enable="YES"
firewall_quiet="YES"
firewall_type="open"
firewall_script="/usr/local/etc/firewall.rules"
firewall_logging="YES"

modification to /usr/libexec/blacklistd-helper is to add one line for log:

# $7 id


echo "`date` $0 run $@" >>/var/log/blacklistd-helper.log
pf=

the ipfw list:
$ sudo ipfw list
00100 allow ip from any to any via lo0
00200 deny ip from any to 127.0.0.0/8
00300 deny ip from 127.0.0.0/8 to any
00400 deny ip from any to ::1
00500 deny ip from ::1 to any
00600 allow ipv6-icmp from :: to ff02::/16
00700 allow ipv6-icmp from fe80::/10 to fe80::/10
00800 allow ipv6-icmp from fe80::/10 to ff02::/16
00900 allow ipv6-icmp from any to any ip6 icmp6types 1
01000 allow ipv6-icmp from any to any ip6 icmp6types 2,135,136
02022 deny log tcp from table(port22) to any dst-port 22
65000 allow ip from any to any
65535 deny ip from any to any

the rule "02022 deny log tcp from table(port22) to any dst-port 22" was
added by myself to have log from ipfw

syslog.conf:
!blacklistd
*.*                                             /var/log/blacklistd.log

I did sshd MaxAuthTries =3 and 4.

maxauth =3, the blacklistd-helper.log:

--start sshd maxauth=3; blacklist nfail=2, disable=*; ipfw enabled, removed
/etc/ipfw-blacklist.rc--
Wed Nov 15 09:53:40 CET 2017 /usr/libexec/blacklistd-helper run flush
blacklistd
Wed Nov 15 09:55:47 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 59.120.35.74 32 22
Wed Nov 15 09:55:47 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 59.120.35.74 32 22
Wed Nov 15 09:59:21 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 193.201.224.218 32 22
Wed Nov 15 09:59:21 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 193.201.224.218 32 22
Wed Nov 15 09:59:25 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 193.201.224.218 32 22
Wed Nov 15 09:59:26 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 193.201.224.218 32 22
Wed Nov 15 09:59:26 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 193.201.224.218 32 22
....

blacklistd.log:

Nov 15 09:55:09 res blacklistd[18044]: Connected to blacklist server
Nov 15 10:14:14 res blacklistd[18045]: message too short 144
Nov 15 10:14:14 res blacklistd[18045]: no message (Connection refused)
Nov 15 10:17:33 res blacklistd[18045]: message too short 144
Nov 15 10:17:33 res blacklistd[18045]: no message (Connection refused)
Nov 15 10:17:34 res blacklistd[18045]: message too short 144
Nov 15 10:17:34 res blacklistd[18045]: no message (Connection refused)
Nov 15 10:17:44 res blacklistd[18045]: message too short 144
Nov 15 10:17:44 res blacklistd[18045]: no message (Connection refused)
Nov 15 10:17:54 res blacklistd[18045]: message too short 144
Nov 15 10:17:54 res blacklistd[18045]: no message (Connection refused)
Nov 15 10:18:20 res blacklistd[18045]: message too short 144
Nov 15 10:18:20 res blacklistd[18045]: no message (Connection refused)
Nov 15 10:18:30 res blacklistd[18045]: message too short 144
Nov 15 10:18:30 res blacklistd[18045]: no message (Connection refused)

dump:

$ sudo blacklistctl dump
        address/ma:port id      nfail   last access
   59.120.35.74/32:22           1/2     2017/11/15 09:55:47
 89.135.123.209/32:22           1/2     2017/11/15 10:32:53
193.201.224.218/32:22           1/2     2017/11/15 09:59:20
118.123.245.239/32:22           1/2     2017/11/15 10:15:10

$ sudo blacklistctl dump -b
        address/ma:port id      nfail   last access

maxauth=4, the logs

$ cat blacklistd-helper.log
--start sshd maxauth=4; blacklist nfail=2, disable=*; ipfw enabled, removed
/etc/ipfw-blacklist.rc--
Wed Nov 15 10:53:39 CET 2017 /usr/libexec/blacklistd-helper run flush
blacklistd
Wed Nov 15 10:56:45 CET 2017 /usr/libexec/blacklistd-helper run flush
blacklistd
Wed Nov 15 10:58:44 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 41.73.194.139 32 22
Wed Nov 15 10:58:44 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 41.73.194.139 32 22
Wed Nov 15 11:01:04 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 83.246.164.83 32 22
Wed Nov 15 11:01:04 CET 2017 /usr/libexec/blacklistd-helper run add
blacklistd tcp 83.246.164.83 32 22

$ tail blacklistd.log
Nov 15 10:53:39 res blacklistd[21125]: Connected to blacklist server
Nov 15 10:53:53 res blacklistd[21161]: Connected to blacklist server
Nov 15 10:56:45 res blacklistd[21264]: Connected to blacklist server
Nov 15 10:56:57 res blacklistd[21312]: Connected to blacklist server

$ sudo blacklistctl dump
        address/ma:port id      nfail   last access
  41.73.194.139/32:22           1/2     2017/11/15 10:58:44
  83.246.164.83/32:22           1/2     2017/11/15 11:01:04

$ sudo blacklistctl dump -b
        address/ma:port id      nfail   last access



>
> Moving on ..
>
> cheers, Ian
>



-- 
with kind regards



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAKV%2BxLC=ABe2i3TN8bo4XaVg3KfUbKsS96=6iyVDnsmWw-e8ag>