Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 29 Oct 2018 13:34:40 +0100
From:      Miroslav Lachman <000.fbsd@quip.cz>
To:        freebsd-stable@freebsd.org
Subject:   sync command runs for hours after upgrade to 11.2
Message-ID:  <21a79514-0ada-83bd-bd4a-77fb838042fe@quip.cz>

next in thread | raw e-mail | index | archive | help
I upgraded about 10 machines from 10.4 to 11.2 this month. The last 
machines were upgrade few days ago and one of them exhibits strange 
behaviour with sync(8) command.
We have sync in one script for MRTG which is called each 5 minutes. 
Every day between 8 - 15h the sync calls are not finished in few seconds 
(or fraction of second) as usual but they run for hours and cannot be 
killed.

# ps auxww | grep sync
mrtg     22897 23.9  0.0    4232    2048  -  R    09:20      69:30.86 sync
mrtg     22186 23.8  0.0    4232    2048  -  R    09:15      72:14.36 sync
mrtg     29535 23.2  0.0    4232    2048  -  R    10:05      48:05.80 sync
mrtg     46040 22.5  0.0    4232    2048  -  R    11:25      22:24.87 sync
mrtg     59472 22.4  0.0    4232    2048  -  R    12:40       4:30.85 sync
mrtg     25683 22.2  0.0    4232    2048  -  R    09:40      59:20.61 sync
mrtg     51972 22.1  0.0    4232    2048  -  R    12:05      12:19.05 sync
mrtg     27983 22.0  0.0    4232    2048  -  R    09:55      52:25.62 sync
mrtg      6534 21.8  0.0    4232    2048  -  R    08:00     126:28.14 sync
mrtg     49674 21.7  0.0    4232    2048  -  R    11:50      15:57.96 sync
mrtg     58720 21.6  0.0    4232    2048  -  R    12:35       5:33.81 sync
mrtg      7419 21.5  0.0    4232    2048  -  R    08:05     122:03.06 sync
mrtg     16371 21.5  0.0    4232    2048  -  R    08:50      88:07.14 sync
mrtg     26454 21.5  0.0    4232    2048  -  R    09:45      56:50.32 sync
mrtg     36149 21.5  0.0    4232    2048  -  R    10:35      37:25.89 sync
mrtg     37607 21.5  0.0    4232    2048  -  R    10:45      34:19.37 sync
mrtg      6588 21.3  0.0    4232    2048  -  R    08:00     126:31.50 sync
mrtg     30283 21.1  0.0    4232    2048  -  R    10:10      46:04.96 sync
mrtg     11030 21.0  0.0    4232    2048  -  R    08:15     114:01.56 sync
mrtg     52724 21.0  0.0    4232    2048  -  R    12:10      11:07.51 sync
mrtg     61965 21.0  0.0    4232    2048  -  R    12:55       1:23.08 sync
mrtg     35360 20.9  0.0    4232    2048  -  R    10:30      38:59.89 sync
mrtg     40764 20.9  0.0    4232    2048  -  R    11:05      27:59.73 sync
mrtg     28855 20.8  0.0    4232    2048  -  R    10:00      50:07.67 sync
mrtg     33199 20.8  0.0    4232    2048  -  R    10:15      44:08.61 sync
mrtg     57984 20.8  0.0    4232    2048  -  R    12:30       6:39.88 sync
mrtg     17866 20.7  0.0    4232    2048  -  R    09:00      81:18.44 sync
mrtg     57226 20.7  0.0    4232    2048  -  R    12:25       7:49.24 sync
mrtg     34610 20.6  0.0    4232    2048  -  R    10:25      40:34.89 sync
mrtg     27242 20.5  0.0    4232    2048  -  R    09:50      54:37.43 sync
mrtg      8180 20.4  0.0    4232    2048  -  R    08:10     117:47.97 sync
mrtg     12422 20.4  0.0    4232    2048  -  R    08:25     106:02.85 sync
mrtg     17071 20.4  0.0    4232    2048  -  R    08:55      84:41.97 sync
mrtg     48920 20.4  0.0    4232    2048  -  R    11:45      17:14.73 sync
mrtg     23521 20.3  0.0    4232    2048  -  R    09:25      66:46.53 sync
mrtg     39911 20.3  0.0    4232    2048  -  R    11:00      29:28.89 sync
mrtg     56437 20.2  0.0    4232    2048  -  R    12:20       8:56.43 sync
mrtg     61234 20.2  0.0    4232    2048  -  R    12:50       2:23.54 sync
mrtg     14981 20.1  0.0    4232    2048  -  R    08:40      95:00.02 sync
mrtg     46832 20.1  0.0    4232    2048  -  R    11:30      21:03.80 sync
mrtg     50447 20.1  0.0    4232    2048  -  R    11:55      14:42.42 sync
mrtg     51273 20.1  0.0    4232    2048  -  R    12:00      13:33.55 sync
mrtg     14088 20.0  0.0    4232    2048  -  R    08:35      98:16.55 sync
mrtg     39126 20.0  0.0    4232    2048  -  R    10:55      31:12.26 sync
mrtg     62764 20.0  0.0    4232    2048  -  R    13:00       0:21.80 sync
mrtg     15716 19.9  0.0    4232    2048  -  R    08:45      91:27.30 sync
mrtg     55646 19.9  0.0    4232    2048  -  R    12:15      10:03.76 sync
mrtg     33863 19.7  0.0    4232    2048  -  R    10:20      42:24.80 sync
mrtg     48174 19.6  0.0    4232    2048  -  R    11:40      18:29.01 sync
mrtg     24297 19.5  0.0    4232    2048  -  R    09:30      64:03.59 sync
mrtg     44522 19.5  0.0    4232    2048  -  R    11:15      25:02.50 sync
root     61355 19.5  0.0    4232    2048  2  R+   12:51       2:06.40 sync
mrtg     38358 19.4  0.0    4232    2048  -  R    10:50      32:39.67 sync
mrtg     36847 19.1  0.0    4232    2048  -  R    10:40      35:44.49 sync
mrtg     45339 19.1  0.0    4232    2048  -  R    11:20      23:45.21 sync
mrtg     60318 19.1  0.0    4232    2048  -  R    12:45       3:27.70 sync
mrtg     13200 18.8  0.0    4232    2048  -  R    08:30     102:11.46 sync
mrtg     18627 18.8  0.0    4232    2048  -  R    09:05      78:06.64 sync
mrtg     41498 18.8  0.0    4232    2048  -  R    11:10      26:30.43 sync
mrtg     47563 18.8  0.0    4232    2048  -  R    11:35      19:42.28 sync
mrtg     11700 18.5  0.0    4232    2048  -  R    08:20     110:05.39 sync
mrtg     19368 18.3  0.0    4232    2048  -  R    09:10      75:03.78 sync
mrtg     25043 18.0  0.0    4232    2048  -  R    09:35      61:45.86 sync

The same script is used on each of our servers without any problem.

This machine is running Apache and PHP without any problems. Web is 
still responding even if the system load is about 60:

last pid: 59734;  load averages: 61.76, 59.62, 57.24   up 2+12:33:30 
12:42:31
387 processes: 64 running, 318 sleeping, 5 zombie
CPU: 14.1% user,  0.0% nice, 85.7% system,  0.1% interrupt,  0.1% idle
Mem: 1006M Active, 9907M Inact, 9056M Laundry, 2703M Wired, 1405M Buf, 
1224M Free
Swap: 16G Total, 63M Used, 16G Free

The one thing that can be of some coincidence is rsync backup which is 
running in the time between 6 - 17h.

Can anybody help where should I look to solve this problem?

# uname -srmi
FreeBSD 11.2-RELEASE-p4 amd64 GENERIC

/dev/mirror/gm0root on / (ufs, local, noatime)
/dev/mirror/gm0usr on /usr (ufs, local, noatime, soft-updates)
/dev/mirror/gm0var on /var (ufs, local, noatime, nosuid, soft-updates)
/dev/mirror/gm0tmp on /tmp (ufs, local, noatime, noexec, nosuid, 
soft-updates)
/dev/mirror/gm0vol0 on /vol0 (ufs, local, noatime, noexec, nosuid, 
soft-updates)
/dev/mirror/gm0vol1 on /vol1 (ufs, local, noatime, noexec, nosuid, 
soft-updates)

No ZFS involved.

Kind regards
Miroslav Lachman



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?21a79514-0ada-83bd-bd4a-77fb838042fe>