Date: Sun, 28 Mar 2021 16:39:41 +0200 From: Stefan Esser <se@freebsd.org> To: FreeBSD CURRENT <freebsd-current@freebsd.org> Subject: Strange behavior after running under high load Message-ID: <58bea0f0-5c3d-4263-ebee-f939a7e169e9@freebsd.org>
next in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--8NgVqi8xO790SvAs7GtbpRIhcJkdVL7kf
Content-Type: multipart/mixed; boundary="ElzUDxtRwMaKrPwRmcUzluqPLMh2LVnJA";
protected-headers="v1"
From: Stefan Esser <se@freebsd.org>
To: FreeBSD CURRENT <freebsd-current@freebsd.org>
Message-ID: <58bea0f0-5c3d-4263-ebee-f939a7e169e9@freebsd.org>
Subject: Strange behavior after running under high load
--ElzUDxtRwMaKrPwRmcUzluqPLMh2LVnJA
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
After a period of high load, my now idle system needs 4 to 10 seconds to
run any trivial command - even after 20 minutes of no load ...
I have run some Monte-Carlo simulations for a few hours, with initially 3=
5=20
processes running in parallel for some 10 seconds each.
The load decreased over time since some parameter sets were faster to pro=
cess.
All in all 63000 processes ran within some 3 hours.
When the system became idle, interactive performance was very bad. Runnin=
g
any trivial command (e.g. uptime) takes some 5 to 10 seconds. Since I hav=
e
to have this system working, I plan to reboot it later today, but will ke=
ep
it in this state for some more time to see whether this state persists or=
whether the system recovers from it.
Any ideas what might cause such a system state???
The system has a Ryzen 5 3600 CPU (6 core/12 threads) and 32 GB or RAM.
The following are a few commands that I have tried on this now practicall=
y
idle system:
$ time vmstat -n 1
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr nv0 in sy cs us sy =
id
2 0 0 26G 922M 1.2K 1 4 0 1.4K 239 0 482 7.2K 934 11 1 =
88
real 0m9,357s
user 0m0,001s
sys 0m0,018
---- wait 1 minute ----
$ time vmstat -n 1
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr nv0 in sy cs us sy =
id
1 0 0 26G 925M 1.2K 1 4 0 1.4K 239 0 482 7.2K 933 11 1 =
88
real 0m9,821s
user 0m0,003s
sys 0m0,389s
$ systat -vm
4 users Load 0.10 0.72 3.57 Mar 28 16:15
Mem usage: 97%Phy 55%Kmem VN PAGER SWAP=20
PAGER
Mem: REAL VIRTUAL in out in =20
out
Tot Share Tot Share Free count
Act 2387M 460K 26481M 460K 923M pages
All 2605M 218M 27105M 572M ioflt Interru=
pts
Proc: cow 132 to=
tal
r p d s w Csw Trp Sys Int Sof Flt 52 zfod 96 h=
pet0:t0
316 356 39 225 132 21 53 ozfod nvme0:a=
dmi
%ozfod nvme0:i=
o0
0.1%Sys 0.0%Intr 0.0%User 0.0%Nice 99.9%Idle daefr nvme0:i=
o1
| | | | | | | | | | | prcfr nvme0:io=
2
totfr nvme0:i=
o3
dtbuf react nvme0:i=
o4
Namei Name-cache Dir-cache 620370 maxvn pdwak nvme0:io=
5
Calls hits % hits % 627486 numvn 168 pdpgs 27 x=
hci0 66
18 14 78 65 frevn intrn ahci0 6=
7
17539M wire xhci1 68=
Disks nvd0 ada0 ada1 ada2 ada3 ada4 cd0 430M act 9 re=
0 69
KB/t 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12696M inact hdac0 76=
tps 0 0 0 0 0 0 0 54276K laund vgapci0 =
78
MB/s 0.00 0.00 0.00 0.00 0.00 0.00 0.00 923M free
%busy 0 0 0 0 0 0 0 0 buf
---- 5 minutes later ----
$ time vmstat -n 1
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr nv0 in sy cs us sy i=
d
1 0 0 26G 922M 1.2K 1 4 0 1.4K 239 0 481 7.2K 931 11 1 8=
8
real 0m4,270s
user 0m0,000s
sys 0m0,019s
$ time uptime
16:20 up 23:23, 4 users, load averages: 0,17 0,39 2,68
real 0m10,840s
user 0m0,001s
sys 0m0,374s
$ time uptime
16:37 up 23:40, 4 users, load averages: 0,29 0,27 0,96
real 0m9,273s
user 0m0,000s
sys 0m0,020s
--ElzUDxtRwMaKrPwRmcUzluqPLMh2LVnJA--
--8NgVqi8xO790SvAs7GtbpRIhcJkdVL7kf
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"
-----BEGIN PGP SIGNATURE-----
wsB5BAABCAAjFiEEo3HqZZwL7MgrcVMTR+u171r99UQFAmBglS0FAwAAAAAACgkQR+u171r99USX
Pgf/Yn0RCxuJ5flyciL/EneUYMQPULUiCitPn7nYysbZyWKXa3AsejiFo4+qkSMbzBAvccZAVIhZ
7OuOfDdwoIjgqA5Z06U83vRdCsUUpRvvTinP8Ifzl2VJSQFJY/biFXmzwRqsvifoubguMHlyJt/O
1calEHYYIZj6i/vuymsqZs0olVxg6w+NTN8/6Xo7EVJizsjxLTkWRh9M7xX4p0ugfdaVI7L3ZDKb
40JSPOv6reVQi59O/6E3zOW55lqB2kr4knPe+wPFzZv3GhosHEMqVZf0PAlBp32ep4HYyWJ22aLJ
PUbaV6ZiWuK64jQONsUcHqS/chULWDegOE4/qJqVqQ==
=3yYF
-----END PGP SIGNATURE-----
--8NgVqi8xO790SvAs7GtbpRIhcJkdVL7kf--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?58bea0f0-5c3d-4263-ebee-f939a7e169e9>