Date: Sun, 25 Oct 2020 10:06:29 +0000 From: Jon Schneider <freebsd@jschneider.net> To: Christos Chatzaras <chris@cretaforce.gr> Cc: freebsd-questions@freebsd.org Subject: Re: PHP-FPM processes hang in UFS-state Message-ID: <650EF24A-7444-41BA-AAF1-EC9930F25710@jschneider.net> In-Reply-To: <675E2CFF-C45D-48C2-8E55-4E9876D5098A@cretaforce.gr> References: <675E2CFF-C45D-48C2-8E55-4E9876D5098A@cretaforce.gr>
next in thread | previous in thread | raw e-mail | index | archive | help
I think this might be a job for DTrace which I don=E2=80=99t know enough to e= xactly what to do but you probably want to trace use of all those functions y= ou see in your traces and everything in UFS. Hint: do not send DTrace outpu= t to something on UFS ! You might also want to create a smaller test case to induce the problem qui= ckly. Be interesting to hear what you discover. Jon > On 24 Oct 2020, at 13:28, Christos Chatzaras <chris@cretaforce.gr> wrote: >=20 > =EF=BB=BFThis issue happened in 3 different servers the last months. >=20 > Some PHP-FPM processes hang in UFS-state. >=20 > The websites that cause the hang are all Prestashop and it's related to th= eir var/cache files (I believe it happens when they clear the cache and the f= iles are regenerated). >=20 >=20 > This is a php-fpm process that hang: >=20 > procstat -kk 73548 >=20 > PID TID COMM TDNAME KSTACK > 73548 100630 php-fpm - mi_switch+0xe2 sleepq= _wait+0x2c sleeplk+0x185 lockmgr_xlock_hard+0x1f5 __lockmgr_args+0x2bf ffs_l= ock+0xa5 VOP_LOCK1_APV+0x7c _vn_lock+0x44 vget+0x4a vfs_hash_get+0xb0 ffs_vg= etf+0x3f softdep_sync_buf+0xbee ffs_syncvnode+0x26f ffs_truncate+0x71b ufs_d= irenter+0x5f8 ufs_makeinode+0x5fb ufs_create+0x34 VOP_CREATE_APV+0x76=20 >=20 >=20 > procstat -kk -a : >=20 > https://www.coderz.gr/ufs/procstat.txt >=20 >=20 > mount | grep home : >=20 > https://www.coderz.gr/ufs/mount.txt >=20 >=20 > dumpfs /dev/mirror/gm0p7 : >=20 > https://www.coderz.gr/ufs/dumpfs.txt >=20 >=20 > I have to force reboot the server as /home is not possible to clean unmoun= t. Then I run fsck which fixes some issues because of the unclean reboot. Th= en I run fsck again that shows no errors. Then I mount /home and first renam= e the cache folders and finally delete var/cache and allow it to regenerate p= restashop files. But the same issue happens again after few days. >=20 >=20 > Any idea how to troubleshoot it? > _______________________________________________ > freebsd-questions@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.or= g" >=20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?650EF24A-7444-41BA-AAF1-EC9930F25710>