Date: Tue, 1 Oct 2019 10:28:37 +0200 From: Julien Cigar <julien@perdition.city> To: Reshad Patuck <reshadpatuck1@gmail.com> Cc: freebsd-fs@freebsd.org Subject: Re: [zfs] filesystem reads hanging Message-ID: <20191001082837.GF49734@home.lan> In-Reply-To: <CADaJeD24HV0eW7nQT9jaQwEWp=1f4J2WL3OOLZiv--v1zyepwQ@mail.gmail.com> References: <CADaJeD24HV0eW7nQT9jaQwEWp=1f4J2WL3OOLZiv--v1zyepwQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--eDB11BtaWSyaBkpc Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Oct 01, 2019 at 10:26:32AM +0530, Reshad Patuck wrote: > Hi, Hello, >=20 > I have a FreeBSD 12.0-RELEASE-p9 system running ZFS. > The system runs an application that uses postgres, and python (among other > services). >=20 > I have noticed that python suddenly is not able to connect to postgres. > When I try to investigate further, certain files on disk can not be read. > The commands `cat` and `ls -l` hang (no output and I can not ctrl-c or ki= ll > -9 them), ps -aux shows them in a D+ state. > On killing the SSH session these processes continue running in orphans, I > am not able to kill them. >=20 > Someone on IRC suggested running a zfs scrub to check for data corruption, > but running `zpool scrub zroot` has the same effect. > The command does not return, ctrl-c does not kill it and `zpool scrub -s > zroot` says "cannot cancel scrubbing zroot: there is no active scrub". >=20 > This has happened in the past 1 month to two of my production servers and > since the application was critical they were rebooted and the boxes > function as normal after the reboot. > Files that were not cat-able on the production servers were working fine > and a zfs scrub worked fine to show 0 errors and 0 fixes. > One of these boxes needed a hard reboot as it got stuck in the shutting > down stage of a soft reboot. >=20 > I am not sure where to start debugging this or if there are any ways to g= et > metrics on a box stuck in this state. > Please let me know if you would like me to fetch any metrics or run and > commands, etc. for you. > Any help would be much appreciated. This is a known problem (see PR 236220) and has been fixed by r350894 (and MFC-ed into 12-STABLE, so I guess it should be in the upcoming 12.1-RELEASE) >=20 > Best regards, >=20 > Reshad > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --eDB11BtaWSyaBkpc Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEE7vn2l0to0nV7EWolsrs3EKIEI8AFAl2TDjQACgkQsrs3EKIE I8CLuRAAwiD/LzfBnl33qAynYYHU4G/EHcDlOAigawVd1JNWXrYbBlq5eHhJ4AgA wRwYaWtXjXgnjdyATOS5yQ3Q8uAMrifPdMiBxtWi8g96kC9GNlWeB2IfTYxKKT8q I0Y6Lhtd5dgAr5fFnUuFl9+ksAawU4XxDP98Tl39c3iGSMnVR6Ct9CUqGRVI1O5x SzDhdW3ubmJuzfYFmEWRpyHU/ZSkuB2/BiwRpX0cebPq1xmSG3rZ1ZC+Iy6ow3I2 EYJc5RV+d8A/SkJ3UodSKzku2o7ms8qoCJ4U4TeX/mBtuWUmngdxTzk8Cml6BnqZ WifvKi8AFds2Sw2+vaSP85u6bYULTXkfa6F4fDnleuo5LMQTN3qMk0CaYGNOUJN3 RTyMjMwKnUVNM/5eMlNO432FDTeFW1JolxbvnCdLx56XONMxqr4kwU1Gx2RMbgRF hlEmT1r+2im2SFq03U45L6AgMQHAzKJqQhMyziVFV0JJ1r0v6JjCXdS9XdjEPNqr CqB8uZyjvR100gLi2bJw4c5CJZcgkoPPbqpEzNFdo+xqGLqRueVgvMNAuNxTlLNo hYWRqxiXQ2NEjgeif78o/5neLQyzPiv8naDtenYXi0CVRJ5ZzmJSAyf1hGOKQQKS LTsXsu0/Td8ryuV2LfwhMDpgnEGTeZUiFBy9b8/u7F38qTrTNm0= =4jsJ -----END PGP SIGNATURE----- --eDB11BtaWSyaBkpc--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20191001082837.GF49734>