From owner-freebsd-arm@freebsd.org Tue Mar 8 14:01:36 2016 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1436AC7EA9; Tue, 8 Mar 2016 14:01:36 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Received: from gromit.dlib.vt.edu (gromit.dlib.vt.edu [128.173.126.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gromit.dlib.vt.edu", Issuer "Chumby Certificate Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 87947224; Tue, 8 Mar 2016 14:01:36 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Received: from pmather.lib.vt.edu (pmather.lib.vt.edu [128.173.126.193]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by gromit.dlib.vt.edu (Postfix) with ESMTPSA id 45604E07; Tue, 8 Mar 2016 09:01:29 -0500 (EST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) Subject: Re: Unstable NFS on recent CURRENT From: Paul Mather In-Reply-To: <1482595660.8940439.1457405756110.JavaMail.zimbra@uoguelph.ca> Date: Tue, 8 Mar 2016 09:01:29 -0500 Cc: Ronald Klop , freebsd-fs@freebsd.org, freebsd-arm@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <08710728-3130-49BE-8BD7-AFE85A31C633@gromit.dlib.vt.edu> References: <3DAB3639-8FB8-43D3-9517-94D46EDEC19E@gromit.dlib.vt.edu> <1482595660.8940439.1457405756110.JavaMail.zimbra@uoguelph.ca> To: Rick Macklem X-Mailer: Apple Mail (2.3112) X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Mar 2016 14:01:36 -0000 On Mar 7, 2016, at 9:55 PM, Rick Macklem wrote: > Paul Mather (forwarded by Ronald Klop) wrote: >> On Sun, 06 Mar 2016 02:57:03 +0100, Paul Mather = >> wrote: >>=20 >>> On my BeagleBone Black running 11-CURRENT (r296162) lately I have = been >>> having trouble with NFS. I have been doing a buildworld and = buildkernel >>> with /usr/src and /usr/obj mounted via NFS. Recently, this process = has >>> resulted in the buildworld failing at some point, with a variety of >>> errors (Segmentation fault; Permission denied; etc.). Even a "ls = -alR" >>> of /usr/src doesn't manage to complete. It errors out thus: >>>=20 >>> =3D=3D=3D=3D=3D >>> [[...]] >>> total 0 >>> ls: ./.svn/pristine/fe: Permission denied >>>=20 >>> ./.svn/pristine/ff: >>> total 0 >>> ls: ./.svn/pristine/ff: Permission denied >>> ls: fts_read: Permission denied >>> =3D=3D=3D=3D=3D >>>=20 >>> On the console, I get the following: >>>=20 >>> newnfs: server 'chumby.chumby.lan' error: fileid changed. fsid >>> 94790777:a4385de: expected fileid 0x4, got 0x2. (BROKEN NFS SERVER = OR >>> MIDDLEWARE) >>>=20 >>>=20 >>> I am using a FreeBSD/amd64 10.3-PRERELEASE (r296412) as the NFS = server. >>> On the BeagleBone Black, I am mounting /usr/src and /usr/obj via >>> /etc/fstab as follows: >>>=20 >>> chumby.chumby.lan:/build/src/head /usr/src nfs rw,nfsv4 0 0 >>> chumby.chumby.lan:/build/obj/bbb /usr/obj nfs rw,nfsv4 0 0 >>>=20 >>>=20 >>> /build/src/head and /build/obj/bbb are both ZFS file systems. >>>=20 > Is it possible that a ZFS file system has gotten to the point where = the > i-node# exceeds 32bits? ZFS does support more than 32bits for = i-node#s, > but FreeBSD does not (it truncates to the low order 32bits). > I know diddly about ZFS, so I don't know if you actually have to = create > more than 4billion files to get the i-node# to exceed 32bits or ??? >=20 > There has been work done on making ino_t 64bits, but it hasn't made it > into FreeBSD-current and I have no idea when it might. >=20 > If you could try a build on newly created file systems (or UFS ones > instead of ZFS), that would tell you if the above might be the = problem. I don't think I have that big of a ZFS pool (it's 2 TB). :-) It doesn't seem that there are excessive numbers of inodes, and the = counts match up between the NFS client and server sides. In the information below, chumby is the NFS server and beaglebone the = client: pmather@beaglebone:~ % mount /dev/mmcsd0s2a on / (ufs, local, noatime, soft-updates) devfs on /dev (devfs, local) /dev/mmcsd0s1 on /boot/msdos (msdosfs, local, noatime) tmpfs on /tmp (tmpfs, local) tmpfs on /var/log (tmpfs, local) tmpfs on /var/tmp (tmpfs, local) chumby.chumby.lan:/build/src/head on /usr/src (nfs, nfsv4acls) chumby.chumby.lan:/build/obj/bbb on /usr/obj (nfs, nfsv4acls) pmather@beaglebone:~ % df -i /usr/src /usr/obj Filesystem 1K-blocks Used Avail Capacity = iused ifree %iused Mounted on chumby.chumby.lan:/build/src/head 2097152 1344484 752668 64% = 147835 1505336 9% /usr/src chumby.chumby.lan:/build/obj/bbb 530875884 1949364 528926520 0% = 70814 1057853040 0% /usr/obj paul@chumby:/home/paul> df -i /build/src/head /build/obj/bbb Filesystem 1K-blocks Used Avail Capacity iused = ifree %iused Mounted on zroot/SHARED/build/src/head 2097152 1344484 752668 64% 147835 = 1505336 9% /build/src/head zroot/SHARED/build/obj/bbb 530876268 1949364 528926904 0% 70814 = 1057853808 0% /build/obj/bbb On the NFS client system, these are the only NFS-related settings I have = in /etc/rc.conf: nfsuserd_enable=3D"YES" nfscbd_enable=3D"YES" Would you recommend I try it with nfscbd_enable=3D"NO"? I will try NFS from other clients to see whether it's just this = FreeBSD/arm system that's having problems. Cheers, Paul. >=20 > rick >=20 >>> Has anyone else encountered this? It has only started happening >>> recently for me, it seems. Prior to this, I have been able to do a >>> buildworld and buildkernel successfully over NFS. >>>=20 >>> Cheers, >>>=20 >>> Paul. >>=20 >> I cc this to freebsd-fs for you. >>=20 >> Ronald. >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>=20 > _______________________________________________ > freebsd-arm@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arm > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org"