From owner-freebsd-hackers@freebsd.org Tue Apr 19 14:32:43 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6E9D5B13DE4 for ; Tue, 19 Apr 2016 14:32:43 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from pmta2.delivery6.ore.mailhop.org (pmta2.delivery6.ore.mailhop.org [54.200.129.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 456A6143B for ; Tue, 19 Apr 2016 14:32:43 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: 96a757f1-063b-11e6-9fea-b54670b854a5 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.34.117.227 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.34.117.227]) by outbound2.ore.mailhop.org (Halon Mail Gateway) with ESMTPSA; Tue, 19 Apr 2016 14:32:47 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.14.9) with ESMTP id u3JEWZvL035783; Tue, 19 Apr 2016 08:32:35 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1461076355.1232.17.camel@freebsd.org> Subject: Re: Diskless workstation and some minor issues From: Ian Lepore To: Alan Somers Cc: "freebsd-hackers@freebsd.org" , BERTRAND =?ISO-8859-1?Q?Jo=EBl?= Date: Tue, 19 Apr 2016 08:32:35 -0600 In-Reply-To: References: <57163991.4000100@systella.fr> <1461075243.1232.9.camel@freebsd.org> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.16.5 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Apr 2016 14:32:43 -0000 On Tue, 2016-04-19 at 08:16 -0600, Alan Somers wrote: > On Tue, Apr 19, 2016 at 8:14 AM, Ian Lepore wrote: > > > On Tue, 2016-04-19 at 15:58 +0200, BERTRAND Joël wrote: > > > > Hello, > > > > > > > I have installed a diskless workstation (10.2-RELEASE > > > -p14) that > > mounts > > > its rootfs from a NetBSD nfs server. My fstab is : > > > > > > root@pythagore:~ # cat /etc/fstab > > > # Device Mountpoint FStype Options Dump Pass# > > > 192.168.10.128:/srv/pythagore / nfs > > > nfsv3,tcp,soft,intr,rw > > > 0 0 > > > ... > > > 192.168.10.128:/home /home nfs > > > nfsv3,tcp,soft,intr,rw > > > 0 0 > > > root@pythagore:~ # > > > > > > > First question. I have installed this diskless > > > workstation with > > an USB > > > stick. Now, FreeBSD kernel mounts rootfs over NFS. But in a first > > > time, > > > it tries to mount usbus0 (!) : > > > > > > Apr 9 10:48:00 pythagore kernel: Root mount waiting for: usbus2 > > > usbus1 > > > usbus0 > > > Apr 9 10:48:00 pythagore kernel: uhub0: 2 ports with 2 > > > removable, self > > > powered > > > Apr 9 10:48:00 pythagore kernel: uhub2: 2 ports with 2 > > > removable, self > > > powered > > > Apr 9 10:48:00 pythagore kernel: uhub1: 21 ports with 21 > > > removable, > > > self powered > > > Apr 9 10:48:00 pythagore kernel: Root mount waiting for: usbus2 > > > usbus1 > > > usbus0 > > > Apr 9 10:48:00 pythagore kernel: ugen0.2: at usbus0 > > > Apr 9 10:48:00 pythagore kernel: ugen2.2: at usbus2 > > > Apr 9 10:48:00 pythagore kernel: uhub3: > > > class 9/0, rev 2.00/0.00, addr 2> on usbus2 > > > Apr 9 10:48:00 pythagore kernel: ugen1.2: at usbus1 > > > Apr 9 10:48:01 pythagore kernel: uhub4: > > > class 9/0, rev 2.00/0.00, addr 2> on usbus1 > > > Apr 9 10:48:01 pythagore kernel: Root mount waiting for: usbus2 > > > usbus1 > > > usbus0 > > > Apr 9 10:48:01 pythagore kernel: uhub4: 6 ports with 6 > > > removable, self > > > powered > > > Apr 9 10:48:01 pythagore kernel: uhub3: 8 ports with 8 > > > removable, self > > > powered > > > Apr 9 10:48:01 pythagore kernel: ugen0.3: at usbus0 > > > Apr 9 10:48:01 pythagore kernel: ukbd0: > > > 0/0, rev 1.10/1.12, addr 2> on usbus0 > > > Apr 9 10:48:01 pythagore kernel: kbd2 at ukbd0 > > > Apr 9 10:48:01 pythagore kernel: Root mount waiting for: usbus0 > > > Apr 9 10:48:01 pythagore kernel: usbd_setup_device_desc: getting > > > device > > > descriptor at addr 3 failed, USB_ERR_IOERROR > > > Apr 9 10:48:01 pythagore kernel: Root mount waiting for: usbus0 > > > Apr 9 10:48:01 pythagore kernel: usbd_setup_device_desc: getting > > > device > > > descriptor at addr 3 failed, USB_ERR_IOERROR > > > Apr 9 10:48:01 pythagore kernel: Root mount waiting for: usbus0 > > > Apr 9 10:48:01 pythagore kernel: Root mount waiting for: usbus0 > > > Apr 9 10:48:01 pythagore kernel: usbd_setup_device_desc: getting > > > device > > > descriptor at addr 3 failed, USB_ERR_IOERROR > > > Apr 9 10:48:02 pythagore kernel: Root mount waiting for: usbus0 > > > Apr 9 10:48:02 pythagore kernel: usbd_setup_device_desc: getting > > > device > > > descriptor at addr 3 failed, USB_ERR_IOERROR > > > Apr 9 10:48:02 pythagore kernel: Root mount waiting for: usbus0 > > > Apr 9 10:48:02 pythagore kernel: Root mount waiting for: usbus0 > > > Apr 9 10:48:02 pythagore kernel: usbd_setup_device_desc: getting > > > device > > > descriptor at addr 3 failed, USB_ERR_IOERROR > > > Apr 9 10:48:02 pythagore kernel: ugen0.4: at usbus0 > > > (disconnected) > > > Apr 9 10:48:02 pythagore kernel: uhub_reattach_port: could not > > > allocate > > > new device > > > Apr 9 10:48:02 pythagore kernel: Trying to mount root from > > > nfs:192.168.10.128:/srv/pythagore [nfsv3,tcp,soft,intr,rw]... > > > Apr 9 10:48:02 pythagore kernel: NFS ROOT: 192.168.10.128: > > /srv/pythagore > > > > > > > I don't understand why kernel tries to mount an USB > > > rootfs... > > And I > > > haven't found how fix this trouble. > > > > > > > At this point during booting, the system doesn't know what physical > > device may hold the rootfs, so it waits until it has enumerated all > > the > > devices on all the USB busses, in case one of them is required to > > mount > > root (even in the NFS case, it could require a USB network device). > > > > To disable this wait, put hw.usb.no_boot_wait=1 in loader.conf. > > > > > Second trouble. /var/log/message contains a lot of : > > > Apr 9 10:50:00 pythagore atrun[862]: cannot lock /var/at/jobs/: > > > Permission denied > > > > > > > Of course, lockd and statd are running on nfs server. > > > Permission > > on > > > /var/at are : > > > > > > root@pythagore:/var/at # ls -al > > > total 8 > > > drwxr-xr-x 4 root wheel 512 Aug 12 2015 . > > > drwxr-xr-x 28 root wheel 512 Apr 15 09:14 .. > > > drwxr-xr-x 2 daemon wheel 512 Aug 12 2015 jobs > > > drwxr-xr-x 2 daemon wheel 512 Aug 12 2015 spool > > > root@pythagore:/var/at # > > > > > > > I don't understand where is the mistake. > > > > > > > This is a more serious problem. I have found it to be impossible > > to > > run a diskless workstation with a persistant /var mounted via NFS > > (either by itself or as a directory within the nfs rootfs). It's > > been > > this way for several years. You can add varmfs=yes to your rc.conf > > to > > get a working system, but then you have a non-persistant /var which > > really isn't very useful. > > > > Hmm, but the problems I usually have are with /var/run and > > pidfiles. > > I've never noticed this /var/at problem (maybe just because I gave > > up > > trying to run with an nfs-mounted /var before I noticed them). > > > > > An third trouble, this workstation runs cupsd. When I send > > > a job to > > > printer, cupsd run as expected but aborts after job is completed. > > > In > > > /var/log/cups/error_log, I have : > > > > > > E [19/Apr/2016:12:44:34 +0200] Unable to create > > > "/usr/local/etc/cups/printers.conf.N": Operation not permitted > > > E [19/Apr/2016:12:45:12 +0200] Unable to create > > > "/var/cache/cups/job.cache.N": Operation not permitted > > > E [19/Apr/2016:12:45:12 +0200] Unable to create > > > "/var/spool/cups/c00032.N": Operation not permitted > > > > > > > I have checked permissions on directories (and config > > > file) > > without any > > > result, but I have seen that permissions on /var/spool/cups are > > > overwrittem by cups (to drwx--x---). > > > > > > > This one I can't help with at all, because I gave up on trying to > > get > > printers to work on freebsd about 10 years ago and haven't had the > > energy to ever try again. :( > > > > -- Ian > > > > Check your /etc/exports on the server. Both of these problems could > be > explained if your server has -maproot set (or whatever NetBSD calls > it). > For diskless workstations, you must turn that off at the server. > > -Alan This sounds backwards. With no -maproot, the default is to use a uid/gid of -2/-2; how would the rc scripts be able to create files in /var and manipulate entropy files and all the other root-y things they do? All my nfsroot exports use -maproot 0. Hmmm, I wonder if -maproot 0:0 would be better (not that I expect that change alone to make diskless /var start working again). -- Ian