From owner-freebsd-current@FreeBSD.ORG Tue Mar 13 15:49:43 2007 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1C62A16A403 for ; Tue, 13 Mar 2007 15:49:43 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from cs1.cs.huji.ac.il (cs1.cs.huji.ac.il [132.65.16.10]) by mx1.freebsd.org (Postfix) with ESMTP id BE9CB13C44B for ; Tue, 13 Mar 2007 15:49:42 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by cs1.cs.huji.ac.il with esmtp id 1HR9Fs-000CoX-Jj; Tue, 13 Mar 2007 17:49:40 +0200 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: freebsd-current@freebsd.org In-reply-to: Your message of Tue, 13 Mar 2007 14:02:43 +0200 . Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 13 Mar 2007 17:49:40 +0200 From: Danny Braniss Message-ID: Cc: pyunyh@gmail.com Subject: Re: nfe/PXE problem X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Mar 2007 15:49:43 -0000 > > On Tue, Mar 13, 2007 at 09:10:38AM +0200, Danny Braniss wrote: > > > > > > > well, I can PXE boot this box if I use an fxp NIC, with the nfe I tracked the > > > > problem > > > > to sys/nfsclient/nfs_diskless, where in nfs_setup_diskless(void) > > > > it does not find the nfe interface, ie, something does not match, but the > > > > interface > > > > was detected!. > > > > i'll try and do some more debugging. > > > > > > > ok, I found the problem, in nfs_diskless.c nfs_setup_diskless(), > > > there is a loop to search for the interface that was used to boot from, > > > and no match is found because the hadrware ethernet address > > > in the nfe is in the wrong byte order! and so the bcmp(...) fails. > > > > Good catch! > > > > > Interestingly, if booting NOT via PXE the Ethernet address is OK! > > > > > > nfe0: port 0xdc00-0xdc07 mem > > > 0xfe02c000-0xfe02cfff irq 22 at device 20.0 on pci0 > > > nfe0: Ethernet address: 00:18:f3:a9:6c:57 > > > and via PXE: > > > nfe0: Ethernet address: 57:6c:a9:f3:18:00 > > > > > > can someone with the right knowledge fix this? > > > > > > > AFAIK there is no known way to get an ethernet address via EEPROM on > > NVIDIA NIC so nfe(4) reads specific address registers to get ethernet > > address as a workaround. The address registers are filled > > automatically by hardware after hardware reset. This type of > > acquisition of ethernet address is *NOT* correct way as it could get > > a fake etherent address since adiministor can program other ethernet > > address into the NIC.(e.g. ifconfig(8) can change ethernet address.) > > Therefore nfe(4) is very careful to save original ethernet address > > in device attach phase and restores the ethernet address at device > > detach time. > > To make matters worse, it seems that ethernet address in NVIDIA NIC > > is loaded backwards into registers so nfe(4) corrects it in > > nfe_init_locked(). However, it seems PXE code does not do necessary > > swapping for ethernet address and does not restore original MAC > > address in the end of PXE phase so the NIC will have bogusly > > programmed MAC address. When kernel boots and nfe(4) is attached it > > will load the bogus ethernet address which was already programmed by > > PXE. If /etc/start_if.nfe0 is honored in diskless boot you can override > > the ethernet address to use the same ethernet address as PXE did. > > > > Even if there is a way to get an ethernet address from EEPROM, PXE > > should be fixed first since it uses bogus ethernet address in booting > > stage. If I encounter the same situation I would contact vendor for > > updated PXE image for the NVIDIA NIC. If that is not available I think > > the only remaining option would be adding a new tunable to swap the > > ethernet address. > > > > I'm not familiar with PXE and I din't use PXE for a long time so I > > could be completely wrong. > > not completely :-) > the DHCP/PXE boot works with the 'correct ethernet address'. > this is because I use the static/registered mac, and so a bogus mac address > can't get past the DHCP/PXE stage. > so, the PXE is using the correct mac address. > I can't use /etc/... because of the horse-cart problem, it's booting > diskless, and the 'disk' is nfs readable via the NIC, but the NIC is not > working > > the question now, is to see if the nfe driver can check if it was > used by the PXE, and flip the address. or have NVIDIA comeup with > a patch ... new twist to the Nvidia saga: the problem (of the inverted mac) happens with the latest bios, a previous version is ok - unfourtunately, it seems impossible to downgrade. danny