From owner-freebsd-current@FreeBSD.ORG Tue Mar 13 11:35:21 2007 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5E40C16A401 for ; Tue, 13 Mar 2007 11:35:21 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1C5FE13C468 for ; Tue, 13 Mar 2007 11:35:20 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: by py-out-1112.google.com with SMTP id f47so558349pye for ; Tue, 13 Mar 2007 04:35:20 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:received:received:date:from:to:cc:subject:message-id:reply-to:references:mime-version:content-type:content-disposition:in-reply-to:user-agent; b=eyNK4uevH1Yka3k2iKeKgHw1I70mybsd7goziefspxpEAziAVr+s5kqABPRWJ/VwJznsqRQjtHM0IiMLi0690et/mv4CtrnrtWbd4hkVtpTnplK9fCedQorpinEBnglNkgG/jSe0Byr0DGdFDeo4e/JVMQ7Hd9dpyizTPaZWL3c= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:date:from:to:cc:subject:message-id:reply-to:references:mime-version:content-type:content-disposition:in-reply-to:user-agent; b=XixHnP+TCDlpYbArsFf5Y+1t0aEa9hDIGDkMHAZeIi5XX6qpewaSTaqTW4jfITH0GQNq2DS0EEx5p2jEfvFnvK2A68hsFbuEpBYIWN3Sj+Us5GsA1nxqnmDV4dJAwOe1LAkrNxkL+Ts7FThQ8AulyLfiATKOsA9EDQstxwyzWKo= Received: by 10.35.121.2 with SMTP id y2mr1537858pym.1173785720456; Tue, 13 Mar 2007 04:35:20 -0700 (PDT) Received: from michelle.cdnetworks.co.kr ( [211.53.35.84]) by mx.google.com with ESMTP id 38sm34779265nza.2007.03.13.04.35.18; Tue, 13 Mar 2007 04:35:19 -0700 (PDT) Received: from michelle.cdnetworks.co.kr (localhost.cdnetworks.co.kr [127.0.0.1]) by michelle.cdnetworks.co.kr (8.13.5/8.13.5) with ESMTP id l2DBbgnh090504 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 13 Mar 2007 20:37:42 +0900 (KST) (envelope-from pyunyh@gmail.com) Received: (from yongari@localhost) by michelle.cdnetworks.co.kr (8.13.5/8.13.5/Submit) id l2DBbfqg090503; Tue, 13 Mar 2007 20:37:41 +0900 (KST) (envelope-from pyunyh@gmail.com) Date: Tue, 13 Mar 2007 20:37:41 +0900 From: Pyun YongHyeon To: Danny Braniss Message-ID: <20070313113741.GE87608@cdnetworks.co.kr> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.1i Cc: freebsd-current@freebsd.org Subject: Re: nfe/PXE problem X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Mar 2007 11:35:21 -0000 On Tue, Mar 13, 2007 at 09:10:38AM +0200, Danny Braniss wrote: > > > well, I can PXE boot this box if I use an fxp NIC, with the nfe I tracked the > > problem > > to sys/nfsclient/nfs_diskless, where in nfs_setup_diskless(void) > > it does not find the nfe interface, ie, something does not match, but the > > interface > > was detected!. > > i'll try and do some more debugging. > > > ok, I found the problem, in nfs_diskless.c nfs_setup_diskless(), > there is a loop to search for the interface that was used to boot from, > and no match is found because the hadrware ethernet address > in the nfe is in the wrong byte order! and so the bcmp(...) fails. Good catch! > Interestingly, if booting NOT via PXE the Ethernet address is OK! > > nfe0: port 0xdc00-0xdc07 mem > 0xfe02c000-0xfe02cfff irq 22 at device 20.0 on pci0 > nfe0: Ethernet address: 00:18:f3:a9:6c:57 > and via PXE: > nfe0: Ethernet address: 57:6c:a9:f3:18:00 > > can someone with the right knowledge fix this? > AFAIK there is no known way to get an ethernet address via EEPROM on NVIDIA NIC so nfe(4) reads specific address registers to get ethernet address as a workaround. The address registers are filled automatically by hardware after hardware reset. This type of acquisition of ethernet address is *NOT* correct way as it could get a fake etherent address since adiministor can program other ethernet address into the NIC.(e.g. ifconfig(8) can change ethernet address.) Therefore nfe(4) is very careful to save original ethernet address in device attach phase and restores the ethernet address at device detach time. To make matters worse, it seems that ethernet address in NVIDIA NIC is loaded backwards into registers so nfe(4) corrects it in nfe_init_locked(). However, it seems PXE code does not do necessary swapping for ethernet address and does not restore original MAC address in the end of PXE phase so the NIC will have bogusly programmed MAC address. When kernel boots and nfe(4) is attached it will load the bogus ethernet address which was already programmed by PXE. If /etc/start_if.nfe0 is honored in diskless boot you can override the ethernet address to use the same ethernet address as PXE did. Even if there is a way to get an ethernet address from EEPROM, PXE should be fixed first since it uses bogus ethernet address in booting stage. If I encounter the same situation I would contact vendor for updated PXE image for the NVIDIA NIC. If that is not available I think the only remaining option would be adding a new tunable to swap the ethernet address. I'm not familiar with PXE and I din't use PXE for a long time so I could be completely wrong. > thanks, > danny > > -- Regards, Pyun YongHyeon