From owner-freebsd-stable@FreeBSD.ORG Thu Oct 26 07:32:44 2006 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1721F16A403; Thu, 26 Oct 2006 07:32:44 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from koef.zs64.net (koef.zs64.net [212.12.50.230]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3E10D43D49; Thu, 26 Oct 2006 07:32:42 +0000 (GMT) (envelope-from stb@lassitu.de) Received: (from stb@koef.zs64.net) (authenticated) by koef.zs64.net (8.13.8/8.13.8) with ESMTP id k9Q7WXLJ014094 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Thu, 26 Oct 2006 09:32:33 +0200 (CEST) (envelope-from stb@lassitu.de) In-Reply-To: <45400286.9020402@samsco.org> References: <20061025183308.L33725@fledge.watson.org> <838FCA83-20F8-4A09-A025-E69956032F86@lassitu.de> <45400286.9020402@samsco.org> Mime-Version: 1.0 (Apple Message framework v752.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Stefan Bethke Date: Thu, 26 Oct 2006 09:32:32 +0200 To: Scott Long X-Mailer: Apple Mail (2.752.2) Cc: Andreas Sons , Robert Watson , FreeBSD Stable Subject: Re: panic: kmem_map too small X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Oct 2006 07:32:44 -0000 Am 26.10.2006 um 02:34 schrieb Scott Long: > There are no obvious culprits from what you posted. [...] Can you > try the following two commands: > > vmstat -m > sysctl hw.busdma Just reset the machine to run these two commands. Will set up a cron job to log them every five minutes so they're available when the next panic occurs. endeavour:~# vmstat -m Type InUse MemUse HighUse Requests Size(s) DEVFS1 84 21K - 84 256 linker 30 2K - 54 16,32,256 DEVFS 12 1K - 13 16,128 lockf 6 1K - 30 64 devbuf 1562 3592K - 1563 16,32,64,128,256,512,1024,2048,4096 temp 14 227K - 4028 16,32,64,128,256,512,1024,2048,4096 module 180 12K - 180 64,128 mtx_pool 1 8K - 1 GEOM 98 12K - 429 16,32,64,128,256,512,1024,2048 pgrp 25 2K - 27 64 session 23 3K - 24 128 proc 2 8K - 2 4096 subproc 148 300K - 792 256,4096 cred 14 2K - 1357 128 plimit 14 4K - 150 256 uidinfo 4 2K - 5 32,1024 sysctl 0 0K - 156 16,32,64 sysctloid 3189 97K - 3189 16,32,64 sysctltmp 0 0K - 204 16,32,64,128 umtx 90 6K - 90 64 SWAP 2 549K - 2 64 bus 793 38K - 4342 16,32,64,128,256,1024 bus-sc 82 32K - 1841 16,32,64,128,256,512,2048,4096 devstat 8 17K - 8 16,4096 eventhandler 44 3K - 44 32,128 kobj 115 230K - 134 2048 acd_driver 1 2K - 1 2048 kbdmux 6 9K - 6 16,128,256,2048,4096 rman 176 11K - 542 16,64 sbuf 0 0K - 246 16,32,64,128,256,512,1024,2048,4096 sleep queues 91 3K - 91 32 taskqueue 9 1K - 9 16,128 turnstiles 91 6K - 91 64 Unitno 6 1K - 8 16,64 ioctlops 0 0K - 487 16,32,64,256,512,1024 iov 0 0K - 292 16,64,128 msg 4 25K - 4 1024,4096 sem 4 7K - 4 512,1024,4096 shm 1 12K - 1 ttys 1072 152K - 2543 128,1024 mbuf_tag 0 0K - 2 32 soname 4 1K - 412 16,32,128 pcb 22 5K - 43 16,32,64,2048 isadev 18 2K - 18 64 BIO buffer 42 84K - 51 2048 vfscache 1 512K - 1 Export Host 1 1K - 1 256 VFS hash 1 256K - 1 vnodes 1 1K - 1 128 mount 76 3K - 251 16,32,64,128,512,2048 vnodemarker 0 0K - 52 512 ata_generic 3 3K - 3 1024 BPF 3 1K - 3 64 ifnet 4 4K - 4 256,1024 ifaddr 22 5K - 22 32,256,512,2048 ether_multi 12 1K - 14 16,32,64 clone 2 8K - 2 4096 arpcom 2 1K - 2 16 lo 1 1K - 1 16 ad_driver 2 1K - 2 32 ata_dma 6 1K - 6 128 entropy 1024 64K - 1024 64 routetbl 14 2K - 55 16,32,64,128,256 in_multi 3 1K - 3 32 hostcache 1 24K - 1 USB 31 3K - 31 16,32,64,128,256 syncache 1 8K - 1 NFS srvsock 1 1K - 1 128 NFS daemon 5 10K - 5 512 p1003.1b 1 1K - 1 16 pagedep 1 64K - 1 inodedep 1 256K - 1 newblk 1 1K - 1 256 UFS dirhash 30 6K - 30 16,32,512 UFS mount 9 19K - 9 256,2048,4096 UMAHash 1 1K - 3 256,512,1024 USBdev 3 1K - 9 16,256,512 cdev 19 3K - 19 128 file desc 74 22K - 730 32,256,2048 VM pgdata 2 65K - 2 64 sigio 1 1K - 1 32 atkbddev 2 1K - 2 32 kenv 113 8K - 114 16,32,64,4096 kqueue 0 0K - 30 256,1024 proc-args 30 2K - 317 32,64,128,256 zombie 0 0K - 644 128 I/O APIC 1 1K - 1 1024 ithread 66 6K - 66 16,64,128 memdesc 1 4K - 1 4096 nexusdev 3 1K - 3 16 DEVFS3 95 12K - 96 128 KTRACE 100 13K - 100 128 acpica 3001 158K - 42411 16,32,64,128,256,512,1024,2048 acpitask 0 0K - 2 32 PCI Link 64 6K - 64 16,64,128 acpisem 17 2K - 17 64 acpidev 93 3K - 93 32 endeavour:~# sysctl hw.busdma hw.busdma.total_bpages: 609 hw.busdma.zone0.total_bpages: 512 hw.busdma.zone0.free_bpages: 512 hw.busdma.zone0.reserved_bpages: 0 hw.busdma.zone0.active_bpages: 0 hw.busdma.zone0.total_bounced: 0 hw.busdma.zone0.total_deferred: 0 hw.busdma.zone0.lowaddr: 0xffffffff hw.busdma.zone0.alignment: 256 hw.busdma.zone0.boundary: 0 hw.busdma.zone1.total_bpages: 1 hw.busdma.zone1.free_bpages: 1 hw.busdma.zone1.reserved_bpages: 0 hw.busdma.zone1.active_bpages: 0 hw.busdma.zone1.total_bounced: 0 hw.busdma.zone1.total_deferred: 0 hw.busdma.zone1.lowaddr: 0xffffffff hw.busdma.zone1.alignment: 4096 hw.busdma.zone1.boundary: 0 hw.busdma.zone2.total_bpages: 96 hw.busdma.zone2.free_bpages: 96 hw.busdma.zone2.reserved_bpages: 0 hw.busdma.zone2.active_bpages: 0 hw.busdma.zone2.total_bounced: 0 hw.busdma.zone2.total_deferred: 0 hw.busdma.zone2.lowaddr: 0xffffffff hw.busdma.zone2.alignment: 2 hw.busdma.zone2.boundary: 65536 > Also, what version of FreeBSD is this? -stable from two days ago, plus this patch: endeavour:~# cat if_bge.patch # based on # if_bge.c.diff?r1=1.143&r2=1.144 # if_bgereg.h.diff?r1=1.54&r2=1.55 --- sys/dev/bge/if_bge.c.orig Fri Oct 13 10:05:55 2006 +++ sys/dev/bge/if_bge.c Fri Oct 20 13:00:14 2006 @@ -325,6 +325,7 @@ static void bge_txeof(struct bge_softc *); static void bge_rxeof(struct bge_softc *); +static void bge_asf_driver_up (struct bge_softc *); static void bge_tick_locked(struct bge_softc *); static void bge_tick(void *); static void bge_stats_update(struct bge_softc *); @@ -375,7 +376,12 @@ static void bge_poll(struct ifnet *ifp, enum poll_cmd cmd, int count); #endif -static void bge_reset(struct bge_softc *); +#define BGE_RESET_START 1 +#define BGE_RESET_STOP 2 +static void bge_sig_post_reset(struct bge_softc *, int); +static void bge_sig_legacy(struct bge_softc *, int); +static void bge_sig_pre_reset(struct bge_softc *, int); +static int bge_reset(struct bge_softc *); static void bge_link_upd(struct bge_softc *); static device_method_t bge_methods[] = { @@ -645,7 +651,6 @@ { struct bge_softc *sc; struct mii_data *mii; - sc = device_get_softc(dev); mii = device_get_softc(sc->bge_miibus); @@ -967,6 +972,84 @@ CSR_WRITE_4(sc, BGE_MAR0 + (i * 4), hashes[i]); } +static void +bge_sig_pre_reset(sc, type) + struct bge_softc *sc; + int type; +{ + /* + * Some chips don't like this so only do this if ASF is enabled + */ + if (sc->bge_asf_mode) + bge_writemem_ind(sc, BGE_SOFTWARE_GENCOMM, BGE_MAGIC_NUMBER); + + if (sc->bge_asf_mode & ASF_NEW_HANDSHAKE) { + switch (type) { + case BGE_RESET_START: + bge_writemem_ind(sc, BGE_SDI_STATUS, 0x1); /* START */ + break; + case BGE_RESET_STOP: + bge_writemem_ind(sc, BGE_SDI_STATUS, 0x2); /* UNLOAD */ + break; + } + } +} + +static void +bge_sig_post_reset(sc, type) + struct bge_softc *sc; + int type; +{ + if (sc->bge_asf_mode & ASF_NEW_HANDSHAKE) { + switch (type) { + case BGE_RESET_START: + bge_writemem_ind(sc, BGE_SDI_STATUS, 0x80000001); + /* START DONE */ + break; + case BGE_RESET_STOP: + bge_writemem_ind(sc, BGE_SDI_STATUS, 0x80000002); + break; + } + } +} + +static void +bge_sig_legacy(sc, type) + struct bge_softc *sc; + int type; +{ + if (sc->bge_asf_mode) { + switch (type) { + case BGE_RESET_START: + bge_writemem_ind(sc, BGE_SDI_STATUS, 0x1); /* START */ + break; + case BGE_RESET_STOP: + bge_writemem_ind(sc, BGE_SDI_STATUS, 0x2); /* UNLOAD */ + break; + } + } +} + +void bge_stop_fw(struct bge_softc *); +void +bge_stop_fw(sc) + struct bge_softc *sc; +{ + int i; + + if (sc->bge_asf_mode) { + bge_writemem_ind(sc, BGE_SOFTWARE_GENCOMM_FW, BGE_FW_PAUSE); + CSR_WRITE_4(sc, BGE_CPU_EVENT, + CSR_READ_4(sc, BGE_CPU_EVENT) != (1 << 14)); + + for (i = 0; i < 100; i++ ) { + if (!(CSR_READ_4(sc, BGE_CPU_EVENT) & (1 << 14))) + break; + DELAY(10); + } + } +} + /* * Do endian, PCI and DMA initialization. Also check the on-board ROM * self-test results. @@ -977,7 +1060,7 @@ uint32_t dma_rw_ctl; int i; - /* Set endian type before we access any non-PCI registers. */ + /* Set endianness before we access any non-PCI registers. */ pci_write_config(sc->bge_dev, BGE_PCI_MISC_CTL, BGE_INIT, 4); /* @@ -1069,6 +1152,12 @@ BGE_MODECTL_TX_NO_PHDR_CSUM); /* + * Tell the firmware the driver is running + */ + if (sc->bge_asf_mode & ASF_STACKUP) + BGE_SETBIT(sc, BGE_MODE_CTL, BGE_MODECTL_STACKUP); + + /* * Disable memory write invalidate. Apparently it is not supported * properly by these devices. */ @@ -1990,6 +2079,7 @@ uint32_t mac_tmp = 0; u_char eaddr[6]; int error = 0, rid; + int trys; sc = device_get_softc(dev); sc->bge_dev = dev; @@ -2057,7 +2147,38 @@ sc->bge_pcix = 1; /* Try to reset the chip. */ - bge_reset(sc); + if (bge_reset(sc)) { + device_printf(sc->bge_dev, "chip reset failed\n"); + bge_release_resources(sc); + error = ENXIO; + goto fail; + } + + sc->bge_asf_mode = 0; + if (bge_readmem_ind(sc, BGE_SOFTWARE_GENCOMM_SIG) + == BGE_MAGIC_NUMBER) { + if (bge_readmem_ind(sc, BGE_SOFTWARE_GENCOMM_NICCFG) + & BGE_HWCFG_ASF) { + sc->bge_asf_mode |= ASF_ENABLE; + sc->bge_asf_mode |= ASF_STACKUP; + if (sc->bge_asicrev == BGE_ASICREV_BCM5750) { + sc->bge_asf_mode |= ASF_NEW_HANDSHAKE; + } + } + } + + /* Try to reset the chip again the nice way. */ + bge_stop_fw(sc); + bge_sig_pre_reset(sc, BGE_RESET_STOP); + if (bge_reset(sc)) { + device_printf(sc->bge_dev, "chip reset failed\n"); + bge_release_resources(sc); + error = ENXIO; + goto fail; + } + + bge_sig_legacy(sc, BGE_RESET_STOP); + bge_sig_post_reset(sc, BGE_RESET_STOP); if (bge_chipinit(sc)) { device_printf(sc->bge_dev, "chip initialization failed\n"); @@ -2184,15 +2305,36 @@ sc->bge_ifmedia.ifm_media = sc->bge_ifmedia.ifm_cur- >ifm_media; } else { /* - * Do transceiver setup. + * Do transceiver setup and tell the firmware the + * driver is down so we can try to get access the + * probe if ASF is running. Retry a couple of times + * if we get a conflict with the ASF firmware accessing + * the PHY. */ + BGE_CLRBIT(sc, BGE_MODE_CTL, BGE_MODECTL_STACKUP); +again: + bge_asf_driver_up(sc); + + trys = 0; if (mii_phy_probe(dev, &sc->bge_miibus, bge_ifmedia_upd, bge_ifmedia_sts)) { + if (trys++ < 4) { + device_printf(sc->bge_dev, "Try again \n"); + bge_miibus_writereg(sc->bge_dev, 1, MII_BMCR, BMCR_RESET); + goto again; + } + device_printf(sc->bge_dev, "MII without any PHY!\n"); bge_release_resources(sc); error = ENXIO; goto fail; } + + /* + * Now tell the firmware we are going up after probing the PHY + */ + if (sc->bge_asf_mode & ASF_STACKUP) + BGE_SETBIT(sc, BGE_MODE_CTL, BGE_MODECTL_STACKUP); } /* @@ -2292,7 +2434,7 @@ BGE_LOCK_DESTROY(sc); } -static void +static int bge_reset(struct bge_softc *sc) { device_t dev; @@ -2378,7 +2520,7 @@ if (i == BGE_TIMEOUT) { device_printf(sc->bge_dev, "firmware handshake timed out\n"); - return; + return(0); } /* @@ -2399,6 +2541,10 @@ CSR_WRITE_4(sc, BGE_MODE_CTL, BGE_DMA_SWAP_OPTIONS| BGE_MODECTL_BYTESWAP_DATA); + /* Tell the ASF firmware we are up */ + if (sc->bge_asf_mode & ASF_STACKUP) + BGE_SETBIT(sc, BGE_MODE_CTL, BGE_MODECTL_STACKUP); + CSR_WRITE_4(sc, BGE_MAC_MODE, 0); /* @@ -2421,6 +2567,8 @@ CSR_WRITE_4(sc, 0x7c00, v | (1<<25)); } DELAY(10000); + + return(0); } /* @@ -2734,6 +2882,25 @@ } static void +bge_asf_driver_up(struct bge_softc *sc) +{ + if (sc->bge_asf_mode & ASF_STACKUP) { + /* Send ASF heartbeat aprox. every 2s */ + if (sc->bge_asf_count) + sc->bge_asf_count --; + else { + sc->bge_asf_count = 5; + bge_writemem_ind(sc, BGE_SOFTWARE_GENCOMM_FW, + BGE_FW_DRV_ALIVE); + bge_writemem_ind(sc, BGE_SOFTWARE_GENNCOMM_FW_LEN, 4); + bge_writemem_ind(sc, BGE_SOFTWARE_GENNCOMM_FW_DATA, 3); + CSR_WRITE_4(sc, BGE_CPU_EVENT, + CSR_READ_4(sc, BGE_CPU_EVENT) != (1 << 14)); + } + } +} + +static void bge_tick_locked(struct bge_softc *sc) { struct mii_data *mii = NULL; @@ -2747,7 +2914,9 @@ if (!sc->bge_tbi) { mii = device_get_softc(sc->bge_miibus); - mii_tick(mii); + /* Don't mess with the PHY in IPMI/ASF mode */ + if (!((sc->bge_asf_mode & ASF_STACKUP) && (sc- >bge_link))) + mii_tick(mii); } else { /* * Since in TBI mode auto-polling can't be used we should poll @@ -2764,6 +2933,8 @@ } } + bge_asf_driver_up(sc); + callout_reset(&sc->bge_stat_ch, hz, bge_tick, sc); } @@ -3101,7 +3272,13 @@ /* Cancel pending I/O and flush buffers. */ bge_stop(sc); + + bge_stop_fw(sc); + bge_sig_pre_reset(sc, BGE_RESET_START); bge_reset(sc); + bge_sig_legacy(sc, BGE_RESET_START); + bge_sig_post_reset(sc, BGE_RESET_START); + bge_chipinit(sc); /* @@ -3184,7 +3361,7 @@ CSR_WRITE_4(sc, BGE_HCC_TX_MAX_COAL_BDS_INT, 1); } else #endif - + /* Enable host interrupts. */ { BGE_SETBIT(sc, BGE_PCI_MISC_CTL, BGE_PCIMISCCTL_CLEAR_INTA); @@ -3535,7 +3712,20 @@ /* * Tell firmware we're shutting down. */ - BGE_CLRBIT(sc, BGE_MODE_CTL, BGE_MODECTL_STACKUP); + + bge_stop_fw(sc); + bge_sig_pre_reset(sc, BGE_RESET_STOP); + bge_reset(sc); + bge_sig_legacy(sc, BGE_RESET_STOP); + bge_sig_post_reset(sc, BGE_RESET_STOP); + + /* + * Keep the ASF firmware running if up. + */ + if (sc->bge_asf_mode & ASF_STACKUP) + BGE_SETBIT(sc, BGE_MODE_CTL, BGE_MODECTL_STACKUP); + else + BGE_CLRBIT(sc, BGE_MODE_CTL, BGE_MODECTL_STACKUP); /* Free the RX lists. */ bge_free_rx_ring_std(sc); --- sys/dev/bge/if_bgereg.h.orig Fri Oct 13 10:05:55 2006 +++ sys/dev/bge/if_bgereg.h Fri Oct 20 13:01:32 2006 @@ -74,6 +74,11 @@ #define BGE_SOFTWARE_GENCOMM 0x00000B50 #define BGE_SOFTWARE_GENCOMM_SIG 0x00000B54 #define BGE_SOFTWARE_GENCOMM_NICCFG 0x00000B58 +#define BGE_SOFTWARE_GENCOMM_FW 0x00000B78 +#define BGE_FW_DRV_ALIVE 0x00000001 +#define BGE_FW_PAUSE 0x00000002 +#define BGE_SOFTWARE_GENNCOMM_FW_LEN 0x00000B7C +#define BGE_SOFTWARE_GENNCOMM_FW_DATA 0x00000B80 #define BGE_SOFTWARE_GENCOMM_END 0x00000FFF #define BGE_UNMAPPED 0x00001000 #define BGE_UNMAPPED_END 0x00001FFF @@ -1651,6 +1656,7 @@ #define BGE_MODE_CTL 0x6800 #define BGE_MISC_CFG 0x6804 #define BGE_MISC_LOCAL_CTL 0x6808 +#define BGE_CPU_EVENT 0x6810 #define BGE_EE_ADDR 0x6838 #define BGE_EE_DATA 0x683C #define BGE_EE_CTL 0x6840 @@ -2062,6 +2068,7 @@ #define BGE_HWCFG_VOLTAGE 0x00000003 #define BGE_HWCFG_PHYLED_MODE 0x0000000C #define BGE_HWCFG_MEDIA 0x00000030 +#define BGE_HWCFG_ASF 0x00000080 #define BGE_VOLTAGE_1POINT3 0x00000000 #define BGE_VOLTAGE_1POINT8 0x00000001 @@ -2432,6 +2439,10 @@ int val; }; +#define ASF_ENABLE 1 +#define ASF_NEW_HANDSHAKE 2 +#define ASF_STACKUP 4 + struct bge_softc { struct ifnet *bge_ifp; /* interface info */ device_t bge_dev; @@ -2449,6 +2460,8 @@ uint32_t bge_chipid; uint8_t bge_asicrev; uint8_t bge_chiprev; + uint8_t bge_asf_mode; + uint8_t bge_asf_count; uint8_t bge_no_3_led; uint8_t bge_pcie; uint8_t bge_pcix; -- Stefan Bethke Fon +49 170 346 0140