Date: Wed, 20 Oct 2004 21:05:16 -0400 From: Stephan Uphoff <ups@tree.com> To: Nik Azim Azam <nskyline_r35@yahoo.com> Cc: freebsd-current@freebsd.org Subject: Re: number of CPUs and IPI panic Message-ID: <1098320716.20390.416.camel@palm.tree.com> In-Reply-To: <20041020221659.33824.qmail@web21124.mail.yahoo.com> References: <20041020221659.33824.qmail@web21124.mail.yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--=-N+LMpHcCO2HdAmv0V0r5 Content-Type: text/plain Content-Transfer-Encoding: 7bit Can you try the attached patch? ( Hope it still applies since it is a few weeks old) If this does not work I have a few more questions next week. Stephan On Wed, 2004-10-20 at 18:16, Nik Azim Azam wrote: > sorry for not posting this information earlier. > > i'm running on SCHED_4BSD, with PREEMPTION. the same > panic occours without PREEMPTION. the kernel is > generic except that i've removed some devices that i > dont need. > > --- Gavin Atkinson <gavin.atkinson@ury.york.ac.uk> > wrote: > > > On Wed, 20 Oct 2004, Nik Azim Azam wrote: > > > > > hello list, > > > > > > i'm doing some testing on a 6 cpu pentium II > > overdrive > > > computer with 6-CURRENT. with 6 cpus, the system > > will > > > panic easily with moderate load (previous IPI is > > > stuck). i edited sys/i386/sys/param.h file and > > lowered > > > the MAXCPU to 4. now the system is running with 4 > > cpus > > > and it survived a make buildworld + portupgrade at > > the > > > same time. > > > > > > is this a known problem? i can give more > > information > > > if anyone interested. > > > > Are you using SCHED_ULE? (I believe it's the default > > in 6-CURRENT...) > > > > If so, try with SCHED_4BSD. ULE has known problems > > at the moment. > > > > Gavin > > > > > > > _______________________________ > Do you Yahoo!? > Declare Yourself - Register online to vote today! > http://vote.yahoo.com > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > > --=-N+LMpHcCO2HdAmv0V0r5 Content-Disposition: attachment; filename=ipi_patch Content-Type: text/x-patch; name=ipi_patch; charset=ASCII Content-Transfer-Encoding: 7bit Index: local_apic.c =================================================================== RCS file: /cvsroot/src/sys/i386/i386/local_apic.c,v retrieving revision 1.9 diff -u -r1.9 local_apic.c --- local_apic.c 14 Jul 2004 18:12:15 -0000 1.9 +++ local_apic.c 4 Oct 2004 02:56:22 -0000 @@ -693,6 +693,62 @@ intr_restore(eflags); } + +static void +lapic_ipi_wait_and_raw(int delay,register_t icrlo, u_int dest) +{ + int x, incr,idle; + register_t value, eflags; + + +/* XXX: Need more sanity checking of icrlo? */ + KASSERT(lapic != NULL, ("%s called too early", __func__)); + KASSERT((dest & ~(APIC_ID_MASK >> APIC_ID_SHIFT)) == 0, + ("%s: invalid dest field", __func__)); + KASSERT((icrlo & APIC_ICRLO_RESV_MASK) == 0, + ("%s: reserved bits set in ICR LO register", __func__)); + + if (delay == -1) { + incr = 0; + delay = 1; + } else + incr = 1; + + /* Set destination in ICR HI register if it is being used. */ + eflags = intr_disable(); + + idle = 0; + + for (x = 0; x < delay; x += incr) { + if ((lapic->icr_lo & APIC_DELSTAT_MASK) == APIC_DELSTAT_IDLE) + { + idle = 1; + break; + } + intr_restore(eflags); + ia32_pause(); + eflags = intr_disable(); + + } + + if (!idle) panic("APIC: Previous IPI is stuck"); + + if ((icrlo & APIC_DEST_MASK) == APIC_DEST_DESTFLD) { + value = lapic->icr_hi; + value &= ~APIC_ID_MASK; + value |= dest << APIC_ID_SHIFT; + lapic->icr_hi = value; + } + + /* Program the contents of the IPI and dispatch it. */ + value = lapic->icr_lo; + value &= APIC_ICRLO_RESV_MASK; + value |= icrlo; + lapic->icr_lo = value; + intr_restore(eflags); +} + + #define BEFORE_SPIN 1000000 #ifdef DETECT_DEADLOCK #define AFTER_SPIN 1000 @@ -725,11 +781,8 @@ destfield = dest; } - /* Wait for an earlier IPI to finish. */ - if (!lapic_ipi_wait(BEFORE_SPIN)) - panic("APIC: Previous IPI is stuck"); - - lapic_ipi_raw(icrlo, destfield); + + lapic_ipi_wait_and_raw(BEFORE_SPIN,icrlo, destfield); #ifdef DETECT_DEADLOCK /* Wait for IPI to be delivered. */ Index: mp_machdep.c =================================================================== RCS file: /cvsroot/src/sys/i386/i386/mp_machdep.c,v retrieving revision 1.239 diff -u -r1.239 mp_machdep.c --- mp_machdep.c 23 Sep 2004 16:06:27 -0000 1.239 +++ mp_machdep.c 4 Oct 2004 02:56:22 -0000 @@ -1002,12 +1002,24 @@ smp_tlb_addr1 = addr1; smp_tlb_addr2 = addr2; atomic_store_rel_int(&smp_tlb_wait, 0); + + + /* Enable interrupts */ + /* Thread switching still disabled */ + + enable_intr(); + if (mask == (u_int)-1) ipi_all_but_self(vector); else ipi_selected(mask, vector); + while (smp_tlb_wait < ncpu) ia32_pause(); + + /* disable interrupts */ + disable_intr(); + } void Index: pmap.c =================================================================== RCS file: /cvsroot/src/sys/i386/i386/pmap.c,v retrieving revision 1.509 diff -u -r1.509 pmap.c --- pmap.c 29 Sep 2004 19:20:39 -0000 1.509 +++ pmap.c 4 Oct 2004 02:56:22 -0000 @@ -1317,12 +1317,20 @@ atomic_store_rel_int((u_int *)&lazymask, (u_int)&pmap->pm_active); atomic_store_rel_int(&lazywait, 0); + /* Enable interrupts */ + /* Thread switching still disabled */ + + enable_intr(); ipi_selected(mask, IPI_LAZYPMAP); while (lazywait == 0) { ia32_pause(); if (--spins == 0) break; } + + /* disable interrupts */ + disable_intr(); + } mtx_unlock_spin(&smp_ipi_mtx); if (spins == 0) --=-N+LMpHcCO2HdAmv0V0r5--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1098320716.20390.416.camel>