From owner-svn-src-all@FreeBSD.ORG Tue Nov 24 03:32:42 2009 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 700AD1065670; Tue, 24 Nov 2009 03:32:42 +0000 (UTC) (envelope-from marcel@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 5E44B8FC12; Tue, 24 Nov 2009 03:32:42 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAO3WgMH004133; Tue, 24 Nov 2009 03:32:42 GMT (envelope-from marcel@svn.freebsd.org) Received: (from marcel@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAO3WgnK004124; Tue, 24 Nov 2009 03:32:42 GMT (envelope-from marcel@svn.freebsd.org) Message-Id: <200911240332.nAO3WgnK004124@svn.freebsd.org> From: Marcel Moolenaar Date: Tue, 24 Nov 2009 03:32:42 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-8@freebsd.org X-SVN-Group: stable-8 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199732 - in stable/8/sys: conf ia64/ia64 ia64/include X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Nov 2009 03:32:42 -0000 Author: marcel Date: Tue Nov 24 03:32:42 2009 New Revision: 199732 URL: http://svn.freebsd.org/changeset/base/199732 Log: MFC r198733: Reimplement the lazy FP context switching... ...This change fixes the high FP inconsistency panics. Added: stable/8/sys/ia64/ia64/highfp.c - copied unchanged from r198733, head/sys/ia64/ia64/highfp.c Modified: stable/8/sys/conf/files.ia64 stable/8/sys/ia64/ia64/interrupt.c stable/8/sys/ia64/ia64/machdep.c stable/8/sys/ia64/ia64/trap.c stable/8/sys/ia64/ia64/vm_machdep.c stable/8/sys/ia64/include/md_var.h stable/8/sys/ia64/include/proc.h Directory Properties: stable/8/sys/ (props changed) Modified: stable/8/sys/conf/files.ia64 ============================================================================== --- stable/8/sys/conf/files.ia64 Tue Nov 24 03:28:35 2009 (r199731) +++ stable/8/sys/conf/files.ia64 Tue Nov 24 03:32:42 2009 (r199732) @@ -85,6 +85,7 @@ ia64/ia64/elf_machdep.c standard ia64/ia64/emulate.c standard ia64/ia64/exception.S standard ia64/ia64/gdb_machdep.c optional gdb +ia64/ia64/highfp.c standard ia64/ia64/in_cksum.c optional inet ia64/ia64/interrupt.c standard ia64/ia64/locore.S standard no-obj Copied: stable/8/sys/ia64/ia64/highfp.c (from r198733, head/sys/ia64/ia64/highfp.c) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ stable/8/sys/ia64/ia64/highfp.c Tue Nov 24 03:32:42 2009 (r199732, copy of r198733, head/sys/ia64/ia64/highfp.c) @@ -0,0 +1,181 @@ +/*- + * Copyright (c) 2009 Marcel Moolenaar + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +__FBSDID("$FreeBSD$"); + +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +static struct mtx ia64_highfp_mtx; + +static void +ia64_highfp_init(void *_) +{ + mtx_init(&ia64_highfp_mtx, "High FP lock", NULL, MTX_SPIN); +} +SYSINIT(ia64_highfp_init, SI_SUB_LOCK, SI_ORDER_ANY, ia64_highfp_init, NULL); + +#ifdef SMP +static int +ia64_highfp_ipi(struct pcpu *cpu) +{ + int error; + + ipi_send(cpu, IPI_HIGH_FP); + error = msleep_spin(&cpu->pc_fpcurthread, &ia64_highfp_mtx, + "High FP", 0); + return (error); +} +#endif + +int +ia64_highfp_drop(struct thread *td) +{ + struct pcb *pcb; + struct pcpu *cpu; + + pcb = td->td_pcb; + + mtx_lock_spin(&ia64_highfp_mtx); + cpu = pcb->pcb_fpcpu; + if (cpu != NULL) { + KASSERT(cpu->pc_fpcurthread == td, + ("cpu->pc_fpcurthread != td")); + td->td_frame->tf_special.psr |= IA64_PSR_DFH; + pcb->pcb_fpcpu = NULL; + cpu->pc_fpcurthread = NULL; + } + mtx_unlock_spin(&ia64_highfp_mtx); + + return ((cpu != NULL) ? 1 : 0); +} + +int +ia64_highfp_enable(struct thread *td, struct trapframe *tf) +{ + struct pcb *pcb; + struct pcpu *cpu; + struct thread *td1; + + pcb = td->td_pcb; + + mtx_lock_spin(&ia64_highfp_mtx); + KASSERT((tf->tf_special.psr & IA64_PSR_DFH) != 0, + ("(tf->tf_special.psr & IA64_PSR_DFH) == 0")); + cpu = pcb->pcb_fpcpu; +#ifdef SMP + if (cpu != NULL && cpu != pcpup) { + KASSERT(cpu->pc_fpcurthread == td, + ("cpu->pc_fpcurthread != td")); + ia64_highfp_ipi(cpu); + } +#endif + td1 = PCPU_GET(fpcurthread); + if (td1 != NULL && td1 != td) { + KASSERT(td1->td_pcb->pcb_fpcpu == pcpup, + ("td1->td_pcb->pcb_fpcpu != pcpup")); + save_high_fp(&td1->td_pcb->pcb_high_fp); + td1->td_frame->tf_special.psr |= IA64_PSR_DFH; + td1->td_pcb->pcb_fpcpu = NULL; + PCPU_SET(fpcurthread, NULL); + td1 = NULL; + } + if (td1 == NULL) { + KASSERT(pcb->pcb_fpcpu == NULL, ("pcb->pcb_fpcpu != NULL")); + KASSERT(PCPU_GET(fpcurthread) == NULL, + ("PCPU_GET(fpcurthread) != NULL")); + restore_high_fp(&pcb->pcb_high_fp); + PCPU_SET(fpcurthread, td); + pcb->pcb_fpcpu = pcpup; + tf->tf_special.psr &= ~IA64_PSR_MFH; + } + tf->tf_special.psr &= ~IA64_PSR_DFH; + mtx_unlock_spin(&ia64_highfp_mtx); + + return ((td1 != NULL) ? 1 : 0); +} + +int +ia64_highfp_save(struct thread *td) +{ + struct pcb *pcb; + struct pcpu *cpu; + + pcb = td->td_pcb; + + mtx_lock_spin(&ia64_highfp_mtx); + cpu = pcb->pcb_fpcpu; +#ifdef SMP + if (cpu != NULL && cpu != pcpup) { + KASSERT(cpu->pc_fpcurthread == td, + ("cpu->pc_fpcurthread != td")); + ia64_highfp_ipi(cpu); + } else +#endif + if (cpu != NULL) { + KASSERT(cpu->pc_fpcurthread == td, + ("cpu->pc_fpcurthread != td")); + save_high_fp(&pcb->pcb_high_fp); + td->td_frame->tf_special.psr |= IA64_PSR_DFH; + pcb->pcb_fpcpu = NULL; + cpu->pc_fpcurthread = NULL; + } + mtx_unlock_spin(&ia64_highfp_mtx); + + return ((cpu != NULL) ? 1 : 0); +} + +#ifdef SMP +int +ia64_highfp_save_ipi(void) +{ + struct thread *td; + + mtx_lock_spin(&ia64_highfp_mtx); + td = PCPU_GET(fpcurthread); + if (td != NULL) { + KASSERT(td->td_pcb->pcb_fpcpu == pcpup, + ("td->td_pcb->pcb_fpcpu != pcpup")); + save_high_fp(&td->td_pcb->pcb_high_fp); + td->td_frame->tf_special.psr |= IA64_PSR_DFH; + td->td_pcb->pcb_fpcpu = NULL; + PCPU_SET(fpcurthread, NULL); + } + mtx_unlock_spin(&ia64_highfp_mtx); + wakeup(&PCPU_GET(fpcurthread)); + + return ((td != NULL) ? 1 : 0); +} +#endif Modified: stable/8/sys/ia64/ia64/interrupt.c ============================================================================== --- stable/8/sys/ia64/ia64/interrupt.c Tue Nov 24 03:28:35 2009 (r199731) +++ stable/8/sys/ia64/ia64/interrupt.c Tue Nov 24 03:32:42 2009 (r199732) @@ -216,14 +216,7 @@ interrupt(struct trapframe *tf) asts[PCPU_GET(cpuid)]++; CTR1(KTR_SMP, "IPI_AST, cpuid=%d", PCPU_GET(cpuid)); } else if (vector == ipi_vector[IPI_HIGH_FP]) { - struct thread *thr = PCPU_GET(fpcurthread); - if (thr != NULL) { - mtx_lock_spin(&thr->td_md.md_highfp_mtx); - save_high_fp(&thr->td_pcb->pcb_high_fp); - thr->td_pcb->pcb_fpcpu = NULL; - PCPU_SET(fpcurthread, NULL); - mtx_unlock_spin(&thr->td_md.md_highfp_mtx); - } + ia64_highfp_save_ipi(); } else if (vector == ipi_vector[IPI_RENDEZVOUS]) { rdvs[PCPU_GET(cpuid)]++; CTR1(KTR_SMP, "IPI_RENDEZVOUS, cpuid=%d", PCPU_GET(cpuid)); Modified: stable/8/sys/ia64/ia64/machdep.c ============================================================================== --- stable/8/sys/ia64/ia64/machdep.c Tue Nov 24 03:28:35 2009 (r199731) +++ stable/8/sys/ia64/ia64/machdep.c Tue Nov 24 03:32:42 2009 (r199732) @@ -1467,81 +1467,6 @@ set_fpregs(struct thread *td, struct fpr return (0); } -/* - * High FP register functions. - */ - -int -ia64_highfp_drop(struct thread *td) -{ - struct pcb *pcb; - struct pcpu *cpu; - struct thread *thr; - - mtx_lock_spin(&td->td_md.md_highfp_mtx); - pcb = td->td_pcb; - cpu = pcb->pcb_fpcpu; - if (cpu == NULL) { - mtx_unlock_spin(&td->td_md.md_highfp_mtx); - return (0); - } - pcb->pcb_fpcpu = NULL; - thr = cpu->pc_fpcurthread; - cpu->pc_fpcurthread = NULL; - mtx_unlock_spin(&td->td_md.md_highfp_mtx); - - /* Post-mortem sanity checking. */ - KASSERT(thr == td, ("Inconsistent high FP state")); - return (1); -} - -int -ia64_highfp_save(struct thread *td) -{ - struct pcb *pcb; - struct pcpu *cpu; - struct thread *thr; - - /* Don't save if the high FP registers weren't modified. */ - if ((td->td_frame->tf_special.psr & IA64_PSR_MFH) == 0) - return (ia64_highfp_drop(td)); - - mtx_lock_spin(&td->td_md.md_highfp_mtx); - pcb = td->td_pcb; - cpu = pcb->pcb_fpcpu; - if (cpu == NULL) { - mtx_unlock_spin(&td->td_md.md_highfp_mtx); - return (0); - } -#ifdef SMP - if (td == curthread) - sched_pin(); - if (cpu != pcpup) { - mtx_unlock_spin(&td->td_md.md_highfp_mtx); - ipi_send(cpu, IPI_HIGH_FP); - if (td == curthread) - sched_unpin(); - while (pcb->pcb_fpcpu == cpu) - DELAY(100); - return (1); - } else { - save_high_fp(&pcb->pcb_high_fp); - if (td == curthread) - sched_unpin(); - } -#else - save_high_fp(&pcb->pcb_high_fp); -#endif - pcb->pcb_fpcpu = NULL; - thr = cpu->pc_fpcurthread; - cpu->pc_fpcurthread = NULL; - mtx_unlock_spin(&td->td_md.md_highfp_mtx); - - /* Post-mortem sanity cxhecking. */ - KASSERT(thr == td, ("Inconsistent high FP state")); - return (1); -} - void ia64_sync_icache(vm_offset_t va, vm_offset_t sz) { Modified: stable/8/sys/ia64/ia64/trap.c ============================================================================== --- stable/8/sys/ia64/ia64/trap.c Tue Nov 24 03:28:35 2009 (r199731) +++ stable/8/sys/ia64/ia64/trap.c Tue Nov 24 03:32:42 2009 (r199732) @@ -652,66 +652,10 @@ trap(int vector, struct trapframe *tf) break; case IA64_VEC_DISABLED_FP: { - struct pcpu *pcpu; - struct pcb *pcb; - struct thread *thr; - - /* Always fatal in kernel. Should never happen. */ - if (!user) + if (user) + ia64_highfp_enable(td, tf); + else trap_panic(vector, tf); - - sched_pin(); - thr = PCPU_GET(fpcurthread); - if (thr == td) { - /* - * Short-circuit handling the trap when this CPU - * already holds the high FP registers for this - * thread. We really shouldn't get the trap in the - * first place, but since it's only a performance - * issue and not a correctness issue, we emit a - * message for now, enable the high FP registers and - * return. - */ - printf("XXX: bogusly disabled high FP regs\n"); - tf->tf_special.psr &= ~IA64_PSR_DFH; - sched_unpin(); - goto out; - } else if (thr != NULL) { - mtx_lock_spin(&thr->td_md.md_highfp_mtx); - pcb = thr->td_pcb; - save_high_fp(&pcb->pcb_high_fp); - pcb->pcb_fpcpu = NULL; - PCPU_SET(fpcurthread, NULL); - mtx_unlock_spin(&thr->td_md.md_highfp_mtx); - thr = NULL; - } - - mtx_lock_spin(&td->td_md.md_highfp_mtx); - pcb = td->td_pcb; - pcpu = pcb->pcb_fpcpu; - -#ifdef SMP - if (pcpu != NULL) { - mtx_unlock_spin(&td->td_md.md_highfp_mtx); - ipi_send(pcpu, IPI_HIGH_FP); - while (pcb->pcb_fpcpu == pcpu) - DELAY(100); - mtx_lock_spin(&td->td_md.md_highfp_mtx); - pcpu = pcb->pcb_fpcpu; - thr = PCPU_GET(fpcurthread); - } -#endif - - if (thr == NULL && pcpu == NULL) { - restore_high_fp(&pcb->pcb_high_fp); - PCPU_SET(fpcurthread, td); - pcb->pcb_fpcpu = pcpup; - tf->tf_special.psr &= ~IA64_PSR_MFH; - tf->tf_special.psr &= ~IA64_PSR_DFH; - } - - mtx_unlock_spin(&td->td_md.md_highfp_mtx); - sched_unpin(); goto out; } Modified: stable/8/sys/ia64/ia64/vm_machdep.c ============================================================================== --- stable/8/sys/ia64/ia64/vm_machdep.c Tue Nov 24 03:28:35 2009 (r199731) +++ stable/8/sys/ia64/ia64/vm_machdep.c Tue Nov 24 03:32:42 2009 (r199732) @@ -120,14 +120,11 @@ cpu_thread_alloc(struct thread *td) sp -= sizeof(struct trapframe); td->td_frame = (struct trapframe *)sp; td->td_frame->tf_length = sizeof(struct trapframe); - mtx_init(&td->td_md.md_highfp_mtx, "High FP lock", NULL, MTX_SPIN); } void cpu_thread_free(struct thread *td) { - - mtx_destroy(&td->td_md.md_highfp_mtx); } void @@ -148,6 +145,8 @@ cpu_set_upcall(struct thread *td, struct struct pcb *pcb; struct trapframe *tf; + ia64_highfp_save(td0); + tf = td->td_frame; KASSERT(tf != NULL, ("foo")); bcopy(td0->td_frame, tf, sizeof(*tf)); Modified: stable/8/sys/ia64/include/md_var.h ============================================================================== --- stable/8/sys/ia64/include/md_var.h Tue Nov 24 03:28:35 2009 (r199731) +++ stable/8/sys/ia64/include/md_var.h Tue Nov 24 03:32:42 2009 (r199732) @@ -86,7 +86,9 @@ int ia64_emulate(struct trapframe *, str int ia64_flush_dirty(struct thread *, struct _special *); uint64_t ia64_get_hcdp(void); int ia64_highfp_drop(struct thread *); +int ia64_highfp_enable(struct thread *, struct trapframe *); int ia64_highfp_save(struct thread *); +int ia64_highfp_save_ipi(void); struct ia64_init_return ia64_init(void); void ia64_probe_sapics(void); void ia64_sync_icache(vm_offset_t, vm_size_t); Modified: stable/8/sys/ia64/include/proc.h ============================================================================== --- stable/8/sys/ia64/include/proc.h Tue Nov 24 03:28:35 2009 (r199731) +++ stable/8/sys/ia64/include/proc.h Tue Nov 24 03:32:42 2009 (r199732) @@ -30,7 +30,6 @@ #define _MACHINE_PROC_H_ struct mdthread { - struct mtx md_highfp_mtx; int md_spinlock_count; /* (k) */ int md_saved_intr; /* (k) */ };