From owner-p4-projects@FreeBSD.ORG Thu Mar 2 08:00:19 2006 Return-Path: X-Original-To: p4-projects@freebsd.org Delivered-To: p4-projects@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 32767) id ACCE316A424; Thu, 2 Mar 2006 08:00:18 +0000 (GMT) X-Original-To: perforce@freebsd.org Delivered-To: perforce@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4467D16A420 for ; Thu, 2 Mar 2006 08:00:18 +0000 (GMT) (envelope-from imp@freebsd.org) Received: from repoman.freebsd.org (repoman.freebsd.org [216.136.204.115]) by mx1.FreeBSD.org (Postfix) with ESMTP id DC05743D46 for ; Thu, 2 Mar 2006 08:00:17 +0000 (GMT) (envelope-from imp@freebsd.org) Received: from repoman.freebsd.org (localhost [127.0.0.1]) by repoman.freebsd.org (8.13.1/8.13.1) with ESMTP id k2280Hin012952 for ; Thu, 2 Mar 2006 08:00:17 GMT (envelope-from imp@freebsd.org) Received: (from perforce@localhost) by repoman.freebsd.org (8.13.1/8.13.1/Submit) id k2280GM5012949 for perforce@freebsd.org; Thu, 2 Mar 2006 08:00:16 GMT (envelope-from imp@freebsd.org) Date: Thu, 2 Mar 2006 08:00:16 GMT Message-Id: <200603020800.k2280GM5012949@repoman.freebsd.org> X-Authentication-Warning: repoman.freebsd.org: perforce set sender to imp@freebsd.org using -f From: Warner Losh To: Perforce Change Reviews Cc: Subject: PERFORCE change 92639 for review X-BeenThere: p4-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: p4 projects tree changes List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Mar 2006 08:00:19 -0000 http://perforce.freebsd.org/chv.cgi?CH=92639 Change 92639 by imp@imp_harmony on 2006/03/02 07:59:25 IFC @92637 Affected files ... .. //depot/projects/arm/src/sys/amd64/amd64/intr_machdep.c#5 integrate .. //depot/projects/arm/src/sys/amd64/amd64/io_apic.c#4 integrate .. //depot/projects/arm/src/sys/amd64/amd64/local_apic.c#8 integrate .. //depot/projects/arm/src/sys/amd64/amd64/mp_machdep.c#7 integrate .. //depot/projects/arm/src/sys/amd64/include/apicvar.h#6 integrate .. //depot/projects/arm/src/sys/amd64/include/intr_machdep.h#4 integrate .. //depot/projects/arm/src/sys/amd64/include/mptable.h#2 integrate .. //depot/projects/arm/src/sys/amd64/isa/atpic.c#4 integrate .. //depot/projects/arm/src/sys/arm/arm/busdma_machdep.c#7 integrate .. //depot/projects/arm/src/sys/arm/arm/pmap.c#9 integrate .. //depot/projects/arm/src/sys/arm/arm/trap.c#8 integrate .. //depot/projects/arm/src/sys/arm/arm/undefined.c#7 integrate .. //depot/projects/arm/src/sys/arm/arm/vm_machdep.c#5 integrate .. //depot/projects/arm/src/sys/arm/include/pmap.h#5 integrate .. //depot/projects/arm/src/sys/compat/freebsd32/freebsd32_misc.c#7 integrate .. //depot/projects/arm/src/sys/compat/freebsd32/freebsd32_proto.h#8 integrate .. //depot/projects/arm/src/sys/compat/freebsd32/freebsd32_syscall.h#8 integrate .. //depot/projects/arm/src/sys/compat/freebsd32/freebsd32_syscalls.c#8 integrate .. //depot/projects/arm/src/sys/compat/freebsd32/freebsd32_sysent.c#8 integrate .. //depot/projects/arm/src/sys/compat/freebsd32/syscalls.master#8 integrate .. //depot/projects/arm/src/sys/contrib/altq/altq/altq_subr.c#3 integrate .. //depot/projects/arm/src/sys/dev/ata/ata-all.h#10 integrate .. //depot/projects/arm/src/sys/dev/ata/ata-queue.c#9 integrate .. //depot/projects/arm/src/sys/dev/ce/tau32-ddk.c#2 integrate .. //depot/projects/arm/src/sys/dev/ce/tau32-ddk.h#2 integrate .. //depot/projects/arm/src/sys/dev/iir/iir.c#3 integrate .. //depot/projects/arm/src/sys/dev/iir/iir.h#4 integrate .. //depot/projects/arm/src/sys/dev/iir/iir_pci.c#4 integrate .. //depot/projects/arm/src/sys/dev/kbd/kbd.c#3 integrate .. //depot/projects/arm/src/sys/dev/kbdmux/kbdmux.c#3 integrate .. //depot/projects/arm/src/sys/dev/mpt/mpt.c#6 integrate .. //depot/projects/arm/src/sys/dev/mpt/mpt_cam.c#7 integrate .. //depot/projects/arm/src/sys/dev/syscons/syscons.c#6 integrate .. //depot/projects/arm/src/sys/dev/usb/usb_port.h#5 integrate .. //depot/projects/arm/src/sys/geom/geom_io.c#3 integrate .. //depot/projects/arm/src/sys/geom/geom_vfs.c#3 integrate .. //depot/projects/arm/src/sys/i386/i386/intr_machdep.c#5 integrate .. //depot/projects/arm/src/sys/i386/i386/io_apic.c#4 integrate .. //depot/projects/arm/src/sys/i386/i386/local_apic.c#7 integrate .. //depot/projects/arm/src/sys/i386/i386/mp_machdep.c#7 integrate .. //depot/projects/arm/src/sys/i386/include/apicvar.h#5 integrate .. //depot/projects/arm/src/sys/i386/include/intr_machdep.h#4 integrate .. //depot/projects/arm/src/sys/i386/isa/atpic.c#4 integrate .. //depot/projects/arm/src/sys/kern/init_sysent.c#9 integrate .. //depot/projects/arm/src/sys/kern/kern_time.c#6 integrate .. //depot/projects/arm/src/sys/kern/syscalls.c#9 integrate .. //depot/projects/arm/src/sys/kern/syscalls.master#10 integrate .. //depot/projects/arm/src/sys/kern/uipc_mqueue.c#5 integrate .. //depot/projects/arm/src/sys/kern/uipc_syscalls.c#6 integrate .. //depot/projects/arm/src/sys/kern/vfs_subr.c#12 integrate .. //depot/projects/arm/src/sys/modules/kbdmux/Makefile#2 integrate .. //depot/projects/arm/src/sys/netinet/tcp_input.c#8 integrate .. //depot/projects/arm/src/sys/netinet/tcp_syncache.c#8 integrate .. //depot/projects/arm/src/sys/netinet/tcp_var.h#5 integrate .. //depot/projects/arm/src/sys/nfsserver/nfs_srvsock.c#4 integrate .. //depot/projects/arm/src/sys/sparc64/sparc64/trap.c#5 integrate .. //depot/projects/arm/src/sys/sys/_types.h#4 integrate .. //depot/projects/arm/src/sys/sys/bio.h#4 integrate .. //depot/projects/arm/src/sys/sys/mount.h#8 integrate .. //depot/projects/arm/src/sys/sys/proc.h#11 integrate .. //depot/projects/arm/src/sys/sys/syscall.h#9 integrate .. //depot/projects/arm/src/sys/sys/syscall.mk#9 integrate .. //depot/projects/arm/src/sys/sys/syscallsubr.h#5 integrate .. //depot/projects/arm/src/sys/sys/sysproto.h#9 integrate .. //depot/projects/arm/src/sys/sys/timers.h#4 integrate .. //depot/projects/arm/src/sys/sys/vnode.h#9 integrate .. //depot/projects/arm/src/sys/ufs/ffs/ffs_alloc.c#4 integrate .. //depot/projects/arm/src/sys/ufs/ffs/ffs_extern.h#3 integrate .. //depot/projects/arm/src/sys/ufs/ffs/ffs_snapshot.c#8 integrate .. //depot/projects/arm/src/sys/ufs/ffs/ffs_softdep.c#7 integrate .. //depot/projects/arm/src/sys/ufs/ffs/softdep.h#3 integrate .. //depot/projects/arm/src/sys/ufs/ufs/ufsmount.h#2 integrate Differences ... ==== //depot/projects/arm/src/sys/amd64/amd64/intr_machdep.c#5 (text+ko) ==== @@ -26,7 +26,7 @@ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * - * $FreeBSD: src/sys/amd64/amd64/intr_machdep.c,v 1.18 2005/12/08 18:33:29 jhb Exp $ + * $FreeBSD: src/sys/amd64/amd64/intr_machdep.c,v 1.19 2006/02/28 22:24:54 jhb Exp $ */ /* @@ -63,6 +63,12 @@ static struct intsrc *interrupt_sources[NUM_IO_INTS]; static struct mtx intr_table_lock; +#ifdef SMP +static int assign_cpu; + +static void intr_assign_next_cpu(struct intsrc *isrc); +#endif + static void intr_init(void *__dummy); static void intrcnt_setname(const char *name, int index); static void intrcnt_updatename(struct intsrc *is); @@ -93,6 +99,7 @@ } intrcnt_register(isrc); interrupt_sources[vector] = isrc; + isrc->is_enabled = 0; mtx_unlock_spin(&intr_table_lock); return (0); } @@ -118,7 +125,17 @@ intr_priority(flags), flags, cookiep); if (error == 0) { intrcnt_updatename(isrc); - isrc->is_pic->pic_enable_intr(isrc); + mtx_lock_spin(&intr_table_lock); + if (!isrc->is_enabled) { + isrc->is_enabled = 1; +#ifdef SMP + if (assign_cpu) + intr_assign_next_cpu(isrc); +#endif + mtx_unlock_spin(&intr_table_lock); + isrc->is_pic->pic_enable_intr(isrc); + } else + mtx_unlock_spin(&intr_table_lock); isrc->is_pic->pic_enable_source(isrc); } return (error); @@ -335,3 +352,76 @@ db_dump_intr_event((*isrc)->is_event, verbose); } #endif + +#ifdef SMP +/* + * Support for balancing interrupt sources across CPUs. For now we just + * allocate CPUs round-robin. + */ + +static u_int cpu_apic_ids[MAXCPU]; +static int current_cpu, num_cpus; + +static void +intr_assign_next_cpu(struct intsrc *isrc) +{ + struct pic *pic; + u_int apic_id; + + /* + * Assign this source to a local APIC in a round-robin fashion. + */ + pic = isrc->is_pic; + apic_id = cpu_apic_ids[current_cpu]; + current_cpu++; + if (current_cpu >= num_cpus) + current_cpu = 0; + if (bootverbose) { + printf("INTR: Assigning IRQ %d", pic->pic_vector(isrc)); + printf(" to local APIC %u\n", apic_id); + } + pic->pic_assign_cpu(isrc, apic_id); +} + +/* + * Add a local APIC ID to our list of valid local APIC IDs that can + * be destinations of interrupts. + */ +void +intr_add_cpu(u_int apic_id) +{ + + if (bootverbose) + printf("INTR: Adding local APIC %d as a target\n", apic_id); + if (num_cpus >= MAXCPU) + panic("WARNING: Local APIC IDs exhausted!"); + cpu_apic_ids[num_cpus] = apic_id; + num_cpus++; +} + +/* + * Distribute all the interrupt sources among the available CPUs once the + * AP's have been launched. + */ +static void +intr_shuffle_irqs(void *arg __unused) +{ + struct intsrc *isrc; + int i; + + /* Don't bother on UP. */ + if (num_cpus <= 1) + return; + + /* Round-robin assign each enabled source a CPU. */ + mtx_lock_spin(&intr_table_lock); + assign_cpu = 1; + for (i = 0; i < NUM_IO_INTS; i++) { + isrc = interrupt_sources[i]; + if (isrc != NULL && isrc->is_enabled) + intr_assign_next_cpu(isrc); + } + mtx_unlock_spin(&intr_table_lock); +} +SYSINIT(intr_shuffle_irqs, SI_SUB_SMP, SI_ORDER_SECOND, intr_shuffle_irqs, NULL) +#endif ==== //depot/projects/arm/src/sys/amd64/amd64/io_apic.c#4 (text+ko) ==== @@ -28,7 +28,7 @@ */ #include -__FBSDID("$FreeBSD: src/sys/amd64/amd64/io_apic.c,v 1.20 2005/11/16 20:29:29 jhb Exp $"); +__FBSDID("$FreeBSD: src/sys/amd64/amd64/io_apic.c,v 1.21 2006/02/28 22:24:54 jhb Exp $"); #include "opt_atpic.h" #include "opt_isa.h" @@ -61,8 +61,6 @@ #define IRQ_SMI (NUM_IO_INTS + 3) #define IRQ_DISABLED (NUM_IO_INTS + 4) -#define DEST_NONE -1 - #define TODO printf("%s: not implemented!\n", __func__) static MALLOC_DEFINE(M_IOAPIC, "io_apic", "I/O APIC structures"); @@ -82,10 +80,10 @@ u_int io_irq; u_int io_intpin:8; u_int io_vector:8; + u_int io_cpu:8; u_int io_activehi:1; u_int io_edgetrigger:1; u_int io_masked:1; - int io_dest:5; int io_bus:4; }; @@ -114,7 +112,7 @@ enum intr_polarity pol); static void ioapic_suspend(struct intsrc *isrc); static void ioapic_resume(struct intsrc *isrc); -static void ioapic_program_destination(struct ioapic_intsrc *intpin); +static void ioapic_assign_cpu(struct intsrc *isrc, u_int apic_id); static void ioapic_program_intpin(struct ioapic_intsrc *intpin); static STAILQ_HEAD(,ioapic) ioapic_list = STAILQ_HEAD_INITIALIZER(ioapic_list); @@ -122,10 +120,10 @@ ioapic_eoi_source, ioapic_enable_intr, ioapic_vector, ioapic_source_pending, ioapic_suspend, ioapic_resume, - ioapic_config_intr }; - -static int bsp_id, current_cluster, logical_clusters, next_ioapic_base; -static u_int next_id, program_logical_dest; + ioapic_config_intr, ioapic_assign_cpu }; + +static int next_ioapic_base; +static u_int next_id; SYSCTL_NODE(_hw, OID_AUTO, apic, CTLFLAG_RD, 0, "APIC options"); static int enable_extint; @@ -273,14 +271,8 @@ } /* Set the destination. */ - if (intpin->io_dest == DEST_NONE) { - low = IOART_DESTPHY; - high = bsp_id << APIC_ID_SHIFT; - } else { - low = IOART_DESTLOG; - high = (intpin->io_dest << APIC_ID_CLUSTER_SHIFT | - APIC_ID_CLUSTER_ID) << APIC_ID_SHIFT; - } + low = IOART_DESTPHY; + high = intpin->io_cpu << APIC_ID_SHIFT; /* Program the rest of the low word. */ if (intpin->io_edgetrigger) @@ -312,7 +304,7 @@ default: KASSERT(intpin->io_vector != 0, ("No vector for IRQ %u", intpin->io_irq)); - low |= IOART_DELLOPRI | intpin->io_vector; + low |= IOART_DELFIXED | intpin->io_vector; } /* Write the values to the APIC. */ @@ -325,60 +317,31 @@ mtx_unlock_spin(&icu_lock); } -/* - * Program an individual intpin's logical destination. - */ static void -ioapic_program_destination(struct ioapic_intsrc *intpin) +ioapic_assign_cpu(struct intsrc *isrc, u_int apic_id) { - struct ioapic *io = (struct ioapic *)intpin->io_intsrc.is_pic; + struct ioapic_intsrc *intpin = (struct ioapic_intsrc *)isrc; + struct ioapic *io = (struct ioapic *)isrc->is_pic; - KASSERT(intpin->io_dest != DEST_NONE, - ("intpin not assigned to a cluster")); + intpin->io_cpu = apic_id; if (bootverbose) { - printf("ioapic%u: routing intpin %u (", io->io_id, - intpin->io_intpin); + printf("ioapic%u: Assigning ", io->io_id); ioapic_print_irq(intpin); - printf(") to cluster %u\n", intpin->io_dest); + printf(" to local APIC %u\n", intpin->io_cpu); } ioapic_program_intpin(intpin); } static void -ioapic_assign_cluster(struct ioapic_intsrc *intpin) -{ - - /* - * Assign this intpin to a logical APIC cluster in a - * round-robin fashion. We don't actually use the logical - * destination for this intpin until after all the CPU's - * have been started so that we don't end up with interrupts - * that don't go anywhere. Another alternative might be to - * start up the CPU's earlier so that they can handle interrupts - * sooner. - */ - intpin->io_dest = current_cluster; - current_cluster++; - if (current_cluster >= logical_clusters) - current_cluster = 0; - if (program_logical_dest) - ioapic_program_destination(intpin); -} - -static void ioapic_enable_intr(struct intsrc *isrc) { struct ioapic_intsrc *intpin = (struct ioapic_intsrc *)isrc; struct ioapic *io = (struct ioapic *)isrc->is_pic; - if (intpin->io_dest == DEST_NONE) { + if (intpin->io_vector == 0) { /* * Allocate an APIC vector for this interrupt pin. Once - * we have a vector we program the interrupt pin. Note - * that after we have booted ioapic_assign_cluster() - * will program the interrupt pin again, but it doesn't - * hurt to do that and trying to avoid that adds needless - * complication. + * we have a vector we program the interrupt pin. */ intpin->io_vector = apic_alloc_vector(intpin->io_irq); if (bootverbose) { @@ -388,7 +351,6 @@ printf(") to vector %u\n", intpin->io_vector); } ioapic_program_intpin(intpin); - ioapic_assign_cluster(intpin); apic_enable_vector(intpin->io_vector); } } @@ -469,22 +431,6 @@ } /* - * Allocate and return a logical cluster ID. Note that the first time - * this is called, it returns cluster 0. ioapic_enable_intr() treats - * the two cases of logical_clusters == 0 and logical_clusters == 1 the - * same: one cluster of ID 0 exists. The logical_clusters == 0 case is - * for UP kernels, which should never call this function. - */ -int -ioapic_next_logical_cluster(void) -{ - - if (logical_clusters >= APIC_MAX_CLUSTER) - panic("WARNING: Local APIC cluster IDs exhausted!"); - return (logical_clusters++); -} - -/* * Create a plain I/O APIC object. */ void * @@ -568,11 +514,10 @@ } /* - * Route interrupts to the BSP by default using physical - * addressing. Vectored interrupts get readdressed using - * logical IDs to CPU clusters when they are enabled. + * Route interrupts to the BSP by default. Interrupts may + * be routed to other CPUs later after they are enabled. */ - intpin->io_dest = DEST_NONE; + intpin->io_cpu = PCPU_GET(apic_id); if (bootverbose && intpin->io_irq != IRQ_DISABLED) { printf("ioapic%u: intpin %d -> ", io->io_id, i); ioapic_print_irq(intpin); @@ -778,29 +723,9 @@ printf("ioapic%u irqs %u-%u on motherboard\n", io->io_id, flags >> 4, flags & 0xf, io->io_intbase, io->io_intbase + io->io_numintr - 1); - bsp_id = PCPU_GET(apic_id); /* Register valid pins as interrupt sources. */ for (i = 0, pin = io->io_pins; i < io->io_numintr; i++, pin++) if (pin->io_irq < NUM_IO_INTS) intr_register_source(&pin->io_intsrc); } - -/* - * Program all the intpins to use logical destinations once the AP's - * have been launched. - */ -static void -ioapic_set_logical_destinations(void *arg __unused) -{ - struct ioapic *io; - int i; - - program_logical_dest = 1; - STAILQ_FOREACH(io, &ioapic_list, io_next) - for (i = 0; i < io->io_numintr; i++) - if (io->io_pins[i].io_dest != DEST_NONE) - ioapic_program_destination(&io->io_pins[i]); -} -SYSINIT(ioapic_destinations, SI_SUB_SMP, SI_ORDER_SECOND, - ioapic_set_logical_destinations, NULL) ==== //depot/projects/arm/src/sys/amd64/amd64/local_apic.c#8 (text+ko) ==== @@ -32,7 +32,7 @@ */ #include -__FBSDID("$FreeBSD: src/sys/amd64/amd64/local_apic.c,v 1.23 2005/12/22 22:16:05 jhb Exp $"); +__FBSDID("$FreeBSD: src/sys/amd64/amd64/local_apic.c,v 1.24 2006/02/28 22:24:54 jhb Exp $"); #include "opt_hwpmc_hooks.h" @@ -217,6 +217,7 @@ /* Set BSP's per-CPU local APIC ID. */ PCPU_SET(apic_id, lapic_id()); + intr_add_cpu(PCPU_GET(apic_id)); /* Local APIC timer interrupt. */ setidt(APIC_TIMER_INT, IDTVEC(timerint), SDT_SYSIGT, SEL_KPL, 0); @@ -279,7 +280,7 @@ lapic_setup(void) { struct lapic *la; - u_int32_t value, maxlvt; + u_int32_t maxlvt; register_t eflags; char buf[MAXCOMLEN + 1]; @@ -291,19 +292,6 @@ /* Initialize the TPR to allow all interrupts. */ lapic_set_tpr(0); - /* Use the cluster model for logical IDs. */ - value = lapic->dfr; - value &= ~APIC_DFR_MODEL_MASK; - value |= APIC_DFR_MODEL_CLUSTER; - lapic->dfr = value; - - /* Set this APIC's logical ID. */ - value = lapic->ldr; - value &= ~APIC_ID_MASK; - value |= (la->la_cluster << APIC_ID_CLUSTER_SHIFT | - 1 << la->la_cluster_id) << APIC_ID_SHIFT; - lapic->ldr = value; - /* Setup spurious vector and enable the local APIC. */ lapic_enable(); ==== //depot/projects/arm/src/sys/amd64/amd64/mp_machdep.c#7 (text+ko) ==== @@ -25,7 +25,7 @@ */ #include -__FBSDID("$FreeBSD: src/sys/amd64/amd64/mp_machdep.c,v 1.268 2006/02/07 21:22:01 phk Exp $"); +__FBSDID("$FreeBSD: src/sys/amd64/amd64/mp_machdep.c,v 1.269 2006/02/28 22:24:54 jhb Exp $"); #include "opt_cpu.h" #include "opt_kstack_pages.h" @@ -160,7 +160,7 @@ static u_int boot_address; -static void set_logical_apic_ids(void); +static void set_interrupt_apic_ids(void); static int start_all_aps(void); static int start_ap(int apic_id); static void release_aps(void *dummy); @@ -405,7 +405,7 @@ hyperthreading_cpus = logical_cpus; } - set_logical_apic_ids(); + set_interrupt_apic_ids(); } @@ -596,33 +596,29 @@ */ /* - * Set the APIC logical IDs. - * - * We want to cluster logical CPU's within the same APIC ID cluster. - * Since logical CPU's are aligned simply filling in the clusters in - * APIC ID order works fine. Note that this does not try to balance - * the number of CPU's in each cluster. (XXX?) + * We tell the I/O APIC code about all the CPUs we want to receive + * interrupts. If we don't want certain CPUs to receive IRQs we + * can simply not tell the I/O APIC code about them in this function. + * We also do not tell it about the BSP since it tells itself about + * the BSP internally to work with UP kernels and on UP machines. */ static void -set_logical_apic_ids(void) +set_interrupt_apic_ids(void) { - u_int apic_id, cluster, cluster_id; + u_int apic_id; - /* Force us to allocate cluster 0 at the start. */ - cluster = -1; - cluster_id = APIC_MAX_INTRACLUSTER_ID; for (apic_id = 0; apic_id < MAXCPU; apic_id++) { if (!cpu_info[apic_id].cpu_present) continue; - if (cluster_id == APIC_MAX_INTRACLUSTER_ID) { - cluster = ioapic_next_logical_cluster(); - cluster_id = 0; - } else - cluster_id++; - if (bootverbose) - printf("APIC ID: physical %u, logical %u:%u\n", - apic_id, cluster, cluster_id); - lapic_set_logical_id(apic_id, cluster, cluster_id); + if (cpu_info[apic_id].cpu_bsp) + continue; +#if 0 + /* Don't let hyperthreads service interrupts. */ + if (hyperthreading_cpus > 1 && + apic_id % hyperthreading_cpus != 0) + continue; +#endif + intr_add_cpu(apic_id); } } ==== //depot/projects/arm/src/sys/amd64/include/apicvar.h#6 (text+ko) ==== @@ -26,7 +26,7 @@ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * - * $FreeBSD: src/sys/amd64/include/apicvar.h,v 1.16 2005/12/22 22:16:05 jhb Exp $ + * $FreeBSD: src/sys/amd64/include/apicvar.h,v 1.17 2006/02/28 22:24:54 jhb Exp $ */ #ifndef _MACHINE_APICVAR_H_ @@ -181,7 +181,6 @@ void *ioapic_create(uintptr_t addr, int32_t id, int intbase); int ioapic_disable_pin(void *cookie, u_int pin); int ioapic_get_vector(void *cookie, u_int pin); -int ioapic_next_logical_cluster(void); void ioapic_register(void *cookie); int ioapic_remap_vector(void *cookie, u_int pin, int vector); int ioapic_set_bus(void *cookie, u_int pin, int bus_type); ==== //depot/projects/arm/src/sys/amd64/include/intr_machdep.h#4 (text+ko) ==== @@ -23,7 +23,7 @@ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * - * $FreeBSD: src/sys/amd64/include/intr_machdep.h,v 1.8 2005/12/08 18:33:30 jhb Exp $ + * $FreeBSD: src/sys/amd64/include/intr_machdep.h,v 1.9 2006/02/28 22:24:54 jhb Exp $ */ #ifndef __MACHINE_INTR_MACHDEP_H__ @@ -85,6 +85,7 @@ void (*pic_resume)(struct intsrc *); int (*pic_config_intr)(struct intsrc *, enum intr_trigger, enum intr_polarity); + void (*pic_assign_cpu)(struct intsrc *, u_int apic_id); }; /* Flags for pic_disable_source() */ @@ -105,6 +106,7 @@ u_long *is_count; u_long *is_straycount; u_int is_index; + u_int is_enabled:1; }; struct trapframe; @@ -117,6 +119,11 @@ enum intr_trigger elcr_read_trigger(u_int irq); void elcr_resume(void); void elcr_write_trigger(u_int irq, enum intr_trigger trigger); +#ifdef SMP +void intr_add_cpu(u_int apic_id); +#else +#define intr_add_cpu(apic_id) +#endif int intr_add_handler(const char *name, int vector, driver_intr_t handler, void *arg, enum intr_type flags, void **cookiep); int intr_config_intr(int vector, enum intr_trigger trig, ==== //depot/projects/arm/src/sys/amd64/include/mptable.h#2 (text+ko) ==== @@ -22,7 +22,7 @@ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * - * $FreeBSD: src/sys/amd64/include/mptable.h,v 1.222 2005/01/05 20:17:20 imp Exp $ + * $FreeBSD: src/sys/amd64/include/mptable.h,v 1.223 2006/03/01 05:59:56 sam Exp $ */ #ifndef __MACHINE_MPTABLE_H__ @@ -139,7 +139,8 @@ char name[16]; } basetable_entry; +#ifdef _KERNEL int mptable_pci_probe_table(int bus); int mptable_pci_route_interrupt(device_t pcib, device_t dev, int pin); - +#endif #endif /* !__MACHINE_MPTABLE_H__ */ ==== //depot/projects/arm/src/sys/amd64/isa/atpic.c#4 (text+ko) ==== @@ -32,7 +32,7 @@ */ #include -__FBSDID("$FreeBSD: src/sys/amd64/isa/atpic.c,v 1.18 2005/12/08 18:33:30 jhb Exp $"); +__FBSDID("$FreeBSD: src/sys/amd64/isa/atpic.c,v 1.19 2006/02/28 22:24:54 jhb Exp $"); #include "opt_auto_eoi.h" #include "opt_isa.h" @@ -108,8 +108,8 @@ #define ATPIC(io, base, eoi, imenptr) \ { { atpic_enable_source, atpic_disable_source, (eoi), \ atpic_enable_intr, atpic_vector, atpic_source_pending, NULL, \ - atpic_resume, atpic_config_intr }, (io), (base), \ - IDT_IO_INTS + (base), (imenptr) } + atpic_resume, atpic_config_intr, atpic_assign_cpu }, (io), \ + (base), IDT_IO_INTS + (base), (imenptr) } #define INTSRC(irq) \ { { &atpics[(irq) / 8].at_pic }, IDTVEC(atpic_intr ## irq ), \ @@ -142,6 +142,7 @@ static int atpic_source_pending(struct intsrc *isrc); static int atpic_config_intr(struct intsrc *isrc, enum intr_trigger trig, enum intr_polarity pol); +static void atpic_assign_cpu(struct intsrc *isrc, u_int apic_id); static void i8259_init(struct atpic *pic, int slave); static struct atpic atpics[] = { @@ -353,6 +354,17 @@ } static void +atpic_assign_cpu(struct intsrc *isrc, u_int apic_id) +{ + + /* + * 8259A's are only used in UP in which case all interrupts always + * go to the sole CPU and this function shouldn't even be called. + */ + panic("%s: bad cookie", __func__); +} + +static void i8259_init(struct atpic *pic, int slave) { int imr_addr; ==== //depot/projects/arm/src/sys/arm/arm/busdma_machdep.c#7 (text+ko) ==== @@ -29,7 +29,7 @@ */ #include -__FBSDID("$FreeBSD: src/sys/arm/arm/busdma_machdep.c,v 1.24 2005/10/03 22:44:54 cognet Exp $"); +__FBSDID("$FreeBSD: src/sys/arm/arm/busdma_machdep.c,v 1.25 2006/03/01 23:04:25 cognet Exp $"); /* * MacPPC bus dma support routines @@ -93,6 +93,8 @@ bus_dma_tag_t dmat; int flags; void *buffer; + void *origbuffer; + void *allocbuffer; TAILQ_ENTRY(bus_dmamap) freelist; int len; }; @@ -416,6 +418,23 @@ *mapp = NULL; return (ENOMEM); } + if (flags & BUS_DMA_COHERENT) { + void *tmpaddr = arm_remap_nocache( + (void *)((vm_offset_t)*vaddr &~ PAGE_MASK), + dmat->maxsize + ((vm_offset_t)*vaddr & PAGE_MASK)); + + if (tmpaddr) { + tmpaddr = (void *)((vm_offset_t)(tmpaddr) + + ((vm_offset_t)*vaddr & PAGE_MASK)); + newmap->origbuffer = *vaddr; + newmap->allocbuffer = tmpaddr; + cpu_idcache_wbinv_range((vm_offset_t)*vaddr, + dmat->maxsize); + *vaddr = tmpaddr; + } else + newmap->origbuffer = newmap->allocbuffer = NULL; + } else + newmap->origbuffer = newmap->allocbuffer = NULL; return (0); } @@ -426,6 +445,12 @@ void bus_dmamem_free(bus_dma_tag_t dmat, void *vaddr, bus_dmamap_t map) { + if (map->allocbuffer) { + KASSERT(map->allocbuffer == vaddr, + ("Trying to freeing the wrong DMA buffer")); + vaddr = map->origbuffer; + arm_unmap_nocache(map->allocbuffer, dmat->maxsize); + } if (dmat->maxsize <= PAGE_SIZE) free(vaddr, M_DEVBUF); else { ==== //depot/projects/arm/src/sys/arm/arm/pmap.c#9 (text+ko) ==== @@ -147,7 +147,7 @@ #include "opt_vm.h" #include -__FBSDID("$FreeBSD: src/sys/arm/arm/pmap.c,v 1.41 2005/11/20 06:09:48 alc Exp $"); +__FBSDID("$FreeBSD: src/sys/arm/arm/pmap.c,v 1.42 2006/03/01 23:04:25 cognet Exp $"); #include #include #include @@ -2553,9 +2553,12 @@ virtual_avail = round_page(virtual_avail); virtual_end = lastaddr; kernel_vm_end = pmap_curmaxkvaddr; + arm_nocache_startaddr = lastaddr; + #ifdef ARM_USE_SMALL_ALLOC mtx_init(&smallalloc_mtx, "Small alloc page list", NULL, MTX_DEF); - alloc_firstaddr = alloc_curaddr = lastaddr; + alloc_firstaddr = alloc_curaddr = arm_nocache_startaddr + + ARM_NOCACHE_KVA_SIZE; #endif } @@ -2916,6 +2919,13 @@ } void +pmap_kenter_nocache(vm_offset_t va, vm_paddr_t pa) +{ + + pmap_kenter_internal(va, pa, 0); +} + +void pmap_kenter_user(vm_offset_t va, vm_paddr_t pa) { ==== //depot/projects/arm/src/sys/arm/arm/trap.c#8 (text+ko) ==== @@ -82,7 +82,7 @@ #include "opt_ktrace.h" #include -__FBSDID("$FreeBSD: src/sys/arm/arm/trap.c,v 1.21 2006/02/08 08:09:15 phk Exp $"); +__FBSDID("$FreeBSD: src/sys/arm/arm/trap.c,v 1.23 2006/03/01 20:43:51 cognet Exp $"); #include @@ -428,15 +428,14 @@ error = vm_fault(map, va, ftype, (ftype & VM_PROT_WRITE) ? VM_FAULT_DIRTY : VM_FAULT_NORMAL); pcb->pcb_onfault = onfault; - if (__predict_true(error == 0)) { - goto out; - } if (map != kernel_map) { PROC_LOCK(p); p->p_lock--; PROC_UNLOCK(p); } + if (__predict_true(error == 0)) + goto out; if (user == 0) { if (pcb->pcb_onfault) { tf->tf_r0 = error; @@ -930,6 +929,8 @@ AUDIT_SYSCALL_ENTER(code, td); error = (*callp->sy_call)(td, args); AUDIT_SYSCALL_EXIT(error, td); + KASSERT(td->td_ar == NULL, + ("returning from syscall with td_ar set!")); } switch (error) { case 0: ==== //depot/projects/arm/src/sys/arm/arm/undefined.c#7 (text+ko) ==== @@ -48,7 +48,7 @@ #include "opt_ddb.h" #include -__FBSDID("$FreeBSD: src/sys/arm/arm/undefined.c,v 1.11 2006/02/22 18:57:49 jhb Exp $"); +__FBSDID("$FreeBSD: src/sys/arm/arm/undefined.c,v 1.12 2006/03/01 18:33:45 cognet Exp $"); #include #include ==== //depot/projects/arm/src/sys/arm/arm/vm_machdep.c#5 (text+ko) ==== @@ -41,7 +41,7 @@ */ #include -__FBSDID("$FreeBSD: src/sys/arm/arm/vm_machdep.c,v 1.21 2006/01/27 21:07:04 cognet Exp $"); +__FBSDID("$FreeBSD: src/sys/arm/arm/vm_machdep.c,v 1.23 2006/03/02 00:37:40 cognet Exp $"); #include #include @@ -368,6 +368,69 @@ { } +#define BITS_PER_INT (8 * sizeof(int)) +vm_offset_t arm_nocache_startaddr; +static int arm_nocache_allocated[ARM_NOCACHE_KVA_SIZE / (PAGE_SIZE * + BITS_PER_INT)]; + +/* + * Functions to map and unmap memory non-cached into KVA the kernel won't try + * to allocate. The goal is to provide uncached memory to busdma, to honor + * BUS_DMA_COHERENT. + * We can allocate at most ARM_NOCACHE_KVA_SIZE bytes. + * The allocator is rather dummy, each page is represented by a bit in + * a bitfield, 0 meaning the page is not allocated, 1 meaning it is. + * As soon as it finds enough contiguous pages to satisfy the request, + * it returns the address. + */ +void * +arm_remap_nocache(void *addr, vm_size_t size) +{ + int i, j; + + size = round_page(size); + for (i = 0; i < MIN(ARM_NOCACHE_KVA_SIZE / (PAGE_SIZE * BITS_PER_INT), + ARM_TP_ADDRESS); i++) { + if (!(arm_nocache_allocated[i / BITS_PER_INT] & (1 << (i % + BITS_PER_INT)))) { + for (j = i; j < i + (size / (PAGE_SIZE)); j++) + if (arm_nocache_allocated[j / BITS_PER_INT] & + (1 << (j % BITS_PER_INT))) + break; + if (j == i + (size / (PAGE_SIZE))) + break; + } + } + if (i < MIN(ARM_NOCACHE_KVA_SIZE / (PAGE_SIZE * BITS_PER_INT), + ARM_TP_ADDRESS)) { + vm_offset_t tomap = arm_nocache_startaddr + i * PAGE_SIZE; + void *ret = (void *)tomap; + vm_paddr_t physaddr = vtophys((vm_offset_t)addr); + + for (; tomap < (vm_offset_t)ret + size; tomap += PAGE_SIZE, + physaddr += PAGE_SIZE, i++) { + pmap_kenter_nocache(tomap, physaddr); + arm_nocache_allocated[i / BITS_PER_INT] |= 1 << (i % + BITS_PER_INT); + } + return (ret); + } + return (NULL); +} + +void +arm_unmap_nocache(void *addr, vm_size_t size) +{ + vm_offset_t raddr = (vm_offset_t)addr; + int i; + + size = round_page(size); + i = (raddr - arm_nocache_startaddr) / (PAGE_SIZE); + for (; size > 0; size -= PAGE_SIZE, i++) + arm_nocache_allocated[i / BITS_PER_INT] &= ~(1 << (i % + BITS_PER_INT)); +} + #ifdef ARM_USE_SMALL_ALLOC static TAILQ_HEAD(,arm_small_page) pages_normal = @@ -393,7 +456,7 @@ { struct arm_small_page *pg; - bytes &= ~PAGE_SIZE; + bytes &= ~PAGE_MASK; while (bytes > 0) { pg = (struct arm_small_page *)list; pg->addr = mem; ==== //depot/projects/arm/src/sys/arm/include/pmap.h#5 (text+ko) ==== @@ -44,7 +44,7 @@ * from: @(#)pmap.h 7.4 (Berkeley) 5/12/91 * from: FreeBSD: src/sys/i386/include/pmap.h,v 1.70 2000/11/30 * - * $FreeBSD: src/sys/arm/include/pmap.h,v 1.14 2005/11/21 19:10:44 cognet Exp $ + * $FreeBSD: src/sys/arm/include/pmap.h,v 1.15 2006/03/01 23:04:25 cognet Exp $ */ #ifndef _MACHINE_PMAP_H_ @@ -210,6 +210,7 @@ void pmap_bootstrap(vm_offset_t, vm_offset_t, struct pv_addr *); void pmap_kenter(vm_offset_t va, vm_paddr_t pa); +void pmap_kenter_nocache(vm_offset_t va, vm_paddr_t pa); void pmap_kenter_user(vm_offset_t va, vm_paddr_t pa); void pmap_kremove(vm_offset_t); void *pmap_mapdev(vm_offset_t, vm_size_t); @@ -525,6 +526,12 @@ }; #endif + +#define ARM_NOCACHE_KVA_SIZE 0x600000 +extern vm_offset_t arm_nocache_startaddr; +void *arm_remap_nocache(void *, vm_size_t); +void arm_unmap_nocache(void *, vm_size_t); + extern vm_paddr_t dump_avail[]; #endif /* _KERNEL */ ==== //depot/projects/arm/src/sys/compat/freebsd32/freebsd32_misc.c#7 (text+ko) ==== @@ -25,7 +25,7 @@ */ #include -__FBSDID("$FreeBSD: src/sys/compat/freebsd32/freebsd32_misc.c,v 1.50 2006/02/06 22:06:53 jhb Exp $"); +__FBSDID("$FreeBSD: src/sys/compat/freebsd32/freebsd32_misc.c,v 1.51 2006/02/28 19:39:17 ps Exp $"); #include "opt_compat.h" @@ -1427,37 +1427,83 @@ return (ftruncate(td, &ap)); } -#ifdef COMPAT_FREEBSD4 -int -freebsd4_freebsd32_sendfile(struct thread *td, - struct freebsd4_freebsd32_sendfile_args *uap) +struct sf_hdtr32 { + uint32_t headers; + int hdr_cnt; + uint32_t trailers; + int trl_cnt; +}; + +static int +freebsd32_do_sendfile(struct thread *td, + struct freebsd32_sendfile_args *uap, int compat) >>> TRUNCATED FOR MAIL (1000 lines) <<<