Date: Wed, 14 Mar 2018 04:14:14 +0000 (UTC) From: Gordon Tetlow <gordon@FreeBSD.org> To: doc-committers@freebsd.org, svn-doc-all@freebsd.org, svn-doc-head@freebsd.org Subject: svn commit: r51482 - in head/share: security/advisories security/patches/SA-18:03 xml Message-ID: <201803140414.w2E4EEd4066873@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: gordon (src,ports committer) Date: Wed Mar 14 04:14:13 2018 New Revision: 51482 URL: https://svnweb.freebsd.org/changeset/doc/51482 Log: Add FreeBSD-SA-18:03.speculative_execution. Approved by: so Added: head/share/security/advisories/FreeBSD-SA-18:03.speculative_execution.asc (contents, props changed) head/share/security/patches/SA-18:03/ head/share/security/patches/SA-18:03/speculative_execution-amd64-11.patch (contents, props changed) head/share/security/patches/SA-18:03/speculative_execution-amd64-11.patch.asc (contents, props changed) Modified: head/share/xml/advisories.xml Added: head/share/security/advisories/FreeBSD-SA-18:03.speculative_execution.asc ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/share/security/advisories/FreeBSD-SA-18:03.speculative_execution.asc Wed Mar 14 04:14:13 2018 (r51482) @@ -0,0 +1,206 @@ +-----BEGIN PGP SIGNED MESSAGE----- +Hash: SHA512 + +============================================================================= +FreeBSD-SA-18:03.speculative_execution Security Advisory + The FreeBSD Project + +Topic: Speculative Execution Vulnerabilities + +Category: core +Module: kernel +Announced: 2018-03-14 +Credits: Jann Horn (Google Project Zero); Werner Haas, Thomas + Prescher (Cyberus Technology); Daniel Gruss, Moritz Lipp, + Stefan Mangard, Michael Schwarz (Graz University of + Technology); Paul Kocher; Daniel Genkin (University of + Pennsylvania and University of Maryland), Mike Hamburg + (Rambus); Yuval Yarom (University of Adelaide and Data6) +Affects: All supported versions of FreeBSD. +Corrected: 2018-02-17 18:00:01 UTC (stable/11, 11.1-STABLE) + 2018-03-14 04:00:00 UTC (releng/11.1, 11.1-RELEASE-p8) +CVE Name: CVE-2017-5715, CVE-2017-5754 + +Special Note: Speculative execution vulnerability mitigation is a work + in progress. This advisory addresses the most significant + issues for FreeBSD 11.1 on amd64 CPUs. We expect to update + this advisory to include 10.x for amd64 CPUs. Future FreeBSD + releases will address this issue on i386 and other CPUs. + freebsd-update will include changes on i386 as part of this + update due to common code changes shared between amd64 and + i386, however it contains no functional changes for i386 (in + particular, it does not mitigate the issue on i386). + +For general information regarding FreeBSD Security Advisories, +including descriptions of the fields above, security branches, and the +following sections, please visit <URL:https://security.FreeBSD.org/>. + +I. Background + +Many modern processors have implementation issues that allow unprivileged +attackers to bypass user-kernel or inter-process memory access restrictions +by exploiting speculative execution and shared resources (for example, +caches). + +II. Problem Description + +A number of issues relating to speculative execution were found last year +and publicly announced January 3rd. Two of these, known as Meltdown and +Spectre V2, are addressed here. + +CVE-2017-5754 (Meltdown) +- ------------------------ + +This issue relies on an affected CPU speculatively executing instructions +beyond a faulting instruction. When this happens, changes to architectural +state are not committed, but observable changes may be left in micro- +architectural state (for example, cache). This may be used to infer +privileged data. + +CVE-2017-5715 (Spectre V2) +- -------------------------- + +Spectre V2 uses branch target injection to speculatively execute kernel code +at an address under the control of an attacker. + +III. Impact + +An attacker may be able to read secret data from the kernel or from a +process when executing untrusted code (for example, in a web browser). + +IV. Workaround + +No workaround is available. + +V. Solution + +Perform one of the following: + +1) Upgrade your vulnerable system to a supported FreeBSD stable or +release / security branch (releng) dated after the correction date, +and reboot. + +2) To update your vulnerable system via a binary patch: + +Systems running a RELEASE version of FreeBSD on the i386 or amd64 +platforms can be updated via the freebsd-update(8) utility, followed +by a reboot into the new kernel: + +# freebsd-update fetch +# freebsd-update install +# shutdown -r now + +3) To update your vulnerable system via a source code patch: + +The following patches have been verified to apply to the applicable +FreeBSD release branches. + +a) Download the relevant patch from the location below, and verify the +detached PGP signature using your PGP utility. + +[FreeBSD 11.1] +# fetch https://security.FreeBSD.org/patches/SA-18:03/speculative_execution-amd64-11.patch +# fetch https://security.FreeBSD.org/patches/SA-18:03/speculative_execution-amd64-11.patch.asc +# gpg --verify speculative_execution-amd64-11.patch.asc + +b) Apply the patch. Execute the following commands as root: + +# cd /usr/src +# patch < /path/to/patch + +c) Recompile your kernel as described in +<URL:https://www.FreeBSD.org/handbook/kernelconfig.html>; and reboot the +system. + +VI. Correction details + +CVE-2017-5754 (Meltdown) +- ------------------------ + +The mitigation is known as Page Table Isolation (PTI). PTI largely separates +kernel and user mode page tables, so that even during speculative execution +most of the kernel's data is unmapped and not accessible. + +A demonstration of the Meltdown vulnerability is available at +https://github.com/dag-erling/meltdown. A positive result is definitive +(that is, the vulnerability exists with certainty). A negative result +indicates either that the CPU is not affected, or that the test is not +capable of demonstrating the issue on the CPU (and may need to be modified). + +A patched kernel will automatically enable PTI on Intel CPUs. The status can +be checked via the vm.pmap.pti sysctl: + +# sysctl vm.pmap.pti +vm.pmap.pti: 1 + +The default setting can be overridden by setting the loader tunable +vm.pmap.pti to 1 or 0 in /boot/loader.conf. This setting takes effect only +at boot. + +PTI introduces a performance regression. The observed performance loss is +significant in microbenchmarks of system call overhead, but is much smaller +for many real workloads. + +CVE-2017-5715 (Spectre V2) +- -------------------------- + +There are two common mitigations for Spectre V2. This patch includes a +mitigation using Indirect Branch Restricted Speculation, a feature available +via a microcode update from processor manufacturers. The alternate +mitigation, Retpoline, is a feature available in newer compilers. The +feasibility of applying Retpoline to stable branches and/or releases is under +investigation. + +The patch includes the IBRS mitigation for Spectre V2. To use the mitigation +the system must have an updated microcode; with older microcode a patched +kernel will function without the mitigation. + +IBRS can be disabled via the hw.ibrs_disable sysctl (and tunable), and the +status can be checked via the hw.ibrs_active sysctl. IBRS may be enabled or +disabled at runtime. Additional detail on microcode updates will follow. + +The following list contains the correction revision numbers for each +affected branch. + +Branch/path Revision +- ------------------------------------------------------------------------- +stable/11/ r329462 +releng/11.1/ r330908 +- ------------------------------------------------------------------------- + +To see which files were modified by a particular revision, run the +following command, replacing NNNNNN with the revision number, on a +machine with Subversion installed: + +# svn diff -cNNNNNN --summarize svn://svn.freebsd.org/base + +Or visit the following URL, replacing NNNNNN with the revision number: + +<URL:https://svnweb.freebsd.org/base?view=revision&revision=NNNNNN>; + +VII. References + +<URL:https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-5715>; + +<URL:https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-5754>; + +The latest revision of this advisory is available at +<URL:https://security.FreeBSD.org/advisories/FreeBSD-SA-18:03.speculative_execution.asc>; +-----BEGIN PGP SIGNATURE----- + +iQKTBAEBCgB9FiEE/A6HiuWv54gCjWNV05eS9J6n5cIFAlqon0RfFIAAAAAALgAo +aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEZD +MEU4NzhBRTVBRkU3ODgwMjhENjM1NUQzOTc5MkY0OUVBN0U1QzIACgkQ05eS9J6n +5cKORw/+Lc5lxLhDgU1rQ0JF6sb2b80Ly5k+rJLXFWBvmEQt0uVyVkF4TMJ99M04 +bcmrLbT4Pl0Csh/iEYvZQ4el12KvPDApHszsLTBgChD+KfCLvCZvBZzasgDWGD0E +JhL4eIX0wjJ4oGGsT+TAqkmwXyAMJgWW/ZgZPFVXocylZTL3fV4g52VdG1Jnd2yu +hnkViH2kVlVJqXX9AHlenIUfEmUiRUGrMh5oPPpFYDDmfJ+enZ8QLxfZtOKIliD7 +u+2GP8V/bvaErkxqF5wwobybrBOMXpq9Y/fWw0EH/om7myevj/oORqK+ZmGZ17bl +IRbdWxgjc1hN2TIMVn9q9xX6i0I0wSPwbpLYagKnSnE8WNVUTZUteaj1GKGTG1rj +DFH2zOLlbRr/IXUFldM9b6VbZX6G5Ijxwy1DJzB/0KL5ZTbAReUR0pqHR7xpulbJ +eDv8SKCwYiUpMuwPOXNdVlVLZSsH5/9A0cyjH3+E+eIhM8qyxw7iRFwA0DxnGVkr +tkMo51Vl3Gl7JFFimGKljsE9mBh00m8B0WYJwknvfhdehO4WripcwI7/V5zL6cwj +s018kaW4Xm77LOz6P1iN8nbcjZ9gN2AsPYUYYZqJxjCcZ7r489Hg9BhbDf0QtC0R +gnwZWiZ/KuAy0C6vaHljsm0xPEM5nBz/yScFXDbuhEdmEgBBD6w= +=fqrI +-----END PGP SIGNATURE----- Added: head/share/security/patches/SA-18:03/speculative_execution-amd64-11.patch ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/share/security/patches/SA-18:03/speculative_execution-amd64-11.patch Wed Mar 14 04:14:13 2018 (r51482) @@ -0,0 +1,4618 @@ +--- sys/amd64/amd64/apic_vector.S.orig ++++ sys/amd64/amd64/apic_vector.S +@@ -2,7 +2,13 @@ + * Copyright (c) 1989, 1990 William F. Jolitz. + * Copyright (c) 1990 The Regents of the University of California. + * All rights reserved. ++ * Copyright (c) 2014-2018 The FreeBSD Foundation ++ * All rights reserved. + * ++ * Portions of this software were developed by ++ * Konstantin Belousov <kib@FreeBSD.org> under sponsorship from ++ * the FreeBSD Foundation. ++ * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: +@@ -38,12 +44,12 @@ + + #include "opt_smp.h" + ++#include "assym.s" ++ + #include <machine/asmacros.h> + #include <machine/specialreg.h> + #include <x86/apicreg.h> + +-#include "assym.s" +- + #ifdef SMP + #define LK lock ; + #else +@@ -73,30 +79,28 @@ + * translates that into a vector, and passes the vector to the + * lapic_handle_intr() function. + */ +-#define ISR_VEC(index, vec_name) \ +- .text ; \ +- SUPERALIGN_TEXT ; \ +-IDTVEC(vec_name) ; \ +- PUSH_FRAME ; \ +- FAKE_MCOUNT(TF_RIP(%rsp)) ; \ +- cmpl $0,x2apic_mode ; \ +- je 1f ; \ +- movl $(MSR_APIC_ISR0 + index),%ecx ; \ +- rdmsr ; \ +- jmp 2f ; \ +-1: ; \ +- movq lapic_map, %rdx ; /* pointer to local APIC */ \ +- movl LA_ISR + 16 * (index)(%rdx), %eax ; /* load ISR */ \ +-2: ; \ +- bsrl %eax, %eax ; /* index of highest set bit in ISR */ \ +- jz 3f ; \ +- addl $(32 * index),%eax ; \ +- movq %rsp, %rsi ; \ +- movl %eax, %edi ; /* pass the IRQ */ \ +- call lapic_handle_intr ; \ +-3: ; \ +- MEXITCOUNT ; \ ++ .macro ISR_VEC index, vec_name ++ INTR_HANDLER \vec_name ++ FAKE_MCOUNT(TF_RIP(%rsp)) ++ cmpl $0,x2apic_mode ++ je 1f ++ movl $(MSR_APIC_ISR0 + \index),%ecx ++ rdmsr ++ jmp 2f ++1: ++ movq lapic_map, %rdx /* pointer to local APIC */ ++ movl LA_ISR + 16 * (\index)(%rdx), %eax /* load ISR */ ++2: ++ bsrl %eax, %eax /* index of highest set bit in ISR */ ++ jz 3f ++ addl $(32 * \index),%eax ++ movq %rsp, %rsi ++ movl %eax, %edi /* pass the IRQ */ ++ call lapic_handle_intr ++3: ++ MEXITCOUNT + jmp doreti ++ .endm + + /* + * Handle "spurious INTerrupts". +@@ -108,26 +112,21 @@ + .text + SUPERALIGN_TEXT + IDTVEC(spuriousint) +- + /* No EOI cycle used here */ +- + jmp doreti_iret + +- ISR_VEC(1, apic_isr1) +- ISR_VEC(2, apic_isr2) +- ISR_VEC(3, apic_isr3) +- ISR_VEC(4, apic_isr4) +- ISR_VEC(5, apic_isr5) +- ISR_VEC(6, apic_isr6) +- ISR_VEC(7, apic_isr7) ++ ISR_VEC 1, apic_isr1 ++ ISR_VEC 2, apic_isr2 ++ ISR_VEC 3, apic_isr3 ++ ISR_VEC 4, apic_isr4 ++ ISR_VEC 5, apic_isr5 ++ ISR_VEC 6, apic_isr6 ++ ISR_VEC 7, apic_isr7 + + /* + * Local APIC periodic timer handler. + */ +- .text +- SUPERALIGN_TEXT +-IDTVEC(timerint) +- PUSH_FRAME ++ INTR_HANDLER timerint + FAKE_MCOUNT(TF_RIP(%rsp)) + movq %rsp, %rdi + call lapic_handle_timer +@@ -137,10 +136,7 @@ + /* + * Local APIC CMCI handler. + */ +- .text +- SUPERALIGN_TEXT +-IDTVEC(cmcint) +- PUSH_FRAME ++ INTR_HANDLER cmcint + FAKE_MCOUNT(TF_RIP(%rsp)) + call lapic_handle_cmc + MEXITCOUNT +@@ -149,10 +145,7 @@ + /* + * Local APIC error interrupt handler. + */ +- .text +- SUPERALIGN_TEXT +-IDTVEC(errorint) +- PUSH_FRAME ++ INTR_HANDLER errorint + FAKE_MCOUNT(TF_RIP(%rsp)) + call lapic_handle_error + MEXITCOUNT +@@ -163,10 +156,7 @@ + * Xen event channel upcall interrupt handler. + * Only used when the hypervisor supports direct vector callbacks. + */ +- .text +- SUPERALIGN_TEXT +-IDTVEC(xen_intr_upcall) +- PUSH_FRAME ++ INTR_HANDLER xen_intr_upcall + FAKE_MCOUNT(TF_RIP(%rsp)) + movq %rsp, %rdi + call xen_intr_handle_upcall +@@ -183,59 +173,59 @@ + SUPERALIGN_TEXT + invltlb_ret: + call as_lapic_eoi +- POP_FRAME +- jmp doreti_iret ++ jmp ld_regs + + SUPERALIGN_TEXT +-IDTVEC(invltlb) +- PUSH_FRAME +- ++ INTR_HANDLER invltlb + call invltlb_handler + jmp invltlb_ret + +-IDTVEC(invltlb_pcid) +- PUSH_FRAME +- ++ INTR_HANDLER invltlb_pcid + call invltlb_pcid_handler + jmp invltlb_ret + +-IDTVEC(invltlb_invpcid) +- PUSH_FRAME +- ++ INTR_HANDLER invltlb_invpcid_nopti + call invltlb_invpcid_handler + jmp invltlb_ret + ++ INTR_HANDLER invltlb_invpcid_pti ++ call invltlb_invpcid_pti_handler ++ jmp invltlb_ret ++ + /* + * Single page TLB shootdown + */ +- .text ++ INTR_HANDLER invlpg ++ call invlpg_handler ++ jmp invltlb_ret + +- SUPERALIGN_TEXT +-IDTVEC(invlpg) +- PUSH_FRAME ++ INTR_HANDLER invlpg_invpcid ++ call invlpg_invpcid_handler ++ jmp invltlb_ret + +- call invlpg_handler ++ INTR_HANDLER invlpg_pcid ++ call invlpg_pcid_handler + jmp invltlb_ret + + /* + * Page range TLB shootdown. + */ +- .text +- SUPERALIGN_TEXT +-IDTVEC(invlrng) +- PUSH_FRAME +- ++ INTR_HANDLER invlrng + call invlrng_handler + jmp invltlb_ret + ++ INTR_HANDLER invlrng_invpcid ++ call invlrng_invpcid_handler ++ jmp invltlb_ret ++ ++ INTR_HANDLER invlrng_pcid ++ call invlrng_pcid_handler ++ jmp invltlb_ret ++ + /* + * Invalidate cache. + */ +- .text +- SUPERALIGN_TEXT +-IDTVEC(invlcache) +- PUSH_FRAME +- ++ INTR_HANDLER invlcache + call invlcache_handler + jmp invltlb_ret + +@@ -242,15 +232,9 @@ + /* + * Handler for IPIs sent via the per-cpu IPI bitmap. + */ +- .text +- SUPERALIGN_TEXT +-IDTVEC(ipi_intr_bitmap_handler) +- PUSH_FRAME +- ++ INTR_HANDLER ipi_intr_bitmap_handler + call as_lapic_eoi +- + FAKE_MCOUNT(TF_RIP(%rsp)) +- + call ipi_bitmap_handler + MEXITCOUNT + jmp doreti +@@ -258,13 +242,8 @@ + /* + * Executed by a CPU when it receives an IPI_STOP from another CPU. + */ +- .text +- SUPERALIGN_TEXT +-IDTVEC(cpustop) +- PUSH_FRAME +- ++ INTR_HANDLER cpustop + call as_lapic_eoi +- + call cpustop_handler + jmp doreti + +@@ -271,11 +250,7 @@ + /* + * Executed by a CPU when it receives an IPI_SUSPEND from another CPU. + */ +- .text +- SUPERALIGN_TEXT +-IDTVEC(cpususpend) +- PUSH_FRAME +- ++ INTR_HANDLER cpususpend + call cpususpend_handler + call as_lapic_eoi + jmp doreti +@@ -285,10 +260,7 @@ + * + * - Calls the generic rendezvous action function. + */ +- .text +- SUPERALIGN_TEXT +-IDTVEC(rendezvous) +- PUSH_FRAME ++ INTR_HANDLER rendezvous + #ifdef COUNT_IPIS + movl PCPU(CPUID), %eax + movq ipi_rendezvous_counts(,%rax,8), %rax +@@ -328,4 +300,8 @@ + popq %rax + jmp doreti_iret + ++ INTR_HANDLER justreturn1 ++ call as_lapic_eoi ++ jmp doreti ++ + #endif /* SMP */ +--- sys/amd64/amd64/atpic_vector.S.orig ++++ sys/amd64/amd64/atpic_vector.S +@@ -36,38 +36,35 @@ + * master and slave interrupt controllers. + */ + ++#include "assym.s" + #include <machine/asmacros.h> + +-#include "assym.s" +- + /* + * Macros for interrupt entry, call to handler, and exit. + */ +-#define INTR(irq_num, vec_name) \ +- .text ; \ +- SUPERALIGN_TEXT ; \ +-IDTVEC(vec_name) ; \ +- PUSH_FRAME ; \ +- FAKE_MCOUNT(TF_RIP(%rsp)) ; \ +- movq %rsp, %rsi ; \ +- movl $irq_num, %edi; /* pass the IRQ */ \ +- call atpic_handle_intr ; \ +- MEXITCOUNT ; \ ++ .macro INTR irq_num, vec_name ++ INTR_HANDLER \vec_name ++ FAKE_MCOUNT(TF_RIP(%rsp)) ++ movq %rsp, %rsi ++ movl $\irq_num, %edi /* pass the IRQ */ ++ call atpic_handle_intr ++ MEXITCOUNT + jmp doreti ++ .endm + +- INTR(0, atpic_intr0) +- INTR(1, atpic_intr1) +- INTR(2, atpic_intr2) +- INTR(3, atpic_intr3) +- INTR(4, atpic_intr4) +- INTR(5, atpic_intr5) +- INTR(6, atpic_intr6) +- INTR(7, atpic_intr7) +- INTR(8, atpic_intr8) +- INTR(9, atpic_intr9) +- INTR(10, atpic_intr10) +- INTR(11, atpic_intr11) +- INTR(12, atpic_intr12) +- INTR(13, atpic_intr13) +- INTR(14, atpic_intr14) +- INTR(15, atpic_intr15) ++ INTR 0, atpic_intr0 ++ INTR 1, atpic_intr1 ++ INTR 2, atpic_intr2 ++ INTR 3, atpic_intr3 ++ INTR 4, atpic_intr4 ++ INTR 5, atpic_intr5 ++ INTR 6, atpic_intr6 ++ INTR 7, atpic_intr7 ++ INTR 8, atpic_intr8 ++ INTR 9, atpic_intr9 ++ INTR 10, atpic_intr10 ++ INTR 11, atpic_intr11 ++ INTR 12, atpic_intr12 ++ INTR 13, atpic_intr13 ++ INTR 14, atpic_intr14 ++ INTR 15, atpic_intr15 +--- sys/amd64/amd64/cpu_switch.S.orig ++++ sys/amd64/amd64/cpu_switch.S +@@ -191,9 +191,11 @@ + done_tss: + movq %r8,PCPU(RSP0) + movq %r8,PCPU(CURPCB) +- /* Update the TSS_RSP0 pointer for the next interrupt */ ++ /* Update the COMMON_TSS_RSP0 pointer for the next interrupt */ ++ cmpb $0,pti(%rip) ++ jne 1f + movq %r8,COMMON_TSS_RSP0(%rdx) +- movq %r12,PCPU(CURTHREAD) /* into next thread */ ++1: movq %r12,PCPU(CURTHREAD) /* into next thread */ + + /* Test if debug registers should be restored. */ + testl $PCB_DBREGS,PCB_FLAGS(%r8) +@@ -270,7 +272,12 @@ + shrq $8,%rcx + movl %ecx,8(%rax) + movb $0x89,5(%rax) /* unset busy */ +- movl $TSSSEL,%eax ++ cmpb $0,pti(%rip) ++ je 1f ++ movq PCPU(PRVSPACE),%rax ++ addq $PC_PTI_STACK+PC_PTI_STACK_SZ*8,%rax ++ movq %rax,COMMON_TSS_RSP0(%rdx) ++1: movl $TSSSEL,%eax + ltr %ax + jmp done_tss + +--- sys/amd64/amd64/db_trace.c.orig ++++ sys/amd64/amd64/db_trace.c +@@ -200,6 +200,7 @@ + if (name != NULL) { + if (strcmp(name, "calltrap") == 0 || + strcmp(name, "fork_trampoline") == 0 || ++ strcmp(name, "mchk_calltrap") == 0 || + strcmp(name, "nmi_calltrap") == 0 || + strcmp(name, "Xdblfault") == 0) + frame_type = TRAP; +--- sys/amd64/amd64/exception.S.orig ++++ sys/amd64/amd64/exception.S +@@ -1,12 +1,16 @@ + /*- + * Copyright (c) 1989, 1990 William F. Jolitz. + * Copyright (c) 1990 The Regents of the University of California. +- * Copyright (c) 2007 The FreeBSD Foundation ++ * Copyright (c) 2007-2018 The FreeBSD Foundation + * All rights reserved. + * + * Portions of this software were developed by A. Joseph Koshy under + * sponsorship from the FreeBSD Foundation and Google, Inc. + * ++ * Portions of this software were developed by ++ * Konstantin Belousov <kib@FreeBSD.org> under sponsorship from ++ * the FreeBSD Foundation. ++ * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: +@@ -38,13 +42,13 @@ + #include "opt_compat.h" + #include "opt_hwpmc_hooks.h" + ++#include "assym.s" ++ + #include <machine/asmacros.h> + #include <machine/psl.h> + #include <machine/trap.h> + #include <machine/specialreg.h> + +-#include "assym.s" +- + #ifdef KDTRACE_HOOKS + .bss + .globl dtrace_invop_jump_addr +@@ -100,69 +104,62 @@ + MCOUNT_LABEL(user) + MCOUNT_LABEL(btrap) + +-/* Traps that we leave interrupts disabled for.. */ +-#define TRAP_NOEN(a) \ +- subq $TF_RIP,%rsp; \ +- movl $(a),TF_TRAPNO(%rsp) ; \ +- movq $0,TF_ADDR(%rsp) ; \ +- movq $0,TF_ERR(%rsp) ; \ ++/* Traps that we leave interrupts disabled for. */ ++ .macro TRAP_NOEN l, trapno ++ PTI_ENTRY \l,X\l ++ .globl X\l ++ .type X\l,@function ++X\l: subq $TF_RIP,%rsp ++ movl $\trapno,TF_TRAPNO(%rsp) ++ movq $0,TF_ADDR(%rsp) ++ movq $0,TF_ERR(%rsp) + jmp alltraps_noen +-IDTVEC(dbg) +- TRAP_NOEN(T_TRCTRAP) +-IDTVEC(bpt) +- TRAP_NOEN(T_BPTFLT) ++ .endm ++ ++ TRAP_NOEN dbg, T_TRCTRAP ++ TRAP_NOEN bpt, T_BPTFLT + #ifdef KDTRACE_HOOKS +-IDTVEC(dtrace_ret) +- TRAP_NOEN(T_DTRACE_RET) ++ TRAP_NOEN dtrace_ret, T_DTRACE_RET + #endif + + /* Regular traps; The cpu does not supply tf_err for these. */ +-#define TRAP(a) \ +- subq $TF_RIP,%rsp; \ +- movl $(a),TF_TRAPNO(%rsp) ; \ +- movq $0,TF_ADDR(%rsp) ; \ +- movq $0,TF_ERR(%rsp) ; \ ++ .macro TRAP l, trapno ++ PTI_ENTRY \l,X\l ++ .globl X\l ++ .type X\l,@function ++X\l: ++ subq $TF_RIP,%rsp ++ movl $\trapno,TF_TRAPNO(%rsp) ++ movq $0,TF_ADDR(%rsp) ++ movq $0,TF_ERR(%rsp) + jmp alltraps +-IDTVEC(div) +- TRAP(T_DIVIDE) +-IDTVEC(ofl) +- TRAP(T_OFLOW) +-IDTVEC(bnd) +- TRAP(T_BOUND) +-IDTVEC(ill) +- TRAP(T_PRIVINFLT) +-IDTVEC(dna) +- TRAP(T_DNA) +-IDTVEC(fpusegm) +- TRAP(T_FPOPFLT) +-IDTVEC(mchk) +- TRAP(T_MCHK) +-IDTVEC(rsvd) +- TRAP(T_RESERVED) +-IDTVEC(fpu) +- TRAP(T_ARITHTRAP) +-IDTVEC(xmm) +- TRAP(T_XMMFLT) ++ .endm + +-/* This group of traps have tf_err already pushed by the cpu */ +-#define TRAP_ERR(a) \ +- subq $TF_ERR,%rsp; \ +- movl $(a),TF_TRAPNO(%rsp) ; \ +- movq $0,TF_ADDR(%rsp) ; \ ++ TRAP div, T_DIVIDE ++ TRAP ofl, T_OFLOW ++ TRAP bnd, T_BOUND ++ TRAP ill, T_PRIVINFLT ++ TRAP dna, T_DNA ++ TRAP fpusegm, T_FPOPFLT ++ TRAP rsvd, T_RESERVED ++ TRAP fpu, T_ARITHTRAP ++ TRAP xmm, T_XMMFLT ++ ++/* This group of traps have tf_err already pushed by the cpu. */ ++ .macro TRAP_ERR l, trapno ++ PTI_ENTRY \l,X\l,has_err=1 ++ .globl X\l ++ .type X\l,@function ++X\l: ++ subq $TF_ERR,%rsp ++ movl $\trapno,TF_TRAPNO(%rsp) ++ movq $0,TF_ADDR(%rsp) + jmp alltraps +-IDTVEC(tss) +- TRAP_ERR(T_TSSFLT) +-IDTVEC(missing) +- subq $TF_ERR,%rsp +- movl $T_SEGNPFLT,TF_TRAPNO(%rsp) +- jmp prot_addrf +-IDTVEC(stk) +- subq $TF_ERR,%rsp +- movl $T_STKFLT,TF_TRAPNO(%rsp) +- jmp prot_addrf +-IDTVEC(align) +- TRAP_ERR(T_ALIGNFLT) ++ .endm + ++ TRAP_ERR tss, T_TSSFLT ++ TRAP_ERR align, T_ALIGNFLT ++ + /* + * alltraps entry point. Use swapgs if this is the first time in the + * kernel from userland. Reenable interrupts if they were enabled +@@ -174,25 +171,24 @@ + alltraps: + movq %rdi,TF_RDI(%rsp) + testb $SEL_RPL_MASK,TF_CS(%rsp) /* Did we come from kernel? */ +- jz alltraps_testi /* already running with kernel GS.base */ ++ jz 1f /* already running with kernel GS.base */ + swapgs + movq PCPU(CURPCB),%rdi + andl $~PCB_FULL_IRET,PCB_FLAGS(%rdi) +- movw %fs,TF_FS(%rsp) +- movw %gs,TF_GS(%rsp) +- movw %es,TF_ES(%rsp) +- movw %ds,TF_DS(%rsp) +-alltraps_testi: +- testl $PSL_I,TF_RFLAGS(%rsp) +- jz alltraps_pushregs_no_rdi ++1: SAVE_SEGS ++ movq %rdx,TF_RDX(%rsp) ++ movq %rax,TF_RAX(%rsp) ++ movq %rcx,TF_RCX(%rsp) ++ testb $SEL_RPL_MASK,TF_CS(%rsp) ++ jz 2f ++ call handle_ibrs_entry ++2: testl $PSL_I,TF_RFLAGS(%rsp) ++ jz alltraps_pushregs_no_rax + sti +-alltraps_pushregs_no_rdi: ++alltraps_pushregs_no_rax: + movq %rsi,TF_RSI(%rsp) +- movq %rdx,TF_RDX(%rsp) +- movq %rcx,TF_RCX(%rsp) + movq %r8,TF_R8(%rsp) + movq %r9,TF_R9(%rsp) +- movq %rax,TF_RAX(%rsp) + movq %rbx,TF_RBX(%rsp) + movq %rbp,TF_RBP(%rsp) + movq %r10,TF_R10(%rsp) +@@ -248,15 +244,18 @@ + alltraps_noen: + movq %rdi,TF_RDI(%rsp) + testb $SEL_RPL_MASK,TF_CS(%rsp) /* Did we come from kernel? */ +- jz 1f /* already running with kernel GS.base */ ++ jz 1f /* already running with kernel GS.base */ + swapgs + movq PCPU(CURPCB),%rdi + andl $~PCB_FULL_IRET,PCB_FLAGS(%rdi) +-1: movw %fs,TF_FS(%rsp) +- movw %gs,TF_GS(%rsp) +- movw %es,TF_ES(%rsp) +- movw %ds,TF_DS(%rsp) +- jmp alltraps_pushregs_no_rdi ++1: SAVE_SEGS ++ movq %rdx,TF_RDX(%rsp) ++ movq %rax,TF_RAX(%rsp) ++ movq %rcx,TF_RCX(%rsp) ++ testb $SEL_RPL_MASK,TF_CS(%rsp) ++ jz alltraps_pushregs_no_rax ++ call handle_ibrs_entry ++ jmp alltraps_pushregs_no_rax + + IDTVEC(dblfault) + subq $TF_ERR,%rsp +@@ -278,10 +277,7 @@ + movq %r13,TF_R13(%rsp) + movq %r14,TF_R14(%rsp) + movq %r15,TF_R15(%rsp) +- movw %fs,TF_FS(%rsp) +- movw %gs,TF_GS(%rsp) +- movw %es,TF_ES(%rsp) +- movw %ds,TF_DS(%rsp) ++ SAVE_SEGS + movl $TF_HASSEGS,TF_FLAGS(%rsp) + cld + testb $SEL_RPL_MASK,TF_CS(%rsp) /* Did we come from kernel? */ +@@ -288,31 +284,54 @@ + jz 1f /* already running with kernel GS.base */ + swapgs + 1: +- movq %rsp,%rdi ++ movq PCPU(KCR3),%rax ++ cmpq $~0,%rax ++ je 2f ++ movq %rax,%cr3 ++2: movq %rsp,%rdi + call dblfault_handler +-2: +- hlt +- jmp 2b ++3: hlt ++ jmp 3b + ++ ALIGN_TEXT ++IDTVEC(page_pti) ++ testb $SEL_RPL_MASK,PTI_CS-2*8(%rsp) ++ jz Xpage ++ swapgs ++ pushq %rax ++ pushq %rdx ++ movq %cr3,%rax ++ movq %rax,PCPU(SAVED_UCR3) ++ PTI_UUENTRY has_err=1 ++ subq $TF_ERR,%rsp ++ movq %rdi,TF_RDI(%rsp) ++ movq %rax,TF_RAX(%rsp) ++ movq %rdx,TF_RDX(%rsp) ++ movq %rcx,TF_RCX(%rsp) ++ jmp page_u + IDTVEC(page) + subq $TF_ERR,%rsp +- movl $T_PAGEFLT,TF_TRAPNO(%rsp) +- movq %rdi,TF_RDI(%rsp) /* free up a GP register */ ++ movq %rdi,TF_RDI(%rsp) /* free up GP registers */ ++ movq %rax,TF_RAX(%rsp) ++ movq %rdx,TF_RDX(%rsp) ++ movq %rcx,TF_RCX(%rsp) + testb $SEL_RPL_MASK,TF_CS(%rsp) /* Did we come from kernel? */ +- jz 1f /* already running with kernel GS.base */ ++ jz page_cr2 /* already running with kernel GS.base */ + swapgs +- movq PCPU(CURPCB),%rdi ++page_u: movq PCPU(CURPCB),%rdi + andl $~PCB_FULL_IRET,PCB_FLAGS(%rdi) +-1: movq %cr2,%rdi /* preserve %cr2 before .. */ ++ movq PCPU(SAVED_UCR3),%rax ++ movq %rax,PCB_SAVED_UCR3(%rdi) ++ call handle_ibrs_entry ++page_cr2: ++ movq %cr2,%rdi /* preserve %cr2 before .. */ + movq %rdi,TF_ADDR(%rsp) /* enabling interrupts. */ +- movw %fs,TF_FS(%rsp) +- movw %gs,TF_GS(%rsp) +- movw %es,TF_ES(%rsp) +- movw %ds,TF_DS(%rsp) ++ SAVE_SEGS ++ movl $T_PAGEFLT,TF_TRAPNO(%rsp) + testl $PSL_I,TF_RFLAGS(%rsp) +- jz alltraps_pushregs_no_rdi ++ jz alltraps_pushregs_no_rax + sti +- jmp alltraps_pushregs_no_rdi ++ jmp alltraps_pushregs_no_rax + + /* + * We have to special-case this one. If we get a trap in doreti() at +@@ -319,30 +338,71 @@ + * the iretq stage, we'll reenter with the wrong gs state. We'll have + * to do a special the swapgs in this case even coming from the kernel. + * XXX linux has a trap handler for their equivalent of load_gs(). ++ * ++ * On the stack, we have the hardware interrupt frame to return ++ * to usermode (faulted) and another frame with error code, for ++ * fault. For PTI, copy both frames to the main thread stack. + */ +-IDTVEC(prot) ++ .macro PROTF_ENTRY name,trapno ++\name\()_pti_doreti: ++ pushq %rax ++ pushq %rdx ++ swapgs ++ movq PCPU(KCR3),%rax ++ movq %rax,%cr3 ++ movq PCPU(RSP0),%rax ++ subq $2*PTI_SIZE-3*8,%rax /* no err, %rax, %rdx in faulted frame */ ++ MOVE_STACKS (PTI_SIZE / 4 - 3) ++ movq %rax,%rsp ++ popq %rdx ++ popq %rax ++ swapgs ++ jmp X\name ++IDTVEC(\name\()_pti) ++ cmpq $doreti_iret,PTI_RIP-2*8(%rsp) ++ je \name\()_pti_doreti ++ testb $SEL_RPL_MASK,PTI_CS-2*8(%rsp) /* %rax, %rdx not yet pushed */ ++ jz X\name ++ PTI_UENTRY has_err=1 ++ swapgs ++IDTVEC(\name) + subq $TF_ERR,%rsp +- movl $T_PROTFLT,TF_TRAPNO(%rsp) ++ movl $\trapno,TF_TRAPNO(%rsp) ++ jmp prot_addrf ++ .endm ++ ++ PROTF_ENTRY missing, T_SEGNPFLT ++ PROTF_ENTRY stk, T_STKFLT ++ PROTF_ENTRY prot, T_PROTFLT ++ + prot_addrf: + movq $0,TF_ADDR(%rsp) + movq %rdi,TF_RDI(%rsp) /* free up a GP register */ ++ movq %rax,TF_RAX(%rsp) ++ movq %rdx,TF_RDX(%rsp) ++ movq %rcx,TF_RCX(%rsp) ++ movw %fs,TF_FS(%rsp) ++ movw %gs,TF_GS(%rsp) + leaq doreti_iret(%rip),%rdi + cmpq %rdi,TF_RIP(%rsp) +- je 1f /* kernel but with user gsbase!! */ ++ je 5f /* kernel but with user gsbase!! */ + testb $SEL_RPL_MASK,TF_CS(%rsp) /* Did we come from kernel? */ +- jz 2f /* already running with kernel GS.base */ +-1: swapgs +-2: movq PCPU(CURPCB),%rdi ++ jz 6f /* already running with kernel GS.base */ ++ swapgs ++ movq PCPU(CURPCB),%rdi ++4: call handle_ibrs_entry + orl $PCB_FULL_IRET,PCB_FLAGS(%rdi) /* always full iret from GPF */ +- movw %fs,TF_FS(%rsp) +- movw %gs,TF_GS(%rsp) + movw %es,TF_ES(%rsp) + movw %ds,TF_DS(%rsp) + testl $PSL_I,TF_RFLAGS(%rsp) +- jz alltraps_pushregs_no_rdi ++ jz alltraps_pushregs_no_rax + sti +- jmp alltraps_pushregs_no_rdi ++ jmp alltraps_pushregs_no_rax + ++5: swapgs ++6: movq PCPU(CURPCB),%rdi ++ jmp 4b ++ *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201803140414.w2E4EEd4066873>