summaryrefslogtreecommitdiff
path: root/sys/arch/amd64/include/cpu.h
AgeCommit message (Collapse)Author
2023-04-09amd64: Make curlwp and curcpu() flushable.riastradh
The only effect of the `volatile' qualifier on an asm block with outputs is to force the instructions to appear in the generated code, even if the outputs end up being unused. Since these instructions have no (architectural) side effects -- provided %gs is set correctly, which must be the case here -- there's no need for the volatile qualifier, so nix it.
2021-11-02In order to prevent _mcount() from being recursively called when built with ↵ryo
COPTS=-O0, sprinkle `__always_inline' to make _mcount() be generated as a single function.
2021-04-17sys/arch/amd64: remove trailing whitespacerillig
2020-03-17Add a redzone between the pcb and the stack. Sent to port-amd64@.maxv
2019-12-08Use the inlines; it is actually fine, since the compiler drops the inlinesmaxv
if the caller is kmsan-instrumented, forcing a white-listing of the memory access.
2019-11-21mi_userret(): take care of calling preempt(), set spc_curpriority directly,ad
and remove MD code that does the same.
2019-11-14Add support for Kernel Memory Sanitizer (kMSan). It detects uninitializedmaxv
memory used by the kernel at run time, and just like kASan and kCSan, it is an excellent feature. It has already detected 38 uninitialized variables in the kernel during my testing, which I have since discreetly fixed. We use two shadows: - "shad", to track uninitialized memory with a bit granularity (1:1). Each bit set to 1 in the shad corresponds to one uninitialized bit of real kernel memory. - "orig", to track the origin of the memory with a 4-byte granularity (1:1). Each uint32_t cell in the orig indicates the origin of the associated uint32_t of real kernel memory. The memory consumption of these shadows is consequent, so at least 4GB of RAM is recommended to run kMSan. The compiler inserts calls to specific __msan_* functions on each memory access, to manage both the shad and the orig and detect uninitialized memory accesses that change the execution flow (like an "if" on an uninitialized variable). We mark as uninit several types of memory buffers (stack, pools, kmem, malloc, uvm_km), and check each buffer passed to copyout, copyoutstr, bwrite, if_transmit_lock and DMA operations, to detect uninitialized memory that leaves the system. This allows us to detect kernel info leaks in a way that is more efficient and also more user-friendly than KLEAK. Contrary to kASan, kMSan requires comprehensive coverage, ie we cannot tolerate having one non-instrumented function, because this could cause false positives. kMSan cannot instrument ASM functions, so I converted most of them to __asm__ inlines, which kMSan is able to instrument. Those that remain receive special treatment. Contrary to kASan again, kMSan uses a TLS, so we must context-switch this TLS during interrupts. We use different contexts depending on the interrupt level. The orig tracks precisely the origin of a buffer. We use a special encoding for the orig values, and pack together in each uint32_t cell of the orig: - a code designating the type of memory (Stack, Pool, etc), and - a compressed pointer, which points either (1) to a string containing the name of the variable associated with the cell, or (2) to an area in the kernel .text section which we resolve to a symbol name + offset. This encoding allows us not to consume extra memory for associating information with each cell, and produces a precise output, that can tell for example the name of an uninitialized variable on the stack, the function in which it was pushed on the stack, and the function where we accessed this uninitialized variable. kMSan is available with LLVM, but not with GCC. The code is organized in a way that is similar to kASan and kCSan, so it means that other architectures than amd64 can be supported.
2019-02-11We reorganise definitions for XEN source support as follows:cherry
XEN - common sources required for baseline XEN support. XENPV - sources required for support of XEN in PV mode. XENPVHVM - sources required for support for XEN in HVM mode. XENPVH - sources required for support for XEN in PVH mode.
2018-11-18On Xen, copy just the bits we need from the trapframe for hardclock(9)cherry
and statclock(9). Current, the macros that use the trapframe are: CLKF_USERMODE() CLKF_PC() CLKF_INTR() Of these, CLKF_INTR() already ignores the frame and uses the ci_idepth variable to do its job. Convert the two remaining ones to do this, but only for XEN.
2018-03-16Remove the prototypes for cpu_uarea_*, I removed these functions twomaxv
minutes ago.
2017-09-17Remove the second argument from USERMODE and KERNELMODE, it is unusedmaxv
now that we don't have vm86 anymore.
2012-01-21allocate uareas contiguously and access them via the direct map.chs
2008-12-30_LKM -> _MODULEpooka
2008-10-26put the contents of these header files around #ifdef __x86_64__, andmrg
#include the <i386/foo.h> in the #else clause, making these files largely bit-size independant.
2008-05-22Mark x86_curlwp() with __attribute__ ((const)), so gcc can CSE it and knowad
that it does not clobber global data.
2008-05-11Wrap stuff in #ifdef _KERNELad
2008-05-11Share cpu.h between the x86 ports.ad
2008-05-11Simplify x86 identcpu code, and share between i386/amd64.ad
2008-05-10Improve x86 tsc handling:ad
- Ditch the cross-CPU calibration stuff. It didn't work properly, and it's near impossible to synchronize the CPUs in a running system, because bus traffic will interfere with any calibration attempt, messing up the timings. - Only enable the TSC on CPUs where we are sure it does not drift. If we are On a known good CPU, give the TSC high timecounter quality, making it the default. - When booting CPUs, detect TSC skew and account for it. Most Intel MP systems have synchronized counters, but that need not be true if the system has a complicated bus structure. As far as I know, AMD systems do not have synchronized TSCs and so we need to handle skew. - While an AP is waiting to be set running, try and make the TSC drift by entering a reduced power state. If we detect drift, ensure that the TSC does not get a high timecounter quality. This should not happen and is only for safety. - Make cpu_counter() stuff LKM safe.
2008-05-09Make cpu_idle a macro calling a function pointer on x86.joerg
Select the Xen idle routine for Xen, mwait if supported by the CPU and it is not AMD and halt otherwise. As reported by Christoph Egger, AMD Barcelona keeps the CPU in C0 state with MWAIT, contrary to HLT, which uses C1 and therefore much less power.
2008-04-30Avoid unneeded AST faults.ad
2008-04-28Add support for kernel preeemption to the i386 and amd64 ports. Notes:ad
- I have seen one isolated panic in the x86 pmap, but otherwise i386 seems stable with preemption enabled. - amd64 is missing the FPU handling changes and it's not yet safe to enable it there. - The usual level for kern.sched.kpreempt_pri will be 128 once enabled by default. For testing, setting it to 0 helps to shake out bugs.
2008-04-24- Give ci_want_resched a single cache line, and align. This is for monitor/ad
mwait. At least one errata sheet from Intel notes that a single line should be used. - Align cc_microtime.
2008-04-23Ensure that offsetof() is in scope by including <sys/systm.h>.he
Fixes build problem found while building swapnetbsd.o for XEN3_DOM0.
2008-04-21Access Xen's vcpu info structure per-CPU.cegger
Tested on i386 and amd64 (both dom0 and domU) by me. Xen2 tested (both dom0 and domU) by bouyer. OK bouyer
2008-04-16use POSIX integer typescegger
2008-02-27Remove CTL_MACHDEP_NAMES, it's not used anywhere.xtraeme
Ok by martin@.
2008-01-22GC i8254_microtime.joerg
2008-01-05remove no longer necessary cpu_maxproc.yamt
2008-01-05- make amd64 use per-cpu tss.yamt
- fix iopl syscall for amd64+xen.
2008-01-05g/c unused membersyamt
2008-01-05g/c ci_idle_pcb_paddryamt
2008-01-01try to detect processor resource sharing topologies. ie. package/core/smt IDs.yamt
2007-12-25Convert many of the uses of __attribute__ to equivalentperry
__packed, __unused and __dead macros from cdefs.h
2007-12-22Define 'struct intrframe' in terms of 'struct trapframe' since the two aredsl
assumed to match by a lot of code (including that which saves the regs). This only slightly reduces the number of places the trapframe register layout is defined.
2007-12-18Add new IPI for saving CPU state explicitly, share high-level part ofjoerg
ACPI wakeup code and teach it how to start the APs again. As a side effect the CPU_START interface allows choosing between different bootstrap codes more easily now.
2007-12-09Merge jmcneill-pm branch.jmcneill
2007-12-03Add a CPU local timer based on the LAPIC. This is consistently fasterjoerg
than TSC, but doesn't suffer from SpeedStep as TSC does. The default quality is higher than HPET for UP, but -100 for MULTIPROCESSOR as it needs CPU local state which doesn't exist yet.
2007-11-22Pull up the bouyer-xenamd64 branch to HEAD. This brings in amd64 supportbouyer
to NetBSD/Xen, both Dom0 and DomU.
2007-11-12- cpu_vendor was both an int and char[] on amd64 - fix it.ad
- Run the errata check/patch on all CPUs, not just the boot processor.
2007-10-29Mark cpu_info::ci_tlbstate volatile to ensure that the compiler doesn'tad
reorder accesses to it. It's updated from the TLB IPI handlers and we don't block those, so the order in which things are read/updated is important.
2007-10-26Match delay/DELAY on x86 with delay(9). It takes an unsigned int asjoerg
argument. Use this and replace the inline assembly (mul + div using the 64bit intermediate result) with normal 32bit multiplication and division. The compiler can turn the division into a multiplication and shift, making it even cheaper then the original assembly. For extreme long delays, just use 64bit arithmetic.
2007-10-18merge yamt-x86pmap branch.yamt
- reduce differences between amd64 and i386. notably, share pmap.c between them. it makes several i386 pmap improvements available to amd64, including tlb shootdown reduction and bug fixes from Stephan Uphoff. - implement deferred pmap switching for amd64. - remove LARGEPAGES option. always use large pages if available. also, make it work on amd64.
2007-10-17Merge the ppcoea-renovation branch to HEAD.garbled
This branch was a major cleanup and rototill of many of the various OEA cpu based PPC ports that focused on sharing as much code as possible between the various ports to eliminate near-identical copies of files in every tree. Additionally there is a new PIC system that unifies the interface to interrupt code for all different OEA ppc arches. The work for this branch was done by a variety of people, too long to list here. TODO: bebox still needs work to complete the transition to -renovation. ofppc still needs a bunch of work, which I will be looking at. ev64260 still needs to be renovated amigappc was not attempted. NOTES: pmppc was removed as an arch, and moved to a evbppc target.
2007-09-26x86 changes for pcc and LKMs.ad
- Replace most inline assembly with proper functions. As a side effect this reduces the size of amd64 GENERIC by about 120kB, and i386 by a smaller amount. Nearly all of the inlines did something slow, or something that does not need to be fast. - Make curcpu() and curlwp functions proper, unless __GNUC__ && _KERNEL. In that case make them inlines. Makes curlwp LKM and preemption safe. - Make bus_space and bus_dma more LKM friendly. - Share a few more files between the ports. - Other minor changes.
2007-09-25ci_astpending is no more.ad
2007-08-29Merge most x86 changes from the vmlocking branch, except the threaded softad
interrupt stuff. This is mostly comprised of changes to the pmap modules to work on multiprocessor systems without kernel_lock, and changes to speed up tlb shootdowns.
2007-05-21Revert fs/gs changes until I figure out issues with them.fvdl
2007-05-17merge yamt-idlelwp branch. asked by core@. some ports still needs work.yamt
from doc/BRANCHES: idle lwp, and some changes depending on it. 1. separate context switching and thread scheduling. (cf. gmcgarry_ctxsw) 2. implement idle lwp. 3. clean up related MD/MI interfaces. 4. make scheduler(s) modular.
2007-05-11Don't save/restore %fs and %gs in trapframe. The kernel won't touch them.fvdl
Instead, save/restore them on context switch. For 32bit processes, save/restore the selector values only, for 64bit processes, save/restore the appropriate MSRs. Iff the defaults have been changed.