| Age | Commit message (Collapse) | Author |
|
This so that the rather large cc1 from gcc12 can be run.
OK'ed by simonb@
|
|
|
|
|
|
Details in comments.
PR kern/57240
XXX pullup-8
XXX pullup-9
XXX pullup-10
|
|
For !MULTIPROCESSOR, define them to be empty or nop as appropriate.
|
|
|
|
|
|
|
|
This is based on code given to me by Matt Thomas a long time ago with
many updates and bugs fixes from me.
|
|
|
|
|
|
|
|
|
|
Using <mips/asm.h> in a .c file is kinda grody but CALLFRAME_SIZ
doesn't seem to be defined anywhere else. Not sure how this was ever
supposed to work...
|
|
This change deletes memory barriers on non-Octeon MP. However, all
the appropriate acquire and release barriers are already used in
mutex stubs, and no barriers are needed in atomic_* unless we set
__HAVE_ATOMIC_AS_MEMBAR which we don't on MIPS. So this should be
safe.
Unclear whether we need this even on Octeon -- don't have a clear
reference on why it's here.
|
|
BDSYNC is used for membar_sync, which is supposed to be a full
sequential consistency barrier, which is not provided by syncw, so
this is necessary for correctness.
BDSYNC is not used for anything else, so this can't hurt performance,
except where it was necessary for correctness anyway or where the
semantic choice of membar_sync was too strong anyway.
|
|
This change deletes a memory barrier. However, it should be safe:
The semantic requirement for this is already provided by the SYNC_REL
above, before the ll. And as currently defined, SYNC_REL is at least
as strong as SYNC, so this change can't hurt correctness on its own
(barring CPU errata, which would apply to other users of SYNC_REL and
can be addressed in the definition of SYNC_REL).
Later, perhaps we can relax SYNC_REL to syncw on Octeon if we prove
that it is correct (e.g., if Octeon follows the SPARCv9 partial store
order semantics).
Nix now-unused SYNC macro in asm.h.
|
|
This change should be safe because it doesn't remove or weaken any
memory barriers, but does add, clarify, or strengthen barriers.
Goals:
- Make sure mutex_enter/exit and mutex_spin_enter/exit have
acquire/release semantics.
- New macros make maintenance easier and purpose clearer:
. SYNC_ACQ is for load-before-load/store barrier, and BDSYNC_ACQ
for a branch delay slot -- currently defined as plain sync for MP
and nothing, or nop, for UP; thus it is no weaker than SYNC and
BDSYNC as currently defined, which is syncw on Octeon, plain sync
on non-Octeon MP, and nothing/nop on UP.
It is not clear to me whether load-then-syncw or ll/sc-then-syncw
or even bare load provides load-acquire semantics on Octeon -- if
no, this will fix bugs; if yes (like it is on SPARC PSO), we can
relax SYNC_ACQ to be syncw or nothing later.
. SYNC_REL is for load/store-before-store barrier -- currently
defined as plain sync for MP and nothing for UP.
It is not clear to me whether syncw-then-store is enough for
store-release on Octeon -- if no, we can leave this as is; if
yes, we can relax SYNC_REL to be syncw on Octeon.
. SYNC_PLUNGER is there to flush clogged Cavium store buffers, and
BDSYNC_PLUNGER for a branch delay slot -- syncw on Octeon,
nothing or nop on non-Octeon.
=> This is not necessary (or, as far as I'm aware, sufficient)
for acquire semantics -- it serves only to flush store buffers
where stores might otherwise linger for hundreds of thousands
of cycles, which would, e.g., cause spin locks to be held for
unreasonably long durations.
Newerish revisions of the MIPS ISA also have finer-grained sync
variants that could be plopped in here.
Mechanism:
Insert these barriers in the right places, replacing only those where
the definition is currently equivalent, so this change is safe.
- Replace #ifdef _MIPS_ARCH_OCTEONP / syncw / #endif at the end of
atomic_cas_* by SYNC_PLUNGER, which is `sync 4' (a.k.a. syncw) if
__OCTEON__ and empty otherwise.
=> From what I can tell, __OCTEON__ is defined in at least as many
contexts as _MIPS_ARCH_OCTEONP -- i.e., there are some Octeons
with no _MIPS_ARCH_OCTEONP, but I don't know if any of them are
relevant to us or ever saw the light of day outside Cavium; we
seem to buid with `-march=octeonp' so this is unlikely to make a
difference. If it turns out that we do care, well, now there's
a central place to make the distinction for sync instructions.
- Replace post-ll/sc SYNC by SYNC_ACQ in _atomic_cas_*, which are
internal kernel versions used in sys/arch/mips/include/lock.h where
it assumes they have load-acquire semantics. Should move this to
lock.h later, since we _don't_ define __HAVE_ATOMIC_AS_MEMBAR on
MIPS and so the extra barrier might be costly.
- Insert SYNC_REL before ll/sc, and replace post-ll/sc SYNC by
SYNC_ACQ, in _ucas_*, which is used without any barriers in futex
code and doesn't mention barriers in the man page so I have to
assume it is required to be a release/acquire barrier.
- Change BDSYNC to BDSYNC_ACQ in mutex_enter and mutex_spin_enter.
This is necessary to provide load-acquire semantics -- unclear if
it was provided already by syncw on Octeon, but it seems more
likely that either (a) no sync or syncw is needed at all, or (b)
syncw is not enough and sync is needed, since syncw is only a
store-before-store ordering barrier.
- Insert SYNC_REL before ll/sc in mutex_exit and mutex_spin_exit.
This is currently redundant with the SYNC already there, but
SYNC_REL more clearly identifies the necessary semantics in case we
want to define it differently on different systems, and having a
sync in the middle of an ll/sc is a bit weird and possibly not a
good idea, so I intend to (carefully) remove the redundant SYNC in
a later change.
- Change BDSYNC to BDSYNC_PLUNGER at the end of mutex_exit. This has
no semantic change right now -- it's syncw on Octeon, sync on
non-Octeon MP, nop on UP -- but we can relax it later to nop on
non-Cavium MP.
- Leave LLSCSYNC in for now -- it is apparently there for a Cavium
erratum, but I'm not sure what the erratum is, exactly, and I have
no reference for it. I suspect these can be safely removed, but we
might have to double up some other syncw instructions -- Linux uses
it only in store-release sequences, not at the head of every ll/sc.
|
|
- Eradicate last vestiges of mb_* barriers.
- In __cpu_simple_lock_init, omit needless barrier. It is the
caller's responsibility to ensure __cpu_simple_lock_init happens
before other operations on it anyway, so there was never any need
for a barrier here.
- In __cpu_simple_lock_try, leave comments about memory ordering
guarantees of the kernel's _atomic_cas_uint, which are inexplicably
different from the non-underscored atomic_cas_uint.
- In __cpu_simple_unlock, use membar_exit instead of mb_memory, and do
it unconditionally.
This ensures that in __cpu_simple_lock/.../__cpu_simple_unlock, all
memory operations in the ellipsis happen before the store that
releases the lock.
- On Octeon, the barrier was omitted altogether, which is a bug --
it needs to be there or else there is no happens-before relation
and whoever takes the lock next might see stale values stored or
even stomp over the unlocking CPU's delayed loads.
- On non-Octeon, the mb_memory was sync. Using membar_exit
preserves this.
XXX On Octeon, membar_exit only issues syncw -- this seems wrong,
only store-before-store and not load/store-before-store, unless the
CNMIPS architecture guarantees it is sufficient here like
SPARCv8/v9 PSO (`Partial Store Order').
- Leave an essay with citations about why we have an apparently
pointless syncw _after_ releasing a lock, to work around a design
bug^W^Wquirk in cnmips which sometimes buffers stores for hundreds
of thousands of cycles for fun unless you issue syncw.
|
|
everywhere.
|
|
|
|
another copyright claim line. again. (i did this in 2008 and then
did not update all of my personal templates.)
|
|
register.
|
|
|
|
|
|
|
|
- Define __HAVE_STRUCT_SIGCONTEXT if _KERNEL (because it's needed for
32-bit binary compatibility) or if the O32 ABI is active (because
that's the only ABI that ever used sigcontext for signal delivery).
- For _KERNEL, define a "struct sigcontext" suitable only for 32-bit
compatible signal delivery.
- For userspace, define a "struct sigcontext" appropriate for any ABI
if _LIBC is defined (it's used for setjmp / longjmp) or if O32 is
the active ABI (because it was part of the old BSD signal API).
|
|
|
|
- sparc and sparc64 were not using version 0 sigcontext when there were
no arguments in the signal version. This was probably a bug.
- vax is using +1 the version numbers of the other archs.
- Only hppa was defining __LIBC12_SOURCE__ so it was getting a working
sigcontext before. all the other ports that supported sigcontext had
the compat code disabled.
[pointed out by thorpej, thanks!]
If we want to remove sigcontext support from userland at least now there
is less work to do so.
|
|
|
|
atomic_cas_ulong().
- For arm, ia64, m68k, mips, or1k, riscv, vax: don't define our own
MUTEX_CAS(), as they either use atomic_cas_ulong() or equivalent
(atomic_cas_uint() on m68k).
- For alpha and sparc64, don't define MUTEX_CAS() in terms of their own
_lock_cas(), which has its own memory barriers; the call sites in
kern_mutex.c already have the appropriate memory barrier calls. Thus,
alpha and sparc64 can use default definition.
- For sh3, don't define MUTEX_CAS() in terms of its own _lock_cas();
atomic_cas_ulong() is strong-aliased to _lock_cas(), therefore defining
our own MUTEX_CAS() is redundant.
Per thread:
https://mail-index.netbsd.org/tech-kern/2021/07/25/msg027562.html
|
|
to get profiling data for each CPU.
In the current implementation, locks are acquired at the entrance of the mcount
internal function, so the higher the number of cores, the more lock conflict
occurs, making profiling performance in a MULTIPROCESSOR environment unusable
and slow. Profiling buffers has been changed to be reserved for each CPU,
improving profiling performance in MP by several to several dozen times.
- Eliminated cpu_simple_lock in mcount internal function, using per-CPU buffers.
- Add ci_gmon member to struct cpu_info of each MP arch.
- Add kern.profiling.percpu node in sysctl tree.
- Add new -c <cpuid> option to kgmon(8) to specify the cpuid, like openbsd.
For compatibility, if the -c option is not specified, the entire system can be
operated as before, and the -p option will get the total profiling data for
all CPUs.
|
|
|
|
|
|
referenced since some sources include <machine/param.h>.
|
|
modern (MIPS32/MIPS64) and convert to __BIT/__BITS.
|
|
mips_emul_ll() and mips_emul_sc(); make these static to mips_emul.c.
|
|
this has been returning "mipsn64eb" on my edgerouter4 with the
32 bit uname binary.
introduce o32, n32, and n64 versions of MACHINE_ARCH, and use
them appropriately in PROC_MACHINE_ARCH32(). now o32, n32 and
n64 "uname -p" all return different values.
|
|
|
|
|
|
|
|
|
|
differentiate them from n32/o32 binaries.
|
|
Update a few comments.
|
|
|
|
|
|
comment.
|
|
|
|
|
|
|