| Age | Commit message (Collapse) | Author |
|
|
|
arrays to use DEVMAP_ENTRY{,_END}
|
|
It's less letters, matches other similar variables and will help with
sharing code between the two architectures.
NFCI.
|
|
Sprinkle KASSERT (or KDASSERT in hot paths) for kpreempt_disabled()
when we use curcpu() and it's not immediately obvious that the caller
has preemption disabled but closer scrutiny suggests the caller has.
Note unsafe curcpu()s for syscall event counting. Not sure this is
worth changing.
Possible bugs fixed:
- cpu_irq and cpu_fiq could be preempted while trying to run softints
on this CPU.
- data_abort_handler might incorrectly think it was invoked in
interrupt context when it was only preempted and migrated to
another CPU.
- pmap_fault_fixup might report the wrong CPU logs.
(However, we don't currently run with kpreemption on aarch64, so
these are not yet real bugs fixed except if you patch it to build
with __HAVE_PREEMPTION.)
|
|
pmap_md_pdetab_init.
Call pmap_md_pdetab_fini from pmap_segtab_destroy.
|
|
- Multiple events can now be handled simultaneously.
- Counters should be configured with TPROF_IOC_CONFIGURE_EVENT in advance,
instead of being configured at TPROF_IOC_START.
- The configured counters can be started and stopped repeatedly by
PROF_IOC_START/TPROF_IOC_STOP.
- The value of the performance counter can be obtained at any timing as a 64bit
value with TPROF_IOC_GETCOUNTS.
- Backend common parts are handled in tprof.c as much as possible, and functions
on the tprof_backend side have been reimplemented to be more primitive.
- The reset value of counter overflows for profiling can now be adjusted.
It is calculated by default from the CPU clock (speed of cycle counter) and
TPROF_HZ, but for some events the value may be too large to be sufficient for
profiling. The event counter can be specified as a ratio to the default or as
an absolute value when configuring the event counter.
- Due to overall changes, API and ABI have been changed. TPROF_VERSION and
TPROF_BACKEND_VERSION were updated.
|
|
PMCR.E controls not only performance event counters but also the cycle
counter operation, and the cycle counter may be used for cpu_counter.
Similarly, the 31st bit in PMINTENCLR and PMCNTENCLR controls the cycle
counter, not performance event counters, and should not be modified.
|
|
|
|
|
|
|
|
|
|
Rename the following defines:
- _ARM_BUS_SPACE_MAP_STRONGLY_ORDERED to BUS_SPACE_MAP_NONPOSTED
- PMAP_DEV_SO to PMAP_DEV_NP
- LX_BLKPAG_ATTR_DEVICE_MEM_SO to LX_BLKPAG_ATTR_DEVICE_MEM_NP
Rename the following option:
- AARCH64_DEVICE_MEM_STRONGLY_ORDERED to AARCH64_DEVICE_MEM_NONPOSTED
|
|
|
|
above 0x0001000000000000 in /dev/mem with mmap().
|
|
Add include guard while here.
XXX Why does this aarch64-specific file have #ifdef __aarch64__?
|
|
|
|
|
|
Not all ICs need interrupts disabled to update the priority. DAIF accesses
are not cheap, so push the update of ci_cpl from pic_set_priority to the
IC's pic_set_priority callback, and let the IC driver determine whether
or not it needs interrupts disabled.
|
|
|
|
Group the fields in the header.
|
|
by creating an EFI RT pmap that can be activated / deactivated when
required.
Adds support for EFI RT to ARM_MMU_EXTENDED (ASID) 32-bit Arm machines.
On Arm64 the usage of pmapboot_enter is reduced and the mappings are
created much later in the boot process -- now in cpu_startup_hook.
Backward compatiblity for KVA mapped RT from old bootaa64.efi is
maintained.
Adding support to other platforms should be easier as a result.
|
|
- The DBM bit of the PTE is now used to determine if it is writable, and
the AF bit is treated entirely as a reference bit. A valid PTE is always
treated as readable. There can be no valid PTE that is not readable.
- LX_BLKPAG_OS_{READ,WRITE} are used only for debugging purposes,
and has been superseded by LX_BLKPAG_AF and LX_BLKPAG_DBM.
- Improve comment
The need for reference/modify emulation has been eliminated,
and access/permission faults have been reduced, however,
there has been little change in overall performance.
|
|
|
|
the n1sdp couldn't complete a build.
No noticeable change in kernel build performance.
|
|
|
|
__HAVE_STRUCT_SIGCONTEXT in order to support the old
"sigcontext" style of handlers for 32-bit binaries.
However, we only support 32-bit EABI binaries on AArch64,
and by happy accident (due to a libc bug introduced in
2006), 32-bit NetBSD EABI binaries never used "sigcontext"
style handlers. So, we don't need to carry any of this
baggage forward.
This addresses the AArch64 case of PR kern/56487.
|
|
COPTS=-O0,
sprinkle `__always_inline' to make _mcount() be generated as a single function.
|
|
works.
|
|
|
|
The AP initialisation code in cpu_init_secondary_processor will read and
initialise the required system registers and state for the BP to attach
and report.
Rework the interrupt handler code for this new sequence. Thankfully,
this removes a bunch of code for bcm2836mp.
The VFP detection handler on <= armv7 relies on the global undefined
handler being in place until the BP attaches vfp. That is, after the
APs have been spun up.
gicv3_its.c has a serialisation issue which is protected against in
the gicv3_its_cpu_init, which is called from cpu_hatch, with a spin
lock. The serialisation issue needs addressing more completely.
Tested on RPI3, Apple M1, QEMU, and lx2k
Fixes PR port-arm/56264:
diagnostic assertion "l->l_stat == LSONPROC" failed on RPI3
|
|
|
|
versions. Instead, use the version constants from <sys/signal.h>
and automatically (and correctly) handle cases where multiple versions
of a particular trampoline flavor exist. Conditionalize support
for sigcontext trampolines on __HAVE_STRUCT_SIGCONTEXT.
- aarch64 and amd64 don't use sigcontext natively, but do need to
support it for 32-bit compatibility; define __HAVE_STRUCT_SIGCONTEXT
conditionally on _KERNEL.
|
|
|
|
|
|
|
|
|
|
and some PPC kernels do. This removes the limitation of 256 processes on
CPUs with 8bit ASID field, e.g. Apple M1.
Additionally the following changes have been made
- removed a couple of unnecessary aarch64_tlbi_all calls
- removed any invalidation after freeing page tables due to
_pmap_sweep_pdp. This was never necessary afaict.
- all kernel mappings are marked global and userland mapping not-global.
Performance testing hasn't show a significant difference. The data here
is from building a kernel on an lx2k system with nvme.
before
1489.6u 400.4s 2:40.65 1176.5% 228+224k 0+32289io 57pf+0w
1482.6u 403.2s 2:38.49 1189.9% 228+222k 0+32274io 46pf+0w
1485.4u 402.2s 2:37.27 1200.2% 228+222k 0+32275io 12pf+0w
after
1493.9u 404.6s 2:37.50 1205.4% 227+221k 0+32265io 48pf+0w
1485.0u 408.0s 2:38.54 1194.0% 227+222k 0+32272io 36pf+0w
1484.3u 407.0s 2:35.88 1213.3% 228+224k 0+32268io 14pf+0w
>>> stats.ttest_ind([160.65,158.49,157.27], [157.5,158.54,155.88])
Ttest_indResult(statistic=1.1923622711296888, pvalue=0.2990182944606766)
>>>
|
|
hold ASID_MAX + 1.
|
|
|
|
Avoid any kind of register access (DAIF, PMR, etc), barriers, and atomic
operations in the common case where no interrupt fires between spl being
raised and lowered.
This introduces a per-CPU return address (ci_splx_restart) used by the
vector handler to restart a sequence in splx that compares the new ipl
with the per-CPU hardware priority state stored in ci_hwpl.
|
|
|
|
|
|
to get profiling data for each CPU.
In the current implementation, locks are acquired at the entrance of the mcount
internal function, so the higher the number of cores, the more lock conflict
occurs, making profiling performance in a MULTIPROCESSOR environment unusable
and slow. Profiling buffers has been changed to be reserved for each CPU,
improving profiling performance in MP by several to several dozen times.
- Eliminated cpu_simple_lock in mcount internal function, using per-CPU buffers.
- Add ci_gmon member to struct cpu_info of each MP arch.
- Add kern.profiling.percpu node in sysctl tree.
- Add new -c <cpuid> option to kgmon(8) to specify the cpuid, like openbsd.
For compatibility, if the -c option is not specified, the entire system can be
operated as before, and the -p option will get the total profiling data for
all CPUs.
|
|
Move 'struct pic_pending' from percpu to struct cpu_info. Saves a few
instructions in splx.
There is(/was) no need to use atomic operations on the percpu / cpu_info
members, so don't.
Finally removng the use of percpu should help avoid problems with "late"
attaching cpus.
|
|
PMU implementation.
|
|
|
|
referenced since some sources include <machine/param.h>.
|
|
recognise CPUs with only 8bit ASIDs.
|
|
|
|
|