summaryrefslogtreecommitdiff
path: root/sys/kern/init_main.c
AgeCommit message (Collapse)Author
2023-07-07heartbeat(9): New mechanism to check progress of kernel.riastradh
This uses hard interrupts to check progress of low-priority soft interrupts, and one CPU to check progress of another CPU. If no progress has been made after a configurable number of seconds (kern.heartbeat.max_period, default 15), then the system panics -- preferably on the CPU that is stuck so we get a stack trace in dmesg of where it was stuck, but if the stuckness was detected by another CPU and the stuck CPU doesn't acknowledge the request to panic within one second, the detecting CPU panics instead. This doesn't supplant hardware watchdog timers. It is possible for hard interrupts to be stuck on all CPUs for some reason too; in that case heartbeat(9) has no opportunity to complete. Downside: heartbeat(9) relies on hardclock to run at a reasonably consistent rate, which might cause trouble for the glorious tickless future. However, it could be adapted to take a parameter for an approximate number of units that have elapsed since the last call on the current CPU, rather than treating that as a constant 1. XXX kernel revbump -- changes struct cpu_info layout
2022-10-26kern/init_main.c: Get extern lwp0 from sys/lwp.h.riastradh
2022-07-21Removed unused opt_wapbl.h include.simonb
2022-06-18fix typos in word "functions" in comments, mainly s/fuctions/functions/.andvar
2022-03-19Fix locking after opendisk(), VOP_IOCTL() needs an unlocked vnode,hannken
vn_rdwr() needs flag IO_NODELOCKED.
2022-03-18entropy(9): Establish the softint a little earlier.riastradh
Just need to wait until softint_establish and high-priority xcalls will work, no later than that. Doing this earlier gives us slightly more of a chance to ensure cprng_fast and ssp get entropy from hardware RNG devices that rely on interrupts.
2022-01-26remove double t from targeted, add missing r to arbitraryandvar
And fix few more typos along the way in comments and man pages.
2021-04-01Expose olde style intrcnt interrupt accounting via event counters.simonb
This code will be garbage collected once our last legacy intrcnt user is update to native evcnts.
2020-12-05Refactor interval timers to make it possible to support types other thanthorpej
the BSD/POSIX per-process timers: - "struct ptimer" is split into "struct itimer" (common interval timer data) and "struct ptimer" (per-process timer data, which contains a "struct itimer"). - Introduce a new "struct itimer_ops" that supplies information about the specific kind of interval timer, including it's processing queue, the softint handle used to schedule processing, the function to call when the timer fires (which adds it to the queue), and an optional function to call when the CLOCK_REALTIME clock is changed by a call to clock_settime() or settimeofday(). - Rename some fuctions to clearly identify what they're operating on (ptimer vs itimer). - Use kmem(9) to allocate ptimer-related structures, rather than having dedicated pools for them. Welcome to NetBSD 9.99.77.
2020-11-12Set a better default for MAXFILES on larger RAM machines if notsimonb
otherwise specified the kernel config file. Arbitary numbers are 20,000 files for 16GB RAM or more and 10,000 files for 1GB RAM or more. TODO: Adjust this and other values totally dynamically.
2020-11-04In uvmpd_tryownerlock(), if the initial try-lock of the owner lock failschs
then rather than do more try-locks and eventually sleep for a tick, take a hold on the current owner's lock, drop the page interlock, and acquire the lock that we took the hold on in a blocking fashion. After we get the lock, check if the lock that we acquired is still the lock for the owner of the page that we're interested in. If the owner hasn't changed then can proceed with this page, otherwise we will skip this page and move on to a different page. This dramatically reduces the amount of time that the pagedaemon sleeps trying to get locks, since even 1 tick is an eternity to sleep in this context and it was easy to trigger that case in practice, and with this new method the pagedaemon only very rarely actually blocks to acquire the lock that it wants since the object locks are adaptive, and when the pagedaemon does block then the amount of time it spends sleeping will be generally be much less than 1 tick.
2020-09-08ipi: Split up initialization into two parts.riastradh
First part runs early so ipi_register can be used in module initialization, e.g. via pktqueue_create; second part runs after CPUs have been detected.
2020-09-07Add the ability to set an alternate cnmagic in the kernel configthorpej
file, e.g.: options CNMAGIC="\"+++++\""
2020-08-27Move address hashing from init_main.c to kern_sysctl.c.riastradh
This way rump gets it automatically. Make sure blake2s is in librumpkern.so, not just in librumpkern_crypto.so, for this to work.
2020-08-26Instead of returning 0 when sysctl kern.expose_address=0, return a randomchristos
hashed value of the data. This allows sockstat to work without exposing kernel addresses or being setgid kmem.
2020-06-11uvm_availmem(): give it a boolean argument to specify whether a recentad
cached value will do, or if the very latest total must be fetched. It can be called thousands of times a second and fetching the totals impacts not only the calling LWP but other CPUs doing unrelated activity in the VM system.
2020-05-23Move proc_lock into the data segment. It was dynamically allocated becausead
at the time we had mutex_obj_alloc() but not __cacheline_aligned.
2020-05-11Move cprng_init before configure.riastradh
This makes it available to device drivers, e.g. to generate MAC addresses at random, without initialization order hacks. Requires a minor initialization hack for cpu_name(primary cpu) early on, since that doesn't get set until mi_cpu_attach which may not run until the middle of configure. But this hack is less bad than other initialization order hacks.
2020-04-30Rewrite entropy subsystem.riastradh
Primary goals: 1. Use cryptography primitives designed and vetted by cryptographers. 2. Be honest about entropy estimation. 3. Propagate full entropy as soon as possible. 4. Simplify the APIs. 5. Reduce overhead of rnd_add_data and cprng_strong. 6. Reduce side channels of HWRNG data and human input sources. 7. Improve visibility of operation with sysctl and event counters. Caveat: rngtest is no longer used generically for RND_TYPE_RNG rndsources. Hardware RNG devices should have hardware-specific health tests. For example, checking for two repeated 256-bit outputs works to detect AMD's 2019 RDRAND bug. Not all hardware RNGs are necessarily designed to produce exactly uniform output. ENTROPY POOL - A Keccak sponge, with test vectors, replaces the old LFSR/SHA-1 kludge as the cryptographic primitive. - `Entropy depletion' is available for testing purposes with a sysctl knob kern.entropy.depletion; otherwise it is disabled, and once the system reaches full entropy it is assumed to stay there as far as modern cryptography is concerned. - No `entropy estimation' based on sample values. Such `entropy estimation' is a contradiction in terms, dishonest to users, and a potential source of side channels. It is the responsibility of the driver author to study the entropy of the process that generates the samples. - Per-CPU gathering pools avoid contention on a global queue. - Entropy is occasionally consolidated into global pool -- as soon as it's ready, if we've never reached full entropy, and with a rate limit afterward. Operators can force consolidation now by running sysctl -w kern.entropy.consolidate=1. - rndsink(9) API has been replaced by an epoch counter which changes whenever entropy is consolidated into the global pool. . Usage: Cache entropy_epoch() when you seed. If entropy_epoch() has changed when you're about to use whatever you seeded, reseed. . Epoch is never zero, so initialize cache to 0 if you want to reseed on first use. . Epoch is -1 iff we have never reached full entropy -- in other words, the old rnd_initial_entropy is (entropy_epoch() != -1) -- but it is better if you check for changes rather than for -1, so that if the system estimated its own entropy incorrectly, entropy consolidation has the opportunity to prevent future compromise. - Sysctls and event counters provide operator visibility into what's happening: . kern.entropy.needed - bits of entropy short of full entropy . kern.entropy.pending - bits known to be pending in per-CPU pools, can be consolidated with sysctl -w kern.entropy.consolidate=1 . kern.entropy.epoch - number of times consolidation has happened, never 0, and -1 iff we have never reached full entropy CPRNG_STRONG - A cprng_strong instance is now a collection of per-CPU NIST Hash_DRBGs. There are only two in the system: user_cprng for /dev/urandom and sysctl kern.?random, and kern_cprng for kernel users which may need to operate in interrupt context up to IPL_VM. (Calling cprng_strong in interrupt context does not strike me as a particularly good idea, so I added an event counter to see whether anything actually does.) - Event counters provide operator visibility into when reseeding happens. INTEL RDRAND/RDSEED, VIA C3 RNG (CPU_RNG) - Unwired for now; will be rewired in a subsequent commit.
2020-04-26Add a NetBSD native futex implementation, mostly written by riastradh@.thorpej
Map the COMPAT_LINUX futex calls to the native ones.
2020-02-24move config_init_mi() call before vfsinit(), which can trigger loadingjdolecek
of VFS modules fixes crash with LOCKDEBUG due to uninitialized mutex when zfs module is loaded in boot, because zfs's spa_init() calls config_mountroot() which now requires the config init having been done
2020-02-18remove the aiodoned thread. I originally added this to provide a thread contextchs
for doing page cache iodone work, but since then biodone() has changed to hand off all iodone work to a softint thread, so we no longer need the special-purpose aiodoned thread.
2020-02-15- Move the LW_RUNNING flag back into l_pflag: updating l_flag without lockad
in softint_dispatch() is risky. May help with the "softint screwup" panic. - Correct the memory barriers around zombies switching into oblivion.
2020-01-28Call radix_tree_init() earlier, so more stuff can make use of radixtree.ad
2020-01-08Hopefully fix some problems seen with MP support on non-x86, in particularad
where curcpu() is defined as curlwp->l_cpu: - mi_switch(): undo the ~2007ish optimisation to unlock curlwp before calling cpu_switchto(). It's not safe to let other actors mess with the LWP (in particular l->l_cpu) while it's still context switching. This removes l->l_ctxswtch. - Move the LP_RUNNING flag into l->l_flag and rename to LW_RUNNING since it's now covered by the LWP's lock. - Ditch lwp_exit_switchaway() and just call mi_switch() instead. Everything is in cache anyway so it wasn't buying much by trying to avoid saving old state. This means cpu_switchto() will never be called with prevlwp == NULL. - Remove some KERNEL_LOCK handling which hasn't been needed for years.
2020-01-02- Eliminate the global "boottime" variable, which was being accessedthorpej
without any synchronization against changes by e.g. clock_settime(). - Replace with new getbinboottime() / getnanoboottime() / getmicroboottime() functions (naming mirrors that of other time access functions in kern_tc.c). It returns the (maybe-converted) value of timebasebin, which also tracks our estimate of when the system was booted (i.e. the legacy "boottime" was redundant). XXX There needs to be a lockless synchronization mechanism for reading timebasebin, but this is a problem in kern_tc.c that pre-existed these "boottime" changes. At least now the problem is centralized in one location.
2020-01-01- Introduce a new global kernel variable "shutting_down" to indicate thatthorpej
the system is shutting down or rebooting. - Set this global in a new function called kern_reboot(), which is currently just a basic wrapper around cpu_reboot(). - Call kern_reboot() instead of cpu_reboot() almost everywhere; a few places remain where it's still called directly, but those are in early pre-main() machdep locations. Eventually, all of the various cpu_reboot() functions should be re-factored and common functionality moved to kern_reboot(), but that's for another day.
2020-01-01First steps towards properly serializing access to the TOD clock.thorpej
- Add a mutex around the TODR, and provide lock/unlock/lock-owned functions to manipulate it. - Rename inittodr() to todr_set_systime() and resettodr() to todr_save_systime() to better reflect what they do. These functions are intended to be called with the TODR lock held, which will allow for a pattern like: -> todr_lock() -> todr_save_systime() -> [do machine-dependent stuff to sleep/suspend] -> [magically awaken] -> todr_set_systime(...) -> todr_unlock() - Provide historically-named wrappers inittodr() and resettodr() that do the dance of acquiring / releasing the lock around the actual substance. NOTE: resettodr()'s use of the TODR lock is currently disabled (and todr_save_systime() does not assert it's held) until such time as issues around shutdown / reboot under duress can be addressed.
2019-12-31Rename uvm_free() -> uvm_availmem().ad
2019-12-27Redo the page allocator to perform better, especially on multi-core andad
multi-socket systems. Proposed on tech-kern. While here: - add rudimentary NUMA support - needs more work. - remove now unused "listq" from vm_page.
2019-12-22Fix integer overflow when printing available memory size (resulting fromad
a cast lost during merges). Reported-by: syzbot+f02ca5f83ac7196b8afd@syzkaller.appspotmail.com
2019-12-21uvmexp.free -> uvm_free()ad
2019-12-14Include radixtree in the kernel.ad
2019-12-12Eliminate per-hook duplication of common code as suggested bypgoyette
(and with major contributions from) riastradh@ Welcome to 9.99.23
2019-12-02Take the basic CPU topology information we already collect, and use itad
to make circular lists of CPU siblings in the same core, and in the same package. Nothing fancy, just enough to have a bit of fun in the scheduler trying out different tactics.
2019-12-01Init kern_runq and kern_synch before booting secondary CPUs.ad
2019-10-03Remove compile-time asserts checking whether intptr_t and void* are compatkamil
The checks were requested by core@ as a prerequisite for kevent::udata type switch from intptr_t to void*.
2019-09-24Add a temporary ctassert checking whether void* and intptr_t are compatiblekamil
2019-05-17Implement an aggressive psref leak detectorozaki-r
It is yet another psref leak detector that enables to tell where a leak occurs while a simpler version that is already committed just tells an occurrence of a leak. Investigating of psref leaks is hard because once a leak occurs a percpu list of psref that tracks references can be corrupted. A reference to a tracking object is memorized in the list via an intermediate object (struct psref) that is normally allocated on a stack of a thread. Thus, the intermediate object can be overwritten on a leak resulting in corruption of the list. The tracker makes a shadow entry to an intermediate object and stores some hints into it (currently it's a caller address of psref_acquire). We can detect a leak by checking the entries on certain points where any references should be released such as the return point of syscalls and the end of each softint handler. The feature is expensive and enabled only if the kernel is built with PSREF_DEBUG. Proposed on tech-kern
2019-02-20Attach "mnt_transinfo" to "dead_rootmount" so every mount has ahannken
valid "mnt_transinfo" and remove now unneeded flag IMNT_HAS_TRANS. Run fstrans_start()/fstrans_done() on dead_rootmount if FSTRANS_DEAD_ENABLED. Should become the default for DIAGNOSTIC in the future.
2019-01-23Change the place of initproc initializationkamil
The initproc variable cannot be initialized in start_init as there is a race between vfs_mountroot and start_init. PR kern/53817 by Andreas Gustafsson
2018-12-26Rather than performing lazy initialization, statically initialize earlythorpej
in the respective kernel startup routines.
2018-10-30Correct the 6 second offset issue between the time reported bykre
dmesg -T and the actual time a message was produced, noted on current-users by Geoff Wing (Oct 27, 2018). The size of the offset would depend upon architecture, and processor, but was the delay from starting the clocks to initialising the time of day (after mounting root, in case that is needed). Change the kernel to set boottime to be the time at which the clocks were started, rather than the time at which it is init'd (by subtracting the interval between). Correct dmesg to properly compute the ToD based upon the boottime (which is a timespec, not a timeval, and has been since Jan 2009) and the time logged in the message. Note that this can (rarely) be 1 second earlier than date reports. This occurs when the time when the message was logged was actually in the next second, but the timecounters have not yet processed the tick, and so the time of the last tick, near the end of the previous second, is reported instead. Since times are always truncated, rather than rounded, it is occasionally possible to observe that disparity (if you try hard enough). IOW: sys/kern/subr_prf.c:addtstamp() uses getnanouptime() rather than nanouptime(). Note in dmesg(8) that -T conversions are gibberish other than when the message comes from current the running kernel. (It could be fixed when -M is used, for messages generated by the kernel whose corpse is being observed. But hasn't been...)
2018-10-26Only print the "no console" warning when booting verbose or debug.martin
It is a normal condition in many setups and has no consequences for the user, so do not scare them.
2018-07-03Fix net.inet6.ip6.ifq node doesn't existozaki-r
The node (and child nodes) is initialized in sysctl_net_pktq_setup, but the call of sysctl_net_pktq_setup is skipped unexpectedly. sysctl_net_pktq_setup is skipped if in6_present is false that indicates the netinet6 component isn't loaded on rump kernels. However the flag is accidentally always false because the flag is turned on in in6_dom_init that is called after if_sysctl_setup on both normal and rump kernels. Fix the issue by moving if_sysctl_setup after in6_dom_init (domaininit on normal kernels). This fix is ad-hoc but good enough for netbsd-8. We should refine the initialization order of network components in the future. Pointed out by hikaru@
2018-04-16Remove the rnewprocp argument from fork1(9)kamil
It's now unused and it can cause use-after-free scenarios as noted by <Mateusz Guzik>. Reference: http://mail-index.netbsd.org/tech-kern/2017/09/08/msg022267.html Sponsored by <The NetBSD Foundation>
2018-04-16Set initproc inside start_init()kamil
This allows us to stop using the rnewprocp argument in fork1(9). The rnewprocp argument will be removed soon from the API, as it can cause use-after-free scenarios. No functional change intended. Noted by <Mateusz Guzik> Reference: http://mail-index.netbsd.org/tech-kern/2017/09/08/msg022267.html Sponsored by <The NetBSD Foundation>
2018-02-04Add a proper defflag for GPROF, and include opt_gprof.h, otherwise we'remaxv
not gonna go very far.
2017-12-26 Make cold __read_mostly like mp_online.msaitoh
2017-12-15add some assertions to verify that CPU_INFO_FOREACH() works rightchs
early in the boot process. this detects existing bugs on some platforms.