summaryrefslogtreecommitdiff
path: root/sys/kern/kern_exec.c
AgeCommit message (Collapse)Author
2022-07-01posix_spawn(2): Plug leak in proc_alloc error branch.riastradh
2022-04-09sys: Use membar_release/acquire around reference drop.riastradh
This just goes through my recent reference count membar audit and changes membar_exit to membar_release and membar_enter to membar_acquire -- this should make everything cheaper on most CPUs without hurting correctness, because membar_acquire is generally cheaper than membar_enter.
2022-03-12sys: Membar audit around reference count releases.riastradh
If two threads are using an object that is freed when the reference count goes to zero, we need to ensure that all memory operations related to the object happen before freeing the object. Using an atomic_dec_uint_nv(&refcnt) == 0 ensures that only one thread takes responsibility for freeing, but it's not enough to ensure that the other thread's memory operations happen before the freeing. Consider: Thread A Thread B obj->foo = 42; obj->baz = 73; mumble(&obj->bar); grumble(&obj->quux); /* membar_exit(); */ /* membar_exit(); */ atomic_dec -- not last atomic_dec -- last /* membar_enter(); */ KASSERT(invariant(obj->foo, obj->bar)); free_stuff(obj); The memory barriers ensure that obj->foo = 42; mumble(&obj->bar); in thread A happens before KASSERT(invariant(obj->foo, obj->bar)); free_stuff(obj); in thread B. Without them, this ordering is not guaranteed. So in general it is necessary to do membar_exit(); if (atomic_dec_uint_nv(&obj->refcnt) != 0) return; membar_enter(); to release a reference, for the `last one out hit the lights' style of reference counting. (This is in contrast to the style where one thread blocks new references and then waits under a lock for existing ones to drain with a condvar -- no membar needed thanks to mutex(9).) I searched for atomic_dec to find all these. Obviously we ought to have a better abstraction for this because there's so much copypasta. This is a stop-gap measure to fix actual bugs until we have that. It would be nice if an abstraction could gracefully handle the different styles of reference counting in use -- some years ago I drafted an API for this, but making it cover everything got a little out of hand (particularly with struct vnode::v_usecount) and I ended up setting it aside to work on psref/localcount instead for better scalability. I got bored of adding #ifdef __HAVE_ATOMIC_AS_MEMBAR everywhere, so I only put it on things that look performance-critical on 5sec review. We should really adopt membar_enter_preatomic/membar_exit_postatomic or something (except they are applicable only to atomic r/m/w, not to atomic_load/store_*, making the naming annoying) and get rid of all the ifdefs.
2022-02-05Prevent escallation of privilege due to poor handling of argc == 0 in set*idchristos
binaries by refusing to execute them.
2021-11-26Fix anonymous memory object leak for sigcode.ryo
- Repeating "modload compat_linux && /emul/linux/bin/ls && modunload compat_linux" will reproduce this problem. - It cause in exec_sigcode_map(), anon-object for sigcode was created at first exec, but it remained even after exec_remove. - Fixed that the anon-object for sigcode is created at exec_add(), and the anon-object reference is removed at exec_remove(). - sigobject_lock is no longer needed since it is locked by exec_lock. - The compat_16 module rewrites the e_sigcode entry in emul_netbsd directly and does not use exec_add()/exec_remove(), so it needs to call sigcode_alloc()/sigcode_free() on its own.
2021-11-25Reverte my previous changes kern_exec.c r1.512. It panics.ryo
This changes was insufficient because es_emul is referenced by multiple execsw.
2021-11-25Fix anonymous memory object leak for sigcode.ryo
- Repeating "modload compat_linux && /emul/linux/bin/ls && modunload compat_linux" will reproduce this problem. - It cause in exec_sigcode_map(), anon-object for sigcode was created at first exec, but it remained even after exec_remove. - Fixed that the anon-object for sigcode is created at exec_add(), and the anon-object reference is removed at exec_remove(). - sigobject_lock is no longer needed since it is locked by exec_lock.
2021-11-07Merge the kernel portion of the posix-spawn-chdir project by Piyush Sachdeva.christos
2021-10-10Changes to make EVFILT_PROC MP-safe:thorpej
Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-09-28Make sure the robust futex head is zeroed out, since this LWPthorpej
will live on with a different program image. (Thanks ryo@ for pointing out my mistake.)
2021-09-28futex_release_all_lwp(): No need to pass the "tid" argument separately; thatthorpej
is a vestige of an older version of the code. Also, move a KASSERT() that both futex_release_all_lwp() call sites had inside of futex_release_all_lwp() itself.
2021-09-28In the exec path, multi-LWP programs dispose of their robust futexesthorpej
by calling exit_lwps(), except for the last LWP. So, dispose of that LWP's robust futexes right before calling lwp_ctl_exit(). Fixes a "WARNING: ... : unmapped robust futex list head" message when running bash under Linux emulation on aarch64. Root caused and patch proposed by ryo@. I have tweaked it slightly, just to add a comment and a KASSERT().
2021-06-11Fix the order of handling of posix_spawn attributes and file actions.martin
The standard is explicit about it and it matters if e.g. RESETIDS is used as an attribute and file actions depend on the group rights for opening a file.
2021-05-02Fix copy&pasto in handling of POSIX_SPAWN_RESETIDS in posix_spawn(3)martin
2020-12-05Refactor interval timers to make it possible to support types other thanthorpej
the BSD/POSIX per-process timers: - "struct ptimer" is split into "struct itimer" (common interval timer data) and "struct ptimer" (per-process timer data, which contains a "struct itimer"). - Introduce a new "struct itimer_ops" that supplies information about the specific kind of interval timer, including it's processing queue, the softint handle used to schedule processing, the function to call when the timer fires (which adds it to the queue), and an optional function to call when the CLOCK_REALTIME clock is changed by a call to clock_settime() or settimeofday(). - Rename some fuctions to clearly identify what they're operating on (ptimer vs itimer). - Use kmem(9) to allocate ptimer-related structures, rather than having dedicated pools for them. Welcome to NetBSD 9.99.77.
2020-11-25Define LMSG outside the MAXTSIZ check so it also exists in non-MAXTSIZ kernels.wiz
2020-10-06Make MAXTSIZ optional.christos
2020-05-23Move proc_lock into the data segment. It was dynamically allocated becausead
at the time we had mutex_obj_alloc() but not __cacheline_aligned.
2020-05-07On debugger attach to a prestarted process don't report SIGTRAPkamil
Introduce PSL_TRACEDCHILD that indicates tracking of birth of a process. A freshly forked process checks whether it is traced and if so, reports SIGTRAP + TRAP_CHLD event to a debugger as a result of tracking forks-like events. There is a time window when a debugger can attach to a newly created process and receive SIGTRAP + TRAP_CHLD instead of SIGSTOP. Fixes races in t_ptrace_wait* tests when a test hangs or misbehaves, especially the ones reported in tracer_sysctl_lookup_without_duplicates.
2020-04-24Overhaul the way LWP IDs are allocated. Instead of each LWP having it'sthorpej
own LWP ID space, LWP IDs came from the same number space as PIDs. The lead LWP of a process gets the PID as its LID. If a multi-LWP process's lead LWP exits, the PID persists for the process. In addition to providing system-wide unique thread IDs, this also lets us eliminate the per-process LWP radix tree, and some associated locks. Remove the separate "global thread ID" map added previously; it is no longer needed to provide this functionality. Nudged in this direction by ad@ and chs@.
2020-04-21Revert the changes made in February to make cwdinfo use mostly lockless,ad
which relied on taking extra vnode refs. Having benchmarked various experimental changes over the past few months it seems that it's better to avoid vnode refs as much as possible. cwdi_lock as a RW lock already did that to some extent for getcwd() and will permit the same for namei() too.
2020-04-19- Only increment nprocs when we're creating a new process, not justthorpej
when allocating a PID. - Per above, proc_free_pid() no longer decrements nprocs. It's now done in proc_free() right after proc_free_pid(). - Ensure nprocs is accessed using atomics everywhere.
2020-04-14Set p_oppid always, not just when a parent is tracedkamil
PR kern/55151 by Martin Husemann
2020-04-06Reintroduce struct proc::p_oppidkamil
Relying on p_opptr is not safe as there is a race between: - spawner giving a birth to a child process and being killed - spawnee accessng p_opptr and reporting TRAP_CHLD PR kern/54786 by Andreas Gustafsson
2020-04-05- Untangle spawn_return by splitting it up to sub-functions.christos
- Merge the eventswitch parent notification code which was copied in two places (eventswitchchild) - Fix bugs in the eventswitch parent notification code: 1. p_slflags should be accessed holding both proc_lock and p->p_lock 2. p->p_opptr can be NULL if the parent was PSL_CHTRACED and exited. Fixes random crashes the posix_spawn_kill_spawner unit test which tried to dereference a NULL pptr.
2020-02-23Merge from ad-namecache:ad
- Have a stab at clustering the members of vnode_t and vnode_impl_t in a more cache-conscious way. With that done, go back to adjusting v_usecount with atomics and keep vi_lock directly in vnode_impl_t (saves KVA). - Allow VOP_LOCK(LK_NONE) for the benefit of VFS_VGET() and VFS_ROOT(). Make sure LK_UPGRADE always comes with LK_NOWAIT. - Make cwdinfo use mostly lockless.
2020-02-15PR kern/54922: 9.99.45@20200202 panic: diagnostic assertion linux ldconfig ↵ad
triggers vpp != NULL in exit1()->radixtree.c line 674 Create an lwp_renumber() from the code in emulexec() and use in linux_e_proc_exec() and linux_e_proc_fork() too.
2020-02-10- check for errors in exec_resolvename() and failchristos
- put back the compat_linux modules in the exec array (commented out) - remove extra parens
2020-01-29- Track LWPs in a per-process radixtree. It uses no extra memory in thead
single threaded case. Replace scans of p->p_lwps with lookups in the tree. Find free LIDs for new LWPs in the tree. Replace the hashed sleep queues for park/unpark with lookups in the tree under cover of a RW lock. - lwp_wait(): if waiting on a specific LWP, find the LWP via tree lookup and return EINVAL if it's detached, not ESRCH. - Group the locks in struct proc at the end of the struct in their own cache line. - Add some comments.
2020-01-23exec_lock: declare it in the header, and mark with __cachline_aligned.ad
2020-01-12A final set of scheduler tweaks:ad
- Try hard to keep vfork() parent and child on the same CPU until execve(), failing that on the same core, but in all other cases scatter new LWPs among the different CPU packages, round robin, to try and get the best out of the available cache and bus bandwidth. - Remove attempts at balancing. Replace with a rate-limited skim of other CPU's run queues in sched_idle(), starting in the current package and moving outwards. Add a sysctl tunable to change the interval. - Make the cacheht_time tuneable take a milliseconds value. - It's possible to configure things such that there's no CPU allowed to run an LWP. Defeat this by always having a default: Reported-by: syzbot+46968944dd9359ab93bc@syzkaller.appspotmail.com Reported-by: syzbot+7f750a4cc230d1e831f9@syzkaller.appspotmail.com Reported-by: syzbot+88d7675158f5cb4684db@syzkaller.appspotmail.com Reported-by: syzbot+d409c2338150e9a8ae1e@syzkaller.appspotmail.com Reported-by: syzbot+e152dc5bff188f67358a@syzkaller.appspotmail.com
2020-01-12Tidy up the vnode locking around execve() on ELF images to acquire andad
release the locks fewer times. Proposed on tech-kern a very long time go.
2020-01-08Hopefully fix some problems seen with MP support on non-x86, in particularad
where curcpu() is defined as curlwp->l_cpu: - mi_switch(): undo the ~2007ish optimisation to unlock curlwp before calling cpu_switchto(). It's not safe to let other actors mess with the LWP (in particular l->l_cpu) while it's still context switching. This removes l->l_ctxswtch. - Move the LP_RUNNING flag into l->l_flag and rename to LW_RUNNING since it's now covered by the LWP's lock. - Ditch lwp_exit_switchaway() and just call mi_switch() instead. Everything is in cache anyway so it wasn't buying much by trying to avoid saving old state. This means cpu_switchto() will never be called with prevlwp == NULL. - Remove some KERNEL_LOCK handling which hasn't been needed for years.
2019-12-06Make it possible to call mi_switch() and immediately switch to another CPU.ad
This seems to take about 3us on my Intel system. Two changes required: - Have the caller to mi_switch() be responsible for calling spc_lock(). - Avoid using l->l_cpu in mi_switch(). While here: - Add a couple of calls to membar_enter() - Have the idle LWP set itself to LSIDL, to match softint_thread(). - Remove unused return value from mi_switch().
2019-11-23Minor scheduler cleanup:ad
- Adapt to cpu_need_resched() changes. Avoid lost & duplicate IPIs and ASTs. sched_resched_cpu() and sched_resched_lwp() contain the logic for this. - Changes for LSIDL to make the locking scheme match the intended design. - Reduce lock contention and false sharing further. - Numerous small bugfixes, including some corrections for SCHED_FIFO/RT. - Use setrunnable() in more places, and merge cut & pasted code.
2019-10-12Remove p_oppid from struct prockamil
This field is not needed as it duplicated p_opptr that is alread safe to use, unless proven otherwise. eventswitch() already contained a check for != initproc (pid1). Ride ABI bump for 9.99.16.
2019-09-30Move TRAP_CHLD/TRAP_LWP ptrace information from struct proc to siginfokamil
Storing struct ptrace_state information inside struct proc was vulnerable to synchronization bugs, as multiple events emitted in the same time were overwritting other ones. Cache the original parent process id in p_oppid. Reusing here p_opptr is in theory prone to slight race codition. Change the semantics of PT_GET_PROCESS_STATE, reutning EINVAL for calls prompting for the value in cases when there wasn't registered an appropriate event. Add an alternative approach to check the ptrace_state information, directly from the siginfo_t value returned from PT_GET_SIGINFO. The original PT_GET_PROCESS_STATE approach is kept for compat with older NetBSD and OpenBSD. New code is recommended to keep using PT_GET_PROCESS_STATE. Add a couple of compile-time asserts for assumptions in the code. No functional change intended in existing ptrace(2) software. All ATF ptrace(2) and ATF GDB tests pass. This change improves reliability of the threading ptrace(2) code.
2019-09-17Add a boolean argument to indicate if we have a path/true (execve) or anchristos
fd/false (fexecve). This is needed to differentiate between them because NULL/-1 can be readily passed from userland.
2019-09-15- Add support for fexecvechristos
- get the vnode from the fd passed instead of calling namei() on the path - try to reverse resolve the vnode to extract the pathname - deal with not having a resolved path available - rename variable that was not a pathbuf
2019-09-07- move quadruplicated code into a functionchristos
- delete #if 1 and #if 0 code
2019-07-05Fix info leak. The padding of 'sigact' is not initialized, it gets copiedmaxv
in the proc, and can later be obtained by userland.
2019-06-27Fix this fucking shit once and for all, for fuck's sake.maxv
2019-06-27remove offs initialization and XXX gcc comment. Offs should always bechristos
initialized. Pointed out by maxv.
2019-06-27Return an error if the path was too long. Pointed out by maxvchristos
2019-06-26Remove useless debugging messages which achieved nothing but hiding bugs.maxv
2019-06-26whitespace around operatorschristos
2019-06-25Fail if getcwd fails. Pointed out by maxv@christos
2019-06-25Fix word (direct -> directory) in comment.wiz
2019-06-25add a comment explaining what this does.christos
2019-06-25Fix buffer overflow. It seems that some people need to go back to themaxv
basics of C programming. Reported-by: syzbot+8665827f389a9fac5cc9@syzkaller.appspotmail.com