powerpc: powerpc_ipi_handler(): Stop first, preempt last
Preemption should happen last so that other IPIs have a chance to be
executed in due course, in particular hardclockintr(). So, move
handling of IPI_PREEMPT at the end.
An IPI_STOP/IPI_STOP_HARD request should be processed immediately to
avoid risks that further processing changes the system state on panic,
or worse itself causes another panic on corrupted kernel state.
While here, move IPI_AST handling just before IPI_PREEMPT. We currently
do not need to do anything for it, but if we were to, it is likely that
would have to happen after other IPIs but IPI_PREEMPT.
Reviewed by: jhibbits
Fixes: 9d5a22b928c2 ("MFp4: SMP support")
Fixes: 12640815f815 ("MFp4: SMP support")
Fixes: 7ed09a6778d3 ("powerpc: Rework IPI message processing")
MFC after: 1 week
[4 lines not shown]
x86: ipi_bitmap_handler(): Preempt last, reduce critical section
Preemption should happen last so that other IPIs have a chance to be
executed in due course, in particular hardclockintr(). So, move
handling of IPI_PREEMPT at the end.
Reduce the critical section to be in force only around hardclockintr(),
which needs it to execute entirely without preemption (because of code
it calls that can enter a critical section; interrupts are already
disabled in this handler, as usual).
The critical_exit() following hardclockintr() may cause preemption if
due. If that happens, there is no need to preempt again on IPI_PREEMPT,
so make sure sched_preempt() is not called in this case.
This fixes inadvertent removal of the optimization that sched_preempt()
should be called outside of a critical section to perform an immediate
context switch in the case of IPI_HARDCLOCK and IPI_PREEMPT being
handled at the same time.
[12 lines not shown]
x86: ipi_bitmap_handler(): Account for IPIs correctly on COUNT_IPIS
A preemption could occur before the updates to the initial CPU's IPI
counters, leading to a potential non-atomic updates of these counters
from another CPU.
Reviewed by: kib
Fixes: ff326a187907 ("x86: Restore the critical section (...)")
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D60862
(cherry picked from commit 3a0e2ca3eecd3176fa7dd139e59913a67c858ac9)
sched_ule: Use TDQ_CPU() macro instead of fiddling with DPCPU
No functional change (intended).
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
(cherry picked from commit 79ca9f787ca92018a1374242bb77252af6fce0d0)
sched: INVARIANTS: Check sched_preempt() is called on 'curthread'
sched_preempt() does not support operating on a remote thread. If you
are looking to influence CPU allocation on another thread, see, e.g.,
cpuset_setthread(), which in the end calls sched_affinity().
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
(cherry picked from commit f19009020a33d07d7cad9393b4bf0a36f267cd07)
sched_ule: sched_affinity(): Delineate steps
No functional change (intended).
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
(cherry picked from commit a1fe7f00bc452653426493c511d288e5a4e7862d)
sched_ule: tdq_runq_add(): Scope variables, delineate the different steps
For better readability.
No functional change (intended).
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
(cherry picked from commit d5e9c642307c13b8058609f421b63e2c72bab45f)
sched_ule: tdq_notify(): Fix a typo in a comment
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
(cherry picked from commit ea1da6a51a5aa0c949fc125aece6b88a68e67a3a)
systm.h: Remove unused <sys/kpilite.h>
It only contains sched_pin_lite() and sched_unpin_lite(), which have
been unused in-tree for the past 8 years, and were only briefly used
between 6573d7580b85 ("epoch(9): allow preemptible epochs to compose")
and a760c50c9ea7 ("With epoch not inlined, there is no point in using
_lite KPI. (...)").
Additionally, these functions are redundant with the existing
sched_pin() and sched_unpin() in <sys/sched.h>, which are already
themselves inline functions. The only difference is that the
sched_*_lite() ones do not require the full definition of 'struct
thread', which may have been useful to circumvent namespace pollution.
That problem, if it materializes again, could be quickly circumvented
without this trickery by the use of non-inline functions, or using
inline functions referencing a variable filled at kernel startup with
the desired offset. The trickery of using offsets computed at
compile-time appears necessary only in the case of extremely performance
[9 lines not shown]
sched.h: Include <sys/pcpu.h> unconditionally
Even if SCHED_STATS is not defined, this header is necessary to provide
a definition of 'curthread' used in sched_pin() and sched_unpin().
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
(cherry picked from commit b6d10a962448bf03400dd381b2083d1f547f869c)
sched.h: Fix a typo and remove an extra line
Fixes: 42490d5cd29d ("sched: New scheduler interface definition scheme")
Sponsored by: The FreeBSD Foundation
(cherry picked from commit 0bccbb9e98930f55bff5a943c881e460d4f31aec)
sched: New scheduler interface definition scheme
Define the scheduler interface once and for all (in 'sys/sys/sched.h')
and remove all code duplication related to it (function signatures, slot
names, dispatch, scheduler instance declaration), making it easier to
modify the interface or to add new schedulers.
This is implemented by defining the interface as X macros, which are
passed a macro (and additional arguments for it) that is "called" with
a variable number of arguments describing a single function of the
interface. The convention used for a function's parameters is that each
parameter is represented with two macro arguments, the first one being
the type and the second one being the name. Helper macros allow to
process arguments described in this convention in order to generate
a list of arguments for function definitions (currently, up to
4 function parameters). The chosen convention eliminates the need for
any specific declaration of functions depending on their number of
arguments.
[50 lines not shown]
cred: freebsd14_setgroups(): Remove a now-unnecessary local variable
MFC with: e6fbef451dd4 ("cred: Fix a race in the FreeBSD-14-compatible setgroups(2)")
Sponsored by: The FreeBSD Foundation
(cherry picked from commit e40d60838c9a0ebe5e5fb0e3cb00e7b08dd8c959)
cred: Fix a race in the FreeBSD-14-compatible setgroups(2)
The freebsd14_setgroups() function would try to modify the effective GID
on the current process' credentials without holding the process lock,
allowing races with other threads concurrently modifying the process
credentials. In the worst case, freebsd14_setgroups() could be
manipulating a 'struct ucred' already freed by another thread (in the
very small window after reading 'p_ucred' without lock but before
modifying the effective GID). Concurrent uses of freebsd14_setgroups()
or setcred() could also lead to non-atomic credentials modifications.
Fix this by making kern_setgroups() take a new boolean indicating
whether the passed array includes the effective GID in its first slot.
When this boolean is true, it internally keeps the effective GID in
a separate variable, pretends that the groups[] array that was passed
actually starts at 'groups + 1', do the usual steps to set the
supplementary groups and new extra ones to set the effective GID along,
without releasing the process lock in between.
[7 lines not shown]
amdsmu(4): Fix 32-bit compile
For what is worth, since i386 is going away soon and still does not
compile.
Fixes: cd96657b6708 ("amdsmu: Print how long system spent in S0i3")
Sponsored by: The FreeBSD Foundation
sys/modules.h: Fix too long line
No functional change (intended).
Fixes: f4b3640475ce ("inline atomics and allow tied modules to inline locks")
Sponsored by: The FreeBSD Foundation
bnxt_en: update the driver version to 239.1.153.0
bnxt_en driver version has not been updated for
a while, updating the version now so it better
reflects the current state of the driver.
Sponsored by: Broadcom Inc
linuxkpi: Do not wait for a disabled tasklet in tasklet_unlock_wait()
tasklet_unlock_wait() waited for the tasklet state to become idle. Linux
only waits for TASKLET_STATE_RUN, i.e. for a tasklet that is currently
executing. A tasklet which is scheduled but disabled does not run until
it is enabled again, so in LinuxKPI it stays in the BUSY (queued) state and
the waiter never returns.
This is what happens when unloading i915: execlists_reset_prepare()
disables the engine tasklet with __tasklet_disable_sync_once() while the
tasklet is queued and kldunload loops forever in pause().
Do not wait if the tasklet is only queued and disabled.
Reviewed by: bz
Sponsored by: Beckhoff Automation GmbH & Co. KG
linuxkpi: Fix lkpi_i2c_del_adapter() leaving the devices behind
lkpi_i2c_del_adapter() looked for the lkpi_iic or lkpi_iicbb device to
delete with device_find_child() on increasing unit numbers, and stopped at
the first unit that does not exist. Units are allocated globally per
devclass, so as soon as one adapter was removed (leaving a hole) or when the
units of a parent do not start at 0, the adapter was not found and its
device was never deleted. The i2c devices of a driver such as i915 then
accumulate, with a new set of unit numbers, each time it is loaded.
Walk the children of the parent instead and match on the adapter.
Sponsored by: Beckhoff Automation GmbH & Co. KG
linuxkpi: Release devres on PCI driver detach
linux_pci_detach_device() called the driver remove method and then only
dropped its reference on the device, expecting the devres list to be
released from lkpi_pci_dev_release() once the last reference is gone.
That never happens when a devres action itself holds a reference on the
device. The typical case is devm_drm_dev_alloc(): the drm_device takes a
reference on its parent and the matching drm_dev_put() is a devres
action. The device is thus never released, drm_dev_release() is never
called and the DRM device and everything it owns is leaked, which makes
unloading drm-kmod drivers fail ("failed due to N other pending
references on the LinuxKPI device") and leaves the module unloadable or
crashing.
Linux releases the devres resources when a driver is unbound, so do the
same: call lkpi_devres_release_free_list() right after the remove method.
The list is empty when lkpi_pci_dev_release() runs later.
[2 lines not shown]
linuxkpi: Fix sysfs_remove_group() for groups without a name
sysfs_create_group() handles a group with a NULL name by creating its
attributes directly under the object node, but sysfs_remove_group() passed
the NULL name to sysctl_remove_name(), which then panics in strcmp().
Mirror the creation path: for an unnamed group, remove each attribute and
binary attribute individually.
This was not hit before because the devres actions of a PCI device
(e.g. the one registered by devm_device_add_group()) were not run when
the driver was detached.
Sponsored by: Beckhoff Automation Gmbh & Co. KG
libc/locale: Check EILSEQ return
_citrus_iconv_std_iconv_convert can return EILSEQ without setting
invlen. As a result, we should check EILSEQ in the return value or it
will incorrectly return -2
Reviewed by: kevans
Fixes: e2030ca246a7 ("libc: fix c*rtomb/mbrtoc*")
MFC after: 2 days
Differential Revision: https://reviews.freebsd.org/D61151
sched: Fix building kernel independently of world
Regular '-I' directives must be passed to ${CPP} so that when
pre-processed, 'sched_ns.h' can find the files it includes in the proper
places, instead of relying on what is in ${WORLDTMP}, i.e., nothing if
'_includes' was not executed, which happens, e.g., when using 'make
kernel-toolchain' and then 'make buildkernel' without having executed
a 'make buildworld', or possibly some outdated copy not matching an
updated source.
Reported by: Claude Opus 5.5, imp
Fixes: 510331fa3763 ("sched: Automatically namespace schedulers' implementation functions")
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D61167
Makefile.inc1: Fix bare $ characters
The '$' characters inside awk commands need to be $$ when appearing
inside a Makefile. Without this, we generate empty src and src-sys
packages.
Reported by: Stefan Grundmann
Fixes: a8be89928930 ("build: Support got")
MFC after: 1 day
bnxt: correct more mappings for 400G optics and 100G LR
- Support 400G-DR4 optics
- Support 400G-CR8 and -SR8 with their types, rather than generic 400G-AUI8
- Use the generic 400G-AUI8 for 400G-SR4, rather than mislabeling it -DR4
(We probably should add a 400G-SR media type, but I don't have any to test ATM)
- Add missing legacy PAM4 100G LR entry and matching media-selection cases.
Reviewed by: sumit.saxena_broadcom.com
Differential Revision: https://reviews.freebsd.org/D61162
Sponsored by: Netflix
sched: Automatically namespace schedulers' implementation functions
This renames back scheduler implementation functions to their old names,
i.e., the same name as the interface functions, in the source code,
while providing the infrastructure to still produce from them C symbols
namespaced by the scheduler implementation, in order to avoid symbol
collisions with interface dispatching functions and other scheduler
implementations when multiple ones are compiled in. In a nutshell, the
implementation consists in re-routing interface names to implementation
names via the C preprocessor.
The main reasons for this change are that different implementation names:
1. Break navigating code with ctags-like tools like in-tree ctags,
Global and Universal Ctags.
2. Spoils inlining opportunities in internal scheduler code if not
careful. This alternatively could be regained by changing call
points to use scheduler-specific names, but that does not prevent
people from inadvertently adding more calls to interface functions.
Also, having to namespace calls with the implementation name in the
[75 lines not shown]
