arm64: Use the fault handler when one is provided
In align_abort() and tag_check_abort(), if we got a fault while in kernel,
do not panic if a fault handler has been provided. We may get such a fault
when trying to read or write userland data, it can at least happen with
_umtx_op() if an unaligned pointer is provided. Instead, just let the
fault handler deal with it.
MFC After: 1 week
Approved by: andrew
Differential Revision: https://reviews.freebsd.org/D58426
(cherry picked from commit c6f5d8fb269fd67a8206420b4e7d67a93bc80733)
Signed-off-by: Olivier Houchard <cognet at FreeBSD.org>
(cherry picked from commit 60dd46db19f473af7aa505c0d8dcf9bb69847019)
Signed-off-by: Olivier Houchard <cognet at FreeBSD.org>
arm64: Use the fault handler when one is provided
In align_abort() and tag_check_abort(), if we got a fault while in kernel,
do not panic if a fault handler has been provided. We may get such a fault
when trying to read or write userland data, it can at least happen with
_umtx_op() if an unaligned pointer is provided. Instead, just let the
fault handler deal with it.
MFC After: 1 week
Approved by: andrew
Differential Revision: https://reviews.freebsd.org/D58426
(cherry picked from commit c6f5d8fb269fd67a8206420b4e7d67a93bc80733)
Signed-off-by: Olivier Houchard <cognet at FreeBSD.org>
sys/socket.h: Fix AF_MAX
AF_MAX was always intended to be one more than the greatest allocated
value. Jeff broke this in 2013. Unfortunately, a bunch of people then
decided to adapt to the mistake instead of correcting it.
Fixes: 863c7e45628d (" - Reserve a special AF for SDP. The one we were incorrectly using before was taken by another AF.")
MFC after: 3 days
Sponsored by: Klara, Inc.
Sponsored by: NetApp, Inc.
Reviewed by: kevans, glebius
Differential Revision: https://reviews.freebsd.org/D58597
e1000: report UDP RSS hash type on igb/em
{em,igb}_determine_rsstype() mapped only the TCP and bare-IP RSS descriptor
types; the UDP types returned M_HASHTYPE_NONE.
The hardware does hash UDP, but with a NONE hashtype iflib skips its
flowid-based TX queue spread, so all forwarded UDP egressed on a single queue
and serialized transmit on one core.
Add the three UDP cases (IPV4_UDP, IPV6_UDP, IPV6_UDP_EX) so egress spreads
across all TX queues.
Reviewed by: kbowling, gallatin
Approved by: kbowling
MFC after: 1 week
MFC to: stable/14, stable/15
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D58513
(cherry picked from commit 285c749f575ed7f9e60555037f23ac673084c62a)
hwpmc: fix event allocation on pre-Zen AMD CPUs
amd_allocate_pmc() chose the pmu-events code path whenever pmc_cpuid was
non-empty, and rejected any allocation lacking PMC_F_EV_PMU.
But pmc_cpuid is set for every AMD CPU, while the pmu-events tables only cover
Zen and later.
On older families (K8, Bobcat, Jaguar/16h, Bulldozer) libpmc finds no
pmu-events entry and falls back to the legacy path, which never sets
PMC_F_EV_PMU.
Reviewed by: mhorne
Approved by: mhorne
MFC after: 1 week
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D58468
(cherry picked from commit 6c4d9b9af1a3b247bf82a4228c835d106f535613)
e1000: report UDP RSS hash type on igb/em
{em,igb}_determine_rsstype() mapped only the TCP and bare-IP RSS descriptor
types; the UDP types returned M_HASHTYPE_NONE.
The hardware does hash UDP, but with a NONE hashtype iflib skips its
flowid-based TX queue spread, so all forwarded UDP egressed on a single queue
and serialized transmit on one core.
Add the three UDP cases (IPV4_UDP, IPV6_UDP, IPV6_UDP_EX) so egress spreads
across all TX queues.
Reviewed by: kbowling, gallatin
Approved by: kbowling
MFC after: 1 week
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D58513
(cherry picked from commit 285c749f575ed7f9e60555037f23ac673084c62a)
hwpmc: fix event allocation on pre-Zen AMD CPUs
amd_allocate_pmc() chose the pmu-events code path whenever pmc_cpuid was
non-empty, and rejected any allocation lacking PMC_F_EV_PMU.
But pmc_cpuid is set for every AMD CPU, while the pmu-events tables only cover
Zen and later.
On older families (K8, Bobcat, Jaguar/16h, Bulldozer) libpmc finds no
pmu-events entry and falls back to the legacy path, which never sets
PMC_F_EV_PMU.
Reviewed by: mhorne
Approved by: mhorne
MFC after: 1 week
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D58468
(cherry picked from commit 6c4d9b9af1a3b247bf82a4228c835d106f535613)
stand: consume UEFI PXE Base Code cache to seed DHCP
Populate stand/libsa/bootp.c's bootp_response global from the UEFI PXE
Base Code Protocol's cached DhcpAck, so bootp() can enter RFC 2131
INIT-REBOOT and skip DISCOVER/OFFER instead of running a fresh DHCP
transaction after the firmware has already done one.
manuals: Fix Fx and nearby mechanical typos
Fix compiler warnings related to the Fx macro, as well as all other
mechanical typos that were visible within one screenful of them. These
cause rendering glitches on various toolchains with various of the five
and a half decades of rich output formats and tooling manpages scale to.
The *x macro set specifies operating systems. These macros take the rest
of the line as an argument. Sometimes, a space was not used to separate
the argument of Fx and the trailing period. Another, FreeBSD Foundation
was misrepresented as an operating system version instead of an author.
Two more had other parts of the sentence supplied as an argument to Fx.
While I had those open, fix the other mechancial typos visible on those
specific screenfulls. Fix a list width glitch, correct section typo
AUTHOR to AUTHORS, and switch AUTHORS sections containing prose to
prose-mode so that they wrap freely when rendered.
PR: 297248
[5 lines not shown]
nfs_commonkrpc.c: Fix recovery that was broken by 4d80d4913e79
Commit 4d80d4913e79 added a check for nfsess_defunct already
being set. This was incorrect because, once set, nfsess_defunct
remains set and an additional recovery might be needed.
This patch reverts this part of 4d80d4913e79.
PR: 297252
Suggested by: olivier
MFC after: 3 days
Fixes: 4d80d4913e79 ("nfs: Fix argument typo to avoid a crash")
ng_ipfw: provide a tree for faster hook lookup
This should assist setups that connect a lot of nodes to ipfw: and then
distribute traffic with ipfw(4) tablearg feature.
Reviewed by: pouria
Differential Revision: https://reviews.freebsd.org/D58547
uexterr_gettext(3): support dynamic kernel categories
Add an uncached sysctl based implementation which retrieves individual
categories. A cache would be an obvious extension should this optional
feature that can only be enabled by an environmental varible have a
noticable performance impact in a case that matters.
Reviewed by: kib
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D58238
kdump(1): minimally adapt to exterror category sources
Add some minimal handling of category sources other than static kernel
sources. We don't actually look up dynamic sources yet (that would
require extended trace records to add the file names to the trace file
since we can't assume the trace file is running on a kernel with the
same numbers.)
Make the decision to append a "src/" prefix to each file name
dependent on the category source.
Reviewed by: kib
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D58412
exterror(9): dynamic kernel categories
Make it possible to define categories without compiling their
paths into libc (important for third-party modules). The
EXTERR_CATEGORY_DYNAMIC macro can be defined to a string describing the
compilation unit (generally the path relative to src/sys) which takes
the place of EXTERR_CATEGORY.
These strings are assembled in linker sets with category numbers
assigned at system startup or module load time. The strings can be
retrieved from the kern.exterr.categories.<category> sysctl.
Reviewed by: kib
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D58237
exterror(9): split category into multiple sources
The static, global index into an array of strings is simple, cheap,
and works for the base kernel, but is unworkable for (potentially
third-party) kernel modules or for arbitrary userspace code. Swipe a
few of the top bits of category to indicate a source with all-zeros
being the current model (EXTERR_CAT_SRC_KERN_STATIC). Add two
additional sources EXTERR_CAT_SRC_KERN_DYNAMIC and EXTERR_CAT_SRC_USER
with stub implementations.
Reviewed by: kib
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D58236
bin/sh: Fix history long line truncation/corruption
When reading from standard input with editline history enabled, increase
buffer size to accomodate long lines so that history is recorded
correctly. Cleanup el_gets() handling avoiding potentially dangerous
retention of pointers to editline buffers across calls. Ensure struct
parsefile objects are properly zero initialised when created. Remove
push argument from setinputstring() and simplify logic as it was always
called with a value of one and as was written was potentially dangerous
if ever called with a value of zero.
This commit does not fix long lines when history is enabled but editing
is not (e.g. if there is no terminal).
MFC after: 3 weeks
Pull Request: https://github.com/freebsd/freebsd-src/pull/2028
Signed-off-by: Kristofer Peterson <kris at tranception.com>
(cherry picked from commit 95e4fce8f0c4fc6bf828288b1d63faf0f1300198)
ip_mroute: Don't assume that a multicast router is running
The SIOCGETSGCNT handler may be invoked in this scenario, and if no
router has initialized the lookup table, we'll have
mfct->mfchashtbl == NULL.
PR: 297148
Reported by: Robert Morris
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
sys/ofed: don't stop removing stale RoCE GIDs at the first hole
When cleaning up stale GIDs the scan stopped as soon as
rdma_get_gid_attr() failed. But that can also happen for empty entries
in the middle of the table, so a single gap left everything after it
behind and the GID entries could eventually run out.
Now the whole table is scanned and the empty slots are simply skipped.
Reviewed by: kib, jhb
Sponsored by: Nvidia networking
Fixes: 6a75471dbcf0 ("OFED: Various changes from Linux 4.19")
Differential revision: https://reviews.freebsd.org/D58510
sys/ofed: fix GID table reference leak in roce_gid_update_addr_callback()
The "add missing GIDs" loop uses rdma_find_gid_by_port() to test whether
a GID already exists, but forgets to drop the reference it returns. So
every rescan that finds an existing GID leaks one, which pins the entry
and prevents its slot from ever being freed on delete.
Just release the reference once the GID is found, like the "remove stale
GIDs" loop already does.
Reported by: Wafa Hamzah <wafah at nvidia.com>
Reviewed by: kib, jhb
Sponsored by: Nvidia networking
Fixes: 6a75471dbcf0 ("OFED: Various changes from Linux 4.19")
Differential revision: https://reviews.freebsd.org/D58511
fts: refactor to use fd-relative operations internally
Replace all _open() calls with _openat() in __fts_open(), fts_read(),
and fts_children(). Replace statfs() with _fstatfs().
Add fts_dirfd to struct _ftsent, set to the file descriptor of the
parent directory. Callers can use openat(ent->fts_dirfd, ent->fts_name,
...) to access files safely without relying on fts_accpath, which
enables programs in capability mode to open the files described by
_ftsent.
This is a preparatory change for fts_openat() which will allow callers
to provide a pre-opened directory fd, enabling fts(3) traversal inside
Capsicum capability mode.
Mirror all fts_open() changes to fts_open_b().
As a result of expanding _ftsend, publish new ELF symbol versions for
fts_openat and related functions.
[4 lines not shown]
kqueue: stream the knote report instead of buffering all of it
kern_proc_kqueues_out() sized its intermediate sbuf from the preceding
sizing pass, so dumping core for a process with many knotes wired a
buffer as large as the entire report.
Shrank the intermediate to one page and added a drain that copied into
the caller's sbuf up to maxlen, stopping the walk once it was reached.
Truncation stayed byte exact.
A dump of 384k knotes peaked at 20 KB of M_SBUF instead of 445 MB.
Reviewed by: adrian, markj
Differential Revision: https://reviews.freebsd.org/D58584
MFC after: 1 week
kqueue: fix the always-empty NT_PROCSTAT_KQUEUES core note
sbuf reserves a byte of its buffer for the terminator, so the sbuf
created with maxlen held one byte less than the sizing pass had
computed. The last record overflowed it, sbuf_bcat() failed, and the
error == 0 guard skipped the copy into the caller's sbuf, so the note
has been emitted at full size but zero filled since 5e7c43ff02dc.
Fixes: 5e7c43ff02dc
Reviewed by: adrian, markj
Differential Revision: https://reviews.freebsd.org/D58583
MFC after: 1 week
tests/sys/pmc: only build if MK_PMC != no
This unbreaks the build when pmc support is explicitly disabled via the
aforementioned build knob.
MFC after: 10 days
Fixes: 2cfd82f74 ("hwpmc: add regression tests for ...")
Differential Revision: https://reviews.freebsd.org/D58401
(cherry picked from commit a18e773d2776a65c1a2a0418cbec9cf5ef526b53)
hwpmc: drain a process-mode PMC's runcount when a live target detaches
A process-mode PMC's runcount tracks how many CPUs currently have it
loaded in hardware. It is decremented only by the context-switch-out
and process-exit reclaim paths, both of which the scheduler invokes
only for processes flagged P_HWPMC. Detaching a target that still has
the PMC live in hardware dropped the target and cleared P_HWPMC without
taking the PMC off the hardware or dropping the runcount reference, so
the reference leaked. A subsequent release then spun in
pmc_wait_for_pmc_idle() forever waiting for the runcount to reach zero:
on an INVARIANTS kernel this panics ("waiting too long for pmc to be
free"), otherwise it is an unkillable loop holding the hwpmc lock. Any
process able to allocate a PMC can trigger this by attaching a counting
PMC to itself and detaching it before releasing.
Take the PMC off the hardware and drop the runcount reference as part
of detaching, before P_HWPMC is cleared: reclaim it from the detaching
thread's own CPU directly, and, when the detach removes the PMC's last
target, wait for any references held by the target's other threads to
[8 lines not shown]
hwpmc: add regression tests for detaching a live process-mode PMC
Attach a process-mode counting PMC to the current process, start it,
then detach and release it while it is still loaded on the hardware -
the case that previously leaked the PMC's runcount reference and
wedged pmc_wait_for_pmc_idle() at release. A second case does the same
from a multi-threaded process so the sibling threads' references have
to be drained too.
The tests need an allocatable process-mode counting event and skip
where none is available (hwpmc(4) not loaded, or a VM without a vPMU).
Reviewed by: adrian
MFC after: 2 weeks
Assisted-by: Claude Code (Fable 5)
Differential Revision: https://reviews.freebsd.org/D58343
(cherry picked from commit 3c3f886e4bc7619f7847ad0d0f996088ddf5915a)
hwpmc: handle counter wraparound for process-mode counting PMCs
The accumulated count of a process-mode counting PMC is kept in a
64-bit software counter and seeded into the hardware counter at every
context switch in. Hardware counters are narrower than that - each
PMC class discovers and records its own counter width, e.g. 48 bits
on current x86 (queried from CPUID on Intel, architectural on AMD) -
so once the accumulated count approaches the end of the hardware
counter range, the counter wraps during a time slice and the value
read back at switch out is smaller than the value seeded. The
increment was computed assuming a full 64-bit counter: on INVARIANTS
kernels a long enough counting run panics with "negative increment"
the moment the accumulated count first crosses the hardware counter
range, and on other kernels the totals silently lose a full counter
range per wrap.
Compute the increment modulo the per-class hardware counter width
instead, in both places that accumulate switch-out deltas.
[6 lines not shown]
hwpmc: add regression tests for counting-PMC counter wraparound
Exercise a process-mode counting PMC whose accumulated count crosses,
or already exceeds, the range of the underlying hardware counter.
Before the previous commit, the first context switch after the
hardware counter wrapped panicked INVARIANTS kernels with "negative
increment" and silently corrupted the accumulated count on other
kernels.
The tests need a hardware counting event backed by a counter narrower
than 64 bits and skip where none is available (hwpmc(4) not loaded,
or a VM without a vPMU).
Reviewed by: adrian
MFC after: 2 weeks
Assisted-by: Claude Code (Fable 5)
Differential Revision: https://reviews.freebsd.org/D58341
(cherry picked from commit 2cfd82f747c04f68f679824ba627460e87ab3848)
