powerpc: keep FP, VMX and VSX ownership changes atomic with preemption
trap() and set_mcontext() change the FP/VMX/VSX ownership state in
several steps with preemption enabled. A context switch in between
leaves the thread computing with another thread's register contents:
- enable_fpu()/enable_vec() set PCB_FPU/PCB_VEC before loading the
registers, so a switch mid-load saves a half-loaded unit over the PCB.
- set_mcontext() clears the frame's MSR bits and then the PCB flags;
a switch in between re-enables the unit, and the thread returns to
user space with it enabled but not owned, so it is neither saved nor
restored until the next signal.
Both paths run after every sigreturn(2), setcontext(2) and
swapcontext(3). A POWER9 test that keeps f14-f31 live across a signal,
with other threads using the FPU on the same CPU, saw 283 corrupted
round trips out of 882000; none after this change.
Hold a critical section around both, as amd64 and arm64 do.
[7 lines not shown]
powerpc: keep FP, VMX and VSX ownership changes atomic with preemption
trap() and set_mcontext() change the FP/VMX/VSX ownership state in
several steps with preemption enabled. A context switch in between
leaves the thread computing with another thread's register contents:
- enable_fpu()/enable_vec() set PCB_FPU/PCB_VEC before loading the
registers, so a switch mid-load saves a half-loaded unit over the PCB.
- set_mcontext() clears the frame's MSR bits and then the PCB flags;
a switch in between re-enables the unit, and the thread returns to
user space with it enabled but not owned, so it is neither saved nor
restored until the next signal.
Both paths run after every sigreturn(2), setcontext(2) and
swapcontext(3). A POWER9 test that keeps f14-f31 live across a signal,
with other threads using the FPU on the same CPU, saw 283 corrupted
round trips out of 882000; none after this change.
Hold a critical section around both, as amd64 and arm64 do.
[6 lines not shown]
pf: Change source limiter tables from a task
pf_source_used() puts a source in the table of its source limiter when
the source has more states than the high-water mark, and
pf_source_rele() takes it out again below the low-water mark. Both
changed the table in place: pf_source_used() from pf_create_state(),
with the rules lock held for reading, and pf_source_rele() from
pf_remove_state(), with no rules lock at all. A table may only be
changed with the rules lock held for writing, as packets look addresses
up in its radix tree at the same time. With INVARIANTS the first
insertion panics ("Lock pf rulesets not exclusively locked" in
pfr_route_kentry()); without, the tree can be corrupted.
Queue the changes instead, as pf_overload_task() does for overload
tables, and make them from a task with the rules lock held for writing.
An entry names the limiter by its id and remembers its table: the
limiter, or its table, may be gone or replaced by the time the task
runs, and then the entry is dropped. pfsr_intable changes only once
the change is queued, so a failed allocation is retried the next time
[9 lines not shown]
Fix build failure after 904efcc64398
On main, casting PHYS_TO_DMAP(pa) does not require casting to pt_entry_t
* because it returns void * after fe3e92e6868d ("PHYS_TO_DMAP: Return a
void * instead of a vm_offset_t"). That was not MFCed to stable/14 and
stable/15, thus cast the return value of PHYS_TO_DMAP(pa) to
pt_entry_t * to fix build failure.
Approved by: jhb (mentor)
Fixes: 904efcc64398 ("powerpc/radix: include page tables...")
Sponsored by: FreeBSD Foundation
Fix build failure after f02a5391295e
On main, casting PHYS_TO_DMAP(pa) does not require casting to pt_entry_t
* because it returns void * after fe3e92e6868d ("PHYS_TO_DMAP: Return a
void * instead of a vm_offset_t"). That was not MFCed to stable/14 and
stable/15, thus cast the return value of PHYS_TO_DMAP(pa) to
pt_entry_t * to fix build failure.
Approved by: jhb (mentor)
Fixes: f02a5391295e ("powerpc/radix: include page tables...")
Sponsored by: FreeBSD Foundation
aq: Use the common RSS key without options RSS
Fetch the common RSS key independently of options RSS instead of
generating a private key in non-RSS kernels. Keep the existing
software-RSS or round-robin indirection policy and hardware key
serialization unchanged.
Reviewed by: gallatin
MFC after: 2 weeks
Sponsored by: BBOX.io
Differential Revision: https://reviews.freebsd.org/D59793
(cherry picked from commit 772994d7748287be70fe850ec9e1117d4e7b1747)
aq: Report partial initialization failures to iflib
Stop initialization when hardware setup, ring initialization/start, or
datapath start fails. Run the existing best-effort stop/cache/reset
cleanup and report the failure through iflib_init_failed(). Do not keep
configuring later rings or publish the interface as running.
Reviewed by: nprice
MFC after: 2 weeks
Sponsored by: BBOX.io
Differential Revision: https://reviews.freebsd.org/D59853
(cherry picked from commit 050683bb8e136981e4a1f66e845ab05c135630f4)
aq: Invalidate the descriptor cache after stopping all rings
Atlantic controllers can retain receive descriptors and their data
addresses after their rings are disabled. Reusing or releasing those
mappings without invalidating the device cache has caused observed
IOMMU and SMMU faults in the referenced Linux reports (7a1bb49461b1,
ed4d81c4b3f2 and 7526183cfdbe).
Move global cache invalidation out of the per-ring stop routine. Disable
every ring first, toggle invalidation once, and wait for its completion
indication. Exclude Atlantic A0, as in the upstream workaround. Report
a completion timeout rather than silently discarding it.
Reviewed by: nprice
MFC after: 2 weeks
Sponsored by: BBOX.io
Differential Revision: https://reviews.freebsd.org/D59852
(cherry picked from commit 4993c1052f899b9a971815df5661ca241641ad8e)
mgb: Propagate DMA setup errors and use the requested channel
Check the receive DMA and frame transfer setup results that were
previously discarded. Return EINVAL for invalid ring/writeback addresses
rather than reporting success, and stop initialization through
iflib_init_failed() when a ring could not be configured.
Use the transmit queue count for transmit initialization and the supplied
channel index for DMA operations. The driver currently allocates one
queue in each direction, so those index/count corrections are latent.
MFC after: 2 weeks
Sponsored by: BBOX.io
(cherry picked from commit 3f6c1e731376ffaf8612e7e196119acabbf357e4)
mgb: Correct TX interrupt bit
The TX interrupt handler checked MGB_INTR_STS_RX(qidx) instead of
MGB_INTR_STS_TX(qidx) -- presumably a copy-paste issue.
Because the driver does not use TX interrupts so there was no actual
issue in practice.
Reviewed by: adrian
Sponsored by: The FreeBSD Foundation
Fixes: 8890ab7758b8 ("Introduce if_mgb driver for Microchip LAN743x PCIe NIC")
Differential Revision: https://reviews.freebsd.org/D60550
(cherry picked from commit 6c3f2b8be268de075d60387158f2889140851725)
pf: check AH header length.
pf_walk_header() advances the packet offset for each AH extension
header. It was not checked that this does not exceed the packet
length if no next header was processed. Fix it like in IPv6.
OK henning@ sashan@
MFC after: 1 week
Obtained from: OpenBSD, bluhm <bluhm at openbsd.org>, fcd4e5522e
Sponsored by: Rubicon Communications, LLC ("Netgate")
pf: check extension header length.
pf_walk_header6() advances the packet offset for each IPv6 extension
header. It was not checked that this does not exceed the packet
length if no next header was processed. When used with af-to rules,
this could trigger a panic: m_copydata: null mbuf.
from Acts1631; OK henning@ sashan@
MFC after: 1 week
Obtained from: OpenBSD, bluhm <bluhm at openbsd.org>, 6d56112588
Sponsored by: Rubicon Communications, LLC ("Netgate")
pf: do not free the caller's ruleset when attaching a table fails
pfr_attach_table() takes a struct pf_ruleset * from its caller and, on its
own error paths, calls pfr_destroy_ktable() on the table it just made. That
drops the table's reference on the ruleset and calls
pf_remove_if_empty_ruleset(), which may unlink and free the anchor the
ruleset is embedded in -- and then pfr_attach_table() returns NULL to a
caller still holding the pointer.
Reached from userland with DIOCADDRULE and a PF_ADDR_DYNIFTL source, after
DIOCSETLIMIT has lowered PF_LIMIT_TABLES to one item of headroom: the first
pfr_create_ktable() takes it and the root table's fails.
Hold a table reference on the ruleset for the duration instead, so the
ruleset outlives every failure path inside pfr_attach_table() and stays the
caller's to remove.
OK sashan@
[2 lines not shown]
pf: Drain the overload task before a vnet's rules are freed
pf_src_connlimit() queues an entry for pf_overload_task() when a
source exceeds max-src-conn or max-src-conn-rate and the rule has an
overload table. The entry holds a pointer to the rule, and the task
runs from taskqueue_swi with the vnet of the queue. Nothing drained
the task when the vnet went away: an entry queued just before could
be applied to a rule pf_unload_vnet_purge() had freed, or to a vnet
that was gone.
Drain the task at the start of pf_unload_vnet_purge(), before the
unlinked rules are freed. No packet can queue another by then, as the
hooks are already removed.
Reviewed by: kp
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D60529
pf: Send syncookies from the receiving thread
pf sends outbound packets by offloading them to a single per-vnet SWI handler
through the `V_pf_sendqueue` mbuf queue. A large DDoS attack may overwhelm
that per-vnet queue with syncookie packets and cause contention in the SWI
handler that negatively affects other pf operations.
Fix this by sending the initial syncookie challenge from the context
of the receiving thread. This avoids the syncookie-induced contention on
the `pf_intr` mbuf queue.
Sponsored by: Klara, Inc.
Sponsored by: Entersekt
MFC after: 3 weeks
Reviewed by: kp
Differential Revision: https://reviews.freebsd.org/D59068
(cherry picked from commit bf05236727cf367ee8e22ef47febebb319db9ddb)
termcap: add missing capabilities to the xterm-ghostty entry
The entry was imported from the termcap source that Ghostty generates
with infotocap during its build. To fit within 1023 bytes, the
conversion dropped a number of capabilities, most notably all color
support, so termcap consumers such as ls -G and ncurses applications
treated Ghostty as a monochrome terminal.
Split the entry into xterm-ghostty and a ghostty+common fragment, and
restore the following from Ghostty's terminfo definition:
- colors: Co, pa, AF, AB, op, oc, ut, AX
- keys: End, keypad Enter, F10-F12, back tab, mouse prefix
- attributes: italics (ZH/ZR), invisible (mk), acsc map (ac)
- misc: cb, ch, cv, SA/RA, r1, bs, NP, 5i
AF and AB use the plain 256-color form, like xterm-256color, because
termcap cannot express the conditional used by the terminfo entry.
[2 lines not shown]
powerpc/radix: include page tables in minidumps
The radix pmap did not implement the minidump pmap callbacks, so radix
minidumps were emitted with an empty pmap section. Such dumps do not
contain enough information for libkvm to translate kernel virtual
addresses.
Snapshot the 64KB radix root table in the pmap section, add lower-level
page-table pages to the sparse dump, and include pages backing non-DMAP
kernel mappings. Keep the bulk direct map out of the dump while
retaining the relocated kernel image.
PR: 298532
Reviewed by: jhb
Approved by: jhb (mentor)
MFC after: 2 weeks
Sponsored by: FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D59743
(cherry picked from commit a38a698704b0289db3260617116a455fecdd54f6)
arm: pad minidump page table
libkvm locates the sparse page array after the page-rounded PTE table
size, but the ARM minidump writer emitted only the unrounded size. When
the table size was not page-aligned, libkvm therefore read every dumped
physical page at the wrong offset.
The mismatch was introduced when libkvm began rounding ptesize. It has
affected ARM minidumps since ffdeef323449 ("libkvm: Improve physical
address lookup scaling."). It is exposed when the dumped KVA span is not
a multiple of 4 MiB.
Zero-pad the final PTE page and include the padding in the dump size.
Reviewed by: jhb
Approved by: jhb (mentor)
Fixes: ffdeef323449 ("libkvm: Improve physical address lookup scaling.")
MFC after: 2 weeks
Sponsored by: FreeBSD Foundation
[3 lines not shown]
libkvm: support powerpc64 radix minidumps
PowerPC64 radix minidumps use a 64KB root directory followed by three
levels of 4KB page tables. Add an MMU backend which walks those tables,
handles large-page leaves, and decodes their always-big-endian entries
on both powerpc64 and powerpc64le.
Reject old radix minidumps whose pmap section is empty with a specific
diagnostic. Add a synthetic powerpc64le dump test which reads a page
through both its kernel and direct-map addresses.
PR: 298532
Reviewed by: jhb
Approved by: jhb (mentor)
MFC after: 2 weeks
Sponsored by: FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D59743
(cherry picked from commit 592ede65bd48f16e5c46a016e92b732ed90094d2)
ci: Remove Cirrus CI files
Cirrus CI shutted down its service on June 1, 2026 and its CI dashboard
website (https://cirrus-ci.com) is now inaccessible. Since these files
are no longer used, remove them from the repository.
Reviewed by: brooks, emaste, lwhsu
Approved by: olce (mentor)
MFC after: 2 weeks
Sponsored by: FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D59858
(cherry picked from commit 8d31b7856f774554ffd5bb382bd2b3f6534817a0)
powerpc/radix: synchronize temporary kernel mappings
The radix kremove implementation cleared a kernel PTE without
invalidating its TLB entry. Reusing crashdumpmap could therefore keep
accessing an old physical page, causing minidumps to contain repeated
stale page contents.
Invalidate the removed kernel mapping and synchronize newly installed
kernel PTEs before they are accessed.
Reviewed by: jhibbits
Approved by: olce (mentor)
Fixes: 65bbba25d214 ("powerpc64: Implement Radix MMU for POWER9 CPUs")
MFC after: 2 weeks
Sponsored by: FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D59728
(cherry picked from commit 8970a9f2ef781ab4f7d3fa446f7c6ac5eda26cc0)
libkvm: route powerpc64 minidumps to minidump backend
The regular powerpc64 core probes only checked the kernel ELF and thus
also matched minidumps. In particular, the powerpc64le probe could
claim a minidump before the minidump backend and then reject it as an
invalid ELF core.
Exclude minidumps from both regular powerpc64 probes and add a
regression test that verifies a powerpc64le minidump reaches the
minidump parser.
Reviewed by: jhibbits
Approved by: olce (mentor)
Fixes: f4eb39ba6bc9 ("[PowerPC64LE] libkvm powerpc64le support.")
MFC after: 2 weeks
Sponsored by: FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D59719
(cherry picked from commit 51b1272fca667ad4c57d94ca24600bf68f29ebd2)
ixl: fix big-endian hardware interface handling
Correct the byte order at the driver interfaces that do not operate in
host order. Encode little-endian Admin Queue VSI and MAC/VLAN fields,
and decode firmware-provided queue handles, statistics indices, and
event parameters before using them.
Decode transmit descriptor writeback before examining it, and construct
VLAN descriptor fields in host order for the final descriptor
conversion. Store paged HMC descriptors in little-endian form and retain
the selected field bits when reading HMC contexts.
Keep MMIO values in host order for bus-space accessors and remove the
obsolete le16_to_cpu no-op macro. In particular, retain the atomic
bus_space_read_8() used for 64-bit registers.
On big-endian systems, the unconverted perfect-match filter flag is
presented to firmware as 0x0100 instead of 0x0001. Firmware rejects that
command with EINVAL, leaving receive traffic functional only in
[9 lines not shown]
powerpc/radix: include page tables in minidumps
The radix pmap did not implement the minidump pmap callbacks, so radix
minidumps were emitted with an empty pmap section. Such dumps do not
contain enough information for libkvm to translate kernel virtual
addresses.
Snapshot the 64KB radix root table in the pmap section, add lower-level
page-table pages to the sparse dump, and include pages backing non-DMAP
kernel mappings. Keep the bulk direct map out of the dump while
retaining the relocated kernel image.
PR: 298532
Reviewed by: jhb
Approved by: jhb (mentor)
MFC after: 2 weeks
Sponsored by: FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D59743
(cherry picked from commit a38a698704b0289db3260617116a455fecdd54f6)
arm: pad minidump page table
libkvm locates the sparse page array after the page-rounded PTE table
size, but the ARM minidump writer emitted only the unrounded size. When
the table size was not page-aligned, libkvm therefore read every dumped
physical page at the wrong offset.
The mismatch was introduced when libkvm began rounding ptesize. It has
affected ARM minidumps since ffdeef323449 ("libkvm: Improve physical
address lookup scaling."). It is exposed when the dumped KVA span is not
a multiple of 4 MiB.
Zero-pad the final PTE page and include the padding in the dump size.
Reviewed by: jhb
Approved by: jhb (mentor)
Fixes: ffdeef323449 ("libkvm: Improve physical address lookup scaling.")
MFC after: 2 weeks
Sponsored by: FreeBSD Foundation
[3 lines not shown]
libkvm: support powerpc64 radix minidumps
PowerPC64 radix minidumps use a 64KB root directory followed by three
levels of 4KB page tables. Add an MMU backend which walks those tables,
handles large-page leaves, and decodes their always-big-endian entries
on both powerpc64 and powerpc64le.
Reject old radix minidumps whose pmap section is empty with a specific
diagnostic. Add a synthetic powerpc64le dump test which reads a page
through both its kernel and direct-map addresses.
PR: 298532
Reviewed by: jhb
Approved by: jhb (mentor)
MFC after: 2 weeks
Sponsored by: FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D59743
(cherry picked from commit 592ede65bd48f16e5c46a016e92b732ed90094d2)
