[PGO] Load GPU wave counts into IR metadata
GPU profiles contain wave counts alongside lane counts, but profile use
does not expose them to optimizations. Wave visits do not obey scalar
flow conservation, so unmeasured blocks cannot use counts reconstructed
from neighboring blocks or ordinary branch weights.
Map wave-counter indices to the blocks selected by PGO instrumentation,
after reproducing its critical-edge splits. For dense profiles, also map the
appended wave-only slots to eligible blocks in function order, after the
original block/select prefix. Select the layout from the profile flags.
Attach measured counts using wave.profile metadata, retaining measured
zeros and marking blocks without measurements unavailable. Require a
measured entry count for normalization and exclude select-counter slots
from the block mapping. Repeated instrumentation sites for the same block
represent one measurement and are not summed.
Validate the wave-counter layout against the accepted lane profile.
Clear old wave metadata when loading a replacement profile, including
[14 lines not shown]
amd64 FRED: normalize the user-provided 'fred' knob value
Assembler accesses 'fred' as byte, ensure that any non-zero
user-provided value works. This is pure cosmetics.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
amd64 resumectx(): restore FRED state before loading %cr4
%cr4 has the FRED enable bit. This only matters for NMI and #MC, and the
kernel is really not ready to handle them while in resumectx(), but do
it for formal correctness.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
amd64 FRED: no need to store fs/gsbase on user_fred_not_syscall path
The stores are only relevant for PCB_FULL_IRET return to userspace,
and setting the flag stores the bases into the pcb anyway. Also, the
hidden issue is that FRED already did implicit SWAPGS on entry, so
stored gsbase was wrong anyway.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
amd64 FRED: handle in-kernel faults same as IDT
Pass #MC and #DF directly to corresponding handlers, avoiding EFI
pcb_onfault case.
If a fault occurs while kdb/ddb is active, handler must call
kdb_reenter() and not do the normal processing, which typically ends up
in panic() for kernel-mode faults. But #MC and #DF should be excluded
there as well.
NMI is not an exception, it is delivered as specific event.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
[IR] Define GPU wave-profile metadata
Existing offload GPU profile counters measure lane executions, while
GPU instructions execute at wave granularity under an active-lane
mask. A block visited by every wave can therefore look cold when only
a few lanes are active. Scalar branch weights also cannot represent a
divergent wave visiting both successors before reconverging. These
profiles are a poor fit for optimizations that estimate work performed
by a wave.
Lane counters remain useful for measuring per-lane branch selectivity
and estimating work that scales with the number of active lanes.
Wave counters cannot replace them: a visit with one active lane and a
visit with all lanes active both count as one. The two profiles provide
complementary information about instruction execution and lane activity.
Introduce wave.profile and wave.profile.block metadata to represent
measured wave visits to IR blocks. Each dynamic visit with at least one
active lane contributes one to the count. A measured zero is distinct
[19 lines not shown]
[RISCV][P-ext] Add isel pattern for mul.w01/mul.w11 (#236323)
`mul.w01` multiplies the sign-extended lower word of rs1 by the
sign-extended upper word of rs2, and `mul.w11` multiplies the
sign-extended upper words of both. Both produce a 64-bit result and
are RV64 only.
Previously, (sra X, 32) was selected as a separate srai feeding `mul`.
Match it directly so the shift is folded.
rtwn: fix 8188f channel offset / width programming
* rtwn_chan2centieee() already calculates the proper centre
frequency, so it doesn't need to be further offset.
* Don't program R92C_RF_CHNLBW with further 20MHz wide bits;
the rest of the programming works correctly for 20/40MHz
operation.
Tested:
* Verified against the linux rtl8xxxu/8188f.c code.
* Tested w/ 20 and 40MHz 11n; 40MHz is now operating at higher
speed.
Fixes: 2c38dc04d6f8bc84620bb3b67c8ccd9f910c4d03
[Support] Add missing cast to uch (#236493)
In #174150, a call to `isdigit(PEEK())` was added, but `PEEK()` expands
to an expression of type `char` and `isdigit` requires its argument to
be `unsigned char`. Elsewhere in this file, this is done by just casting
the argument, `isalpha((uch)PEEK())`, so do the same here.
Revert "[asan] Adopt sanitizer_common operator-new framework (#196388)" (#236607)
#196388 made ASan's throwing operator new throw std::bad_alloc when
allocator_may_return_null=1. That exception does not reach the caller in
two configurations:
- arm64 Darwin linked with ld64.lld. asan_win.cpp compiles to an empty
object that is linked right before asan_new_delete.cpp.o, and ld64.lld
writes an encoding-0 (no unwind information) compact unwind entry at
operator new's address (#224949). The unwinder cannot step out of
operator new, so the std::bad_alloc it throws terminates the program
instead of reaching the caller's handler. The linker fix is #225055.
- Android. Both libclang_rt.asan-\*-android.so and libc++_shared.so link
static libc++abi.a, so the two libraries have separate new-handler state
and std::bad_alloc type info (#235557).
This also reverts #235557, which marked tests XFAIL on Android only
because of #196388.
[4 lines not shown]
Build the vdev config in place instead of copying it.
vdev_config_generate() built each child's nvlist on its own, copied it
into the parent and then freed it. The extended stats and the vdev
tree in spa_config_generate() were added the same way. In a config
with stats, the stats of a leaf under a raidz vdev were copied four
times.
Each nested nvlist is now added empty and filled where it lives. The
output is the same. vdev_config_generate() keeps its interface and
calls the new vdev_config_generate_impl().
On a pool with 1200 disks, building the config that
ZFS_IOC_POOL_STATS returns took about 287 ms before and about 80 ms
after. This was measured in userland with libzpool. In a perf profile
of that ioctl the copies were about 55% of the samples.
Signed-off-by: Caleb St. John <yocalebo at gmail.com>
Reviewed-by: Ameer Hamza <ameer.hamza at truenas.com>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Closes #19355
[lldb] Add libstdcpp atomic data formatters (#174218)
This patch adds a new synthetic child provider and a summary formatter
for the libstdc++ implementation of std::atomic<>.
ZTS: isolate the legacy mount option test dataset
Use a separate dataset for legacy mount option checks so properties left
by an earlier test do not affect the result. Destroy only that dataset
on exit, and skip its destruction if creation failed.
Match complete option names and compare against the paired inverse, so a
positive option cannot match the corresponding negative spelling.
Assisted-by: OpenAI Codex
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: ooonea <35407790+ooonea at users.noreply.github.com>
Co-authored-by: ooonea <ooonea at noreply.codeberg.org>
Closes #19586
[Offload] Remove unused GenericDevice in DeviceTy::launchKernel (NFC) (#236593)
The last use of GenericDevice in DeviceTy::launchKernel was removed on
October 28, 2026 in commit 8a29b68622037494f3e678cd2832b511fe2f3a9b.
Assisted-by: Antigravity
[PGO] Use the iterator-only IRBuilder constructor
Dense wave instrumentation uses a deprecated IRBuilder constructor that
takes both a block and an insertion iterator. This fails builds that
treat deprecation warnings as errors.
Use the iterator-only constructor. It obtains the same block from the
iterator and preserves the insertion point.
Release the upgrade hold before clearing os_upgrade_id
dmu_objset_upgrade_task_cb() clears os_upgrade_id under
os_upgrade_lock and only then, after dropping the lock, releases its
long hold on the dataset. dmu_objset_upgrade_stop() waits for the
task only if it finds os_upgrade_id set, so in that window it returns
at once, dmu_objset_disown() can drop the last hold, and the async
eviction finds the long hold still counted:
VERIFY3U(rc->rc_count, ==, number) failed (1 == 0)
PANIC at refcount.c:91:zfs_refcount_destroy_many()
dsl_dataset_evict_async+0x151/0x290 [zfs]
The task's own release then touches a dataset that may be freed.
dmu_objset_own() starts a userspace upgrade task on every writable
mount, so a pool destroyed right after it is created can hit this:
zpool_create_024_pos did once in CI.
Release the hold while os_upgrade_lock is still held, before clearing
[4 lines not shown]
tailscale: update 1.104.4
- Tailscale SSH performs Unix socket forwarding on behalf of the
authenticated user rather than as root, preventing authorized peers
from using local and remote forwarding to reach root-only Unix
sockets such as the Docker socket or the tailscaled local API. This
fix addresses a security vulnerability described in TS-2026-014.
- Tailscale SSH bounds the memory used for session handling and
releases it when the peer disconnects, preventing denial of service
through unbounded memory growth in tailscaled from continuous
window-change messages and environment variable appends. This fix
addresses a security vulnerability described in TS-2026-015.
- Tailscale Serve restricts TCP forwards to Unix socket backends and
revalidates backend paths on each connection, preventing access to
privileged Unix sockets such as the tailscaled local API socket.
This fix addresses a security vulnerability described in
TS-2026-016.
[6 lines not shown]
Makefile.inc1: Make sure mandoc bootstrap tool depends on libz
It links against both libopenbsd and libz, so reflect this in the
bootstrap tool's dependencies in case of races from highly-parallel
builds.
Reported by: jkh
MFC after: 1 week
Set the peer's ALUA group when LIO becomes ACTIVE
On takeover, set_alua_active() wrote only the local controller group.
The peer's group kept the value this node had given it while STANDBY,
ACTIVE/OPTIMIZED. Nothing else rewrote it until the peer came back,
reached lio_ha 'synced' and called refresh_alua_states() on this node.
Until then, REPORT TARGET PORT GROUPS advertised the down peer as the
optimized path. If that remote call failed, the wrong state stayed,
because the failure is only logged.
set_alua_active() now writes both groups. The local group becomes
OPTIMIZED, and the peer's group gets the state the ACTIVE node gives
for the current ha_state: NONOPTIMIZED when synced, UNAVAILABLE when
disconnected, otherwise TRANSITIONING.
refresh_alua_states() uses the same table, so the disconnected case now
writes UNAVAILABLE rather than TRANSITIONING. That matches the table in
utils/lio/config.py, so the reconciler and the event handlers no longer
disagree about the peer's state.
Add middleware support for LIO ALUA HA
Wire up the middleware side of LIO ALUA high-availability: load
lio_ha.ko with per-node addresses on service start, manage ALUA
state across failover events, clean up STANDBY configfs on pool
export, and add pre-flight validation that targets have static
initiator ACLs before ALUA can be enabled.
For each target, create a portal-less phantom TPG carrying the peer
node's controller group so that a single RTPG response from any
connected port lists both ALUA groups. Write tpgt_N/rtpi explicitly
before enable so that relative target port IDs in RTPG match the
tag formula (portal.tag on Node A, portal.tag + 32000 on Node B)
rather than being auto-assigned sequentially by the kernel.
ALUA group states are driven by role and ha_state:
MASTER + synced local=OPTIMIZED remote=NONOPTIMIZED
MASTER + connected local=OPTIMIZED remote=TRANSITIONING
[4 lines not shown]
