Attributor: Hash all bits of the abstract attribute map keys
The AA IDs are adjacent bytes and the positions of a value only differ
in the low bits, which the default hashes mostly ignore, so many AAMap
keys collide. Hash the full key instead.
attributor-light instruction counts (callgrind): -0.6% to -4.3%, full
attributor -0.5% to -1.5%. CTMark (-attributor-enable=light): O3
-0.01%, ThinLTO -0.03%.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[RISCV][P-ext] Make v2i32 insert_vector_elt with constant indices legal for RV32.
We can select to an insert_subreg.
The default lowering created a store+load to stack that got folded,
but left behind the stack slot allocation.
Assisted-by: Claude
[RISCV][P-ext] Lower 2 element insert_vector_elt to scalar_to_vec+ppaire/ppaireo.
This avoids needing to match the srl+build_vec later which we only
do on RV64.
[RISCV] Add SMT_ prefix to SpacemiT instruction enum names. (#241053)
Add "vtype" to end of names that become ambiguous. This represents the
additional operand the VMADOT II instructions have.
[BPF] Regenerate adjust-opt-icmp7.ll checks after #237690 (#241074)
#231257 added this test with checks generated before #237690 started
marking global functions noipa, so it fails on main.
cc @eaeltsin
Assisted-by: Claude Opus 5.5
mlx5: add per-flow-group hash table for FTE lookup
lookup_fte_locked() scanned every FTE in a flow group with xa_next()
and memcmp() on each one. A new RX TLS session always misses, so the
whole group was walked under the group lock for every insertion. With
tens of thousands of RX-offloaded sessions in the same IPv6 TCP group
this costs milliseconds per call and grows linearly with the number of
FTEs.
Keep a hash table of the FTEs in each flow group, keyed by a jhash of
the match value. The table is sized from the group size, capped at
16K buckets. It is updated in insert_fte() and del_sw_fte(), both of
which hold the group write lock, and read in lookup_fte_locked() under
the group read or write lock. The xarray is kept for index
allocation. No lockless reads are introduced.
Reported by: NetApp
Reviewed by: kib
Sponsored by: NVidia networking
MFC after: 1 week
zvol: leave alone a minor that is being removed
zvol_os_remove_minor() clears zv_zso and drops zv_state_lock while it
tears down the OS side of a ZVOL_REMOVING zvol. zvol_set_ro() and
zvol_set_volthreading() can find the zvol by name in that window and
zvol_set_ro() dereferences the NULL zv_zso. zvol_set_volsize() drops
all its locks before it updates the OS side, so the removal can start
under it.
Since d73686867 the thread that changes volmode waits for the minor to
be replaced, but another process setting readonly or volsize on the
same volume still hits the window: the kernel oopses in
zvol_os_set_disk_ro() or zvol_os_update_volsize(), the ioctl thread
dies with the zvol's lock held and every later minor operation hangs
in the zvol taskq.
Skip a ZVOL_REMOVING zvol when setting readonly, volthreading or
volsize, as zvol_suspend() does; the property is still stored, as when
there is no minor. Hold a suspend reference across the OS side of the
[12 lines not shown]
ZTS: race volume property changes with volmode changes
Setting readonly or volsize on a volume whose minor another process is
removing, for a volmode change, dereferences the minor's OS state after
it has been cleared. Run volmode, readonly, volthreading and volsize
changes on the same volume at the same time, and check that they all
finish and the volume still works afterwards, with the properties set.
A loop still running two minutes after it should have stopped is stuck
in the kernel: fail with the stuck zfs processes' stacks rather than
wait for the test timeout.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Closes #19319
x11/nvidia-kmod*, graphics/nvidia-drm-*-kmod*: Use sys/stdarg.h
machine/stdarg.h is deprecated and will be removed in due course.
This deorbits machine/stdarg.h and use sys/stdarg.h instead.
Not bumping PORTREVISION, as this is a build fix for future
removal of machine/stdarg.h and shouldn't cause changes
in contents of pkgs.
PR: 300622
Co-authored-by: Brooks Davis <brooks at FreeBSD.org>
Differential Revision: https://reviews.freebsd.org/D61415
libzfs_input_check: Fix integer to pointer casts
The zc_nvlist_foo fields are uint64_t and must be first cast to a
pointer-sized integer (uintptr_t) before being cast to a pointer to
avoid build-breakage for some targets (e.g. FreeBSD i386 w/GCC).
Fixes: 59dc88602
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Garth Snyder <garth at garthsnyder.com>
Reviewed-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Signed-off-by: Ryan Moeller <rmoeller at FreeBSD.org>
Closes #19793
libspl/kmem: Make prototypes safe for C++
restrict is a C99 keyword and is not available in C++, so including
this header from C++ fails to compile. FreeBSD's zfsd is one such
user, through sys/vdev_impl.h.
Use __restrict, which gcc and clang accept in both languages.
Fixes: 8b5d919d4 ("libspl: move kmem definitions from zfs_context.h")
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Garth Snyder <garth at garthsnyder.com>
Reviewed-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Signed-off-by: Ryan Moeller <rmoeller at FreeBSD.org>
Closes #19793
Fix uint64_t format strings on 32-bit targets
These print 64-bit values with %lu, %ld or %lx, which only matches
where long is 64 bits, and breaks builds with -Werror.
Use %llu and %lld with (u_longlong_t) and (longlong_t) casts, as is
done elsewhere.
Fixes: 13ec35ce3 ("Linux/vnops: implement STATX_DIOALIGN")
Fixes: d1b0a6982 ("draid: add failure domains support")
Fixes: 816804dfd ("Add support for anyraid vdevs")
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Garth Snyder <garth at garthsnyder.com>
Reviewed-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Signed-off-by: Ryan Moeller <rmoeller at FreeBSD.org>
Closes #19793
zstream: Use libspl atomics for memory accounting
These were the only bare __atomic calls in zstream, and Clang rejects
the 64-bit ones on i386 with -Watomic-alignment. The rest of zstream
already uses libspl.
Fixes: c878ee452 ("zstream: track and limit memory use")
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Garth Snyder <garth at garthsnyder.com>
Reviewed-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Signed-off-by: Ryan Moeller <rmoeller at FreeBSD.org>
Closes #19793
libspl/atomic: Add sequentially consistent ops
The libspl load and store are relaxed. Add the sequentially consistent
variants that zstream needs, so that it does not have to use bare
__atomic calls or add full barriers around relaxed operations.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Garth Snyder <garth at garthsnyder.com>
Reviewed-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Signed-off-by: Ryan Moeller <rmoeller at FreeBSD.org>
Closes #19793
libspl/atomic: Silence Clang -Watomic-alignment
Since the implementation moved into the header as inlines, the 64-bit
operations are compiled in every file that includes it, not just in
libspl where NO_ATOMIC_ALIGNMENT is applied. On 32-bit targets Clang
warns because the operations become library calls, as uint64_t may be
under-aligned (i386) or too wide to be lock-free (powerpc). This is
fatal with -Werror.
Those library calls are expected, so ignore the warning for the
inlines only.
Fixes: 32ba39f47
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Garth Snyder <garth at garthsnyder.com>
Reviewed-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Signed-off-by: Ryan Moeller <rmoeller at FreeBSD.org>
Closes #19793
[X86][APX] Test the relocation of an NF add
test_regclass_not_updated_by_regalloc_2 only produced an NDD/NF add
with a GOT operand because the allocator happened to give the add a
destination different from its source. With SlotIndexes compaction it
picks the same register, so the add is compressed to the legacy form
and the CODE_6 TODO lost its test.
Add a function whose add keeps its source live, so it stays NDD and
becomes NF under any allocation.
[X86] Enable SlotIndexes compaction by default
Compacting SlotIndexes before greedy allocation drops the entries of
erased instructions, so live range sizes and instruction distances
reflect the code being allocated rather than how much was erased
before it. AMDGPU has done this since #235806; do the same for X86.
Codegen changes only through allocation: no new crashes, verifier
errors or allocation failures in the lit suites, including
EXPENSIVE_CHECKS, and a stage-2 build of LLVM, clang and lld with
-verify-machineinstrs passes its own tests. On llvm-test-suite at -O3,
spills drop 0.3% and .text 0.06%, with compile time unchanged.
Most test updates are regenerated checks. misched-matmul now inserts
20 spills instead of 24. live-debug-vars-stale-slotindexes.mir gets
one more use of %27 so that accumulate_fit still reaches the stale
index path it was written for.
freebsd/kstat: clean up properly when kstat_create fails
On error, kstat_create() would free the sysctl context and the kstat
itself, but missed the lock and most importantly, any memory allocated
for ks_data. Sending those error cases through to kstat_delete() handles
all the cleanup necessary.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Matthew Ahrens <matt at mahrens.org>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19789
kstat: don't set ks_data for KSTAT_TYPE RAW
It's used for internal kstat stuff, nothing to do with us.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Matthew Ahrens <matt at mahrens.org>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19789
kstat: fix use of ks_data and KSTAT_FLAG_VIRTUAL
We've been holding this wrong in a few places. KSTAT_FLAG_VIRTUAL is for
when ks_data will outlive the kstat (ie continue to exist after
kstat_delete() returns). In a few places, we allocated and set up
ks_data before the kstat is published (allowed!), but free it before it
is unpublished in kstat_delete(), so if the kstat is read then, it would
access already-freed memory.
The kstat API already allows for this case. If we don't set
KSTAT_FLAG_VIRTUAL, kstat_create() will allocate ks_data for us, and
then free it at a safe time in kstat_delete() after the kstat has been
unpublished.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Matthew Ahrens <matt at mahrens.org>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19789
kstat: always NULL the kstat_t* after freeing it
Doing it in some places but not others is confusing.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Matthew Ahrens <matt at mahrens.org>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19789
L2ARC: Do not stop writing when l2arc_headroom is 0
zfs.4 documents l2arc_headroom=0 as removing the per-cycle headroom
limit, and l2arc_write_sublist() still treats it so. Since 3523b5f3f,
however, l2arc_write_buffers() also stops a pass once its consumed
headroom reaches the headroom, which with l2arc_headroom=0 is 0 from
the start: no sublist is scanned, and nothing is written to any cache
device. Skip that check when l2arc_headroom is 0, as the check it
replaced did.
Add l2arc_headroom_zero_pos, which reads data with l2arc_headroom=0
and checks that the cache device received buffers.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Closes #19628
Don't set the R92C_CMD_FLAG_EXT bit in H2C commands on non-RTL8192C-family
chips. The extended command flag only exists in the RTL8192C-family
H2C command ids.
This matches Linux rtlwifi/rtl8xxxu.
ok stsp@
[CIR][SYCL] Emit sycl_in_device field to the for cir.lowering_options (#238117)
Add sycl_is_device field to the `cir.lowering_options`. This is
populated during CIRGen time from `langOpts.SYCLIsDevice`.
