ZTS: sync checksum test data before fault injection
Ensure that the checksum configuration test reads persisted blocks.
mkfile does not sync its writes, and flushing the ARC with zinject -a
does not make dirty dbufs evictable. Cached reads can miss the injected
checksum errors needed to reach the configured threshold.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: ooonea <35407790+ooonea at users.noreply.github.com>
Assisted-by: OpenAI Codex
Closes #19545
ZTS: allow ctypes library discovery
Include the loader discovery tools used by Python ctypes in the
constrained Linux test path.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: ooonea <35407790+ooonea at users.noreply.github.com>
Assisted-by: OpenAI Codex
Closes #19576
Add tests for stale mmap pages after rollback and recv -F
A file that has been mapped with mmap() keeps its pages in the Linux
page cache. If the dataset is then rolled back, or receives an
incremental stream with "zfs receive -F", and the file keeps its
object number, generation and size but has different data, reads
and new mappings kept returning the pre-rollback data until the
cache was dropped or the dataset remounted (#10931). The on-disk
data was always correct.
Add mmap_cached_rollback and mmap_cached_receive to the mmap group.
Both fault the file in through a read-only mapping, change the
dataset underneath it, and check that read() and a fresh mapping
see the new contents. The mmap_stale_check helper does the mapping
and checksumming.
Both tests fail on Linux without the zfs_rezget() page cache fix
and pass with it. They also pass on FreeBSD.
[3 lines not shown]
Linux 7.3 compat: META
Update the META file to reflect compatibility with the 7.3
kernel.
Assisted-by: OpenAI Codex
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: ooonea <35407790+ooonea at users.noreply.github.com>
Closes #19553
libzutil: keep the latest label when two vdevs claim one id
When it builds a pool's config from the labels found by a scan,
get_configs() puts each top-level vdev at the id its label gives,
replacing whatever was there, and leaking it. A device can keep the
label of a top-level vdev that the pool no longer has, such as one
added after a checkpoint that the pool was then rewound to, while a
vdev added since has the same id. Whichever comes last wins, so the
stale vdev can replace the current one in the config. The import, or
"zdb -e", then fails with an I/O error when the pool's metadata has
blocks on that vdev. When it has none there, the import falls back
on the config stored in the pool, which works only while the paths
recorded there still lead to the devices.
For each id, keep the top-level vdev whose label has the latest txg,
and free the one it replaces.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Signed-off-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Closes #19496
abi: update for libspl kthread changes
Sponsored-by: TrueNAS
Reviewed-by: Kamil Monicz <kamil at monicz.dev>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes 19508
libspl: updated "owned" primitives to track curthread
kmutex_t and krwlock_t both have an owner field, tracking the owner.
These can now use a kthread_t* and the curthread pointer like everything
else.
Sponsored-by: TrueNAS
Reviewed-by: Kamil Monicz <kamil at monicz.dev>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes 19508
libspl/thread: rewrite for kthread semantics and to hide pthread
The libspl kthread implementation was previously a simple wrapper around
POSIX threads. This was subtly wrong in a handful of ways, mostly in
that a pthread does not have exactly the same lifetime as a kthread, and
pthread_t neither represents exactly the same thing as kthread_t*, nor
is it guaranteed to be the same size and shape.
This commit properly implements the kthread semantics and API on top of
pthreads, while largely hiding the pthread use inside as an
implementation detail.
Sponsored-by: TrueNAS
Reviewed-by: Kamil Monicz <kamil at monicz.dev>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes 19508
spa: extend HAVE_SPA_THREAD gate to all uses of spa_did
thread_join(spa->spa_did) was always compiled in, which was ok while
thread_join() is either a no-op or takes a type trivially convertible to
a uintptr_t. Next commit will change the latter for userspace, breaking
the build.
Since this is all part of the currently-unused "SPA thread" mechanism,
it should go behind the same compilation gate.
Sponsored-by: TrueNAS
Reviewed-by: Kamil Monicz <kamil at monicz.dev>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes 19508
raidz: don't lose the expansion wakeup when a resilver ends early
When a pass of a RAID-Z expansion fails, because a child cannot be
read or its DTL_MISSING is not empty, spa_raidz_expand_thread() waits
for the txgs of that pass to sync, and only then sets
vre_waiting_for_resilver. raidz_dtl_reassessed() resumes the
expansion only if that flag is already set. A DTL reassessment that
lands in between is lost, and the expansion stays paused with every
child healthy, until something else reassesses the DTLs.
This is what happens after "zpool online" of a child that was offline
during an expansion. The online wakes the thread, the next pass fails
again because the child's DTL_MISSING is not empty yet, and the
resilver of a short outage ends, and reassesses the DTLs, while the
thread is still in txg_wait_synced().
Count the reassessments in vre_dtl_gen, and do not set
vre_waiting_for_resilver if the count has moved since the pass
started: the zthr then runs the next pass at once, as a wakeup would
[13 lines not shown]
zarcsummary: handle reset output pipes
A reader closing the output early can raise ConnectionResetError rather
than BrokenPipeError. Handle both with the existing successful exit path
and preserve the handling of other exceptions.
Assisted-by: OpenAI Codex
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: ooonea <35407790+ooonea at users.noreply.github.com>
Closes #19547
Linux: fix zfs_sha256_available() on ARM CPUs that also have SHA-512
ID_AA64ISAR0_EL1.SHA2 is 0b0001 when SHA-256 is implemented and 0b0010
when SHA-256 and SHA-512 both are. zfs_sha256_available() tested bit 0
of the field, so on every CPU that also implements SHA-512, which
includes Apple silicon and recent Cortex and Neoverse cores, it reported
SHA-256 as absent. The sha256 armv8-ce implementation was then never
listed and the slower NEON variant was the fastest available, while
zfs_sha512_impl correctly offered armv8-ce on the same CPU.
Treat any nonzero field value as SHA-256 present, matching the kernel's
HWCAP_SHA2 logic. FreeBSD and userspace use HWCAP and were already
correct.
Reviewed-by: Tino Reichardt <milky-zfs at mcmilk.de>
Reviewed-by: Rob Norris <rob.norris at truenas.com>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Walter McKelvie <walt at mckelvie.org>
Closes #19536
ZTS: preserve failures across repeated test executions
The reporter keeps only the last result for each test pathname. A pass
from another runfile group or a later iteration can overwrite an
earlier failure and make zfs-tests.sh report success.
Keep each result record and apply the existing pathname-based known
and intermittent failure policies. Cover duplicate groups and repeated
iterations with logs emitted by the test runner.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Signed-off-by: Kamil Monicz <kamil at monicz.dev>
Closes #19438
ZTS: age MRU buffers before checking MFU promotion
A repeated read only promotes MRU buffers after ARC_MINTIME has elapsed.
Wait one second before the second read so the test establishes that
precondition without changing ARC behavior or its assertions.
Update the strategy to describe the operations the test performs.
Assisted-by: OpenAI Codex
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: ooonea <35407790+ooonea at users.noreply.github.com>
Closes #19544
unit/nvpair: verify complete copied results
Read the copied string from the destination list rather than the source.
Require both pair iterators to reach the end when comparing a duplicate.
The current loop accepts a copy truncated after its first pair.
Check that duplication preserves the list's uniqueness flags as well as
its entries. Distinct input names otherwise hide a changed list mode.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Kamil Monicz <kamil at monicz.dev>
Closes #19455
unit: exercise the selected nvlist and normalization modes
The byte and uint8 cases ignore their uniqueness parameters, and the
pair-lookup case always constructs an NV_UNIQUE_NAME list. Use the
selected fixtures and check ENOTSUP for pair lookup in the other modes.
Select NFKD in the compatibility-decomposition test, which currently
passes NFKC and repeats the composition mode instead.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Kamil Monicz <kamil at monicz.dev>
Closes #19455
unit: check fixture setup and output cleanup results
Check the fat-ZAP upgrade operations and memory-stream closes directly,
so their failures cannot be hidden by later assertions on the contents.
Reject a failure to disable core dumps in the btree death-test child
with a nonzero exit. An assertion there would produce the same SIGABRT
that the test expects from the btree operation.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Rob Norris <rob.norris at truenas.com>
Signed-off-by: Kamil Monicz <kamil at monicz.dev>
Closes #19467
libspl: keep native backtraces usable under Valgrind
Native unwinding inspects register snapshots and probes process memory.
Memcheck can report these operations while handling an assertion,
obscuring the failure that triggered the backtrace.
Disable Valgrind error reporting on the calling thread while generating
the native backtrace, then restore the previous reporting state. Keep
the native registers, symbolization and caller-selected descriptor,
including stderr redirected after Valgrind starts. Detect the optional
client-request headers during userspace configuration.
This scope also hides genuine errors encountered while unwinding.
Add destination tests for redirected stderr and custom descriptors.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Kamil Monicz <kamil at monicz.dev>
Closes #19459
build: generate zfs_gitrev.h before building the kernel modules
module/Makefile lists modules and gitrev as two prerequisites of all,
with nothing that orders them. Under make -j the kernel build can then
compile spa_history.c or spl-proc.c before zfs_gitrev.h exists. Seen
with make -j24 -l25:
spa_history.c:32:10: fatal error: 'zfs_gitrev.h' file not found
Make modules-Linux and modules-FreeBSD depend on gitrev instead.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alexander Moch <mail at alexmoch.com>
Closes #19535
test-runner: write the kmemleak report into the log
When kmemleak reports objects after a test, the test runner marks the
test as failed and writes the report to a kmemleak file in the test's
output directory, but not to the log. The CI's Test Summary and job
summary take a failed test's output from the log, so the report only
shows up in the downloaded archive.
Also write the report into the log, after the test's own output, with
each line prefixed "kmemleak:". The kmemleak file is unchanged.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alexander Moch <mail at alexmoch.com>
Closes #19472
ZTS: select failed tests from current runner logs
The rerun filters only recognize unprefixed, unlabelled result lines.
Current timestamps and platform identifiers prevent failures from
being selected, so automatic retries can report success without
rerunning a failed test.
Parse the emitted result headers and retain group identifiers when
matching failures. Match resolved directories first, with relative
runfile group paths allowing reruns after the suite moves. Absolute
groups and standalone tests keep exact path matching.
Let the runner parse retry records after the wrapper filters eligible
test names, retaining older log support.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Signed-off-by: Kamil Monicz <kamil at monicz.dev>
Closes #19437
zio_crypt_os: add Linux kernel crypto API backend
Implement the zio_crypt_os platform interface using the Linux kernel
crypto API. This provides hardware-accelerated encryption on platforms
where the ICP has none, most notably ARM.
The kernel exports these symbols GPL-only by default, so configure
selects this backend only when the kernel exports them with a license
compatible with the module's and the AEAD and hash cores (CRYPTO_AEAD2
and CRYPTO_HASH2) are enabled. A builtin build is not subject to export
licensing, but then needs those cores to be builtin as well. Otherwise
the ICP backend is used. Only the selected backend is compiled.
The backend looks up its algorithms by name at key load time, so it
declares a soft dependency on the modules providing them for initrd
builders to include.
Finally, since crypto accelerators may DMA write to keydata and
hmac_keydata, dsl_crypto_key_sync() will now put these on the heap
[6 lines not shown]
zstream: read BLKSSZGET into an int in zstream raw
The BLKSSZGET ioctl stores an int, but zstream raw passed a pointer to
the uint64_t volume.sectorsize. On little-endian systems this happens
to work because the upper half was already zero, but on big-endian
systems the sector size ends up in the upper 32 bits (512 << 32, or
2 TiB). free_blocks() then rounds the start of a freed range up to
2 TiB and writes zeros from the start of the range towards it. On any
smaller device that write fails with ENOSPC at the end of the device,
so zstream raw exits at the first FREE record with a nonzero offset,
after zeroing from that offset to the end of the device. Every zvol
stream has such a record before its data (the free past the end of the
volume), so zstream raw could not apply a zvol stream to a block
device on big-endian systems at all.
Read the value into an int and assign it.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Closes #19324
zpool: compare special and dedup vdevs only with normal vdevs
Since 1d9c32c16 a special or dedup vdev may be more redundant than the
normal vdevs, but get_replication() still compared each special or dedup
vdev with the previous special or dedup vdev. A pool with a 3-way
special mirror and a 2-way dedup mirror on 2-way mirror normal vdevs,
each of which is individually acceptable, was therefore reported as
inconsistent. check_replication() skips all checks for an inconsistent
pool, so a later non-redundant addition was accepted without -f:
zpool create tank mirror f1 f2 special mirror f3 f4 f5
zpool add tank dedup mirror f6 f7 # accepted
zpool add tank f8 # accepted without -f
At the same time, creating the same layout in one step was rejected:
zpool create tank mirror f1 f2 special mirror f3 f4 f5 \
dedup mirror f6 f7 # rejected
[14 lines not shown]
ZTS: test replication checks with mixed special and dedup
Add alloc_class_018_pos. It creates a 2-way mirror pool with a 3-way
special mirror and a 2-way dedup mirror, both in one step and with
'zpool add', and checks that adding a single disk to it is refused
without -f. It also checks that a less redundant dedup device is
rejected when added together with a more redundant special device.
This currently fails: creating the pool in one step is rejected as an
inconsistent replication level, while building it with 'zpool add'
succeeds and the pool is then considered inconsistent, so the single
disk is accepted without -f. The fix follows in the next commit.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Closes #19288
libspl/membar: silence GCC warnings with ThreadSanitizer
ThreadSanitizer (TSan) does not currently have any support for fences.
GCC will throw an error when trying to emit a fence with
-fsanitize=thread enabled. Silence that warning.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Kamil Monicz <kamil at monicz.dev>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19505
libspl/atomic: remove membar_enter/membar_exit
Not used anywhere, and not implemented for kernel SPLs.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Kamil Monicz <kamil at monicz.dev>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #19505
ZTS: pass only numbers to return
ksh93u+m 1.0 (533ab5f932, 2026-10-22) rejects a non-numeric argument to
return with "bad number" and exit status 128. The Alpine CI runners
build ksh from the head of that branch, so since then
zpool_prefetch_001_pos (return true) and zinject_counts (return [[ ...
]]) fail there on every run.
Older versions silently treated those arguments as 0, so
check_count_freq() in zinject_counts never failed: its equality test was
also malformed ([[ $match -eq 0 $inject ]]), and the object injection
case read /tank/file instead of the test pool's file, so it never
matched the rule. Use the [[ ]] result as the status, compare the
injection count with the match count, and read the right file.
zed_stop() returned "" on FreeBSD; return 0.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Closes #19530
dsl_deadlist_open(): clean up on error
Since c33a55b0c2, dsl_deadlist_open() can fail, but when
dmu_bonus_hold() fails it leaves dl_os set. dsl_deadlist_is_open()
then returns true, and the error paths of dsl_dataset_hold_obj() and
dsl_dir_hold_obj() call dsl_deadlist_close(), which does
dmu_buf_rele(NULL). For a livelist, dsl_dir_livelist_close() also
destroys dd_pending_allocs and dd_pending_frees, which were never
created. Importing a pool with a damaged deadlist object can thus
crash the kernel (#17809).
Make dsl_deadlist_open() set dl_os, dl_object and dl_lock only once
it has succeeded, also for an old-format deadlist whose bpobj_open()
fails, so that a failed open leaves the deadlist not open. Likewise
initialize bpo_lock only once bpobj_open() has succeeded.
Add a test that damages the MOS dnode block holding a snapshot's
deadlist object and holds that snapshot.
[5 lines not shown]
zoned_uid: preserve global-only zoning property checks (#19521)
The UID-based property policy returns before the global-zone check for
zoned. A namespace owner with delegated zoned permission can change it
through ZFS_IOC_SET_PROP, although the CLI rejects the request.
Check both zoning properties before choosing the UID or regular policy.
Use dsl_deleg_access() instead of repeating its dataset-hold sequence,
retaining the delegation-disabled rejection.
Grant zoned permission in the existing negative test so absence of the
grant does not hide the property restriction.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Kamil Monicz <kamil at monicz.dev>
Closes #19521
