ZTS: verify full mapped AIO transfers
mmap_libaio maps one page even when its I/O buffer spans more pages.
It also ignores the completion result, so failed or partial transfers
can incorrectly pass the test.
Map the full buffer and require each completion to transfer all bytes.
Assisted-by: OpenAI Codex
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Giuseppe Piscitelli <ooonea at gmail.com>
Closes #19754
zfs.4: fix the order of the fixed,N,M taskq values
The zio_taskq_read, zio_taskq_write and zio_taskq_free entries say
that fixed,N,M means M queues with N threads each, but
spa_taskq_param_set() reads the number of queues first and the number
of threads per queue second, and spa_taskq_param_get() prints them in
that order. The documented default of zio_taskq_read, fixed,1,8, is
one queue of eight threads (ZTI_N(8)), not eight queues of one thread.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Closes #19775
Set fletcher4 fastest uses_fpu from both byte orders
The "fastest" fletcher4 implementation is assembled from two
benchmark winners: the native functions come from the fastest native
implementation and the byteswap functions from the fastest byteswap
implementation. The two can differ. It still has a single uses_fpu
flag, which FLETCHER_4_FASTEST_FN_COPY() overwrites with each new
winner. The byteswap pass runs first, so the flag ends up as that of
the native winner.
When a SIMD implementation wins only the byteswap benchmark, as #14219
reports for the in-tree NEON code on a Yitian 710, byteswapped
fletcher4 then runs SIMD code without kfpu_begin() and kfpu_end().
That is reached by reading blocks written on a host of the other byte
order and by receiving such a send stream. On Linux this can corrupt
the FPU/SIMD register state of user tasks; FreeBSD requires
fpu_kern_enter() for kernel FPU use. This came in with 78289b8458,
which moved kfpu_begin()/kfpu_end() from each implementation's
init/fini, where the byte order was implied, into the callers.
[11 lines not shown]
libspl: remove the unused sys/asm_linkage.h
lib/libspl/include/sys/asm_linkage.h is a userspace copy of
include/sys/asm_linkage.h, but nothing includes it: config/Rules.am
puts include/ before lib/libspl/include, so <sys/asm_linkage.h> is
always include/sys/asm_linkage.h, which already covers userspace and
also defines ASMABI, which this copy lacks. Since 0b2819872 removed
the gate around its FRAME_BEGIN and FRAME_END but kept the #endif,
the copy does not even preprocess.
Remove it, and stop installing it as libspl/sys/asm_linkage.h.
include/sys/asm_linkage.h is installed with the libzfs headers, so
code built against the installed headers still finds a working one.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Rob Norris <rob.norris at truenas.com>
Signed-off-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Closes #19555
ZTS: fix send_raw_inherited_key's license boilerplate
The test added in f53b6403ea points at the old illumos.org CDDL URL,
so 'make checkstyle' fails in spdxcheck.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Matthew Ahrens <matt at mahrens.org>
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
Closes #19842
ZTS: test change-key key mismatch with a pool image
Since f53b6403ea, an incremental raw receive into a dataset that
inherits its key from a passphrase-keyed encryption root keeps the
local key. zfs_change-key_mismatch used that receive to make a dataset
whose key its encryption root can't unwrap, so the mismatch no longer
happens, 'zfs change-key' succeeds and the test fails every run.
Datasets damaged that way before that change still exist, and
change-key's error path for them (#17425) needs coverage. Add a small
pool image, made with a build from before that change, that already
holds such a dataset, and run the same checks against it.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Matthew Ahrens <matt at mahrens.org>
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
Closes #19842
freebsd: prefer sys/stdarg.h where available
While the type of va_list and implementation of va_*() psuedo functions
varies (sometimes greatly) by architecture, they will always be defined
by the compiler in a consistent way that does not require machine
dependent handling.
Sponsored by: DARPA, AFRL
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Ryan Moeller <ryan.moeller at klarasystems.com>
Signed-off-by: Brooks Davis <brooks at capabilitieslimited.co.uk>
Closes #19753
FreeBSD: Drop some no-op Solaris compat macros
These macros do nothing and are only used in FreeBSD platform code.
Drop them and the invocations.
Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Rob Norris <rob.norris at truenas.com>
Signed-off-by: Ryan Moeller <ryan.moeller at klarasystems.com>
Closes #19746
FreeBSD: Drop some Solaris compat vnode macros
These macros provide Solaris-compatible names but are only used in
FreeBSD platform code. Use the native FreeBSD names and drop the shims.
Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Rob Norris <rob.norris at truenas.com>
Signed-off-by: Ryan Moeller <ryan.moeller at klarasystems.com>
Closes #19746
icp: call armv7 SHA2 asm through C wrappers
sha256_armv7_impl and sha512_armv7_impl stored the assembly entry
points zfs_sha256_block_armv7() and zfs_sha512_block_armv7() directly
in their ops tables. Assembly functions carry no kCFI type hash, so on
an arm64 or arm kernel built with CONFIG_CFI every indirect call
through ops->transform traps:
CFI failure at SHA2Update+0xe0/0x250 [zfs]
(target: zfs_sha256_block_armv7+0x0/0xe40 [zfs]; expected type: ...)
On arm64 armv7 is typically the fastest SHA-512 implementation, as
NEON is unavailable to zfs, so this hits every sha512 checksum.
Call them from C wrappers, as the x64 implementation does. No
kfpu_begin() is needed, as this code does not use NEON.
Tested on Linux 7.3 arm64 with CONFIG_CFI=y and CONFIG_CFI_PERMISSIVE=y
in QEMU: before, loading zfs.ko logged 1064 CFI failures from the
checksum benchmark; after, none, and both armv7 implementations
[6 lines not shown]
zstdcheck: allow __cfi_ prefixed symbols
On kernels built with CONFIG_CFI, clang emits a global __cfi_<func>
symbol for each function's type-hash preamble, just as
CONFIG_CALL_PADDING emits __pfx_<func>. check-zstd-symbols already
accepts __pfx_zfs_*, but reports every __cfi_zfs_* symbol as an
unprefixed zstd symbol, so make checkstyle fails on such kernels.
Accept __cfi_zfs_* as well.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
Signed-off-by: Cytops <cyridge at proton.me>
Closes #19562
icp: call x86_64 AES asm through C wrappers
aes_x86_64_impl stored the assembly entry points aes_encrypt_amd64()
and aes_decrypt_amd64() directly in its ops table. Assembly functions
carry no kCFI type hash, so on a kernel built with CONFIG_CFI every
indirect call through ops->encrypt/decrypt traps:
CFI failure at aes_encrypt_block+0xf4/0x120 [zfs]
(target: aes_encrypt_amd64+0x0/0x9f0 [zfs]; expected type: ...)
The path is used when AES-NI is unavailable or icp_aes_impl=x86_64.
Call them from C wrappers, as the SHA2 and BLAKE3 code already does.
Tested on Linux 7.3 with CONFIG_CFI=y and CONFIG_CFI_PERMISSIVE=y in
a VM with icp_aes_impl=x86_64 and icp_gcm_impl=generic: before, ~47k
CFI failures during encrypted dataset I/O; after, none, and
create/write/scrub/export/import/load-key of AES-GCM and AES-CCM
datasets succeeds with no pool errors.
[4 lines not shown]
zfs-mount-generator: fix unit names of non-ASCII mountpoints
systemd_escape() writes each byte it escapes with "\\x%02x" from
(int)*cur. Where char is signed (such as how the x86 ABIs define it), a
byte from 0x80 on is negative, so sprintf() writes \xffffff and two
more digits, and the next byte's escape overwrites all but \xff. So
/mnt/josé gets the unit mnt-jos\xff\xff.mount instead of the correct
mnt-jos\xc3\xa9.mount, and systemd refuses it, since its name doesn't
match its Where=. Mountpoints that differ only in such bytes also get
the same name, so only one of them gets a unit. And the last byte's
sprintf() writes 7 bytes past the 4 it has, into the room for the
suffix.
Print the byte as a uint8_t with %hhx, as zfs diff does with %hho, so
that it always prints as two digits.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: R. Andrew Burza <andrew.ohana at gmail.com>
Closes #19540
CI: install userspace build dependencies for unit tests
The unit job uses VM preparation that upgrades installed packages,
installs kernel and ZTS dependencies, and removes packaged ZFS modules.
None of that machine preparation is needed for the userspace unit build.
Share the Debian userspace build packages between the unit job and the
full VM installer, leaving the latter's other dependencies and setup in
place. Deliver the shared installer alongside the VM script when it runs
before the repository is copied into the guest.
Use the native unit target in the documented minimum-build example.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Kamil Monicz <kamil at monicz.dev>
Closes #19456
Keep the local key where a raw receive brings a foreign one
A raw receive writes the DSL Crypto Key from the stream over the one on
disk but leaves the encryption root it is filed under alone. The key in
the stream is wrapped by the wrapping key of the sender's encryption
root, so this only holds together for as long as ours is the same one.
Where the receiving filesystem inherits from a root with a wrapping key
of its own, which is what a replica has once 'zfs change-key -i' hands
it to a locally keyed dataset, the wrapped master key on disk becomes
the sender's while the root it is filed under is still ours, and nothing
on the pool can unwrap it. The unwrapped key stays cached until it is
unloaded, so the receive and everything after it look fine; the next
'zfs load-key' loads the root happily and the filesystem then refuses to
mount with EACCES. Getting the data back takes the sending pool's
wrapping key and a hand built encryption hierarchy, which is why
reporters have ended up rebuilding their backups instead.
The keyformat, salt and iteration count travel with the key and describe
the wrapping key it was wrapped with, so where the receiving filesystem
[30 lines not shown]
CI: Let the labels jobs ride out GitHub server errors
One transient error from GitHub fails a whole job of the labels
workflow, and with it every PR the job had not labelled yet:
area gh: Server Error: Sorry, this diff is temporarily
unavailable due to heavy server load. (HTTP 500)
conflict HTTP 502: 502 Bad Gateway (https://api.github.com/graphql)
inactive invalid character '<' looking for beginning of value
The area job is the worst off: it runs once per push, for that PR
only, so a PR it fails on has no area labels until its next push.
The area job now lists the files of a PR with git rather than with
the pulls/N/files endpoint, which needs GitHub's diff service: one
partial fetch, without file contents, of the PR heads and their base
branches into a bare repository, then git diff --name-only from the
merge base, which lists what GitHub shows, renames included. Nothing
of a PR is checked out or run. The fetch takes about 2 s and 20 MB
[22 lines not shown]
ZTS: refreserv_004_pos fails: its space limits are off
The test sets a dataset's refreservation to the dataset's available
space and expects any larger value to fail. But a refreservation also
covers the space the dataset already uses, so it can grow to available +
used; setting available + 1 succeeds. The test is listed as a known
failure on every platform.
It also writes a file into the refreservation while the parent is at its
quota. That write can fail with EDQUOT however small it is: a
transaction's worst-case space estimate is far larger than the data, and
the part that exceeds the unused refreservation is charged to the
parent. The write only goes through while the parent is a few KiB short
of its quota, which for the pool's root dataset ends as soon as the MOS
grows.
Check that available + used + 1 MiB fails. The margin is needed because
the kernel checks against the on-disk available space, which differs
from the avail property by the writes in flight. Drop both writes; the
[9 lines not shown]
Linux: log size changes and punches before dropping the range lock
zfs_freesp() makes the change under a range lock in zfs_trunc(),
zfs_extend() or zfs_free_range(), drops the lock, and only then
assigns TX_TRUNCATE in a second transaction. zfs_setattr() calls it
with log=FALSE and logs the new size later still, in TX_SETATTR.
zpl_iter_write() takes no inode lock, so a write can run in between.
Its TX_WRITE is assigned first, and replay applies the write before
the size change or the punch: a write fsync'ed after ftruncate()
returned is cut off at the new size, and one written into a punched
hole is zeroed again.
Take the range lock in zfs_freesp() and hold it until TX_TRUNCATE is
assigned, with the helpers running under it. As before, the lock
covers the whole file when the size may change and only the range
otherwise; the choice between extending, truncating and freeing is
now made under it. The page cache is still truncated after the lock
is dropped.
[27 lines not shown]
spa: keep ZIL claims relative to the loaded txg
5f1957a15 numbers the new txgs after a load of an explicitly requested
txg above every uberblock still on disk, by raising spa_first_txg.
spa_min_claim_txg() returned spa_first_txg, so the threshold for "log
block not yet committed" moved with it. A writable 'zpool import -T'
doesn't clear the log (only the rewind retry path does), so zil_claim()
then treats log blocks and WR_INDIRECT data born after the loaded txg,
which are free in its space maps, as already committed and doesn't
claim them. Debug builds fail the assertion at the end of zil_claim();
release builds free those log blocks unclaimed when the log is
destroyed: a double free, or a free of space that has been reused since.
Record the claim threshold before raising spa_first_txg and return it
from spa_min_claim_txg(). It is fixed for the life of the import, as
dsl_scan and traverse also use it after the load. The threshold is
also stored as zh_claim_txg, which replay and the log's frees compare
births against, so it has to stay the txg after the loaded one.
[11 lines not shown]
ZTS: test a writable import -T of a pool with a live ZIL
Run a synchronous writer so the ZIL is never empty and freeze the pool
while it runs: the log keeps growing, but nothing after the freeze txg
reaches the disk. Stop the writer, export, and import the pool
read-write at the txg before the newest uberblock, so the log blocks
written while the newest txg was open are born after the imported txg.
The import must succeed, and after another export and import the pool
must scrub clean and pass zdb's leak check. (Freezing keeps the test
deterministic: a pool that syncs more txgs before the export can reuse
blocks freed after the imported txg, and the rewound pool then has
data errors.)
Since 5f1957a15 the new txgs after such an import are numbered above
the newest uberblock on disk, and spa_min_claim_txg() follows them, so
the log is claimed against the wrong txg.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
[2 lines not shown]
zio: don't write dedup copies from an override BP's data
An fsync'd write reaches syncing context with an override BP that
dmu_sync() already wrote, so zio_write_bp_init() skips the compress
and encrypt stages and the zio only holds the logical data. If the
block is deduplicated against an entry with fewer DVAs than copies=
needs, zio_ddt_write() wipes the override DVAs and issues a child write
for the missing copies, pushing that logical buffer under a BP whose
PSIZE and compression come from the override.
Debug builds then fail VERIFY3U(zio->io_size, ==, BP_GET_PSIZE(bp)) in
zio_dva_allocate() and hang txg_sync. Release builds write the extra
copy from the wrong buffer, so it fails its checksum from the start:
reads use the other copy, and a scrub reports and repairs it.
When an override write needs more DVAs, restart it as a regular dedup
write instead, as zio_write_bp_init() does when it can't use an
override. The data is compressed again and the missing copies are
written from it; dbuf_write_override_done() frees the override block.
[9 lines not shown]
ZTS: test dedup of an fsync'd block that needs more copies
An fsync'd write reaches syncing context as an override BP written by
dmu_sync(). When it is deduplicated against an entry that has fewer
DVAs than its copies= needs, zio_ddt_write() issues a write for the
extra copies. Add a test that sets this up with fast dedup, lz4 and
copies=1 then 2, and checks that both files read back intact, a scrub
finds no errors and zdb finds no leaks.
This fails on a debug build with
VERIFY3U(zio->io_size, ==, BP_GET_PSIZE(bp)) in zio_dva_allocate().
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Matthew Ahrens <matt at mahrens.org>
Reviewed-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Issue #19226
Closes #19303
zvol: leave alone a minor that is being removed
zvol_os_remove_minor() clears zv_zso and drops zv_state_lock while it
tears down the OS side of a ZVOL_REMOVING zvol. zvol_set_ro() and
zvol_set_volthreading() can find the zvol by name in that window and
zvol_set_ro() dereferences the NULL zv_zso. zvol_set_volsize() drops
all its locks before it updates the OS side, so the removal can start
under it.
Since d73686867 the thread that changes volmode waits for the minor to
be replaced, but another process setting readonly or volsize on the
same volume still hits the window: the kernel oopses in
zvol_os_set_disk_ro() or zvol_os_update_volsize(), the ioctl thread
dies with the zvol's lock held and every later minor operation hangs
in the zvol taskq.
Skip a ZVOL_REMOVING zvol when setting readonly, volthreading or
volsize, as zvol_suspend() does; the property is still stored, as when
there is no minor. Hold a suspend reference across the OS side of the
[12 lines not shown]
ZTS: race volume property changes with volmode changes
Setting readonly or volsize on a volume whose minor another process is
removing, for a volmode change, dereferences the minor's OS state after
it has been cleared. Run volmode, readonly, volthreading and volsize
changes on the same volume at the same time, and check that they all
finish and the volume still works afterwards, with the properties set.
A loop still running two minutes after it should have stopped is stuck
in the kernel: fail with the stuck zfs processes' stacks rather than
wait for the test timeout.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Closes #19319
libzfs_input_check: Fix integer to pointer casts
The zc_nvlist_foo fields are uint64_t and must be first cast to a
pointer-sized integer (uintptr_t) before being cast to a pointer to
avoid build-breakage for some targets (e.g. FreeBSD i386 w/GCC).
Fixes: 59dc88602
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Garth Snyder <garth at garthsnyder.com>
Reviewed-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Signed-off-by: Ryan Moeller <rmoeller at FreeBSD.org>
Closes #19793
libspl/kmem: Make prototypes safe for C++
restrict is a C99 keyword and is not available in C++, so including
this header from C++ fails to compile. FreeBSD's zfsd is one such
user, through sys/vdev_impl.h.
Use __restrict, which gcc and clang accept in both languages.
Fixes: 8b5d919d4 ("libspl: move kmem definitions from zfs_context.h")
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Garth Snyder <garth at garthsnyder.com>
Reviewed-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Signed-off-by: Ryan Moeller <rmoeller at FreeBSD.org>
Closes #19793
Fix uint64_t format strings on 32-bit targets
These print 64-bit values with %lu, %ld or %lx, which only matches
where long is 64 bits, and breaks builds with -Werror.
Use %llu and %lld with (u_longlong_t) and (longlong_t) casts, as is
done elsewhere.
Fixes: 13ec35ce3 ("Linux/vnops: implement STATX_DIOALIGN")
Fixes: d1b0a6982 ("draid: add failure domains support")
Fixes: 816804dfd ("Add support for anyraid vdevs")
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Garth Snyder <garth at garthsnyder.com>
Reviewed-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Signed-off-by: Ryan Moeller <rmoeller at FreeBSD.org>
Closes #19793
zstream: Use libspl atomics for memory accounting
These were the only bare __atomic calls in zstream, and Clang rejects
the 64-bit ones on i386 with -Watomic-alignment. The rest of zstream
already uses libspl.
Fixes: c878ee452 ("zstream: track and limit memory use")
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Garth Snyder <garth at garthsnyder.com>
Reviewed-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Signed-off-by: Ryan Moeller <rmoeller at FreeBSD.org>
Closes #19793
libspl/atomic: Add sequentially consistent ops
The libspl load and store are relaxed. Add the sequentially consistent
variants that zstream needs, so that it does not have to use bare
__atomic calls or add full barriers around relaxed operations.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Garth Snyder <garth at garthsnyder.com>
Reviewed-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Signed-off-by: Ryan Moeller <rmoeller at FreeBSD.org>
Closes #19793
