L2ARC: bound the rebuild by the write hand on a first sweep
l2arc_log_blkptr_valid() ends with (!evicted || dev->l2ad_first), which
disables the eviction-overlap test entirely on a first sweep. That test
is meaningless in that state, since l2arc_evict() returns immediately
and l2ad_evict never advances off l2ad_start, but dropping it leaves the
log block bounded only by device geometry. On a first sweep only the
region below the write hand has been written, so a block at or beyond
l2ad_hand describes data this incarnation never wrote.
l2arc_hdr_restore() performs no per-entry validation, so every such
entry inflates arcstat_l2_psize and, via vdev_space_update(), the cache
vdev's vs_alloc. Nothing reconciles it because l2arc_evict() never runs
on a first sweep, and once vs_alloc exceeds vs_space the unclamped
subtraction in zpool(8) wraps and the device reports 16.0E free. Stale
entries also let the L2ARC read offsets that were never written, which
then fail checksum verification.
Removing and re-adding a cache vdev is enough to set this up:
l2arc_add_vdev() resets l2ad_hand to l2ad_start while the previous
[21 lines not shown]
ZTS: ensure zfs_channel_program_support test is installed
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18836
zfs recv: warn when a non-raw incremental diverges a raw IV set
A non-raw incremental received onto a snapshot that was itself received
raw re-stamps the destination's IV set guid with a fresh value, so it no
longer matches the sending lineage. Nothing rejects this at the time,
but a later raw incremental taken from that snapshot is rejected with an
IV set guid mismatch (#8758). The failure surfaces arbitrarily later,
often on a different host, with no hint that an earlier non-raw receive
was the cause.
The receive itself cannot tell that a snapshot was received raw from
anything on disk: a raw-received snapshot and an ordinary locally-keyed
one both carry an IV set guid of the same shape, one stamped from the
stream and one from unique_create(), with no recorded provenance. So
warning on every non-raw incremental into an encrypted dataset would
fire on the common keys-local backup workflow that never hits the bug.
Record the provenance instead. A raw receive now stamps a
DS_FIELD_RAW_RECEIVED marker on the new snapshot, next to the ivset
[23 lines not shown]
libzfs: clarify the raw incremental "IV set guid mismatch" error
When a raw incremental receive is rejected because the IV set guid of
the incremental source snapshot does not match the send stream, the
error only said "IV set guid mismatch" and pointed at the man page.
That leaves the user with no idea what actually happened or how to
recover, which is a recurring complaint in #8758: the state that
triggers the error is created silently, most commonly by a single
non-raw incremental receive in an otherwise raw chain (e.g. a
replication tool falling back or omitting -w once). Such a receive
copies the dataset guid from the stream but re-encrypts the data with
a freshly generated IV set, so the destination snapshot looks
identical to guid-matching tools while any later raw incremental based
on it must be rejected to avoid mixing two incompatible ciphertext
lineages.
Name the non-raw-receive cause and the recovery (roll the destination
back to the most recent raw-received snapshot and take the raw
incremental from there, or receive a new full raw stream) directly in
[14 lines not shown]
ZTS: assert the IV set guid divergence behind mixed raw/non-raw sends
send_mixed_raw already exercises the failing sequence from #8758: a
raw full receive, then a non-raw incremental receive, then a raw
incremental receive that is rejected with "IV set guid mismatch".
However it only asserted the final failure, not the mechanism, so a
regression in how receives stamp the IV set guid could go unnoticed
as long as something still failed.
Extend the test to pin down the actual state transitions:
- A raw receive must preserve both the dataset guid and the IV set
guid of the received snapshot.
- A non-raw incremental receive copies the dataset guid from the
stream but generates a fresh IV set guid on the destination. This
is the silent divergence that later raw incrementals trip over:
the snapshots look identical to guid-matching tools while their
IV sets no longer match.
[14 lines not shown]
man: conditionally install pam_zfs_key.8
When built with --enable-pam include the pam_zfs_key.8 man
page, otherwise omit it. This aligns man/Makefile.am with
the build logic in contrib/Makefile.am.
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #18815
Closes #18817
zstream: add "drop records" chain module
This patch adds a module to remove records from a stream according to a
user-specified record-type mask. For example:
```
zstream_chain_t drop_writes {
...
serial_drop_record_types(DROP_WRITE_EMBEDDED | DROP_WRITE),
...
}
```
This is a trivial module, but there are a couple of reasons to use it
instead of just ignoring records you don't care about:
- It makes filtering a declarable API contract. Past the filter point,
you do not need to double-check record types beyond those you're
actually interested in.
[21 lines not shown]
tests: add mmap_read_truncate regression test
Exercise the mmap read-past-EOF race fixed by 223b8bc446 (#18715): a
page faulted in for read after the file is truncated below it reaches
zfs_fillpage() with io_off >= i_size. Before the fix the unsigned
io_len = i_size - io_off underflowed and dmu_read() zero-filled far past
the page, corrupting memory. Reader processes mmap and fault every page
of a file while a truncator churns its size; reads beyond EOF raise a
tolerated SIGBUS. A fixed module completes the run; an unfixed debug
module panics on the VERIFY3U(io_off, <, i_size) the fix removed.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Michael Heller <michael.heller at gmail.com>
Closes #18824
Harden recv record validation
Refactors and enhances how we do recv record validation. I've
made sure to keep all existing checks and added a few new ones.
The per-record type validator functions are also used by zstream
so this is now common code. Additionally, more detailed error
messages have been plumbed through to user space.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Garth Snyder <garth at garthsnyder.com>
Signed-off-by: Alek Pinchuk <Alek.Pinchuk at connectwise.com>
Closes #18725
ZTS: make zpool_iostat_interval_all teardown deterministic
zpool_iostat_interval_all runs "zpool iostat" in the background at a
0.1s interval and compares its output, parsed into a sequence of
chunks, against a fixed expected sequence as pools are created,
imported, exported and destroyed. Every step changes the visible pool
list by exactly one pool except the teardown, which used a single
"zpool export -a".
export -a exports the pools one after another rather than atomically,
so there is a brief window in which one pool is already gone and the
other is not. At the 0.1s sampling interval iostat occasionally catches
that intermediate single-pool state and emits an extra chunk that is
not in the expected sequence, and the test fails. Whether a sample
lands in the window depends on timing, which is why it shows up as a
flake.
Export the two pools explicitly, one at a time, and add the
intermediate single-pool state to the expected output, the same way
[9 lines not shown]
Add configure option to disable zfs channel programs
Introduce a configure time option to build OpenZFS without
channel program support. When disabled, the channel program ioctl
is not registered and attempts to invoke a channel program return
CMD_UNAVAILABLE.
This also removes the embedded channel program used for snapshot
deletion and updates the related build files, tests, and manpage
install rules so they reflect whether channel program support is
enabled.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Eduardo Alvarado <edalv.4Q7GW1IdW1+0 at gmail.com>
Closes #18658
libzfs: fallback VDEV_UPATH to VDEV_PATH for non-DM devices
When zfs_get_underlying_path() returns NULL for a non-DM device (e.g.
NVMe), the zfs_prepare_disk script was getting an empty VDEV_UPATH. Per
the man page, VDEV_UPATH should fall back to VDEV_PATH when there is no
underlying path.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: MISAPOR LAB <misapor at lab.misapor.pl>
Closes #18439
Closes #18802
zpool_influxdb: export per-vdev slow_ios counter
OpenZFS 2.4 added slow vdev sit-out detection, visible in 'zpool status
-s' as the SLOW column. Export the same vs_slow_ios counter in
zpool_influxdb output so monitoring stacks can alert on slow vdevs.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: MISAPOR LAB <misapor at lab.misapor.pl>
Closes #18804
Closes #18555
ABD: Validate borrowed buffer length
abd_borrow_buf() currently asserts that an unsigned size is at least
zero. This is tautological and allows a caller to borrow more bytes
than the ABD represents.
Assert that the ABD covers the requested length in the Linux, FreeBSD,
and libzpool implementations.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Kamil Monicz <kamil at monicz.dev>
Closes #18811
Fix reads for blocks freed after being cloned
PR #18421 fixed a case when reads for blocks cloned after being
freed could return zeroes. But it created an opposite problem,
when reads for blocks freed after being cloned could return non-
zero content from the cloning.
This patch fixes the problem by creating a more specialized
form of dnode_block_freed(), taking into account the TXG when
the cloning has happened and checking frees only in TXGs after.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Gary Guo <gary at kernel.org>
Signed-off-by: Alexander Motin <alexander.motin at TrueNAS.com>
Closes #18421
Closes #18724
Fix dmu_zfetch_prime() assuming a stream was created
dmu_zfetch_prime() calls dmu_zfetch_stream_create() and then takes
list_head(&zf->zf_stream), asserting its zs_blkid equals the requested
blkid. But dmu_zfetch_stream_create() only inserts a new head stream
in its normal path; when zf_numstreams has reached zfetch_max_streams
and no stream is reclaimable it bumps zfetchstat_max_streams and
returns without creating one, leaving an unrelated stream at the head.
dmu_zfetch_prime() then trips VERIFY3U(zs->zs_blkid, ==, blkid) (a
debug build panics; otherwise it primes the wrong stream).
Return the created/reused stream from dmu_zfetch_stream_create(), or
NULL when it declines, and have dmu_zfetch_prime() stop priming in
that case instead of trusting list_head. dmu_zfetch_prepare(), the
only other caller, does not use the returned stream.
This is reachable by any dmu_prefetch_stream() caller that drives more
than zfetch_max_streams concurrent streams on a dnode.
[3 lines not shown]
libzfs: don't abort receiving a raw encrypted send that carries holds
A raw send that gathers holds but not properties (zfs send -w -h,
without -p or -R) produces a compound stream whose per-filesystem
nvlist has no "props" entry. recv_fix_encryption_hierarchy() looked
that entry up with fnvlist_lookup_nvlist(), which aborts the process
when the key is missing, so receiving such a stream of an encrypted
dataset crashed with a failed VERIFY0() after the data had already
been received. Release builds abort the same way, since the VERIFY
family is always compiled in.
Look "props" up tolerantly and skip the keylocation fixup when it is
absent: a holds-only raw send sends no properties, and the raw
receive has already established a usable keylocation. The
FORCE_NEW_KEY and FORCE_INHERIT fixups, which do not need
properties, still run.
A new rsend test sends an encrypted dataset with -w -h, both a full
and an incremental stream, and confirms the receive succeeds with
[6 lines not shown]
Rate limit Direct I/O verify zevents
Each vdev initializes a vdev_dio_verify_rl rate limiter (governed by
zfs_dio_write_verify_events_per_second), but
zio_dio_chksum_verify_error_report() never consults it, so
dio_verify_rd and dio_verify_wr zevents are posted with no rate
limiting. A workload that repeatedly trips the Direct I/O verify can
therefore produce an unbounded flood of zevents.
Gate both ereport posts through zfs_ratelimit(&vd->vdev_dio_verify_rl),
as is already done for the other per-vdev ereports (checksum, delay,
deadman). The vs_dio_verify_errors vdev stat still increments on every
event, so the true count remains observable via zpool status -d.
The dio_write_verify test checks on every iteration that a
dio_verify_wr zevent was posted. With rate limiting now in effect the
shared limiter window is exhausted after the first iteration, so later
iterations observe zero events and the test fails. Raise
zfs_dio_write_verify_events_per_second for the duration of that test
[5 lines not shown]
Fix deduce_nblkptr for large dnodes on receive
In Pre-SA filesystems (version < 5) deduce_nblkptr() derived nblkptr
from DN_OLD_MAX_BONUSLEN, the bonus capacity of a single-slot dnode.
The issue is receive_handle_existing_object() uses the value returned
by deduce_nblkptr() to decide whether the object was reallocated.
With the existing formula, multi-slot dnodes get an underestimated
nblkptr so a non-raw incremental receive could free still-needed
contents. Resolve this by passing dn_slots into deduce_nblkptr() and
use the same bonus-capacity formula as dnode_(re)allocate().
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Alek Pinchuk <Alek.Pinchuk at connectwise.com>
Closes #18773
man: thorough scrub documentation update
Drop the paragraph in the -t option description which details the
resume behavior. This is already covered under the -p option and
there's no need to belabor the point.
Reviewed-by: Alek Pinchuk <Alek.Pinchuk at connectwise.com>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #18797
ZTS/block_cloning: test `COPY_FILE_RANGE_CLONE` on FreeBSD 15+
FreeBSD 15 gained a "clone only, never copy" flag for copy_file_range().
This implements it as an option in clonefile, so we can get the test
suite checking it.
Copying the existing FICLONERANGE tests, and changing the clonefile
call to use the copy_file_range(COPY_FILE_RANGE_CLONE) mode available
to ensure correct behaviour on FreeBSD 15.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Christos Longros <chris.longros at gmail.com>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18788
zfs_sync: remove support for impossible scenarios
The superblock pointer will always be set, as will z_log, so remove code
supporting cases that can't occur (on Linux at least).
Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Paul Dagnelie <paul.dagnelie at klarasystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Rob Norris <rob.norris at klarasystems.com>
Closes #17420
(cherry picked from commit 52352dd74869d64070e784573d4a01bb55924017)
Signed-off-by: Andriy Tkachuk <atkachuk at wasabi.com>
zfs_sync: return error when pool suspends
If the pool is suspended, we'll just block in zil_commit(). If the
system is shutting down, blocking wouldn't help anyone. So, we should
keep this test for now, but at least return an error for anyone who is
actually interested.
Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Paul Dagnelie <paul.dagnelie at klarasystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Rob Norris <rob.norris at klarasystems.com>
Closes #17420
(cherry picked from commit e3f5e317e0ed4da44a10fe202ec80abed1c8cc87)
Signed-off-by: Andriy Tkachuk <atkachuk at wasabi.com>
zts: test syncfs() behaviour when pool suspends
Fairly coarse, but if it returns while the pool suspends, it must be
with an error.
Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Paul Dagnelie <paul.dagnelie at klarasystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Rob Norris <rob.norris at klarasystems.com>
Closes #17420
(cherry picked from commit c1b7bc52fe264bf8b96dd667ef55db3bd478c00a)
Signed-off-by: Andriy Tkachuk <atkachuk at wasabi.com>
zpl_sync_fs: work around kernels that ignore sync_fs errors
If the kernel will honour our error returns, use them. If not, fool it
by setting a writeback error on the superblock, if available.
Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Paul Dagnelie <paul.dagnelie at klarasystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Rob Norris <rob.norris at klarasystems.com>
Closes #17420
(cherry picked from commit d1c88cbd4cc2ebe5b97fa5b39e7fd83294583ed8)
Signed-off-by: Andriy Tkachuk <atkachuk at wasabi.com>
zts: test syncfs() behaviour when pool suspends
Fairly coarse, but if it returns while the pool suspends, it must be
with an error.
Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Paul Dagnelie <paul.dagnelie at klarasystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Rob Norris <rob.norris at klarasystems.com>
Closes #17420
(cherry picked from commit c1b7bc52fe264bf8b96dd667ef55db3bd478c00a)
Signed-off-by: Andriy Tkachuk <atkachuk at wasabi.com>
zfs_sync: remove support for impossible scenarios
The superblock pointer will always be set, as will z_log, so remove code
supporting cases that can't occur (on Linux at least).
Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Paul Dagnelie <paul.dagnelie at klarasystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Rob Norris <rob.norris at klarasystems.com>
Closes #17420
(cherry picked from commit 52352dd74869d64070e784573d4a01bb55924017)
Signed-off-by: Andriy Tkachuk <atkachuk at wasabi.com>
zpl_sync_fs: work around kernels that ignore sync_fs errors
If the kernel will honour our error returns, use them. If not, fool it
by setting a writeback error on the superblock, if available.
Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Paul Dagnelie <paul.dagnelie at klarasystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Rob Norris <rob.norris at klarasystems.com>
Closes #17420
(cherry picked from commit d1c88cbd4cc2ebe5b97fa5b39e7fd83294583ed8)
Signed-off-by: Andriy Tkachuk <atkachuk at wasabi.com>
zfs_sync: return error when pool suspends
If the pool is suspended, we'll just block in zil_commit(). If the
system is shutting down, blocking wouldn't help anyone. So, we should
keep this test for now, but at least return an error for anyone who is
actually interested.
Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Paul Dagnelie <paul.dagnelie at klarasystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <mav at FreeBSD.org>
Signed-off-by: Rob Norris <rob.norris at klarasystems.com>
Closes #17420
(cherry picked from commit e3f5e317e0ed4da44a10fe202ec80abed1c8cc87)
Signed-off-by: Andriy Tkachuk <atkachuk at wasabi.com>
CI: Update Alpine Linux runner to 3.24.1
Update the Alpine Linux CI runner from 3.23.2 to 3.24.1.
This refreshes the runner to the latest Alpine release while keeping
the existing CI configuration unchanged.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alexander Moch <mail at alexmoch.com>
Closes #18790
