dmu_recv: Avoid potential null deref
Compilers are smart enough to deref only if the first && operand is
true, so this is mostly to avoid false positives from sanitizers.
Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Igor Ostapenko <igor.ostapenko at klarasystems.com>
Closes #18848
DDT: Fix several bugs in pruning
- Fix variables types to avoid overflows after 2B entries.
- Make ddt_prune_walk() code some more symmetrical.
- Fix zero oldest on exact target to histogram value match.
- Make bin 0 properly start from 0, not 1 hour.
- Take as a cutoff base a time of histogram build start.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alexander Motin <alexander.motin at TrueNAS.com>
Closes #18838
zpool initialize: add -z to write zeroes
zpool initialize writes a non-zero pattern (0xdeadbeefdeadbeee) to free
space. The pattern can be changed globally with the
zfs_initialize_value module parameter, but there was no per-command way
to zero free space, which is what is wanted to reclaim thinly
provisioned backing storage.
Add a -z/--zero flag to "zpool initialize" that writes zeroes for that
run. The fill value is threaded from the CLI through libzfs and the
POOL_INITIALIZE ioctl into vdev_initialize(), and is recorded in the
leaf ZAP so a run that is suspended and resumed (including across export
and import) keeps writing the same value. When no value is supplied the
zfs_initialize_value default is used, preserving existing behavior. -z
is only valid when starting initialization.
The zpool-initialize(8) page notes that -z targets backing storage which
does not support TRIM and points at zpool-trim(8) as the more efficient
option when the device does support it; zfs.4 documents
[12 lines not shown]
cstyle: better tolerance for struct literals
`cstyle.pl` currently doesn't have much patience for code such as:
```
*myvar = (mystruct_t) {
.ms_field = 42,
.ms_other_field = "chow time"
};
```
The first line is a Catch-22. If there's a space before the curly brace,
then it's an illegal cast because of the trailing space. If there isn't
a space, then it's an illegal curly brace without a preceding space.
Either way, tagging the first line as `/* CSTYLED */` gets you nowhere
because `cstyle.pl` doesn't understand the structure. It sees the
fields as continuation lines and complains about the indentation.
[16 lines not shown]
Bound explicit user prefetch to a fraction of the ARC
posix_fadvise(POSIX_FADV_WILLNEED) prefetch goes through dmu_prefetch(),
which issued every block of the request with no cap on outstanding I/O.
A large hint could pin memory without bound and OOM the system (#15776);
only the speculative prefetcher was capped.
Add dmu_prefetch_user() for explicit user requests and route the Linux
and FreeBSD fadvise WILLNEED paths through it. It bounds the
outstanding prefetch to a budget from arc_prefetch_max_bytes() -- a
quarter of the adaptive ARC target when warm, and of the midpoint toward
arc_c_max while the cache is still cold, so a hint right after boot is
not starved. dmu_prefetch() and the internal callers (bpobj, receive,
zvol) are left unthrottled.
Each request takes up to half the remaining budget, so concurrent hints
share it rather than the first taking everything, and that budget also
drives the block-range split -- a tight budget prefetches all of the
indirects and only some data, which suits a following random read. The
[14 lines not shown]
FreeBSD: Enable Direct IO by default
Commits 25eb538778, 178682506f, and 8dc452d907 resolve the remaining
known issues with Direct IO on FreeBSD. Enable it by default on all
platforms.
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #16761
ZTS: add scrub-from-txg into the runfile
zpool_scrub_txg_continue_from_last (#16301) and
zpool_events_scrub_txg_continue_from_last (#17432) are listed in
Makefile.am but not referenced in common.run thus ignored into CI
and ZTS
The test defines that the last_scrubbed_txg is initially 0, which is
only valid on a pool with no scrub history. Recreate the pool so the
test does not depend on its position in the runfile.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Christos Longros <chris.longros at gmail.com>
Closes #18785
dnode_sync: Relax constraint on indirect freeing
One of the roles of the dnode sync thread is freeing. There are two
modes of this operation: freeing a portion of an object, and freeing
the object itself.
Freeing is handled by `free_children()`. This function verifies that
if the dnode has children there must be at least one marked as dirty
before proceeding to recursively free the tree of block pointers.
As described in a comment, this VERIFY is overly strict. In practice,
children may not be marked as dirty when the dnode itself is being
freed. Relax the VERIFY slightly by avoiding the VERIFY when indirect
blocks are to be immediately freed, to more closely match the comment
motivating the VERIFY.
While here, clarify the logic of the test by using VERIFY_IMPLY instead
of chaining multiple tests.
[5 lines not shown]
libspl: Implement VERIFY_IMPLY and VERIFY_EQUIV
The libspl debug header is missing VERIFY_IMPLY and VERIFY_EQUIV macros
and instead directly implements IMPLY and EQUIV. Break out the VERIFY
definitions to match the kernel macros and facilitate code sharing
between kernel and userland.
Sponsored-by: Cybersecure Pty Ltd
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Ryan Moeller <ryan.moeller at klarasystems.com>
Closes #18822
mmp: skip non-writeable vdevs during activity check
The import-time MMP activity check added by c710f8792 writes an
uberblock to each top-level vdev and requires a matching number of
good writes before the pool is claimed. Two vdev types that carry
no writeable device break this:
- A hole vdev (left by removing a log) and an indirect vdev (left
by removing a data device) are counted in the required-write
total but can never be written, so good_writes never reaches
req_writes. The activity check then spuriously fails and the
pool is reported as held by another host with hostid 0.
- mmp_claim_uberblock() also issues a zio_flush() to the root vdev
after the writes. zio_flush() recurses to every leaf, and an
indirect vdev is a childless top-level vdev, so it is issued a
ZIO_TYPE_FLUSH. That trips the ZIO_TYPE_WRITE assertion in
vdev_indirect_io_start() and panics.
[12 lines not shown]
BRT: Implement partial bv_entcount writing
From the introduction of BRT the bv_entcount was always written all
at once, even though there was a mechanism for partial writing.
Implement one finally to slightly reduce per-TXG dirty data and write
inflation on large pools. On 1PB pool it meant 128MB write for each
TXG modifying BRT.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alexander Motin <alexander.motin at TrueNAS.com>
Closes #18821
zdb: output refcounts from verify_spacemap_refcounts()
Output information about refcounts when there is refcount mismatch.
Use plain uint64_t even as the values are expected to not be large.
Also use unsigned as we should never get negative refcounts there.
Reviewed-by: Allan Jude <allan at klarasystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Toomas Soome <tsoome at me.com>
Closes #18809
L2ARC: bound the rebuild by the write hand on a first sweep
l2arc_log_blkptr_valid() ends with (!evicted || dev->l2ad_first), which
disables the eviction-overlap test entirely on a first sweep. That test
is meaningless in that state, since l2arc_evict() returns immediately
and l2ad_evict never advances off l2ad_start, but dropping it leaves the
log block bounded only by device geometry. On a first sweep only the
region below the write hand has been written, so a block at or beyond
l2ad_hand describes data this incarnation never wrote.
l2arc_hdr_restore() performs no per-entry validation, so every such
entry inflates arcstat_l2_psize and, via vdev_space_update(), the cache
vdev's vs_alloc. Nothing reconciles it because l2arc_evict() never runs
on a first sweep, and once vs_alloc exceeds vs_space the unclamped
subtraction in zpool(8) wraps and the device reports 16.0E free. Stale
entries also let the L2ARC read offsets that were never written, which
then fail checksum verification.
Removing and re-adding a cache vdev is enough to set this up:
l2arc_add_vdev() resets l2ad_hand to l2ad_start while the previous
[21 lines not shown]
ZTS: ensure zfs_channel_program_support test is installed
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18836
zfs recv: warn when a non-raw incremental diverges a raw IV set
A non-raw incremental received onto a snapshot that was itself received
raw re-stamps the destination's IV set guid with a fresh value, so it no
longer matches the sending lineage. Nothing rejects this at the time,
but a later raw incremental taken from that snapshot is rejected with an
IV set guid mismatch (#8758). The failure surfaces arbitrarily later,
often on a different host, with no hint that an earlier non-raw receive
was the cause.
The receive itself cannot tell that a snapshot was received raw from
anything on disk: a raw-received snapshot and an ordinary locally-keyed
one both carry an IV set guid of the same shape, one stamped from the
stream and one from unique_create(), with no recorded provenance. So
warning on every non-raw incremental into an encrypted dataset would
fire on the common keys-local backup workflow that never hits the bug.
Record the provenance instead. A raw receive now stamps a
DS_FIELD_RAW_RECEIVED marker on the new snapshot, next to the ivset
[23 lines not shown]
libzfs: clarify the raw incremental "IV set guid mismatch" error
When a raw incremental receive is rejected because the IV set guid of
the incremental source snapshot does not match the send stream, the
error only said "IV set guid mismatch" and pointed at the man page.
That leaves the user with no idea what actually happened or how to
recover, which is a recurring complaint in #8758: the state that
triggers the error is created silently, most commonly by a single
non-raw incremental receive in an otherwise raw chain (e.g. a
replication tool falling back or omitting -w once). Such a receive
copies the dataset guid from the stream but re-encrypts the data with
a freshly generated IV set, so the destination snapshot looks
identical to guid-matching tools while any later raw incremental based
on it must be rejected to avoid mixing two incompatible ciphertext
lineages.
Name the non-raw-receive cause and the recovery (roll the destination
back to the most recent raw-received snapshot and take the raw
incremental from there, or receive a new full raw stream) directly in
[14 lines not shown]
ZTS: assert the IV set guid divergence behind mixed raw/non-raw sends
send_mixed_raw already exercises the failing sequence from #8758: a
raw full receive, then a non-raw incremental receive, then a raw
incremental receive that is rejected with "IV set guid mismatch".
However it only asserted the final failure, not the mechanism, so a
regression in how receives stamp the IV set guid could go unnoticed
as long as something still failed.
Extend the test to pin down the actual state transitions:
- A raw receive must preserve both the dataset guid and the IV set
guid of the received snapshot.
- A non-raw incremental receive copies the dataset guid from the
stream but generates a fresh IV set guid on the destination. This
is the silent divergence that later raw incrementals trip over:
the snapshots look identical to guid-matching tools while their
IV sets no longer match.
[14 lines not shown]
man: conditionally install pam_zfs_key.8
When built with --enable-pam include the pam_zfs_key.8 man
page, otherwise omit it. This aligns man/Makefile.am with
the build logic in contrib/Makefile.am.
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #18815
Closes #18817
zstream: add "drop records" chain module
This patch adds a module to remove records from a stream according to a
user-specified record-type mask. For example:
```
zstream_chain_t drop_writes {
...
serial_drop_record_types(DROP_WRITE_EMBEDDED | DROP_WRITE),
...
}
```
This is a trivial module, but there are a couple of reasons to use it
instead of just ignoring records you don't care about:
- It makes filtering a declarable API contract. Past the filter point,
you do not need to double-check record types beyond those you're
actually interested in.
[21 lines not shown]
tests: add mmap_read_truncate regression test
Exercise the mmap read-past-EOF race fixed by 223b8bc446 (#18715): a
page faulted in for read after the file is truncated below it reaches
zfs_fillpage() with io_off >= i_size. Before the fix the unsigned
io_len = i_size - io_off underflowed and dmu_read() zero-filled far past
the page, corrupting memory. Reader processes mmap and fault every page
of a file while a truncator churns its size; reads beyond EOF raise a
tolerated SIGBUS. A fixed module completes the run; an unfixed debug
module panics on the VERIFY3U(io_off, <, i_size) the fix removed.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Michael Heller <michael.heller at gmail.com>
Closes #18824
Harden recv record validation
Refactors and enhances how we do recv record validation. I've
made sure to keep all existing checks and added a few new ones.
The per-record type validator functions are also used by zstream
so this is now common code. Additionally, more detailed error
messages have been plumbed through to user space.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Garth Snyder <garth at garthsnyder.com>
Signed-off-by: Alek Pinchuk <Alek.Pinchuk at connectwise.com>
Closes #18725
ZTS: make zpool_iostat_interval_all teardown deterministic
zpool_iostat_interval_all runs "zpool iostat" in the background at a
0.1s interval and compares its output, parsed into a sequence of
chunks, against a fixed expected sequence as pools are created,
imported, exported and destroyed. Every step changes the visible pool
list by exactly one pool except the teardown, which used a single
"zpool export -a".
export -a exports the pools one after another rather than atomically,
so there is a brief window in which one pool is already gone and the
other is not. At the 0.1s sampling interval iostat occasionally catches
that intermediate single-pool state and emits an extra chunk that is
not in the expected sequence, and the test fails. Whether a sample
lands in the window depends on timing, which is why it shows up as a
flake.
Export the two pools explicitly, one at a time, and add the
intermediate single-pool state to the expected output, the same way
[9 lines not shown]
Add configure option to disable zfs channel programs
Introduce a configure time option to build OpenZFS without
channel program support. When disabled, the channel program ioctl
is not registered and attempts to invoke a channel program return
CMD_UNAVAILABLE.
This also removes the embedded channel program used for snapshot
deletion and updates the related build files, tests, and manpage
install rules so they reflect whether channel program support is
enabled.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Eduardo Alvarado <edalv.4Q7GW1IdW1+0 at gmail.com>
Closes #18658
libzfs: fallback VDEV_UPATH to VDEV_PATH for non-DM devices
When zfs_get_underlying_path() returns NULL for a non-DM device (e.g.
NVMe), the zfs_prepare_disk script was getting an empty VDEV_UPATH. Per
the man page, VDEV_UPATH should fall back to VDEV_PATH when there is no
underlying path.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: MISAPOR LAB <misapor at lab.misapor.pl>
Closes #18439
Closes #18802
zpool_influxdb: export per-vdev slow_ios counter
OpenZFS 2.4 added slow vdev sit-out detection, visible in 'zpool status
-s' as the SLOW column. Export the same vs_slow_ios counter in
zpool_influxdb output so monitoring stacks can alert on slow vdevs.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: MISAPOR LAB <misapor at lab.misapor.pl>
Closes #18804
Closes #18555
ABD: Validate borrowed buffer length
abd_borrow_buf() currently asserts that an unsigned size is at least
zero. This is tautological and allows a caller to borrow more bytes
than the ABD represents.
Assert that the ABD covers the requested length in the Linux, FreeBSD,
and libzpool implementations.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Kamil Monicz <kamil at monicz.dev>
Closes #18811
Fix reads for blocks freed after being cloned
PR #18421 fixed a case when reads for blocks cloned after being
freed could return zeroes. But it created an opposite problem,
when reads for blocks freed after being cloned could return non-
zero content from the cloning.
This patch fixes the problem by creating a more specialized
form of dnode_block_freed(), taking into account the TXG when
the cloning has happened and checking frees only in TXGs after.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Gary Guo <gary at kernel.org>
Signed-off-by: Alexander Motin <alexander.motin at TrueNAS.com>
Closes #18421
Closes #18724
Fix dmu_zfetch_prime() assuming a stream was created
dmu_zfetch_prime() calls dmu_zfetch_stream_create() and then takes
list_head(&zf->zf_stream), asserting its zs_blkid equals the requested
blkid. But dmu_zfetch_stream_create() only inserts a new head stream
in its normal path; when zf_numstreams has reached zfetch_max_streams
and no stream is reclaimable it bumps zfetchstat_max_streams and
returns without creating one, leaving an unrelated stream at the head.
dmu_zfetch_prime() then trips VERIFY3U(zs->zs_blkid, ==, blkid) (a
debug build panics; otherwise it primes the wrong stream).
Return the created/reused stream from dmu_zfetch_stream_create(), or
NULL when it declines, and have dmu_zfetch_prime() stop priming in
that case instead of trusting list_head. dmu_zfetch_prepare(), the
only other caller, does not use the returned stream.
This is reachable by any dmu_prefetch_stream() caller that drives more
than zfetch_max_streams concurrent streams on a dnode.
[3 lines not shown]
libzfs: don't abort receiving a raw encrypted send that carries holds
A raw send that gathers holds but not properties (zfs send -w -h,
without -p or -R) produces a compound stream whose per-filesystem
nvlist has no "props" entry. recv_fix_encryption_hierarchy() looked
that entry up with fnvlist_lookup_nvlist(), which aborts the process
when the key is missing, so receiving such a stream of an encrypted
dataset crashed with a failed VERIFY0() after the data had already
been received. Release builds abort the same way, since the VERIFY
family is always compiled in.
Look "props" up tolerantly and skip the keylocation fixup when it is
absent: a holds-only raw send sends no properties, and the raw
receive has already established a usable keylocation. The
FORCE_NEW_KEY and FORCE_INHERIT fixups, which do not need
properties, still run.
A new rsend test sends an encrypted dataset with -w -h, both a full
and an incremental stream, and confirms the receive succeeds with
[6 lines not shown]
Rate limit Direct I/O verify zevents
Each vdev initializes a vdev_dio_verify_rl rate limiter (governed by
zfs_dio_write_verify_events_per_second), but
zio_dio_chksum_verify_error_report() never consults it, so
dio_verify_rd and dio_verify_wr zevents are posted with no rate
limiting. A workload that repeatedly trips the Direct I/O verify can
therefore produce an unbounded flood of zevents.
Gate both ereport posts through zfs_ratelimit(&vd->vdev_dio_verify_rl),
as is already done for the other per-vdev ereports (checksum, delay,
deadman). The vs_dio_verify_errors vdev stat still increments on every
event, so the true count remains observable via zpool status -d.
The dio_write_verify test checks on every iteration that a
dio_verify_wr zevent was posted. With rate limiting now in effect the
shared limiter window is exhausted after the first iteration, so later
iterations observe zero events and the test fails. Raise
zfs_dio_write_verify_events_per_second for the duration of that test
[5 lines not shown]
