zed: let autoexpand see capacity changes on partitioned disks
Growing a disk under a whole-disk vdev never triggers autoexpand
(#12505). The kernel reports a capacity change on the disk itself
and nothing for the partitions, whose sizes did not change. But
since zfs owns the whole disk it carries a partition table, and
zed_udev_monitor() drops any disk-with-partitions event on the
assumption that a partition event will follow. For a resize none
ever does, so the ESC_DEV_DLE event that zfsdle_vdev_online() needs
is never generated and the pool stays at the old size until someone
runs zpool online -e by hand. This is the common case for cloud
disks grown online.
Pass change events through when udev marks them RESIZE=1. On the
matching side a disk-level event has no vdev guid to search by (the
label lives on the partition), and udev provides no ID_PATH on some
buses, so the physical path lookup can also come up empty. When
that happens, read the ZFS label off the whole-disk partition and
match by the pool and vdev guids stored in it. Unlike matching the
[33 lines not shown]
zio: don't strip gang stages from the DDT free pipeline
zio_create() adds ZIO_GANG_STAGES to the pipeline of a logical free
of a gang BP, but zio_free_bp_init() replaced the whole pipeline
with ZIO_DDT_FREE_PIPELINE for dedup BPs, discarding those stages.
When zio_ddt_free() takes the pruned-entry fallback (the entry was
legitimately removed by zpool ddtprune, or missing for any other
reason), the plain free that follows only frees the gang header
extents (metaslab_free_dva() frees vdev_gang_header_asize() for a
gang DVA) and every gang member leaks permanently.
OR the full ZIO_DDT_FREE_PIPELINE into the existing pipeline instead
of replacing it. zio_free_sync() is the only creator of these zios,
and for a dedup BP it already builds them with ZIO_FREE_PIPELINE and
ZIO_STAGE_ISSUE_ASYNC, so today this only adds ZIO_STAGE_DDT_FREE --
but ORing the whole declaration keeps this site correct if the DDT
free pipeline ever grows a stage. Adding rather than replacing also
keeps any stage zio_create() adds for this BP, so a future
conditional stage cannot be silently dropped here again.
[18 lines not shown]
ddt: select traditional phys by block identity, not BP DVA count
The write path stores a traditional DDT entry in the phys slot chosen
by the zp_copies value the block was written with, but the free-path
lookup validation (ddt_entry_lookup_is_valid()) selected the slot to
compare by the BP's physical DVA count, and ddt_addref() bumped the
refcount of that same wrongly-selected slot.
The two indexes diverge whenever a deduplicated block's BP carries
more DVAs than its copies value, which is exactly what ganging does:
a gang header is stored in more copies than the data it gangs, so
copies=1 blocks get a two-DVA gang header BP (unconditionally
through 2.3.3, and under the default redundant_metadata=all since
a46ce73ca8 made it configurable in 2.3.4 and 9250403ba6 in 2.4.0),
and from 2.3.4 that default also gives unencrypted copies=2 blocks a
three-DVA header BP. Encryption does not avoid the mismatch at
copies=1: the header BP still carries two counted DVAs (the salt/IV
DVA is separate). For such a block the validation checked
the wrong slot, concluded the entry had been pruned, and
[50 lines not shown]
zpool export: return EBUSY when zvol minors are in use
When a zvol block device is open (e.g., mounted by the OS), exporting
the pool can hang indefinitely in zvol_remove_minors_impl() which calls
cv_wait(&zv->zv_removing_cv) with no timeout. This is because the open
holder never closes the device, so the condition variable is never
signaled.
Fix this by taking an additional spa reference in zvol_first_open() and
releasing it in zvol_last_close(). With this reference,
spa_export_common() will return EBUSY via the existing
spa_refcount_zero() check, matching the behavior already provided for
mounted ZFS datasets.
Additionally, move the zvol_remove_minors() call in spa_export_common()
to after the spa_refcount_zero() check. Previously it was called before
the check, which could still hang if a zvol was open.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Heonje LEE <lhjnano at gmail.com>
Closes #18841
ZTS: make file_check actually compare the resume test results
file_check guards every comparison with a check that the snapshot
directory exists on both sides, and the resume tests receive with -u,
so the receive side is never mounted and the .zfs snapshot paths never
exist. The function has been quietly comparing nothing in
rsend_019-022, rsend_024, rsend_030 and send-c_resume, so the resume
test family verified that receives succeed but not that the received
data matches.
Mount both sides before diffing (some tests also unmount the send
side), still compare only the snapshots both sides carry since several
tests send just one of them, and fail loudly when nothing at all was
compared so the check cannot rot back into a no-op. Two callers
needed their expectations fixed once the checks came alive: rsend_024
streams from the head rather than a snapshot, so it now diffs the
mounted heads directly, and the first file_check in
send_partial_dataset pointed at a partial dataset with no snapshots,
so it now compares against the dataset the stream came from.
[7 lines not shown]
ZTS: save ZAP_MICRO_MAX_SIZE before the large microzap tests change it
send_large_microzap_incremental and send_large_microzap_transitive set
zap_micro_max_size to 1M and call restore_tunable in cleanup, but
restore_tunable is a silent no-op unless save_tunable stored the old
value first, which neither test ever did. The tunable therefore stays
at 1M until the module is reloaded, and any later run of the rsend
group on the same module fails seven tests (rsend_019-022, rsend_024,
rsend_030 and send-c_resume): their setup creates directories large
enough to become large microzaps, and plain zfs send then refuses the
stream without -L. CI never reruns a group on a loaded module, so
this only bites people iterating locally.
Tests: rsend group run twice back to back on one module load; before
the change the second run fails the seven tests above, after it both
runs pass.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: MorganaFuture <103630661+MorganaFuture at users.noreply.github.com>
Closes #18834
dmu_recv: Avoid potential null deref
Compilers are smart enough to deref only if the first && operand is
true, so this is mostly to avoid false positives from sanitizers.
Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Igor Ostapenko <igor.ostapenko at klarasystems.com>
Closes #18848
DDT: Fix several bugs in pruning
- Fix variables types to avoid overflows after 2B entries.
- Make ddt_prune_walk() code some more symmetrical.
- Fix zero oldest on exact target to histogram value match.
- Make bin 0 properly start from 0, not 1 hour.
- Take as a cutoff base a time of histogram build start.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alexander Motin <alexander.motin at TrueNAS.com>
Closes #18838
zpool initialize: add -z to write zeroes
zpool initialize writes a non-zero pattern (0xdeadbeefdeadbeee) to free
space. The pattern can be changed globally with the
zfs_initialize_value module parameter, but there was no per-command way
to zero free space, which is what is wanted to reclaim thinly
provisioned backing storage.
Add a -z/--zero flag to "zpool initialize" that writes zeroes for that
run. The fill value is threaded from the CLI through libzfs and the
POOL_INITIALIZE ioctl into vdev_initialize(), and is recorded in the
leaf ZAP so a run that is suspended and resumed (including across export
and import) keeps writing the same value. When no value is supplied the
zfs_initialize_value default is used, preserving existing behavior. -z
is only valid when starting initialization.
The zpool-initialize(8) page notes that -z targets backing storage which
does not support TRIM and points at zpool-trim(8) as the more efficient
option when the device does support it; zfs.4 documents
[12 lines not shown]
cstyle: better tolerance for struct literals
`cstyle.pl` currently doesn't have much patience for code such as:
```
*myvar = (mystruct_t) {
.ms_field = 42,
.ms_other_field = "chow time"
};
```
The first line is a Catch-22. If there's a space before the curly brace,
then it's an illegal cast because of the trailing space. If there isn't
a space, then it's an illegal curly brace without a preceding space.
Either way, tagging the first line as `/* CSTYLED */` gets you nowhere
because `cstyle.pl` doesn't understand the structure. It sees the
fields as continuation lines and complains about the indentation.
[16 lines not shown]
Bound explicit user prefetch to a fraction of the ARC
posix_fadvise(POSIX_FADV_WILLNEED) prefetch goes through dmu_prefetch(),
which issued every block of the request with no cap on outstanding I/O.
A large hint could pin memory without bound and OOM the system (#15776);
only the speculative prefetcher was capped.
Add dmu_prefetch_user() for explicit user requests and route the Linux
and FreeBSD fadvise WILLNEED paths through it. It bounds the
outstanding prefetch to a budget from arc_prefetch_max_bytes() -- a
quarter of the adaptive ARC target when warm, and of the midpoint toward
arc_c_max while the cache is still cold, so a hint right after boot is
not starved. dmu_prefetch() and the internal callers (bpobj, receive,
zvol) are left unthrottled.
Each request takes up to half the remaining budget, so concurrent hints
share it rather than the first taking everything, and that budget also
drives the block-range split -- a tight budget prefetches all of the
indirects and only some data, which suits a following random read. The
[14 lines not shown]
FreeBSD: Enable Direct IO by default
Commits 25eb538778, 178682506f, and 8dc452d907 resolve the remaining
known issues with Direct IO on FreeBSD. Enable it by default on all
platforms.
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #16761
ZTS: add scrub-from-txg into the runfile
zpool_scrub_txg_continue_from_last (#16301) and
zpool_events_scrub_txg_continue_from_last (#17432) are listed in
Makefile.am but not referenced in common.run thus ignored into CI
and ZTS
The test defines that the last_scrubbed_txg is initially 0, which is
only valid on a pool with no scrub history. Recreate the pool so the
test does not depend on its position in the runfile.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Christos Longros <chris.longros at gmail.com>
Closes #18785
dnode_sync: Relax constraint on indirect freeing
One of the roles of the dnode sync thread is freeing. There are two
modes of this operation: freeing a portion of an object, and freeing
the object itself.
Freeing is handled by `free_children()`. This function verifies that
if the dnode has children there must be at least one marked as dirty
before proceeding to recursively free the tree of block pointers.
As described in a comment, this VERIFY is overly strict. In practice,
children may not be marked as dirty when the dnode itself is being
freed. Relax the VERIFY slightly by avoiding the VERIFY when indirect
blocks are to be immediately freed, to more closely match the comment
motivating the VERIFY.
While here, clarify the logic of the test by using VERIFY_IMPLY instead
of chaining multiple tests.
[5 lines not shown]
libspl: Implement VERIFY_IMPLY and VERIFY_EQUIV
The libspl debug header is missing VERIFY_IMPLY and VERIFY_EQUIV macros
and instead directly implements IMPLY and EQUIV. Break out the VERIFY
definitions to match the kernel macros and facilitate code sharing
between kernel and userland.
Sponsored-by: Cybersecure Pty Ltd
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Ryan Moeller <ryan.moeller at klarasystems.com>
Closes #18822
mmp: skip non-writeable vdevs during activity check
The import-time MMP activity check added by c710f8792 writes an
uberblock to each top-level vdev and requires a matching number of
good writes before the pool is claimed. Two vdev types that carry
no writeable device break this:
- A hole vdev (left by removing a log) and an indirect vdev (left
by removing a data device) are counted in the required-write
total but can never be written, so good_writes never reaches
req_writes. The activity check then spuriously fails and the
pool is reported as held by another host with hostid 0.
- mmp_claim_uberblock() also issues a zio_flush() to the root vdev
after the writes. zio_flush() recurses to every leaf, and an
indirect vdev is a childless top-level vdev, so it is issued a
ZIO_TYPE_FLUSH. That trips the ZIO_TYPE_WRITE assertion in
vdev_indirect_io_start() and panics.
[12 lines not shown]
BRT: Implement partial bv_entcount writing
From the introduction of BRT the bv_entcount was always written all
at once, even though there was a mechanism for partial writing.
Implement one finally to slightly reduce per-TXG dirty data and write
inflation on large pools. On 1PB pool it meant 128MB write for each
TXG modifying BRT.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alexander Motin <alexander.motin at TrueNAS.com>
Closes #18821
zdb: output refcounts from verify_spacemap_refcounts()
Output information about refcounts when there is refcount mismatch.
Use plain uint64_t even as the values are expected to not be large.
Also use unsigned as we should never get negative refcounts there.
Reviewed-by: Allan Jude <allan at klarasystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Toomas Soome <tsoome at me.com>
Closes #18809
L2ARC: bound the rebuild by the write hand on a first sweep
l2arc_log_blkptr_valid() ends with (!evicted || dev->l2ad_first), which
disables the eviction-overlap test entirely on a first sweep. That test
is meaningless in that state, since l2arc_evict() returns immediately
and l2ad_evict never advances off l2ad_start, but dropping it leaves the
log block bounded only by device geometry. On a first sweep only the
region below the write hand has been written, so a block at or beyond
l2ad_hand describes data this incarnation never wrote.
l2arc_hdr_restore() performs no per-entry validation, so every such
entry inflates arcstat_l2_psize and, via vdev_space_update(), the cache
vdev's vs_alloc. Nothing reconciles it because l2arc_evict() never runs
on a first sweep, and once vs_alloc exceeds vs_space the unclamped
subtraction in zpool(8) wraps and the device reports 16.0E free. Stale
entries also let the L2ARC read offsets that were never written, which
then fail checksum verification.
Removing and re-adding a cache vdev is enough to set this up:
l2arc_add_vdev() resets l2ad_hand to l2ad_start while the previous
[21 lines not shown]
ZTS: ensure zfs_channel_program_support test is installed
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18836
zfs recv: warn when a non-raw incremental diverges a raw IV set
A non-raw incremental received onto a snapshot that was itself received
raw re-stamps the destination's IV set guid with a fresh value, so it no
longer matches the sending lineage. Nothing rejects this at the time,
but a later raw incremental taken from that snapshot is rejected with an
IV set guid mismatch (#8758). The failure surfaces arbitrarily later,
often on a different host, with no hint that an earlier non-raw receive
was the cause.
The receive itself cannot tell that a snapshot was received raw from
anything on disk: a raw-received snapshot and an ordinary locally-keyed
one both carry an IV set guid of the same shape, one stamped from the
stream and one from unique_create(), with no recorded provenance. So
warning on every non-raw incremental into an encrypted dataset would
fire on the common keys-local backup workflow that never hits the bug.
Record the provenance instead. A raw receive now stamps a
DS_FIELD_RAW_RECEIVED marker on the new snapshot, next to the ivset
[23 lines not shown]
libzfs: clarify the raw incremental "IV set guid mismatch" error
When a raw incremental receive is rejected because the IV set guid of
the incremental source snapshot does not match the send stream, the
error only said "IV set guid mismatch" and pointed at the man page.
That leaves the user with no idea what actually happened or how to
recover, which is a recurring complaint in #8758: the state that
triggers the error is created silently, most commonly by a single
non-raw incremental receive in an otherwise raw chain (e.g. a
replication tool falling back or omitting -w once). Such a receive
copies the dataset guid from the stream but re-encrypts the data with
a freshly generated IV set, so the destination snapshot looks
identical to guid-matching tools while any later raw incremental based
on it must be rejected to avoid mixing two incompatible ciphertext
lineages.
Name the non-raw-receive cause and the recovery (roll the destination
back to the most recent raw-received snapshot and take the raw
incremental from there, or receive a new full raw stream) directly in
[14 lines not shown]
ZTS: assert the IV set guid divergence behind mixed raw/non-raw sends
send_mixed_raw already exercises the failing sequence from #8758: a
raw full receive, then a non-raw incremental receive, then a raw
incremental receive that is rejected with "IV set guid mismatch".
However it only asserted the final failure, not the mechanism, so a
regression in how receives stamp the IV set guid could go unnoticed
as long as something still failed.
Extend the test to pin down the actual state transitions:
- A raw receive must preserve both the dataset guid and the IV set
guid of the received snapshot.
- A non-raw incremental receive copies the dataset guid from the
stream but generates a fresh IV set guid on the destination. This
is the silent divergence that later raw incrementals trip over:
the snapshots look identical to guid-matching tools while their
IV sets no longer match.
[14 lines not shown]
man: conditionally install pam_zfs_key.8
When built with --enable-pam include the pam_zfs_key.8 man
page, otherwise omit it. This aligns man/Makefile.am with
the build logic in contrib/Makefile.am.
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #18815
Closes #18817
zstream: add "drop records" chain module
This patch adds a module to remove records from a stream according to a
user-specified record-type mask. For example:
```
zstream_chain_t drop_writes {
...
serial_drop_record_types(DROP_WRITE_EMBEDDED | DROP_WRITE),
...
}
```
This is a trivial module, but there are a couple of reasons to use it
instead of just ignoring records you don't care about:
- It makes filtering a declarable API contract. Past the filter point,
you do not need to double-check record types beyond those you're
actually interested in.
[21 lines not shown]
tests: add mmap_read_truncate regression test
Exercise the mmap read-past-EOF race fixed by 223b8bc446 (#18715): a
page faulted in for read after the file is truncated below it reaches
zfs_fillpage() with io_off >= i_size. Before the fix the unsigned
io_len = i_size - io_off underflowed and dmu_read() zero-filled far past
the page, corrupting memory. Reader processes mmap and fault every page
of a file while a truncator churns its size; reads beyond EOF raise a
tolerated SIGBUS. A fixed module completes the run; an unfixed debug
module panics on the VERIFY3U(io_off, <, i_size) the fix removed.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Michael Heller <michael.heller at gmail.com>
Closes #18824
Harden recv record validation
Refactors and enhances how we do recv record validation. I've
made sure to keep all existing checks and added a few new ones.
The per-record type validator functions are also used by zstream
so this is now common code. Additionally, more detailed error
messages have been plumbed through to user space.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Garth Snyder <garth at garthsnyder.com>
Signed-off-by: Alek Pinchuk <Alek.Pinchuk at connectwise.com>
Closes #18725
ZTS: make zpool_iostat_interval_all teardown deterministic
zpool_iostat_interval_all runs "zpool iostat" in the background at a
0.1s interval and compares its output, parsed into a sequence of
chunks, against a fixed expected sequence as pools are created,
imported, exported and destroyed. Every step changes the visible pool
list by exactly one pool except the teardown, which used a single
"zpool export -a".
export -a exports the pools one after another rather than atomically,
so there is a brief window in which one pool is already gone and the
other is not. At the 0.1s sampling interval iostat occasionally catches
that intermediate single-pool state and emits an extra chunk that is
not in the expected sequence, and the test fails. Whether a sample
lands in the window depends on timing, which is why it shows up as a
flake.
Export the two pools explicitly, one at a time, and add the
intermediate single-pool state to the expected output, the same way
[9 lines not shown]
Add configure option to disable zfs channel programs
Introduce a configure time option to build OpenZFS without
channel program support. When disabled, the channel program ioctl
is not registered and attempts to invoke a channel program return
CMD_UNAVAILABLE.
This also removes the embedded channel program used for snapshot
deletion and updates the related build files, tests, and manpage
install rules so they reflect whether channel program support is
enabled.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Eduardo Alvarado <edalv.4Q7GW1IdW1+0 at gmail.com>
Closes #18658
libzfs: fallback VDEV_UPATH to VDEV_PATH for non-DM devices
When zfs_get_underlying_path() returns NULL for a non-DM device (e.g.
NVMe), the zfs_prepare_disk script was getting an empty VDEV_UPATH. Per
the man page, VDEV_UPATH should fall back to VDEV_PATH when there is no
underlying path.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: MISAPOR LAB <misapor at lab.misapor.pl>
Closes #18439
Closes #18802
