ZIL replay can apply part of a split WR_NEED_COPY write
After a crash while an fsync() is still writing the log, ZIL replay can
apply the first part of a write without the rest, leaving the write
torn at an arbitrary 8-byte-aligned offset. This exposes a state that
no reader could have observed before the crash: new data up to the
tear, old data after it. While POSIX doesn't specify anything before
fsync() completes, ZFS intends to preserve the atomicity of writes as
much as possible: writes within a single block are intended to be
atomic, with larger writes torn only on recordsize boundaries.
The problem occurs with WR_NEED_COPY TX_WRITE records. When such a
record does not fit in the current log block, zil_lwb_assign() splits
it into several TX_WRITE records in consecutive log blocks. If the
system crashes after the first of those blocks is on disk but before
all of them are, zil_parse() stops at the missing block, but replay
has already applied the pieces before it. Files are affected for
async writes that a later fsync() commits, and for writes of any size
with a slog. Zvols are affected too, where a tear inside a sector
[75 lines not shown]
vdev_mirror: scrub every copy of a gang block in a sorted scrub
A sorted scrub queues one read per DVA of a block, and each of these
reads carries all of the block's DVAs so that a damaged copy can be
repaired. vdev_mirror_map_init() therefore reads only the first DVA
of a scrub read while a sorted scrub runs, since the other DVAs are
checked by their own queued reads, and reads them all only on retry.
Gang blocks are not queued: dsl_scan_enqueue() issues them at once and
whole, and their gang headers and gang members are read by gang
children of that one read. These inherit ZIO_FLAG_SCRUB and so take
the same shortcut, but nothing else ever reads their other DVAs. A
damaged copy of a gang header or gang member, other than the first, is
neither detected nor repaired by a scrub, unless the first copy is
damaged too and the retry reads them all. With the default
redundant_metadata=all every gang header has at least two copies, and
gang members have as many as the copies property asks for. A legacy
scrub (zfs_scan_legacy=1) reads and repairs every copy.
[11 lines not shown]
ZTS: test that a scrub repairs every copy of a gang block
With the default redundant_metadata=all, a gang header gets one more
copy than the block it describes, up to three, and each gang child
gets as many as the block. A scrub with the default sorted scan reads
only the first copy of each, so the other copies of a gang header or
of a gang child can be damaged without the scrub detecting or
repairing it. A legacy scrub (zfs_scan_legacy=1) reads and repairs
them.
Add a test that forces 128K blocks to gang on a file vdev, overwrites
the second copy of a gang header (copies=1) and of a gang child
(copies=2), scrubs, and checks that both copies were restored.
This fails now: the scrub repairs neither copy.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Closes #19399
spl: avoid reclaim for speculative physical allocations
The constrained spl_kvmalloc() path allows direct reclaim while trying
to obtain physically contiguous memory, even though vmalloc can satisfy
the request with scattered pages. __GFP_NORETRY limits retries but does
not prevent that initial reclaim.
For multipage requests without __GFP_RETRY_MAYFAIL, clear direct reclaim
only from the physical attempt's flags. Keep the original flags for
fallback eligibility and vmalloc so a sleeping allocation can still
reclaim memory there. This follows the policy adopted by Linux 6.16
without changing the fallback needed by older supported kernels.
Retain the existing small-order retry policy and correct its obsolete
explanation of __GFP_RETRY_MAYFAIL.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Kamil Monicz <kamil at monicz.dev>
Closes #19406
raidz: refuse expansions that the boot area cannot hold
A RAID-Z expansion uses the boot area of each child as scratch space,
and raidz_reflow_scratch_sync() needs it to hold one sector for every
child of the expanded vdev:
VERIFY3U(write_size, >=, raidvd->vdev_children << ashift);
vdev_raidz_attach_check() is there to refuse an attach that would
break this, but spa_vdev_attach() calls it before the new leaf is
moved under the RAID-Z vdev, while its parent is still the temporary
root vdev it was parsed under. The check took the width and the
ashift from that root, one child and an ashift of zero, so it never
refused anything. Attaching a 57th disk to a RAID-Z vdev with
ashift=16, or a 113th with ashift=15, was accepted, and the VERIFY
failed in the syncing thread as soon as the expansion started.
Pass the RAID-Z vdev to vdev_raidz_attach_check() and count the child
that is about to be added. Return a new error for it rather than
[9 lines not shown]
CI: reuse fork jobs from any recent push run of the PR head
reuse-ci.py only looks at the newest push run of each workflow for the
PR head SHA. A fork's workflows run on every push, so pushing an already
tested commit to the PR branch starts new push runs for the same SHA
just as the PR event fires. Those runs are still queued when the PR's
reuse step checks, so nothing is reused and every workflow runs in both
repositories (#19410). A cancelled newer run hides an older success the
same way, and with reuse limited to the first attempt a rerun can no
longer pick up the fork results once they finish.
Look at the latest ten push runs for the SHA, newest first, and let the
newest finished result for each job decide it: a success in a completed
run is reused and a failure is not, so an older success still cannot
replace a newer failure. Jobs that were cancelled, skipped or have not
finished have no result and fall through to older runs. A failure that
has already finished in a running workflow still blocks reuse.
Reviewed-by: George Melikov <mail at gmelikov.ru>
[2 lines not shown]
Name dedup, not asize, in the zdb --class error messages
zdb --class accepts normal, special, dedup and other, as its usage
text and the man page say, but both of its error messages list
"asize" instead of "dedup". asize is not a class and is rejected
like any other unknown word, so the message sends the user to a value
that fails. Name dedup instead.
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Closes #19387
Find the dataset of a renamed volume by its object number
A volume looks its dataset up by zv_name when it is first opened, to
own it, after a rollback or a receive in zvol_resume(), to find its
objset again, and in zvol_setup_zv(), which both call, to read the
readonly property. A rename only updates zv_name later, though:
dsl_dir_rename_sync() hands the update to the zvol taskq through
zvol_rename_minors(). Until that task has run, zv_name is the old
name. A first open of the volume, or of any volume below a renamed
dataset, then fails with ENOENT. A rollback or a receive issued by the
old name in that window still finds the minor by it, and when
zvol_resume() then fails in zvol_setup_zv(), the volume stays open
without its dnode and the next read or write dereferences NULL; had the
hold in zvol_resume() failed, its VERIFY0() would have panicked.
Record the object number of the dataset when the minor is allocated and
look the dataset up by it, which a rename does not change, and read the
property from the dataset already held. The pool is still found by
zv_name, whose pool part a rename keeps. dmu_objset_own() also
[9 lines not shown]
Add zfs rewrite -W to skip blocks already at their current size
After a RAID-Z expansion, old blocks keep the data-to-parity ratio of
the old width until they are rewritten, and zfs rewrite is the way to
rewrite them. It rewrites every block it is given, though, including
the blocks written since the expansion, which gain nothing from it: a
rewrite that is interrupted, or run again after a later expansion,
starts over from the first block.
Add ZFS_REWRITE_SKIP_CURRENT_ASIZE, set by -W, which skips a block when
each of its DVAs already has the allocated size that metaslab
allocation would give its physical size on the same vdev, in the
transaction group of the rewrite. On a RAID-Z vdev that size depends
on the logical width of that transaction group, so a block written
before the last expansion completed is rewritten if the new width
stores it in less space. The blocks written since are skipped, and so
are the old blocks smaller than the old width, which take the same
space at the new width and stay in place. Comparing sizes rather than
widths keeps the check valid for whatever else may come to change that
[14 lines not shown]
dsl_crypt: say when change-key finds a key it cannot unwrap
When change-key reaches a dataset whose DSL Crypto Key cannot be
unwrapped with its encryption root's wrapping key, the check returns
EACCES and libzfs reports "Key is not currently loaded." libzfs has
already confirmed through keystatus that the root's wrapping key is
loaded before it issues the ioctl, so the message points at the wrong
problem.
spa_keystore_change_key_check_impl() now holds the root's wrapping
key while it takes the hold on each DSL Crypto Key. The wrapping key
cannot be unloaded in between, so an EACCES from the key hold is an
unwrap failure. Return ZFS_ERR_CRYPTO_KEY_MISMATCH for it and name the
dataset with dprintf_dd(). libzfs reports that a dataset sharing the
target's encryption root has a key that cannot be unwrapped. A
wrapping key that is not loaded still gives EACCES and the old
message.
zfs_change-key_mismatch now checks for the new message.
[4 lines not shown]
libspl: split usleep_range() into seconds and nanoseconds
The userspace usleep_range() put the entire minimum into tv_nsec.
For a minimum of one second or more that is not a valid timespec, so
nanosleep() failed with EINVAL and returned immediately, and the
caller did not sleep at all.
zfs_sleep_until() calls it, so every userspace sleep of a second or
more was skipped: the dmu_tx_delay(), vdev_indirect and zio_inject
delays.
Kernel builds are not affected, as Linux uses the kernel's own
usleep_range() and FreeBSD has its own macro.
Take the minimum modulo MICROSEC for tv_nsec. While here, evaluate
the argument once and parenthesise it, so an expression with side
effects or a lower precedence than the operators no longer misbehaves.
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
[2 lines not shown]
unit/timer: test usleep_range() in libspl
Add a unit test for the userspace usleep_range() macro. It checks
that a sub-second sleep and a sleep of more than one second both last
at least as long as requested, and that the minimum argument is
evaluated only once.
The macro currently fails the last two. It passes the whole minimum
as tv_nsec, which nanosleep() rejects with EINVAL once the minimum
reaches one second, so the call returns without sleeping. It also
evaluates the minimum twice. The fix follows.
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Closes #19414
Fix the in-memory size of indirect mappings in zdb -b
For each indirect vdev, zdb -b prints its number of segments and the
memory its mapping takes, but it formats the number of entries a second
time instead of the size of the mapping, and passes that size as the
length of the 32-byte buffer. The size it prints is therefore the
number of entries. With -P the number goes through snprintf() with
that length, which fortified builds catch as a buffer overflow: zdb -bP
aborts on any pool that has an indirect vdev.
Print vdev_indirect_mapping_size() with zdb_nicebytes(), into the
buffer with its real length. The new removal_indirect_mapping_size
test checks the exact size that zdb -bPL reports against the number of
entries.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Matthew Ahrens <matt at mahrens.org>
Signed-off-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Closes #19418
CI: make kmemleak and no CPU mitigations the Alpine default
Boot the Alpine runner with kmemleak=on and mitigations=off, and run ZTS
with -m whenever the kernel booted with kmemleak=on, so each test is
followed by a kmemleak scan. CPU mitigations roughly double the run time
with kmemleak on AMD runners. Without them, the run fits into the time
limit.
A -nokmemleak or -mitigations suffix keeps the kernel's default for that
option, so alpine3-24-nokmemleak-mitigations boots the plain kernel.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alexander Moch <mail at alexmoch.com>
Closes #19426
ZTS: failmode_dio_unwritable: accept musl's EIO error message
failmode_dio_unwritable expects stride_dd to report the failed write as
"Input/output error", which is glibc's wording for EIO. musl, used on
Alpine, words the same error "I/O error", so the test fails there even
though the write failed with EIO as intended.
Accept both wordings, as procfs_list_stale_read already does.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Matthew Ahrens <matt at mahrens.org>
Signed-off-by: Alexander Moch <mail at alexmoch.com>
Closes #19424
vdev_rebuild: wait for the syncing txg before rebuilding a metaslab
Before a sequential rebuild reads a metaslab, it disables allocations
from it and waits for pending allocations to sync. Later passes of the
syncing txg can still rewrite blocks born in that txg in place: a
rewrite allocates nothing, and the first pass has already moved their
allocations to the space map. A rebuild read which overlaps such a
rewrite sees some columns before it and some after. dRAID then finds
the row's parity inconsistent and withholds its repair, and a read
which completes before the rewrite can write the old block over the new
one on the device being rebuilt.
Wait for the txg that is syncing when the metaslab is disabled. No later
txg can allocate from it, so none can rewrite its blocks in place.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Andriy Tkachuk <atkachuk at wasabi.com>
Signed-off-by: Kamil Monicz <kamil at monicz.dev>
Closes #19198
Report EIO when no vdev can take a Direct I/O write
A Direct I/O write to a pool whose vdevs have all stopped taking
writes fails with ENOSPC. The same write done buffered fails at fsync
with EIO. fstests generic/271, 272 and 278 grep xfs_io's output for
"error", and "No space left on device" does not contain it.
metaslab_group_allocatable() skips a vdev that vdev_allocatable() says
cannot be written. When it skips every vdev in the class, the
allocation fails with ENOSPC, the only error that path returns.
Add metaslab_class_unwritable(), which reports whether a class has
vdevs and none of them is allocatable. When an allocation for a
ZIO_FLAG_CANFAIL zio fails with ENOSPC and its class is unwritable,
report EIO instead. A full pool keeps ENOSPC while any of its vdevs
can still be written, a degraded one included.
The allocator still returns ENOSPC to its callers, because the class
fallback and the gang retry in zio_dva_allocate() depend on it. The
[24 lines not shown]
raidz: resilver every child a block born before an expansion spans
vdev_raidz_need_resilver() assumes that a block spans its sectors
plus one row of parity over consecutive children. A block born
before a RAID-Z expansion keeps its logical width and is laid out in
rows of that width, each with its own parity columns, so it spans
s + p * howmany(s, width - p) children. Once the vdev has been
expanded more than once, this can exceed s + p while staying below
the vdev's width, and when the rows past the first end on a child
whose DTL is dirty, the resilver skips the block and leaves that
child without its sectors. A later scrub repairs them, but a second
failure in the same row before that leaves the row one column short
of what its parity can rebuild.
Count the columns of every row, from the logical width of the block's
birth txg. The count is unchanged for a block of the vdev's width,
which needs one row whenever the count is below the vdev's width.
The lookup takes vd_expand_lock, so it comes after the DTL check and
after the one-row test, which already settles most blocks.
[5 lines not shown]
ZTS: resilver a RAID-Z vdev expanded more than once
A block born before a RAID-Z expansion keeps the logical width it was
written with, so after the vdev has been expanded more than once it
is laid out over more rows than a block of the vdev's width, each row
with its own parity columns, and spans more consecutive children than
its sectors plus one row of parity. No test resilvers such a block.
raidz_expand_010_pos writes 16K blocks on a four-wide RAIDZ1, a
five-wide RAIDZ2 and a six-wide RAIDZ3, expands each vdev twice, then
replaces each child in turn and requires the scrub that follows each
resilver to find nothing to repair.
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Closes #19395
zio: find large gang headers while the pool is loading
zio_gang_tree_assemble() and zio_checksum_error() size gang headers
with spa_feature_is_active(dynamic_gang_header). During import the
MOS is read before spa_ld_check_features() loads the feature
refcounts, so the feature looks inactive: a ganged MOS block with a
large (ashift-sized) header is read as a 512-byte one, fails its
checksum, and the fallback only tries large-to-small. Every txg the
import tries fails the same way, so a pool that ganged MOS blocks
after the feature went active can't be imported.
Add spa_dynamic_gang_header(), which while the pool is loading also
checks the label's features_for_read, where dynamic_gang_header is
listed when active, and use it for both decisions.
This makes gang_blocks/gang_blocks_dyn_header_import pass.
Reviewed-by: Paul Dagnelie <paul.dagnelie at klarasystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
[3 lines not shown]
ZTS: test importing a pool with large gang headers in its MOS
gang_blocks_dyn_header_pos checks that large gang headers are written,
but never re-imports the pool. Add a test that forces ganging on an
ashift=12 pool until dynamic_gang_header is active and MOS blocks have
been ganged, then exports and imports the pool and scrubs it.
Reviewed-by: Paul Dagnelie <paul.dagnelie at klarasystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Issue #19189
Closes #19314
Accept the dedup NOP of zfs rewrite -P in arc_write_done()
A physical rewrite (zfs rewrite -P) of a dedup block that the DDT still
holds unchanged becomes a NOP in zio_ddt_write(), which sets
ZIO_FLAG_NOPWRITE so that the block pointer and its accounting are
kept, as for a nopwrite. arc_write_done() asserts that a write with
that flag has zp_nopwrite set, but dmu_write_policy() turns nopwrite
off whenever dedup is on. When the header of the block is still in
the ARC, a debug build therefore panics in arc_write_done() with
"VERIFY(zio->io_prop.zp_nopwrite) failed", and the pool hangs: the
first zfs rewrite -P -r of a dedup=on dataset holding a file and a
copy of it did so on master. Builds without assertions are not
affected.
Accept the flag for a physical rewrite of a dedup block as well, and
rewrite a deduplicated file and its copy twice with -P in
zfs_rewrite_physical.
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
[2 lines not shown]
Translate initialize and TRIM ranges under the config lock
vdev_initialize_thread() and vdev_trim_thread() drop SCL_CONFIG
before they disable and load each metaslab, then walk its
ms_allocatable tree through vdev_xlate_walk() before they take the
lock again. vdev_xlate() climbs vdev_parent up to the top-level
vdev, so that walk reads the vdev tree without the lock protecting it.
spa_vdev_detach() takes SCL_ALL as writer, unlinks the leaf with
vdev_remove_child(), which clears its vdev_parent, and sets
vdev_detached. The leaf's initialize and TRIM threads are only
stopped afterwards, in spa_vdev_config_exit(), once the lock has been
dropped. A thread inside the walk then passes the NULL parent to
vdev_xlate(). ztest hit this as a SIGSEGV in vdev_xlate() called
from vdev_initialize_thread() while the same leaf was being detached.
Detaching the leaf's sibling, which frees the parent mirror, races
with the walk in the same way.
Take SCL_CONFIG as reader again once the metaslab is disabled, and
[9 lines not shown]
ddt: prune only entries unchanged since the prune walk
The prune walk reads candidates from the stored unique class, which
lags the dedup log. The prune's sync task then skips a candidate only
if it is live or was loaded from the log. If the log is flushed between
the walk and the sync task, a candidate that has gained a reference is
found in the stored duplicate class and cleared anyway. Freeing either
reference afterwards frees the block while the other still uses it,
and the space can be reallocated under it.
Prune a candidate only if its reloaded entry still matches what the
walk read. Releases since 2.3.0 also skip a candidate found in the log
during the walk, but that lookup is unsynchronized and separate from
the read of the stored entry, so a flush between the two still lets the
candidate through: the lookup narrows the race without closing it.
Since fd1ae59c7 (#18889) removed the lookup, which was unsafe outside
txg sync, an entry already in the log when the walk reads it is pruned
as well.
[2 lines not shown]
ZTS: read zpool_get_006_pos class properties together
zpool_get_006_pos checks that the space properties of each allocation
class agree with each other, for example that used is not larger than
allocated. It read each property with a separate zpool get.
Three sections of the test write with sync=always and do not wait for
the pool to sync before the checks. The space counters only change when
a transaction group syncs, and this can happen between two of the zpool
get calls. The values then come from different points in time, and the
check fails. With kmemleak enabled, each zpool get takes longer, and the
test failed in about a third of the runs.
Read all properties of a class with a single zpool get. It returns all
values from the same ioctl, so they agree with each other.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alexander Moch <mail at alexmoch.com>
Closes #19401
ZTS: run zfs_get_009_pos under kmemleak without CPU mitigations
zfs_get_009_pos skips itself when kmemleak is enabled, because it runs
too slowly. Most of the slowdown comes from CPU mitigations: with
mitigations=off, it takes at most about 10 minutes under kmemleak.
Add has_mitigations, and skip the test only when both kmemleak and CPU
mitigations are enabled. Raise the timeout of the zfs_get tests to 900
seconds, so that slower runners can still finish it. Keep expecting a
SKIP in zts-report for systems that still skip it.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alexander Moch <mail at alexmoch.com>
Closes #19382
CI: Cancel outdated push runs on the same branch
Push events have no head_ref, so the run_id fallback puts every push
in a separate concurrency group. Use the pushed ref to group runs from
the same branch and let cancel-in-progress stop outdated runs.
Keep the existing grouping for PRs and independent manual runs.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: George Melikov <mail at gmelikov.ru>
Closes #19412
docs: Add fork CI and PR submission guidance for agents
Wait for successful fork CI on the final PR head before opening an
upstream PR so supported workflows can reuse its results. Limit new
upstream PRs to five per day unless agreed with maintainers in advance.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: George Melikov <mail at gmelikov.ru>
Closes #19412
CI: Extend fork result reuse and honor requested reruns
Extend QEMU result reuse to zloop, unit tests, ARM builds and
checkstyle. Use one helper with workflow file, runner, job name and
required step profiles. Require successful lint and ABI checks for
checkstyle, and exclude manual ARM runs with custom GCC inputs.
Reuse fork results only on the first workflow attempt. Requested reruns
execute in the base repository. Ignore cached Setup outputs on selective
QEMU matrix reruns and collect their new artifacts and summaries.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: George Melikov <mail at gmelikov.ru>
Closes #19412
zio: free the gang header ABD when its read fails
zio_gang_tree_assemble() wraps the gang header buffer in an ABD with
abd_get_from_buf(). zio_gang_tree_assemble_done() frees it, but only
when the read succeeded: on an error it returns first and the abd_t is
never freed. Each failed gang header read, whether from a checksum
or an I/O error, leaks one.
A pool whose MOS has large gang headers that can not be read while
the pool is loading (#19189) shows it. On a debug kernel, the failed
imports of such a pool left six abd_t objects behind and "rmmod zfs"
then panicked on
VERIFY3U(percpu_counter_sum(&skc->skc_linux_alloc), ==, 0). valgrind
on a failed "zdb -e" of the same kind of pool reports the abd_t
allocated by abd_get_from_buf() in zio_gang_tree_assemble() as
definitely lost.
Free the ABD on the error path as well. Nothing else owns it: the
gang tree only frees the header buffer, and the other gang callbacks
[5 lines not shown]
