zio_ddt_write: compute have_dvas after taking dde_io_lock
In zio_ddt_write(), have_dvas and is_ganged were computed before
dde_io_lock was taken. A concurrent zio_ddt_child_write_done() error
path calls ddt_phys_unextend() under dde_io_lock, which can zero
DVA[0] while another thread is between computing have_dvas and taking
dde_io_lock. That thread then uses the stale have_dvas=1 to call
ddt_bp_fill(), copying the zeroed DVA into the BP. A zero DVA resolves
as a hole, producing blocks that read back as zeros with no checksum
error (silent data corruption).
Fix by moving have_dvas and is_ganged computation to after dde_io_lock
is taken, so they always reflect the current state of dde->dde_phys.
Regression introduced by a41ef36858 ("DDT: Reduce global DDT lock
scope during writes").
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
[3 lines not shown]
zap: remove refcount tags from backend functions
Since we now never need to unlock/lock an existing zap_t, we don't need
to thread through the refcount tag everywhere, which lets us simplify a
lot of calls.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18546
zap: lift and simplify zap_t lock upgrade
Most fatzap write ops only take the READER zap_t lock, because the
header block only needs to be updated when a change would add or remove
a leaf block or spill the ptrtbl. When this happens, the lock is
upgraded to WRITER so those changes can be made.
If the lock can't be upgraded directly (not least because
rw_tryupgrade() is a no-op on Linux and userspace), then it has to be
dropped and re-acquired, that is, zap_unlock() and then zap_lock().
However, this method is far heavier than it needs to be, and adds
complication because it fully releases the zap_t, the header dbuf and
the dnode. This gives a window where the dbuf can be evicted and so the
zap_t destroyed. In addition to the IO overhead if this happens, this
means the zap_t returned by zap_lock() may be different to the original,
which means all callers need to be prepared for it to change.
zap_shrink() used an alternate method of simply dropping and reacquiring
[18 lines not shown]
mzap_create_impl: use zap_lock_by_dnode()
The only reason this used zap_lock_impl() directly was to avoid an extra
dbuf hold, but there's no real reason to do that. Just use
zap_lock_by_dnode(), and then zap_lock_impl() can be de-exported.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18546
zap_lock: make it be a simple wrapper around zap_lock_by_dnode()
The only real difference between zap_lock() and zap_lock_by_dnode() is
that the former takes and releases its own dnode hold. If we make it
just delegate to zap_lock_by_dnode(), then the dbuf hold and release can
be handled there, in one place.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18546
zap: rename 'lockdir' to 'lock'
The "dir" part is a holdover from prehistoric times, where ZAPs were
just the filesystem directory object.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18546
CI: Allow testing with a newer GCC on ARM builder
Add a text box to specify a custom GCC version (like '16') when
running the zfs-arm builder. This allows you to test with a newer
GCC than the Ubuntu default.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tony Hutter <hutter2 at llnl.gov>
Closes #18540
zstream: init/fini refcount tracking
When compiled with ZFS_DEBUG and reference_tracking_enable is enabled,
ABD alloc/free will have real refcount tracking, which will crash if the
reference cache hasn't been initialised. Adding it to the init & fini
lists is the quickest way to get that going again.
Sponsored-by: TrueNAS
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18535
zstream: dump backtrace on crash
Same method as zdb and ztest. zstream doesn't get touched much, and
plays a bit fast-and-loose with some core code. Its not hard for a
change to make it crash; this makes debugging easier when it does.
Sponsored-by: TrueNAS
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18535
CI: Remove deprecated Fedora 42
Fedora 42 was deprecated on May 13 2026. Remove it from CI tests.
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #18545
Fix aarch64 build failure by removing earlyclobber (#18532)
The UVR macros used "+&w" (read-write + earlyclobber) as the
constraint for NEON register operands that are declared as explicit
hard-register variables via:
register unsigned char wN asm("vN") __attribute__((vector_size(16)));
The + modifier implicitly makes the operand also an input (reading the
register before the asm runs). The & (earlyclobber) modifier says "this
output may be written before all inputs are consumed." Having an
earlyclobber output on the same hard-register that is simultaneously
an input is a contradiction — GCC 16 now strictly diagnoses this.
The fix removes the & from "+&w", yielding "+w". The earlyclobber
was both incorrect (contradicts the implicit input) and unnecessary
(the physical registers are already hard-bound, so the compiler has no
freedom to assign conflicting registers anyway).
[6 lines not shown]
dsl_bookmark: fix redaction list refcount tag when upgrading spill
rl_bonus and rl_dbuf are expected to have the same hold tag if they are
different. If the spill hold is taken after the redaction_list_t was
created and the bonus hold was taken, it must also be taken with the
same tag. Fortunately, we have it right here, so we can just use it.
Sponsored-by: TrueNAS
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18536
ddt_log: fix refcount tag between ddt_log_begin & ddt_log_commit
We have to hold and release the dbuf array with the same tag. Since the
caller provides the ddt_log_update_t and is managing its lifetime, and
the begin/commit calls must be matched, it's quite reasonable to its
pointer as the refcount tag.
Sponsored-by: TrueNAS
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18536
zap: fix refcount tag use in zap_lookup_length_uint64 and zap_prefetch_uint64
The same tag must be used for zap_lockdir() and zap_unlockdir(), so we have
to follow the pattern used elsewhere: pass the tag used for
zap_lockdir() through to the _impl(), so it can use it for
zap_unlockdir().
Sponsored-by: TrueNAS
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18536
linux: suppress reclaim lockdep in zfs_inactive via rwlock wrappers
kswapd can enter zfs_inactive() from inode reclaim while holding
fs_reclaim. The z_teardown_inactive_lock still serializes teardown,
but the reclaim-thread acquire/release pair can produce a lockdep
cycle through zfs_zinactive() and zfs_rmnode().
Add Linux rwlock nolockdep wrappers alongside the existing rwlock
macros and use them only for the reclaim-thread
z_teardown_inactive_lock acquire/release in zfs_inactive(). Keep
the real rwsem semantics unchanged and leave CONFIG_LOCKDEP
handling in the platform rwlock layer.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: ZhengYuan Huang <gality369 at gmail.com>
Closes #18505
CI: Fix 99.99 META version
We have an option in zfs-qemu-packages to test against a specific kernel
version. However, qemu-3-deps.sh was incorrectly hard coded to look
at $2 for a kernel version argument (which could come in $2 or $3
depending on if --poweroff was also passed). This caused the CI
to incorrectly edit META with a max supported kernel version of 99.99
when we didn't want that.
Fix this by looking at all the arguments for something that looks
like a kernel version and set that as the kernel max in META.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tony Hutter <hutter2 at llnl.gov>
Closes #18526
Closes #18531
Remove arc_bcopy_func() function
While this function could be convenient it appears it's never been
used. In practice, callers end up using the arc_getbuf_func()
instead. Remove this unused function.
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #18534
arc: export additional required symbols
External consumers of arc_read() need to be able to destroy the
returned arc_buf_t. Add the arc_buf_destroy() interface as an
exported symbol.
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #18533
ZTS: zhack_metaslab_leak.ksh busy export
If the pool is active 'zpool export' will fail resulting in
a test failure. Swap log_must with log_must_busy so the export
is retried when reported as busy before failing the test.
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #18512
Integrate DDT and BRT tests
Don't disable block cloning during dedup tests. Just don't use
cp to not trigger it. Add a new test, explicitly mixing dedup
and cloning on the same file, that should be handled by DDT.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Rob Norris <rob.norris at truenas.com>
Signed-off-by: Alexander Motin <alexander.motin at TrueNAS.com>
Closes #18520
Fix double free for blocks cloned after DDT prune
Before this change, for blocks marked with D flag but absent in DDT
(pruned from it), zio_ddt_free() fell back to ZIO_STAGE_DVA_FREE
without trying ZIO_STAGE_BRT_FREE first. Same time such blocks
might be present in BRT, and not handling that would result in
double/multiple free.
This change makes ZIO_DDT_FREE_PIPELINE include ZIO_FREE_PIPELINE,
just adding required ZIO_STAGE_ISSUE_ASYNC and ZIO_STAGE_DDT_FREE,
and moves DDT stages before BRT. This way, if the block is found
in DDT by zio_ddt_free(), the pipeline is short-circuited to
ZIO_INTERLOCK_PIPELINE, similar to what zio_brt_free() does. If
not, then BRT is checked, and if also no match, the block is freed.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Rob Norris <rob.norris at truenas.com>
Signed-off-by: Alexander Motin <alexander.motin at TrueNAS.com>
Closes #18520
linux/zpl_super: handle 'source' option directly
vfs_parse_fs_param_source() didn't appear until 5.14, and was not
backported to kernel.org LTS kernels. It's simple enough that it's
easier to just handle it ourselves rather than use a configure check.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18529
Linux: avoid znode list lock inversion during resume
Lockdep reports a circular locking dependency during mounted filesystem
rollback. zfs_resume_fs() walks z_all_znodes under z_znodes_lock and
calls zfs_rezget(), which takes the per-object znode hold lock via
zfs_znode_hold_enter().
The normal zget path takes these locks in the opposite order.
zfs_zget() takes the per-object hold lock before zfs_znode_alloc()
inserts the znode on z_all_znodes under z_znodes_lock. Resume can
therefore establish z_znodes_lock -> zh_lock while normal lookup
creates zh_lock -> z_znodes_lock.
Pin the current and next znodes with igrab() while holding the list
lock, then drop the list lock before reloading the znode. Existing
stale inode handling is preserved, and both the suspended reference
and temporary walk reference are released asynchronously.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: ZhengYuan Huang <gality369 at gmail.com>
Closes #18517
Linux: expose zfs_arc_no_grow_shift as a module parameter
The zfs_arc_no_grow_shift variable is tunable via sysctl on FreeBSD
but had no module parameter registration on Linux.
Register it once in arc.c using param_get_uint and a per-platform
set handler, replacing the FreeBSD-only registration.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alek Pinchuk <alek.pinchuk at connectwise.com>
Signed-off-by: Christos Longros <chris.longros at gmail.com>
Closes #18461
CI: FreeBSD 15.1 STABLE
Update the freebsd15-1s builder to the released STABLE image.
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #18524
libzfs_pool: document export and initialize functions
Add brief docstrings to zpool_export(), zpool_export_force(),
zpool_initialize() and zpool_initialize_wait().
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Christos Longros <chris.longros at gmail.com>
Closes #18514
Consistently encode DRR_BEGIN packed nvlist payloads with NV_ENCODE_XDR
Currently, zfs send generates a mix of nvlist encodings in DRR_BEGIN
records, some XDR and some in native byte order. The result is that
most streams currently can't be zfs received on opposite-endian systems.
zfs send generates the outer wrappers for compound streams in userspace,
and it explicitly requests NV_ENCODE_XDR format for those records. But
the BEGIN records for individual datasets are generated on the kernel
side, in dmu_send.c, where fnvlist_pack() is used for encoding. That
routine hard-wires NV_ENCODE_NATIVE format.
This PR replaces the fnvlist_pack() call with a direct call to
nvlist_pack() that specifies NV_ENCODE_XDR.
Tests are included to verify that native-encoded nvlists are not
generated by any kernel path that attaches nvlists to BEGIN records.
There's also a check for XDR encoding in the outer wrapper of
replication streams in case there is ever a regression there.
[15 lines not shown]
zap: internal interface cleanup
Similar to previous, though a much lighter touch because these are not
"public" interfaces.
- reorganising functions into groups, by rough function class.
- matching header order to source order, to make it a little easier to
find things.
- adding light documentation to functions that had none.
Note that I've not added any documentation for the mzap_* and fzap_*
functions, as part of this commit series is laying the groundwork to
hide those functions in their backend modules; such documentation would
become obsolete very quickly.
No actual code changes.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
[4 lines not shown]
zap: public interface cleanup
- reorganising functions into groups, collections of the variants of the
same function.
- matching header order to source order, to make it a little easier to
find things.
- moving per-function documentation from source to header.
- adding light documentation to functions that had none.
No actual code changes.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Akash B <akash-b at hpe.com>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18516
