Clone sparse files in time proportional to their data
zfs_clone_range_locked() walks the whole logical range in chunks of up
to zil_max_log_data() / sizeof (blkptr_t) blocks, one transaction per
chunk. A hole costs as much as data. Cloning an entirely sparse file
here took 22.7 ms per GiB, which for the 2^63-1 byte file that fstests
generic/303 reflinks comes to about six years. The test never finishes.
When the destination offset is at or past the destination's end of
file, a stretch that is a hole in the source needs no work. The
destination already reads it as zeros. Find the next data in the
source and step over the hole at once, stopping short of the range's
last block. The end of each data region is remembered, so a region
costs two lookups instead of one per chunk.
The lookup is a new dmu_offset_next_nowait(). dmu_offset_next() waits
for a dirty dnode to sync when zfs_dmu_offset_next_sync is set, and a
clone of a source written in the same txg would then sync it and clone
blocks that zfs_bclone_wait_dirty=0 says to copy.
[49 lines not shown]
Allow cloning a file's tail block past the destination's EOF
zfs_clone_range_locked() accepts a length that is not a whole number
of blocks only when the range reaches the end of the source and of the
destination. It tests the destination as
len < outzp->z_size - outoff
in unsigned arithmetic. When the clone starts past the destination's
end of file, which is the usual case for a new file, the subtraction
wraps to nearly 2^64 and the test always fails. FICLONERANGE then
returns EINVAL for a clone that would only have set the
destination's new end of file.
fstests generic/303 and generic/144 both hit this. 303 clones the
last 65535 bytes of a file to offset 1 MiB of a new file. 144 clones
file1 from 64 KiB to its end, 4 * 64 KiB + 37 bytes, to offset 64 KiB
of a new file. At 16 KiB recordsize, 144 fails on master and passes
with this change. At 128 KiB the 64 KiB offsets are not block aligned
[13 lines not shown]
Zero the last page past EOF when extending a file
POSIX requires that "the system shall never write out any modified
portions of the last page of an object which are beyond its end". On
ZFS, data stored through a shared mapping past the end of a file is
kept in the cached last page, and becomes part of the file as soon as
the file is extended over it, by ftruncate(), fallocate(), a write past
the end of the file, or cloning a block past the end of the file:
fd = open("f", O_RDWR | O_CREAT, 0644); write(fd, "0123456789", 10);
p = mmap(NULL, 4096, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
memcpy(p + 10, "STALE-DATA", 10); munmap(p, 4096);
ftruncate(fd, 64);
pread(fd, buf, 10, 10); /* "STALE-DATA", expected zeros */
zfs_putpage() and zfs_putpages() never write out the part of a page
past the end of the file, but nothing clears it either, so after the
file is extended, read() returns it from the cached page, and if the
page was still dirty (or is dirtied again) it is written to disk, where
[31 lines not shown]
ZTS: test mmap stores past EOF when extending a file
Add mmap_eof_extend and its helper of the same name. For each way of
extending a file (ftruncate(), a write past the end of the file,
fallocate() on Linux and cloning a block past the end of the file), the
helper stores data past the end of a file through a shared mapping of
its last page, extends the file, and checks that the bytes past the old
end of file read back as zeros, both with the page still dirty when
the file is extended and after it was written back. The test then
remounts the file system and checks the bytes on disk as well.
The test runs on Linux and FreeBSD. On FreeBSD the helper clones with
copy_file_range() instead of FICLONERANGE, and the test skips the
fallocate mode, as ZFS on FreeBSD does not support posix_fallocate().
POSIX requires that modifications of the partial page beyond the end
of an object are never written out, and ext4, xfs and btrfs return
zeros here. This currently fails on both platforms: every way of
extending the file exposes the stored data through read(), and if the
[9 lines not shown]
zdb: derive passphrase keys from the salt in little-endian
zdb -K derived the wrapping key from the PBKDF2 salt in host byte
order, while libzfs converts it to little-endian first. On big-endian
hosts zdb therefore derived the wrong key and couldn't load any
passphrase-wrapped key ("Permission denied"). Convert the salt the
same way libzfs does.
Found by running zdb -K in an s390x guest.
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Rob Norris <rob.norris at truenas.com>
Signed-off-by: Matthew Ahrens <matt at mahrens.org>
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
Closes #19510
libzfs: don't abort when a recv target's origin is outside the tree
recv_incremental_replication() compares the stream's origin of a file
system with the local one, and created_before() looks both up in the
local tree with fsavl_find(), passing the result straight to
fnvlist_lookup_string(). If the local origin was moved out of the
received tree, for example by promoting the original file system back
after the target had been cloned and promoted, the lookup returns NULL
and zfs receive aborts.
An origin outside the received tree can't have been promoted inside
it, so only check for a promote when both origins are in the tree, and
make created_before() report "same" for snapshots it can't find.
This makes cli_root/zfs_receive/zfs_receive_promote_outside pass.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Closes #17087
Closes #19308
ZTS: test zfs recv after the origin was promoted away
Add the sequence from #17087: replicate a file system, clone and
promote one of its snapshots on the receiving side, promote the
original back so the clone's origin lies outside the received tree,
then receive a new incremental replication stream into the clone.
The receive must succeed and create the new snapshot.
This fails now: zfs receive aborts in fnvlist_lookup_string() because
created_before() looks up the clone's origin in the received tree and
gets NULL.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Issue #17087
Closes #19308
dsl_crypt: only expect an IVset guid on snapshots
dsl_crypto_populate_key_nvlist() looks up DS_FIELD_IVSET_GUID on the
dataset being sent and asserts that a missing one means the pool has
the ZOL_8308 errata. Only snapshots get an IVset guid, though, and a
raw send of an unmounted encrypted file system, or zdb -B of its
objset, sends the head dataset, so debug builds panic (zdb aborts) on
'zfs send -w pool/fs'. Release builds send with to_ivset_guid 0.
Only assert for snapshots.
This makes cli_root/zfs_send/zfs_send_raw pass.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Closes #19018
Closes #19310
ZTS: test raw sends of an encrypted file system
zfs_send_raw only raw-sends snapshots of the encrypted dataset. Also
raw-send the unmounted file system itself, with zfs send and with
zdb -B, which takes a head dataset's objset id.
This fails now: a file system has no DS_FIELD_IVSET_GUID, which only
snapshots get, and dsl_crypto_populate_key_nvlist() asserts that its
absence means an errata pool, so debug builds panic (zdb aborts).
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Issue #19018
Closes #19310
libzfs: don't require keylocation when receiving an encryption root
recv_fix_encryption_hierarchy() restores the keylocation of a received
encryption root from the stream's properties with
fnvlist_lookup_string(), which aborts if it is missing. 9a02e53ad
handled streams with no properties at all, but a 'zfs send -b' stream
carries only received properties, and keylocation is usually a local
property of a raw-received encryption root, so receiving such a stream
still aborts zfs receive after the data has been received.
Restore the keylocation only if the stream carries it. The raw
receive has already set a usable one.
This makes cli_root/zfs_receive/zfs_receive_raw_-b pass.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Closes #17011
Closes #19309
ZTS: test receiving a 'send -b -w -R' stream of an encryption root
Replicate an encrypted file system raw to a backup, then send the
backup with 'zfs send -b -w -R' and receive it into another dataset.
A '-b' stream carries only received properties, which do not include
keylocation for the raw-received encryption root. The receive must
succeed, and the restored dataset must be an encryption root whose key
loads with the original passphrase.
This fails now: zfs receive aborts in fnvlist_lookup_string() when
recv_fix_encryption_hierarchy() looks up keylocation.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Issue #17011
Closes #19309
Redo a Direct I/O write that fails verify from a private copy
On Linux an application can change an O_DIRECT buffer while the write
is in flight. A QEMU guest on cache=none does this when its
filesystem changes a page under writeback. The block's checksum then
no longer matches the data written, zfs_vdev_direct_write_verify
catches it, and the write fails with EIO. QEMU passes that to the
guest as a disk error or pauses the VM.
When a Direct I/O write fails its verify, write the same range again
from a private copy of the buffer, still as Direct I/O, so the
checksum covers the bytes written. Then mark the file handle so every
later Direct I/O write on it is copied first. The copy is the same
idea as iomap's IOMAP_DIO_BOUNCE. A handle whose writes never fail the
verify stays zero-copy. The ereport and the zpool status -d counter
still record each failure, and a copy that fails the verify returns
the error as before.
FreeBSD holds the user pages stable during a Direct I/O write and
[6 lines not shown]
ZTS: Redo a Direct I/O write that fails verify from a copy
dio_write_verify_bounce writes with O_DIRECT while another thread
changes the buffer, with zfs_vdev_direct_write_verify on and compression
off. A pass of 300 one-block writes must fail verify exactly once and
count exactly 301 Direct I/O writes, the retry included. A pass of four
32M writes must redo the first failing write whole, so the pool counts
exactly five writes' worth of Direct I/O blocks and 1 to 256 verify
failures. Every write must return its full length. The helper's threads
race, so a pass in which no write fails verify runs again from a new
handle, up to ten times, and such a try must not have redone anything.
On master the first write whose buffer changes in flight fails with EIO.
dio_write_verify no longer accepts EIO from its writes with the verify
on, since a failed verify is now redone instead of returned.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Michael Heller <75820586+mkhllr at users.noreply.github.com>
Signed-off-by: Patrick Domack <patrickdk at patrickdk.com>
Closes #19345
zstream: flush buffered raw writes before frees and resizes
zstream raw coalesces contiguous WRITE records in a buffer that is only
issued when a non-contiguous write arrives, the buffer fills, or the
stream ends. FREE records and volume size changes from the properties
ZAP were applied to the image immediately, however, so a write still
sitting in the buffer could land after a later free or truncation of the
same range. In a stream package where one snapshot writes a block and
the next frees it, the image ends up with stale data where the zvol has
zeros:
zfs create -V 64M -o volblocksize=16k -o compression=off tp/v
zfs snap tp/v at a
dd if=/dev/urandom of=/dev/zvol/tp/v bs=16k count=1 oflag=direct
zfs snap tp/v at b
blkdiscard -o 0 -l 16384 /dev/zvol/tp/v; sync; zfs snap tp/v at c
g=$(zfs send tp/v at a | zstream raw /var/tmp/img)
zfs send -I @a tp/v at c | zstream raw -g $g /var/tmp/img
cmp -n 16384 /var/tmp/img /dev/zero # differs
[14 lines not shown]
ZTS: test zstream raw write ordering with frees and resizes
Add zstream_raw_002_pos, which applies stream packages where one
snapshot writes a block and the next frees it, or shrinks the volume
past it, to an image file and to a block device, and checks that the
result matches the zvol.
This currently fails: zstream raw applies the FREE, or the resize,
before the buffered write of the same range, so the image keeps stale
data where the zvol has zeros, or is extended back out. The fix follows
in the next commit.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Garth Snyder <garth at garthsnyder.com>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Closes #19284
cstyle: fix cstyle macro detection, and clean resulting issues
The macro detection in cstyle was allowing for lowercase letters,
which caused continuation line and brace placement detection
to be missed.
These changes provide a common macro definition for both places
that macros are scanned, and then cleans up the cstyle issues
that were uncovered as a result of the fix.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Mark Musante <mmusante at wasabi.com>
Closes #19500
CI: run only the dependency part of qemu-3-deps-vm.sh in zfs-arm
zfs-arm runs qemu-3-deps-vm.sh on the ARM runner itself, without
ONLY_DEPS. Its Ubuntu branch therefore still installs
linux-modules-extra-$(uname -r), and ARM runners on image 20261018.154.1
boot 7.0.0-1017-azure, which has no such package. apt-get fails and the
job fails in "Install dependencies".
zfs-arm needs only the dependencies. After them the script prepares a
test VM: it enables NFS, Samba and the guest agent, rewrites the GRUB
command line and resets cloud-init. zfs-arm then only builds, loads the
zfs.ko it built and creates a pool on a file. Set ONLY_DEPS=1, as the
other workflows that run the script on the runner do.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Alexander Moch <mail at alexmoch.com>
Closes #19502
Linux: don't dereference NULL z_ctldir in zfs_vget() for snapshots
A snapshot's zfsvfs never gets a .zfs control directory (zfsctl_create()
is skipped for z_issnap), but z_show_ctldir keeps its default value
because property callbacks are not registered for snapshots. A short
file handle for ZFSCTL_INO_ROOT or ZFSCTL_INO_SNAPDIR with a zero
generation, presented on a mounted snapshot, therefore sets *ipp to
NULL in zfs_vget(), which a debug build catches with
"VERIFY(*ipp != NULL) failed".
Return ENOENT when there is no control directory, the same as when
snapdir=disabled.
This makes snapdir_fh_ctldir, added in the previous commit, pass.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Closes #19290
ZTS: test .zfs file handles on a mounted snapshot
Add an open_by_fid test helper, which opens a ZFS file handle built
from an object number and generation with open_by_handle_at(2), and
the snapdir_fh_ctldir test. It checks that handles for .zfs and
.zfs/snapshot resolve on a filesystem, and are rejected on a mounted
snapshot, which has no .zfs directory of its own.
This currently panics a debug kernel with "VERIFY(*ipp != NULL)
failed" in zfs_vget(), which sets *ipp to the snapshot's NULL
z_ctldir. The fix follows in the next commit.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Closes #19290
zstream: don't corrupt uncompressed records in decompress
When `zstream decompress` resolves a record's compression type to
"off", either because the user asked for it explicitly or because no
type was given and the record's own drr_compressiontype is 0, it
relabels the record as uncompressed by setting drr_logical_size to
drr_compressed_size and clearing drr_compressed_size. That is the
intended behavior of "off" for a record that is compressed in the
stream: the payload is kept as-is and the header is changed to
describe it as raw data of that size.
For a record that is not compressed in the stream, however,
drr_compressed_size is 0, so drr_logical_size becomes 0 while the
logical-size payload is still emitted. The output stream is then
misframed: `zstream dump` fails with "invalid record type" and
`zfs receive` rejects it with "DRR_WRITE logical size 0 below minimum
512". This is easy to hit, e.g.:
zfs create -o compression=off pool/fs
[21 lines not shown]
ZTS: test zstream decompress of uncompressed records
Add zstream_decompress_007_pos, which runs zstream decompress, without
a compression type and with "off", on WRITE records that are not
compressed in the stream, and checks that the output is byte-identical
to the input, can be parsed by zstream dump, and receives to an
identical file.
This currently fails: zstream decompress sets drr_logical_size to 0
for these records while still emitting their payload, so zstream dump
fails with "invalid record type" and zfs receive rejects the stream.
The fix follows in the next commit.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Closes #19289
Linux: revalidate unmounted .zfs/snapshot entries against the name
When a .zfs/snapshot/<name> entry has been looked up but not mounted,
destroying or renaming the snapshot leaves the dentry in the dcache.
zfsctl_snapshot_unmount() only invalidates entries with something
mounted on them, and zpl_snapdir_revalidate() accepted any positive
dentry. The old name then keeps resolving to the old inode:
zfs snapshot tp/fs at a
stat /tp/fs/.zfs/snapshot/a # look up, no automount
zfs destroy tp/fs at a
stat /tp/fs/.zfs/snapshot/a # succeeds, expected ENOENT
ls /tp/fs/.zfs/snapshot/a/ # ESTALE, expected ENOENT
and a snapshot later created or renamed to that name is reported with
the old snapshot's inode number, birth time and fsid until the dentry
happens to be evicted.
Make zpl_snapdir_revalidate() check, for positive dentries with nothing
[13 lines not shown]
ZTS: test stale .zfs/snapshot entries after destroy and rename
Add snapdir_lookup_stale, which looks up a .zfs/snapshot/<name> entry
without mounting it and then destroys, recreates and renames the
snapshot, and renames a different snapshot onto the cached name. It
checks that the old name returns ENOENT and that a snapshot which takes
over the name is reported with its own inode number.
This currently fails: after the destroy, stat of the old name still
succeeds and listing it fails with ESTALE, and a new snapshot with that
name is reported with the old snapshot's inode number. The fix follows
in the next commit.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Closes #19285
zvol: remove minors in parallel
zvol_remove_minors_impl() shuts down its zvols one at a time, and on
Linux each del_gendisk() waits for an RCU grace period, so exporting a
pool with 1,000 zvols spends about 30 s in this loop.
Move the loop body unchanged into zvol_shutdown_minor(). When there is
more than one zvol, run it for each on a taskq created for the call, as
vdev_open_children() does, with up to 32 threads, and wait for all of
them before freeing the zvols, so the removal stays one synchronous
operation as #17625 made it.
On a 12-CPU VM with Linux 6.12, exporting a pool with 1,000 zvols took
28.7-30.2 s before and 2.5-3.3 s after. FreeBSD doesn't wait for a
grace period per device, and its export times are unchanged.
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Rob Norris <rob.norris at truenas.com>
Signed-off-by: Ameer Hamza <ameer.hamza at truenas.com>
Closes #19430
Linux: report correct d_ino for .zfs/snapshot entries
zpl_snapdir_iterate() has emitted each snapshot with an inode number
of ZFSCTL_INO_SHARES - objsetid since the .zfs control directory was
first added. Every other path which names a snapshot directory, the
lookup (zfsctl_snapdir_lookup), getattr of the unmounted placeholder
and of the automounted snapshot root (zfs_getattr_fast), the file
handle code (zfsctl_fid, zfs_vget), and statfs, uses
ZFSCTL_INO_SNAPDIRS - objsetid. As a result the d_ino returned by
readdir(3) for .zfs/snapshot/<name> is off by two from the st_ino
reported by stat(2) for the same entry, and it can even collide with
the inode number of a different snapshot's directory.
Tools which correlate readdir inode numbers with stat results, such as
find(1), du(1) and rsync(1) hard link and cycle detection, or programs
that use d_ino to avoid a stat per entry (e.g. Python's os.scandir()),
see inconsistent inode numbers. It also breaks deriving the snapshot
objsetid from the directory entry, which is now otherwise possible
without triggering an automount via an O_PATH open and statx(2).
[11 lines not shown]
ZTS: test readdir inode numbers in .zfs
Add snapdir_readdir_ino, which compares the readdir d_ino of every
entry with its statx st_ino for the dataset root, .zfs and
.zfs/snapshot, before and after a snapshot is automounted, and checks
that snapshot directory inode numbers are derived from the objsetid.
This currently fails: the d_ino of each .zfs/snapshot/<name> entry is
off by two from the inode number that stat reports for it. The fix
follows in the next commit.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Will Rouesnel <wrouesnel at wrouesnel.com>
Closes #19286
libzpool: fix the lseek() arguments in zfs_file_off()
zfs_file_off() returns the current offset of a file and does not move
it: the Linux and FreeBSD kernel versions return f_pos and f_offset.
The libzpool version calls lseek(fd, SEEK_CUR, 0), with the offset and
whence swapped. As SEEK_CUR is 1 and SEEK_SET is 0, that seeks to
offset 1, so it returns 1 whatever the offset was and leaves the file
there.
No caller reaches it in userspace today: its callers are in
zfs_ioctl.c, which libzpool does not build. A program linked against
libzpool that calls it, or a future caller in code that libzpool does
build, would read or write at the wrong place without any error.
Swap the arguments, and add a unit test that checks the offset after
open, write and seek, and that reading it does not move the file.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Closes #19451
libzpool: return errno from zfs_file_unlink() and zfs_file_deallocate()
The kernel versions of zfs_file_unlink() and zfs_file_deallocate()
return an errno value on failure, as the rest of the libzpool
zfs_file interface does. The libzpool versions return -1 instead:
zfs_file_unlink() returns what remove() returns, and
zfs_file_deallocate() what fallocate() or fspacectl() returns.
spa_config_write() compares the result of zfs_file_unlink() with
ENOENT, so in userspace, removing a cachefile that is already gone is
reported as a failed cachefile write. A failed TRIM of a file vdev
completes with an io_error of -1.
Return errno from both, and test them.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Closes #19451
unit/zfs_file: cover the other zfs_file I/O calls
Check that zfs_file_pwrite() and zfs_file_pread() leave the file
offset where it was and that zfs_file_read() moves it past what it
read, along with the data each of them moves. Also cover short reads
at the end of the file, with and without resid, the errors of
zfs_file_seek(), and zfs_file_open() creating a file with the mode it
was given, as zfs_file_getattr() reports it.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Closes #19451
libzpool: restore the umask when zfs_file_open() fails
With O_CREAT, zfs_file_open() clears the umask so that the file gets
the mode it was given, and restores it after open64(). When open64()
fails, it returns before restoring it, which leaves the umask of the
whole process at 0: every file the program creates afterwards ignores
its umask.
Restore it before checking the result of open64(), and test it.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tigerblue77 <37409593+tigerblue77 at users.noreply.github.com>
Closes #19451
