CI: Fix race caused by shared ctr file updates
CTR is shared between the VMs and used as a global counter. This
uncoordinated shared access can result in a CI failure due to the
racing updates. From the log:
`qemu-6-tests.sh: line 27: 1`
`6: syntax error in expression
(error token is "6")`
Resolve the issue by using separate ctr files by appending the ID.
The output now prints the total test cases count along with each VMs
individual count.
Reviewed-by: Tino Reichardt <milky-zfs at mcmilk.de>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: tiehexue <tiehexue at hotmail.com>
Closes #18778
config: remove HAVE_IDMAP_MNT_API check
The check itself is not needed, since we only use it to decide whether
or not to use a define, we can just test the define anyway.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
config/acl: narrow checks to userns inode->dentry change
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
config/mkdir: narrow checks to dentry return
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
config/tmpfile: narrow checks to userns dentry->file change
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
config: remove obsolete idmap/userns checks
Everything has been converted to the new helpers built on the "global"
idmap/userns checks, so we can get rid of all the individual function
checks.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
xattr: use generic checks for ZPL_XATTR_SET_WRAPPER selection
We can't (yet) replace these wrappers but we can use the generic checks
as selector, obsoleting more configure checks.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
zpl_generic_permission: generic over idmap style
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
zpl_inode_owner_or_capable: generic over idmap style
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
zpl_setattr_prepare: generic over idmap style
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
zpl_generic_fillattr: generic over idmap style
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
zfs_getattr_fast: single version
The caller always has an idmap of some sort, so we can just call it directly.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
zfs_setattr_idmap: always use idmap version
We added zfs_setattr_idmap in dee19fae05; just use it for all cases now
that we always have an idmap available.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
zpl_inode: use ZPL_IDMAP_IOP_DEFINE for getattr/setattr callbacks
ZPL_GETATTR_WRAPPER and ZPL_SETATTR_WRAPPER were earlier concept of
these defines that didn't actually do enough. Since these are just
regular inode_operations callbacks, we can use ZPL_IDMAP_IOP_DEFINE for
those too and remove the old wrappers.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
zpl_inode: use ZPL_IDMAP_IOP_DEFINE for iops callbacks
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
idmap_compat: macros for handling idmap types in iops callbacks
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
spl/cred: move remaining idmap compat, use generic checks
Move the remaining idmap/cred checks from sys/types.h to sys/cred.h, the
only direct user, and change them to use the generic macros.
These should be folded back into idmap_compat at some point but they
need careful auditing first.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
linux/zpl: standardise on 'idmap' nomenclature
Wherever we use zidmap_t, name it idmap, since that is its purpose and
we have no idea whether or not there's a user_namespace or a mnt_idmap
under there.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
idmap_compat: types and defines for idmap/userns compat
Setting up the header where we'll do all our idmap compat stuff. To
start, lift zidmap_t and zfs_init_idmap out from sys/types.h, and switch
zidmap_t type switch to use the generic checks.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
config: detect idmap method via generic_permission test
Since the switch from implicit to explicit userns, and to idmap,
happened right across the kernel in major releases, so it is enough to
use a single test and apply the results everywhere.
generic_permission() is a nice simple function with a simple interface,
so useful for an unambiguous test.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
zpl_inode: move zpl_set_acl/zpl_get_acl from zpl_xattr
These are inode operations functions, better here than there. This will
make it easier to clean them up with the rest, and keeps the actual
implementation functions tidier.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
config: remove HAVE_SETATTR_PREPARE_NO_USERNS
The change it checks for was introduced in 4.9
(torvalds/linux at 31051c85b5e2a), well beyond the versions we support.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
config: remove HAVE_PATH_IOPS_GETATTR
The change its checking for was introduced in 4.11
(torvalds/linux at a528d35e8bfcc). This syncs up with the
HAVE_*_IOPS_SETATTR checks.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
zpl_inode: remove zpl_rename no-flags variants
Removed in 4.9.
Sponsored-by: TrueNAS
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18769
ZTS: stop zpool_initialize tests racing initialize to completion
zpool_initialize_import_export and zpool_initialize_suspend_resume start
initializing a one-disk pool, wait a fixed couple of seconds, and then
expect initializing to still be running so it can be observed across an
export/import and suspended. On a small or fast vdev the default 1 MiB
initialize chunk lets the whole disk finish within that window, after
which "zpool initialize -s" fails with "there is no active
initialization" and the test fails.
Throttle initializing with zfs_initialize_chunk_size, exactly as the
zpool_wait_initialize_* tests already do, so it stays active long enough
to observe regardless of vdev size or speed. The tunable is saved and
restored per test. Cleanup destroys the pool before restoring the chunk
size: the initialize thread rereads zfs_initialize_chunk_size on every
write but allocates its fill buffer once at the smaller size, so raising
it back while the thread is still running would issue a write larger
than that buffer.
[13 lines not shown]
ZTS: migration/setup: clear stale zfs_member label before new_fs
During a full ZTS run functional/migration/setup fails intermittently
when it mounts the non-ZFS device. That device is often one an earlier
test used as a pool vdev. 'zpool destroy' leaves the vdev labels in
place and new_fs only overwrites the front of the device, so the
trailing labels can survive. libblkid then probes the device as
ambiguous (both the new filesystem and zfs_member) and the
auto-detecting mount refuses, which setup reports as a spurious failure.
Wipe any residual signatures with wipefs before laying down the new
filesystem so the device carries a single, unambiguous type, and let
udev settle before the mount. Skip the wipe in the single-disk case,
where the non-ZFS device is the same one the test pool was just created
on, so the live pool is left untouched.
Verified on Linux: after a pool create and destroy the scratch device
still carries a zfs_member label (blkid -p reports zfs_member); a
wipefs -a removes it so the following new_fs is the only signature and
[6 lines not shown]
zfs bookmark: add recursive (-r) bookmark creation
"zfs snapshot -r" and "zfs destroy -r" already operate recursively, but
there was no way to create bookmarks for a recursive snapshot in one
step, so users had to script a loop over every dataset.
Add a -r flag to "zfs bookmark" that, given a snapshot source, creates
a bookmark of that snapshot on every descendant dataset that has it.
Descendants that lack the snapshot (for example one created after the
recursive snapshot) or that are still inconsistent are skipped instead
of failing the whole request, mirroring how "zfs snapshot -r" gathers
its targets. The pairs are submitted to lzc_bookmark() in a single
call and any per-dataset failures are reported individually. The -r
flag requires a snapshot source; a bookmark source is rejected.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: MorganaFuture <103630661+MorganaFuture at users.noreply.github.com>
Closes #9460
Closes #18763
zstream: open up dump API to non-dump subcommands
The upcoming `zstream raw` would like to dump records as part of its
operation when a `-v` flag is passed.
This PR adds `zstream_dump.h` and publicly declares the
`serial_dump_records()` chain module for access by other parts of
zstream. It also makes a couple of other changes to make the dump
module more friendly toward clients.
- The record-type table no longer has to be passed into
`zstream_dump_records()` as a context struct. It's now a file static.
- `zstream dump` has command-line options for "verbose", "very verbose",
and "dump data", which each enable one or more functions. This patch
replaces the generic CA_VERBOSE and CA_VERY_VERBOSE internal flags
with options that request specific behaviors:
```
[17 lines not shown]
implement thorough scrub support (zpool scrub -t)
This introduces the -t (thorough) flag to 'zpool scrub' command.
A thorough scrub decrypts and decompresses blocks as they are read,
allowing ZFS to catch rare corruption scenarios where the block's
checksum matches the data on disk, but the block fails to decrypt
or decompress.
For encrypted datasets, the keys must be loaded to perform a thorough
scrub. If the keys are not loaded, or are unloaded while the scrub is
in progress, the scrub will fall back to a normal scrub
for those encrypted blocks with key unloaded.
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Signed-off-by: Alek Pinchuk <Alek.Pinchuk at connectwise.com>
Closes #18474
mailmap/AUTHORS: update with recent new contributors
"When you rest, you are a king surveying your estate. Look at the
woodland, the peacocks on the lawn. Be the king of your own calm
kingdom."
Sponsored-by: TrueNAS
Reviewed-by: Alexander Motin <alexander.motin at TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Rob Norris <rob.norris at truenas.com>
Closes #18775
