Displaying 1 50 of 5,158 commits (0.009s)

ZFS on Linux — tests/zfs-tests/include commands.cfg, tests/zfs-tests/tests/perf/regression random_reads.ksh sequential_reads_arc_cached.ksh

Profiling for perf tests

Stack profiling is quite useful and Linux ZFS test suite does not
current collect that data.

Linux perf is a common tool for this purpose though the perf record
data file can be quite large. With this change, Linux ZFS perf tests
capture perf record data if perf is installed on the system and
PERF_DO_PROFILING environment variable is set.

Reviewed by: John Wren Kennedy <jwk404 at gmail.com>
Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Signed-off-by: Tony Nguyen <tony.nguyen at delphix.com>
External-issue: LX-971
Closes #7549

ZFS on Linux — . Makefile.am

Create "bin" directory so that zloop.sh works

Before running zloop.sh, we need to run `scripts/zfs-tests.sh -c` to
create and populate the `bin` directory with symlinks to our utilities.
Rather than making developers remember to do this, `make` should do it
for them.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Matthew Ahrens <mahrens at delphix.com>
Closes #7525
Closes #7547
Delta File
+3 -0 Makefile.am
+3 -0 1 file

ZFS on Linux — .github ISSUE_TEMPLATE.md PULL_REQUEST_TEMPLATE.md

Small cleanup of PR and issue templates

- Add links to PULL_REQUEST_TEMPLATE.md
- Clean `System information` table

It's easier to find needes documentation about
PR process with links.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <guss80 at gmail.com>
Signed-off-by: George Melikov <mail at gmelikov.ru>
Closes #7539

ZFS on Linux — tests README.md

Update `tests/README.md` and fix markdown

- there are more options now
- command examples are more readable in code style

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <guss80 at gmail.com>
Signed-off-by: George Melikov <mail at gmelikov.ru>
Closes #7538
Delta File
+47 -28 tests/README.md
+47 -28 1 file

ZFS on Linux — tests/zfs-tests/tests/functional/cli_root/zpool_scrub zpool_scrub_004_pos.ksh

ZTS: Improve zpool_scrub_004_pos reliability

It's possible for the `zpool attach` portion of this test case
to complete before the `zpool scrub` can be issued.  Update the
test case to force the resilvering phase to take longer.

Reviewed-by: Tim Chase <tim at chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #5444
Closes #7541

ZFS on Linux — tests/zfs-tests/tests/functional/tmpfile setup.ksh tmpfile_test.c

ZTS: Update O_TMPFILE support check

In CentOS 7.5 the kernel provided a compatibility wrapper to support
O_TMPFILE.  This results in the test setup script correctly detecting
kernel support.  But the ZFS module was built without O_TMPFILE
support due to the non-standard CentOS kernel interface.

Handle this case by updating the setup check to fail either when
the kernel or the ZFS module fail to provide support.  The reason
will be clearly logged in the test results.

Reviewed-by: Chunwei Chen <tuxoko at gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7528

ZFS on Linux — . .gitignore, man/man8 .gitignore

Ignore *.o.ur-safe build artifacts

Generated when building on Ubuntu 18.04.  Also ignore the new
dynamically generated zfs-mount-generator.8 man page, and the
module/.cache.mk file.

Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7534

ZFS on Linux — module/zfs vdev_disk.c mmp.c

module param callbacks check for initialized spa

Callbacks provided for module parameters are executed both
after the module is loaded, when a user alters it via sysfs, e.g
        echo bar > /sys/modules/zfs/parameters/foo

as well as when the module is loaded with an argument, e.g.
        modprobe zfs foo=bar

In the latter case, the init functions likely have not run yet,
including spa_init() which initializes the namespace lock so it is safe
to use.

Instead of immediately taking the namespace lock and attemping to
iterate over initialized spa structures, check whether spa_mode_global
is nonzero.  This is set by spa_init() after it has initialized the
namespace lock.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tim Chase <tim at chase2k.com>
Signed-off-by: Olaf Faaland <faaland1 at llnl.gov>
Closes #7496
Closes #7521
Delta File
+13 -11 module/zfs/vdev_disk.c
+2 -1 module/zfs/mmp.c
+15 -12 2 files

ZFS on Linux — cmd/zed/zed.d history_event-zfs-list-cacher.sh.in, etc/systemd/system-generators zfs-mount-generator.in

Add canonical mount options zfs-mount-generator

lib/libzfs/libzfs_mount.c:zfs_add_options provides the canonical
mount options used by a `zfs mount` command. Because we cannot call
`zfs mount` directly from a systemd.mount unit, we mirror that logic
in zfs-mount-generator.

The zed script is updated to cache these properties as well.

Include a mini-tutorial in the manual page, properly substitute
configuration paths in zfs-mount-generator.8.in, and standardize the
Makefile.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Richard Laager <rlaager at wiktel.com>
Signed-off-by: Antonio Russo <antonio.e.russo at gmail.com>
Closes #7453

ZFS on Linux — tests/zfs-tests/tests/functional/cli_root/zpool_import zpool_import.kshlib import_cache_device_removed.ksh

Fix shebangs on import tests

Incorrect shebangs were used when porting.

Reviewed-by: John Kennedy <john.kennedy at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: bunder2015 <omfgbunder at gmail.com>
Closes #7523
Closes #7524

ZFS on Linux — module/zfs spa_misc.c

Unify behavior of deadman parameters

The zfs_deadman_failmode, zfs_deadman_ziotime_ms and
zfs_deadman_synctime_ms paramaters are stored per-pool.  However,
only the zfs_deadman_failmode updates the per-pool state when it's
change.  This patch gives adds the same behavior to the other two
for consistency.

Also, in all 3 three cases, only update the per-pool parameters
if spa_init() has actually been called in order to avoid panicking
when trying to take a lock on the spa_namespace_lock mutex.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tim Chase <tim at chase2k.com>
Closes #7499
Delta File
+52 -6 module/zfs/spa_misc.c
+52 -6 1 file

ZFS on Linux — tests/zfs-tests/tests/functional/no_space enospc_002_pos.ksh

ZTS: enospc_002 path cleanup

Removing hard-coded path used in enospc_002

Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: bunder2015 <omfgbunder at gmail.com>
Closes #7515

ZFS on Linux — module/zfs zfs_ioctl.c, tests/runfiles linux.run

Fix 'zpool create -t <tempname>'

Creating a pool with a temporary name fails when we also specify custom
dataset properties: this is because we mistakenly call
zfs_set_prop_nvlist() on the "real" pool name which, as expected,
cannot be found because the SPA is present in the namespace with the
temporary name.

Fix this by specifying the correct pool name when setting the dataset
properties.

Reviewed-by: Prakash Surya <prakash.surya at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #7502
Closes #7509

ZFS on Linux — module/zfs spa.c, tests/zfs-tests/tests/functional/cli_root/zpool_split zpool_split_props.ksh

ZTS: Re-enable MMP tests

Commit 7fab6361 inadvertently disabled the MMP test cases by creating
and not removing an /etc/hostid file in the new zpool_split_props test
case.  When the file exists the ZTS skips the entire MMP test group
rather than modify what may be a system which is already configured.
Update the test case to remove the file.

Additionally, because the MMP tests were disabled a regression slipped
in as part of commit 9eb7b46ed0.  Fix it.

Reviewed-by: Tim Chase <tim at chase2k.com>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: loli10K <ezomori.nozomu at gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7514

ZFS on Linux — cmd/zdb zdb.c

OpenZFS 9523 - Large alloc in zdb can cause trouble

16MB alloc in zdb_embedded_block() can cause cores in certain
situations (clang, gcc55).

Authored by: Jorgen Lundman <lundman at lundman.net>
Reviewed by: Igor Kozhukhov <igor at dilos.org>
Reviewed by: Andriy Gapon <avg at FreeBSD.org>
Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Approved by: Dan McDonald <danmcd at joyent.com>
Ported-by: Brian Behlendorf <behlendorf1 at llnl.gov>

Porting Notes:
* Replaces an equivalent fix previously made for Linux.

OpenZFS-issue: https://illumos.org/issues/9523
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/2c1964a
Closes #7561
Delta File
+8 -5 cmd/zdb/zdb.c
+8 -5 1 file

ZFS on Linux — tests/zfs-tests/tests/functional/snapshot snapshot_002_pos.ksh snapshot_006_pos.ksh

ZTS: remove dead cleanup code from snapshot tests

Caught during path cleanups, the files referenced do not appear to be
created or used anywhere.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: bunder2015 <omfgbunder at gmail.com>
Closes #7508

ZFS on Linux — contrib/pyzfs/libzfs_core/test __init__.py

Prevent `make distclean` removing 0 sized file

__init__.py used by Python packages typically has nothing in it
including contrib/pyzfs/libzfs_core/test/__init__.py, however this
causes `make distclean` to delete the file.

This is the only file with size 0, and it seems reasonable to have
a comment to avoid being deleted, rather than trying to modify
distclean behavior.

 # find . -size 0
 ./contrib/pyzfs/libzfs_core/test/__init__.py
 # ./autogen.sh ; ./configure ; make -j8
 # make distclean
 # ls contrib/pyzfs/libzfs_core/test/__init__.py
 ls: cannot access 'contrib/pyzfs/libzfs_core/test/__init__.py':
     No such file or directory
 # git diff
 diff --git a/contrib/pyzfs/libzfs_core/test/__init__.py
     b/contrib/pyzfs/libzfs_core/test/__init__.py
 deleted file mode 100644

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tomohiro Kusumi <kusumi.tomohiro at osnexus.com>
Closes #7505

ZFS on Linux — tests/zfs-tests/tests/functional/cli_root/zfs_diff zfs_diff_timestamp.ksh

ZTS: Fix zfs_diff_timestamp

When using mawk instead of gawk zfs_diff_timestamp fails consistently:
this is due to a subtle difference in how mawk handles substr().

From awk(1):
---
Finally, here is how mawk handles exceptional cases not discussed in
the AWK book or the Posix draft.  It is unsafe to assume consistency
across awks and safe to skip to the next section.
substr(s,  i,  n) returns the characters of s in the intersection of
the closed interval [1, length(s)] and the half-open interval [i, i+n).
When this intersection is empty, the empty string is returned; so
substr("ABC", 1, 0) = "" and substr("ABC", -4, 6) = "A".
---

To support running zfs_diff_timestamp with both gawk and mawk change
the second parameter passed to substr().

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #7503
Closes #7510

ZFS on Linux — tests/zfs-tests/tests/functional/cli_root/zpool_import setup.ksh cleanup.ksh

Streamline the zpool_import tests

Don't create an ext4 file system atop $DEV_DISKDIR/$DISK2.
There's likely to not be sufficient space for it to succeed.
Instead, simply create the vdev files in the directory where it
would have been mounted.

Signed-off-by: Tim Chase <tim at chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7459

ZFS on Linux — module/zfs vdev.c

Clear vdev_faulted

Clear vdev_faulted if ZPOOL_CONFIG_AUX_STATE is not set to "external"

ZoL supports "zpool export -f" (force fault), which can be combined
with "-t" (temporary fault; don't persist across export/import) and
causes a MOS configuration to be set with ZPOOL_CONFIG_FAULTED=1
and without ZFS_CONFIG_AUX_STATE set at all.  In this case, the
previously-offlined vdev should be imported in an on-line state and.
Clearing the "vdev_faulted" flag causes the import to treat the
device as on-line.  Typically, resilver will catch it up based on
its DTL.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tim Chase <tim at chase2k.com>
Closes #7459
Delta File
+2 -0 module/zfs/vdev.c
+2 -0 1 file

ZFS on Linux — config always-pyzfs.m4

Fix inverted check for --enable-pyzfs

The --{en,dis}able-pyzfs check is backwards. Fix that.

Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: loli10K <ezomori.nozomu at gmail.com>
Signed-off-by: DHE <git at dehacked.net>
Closes #7493
Delta File
+1 -1 config/always-pyzfs.m4
+1 -1 1 file

ZFS on Linux — rpm/generic zfs.spec.in

Fedora 28: Add BuildRequires: libtirpc-devel

Add "BuildRequires: libtirpc-devel" to fix mock builds on Fedora 28.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tony Hutter <hutter2 at llnl.gov>
Closes #7494
Closes #7495
Delta File
+3 -0 rpm/generic/zfs.spec.in
+3 -0 1 file

ZFS on Linux — cmd/zinject zinject.c, include/sys zio_crypt.h dsl_crypt.h

Add support for decryption faults in zinject

This patch adds the ability for zinject to trigger decryption
and authentication faults in the ZIO and ARC layers. This
functionality is exposed via the new "decrypt" error type, which
may be provided for "data" object types.

This patch also refactors some of the core encryption / decryption
functions so that they have consistent prototypes, handle errors
consistently, and do not have unused arguments.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Closes #7474

ZFS on Linux — config kernel-vfs-iterate.m4, include/sys zpl.h zfs_vnops.h

RHEL 7.5 compat: FMODE_KABI_ITERATE

As of RHEL 7.5 the mainline fops.iterate() method was added to
the file_operations structure and is correctly detected by the
configure script.

Normally this is what we want, but in order to maintain KABI
compatibility the RHEL change additionally does the following:

* Requires that callers intending to use this extended interface
  set the FMODE_KABI_ITERATE flag on the file structure when
  opening the directory.
* Adds the fops.iterate() method to the end of the structure,
  without removing fops.readdir().

This change updates the configure check to ignore the RHEL 7.5+
variant of fops.iterate() when detected.  Instead fallback to
the fops.readdir() interface which will be available.

Finally, add the 'zpl_' prefix to the directory context wrappers
to avoid colliding with the kernel provided symbols when both
the fops.iterate() and fops.readdir() are provided by the kernel.

Reviewed-by: Olaf Faaland <faaland1 at llnl.gov>

    [4 lines not shown]

ZFS on Linux — module/zfs qat_crypt.c qat_compress.c

Fix inst_num overflow in qat_crypt.c

This patch fixes the same issue which was previously addressed in
6051.  The variable "inst_num" was of the incorrect type and
"atomic_inc_32_nv()" could cause an overflow damaging its neighbor.

Cast the return value of atomic_inc_32_nv() to Cpa32U.

Fix a few types for num_inst for clarity.

Reviewed-by: Weigang Li <weigang.li at intel.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7468

ZFS on Linux — module/zfs dmu_diff.c arc.c, tests/runfiles linux.run

Fix issues found with zfs diff

Two deadlocks / ASSERT failures were introduced in a2c2ed1b which
would occur whenever arc_buf_fill() failed to decrypt a block of
data. This occurred because the call to arc_buf_destroy() which
was responsible for cleaning up the newly created buffer would
attempt to take out the hdr lock that it was already holding. This
was resolved by calling the underlying functions directly without
retaking the lock.

In addition, the dmu_diff() code did not properly ensure that keys
were loaded and mapped before begining dataset traversal. It turns
out that this code does not need to look at any encrypted values,
so the code was altered to perform raw IO only.

Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Closes #7354
Closes #7456

ZFS on Linux — module/zfs metaslab.c

Silence compile-time warning on unused variable

ASSERT3U() could be NOP which then leads to having unused pointer *spa.

metaslab.c: In function 'metaslab_condense':
metaslab.c:2075:9: warning: unused variable 'spa' [-Wunused-variable]
  spa_t *spa = msp->ms_group->mg_vd->vdev_spa;

Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Tomohiro Kusumi <kusumi.tomohiro at osnexus.com>
Closes #7489
Delta File
+1 -2 module/zfs/metaslab.c
+1 -2 1 file

ZFS on Linux — lib/libzfs libzfs_sendrecv.c, tests/zfs-tests/tests/functional/cli_root/zfs_receive receive-o-x_props_override.ksh

Fix zfs incremental send remove '-o' properties

When receiving an incremental send stream with intermediary snapshots
zfs_receive_one() does not correctly identify the top-level dataset:
consequently we restore said snapshots as if they were children
datasets in the hierarchy, forcing inheritance of any property received
with 'zfs send -o' and effectively removing any locally set value.

The test case did not correctly verify this situation because it uses
adjacent snapshots, basically testing 'zfs send -i' instead of
'zfs send -I': this commit adds an additional intermediary snapshot to
the test script.

Reviewed-by: Paul Dagnelie <pcd at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #7478

ZFS on Linux — man/man8 zpool.8

Add back iostat -y or -w descriptions

The iostat -y and -w descriptions were left in cda0317e,
get them back.

Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: George Melikov <mail at gmelikov.ru>
Closes #7479
Closes #7483
Delta File
+19 -2 man/man8/zpool.8
+19 -2 1 file

ZFS on Linux — tests/runfiles linux.run, tests/zfs-tests/tests/functional/mv_files random_creation.ksh mv_files.cfg

Add test with two kinds of file creation orders

Data loss was identified in #7401 when many small files were copied.
This adds a reproducer for this bug and other similar ones: randomly
generate N files. Then, listing M of them by `ls -U` order, produce
those same files in a directory of the same name.

This triggers the bug consistently, provided N and M are large enough.
Here, N=2^16 and M=2^13.

Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Antonio Russo <antonio.e.russo at gmail.com>
Closes #7411

ZFS on Linux — module/zfs vdev_indirect.c

Fix CONFIG_GCC_PLUGIN_RANDSTRUCT build

Fix build errors with gcc 7.3.0 on Gentoo with kernel 4.16.3
built with CONFIG_GCC_PLUGIN_RANDSTRUCT=y such as:

module/zfs/vdev_indirect.c:296:2: error:
positional initialization of field in ‘struct’ declared with
‘designated_init’ attribute [-Werror=designated-init]
  vdev_indirect_map_free,
  ^~~~~~~~~~~~~~~~~~~~~~

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: Mark Wright <gienah at gentoo.org>
Closes #7464
Delta File
+2 -2 module/zfs/vdev_indirect.c
+2 -2 1 file

ZFS on Linux — . configure.ac, cmd/zfs zfs_main.c

Fix 'zfs remap <poolname at snapname>'

Only filesystems and volumes are valid 'zfs remap' parameters: when
passed a snapshot name zfs_remap_indirects() does not handle the
EINVAL returned from libzfs_core, which results in failing an assertion
and consequently crashing.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #7454

ZFS on Linux — . configure.ac, include/sys zap_leaf.h

Fix ENOSPC in "Handle zap_add() failures in ..."

Commit cc63068 caused ENOSPC error when copy a large amount of files
between two directories. The reason is that the patch limits zap leaf
expansion to 2 retries, and return ENOSPC when failed.

The intent for limiting retries is to prevent pointlessly growing table
to max size when adding a block full of entries with same name in
different case in mixed mode. However, it turns out we cannot use any
limit on the retry. When we copy files from one directory in readdir
order, we are copying in hash order, one leaf block at a time. Which
means that if the leaf block in source directory has expanded 6 times,
and you copy those entries in that block, by the time you need to expand
the leaf in destination directory, you need to expand it 6 times in one
go. So any limit on the retry will result in error where it shouldn't.

Note that while we do use different salt for different directories, it
seems that the salt/hash function doesn't provide enough randomization
to the hash distance to prevent this from happening.

Since cc63068 has already been reverted. This patch adds it back and
removes the retry limit.

Also, as it turn out, failing on zap_add() has a serious side effect for
mzap_upgrade(). When upgrading from micro zap to fat zap, it will

    [13 lines not shown]

ZFS on Linux — cmd/zstreamdump zstreamdump.c, module/zfs dmu_send.c

Fix issues with raw sends of spill blocks

This patch fixes 2 issues in how spill blocks are processed during
raw sends. The first problem is that compressed spill blocks were
using the logical length rather than the physical length to
determine how much data to dump into the send stream. The second
issue is a typo that caused the spill record's object number to be
used where the objset's ID number was required. Both issues have
been corrected, and the payload_size is now printed in zstreamdump
for future debugging.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Closes #7378
Closes #7432

ZFS on Linux — module/zfs dmu_send.c dnode.c, tests/zfs-tests/tests/functional/rsend send_realloc_dnode_size.ksh

Fix object reclaim when using large dnodes

Currently, when the receive_object() code wants to reclaim an
object, it always assumes that the dnode is the legacy 512 bytes,
even when the incoming bonus buffer exceeds this length. This
causes a buffer overflow if --enable-debug is not provided and
triggers an ASSERT if it is. This patch resolves this issue and
adds an ASSERT to ensure this can't happen again.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Closes #7097
Closes #7433

ZFS on Linux — include/sys dbuf.h dmu.h, module/zfs dmu.c dmu_send.c

assertion in arc_release() during encrypted receive

In the existing code, when doing a raw (encrypted) zfs receive,
we call arc_convert_to_raw() from open context. This creates a
race condition between arc_release()/arc_change_state() and
writing out the block from syncing context (arc_write_ready/done()).

This change makes it so that when we are doing a raw (encrypted)
zfs receive, we save the crypt parameters (salt, iv, mac) of dnode
blocks in the dbuf_dirty_record_t, and call arc_convert_to_raw()
from syncing context when writing out the block of dnodes.

Additionally, we can eliminate dr_raw and associated setters, and
instead know that dnode blocks are always raw when doing a zfs
receive (see the new field os_raw_receive).

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tom Caputi <tcaputi at datto.com>
Signed-off-by: Matthew Ahrens <mahrens at delphix.com>
Closes #7424
Closes #7429

ZFS on Linux — tests/zfs-tests/tests/functional/reservation reservation_013_pos.sh

ZTS: fix reservation_013_pos integer overflow

When using large disks the integers for calculating sizes can
overflow past 2**31.  Changing to long integers with typeset
should correct this.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: bunder2015 <omfgbunder at gmail.com>
Closes #4444
Closes #7451

ZFS on Linux — module/zfs vdev_indirect_mapping.c

Revert "OpenZFS 9036 - zfs: duplicate 'const' declaration specifier"

This reverts commit cbb893321545c2c9052787b556c9375fcb103979.

The original change in OpenZFS 9036 did remove duplicate 'const'
specifiers, but the ZoL port had already done what *should* have been
done in OpenZFS 9036, which is to make the pointers themselves const.
The port of the change to ZoL ended up doing an unnecessary removal
of the constness of the pointers. Undo that.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tim Chase <tim at chase2k.com>
Signed-off-by: Ari Sundholm <ari at tuxera.com>
Closes #7444

ZFS on Linux — tests/zfs-tests/tests/functional/cli_root/zpool_create zpool_create_002_pos.ksh

ZTS: zpool_create_002 clean up leftover filedisk

zpool_create_002_pos did not clean up filedisk files left over from
running the test.

Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: bunder2015 <omfgbunder at gmail.com>
Closes #7435
Closes #7439

ZFS on Linux — module/zfs zthr.c

Avoid Linux hung task message in ZTHR

Use an interruptible to avoid Linux hung task message in
ZTHR and to prevent inflating the load average.

Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tim Chase <tim at chase2k.com>
Closes #7440
Closes #7441
Delta File
+1 -1 module/zfs/zthr.c
+1 -1 1 file

ZFS on Linux — tests/zfs-tests/tests/functional/cli_root/zfs_receive zfs_receive_raw_incremental.ksh

ZTS: clean up leftover ibackup_trunc files

zfs_receive_raw_incremental did not clean up ibackup_trunc.* files
left over from running the test.

Also changing the path of the ibackup files so they can be placed
in the correct directories when /var/tmp is not the temporary
directory.

Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: bunder2015 <omfgbunder at gmail.com>
Closes #7430

ZFS on Linux — include/linux blkdev_compat.h

Linux compat 4.16: blk_queue_flag_{set,clear}

The HAVE_BLK_QUEUE_WRITE_CACHE_GPL_ONLY case was overlooked in
the original 10f88c5c commit because blk_queue_write_cache()
was available for the in-kernel builds.

Update the blk_queue_flag_{set,clear} wrappers to call the locked
versions to avoid confusion.  This is safe for all existing callers.

The blk_queue_set_write_cache() function has been updated to use
these wrappers.  This means setting/clearing both QUEUE_FLAG_WC
and QUEUE_FLAG_FUA is no longer atomic but this only done early
in zvol_alloc() prior to any requests so there is no issue.

Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Kash Pande <kash at tripleback.net>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7428
Closes #7431
Delta File
+6 -8 include/linux/blkdev_compat.h
+6 -8 1 file

ZFS on Linux — config kernel-fst-mount.m4 kernel-mount-nodev.m4, include/linux vfs_compat.h

Allow mounting datasets more than once

Currently mounting an already mounted zfs dataset results in an
error, whereas it is typically allowed with other filesystems.
This causes some bad interactions with mount namespaces. Take
this sequence for example:

- Create a dataset
- Create a snapshot of the dataset
- Create a clone of the snapshot
- Create a new mount namespace
- Rename the original dataset

The rename results in unmounting and remounting the clone in the
original mount namespace, however the remount fails because the
dataset is still mounted in the new mount namespace. (Note that
this means the mount in the new mount namespace is never being
unmounted, so perhaps the unmount/remount of the clone isn't
actually necessary.)

The problem here is a result of the way mounting is implemented
in the kernel module. Since it is not mounting block devices it
uses mount_nodev() instead of the usual mount_bdev(). However,
mount_nodev() is written for filesystems for which each mount is
a new instance (i.e. a new super block), and zfs should be able

    [15 lines not shown]

ZFS on Linux — . configure.ac, lib/libzfs libzfs_pool.c

Add 'zpool split' coverage to the ZFS Test Suite

This change adds five new tests to the ZTS:

 * zpool_split_cliargs: verify command line options and arguments
 * zpool_split_devices: verify zpool split accepts a device list
 * zpool_split_encryption: verify zpool can split encrypted pools
 * zpool_split_props: verify zpool split can set property values
 * zpool_split_vdevs: verify vdev layout when splitting the pool

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #7409

ZFS on Linux — lib/libefi rdwr_efi.c, lib/libshare libshare.c

Fix calloc(3) arguments order

calloc(3) takes `nelem` (or `nmemb` in glibc) first, and then size of
elements.  No difference expected for having these in reverse order,
however should follow the standard.

http://pubs.opengroup.org/onlinepubs/009695399/functions/calloc.html

Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tomohiro Kusumi <kusumi.tomohiro at osnexus.com>
Closes #7405

ZFS on Linux — module/zfs arc.c

Fix zfs_arc_max minimum tuning

When setting `zfs_arc_max` its minimum value is allowed
to be 64 MiB.  There was an off-by-1 error which can matter
on tiny systems.

Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Chris Zubrzycki <github at mid-earth.net>
Closes #7417
Delta File
+1 -1 module/zfs/arc.c
+1 -1 1 file

ZFS on Linux — lib/libzfs libzfs_pool.c, tests/runfiles linux.run

Fix zpool set feature@<feature>=disabled

Commit e4010f2 accidentally allows zpool to set pool features to
"disabled"; this should only be allowed at pool creation. This commit
adds additional checks and test coverage to 'zpool set'.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #7402

ZFS on Linux — lib/libzfs libzfs_dataset.c libzfs_util.c, man/man8 zfs.8

OpenZFS 9286 - want refreservation=auto

Authored by: Mike Gerdts <mike.gerdts at joyent.com>
Reviewed by: Allan Jude <allanjude at freebsd.org>
Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Reviewed by: John Kennedy <john.kennedy at delphix.com>
Reviewed by: Andy Stormont <astormont at racktopsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Approved by: Richard Lowe <richlowe at richlowe.net>
Ported-by: Don Brady <don.brady at delphix.com>

Porting Notes:
* Adopted destroy_dataset in ZTS test cleanup
* Use ksh shebang instead of bash for new tests

OpenZFS-issue: https://www.illumos.org/issues/9286
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/723d0c85
Closes #7387

ZFS on Linux — man/man8 zfs.8

zfs(8): fix dedup omission during mdoc overhaul

The property description has been updated with new algorithms as well.

Reviewed-by: Matt Ahrens <matt at delphix.com>
Reviewed-by: Richard Elling <Richard.Elling at RichardElling.com>
Reviewed-by: loli10K <ezomori.nozomu at gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: DHE <git at dehacked.net>
Closes #7377
Delta File
+30 -0 man/man8/zfs.8
+30 -0 1 file

ZFS on Linux — include/sys dnode.h dmu_impl.h, module/zfs dnode.c dmu_objset.c

Fix race in dnode_check_slots_free()

Currently, dnode_check_slots_free() works by checking dn->dn_type
in the dnode to determine if the dnode is reclaimable. However,
there is a small window of time between dnode_free_sync() in the
first call to dsl_dataset_sync() and when the useraccounting code
is run when the type is set DMU_OT_NONE, but the dnode is not yet
evictable, leading to crashes. This patch adds the ability for
dnodes to track which txg they were last dirtied in and adds a
check for this before performing the reclaim.

This patch also corrects several instances when dn_dirty_link was
treated as a list_node_t when it is technically a multilist_node_t.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Closes #7147
Closes #7388