zpool create: report which device caused failure
When zpool create fails because a vdev is already in use, the
error message now identifies the problematic device and the pool
it belongs to, e.g.:
cannot create 'tank': device '/dev/sdb1' is part of
active pool 'rpool'
Implementation follows the ZPOOL_CONFIG_LOAD_INFO pattern used
by zpool import:
- Add spa_create_info to spa_t to capture error info during
vdev_label_init(), before vdev_close() resets vdev state
- When vdev_inuse() detects a conflict, read the on-disk
label to extract the pool name and store it with the
device path
- Return the info wrapped under ZPOOL_CONFIG_CREATE_INFO
through the ioctl zc_nvlist_dst to userspace
[10 lines not shown]
Replace pure-python filter_list
This commit replaces the pure-python implementation of filter_list
with the version provided by the truenas/truenas_pyos repo
(truenas_pyfilter). The overall new workflow for this is:
1. convert the filters / options to their respective objects from
truenas_pyfilter (compile_filters, compile_options).
2. use the filters / options to either match (if there's single item)
or tnfilter (if there is more than one).
Output is same so this is mostly a drop-in replacement; however,
in some places in our codebase we keep copies of pre-compiled filters
and options because they do not change. The filter_list util is now
replaced with what is largely a thin wrapper around the extension.
API validation also now wraps around validation provided by the
extension.
[clang-doc] Merge data into persistent memory
We have a need for persistent memory for the final info. Since each
group processes a single USR at a time, every USR is only ever processed by
a single thread from the thread pool. This means that we can keep per
thread persistent storage for all the info. There is significant
duplicated data between all the serialized records, so we can just merge
the final/unique items into the persistent arena, and clear out the
scratch/transient arena as we process each record in the bitcode.
The patch adds some APIs to help with managing the data, merging, and
allocation of data in the correct arena. It also safely merges and deep
copies data from the transient arenas into persistent storage that is
never reset until the program completes.
This patch reduces memory by another % over the previous patches,
bringing the total savings over the baseline to 57%. Runtime performance
and benchmarks stay mostly flat with modest improvements.
[31 lines not shown]
[clang-doc] Move Info types into arenas
Info types used to own significant chunks of data. As we move these into
local arenas, these types must be trivially destructible, to avoid
leaking resources when the arena is reset. Unfortunaly, there isn't a
good way to transition all the data types one at a time, since most of
them are tied together in some way. Further, as they're now allocated in
the arenas, they often cannot be treated the same way, and even the
aliases and interfaces put in pLace to simplify the transition cannot
cover the full range of changes required.
We also use some SFINAE tricks to avoid adding boilerplate for helper
APIs, we'd otherwise ahve to support
Though it introduces some additional churn, we also try to keep tests
from using arena allocation as much as possible, since this is not
required to test the implementation of the library. As much of the test
code needed to be rewritten anyway, we take the opportunity to
transition now.
[41 lines not shown]
[clang-doc] Move non-arena allocated types off the OwnedPtr alias
Some types should not be using this alias, which was over applied to
APIs that wont participate in arena style allocation. This patch
restores them to their correct spelling.
[clang-doc] Enforce arena allocated types are trivially destructible
We can enforce at compile-time that the types we want to place in the
arenas are always safe to allocate there.
[clang-doc] Simplify parsing and reading bitcode blocks
Much of the logic int he readBlock implementation is boilerplate, and is
repeated for each implementation/specialization. This will become much
worse as we introduce new custom block reading logic as we migrate
towards arena allocation. In preparation for that, we're introducing the
change in logic now, which should make later refactoring much more
straightforward.
[clang-doc] Consolidate merging logic
As we migrate things in the arena, this logic may get more complex.
Factoring it out now, will give clear extension points to make this
easier to manage.
[clang-doc] Make CommentInfo arena allocated
This patch move the CommentInfo type into the arena. It updates block
handling to collect child info types and serialize the array in one
shot.
We also clean up the test code to avoid using the arenas in the tests.
This has the upside of making the test more hermetic, and avoids churn
in the related code as the allocation API interfaces evolve.
Performance and memory usage regress slightly. This is somewhat expected
as we do not yet aggressively release short term memory during merge
operations. Future patches will reclaim this overhead.
| Metric | Baseline | Prev | This | Culm% | Seq% |
| :--- | :--- | :--- | :--- | :--- | :--- |
| Time | 920.5s | 998.5s | 1010.5s | +9.8% | +1.2% |
| Memory | 86.0G | 43.8G | 47.8G | -44.4% | +9.2% |
[26 lines not shown]
[clang-doc] Support deep copy between arenas for merging
Upcoming changes to the merge step will necessitate that we clear the
transient arenas and merge new items into the persistent arena. However
there are some challenges with that, as the existing types typically
don't want to be copied. We introduce some new APIs to simplify that
task and ensure we don't accidentally leak memory.
On the performance front, we reclaim about 2% of the overhead, bringing
the cumulative overhead from the series of patches down to about 7% over
the baseline.
| Metric | Baseline | Prev | This | Culm% | Seq% |
| :--- | :--- | :--- | :--- | :--- | :--- |
| Time | 920.5s | 1014.5s | 991.5s | +7.7% | -2.3% |
| Memory | 86.0G | 39.9G | 40.0G | -53.4% | +0.3% |
| Benchmark | Baseline | Prev | This | Culm% | Seq% |
| :--- | :--- | :--- | :--- | :--- | :--- |
[28 lines not shown]
[clang] Fix issues with const/pure on varargs function. (#190252)
There are two related issues here. On the declaration/definition side,
we need to make sure the markings are conservative. Then on the caller
side, we need to make sure we don't access parameters that don't exist.
Fixes #187535.
[ARM] Add new test that will demonstrate the cmn node in the ARM backend (NFC) (#179282)
No code changes yet, but this is going to change once the cmn node lands
in the backend.
graphics/nvidia-drm-*-kmod*: Fix build for 13.5, 14.3 and 15.0
Latest upgrade to 595.58.03 caused build failure as of
requirements for a stub function pm_vt_switch_required().
The commit introduced it is not MFC'ed to 13.5-RELEASE,
stable/13, 14.3-RELEASE and 15.0-RELEASE and unlikely
to be done as it's not a security fix, more, stable/13 is going
to be EoL'ed in 1 month.
As this function, defined in sys/compat/linuxkpi/common/linux/pm.h,
is still a blank, stub function even on main, drop the offending line
for affected versions, keeping as-is for versions which has it.
No PORTREVISION bumps, as this is build fix.
PR: 294096
Reported by: alt2600 at icloud.com
Reviewed by: ashafer
[4 lines not shown]
graphics/nvidia-drm-*-kmod*: Fix build for 13.5, 14.3 and 15.0
Latest upgrade to 595.58.03 caused build failure as of
requirements for a stub function pm_vt_switch_required().
The commit introduced it is not MFC'ed to 13.5-RELEASE,
stable/13, 14.3-RELEASE and 15.0-RELEASE and unlikely
to be done as it's not a security fix, more, stable/13 is going
to be EoL'ed in 1 month.
As this function, defined in sys/compat/linuxkpi/common/linux/pm.h,
is still a blank, stub function even on main, drop the offending line
for affected versions, keeping as-is for versions which has it.
No PORTREVISION bumps, as this is build fix.
PR: 294096
Reported by: alt2600 at icloud.com
Reviewed by: ashafer
[2 lines not shown]
Revert "[lldb/test] Codesign executables built with custom Makefile rules" (#190398)
Reverts llvm/llvm-project#189902 because this seems to cause hangs.
