[AArch64][SME] Preserve ZA in agnostic ZA functions without +sme (#190141)
`__arm_agnostic("sme_za_state")` does not require +sme, but we must
still preserve ZA in case the function is used with code that makes use
of ZA:
> The use of `__arm_agnostic("sme_za_state")` allows writing functions
> that are compatible with ZA state without having to share ZA state
> with the caller, as required by `__arm_preserves`. The use of this
> attribute does not imply that SME is available.
[lldb][kernel debug] Add a missing call to scan local fs for kexts (#190281)
A kernel developer noticed that I missed a call to index the local
filesystem in one of our codepaths, and had a use case that depended on
that working.
rdar://173814556
[Runtimes] Gracefully handle invalid LLVM_TARGET_TRIPLE (#190284)
In some situations such as reported at
https://github.com/llvm/llvm-project/pull/177953#issuecomment-4179014239,
LLVM_(DEFAULT_)TARGET_TRIPLE is not set. It is used to derive the output
directory in #177953. Only flang-rt currently uses
RUNTIMES_(INSTALL|OUTPUT)_RESOURCE_LIB_PATH, we should not fail building
other despite a missing LLVM_TARGET_TRIPLE.
Compiler-rt uses COMPILER_RT_DEFAULT_TARGET_TRIPLE instead which it
derives itself. Most other LLVM runtimes libraries just skip the target
portion of the library path (explicitly so since #93354). Do the same
for RUNTIMES_(INSTALL|OUTPUT)_RESOURCE_LIB_PATH which we hope eventually
can replace the other mechanisms.
[clang-doc] Switch to string internment (#190044)
This is the first step in migrating all the Info types to be POD. We
introduced a shared string saver that can be used safely across threads,
and updated the internal represntation of various data types to use
these over owned strings, like SmallString or std::string.
This also required changes to YAMLGenerator to keep the single quoted
string formatting and to update the YAML traits.
This change gives an almost 50% reduction in peak memory when building
documentation for clang, at about a 10% performance loss. Future patches
can mitigate the performance penalties, and further reduce memory use.
| Metric | Baseline | Prev | This | Culm% | Seq% |
| :--- | :--- | :--- | :--- | :--- | :--- |
| Time | 920.5s | 920.5s | 1011.0s | +9.8% | +9.8% |
| Memory | 86.0G | 86.0G | 44.9G | -47.8% | -47.8% |
[30 lines not shown]
[clang-doc] Merge data into persistent memory
We have a need for persistent memory for the final info. Since each
group processes a single USR at a time, every USR is only ever processed by
a single thread from the thread pool. This means that we can keep per
thread persistent storage for all the info. There is significant
duplicated data between all the serialized records, so we can just merge
the final/unique items into the persistent arena, and clear out the
scratch/transient arena as we process each record in the bitcode.
The patch adds some APIs to help with managing the data, merging, and
allocation of data in the correct arena. It also safely merges and deep
copies data from the transient arenas into persistent storage that is
never reset until the program completes.
This patch reduces memory by another % over the previous patches,
bringing the total savings over the baseline to 57%. Runtime performance
and benchmarks stay mostly flat with modest improvements.
[31 lines not shown]
[clang-doc] Support deep copy between arenas for merging
Upcoming changes to the merge step will necessitate that we clear the
transient arenas and merge new items into the persistent arena. However
there are some challenges with that, as the existing types typically
don't want to be copied. We introduce some new APIs to simplify that
task and ensure we don't accidentally leak memory.
On the performance front, we reclaim about 2% of the overhead, bringing
the cumulative overhead from the series of patches down to about 7% over
the baseline.
| Metric | Baseline | Prev | This | Culm% | Seq% |
| :--- | :--- | :--- | :--- | :--- | :--- |
| Time | 920.5s | 1014.5s | 991.5s | +7.7% | -2.3% |
| Memory | 86.0G | 39.9G | 40.0G | -53.4% | +0.3% |
| Benchmark | Baseline | Prev | This | Culm% | Seq% |
| :--- | :--- | :--- | :--- | :--- | :--- |
[28 lines not shown]
[clang-doc] Move Info types into arenas
Info types used to own significant chunks of data. As we move these into
local arenas, these types must be trivially destructible, to avoid
leaking resources when the arena is reset. Unfortunaly, there isn't a
good way to transition all the data types one at a time, since most of
them are tied together in some way. Further, as they're now allocated in
the arenas, they often cannot be treated the same way, and even the
aliases and interfaces put in pLace to simplify the transition cannot
cover the full range of changes required.
We also use some SFINAE tricks to avoid adding boilerplate for helper
APIs, we'd otherwise ahve to support
Though it introduces some additional churn, we also try to keep tests
from using arena allocation as much as possible, since this is not
required to test the implementation of the library. As much of the test
code needed to be rewritten anyway, we take the opportunity to
transition now.
[41 lines not shown]
[clang-doc] Move non-arena allocated types off the OwnedPtr alias
Some types should not be using this alias, which was over applied to
APIs that wont participate in arena style allocation. This patch
restores them to their correct spelling.
[clang-doc] Consolidate merging logic
As we migrate things in the arena, this logic may get more complex.
Factoring it out now, will give clear extension points to make this
easier to manage.
[clang-doc] Enforce arena allocated types are trivially destructible
We can enforce at compile-time that the types we want to place in the
arenas are always safe to allocate there.
[clang-doc] Simplify parsing and reading bitcode blocks
Much of the logic int he readBlock implementation is boilerplate, and is
repeated for each implementation/specialization. This will become much
worse as we introduce new custom block reading logic as we migrate
towards arena allocation. In preparation for that, we're introducing the
change in logic now, which should make later refactoring much more
straightforward.
[clang-doc] Make CommentInfo arena allocated
This patch move the CommentInfo type into the arena. It updates block
handling to collect child info types and serialize the array in one
shot.
We also clean up the test code to avoid using the arenas in the tests.
This has the upside of making the test more hermetic, and avoids churn
in the related code as the allocation API interfaces evolve.
Performance and memory usage regress slightly. This is somewhat expected
as we do not yet aggressively release short term memory during merge
operations. Future patches will reclaim this overhead.
| Metric | Baseline | Prev | This | Culm% | Seq% |
| :--- | :--- | :--- | :--- | :--- | :--- |
| Time | 920.5s | 998.5s | 1010.5s | +9.8% | +1.2% |
| Memory | 86.0G | 43.8G | 47.8G | -44.4% | +9.2% |
[26 lines not shown]
[clang-doc] Introduce TransientArena for short lived allocations
With strings interned, we can move the StringRefs in various Info
structs into a new short lived arena. This change migrates the remaining
SmallVectors in CommentInfo to use an ArrayRef backed by the new
transient arena.
This results in further minor reductions in overall memory usage, but no
significant effect on runtime performance.
| Metric | Baseline | Prev | This | Culm% | Seq% |
| :--- | :--- | :--- | :--- | :--- | :--- |
| Time | 920.5s | 1011.0s | 1005.7s | +9.2% | -0.5% |
| Memory | 86.0G | 44.9G | 42.1G | -51.0% | -6.2% |
| Benchmark | Baseline | Prev | This | Culm% | Seq% |
| :--- | :--- | :--- | :--- | :--- | :--- |
| BM_BitcodeReader_Scale/10 | 67.9us | 70.0us | 68.6us | +1.0% | -2.0% |
| BM_BitcodeReader_Scale/10000 | 70.5ms | 21.3ms | 21.3ms | -69.8% | -0.0% |
[23 lines not shown]
[clang-doc] Switch to string internment
This is the first step in migrating all the Info types to be POD. We
introduced a shared string saver that can be used safely across threads,
and updated the internal represntation of various data types to use
these over owned strings, like SmallString or std::string.
This also required changes to YAMLGenerator to keep the single quoted
string formatting and to update the YAML traits.
This change gives an almost 50% reduction in peak memory when building
documentation for clang, at about a 10% performance loss. Future patches
can mitigate the performance penalties, and further reduce memory use.
| Metric | Baseline | Prev | This | Culm% | Seq% |
| :--- | :--- | :--- | :--- | :--- | :--- |
| Time | 920.5s | 920.5s | 1011.0s | +9.8% | +9.8% |
| Memory | 86.0G | 86.0G | 44.9G | -47.8% | -47.8% |
[22 lines not shown]
Revert "[SimplifyCFG] Extend jump-threading to allow live local defs … (#190269)
…(#135079)"
This reverts commit a757f23404c594f4a48b4ddb6625f88b349d11d5. Commit
causes reduce.cu file in hipcub/warp go from 2 minutes of compilation to
taking several hours.
ena: Minor changes
1. Move parenthesis to correct place in switch and fix include order
2. Add comment at the end of an ifdef for clarity
3. Change include order.
Reviewed by: cperciva
Differential Revision: https://reviews.freebsd.org/D55696
Sponsored by: Amazon, Inc.
(cherry picked from commit 2667a8454cff5896c7b467c78cd4ace5ad40f5eb)
ena: Verify that an ENA ring is in netmap only in native mode
netmap operates in two modes:
1) Emulated - netmap handling is done by the network stack, the
NIC driver operates transparently to netmap.
2) Native - netmap management is done by the NIC driver.
When checking whether a specific ENA ring is running in netmap
mode, only the following checks were done:
1. IFCAP_NETMAP - Check whether netmap capability is enabled on
the device.
2. NKR_NETMAP_ON - Check whether netmap is actively using this
ring.
The above checks implied that the netmap mode is native and the
ENA driver needs to handle the netmap logic.
The code was missing an explicit check on whether native mode
is actually on (NAF_NATIVE).
This led to a case where though emulated mode was used and
[19 lines not shown]
ena: Update driver version to v2.8.2
Bug Fixes:
* Verify that an ENA ring is in netmap only in native mode
Minor Changes:
* Move parenthesis to correct place in switch
* Add comment
* Reorder define
Reviewed by: cperciva
Differential Revision: https://reviews.freebsd.org/D55698
Sponsored by: Amazon, Inc.
(cherry picked from commit 96c5eaf0ac6b98d0832e1037d672064de43a7e00)
ena: Verify that an ENA ring is in netmap only in native mode
netmap operates in two modes:
1) Emulated - netmap handling is done by the network stack, the
NIC driver operates transparently to netmap.
2) Native - netmap management is done by the NIC driver.
When checking whether a specific ENA ring is running in netmap
mode, only the following checks were done:
1. IFCAP_NETMAP - Check whether netmap capability is enabled on
the device.
2. NKR_NETMAP_ON - Check whether netmap is actively using this
ring.
The above checks implied that the netmap mode is native and the
ENA driver needs to handle the netmap logic.
The code was missing an explicit check on whether native mode
is actually on (NAF_NATIVE).
This led to a case where though emulated mode was used and
[19 lines not shown]
