[mlir][tosa] Remove vector unrolling support for apply_scale (#238010)
The vector unrolling support for `tosa.apply_scale` has no tests or
in-tree users, leaving its intended use unclear. This PR removes that
support ahead of the planned vector unrolling refactor, keeping the
refactor focused on demonstrated uses of the interface.
This PR removes that support ahead of the planned vector unrolling
refactor (https://github.com/llvm/llvm-project/issues/233408), keeping
the refactor focused on demonstrated uses of the interface.
This effectively reverts the following commit:
```bash
commit 81566001e0c8b072fa9d39be7dac1c007889d937
Author: Jakub Kuderski <kubak at google.com>
Date: Mon Feb 13 16:51:37 2023 -0500
[mlir][tosa] Enable `apply_scale` unrolling
```
[RFC][AMDGPU][Doc] Add documentation about ABI occupancy (#224103)
This PR introduces ABI occupancy in the documentation. The actual
implementation will come in follow-up PRs once we agree on the spec.
[AArch64] Combine multiple uses of a interleave (#235669)
When result of vector_interleave operation is used in multiple store
operation each can form stN.
[LLVM][CodeGen][SVE] Custom lower ISD::MASK_BEFOREFIRST. (#237325)
Avoid the need to create predicate during isel, which in turn allows
more predicate reuse opportunities.
AArch64TTIImpl::getIntrinsicInstrCost() has been changed to maintain the
existing costs for llvm.mask.beforefirst.
[DAGCombiner] in `visitEXTRACT_SUBVECTOR`, don't peek through bitcasts when looking for demanded elements (#237411)
This patch fixes superlinear-time iteration when DAGCombiner computes
demanded elements of a vector accessed by multiple `extract_subvector`
nodes. Before the fix, the total time to evaluate a sequence of `E`
extracts of a vector of length `V` is `E * V`. After, it's just `E`.
This was a latent bug exposed after
https://github.com/llvm/llvm-project/pull/204351 landed, because before
this DAGCombiner would bail on the extract as it computed no demanded
elements.
[Hexagon] Enable SlotIndexes compaction by default
Opt Hexagon in to dropping dead SlotIndexes entries before greedy
allocation, as AMDGPU already does, so live range sizes count only
instructions that still exist.
The test updates are register allocation changes; two pipeliner tests
get back their original, shorter kernels.
[GlobalISel] Compare element sizes in minScalarSameAs/maxScalarSameAs (#237753)
`minScalarSameAs()`/`maxScalarSameAs()` emit `changeElementSizeTo()`,
which
replaces the element type while keeping the element count, but their
predicates
compared an element size against the other type's *total* size. For `<4 x s32>`
that is 32 vs 128:
- `maxScalarSameAs()` matched `<4 x s32>` against itself and produced a no-op
mutation, tripping `mutationIsSane()` in the legalizer.
- `minScalarSameAs()` required the other element size to exceed the whole
vector, so it was effectively dead for vector types.
Compare element sizes on both sides instead. Scalars are unaffected:
element
size and total size are the same there.
Found while fixing the vector `G_CTPOP` legalizer abort in #235976;
[7 lines not shown]
[clang][unittests] Check TimeProfilerTest graphs by event order (#237163)
The test rebuilt the scope tree from microsecond timestamps, so a scope
that starts and ends in the tick its predecessor ended in looks like a
child of it. Instead of rebuilding a tree, we check the expected graph
against what the trace encodes exactly. So we avoid any potential
overlaps.
EvaluateKnownConstIntCheckOverflow was made fast enough that it was
happening on some runs of ConstantEvaluationC99 test.
Turns out that ConstantEvaluationCxx20 was disabled for the same reason.
PerformPendingInstantiations was a special case for the wrong case. So
its gone.
This is fixing a flaky CI failure introduced by c467c6b. But the
flakyness was just because of the MS issue, the fix is valid. So fixing
the tests is the right way forward.
[wasm] Swift async ABI p2: Reserve local 0 for the swiftasync argument
To make it possible to always recover the async context in wasm frames,
this patch reserves local 0 so that it is never overwritten.
Previously, two places were clobbering the register:
* Register coloring gave it to other values after the last use of the
swiftasync argument, for example the parent context in await resume
funclets.
* Register coalescing merged the swiftasync argument with copies, for
example when the argument is the initial value of a loop variable.
[ADT] Avoid runtime cast on tagged pointer in PointerIntPair (#238334)
PointerIntPair::getFromOpaqueValue(const void *V) calls
PtrTraits::getFromVoidPointer(V) on the unmasked opaque pointer solely
to trigger a compile-time error when V is a const void * and PtrTraits
only accepts a non-const void *, discarding the returned pointer before
constructing the PointerIntPair from const_cast<void *>(V).
When compiled with Control Flow Integrity enabled for derived and
unrelated casts (-fsanitize=cfi-derived-cast and
-fsanitize=cfi-unrelated-cast) and PointerTy is a polymorphic class
pointer, PtrTraits::getFromVoidPointer(V) performs a runtime static_cast
from const void * to the polymorphic pointer type and executes a CFI
vtable check. If the low integer tag bits of V are non-zero, the tagged
address does not point to a valid vtable and traps at runtime.
Replace the discarded runtime call with a static_assert checking
std::is_convertible_v<decltype(PtrTraits::getFromVoidPointer(V)),
PointerTy>
[6 lines not shown]
[LoongArch] Materialize vector zero-test results via `MOVCF2GR` (#230804)
This generalizes the changes in the previous commit so that all
zero-test results can be consumed directly through `$FCC` when possible,
and materialized via `MOVCF2GR`, rather than branchy blocks, when really
needed. Test expectations are updated accordingly.
I created [a
microbench](https://gist.github.com/CSharperMantle/cc31f0320f15e1fc3985dafa567c5f75)
with LLM assistance, which yielded the following results on my Loongson
3B6000 12C24T (units are in cycle, N=31). On all cases, the new sequence
performs better in cycles than the current ("Old") one.
| Scenario | Old geomean | Old +/- % | Old SD | New geomean | New +/- %
| New SD | New/Old | Welch p | Mann-Whitney U p |
| :-------------------------- | ----------: | --------: | ------: |
----------: | --------: | -----: | ------: | --------: |
---------------: |
| Linear, predicted-taken | 3255107 | +/- 0.06% | 1,971 | 1000235 | +/-
[12 lines not shown]
[libc] Implement close_range in unistd (#236913)
This patch implements the `close_range` entrypoint in LLVM libc under
`unistd` as standardized in POSIX.1-2024 and Linux kernel 5.9+.
- Implements internal syscall wrapper `close_range` with fallback error
handling.
- Adds public entrypoint and headers conforming to Doxygen format.
- Adds unit tests covering standard range closure and error conditions
(`EINVAL`).
- Updates entrypoints across Linux targets (`x86_64`, `aarch64`, `arm`,
`riscv`).
Fixes #236896.
[AArch64] Combine SVE AND and XOR to BIC for _m form
Add patterns to combine the expressions of the following type to BIC
when possible:
```
return svand_u32_m(svptrue_b32(), a, sveor_u32_x(svptrue_b32(), a, b));
```
Cover symmetric case where result of AND is used by EOR for the above
expression.
Cover cases where the inner node is of _u form like sveor_u32_u or
sveand_u32_u.
Cover cases where the inner node is of _m/_z form under matching
predicates or all-active predicates for the inner node.
Also, cover the case where the inner node is a normal LLVM AND(or XOR)
[4 lines not shown]
[ARM] Fix A32 single-register PUSH/POP preferred disassembly aliases
For A32, `POP` of a single register encodes as an LDR. When the code
for this was first committed in 2011 in change 8ba76c6d5, the Arm ARM
only mentioned that `POP` is listed as the "standard syntax".
Later, in 2014, the Arm ARM DDI0487A.e, §F7.1.137 was updated to
specifically say that `POP` "is always the preferred disassembly."
Update the code and tests to reflect the Arm ARM.
[analyzer] Improve node creation hygiene in MPIChecker (#238423)
Properly handle the case when node creation fails, remove a pointless
transition and some pointless state lookups.
Fixes #64647
[analyzer] tryRearrange: don't "cancel out" symbol in BO_Add (#237160)
In the special case when the same symbol appeared on both sides of the
top-level operator (perhaps with some integer offsets), it was cancelled
out... EVEN WHEN THE OPERATOR WAS ADDITION!
E.g. it was rearranging `(x + 1) + (x + 2)` to `3` and `x + x` to zero.
This change skips the "cancel out symbol" step for `BO_Add`, which
ensures that e.g. `(x + 1) + (x + 2)` is rearranged to `(x + x) + 3`.
Cancelling symbols is valid for all other supported operators
(subtraction and comparisons).
This bug was introduced in 2018 by
2bbccca9f75b6bce08d77cf19abfb206d0c3bc2e, but after spending 3 months in
trunk, `tryRearrange` was disabled by default for severe unrelated
(performance) issues. After that
`aggressive-binary-operation-simplification` was just one of the many
basically unsupported analyzer options, so it's not a surprise that
[5 lines not shown]
[AArch64] Skip adding BTI before PACI[AB]SPPC in AArch64BranchTargets (#236164)
PACI[AB]SPPC also has an implicit BTI instruction, so should be skipped
in addBTI similar to PACI[AB]SP.
AArch64BranchTargetsImpl is also extended to replace the signing
instruction with PACNBI[AB]SPPC, to prevent a bti instruction moving
to the middle of the prologue.
[clang][OpenMP] Fix crash on `linear(uval)` with a reference parameter (#237754)
Fixes #108166
`linear(uval(x))` crashed when `x` is a reference parameter. For `uval`,
Sema read the initial value from the reference's initializer instead of
the reference, and a parameter has none, so a null `Expr` reached
`DefaultLvalueConversion`. Even with an initializer this was wrong: a
defaulted `int &x = g` made the loop update `g`, and in a `parallel`
region or a lambda it could name an uncaptured variable.
Drop the shortcut in `ActOnOpenMPLinearClause` and
`FinishOpenMPLinearClause`, so a `uval` item is read from and written
back through the reference, like `val`. That is what the spec says for
loop constructs, where `uval` only adds the requirement that the item be
a reference, which is still checked.
AI Assisted!
[UpdateTestChecks] Handle lit's %if in update_cc_test_checks.py (#238255)
Handle run lines containing lit `%if`s by ignoring the condition and
unwrapping, similar to other update scripts.
[AMDGPU] Infer mul24 known bits with unknown operands
Do not stop GlobalISel known-bits analysis when a mul24 operand has no known bits. Truncating to 24 bits and extending to the result width can still establish useful product bounds through the shared helper added by #236738.
Cover both operand orders, ignored high bits, zero and overflow boundaries, and i64 results without limiting known bits to the i32 operand width.
[lldb-dap] Don't collect module statistics for the statistics event (#236413)
Use the stats options to build GetStatistics to avoid computing data
that lldb-dap drops.
[KnownBits] Cover negative multiplication boundaries
Exercise both operand orders and i8, i32, i65 and i128 results. Preserve cases where zero or signed and unsigned wraparound prevent leading-bit inference.
[OpenMP] Build the metadirective AST node for runtime selection. (#236432)
Follow-up to #192455 (Phase 3 of the agreed plan): with
`-fopenmp-metadirective-runtime-selection`, Sema now builds an
`OMPMetaDirective` holding every variant instead of returning the first
one.
**Known limitation**: the node has no associated statement, so
`-Wuninitialized` and `-Wunreachable-code` skip the variant bodies. This
is covered by `metadirective_runtime_flow_warnings.cpp` and will be
fixed in a follow-up.
[KnownBits] Improve multiplication bounds for negative operands
Bound products using operand magnitudes when both operands are negative or one is negative and the other is strictly positive. Retain unsigned overflow checks before inferring leading zeros or ones.
Cover both cases with AArch64 IR tests and update the affected AMDGPU SelectionDAG shift checks.
[Offload][Omp] Split plugin diagnostics into detection & messaging
The plugin interface tracks a allocations and kernels (when TrackAllocationTraces
and TrackNumKernelLaunches are enabled) to emit diagnostic messages.
These messages currently contain OpenMP specific data that needs to be carried
through the Plugin Interface.
This PR splits this support in two pieces: tracking and detection that is
kept inside the plugins, and reporting of the error which is done through
a handler set from libomptarget.
A new diagnostic handler type is added, and the API to associate it to a device
are added to liboffload.
This will allow to handle the OpenMP specific data fromb the libomptarget handler
and remove it from the Plugin Interface (not done in this PR).
[Offload][Omp] Allocate and free device memory through liboffload
Remove access to the PluginContext from libomptarget by providing two helpers
to allocate and free memory.
