[SCEV] Account for use flags in impliesPoison. (#244268)
Use flags only hold where the expression is used, so an exit count with
use flags may be poison when expanded in the preheader, even if the
earlier exit is taken first. Conservatively return false from
impliesPoison for non-canonical uses, keeping them in the umin_seq.
PR: https://github.com/llvm/llvm-project/pull/243812
[mlir][linalg] Scaled contract decomposition (#243872)
Adds decomposition of linalg.scaled_contract into a low-precision
contraction of the unscaled inputs followed by scaling and accumulation
of the partial results. Block scaled dimensions are expanded at scaling
block boundaries, which removes semi-affine indexing maps where
possible.
Also, fixes the op's semantics for scales which the scaling extension
truncated to f8E8M0.
Some quantization schemes deliberately use higher precision scales.
These are now converted to the output type and
multiplied with the input data instead, preserving their precision.
Decomposition follows the OCP MX dot product definition and allows
preserving a separate low-precision contraction toward an efficient
lowering path.
Assisted-by: Claude
[CVP][LVI] Allow inserting freeze to eliminate min/max in select expressions (#240166)
Fot this motivating expression...
```
v = select (m < 0), (..., (min(m, 0))), (other)
^ min is redundant, can just be "m"
```
...we haven't previously been folding it in CVP without a guarantee that `m` is
not undef (i.e. attribute noundef). This PR lifts that restriction, and allows
removing the redundant min operation, and if needed, inserts a freeze.
Also see <https://reviews.llvm.org/D151295>.
Related to #239566.
PR: 240166
Assisted-by: Claude
[X86] lowerShuffleAsBitMask - lower to AND(FREEZE(X),MASK) for blend-with-zero shuffle patterns. (#243097)
Similar to #243178 - ensure we freeze the input to prevent AND(poison,0)
-> poison folds.
X86 has far too many of these mask-with-zero/allones folds that ignore
poison propagation and we're starting to see a lot of regression reports
now that SimplifyDemandedBits/VectorElts is handling UNDEF/POISON/FREEZE
more aggressively.
Until #152107 (#149798) lands we're stuck with a few annoying
regressions, but I'd rather avoid adding temporary workarounds.
[clang][NFC] Remove stale disabled ext-vector comparison test (#243510)
The `#if 0` block in `test6` has been disabled since 2009 because vector
comparisons could crash codegen. That was fixed in r76726
([`5ac69057c000`](https://github.com/llvm/llvm-project/commit/5ac69057c000)),
which changed `ScalarExprEmitter::EmitCompare` to use `isFPOrFPVector()`
and added the unguarded `test8` to this same file.
`test8` already checks the same float4 comparisons with exact `fcmp`
expectations, so this older block is redundant. It also contains a stale
`a < b` duplicate where `test8` has `a > b`.
Remove the disabled block. No functional change; coverage remains in
`test8`.
[VPlan] Clamp the AVL to the maximum safe distance via umin. (#243812)
Use umin to clamp maximum AVL by the safe dependnece distance instead of
select. This removes an unnecessary select without profile info, and in
some cases umin is also cheaper than compare/select.
PR: https://github.com/llvm/llvm-project/pull/243812
[LV] Support reductions used on uncountable early exits (#233139)
Compute the value of a reduction on an early exit directly from the
vector accumulators in the vector early exit block: lanes up to and
including the first active lane of the early exit condition use the
updated accumulator, the remaining lanes the accumulator of the previous
iteration:
```
vector.early.exit:
first.active.lane = first-active-lane ...
mask = active-lane-mask 0, first.active.lane + 1
blend = select mask, rdx.next, rdx.phi
result = compute-reduction-result blend
```
Which reductions used on early exits are supported is decided in VPlan
when handling uncountable early exits, based on the reduction phi
recipes.
[8 lines not shown]
[LoopUtils] Check the first outer iteration for co-moving pointer groups
When runtime checks are hoisted out of the outer loop, the range of each
pointer group is widened to cover all iterations of the outer loop. If both
groups of a check advance by the same step in the outer loop, e.g. two rows
of the same image:
```cpp
for (y = 0; y < H; y += 2)
for (x = 0; x < n; x++) {
img[y*S + 3*x] = ...;
img[(y+1)*S + 3*x] = ...;
}
```
the widened ranges overlap as soon as the outer loop runs twice, so the
check always fails and the vector loop never runs.
If all bounds of both groups are affine AddRecs of the outer loop with the
[7 lines not shown]
[LoopUtils] Factor out the same-step check of runtime check bounds (NFC)
The follow-up change uses four SCEV expressions for this type of change
so it also generalizes it to take an array.
[LoopUtils][LV] Compute runtime check bounds per check (NFCI)
getRuntimeCheckBounds takes a single pointer group, but the bounds to use
for one group can depend on the group it is checked against.
This changes RuntimePointerCheck to include bounds of both of its groups.
RuntimeCheckBounds takes the new role of covering both groups and
RuntimeCheckGroupBounds covering single group bounds.
The per-group logic moved unchanged into a static helper.
In VPlan's createRuntimeChecks, compute and expand the bounds of each check
right before its compares, in a single loop over the checks, instead of
caching the expanded bounds per group.
We can do it because VPSCEVExpander already reuses expansions of the same SCEV,
and VPlan's CSE merges the resulting duplicate freezes.
As the result, this change only affects the order of the instructions in
the runtime checks.
[ValueTracking] Propagate demanded lanes through ADD in sign-bit queries (#244293)
Use the demanded lanes when computing the sign bits of the left operand.
This matches the behavior in SDAG.
Assisted by: GPT 6.1 Sol
[X86] Split WriteBlend scheduler classes into WritePBLENDW/WritePBLENDD classes (#240940)
(V)PBLENDW + VPBLENDD instructions usually have very different scheduler
characteristics - so don't try to get them to share a class and then
have to override most of it
Allows us to remove all PBLEND overrides
LunarlakeP - the instruction override had missed the BLENDWrri
instruction
znver1/2 - not sure the pipe usage is correct (FPU013? why the extra AGU
uop?), but I've kept for now - this can be investigated in a future
patch.
Fixes #240408
[TableGen] Remove unused member functions and variables (NFC) (#244515)
This patch removes unused member functions and struct member variables
across TableGen that have no callers or references.
Assisted-by: Antigravity
[Coroutines] Fix a warning (NFC) (#244544)
This patch fixes:
llvm/include/llvm/Transforms/Coroutines/SuspendCrossingInfo.h:69:13:
error: private field 'F' is not used
[-Werror,-Wunused-private-field]
Assisted-by: Antigravity
[Hexagon] Drop REQUIRES on HLX return inreg test (#244497)
The test drives clang_cc1 with an explicit -triple hexagon, so it does
not need the hexagon-registered-target feature to be gated on.
Assisted by AI
[LAA] Keep checking dependences until unsafe. (#230265)
Once dependence recording stops, areDepsSafe returned at the first
non-safe dependence. If that was Unknown, LAA retried with runtime
checks without seeing later unsafe dependences between accesses through
the same pointer, which runtime checks cannot catch.
Only return early once the status is Unsafe, to fix miscompile.
PR: https://github.com/llvm/llvm-project/pull/230265
[runtimes][offload] Add missing dependencies to check-offload (#235643)
After configuring an OpenMP/offload build, running `ninja check-offload`
without first running the build doesn't build all the necessary
dependencies like flang or clang-linker-wrapper, or the runtimes.
Claude assisted by iteratively trying to run check-offload and adding
dependencies as they were needed.