AMDGPU: Mark dead carry-out when rewriting scalar carry op to VALU (#225738)
If an operation is expanded with a dead scc def, the resultant vcc def
will also be dead, so preserve the dead flag. Reduces implicit reliance on
LiveVariables recomputing dead flags later.
Co-authored-by: Claude (Opus 4.8) <noreply at anthropic.com>
[AMDGPU] Fix assert on empty inline asm register constraint (#225743)
An empty register name reaches this parser from user source, where
StringRef::front() asserts
[clang] Migrate away from PointerUnion::dyn_cast (NFC) (#225614)
Note that PointerUnion::dyn_cast has been soft deprecated in
PointerUnion.h:
// FIXME: Replace the uses of is(), get() and dyn_cast() with
// isa<T>, cast<T> and the llvm::dyn_cast<T>
Literal migration would result in dyn_cast_if_present (see the
definition of PointerUnion::dyn_cast), but this patch uses dyn_cast on
UnexpandedParameterPack::first because it is always nonnull.
Specifically, UnexpandedParameterPack is constructed only in the
following places, all of which store a nonnull pointer in first, and
first is never modified afterward:
- CollectUnexpandedParameterPacksVisitor::addUnexpanded in
SemaTemplateVariadic.cpp
- TransformLambdaExpr in TreeTransform.h
- TransformSizeOfPackExpr in TreeTransform.h
Assisted-by: Antigravity
[CostModel][X86] Add masked memory test coverage for VBMI2 capable CPUs (#225790)
Add tigerlake and znver4/5 coverage - znver4 is going to be a pain as it has awful compressstore perf :(
Prep work for #214613
databases/galera26: Update to 26.4.27
Reset MAINTAINER due to multiple previous timeouts.
PR: 298182
Approved by: devel at galeracluster.com (maintainer timeout)
[SCEV] Fix sign bits of pointers wider than their index type. (#225786)
ComputeNumSignBits counts bits over the full pointer width. If all sign
bits are in the high bits, the code previously did not adjust the number
of signed bits. For narrow index types, this could result in the number
of sign bits > than the narrow bitwidth triggering an assert in APInt
(test example has 64 bit pointer with 16 bit index width.
If all sign bits are in the high bits outside the index width, they do
not add any information for the index, just use 1 in that case.
Fixes a crash in the added tests.
PR: https://github.com/llvm/llvm-project/pull/225786
[AMDGPU][InstCombine] Fold zero dot operands to accumulator
Fold AMDGPU dot intrinsics when either operand is zero.
`dot(a, 0) = 0` and `dot(0, b) = 0`, so replace the intrinsic with its accumulator.
This avoids unrelated clamp and add/sub reassociation cases.
[AMDGPU][InstCombine] Canonicalize dot constant operands
Move constant dot product source operands to the right hand side
and add tests for signed and unsigned dot intrinsics.
[AMDGPU][InstCombine] Fold constant add/sub into the dot accumulator (#225002)
`amdgcn.{s,u}dot{2,4,8}(a, b, C) +/- K -> dot(a, b, C +/- K)` when both
the
accumulator C and K are constants. The new constant is computed with
wrapping APInt arithmetic to match the non-clamping accumulate.
Only applies when clamp is false (the saturating accumulate does not
reassociate) and the dot has a single use. K - dot is left alone since
it
would need the dot product negated.
[AMDGPU] Use synthetic fixtures for generic feature validation tests
Remove the classification test that includes the full AMDGPU target. Real targets are validated when generating AMDGPUTargetParserDef.inc during the build.
Extend the lightweight generic feature tests to cover inherited classifications of backend-only features, support through member-side implications, and unsupported generic-side implied features.
Change-Id: I54dd16affdc93039ce9d2dba9c99e036bd4afe48
Validation: all four AMDGPU target-definition lit tests pass; the enhanced generic feature test takes 3.44 seconds.
[SLP]Try the wider store slice when the leading VF=2 slice fails
With the register VF 2 the instruction count check may reject the
leading slice of a store chain.
Try the double-width slice at the same position first and keep the wider
slices for the rest of the chain once one is vectorized.
Fixes the remaining perf regression from #221717.
Reviewers:
Pull Request: https://github.com/llvm/llvm-project/pull/225823
[AMDGPU] Fold a constant add/sub into the sudot4/sudot8 accumulator
Fold a constant add into the accumulator operand of sudot4 and sudot8 when
clamping is disabled:
```
sudot(a, b, C1, false) + C2 -> sudot(a, b, C1 + C2, false)
```
Subtraction by a constant is canonicalized to addition of its negation.
[SPIRV] Emit NonSemantic DebugValue (#224158)
This adds support for emitting `DebugValue` in
`SPIRVNonSemanticDebugHandler` for `DBG_VALUE` records.
NOTE: Much of the new code was necessary to emit constants and their
types at module scope. I also added a bunch of tests as I started
getting into corner cases.
Since
[`DebugValue`](https://github.khronos.org/SPIRV-Registry/nonsemantic/NonSemantic.Shader.DebugInfo.html#DebugValue)
cannot have forward references, it can only reference already emitted
instructions. `DebugValue` cannot come after merge instructions either,
and SPIR-V requires an id defined in a function block to dominate a
non-phi use. The change has four parts:
1. `analyzeDebugRecords()` resolves placement and availability before
emission begins. Each record stays at its source position unless it lies
between a merge instruction and its terminator, in which case it moves
[16 lines not shown]
[mlir] Avoid rewriting unreachable blocks in the greedy driver
A rewrite can disconnect a block after the iteration's initial CFG sweep.
Track possible reachability changes through rewriter notifications for blocks,
terminators, successors, and region owners. Refresh reachability between
rewrites and skip worklist operations in unreachable blocks until the next
iteration removes them.
Cover inserted and redirected blocks, blocks connected before rewrite
completion, and nested regions.
Assisted-by: Codex
