[MLIR][CMake] Fix runtime libraries with PCH (#182850)
Some MLIR libraries are intended to be dlopen-ed, but currently all MLIR
libraries link against LLVMSupport. After the recent PCH introduction,
this causes these libraries to implicitly use the LLVMSupport PCH, which
results in the definition of llvm::*ABIBreakingChecks, which results in
a ODR violation when loaded with dlopen.
Conceptually, libraries that are designed to be dlopen-ed should not
simply link against LLVM libraries in non-dylib builds for this reason.
(This apparently was a problem before with mlir_apfloat_wrappers.)
To fix builds, remove LLVMSupport from runtime libraries that don't need
it and, as a workaround, disable PCH for libraries that are in a weird
state (use LLVMSupport but happen to not export symbols currently).
[NFCI] Make all SI_KILL* convergent (#183100)
Add convergent property to SI_KILL*TERMINATOR. Now all SI_KILL* are
convergent. SI_KILL*TERMINATOR were already terminators so they could
not be sunk by machine-sink. Thus, this is probably a NFC.
Signed-off-by: John Lu <John.Lu at amd.com>
[InstCombine] Replace alloca with undef size with poison instead of null
When an alloca instruction has an undef (or poison) array size, InstCombine
was previously replacing all uses of the alloca with a null pointer. This
caused invalid IR when the alloca was used by @llvm.lifetime intrinsics.
According to the @llvm.lifetime intrinsic specification, the pointer
argument must be either:
- A pointer to an alloca instruction, or
- A poison value
Since null is neither an alloca pointer nor poison, the previous
transformation violated the intrinsic's requirements and produced
invalid IR.
Fix by replacing the alloca with a poison value instead of null, which
satisfies the @llvm.lifetime requirements and produces valid IR.
[VPlan] Supported conditionally executed single early exits. (#182395)
Add support for a single early exit that is executed conditionally. To
make sure the mask from any non-exiting control flow is combined with
the early exit condition.
To do so, introduce a MaskedCond VPInstruction, which is inserted as
user of the early-exit condition, at the point of the early-exit branch.
The VPInstruction will get masked automatically if needed by the
predicator, ensuring that we properly account for it when checking
whether the early exit has been taken.
Note that this does not allow for instructions that require predication
after the early exit. This requires additional work in progress:
https://github.com/llvm/llvm-project/pull/172454
As an alternative to MaskedCond, we could also predicate before handling
early exiting blocks: https://github.com/llvm/llvm-project/pull/181830
PR: https://github.com/llvm/llvm-project/pull/182395
[LLVM][TTI] Remove the isVScaleKnownToBeAPowerOfTwo hook. (#183292)
After https://github.com/llvm/llvm-project/pull/183080 this is no longer
a configurable property.
NOTE: No test changes expected beyond
llvm/test/Transforms/LoopVectorize/scalable-predication.ll which has
been removed because it only existed to verfiy the now unsupported
functionality.
[clang-tidy] Fix erroneous warning to make deleted function public (#182577)
This PR fixes #54276 and fixes #135249 by only matching private deleted
functions with a public overload or special member functions.
[CIR][AArch64] Add lowering + tests for predicated SVE svdup_lane builtins
This PR adds CIR lowering + tests for SVE `svdup_lane` builtins on
AArch64. The corresponding ACLE intrinsics are documented at:
https://developer.arm.com/architectures/instruction-sets/intrinsics
[SPIRV] Implement Gather and GatherCmp intrinsics (#182578)
This commit implements the intrinsics needed to represent the texture
Gather* instructions in HLSL.
Assisted-by: Gemini
[CIR] Implement compare exchange with dynamic failure ordering (#183110)
In #156253, we implemented the rest of this feature, with compile time
constant failure ordering. This patch follows the incubators direction
(with a little cleanup based on other cleanup that we do) to replace
this situation with a 'switch'.
fixup! Move code to `AArch64ExpandPseudoInsts` and `getTgtMemIntrinsic`
Move code to `AArch64ExpandPseudoInsts` and `getTgtMemIntrinsic`
and use tablegen pattern for intrinsic, plus other small review changes.
[MLIR][Python] Fix typeid support for DynamicType and DynamicAttr (#183076)
Previously, we were using the static `typeid` of `DynamicType` for
checks, which is incorrect. We should instead check against the `typeid`
of `DynamicTypeDefinition` (which is a subclass of `SelfOwningTypeID`),
and register it via `register_type_caster` so that Python-defined types
can use `maybe_downcast`. (The attribute part is same.)
[InstCombine] Replace alloca with poison size using poison instead of null
When an alloca instruction has an undef (poison) array size, InstCombine
was previously replacing all uses of the alloca with a null pointer. This
caused invalid IR when the alloca was used by @llvm.lifetime intrinsics.
According to the @llvm.lifetime intrinsic specification, the pointer
argument must be either:
- A pointer to an alloca instruction, or
- A poison value
Since null is neither an alloca pointer nor poison, the previous
transformation violated the intrinsic's requirements and produced
invalid IR.
Fix by replacing the alloca with a poison value instead of null, which
satisfies the @llvm.lifetime requirements and produces valid IR.
[LLVM] Refine MemoryEffect handling for target-specific intrinsics (#155590)
This patch improves memory alias analysis between calls if they change
inaccessible or target memory locations. The results is
computed by comparing each location ModRefInfo between the calls.
[LAA] NFC: Rename mulSCEVOverflow to mulSCEVNoOverflow (#183096)
The function returns nullptr when the multiplication WOULD overflow,
matching the semantics of its sibling addSCEVNoOverflow. The old name
reads as if the function multiplies with overflow, which is the opposite
of what it does.
[InstCombine] Replace alloca with poison size using poison instead of null
When an alloca instruction has an undef (poison) array size, InstCombine
was previously replacing all uses of the alloca with a null pointer. This
caused invalid IR when the alloca was used by @llvm.lifetime intrinsics.
According to the @llvm.lifetime intrinsic specification, the pointer
argument must be either:
- A pointer to an alloca instruction, or
- A poison value
Since null is neither an alloca pointer nor poison, the previous
transformation violated the intrinsic's requirements and produced
invalid IR.
Fix by replacing the alloca with a poison value instead of null, which
satisfies the @llvm.lifetime requirements and produces valid IR.
[MLIR][NVVM] Fix kFactor for fp8/fp6/fp4 types in MmaSpOp verifier. Improve mma tests. (#183133)
Fix an incorrect kFactor value for e4m3/e5m2, e3m2/e2m3, e2m1 types in
MmaSpOp::verify(). The kFactor for these types was set to 32 but should
be 16.
kFactor is used to compute the expected number of operand A/B register
fragments. With kFactor=32 (wrong) and the only allowed shape m16n8k64,
the fragment count was incorrect. With kFactor=16 (correct), it matches
the PTX ISA definition for mma.sp with fp8/fp6/fp4 A/B operands.
PTX ISA reference:
[https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#warp-level-matrix-instructions-sparse-mma](https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#warp-level-matrix-instructions-sparse-mma)
Also improve existing MLIR dialect tests for nvvm.mma.sp.sync and add
new mlir-translate tests covering mma, mma.sp, and blockscale variants.
[InstSimplify] Add support for llvm.structured.gep (#182874)
Similar to GEP, the SGEP instruction with no indices can be simplified
by directly using the base pointer.
[WebAssembly] optimize ext + shuffle + add into addext (#182849)
cc https://github.com/llvm/llvm-project/issues/179143
This adds a second pattern: we already recognize "shuffle + extend +
add" as `addext`, this adds another pattern for "extend + shuffle +
add", which can come up when programs are optimized.
[SPIRV][AMD] Reenable `SPV_KHR_float_control2` for AMD flavored SPIRV (#182873)
`SPV_KHR_float_controls2` is enabled in the translator after
https://github.com/khronosgroup/spirv-llvm-translator/pull/3475.
This extension was disabled since we were not able to translate it back.
This patch reverts #169659.
Unify expandPow2Division/expandPow2Remainder into expandPow2DivRem.
Merge the two functions into one to share the common signed-path logic (freeze, bias, ashr) and reduce code duplication, as suggested.
[CodeGen] Expand power-of-2 div/rem at IR level in ExpandIRInsts.
Previously, power-of-2 div/rem operations wider than
MaxLegalDivRemBitWidth were excluded from IR expansion and left for
backend peephole optimizations. Some backends can fail to process such
instructions in case we switch off DAGCombiner.
Now ExpandIRInsts expands them into shift/mask sequences:
- udiv X, 2^C -> lshr X, C
- urem X, 2^C -> and X, (2^C - 1)
- sdiv X, 2^C -> bias adjustment + ashr X, C
- srem X, 2^C -> X - (((X + Bias) >> C) << C)
Special cases handled:
- Division/remainder by 1 or -1 (identity, negation, or zero)
- Exact division (sdiv exact skips bias, produces ashr exact)
- Negative power-of-2 divisors (result is negated)
- INT_MIN divisor (correct via countr_zero on bit pattern)
Addressed review comments:
- Added proofs for power-of-2 div/rem expansion in ExpandIRInsts at
https://alive2.llvm.org/ce/z/Y-iWm-
- Tests updated as requested.
Also added CreateFreeze() where needed.
