[Clang][OpenMP] Canonicalize Intra-tiles in Loop Tiling (#191114)
This PR canonicalizes the intra-tile loop in OpenMP loop tiling.
Take a simple example:
```cpp
#pragma omp tile sizes(4)
for (int i = 0; i < 6; ++i)
body(i);
```
`clang` turns the above into **2 nested loops**:
- Outer (aka *floor*) loop iterating in steps of 4.
- Inner (aka *intra-tile*) loop iterating over the tiles — `(0, 1, 2,
3)` and `(4, 5)` — since 6 is not a multiple of 4, this results in the
last **partial tile**.
[42 lines not shown]
[SPARC] Preserve inline asm relocation modifiers (#219971)
Make sure that inline assembly that references a global symbol gets the
correct relocation.
sethi %hi(fsr_storage), %i0
st %fsr, [%i0+fsr_storage]
ld [%i0+fsr_storage], %i0
-->
sethi %hi(fsr_storage), %i0
st %fsr, [%i0+%lo(fsr_storage)]
ld [%i0+%lo(fsr_storage)], %i0
(cherry picked from commit cf584bda4c3434086f7fdfebd1c0ce8ecb9423bb)
release/23.x: Backport '[Clang] Fix C++26 fold expression normalization of PackIndexingExpr'
It turns out that PackIndexingExpr doesn't create any PackExpansionTypes
for unexpanded packs and thus we don't have to remove the packs during the normalization.
[LoongArch] Add omitted LASX patterns for vector extend (#219351)
Adds omitted 128-bit to 256-bit patterns for `sign_extend_vector_inreg`,
including `v16i8 -> v4i64` and `v8i16 -> v4i64`, which will generate by
the combination of `icmp + or/and/xor + zext/sext`, all related tests
are added.
Fix: https://github.com/llvm/llvm-project/issues/219224
(cherry picked from commit 715901096defbb408d9fb1a6f7e1959938d950c8)
[AArch64] Widen v3i8/v4i8 VECTOR_MATCH needles (#218972)
The default legalization behaviour for v3i8 and v4i8 types is to promote
them to v4i16. This currently results in VECTOR_MATCH operations with
needles of v3i8/v4i8 needles being expanded for AArch64.
It's generally preferable to widen the needle instead (especially if the
needle is a constant vector, which becomes an immediate splat). For SVE,
this can fold multiple vector compares into a dup + match.
[LoopInterchange] Bail out on reductions with uses outside their chain. (#220263)
getMinMaxRecurrence() can return a descriptor for a min/max phi that has
users outside of the reduction chain, e.g. the comparison of an
argmin/argmax idiom. Such descriptors have no loop exit instruction, and
the extra uses make the order in which the elements are visited
observable.
Bail out for such reductions, matching what
canParallelizeReductionWhenUnrolling() already does.
This fixes a crash on the added test cases.
PR: https://github.com/llvm/llvm-project/pull/220263
[ConstraintElim] Derive IV bound from an relation predicate in latch. (#220107)
A loop whose latch has a relational predicate of the IV post-increment
as exit condition steps *past* the bound rather than landing on it, so
the
existing EQ/NE step reasoning does not apply.
But the condition (IV+Step Pred N) holds for the induction phi each
iteration after the first (when the header is entered via the guarded
latch).
In the first iteration, the induction phi's value is its start value.
Together with a precondition (Start Pred N), (IV Pred N) hold in each
iteration.
Add this as fact enables removing a few redundant conditions in
practice:
https://github.com/dtcxzyw/llvm-opt-benchmark-nightly/pull/1148.
[4 lines not shown]
[CIR][NFC] Generate getSuccessorRegions for fixed-region ops
Six of the ten CIR ops implementing RegionBranchOpInterface reported a fixed
list of entry regions and the parent operation on every region exit, which
makes the whole method derivable from the region accessor names. Add a
CIR_FixedRegionBranchOp class that appends the generated definition to the one
inherited from CIR_RegionBranchOpBase, retarget the six ops onto it and delete
their hand-written definitions.
IfOp, GlobalOp, TryOp and AwaitOp stay on the base class, since their entry
edges are not a fixed list: IfOp falls back to the parent when the else region
is empty, GlobalOp skips its optional ctor and dtor regions, TryOp iterates
variadic handler regions, and AwaitOp routes ready to resume and suspend.
The generated CIROps.h.inc is unchanged and CIROps.cpp.inc gains exactly the
six definitions removed from CIRDialect.cpp.
[CIR][NFC] Share getSuccessorInputs across region-branch ops
The ten CIR ops implementing RegionBranchOpInterface each hand-wrote
getSuccessorInputs, and all ten bodies were equivalent: regions take no
inputs, and returning to the parent yields the parent's results. Three did
not look equivalent but are: CleanupScopeOp and CoroBodyOp returned an empty
ValueRange unconditionally and declare no results, and AwaitOp returned
region block arguments but carries NoRegionArguments, so those ranges are
always empty.
Add a CIR_RegionBranchOpBase ODS class that declares the method and generates
the single shared body through extraClassDefinition, mirroring the existing
CIR_LoopOpBase, and retarget all ten ops onto it.
The generated CIROps.h.inc is unchanged and CIROps.cpp.inc gains exactly the
ten definitions removed from CIRDialect.cpp.
[CIR] Add RegionBranchOpInterface unit tests and fix cir.await successors
Five of the ten ops implementing RegionBranchOpInterface have no unit test
coverage: cir.case, cir.cleanup.scope, cir.global, cir.await and
cir.coro.body. Add tests for all five.
Covering cir.await exposes a disagreement with its own terminator.
cir.condition terminates the ready region and reports {resume, suspend} as
its successors when the parent is an await, but AwaitOp::getSuccessorRegions
listed all three regions as entry successors and reported the parent op as
the successor of every region exit. Fix it to match cir.condition: ready is
the only entry successor, exiting ready branches to resume or suspend, and
exiting suspend or resume returns to the parent operation.
cir.await declares no results and carries NoRegionArguments, so successor
operand and input counts stay at zero along every edge and the MLIR verifier
is unaffected.
[lldb] Fallback to `__clang_vtable` for robust dynamic type resolution (#210584)
`ItaniumABIRuntime::GetTypeInfo(...)`'s type resolution relies on the
demangler to locate the type corresponding to a vtable address. As
reported in #182762 this can break when the demangler and debug info
don't agree on the spelling of the type's name.
This PR implements an lookup based on a support member emitted by clang
added in #130255 as `_vtable$`, later renamed to `__clang_vtable` in
#183617. If the demangling based approach fails we fall back to the new
route.
The initial implementation aims to only support the modern spelling with
DWARF.
[SelectionDAG] powi and ldexp libcalls deal with signed int arguments (#213673)
While investigating a sign extension problem on an off-tree target that
needs extension of 32-bit integer argument to 64-bit register when
passed as an argument, I found that `__powisf2` was being passed
zero-extended despite the argument being a signed `int`. I narrowed this
down to the fact that `shouldSignExtendTypeInLibCall` was being passed
`false` for `IsSigned` for the `int` argument in these libcalls.
Other libcalls (not an exhaustive review) such as the `XINT_TO_FP`
family do set the `IsSigned` value here.
I utilized the s390x target to write a test for this.
[RISCV] Teach the VL Optimizer about Zvzip instructions (#214405)
Set ElementsDependOn = EltDepsNone on the five Zvzip instructions
(VZIP_VV, VUNZIPE_V, VUNZIPO_V, VPAIRE_VV, VPAIRO_VV) so the pass no
longer conservatively treats their result elements as depending on VL
and Mask (beyond masking of inactive lanes).
Wire the opcodes into the VL Optimizer's operand model:
- getOperandLog2EEW: all five instructions return MILog2SEW (source
and destination EEW both equal SEW).
- getOperandInfo: for VZIP_VV, widen the destination and passthru
EMUL by 2x since it interleaves two LMUL inputs into one 2xLMUL
result. For VUNZIPE_V / VUNZIPO_V, widen the EMUL of the vs2
source operand by 2x instead, as the split consumes a 2xLMUL
vector and produces an LMUL even/odd half. VPAIRE_VV / VPAIRO_VV
leave EMUL unchanged on every operand.
Assisted-by: TRAE CLI (Seed-2.1-Turbo)
[ORC] Register SPS run-as function wrappers with SelfEPC (#220224)
#213265 replaced the direct `ExecutorProcessControl::runAsVoidFunction`
and `runAsIntFunction` calls in `COFFPlatform` and
`COFFVCRuntimeSupport` with SPS call proxies. The target-process
bootstrap registers the corresponding executor-side wrappers for
`SimpleRemoteEPC`, but SelfEPC does not register them. In-process
clients using these paths therefore fail to resolve:
```text
orc_rt_ci_sps_call_int32_void
orc_rt_ci_sps_call_int32_int32
```
This patch factors registration of the two existing run-as function
wrappers out of the complete target-process bootstrap set and adds that
narrow set to SelfEPC.
SelfEPC already provides in-process memory access directly, so it does
[14 lines not shown]
[RISCV] Fix result type checks in the VWABDA DAG combine (#220497)
Fix the result element type check in `performVWABDACombine`. The result
of the widening add operation has already been widened, so the expected
element types are `i16` and `i32`, rather than `i8` and `i16`. The
previous check prevented the i16-i32 form from being combined.
[CIR][NFC] Generate getSuccessorRegions for fixed-region ops
Six of the ten CIR ops implementing RegionBranchOpInterface reported a fixed
list of entry regions and the parent operation on every region exit, which
makes the whole method derivable from the region accessor names. Extend
CIR_RegionBranchOpBase with an entryRegions parameter that generates it and
delete the six hand-written definitions.
IfOp, GlobalOp, TryOp and AwaitOp keep theirs, since their entry edges are not
a fixed list: IfOp falls back to the parent when the else region is empty,
GlobalOp skips its optional ctor and dtor regions, TryOp iterates variadic
handler regions, and AwaitOp routes ready to resume and suspend.
The generated CIROps.h.inc is unchanged and CIROps.cpp.inc gains exactly the
six definitions removed from CIRDialect.cpp.
[clang][CIR][AArch64] Add lowering for SISD variants of rounding intrinsics (#219241)
This PR adds lowering for the remaining SISD "Vector saturating rounding
shift right and narrow " intrinsic:
* https://arm-software.github.io/acle/neon_intrinsics/advsimd.html#vector-saturating-rounding-shift-right-and-narrow
The following builtins are added:
* __builtin_neon_vqrshrns_n_s32:
* __builtin_neon_vqrshrns_n_u32:
* __builtin_neon_vqrshrnh_n_s16:
* __builtin_neon_vqrshrnh_n_u16:
* __builtin_neon_vqrshruns_n_s32:
* __builtin_neon_vqrshrunh_n_s16:
The corresponding tests are moved from:
* clang/test/CodeGen/AArch64/
to:
* clang/test/CodeGen/AArch64/neon/
[2 lines not shown]
[MergeFunctions] Intersect poison-generating flags (#220015)
Instead of comparing the flags in FunctionComparator, intersect them in
MergeFunctions. This both a) fixes miscompiles where we failed to check
the poison flags for GEP and FMF and b) allows more function merging, in
cases where the functions only differ in flags.
This can be extended to combine instruction-level metadata as well in
the future.
Fixes https://github.com/llvm/llvm-project/issues/219664.
[CIR][NFC] Share getSuccessorInputs across region-branch ops
The ten CIR ops implementing RegionBranchOpInterface each hand-wrote
getSuccessorInputs, and all ten bodies were equivalent: regions take no
inputs, and returning to the parent yields the parent's results. Three did
not look equivalent but are: CleanupScopeOp and CoroBodyOp returned an empty
ValueRange unconditionally and declare no results, and AwaitOp returned
region block arguments but carries NoRegionArguments, so those ranges are
always empty.
Add a CIR_RegionBranchOpBase ODS class that declares the method and generates
the single shared body through extraClassDefinition, mirroring the existing
CIR_LoopOpBase, and retarget all ten ops onto it.
The generated CIROps.h.inc is unchanged and CIROps.cpp.inc gains exactly the
ten definitions removed from CIRDialect.cpp.
[SandboxVec][NFC] Hoist getInsertPointAfterInstrs into VecUtils (#215863)
Move BottomUpVec.cpp's file-local getInsertPointAfterInstrs() into
VecUtils. NFC.
Co-authored-by: Claude Opus 5 <noreply at anthropic.com>
[orc-rt] Add ORC_RT_HAS_BUILTIN, stop defining __has_builtin (#220503)
orc-rt/support/Compiler.h supplied a fallback definition of
__has_builtin for compilers that lack it. __has_builtin is a reserved
identifier, and defining one from a public header can collide with the
compiler's own definition or with another library the client includes.
Replace it with ORC_RT_HAS_BUILTIN(X) in orc-rt-c/support/Compiler.h,
which wraps __has_builtin where it exists and expands to 0 otherwise,
and convert the five uses.
[clang][Sema] Allow null caller for HIP kernel launch in incremental mode (#218659)
This PR allows a HIP kernel to be launched without a caller in
incremental mode. This is necessary because in incremental
mode(clang-repl), there isn't a `main` or any function that calls the
kernel.
Also adds a Sema test that compiles a top-level HIP kernel launch in
incremental device mode and verifies it is accepted.
Assisted by Claude Opus 4.8
[CIR] Add RegionBranchOpInterface unit tests and fix cir.await successors
Five of the ten ops implementing RegionBranchOpInterface have no unit test
coverage: cir.case, cir.cleanup.scope, cir.global, cir.await and
cir.coro.body. Add tests for all five.
Covering cir.await exposes a disagreement with its own terminator.
cir.condition terminates the ready region and reports {resume, suspend} as
its successors when the parent is an await, but AwaitOp::getSuccessorRegions
listed all three regions as entry successors and reported the parent op as
the successor of every region exit. Fix it to match cir.condition: ready is
the only entry successor, exiting ready branches to resume or suspend, and
exiting suspend or resume returns to the parent operation.
cir.await declares no results and carries NoRegionArguments, so successor
operand and input counts stay at zero along every edge and the MLIR verifier
is unaffected.
[mlir][SCF] Handle dropped operands in structural type conversion (#220473)
SCF structural type conversion assumes that `scf.for` bounds and
`scf.if` conditions are converted to exactly one value. A 1:N conversion
can drop or split these operands, causing `getSingleElement` to trigger
an assertion.
Reject conversions with non-single-valued bounds or conditions and add
regression coverage.
Fixes: https://github.com/llvm/llvm-project/issues/220459
clang: Distinguish unspecified from disabled exception model
Add ExceptionHandlingKind::Default so clang can tell an unspecified
exception model from an explicit -exception-model=none.
clang: Emit "exception-model" module flag
Record the exception-handling model as an "exception-model" IR module
flag when it differs from the target triple's default, mirroring how
other target ABI properties are recorded. Adds a
CodeGenOptions::toExceptionHandling helper to translate clang's
ExceptionHandlingKind into the LLVM ExceptionHandling enum.
Co-authored-by: Claude (Claude-Opus-4.8) <noreply at anthropic.com>
