[SCEV] Prove monotonic predicates from loop entry in isKnownViaInduction
isKnownViaInduction requires the predicate to hold on loop entry and on
the backedge. The backedge check usually fails when the latch only
bounds the IV from above, so e.g. {s,+,1}<nsw> >=s 0 is not proven even
when s >=s 0 is known on entry.
In a loop guarded by "start <u len", this keeps IndVars from computing
a[i - 32] in i64.
A monotonically increasing predicate against a loop-invariant value can
only change from false to true, so holding on entry is enough.
Use this in isKnownOnEveryIteration when the backedge check fails.
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
[BPF] Use FETCH atomic instruction for monotonic atomicrmw when the result is used (#210371)
For BPF, a monotonic atomicrmw selected the non-fetch instruction
regardless of whether the result was used. The non-fetch form discards
the old value, so the result was wrong if used.
Fix makes ISel select the fetch form when the result is used and the
non-fetch form otherwise via the binop_no_use / binop_has_use PatFrags.
This was already done for the i64 and/or/xor patterns, applied it to the
remaining monotonic patterns: i64 add and sub, and all i32 ops
(add/sub/and/or/xor).
Fixes #210280
Assisted-by: Claude (Anthropic)
[SCEV] Prove monotonic predicates from loop entry in isKnownViaInduction
isKnownViaInduction requires the predicate to hold on loop entry and on
the backedge. The backedge check usually fails when the latch only
bounds the IV from above, so e.g. {s,+,1}<nsw> >=s 0 is not proven even
when s >=s 0 is known on entry.
In a loop guarded by "start <u len", this keeps IndVars from computing
a[i - 32] in i64.
A monotonically increasing predicate against a loop-invariant value can
only change from false to true, so holding on entry is enough.
Use this in isKnownOnEveryIteration when the backedge check fails.
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
[IndVarSimplify][PhaseOrdering][NFC] Add tests for widening nsw IV arithmetic
Add tests for for (int i = start; i < N; i++) a[i] = a[i - 32] + 1,
with zext nneg indices and i - 32 nsw only:
- IndVarSimplify: with guards that do and do not prove start
non-negative, i - 32 is computed in i64 or i32.
- PhaseOrdering (indvars, loop-vectorize): whether a runtime alias check
is emitted.
Move @runtime_start out of ult-nonneg-bound-implies-nonneg.ll, which
duplicated it.
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
[IndVarSimplify][NFC] Add a negative test for zext elimination
Add @foo3_neg: the loop from @foo3 with a guard on an unrelated value,
so %add3 may be negative and its zext must stay.
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
[SCEV][NFC] Use isKnownOnEveryIteration in isKnownViaInduction
isKnownViaInduction and isKnownOnEveryIteration both prove a predicate
on every iteration from loop entry and the backedge, so the next change
can extend one place.
Add a private isKnownOnEveryIteration overload that takes any LHS and
RHS and the loop, and move the entry/backedge check from
isKnownViaInduction into it. The public overload is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
[SCEV][NFC] Factor getMonotonicPredicate out of getLoopInvariantPredicate
The next change needs the same setup as getLoopInvariantPredicate.
Move it into a private helper: put the loop-invariant operand on the
RHS, check that the other side is a recurrence of the loop, and compute
the predicate's monotonicity.
Co-Authored-By: Claude Opus 5.5 <noreply at anthropic.com>
[libc] Moves __cxa_thread_atexit_impl into proper entrypoint (#229347)
This PR resolves (#224664).
---------
Co-authored-by: Pavel Labath <pavel at labath.sk>
[AMDGPU] Handle gfx950 v_pk_{add,mul,fma}_f32 invalid op_sel (#231506)
gfx950 v_pk_add/mul/fma_f32 for certain op_sel combinations:
- Case 1 (v_pk_add/mul_f32): src0/src1 both VGPR with op_sel[1:0] ==
2'b10. Adjusted in GCNHazardRecognizer::fixPkF32InvalidOpSel by
commuting src0/src1 in place.
- Case 2/3 (v_pk_fma_f32): op_sel[2:1] == 2'b01, 2'b10, or 2'b11. Fixed
via SIInstrInfo::decomposePkFMAF32, which splits the packed FMA into two
independent scalar v_fma_f32s; wired in from both
SIInstrInfo::legalizeOperandsVOP3 (SelectionDAG) and
AMDGPUInstructionSelector::select's G_FMA case (GlobalISel), both gated
on SIInstrInfo::isLegalPkFMAF32OpSel. A safety net that crashes the
compiler is added in GCNHazardRecognizer::fixPkF32InvalidOpSel.
Also reject the invalid op_sel encodings directly in the assembler
(AMDGPUAsmParser::validateOpSel) so hand-written/inline asm can't encode
them.
[flang][OpenMP] Match variant selectors in the source construct context
The construct context for METADIRECTIVE and DECLARE VARIANT came from
enclosing MLIR operations. It recognized only a few operations, followed
operation nesting instead of source nesting, kept constructs outside an
enclosing TARGET, and used the body context for clause expressions and
loop bounds.
Build the context from the evaluation's source ancestors and from frames
for construct regions entered during lowering:
- A selected METADIRECTIVE replacement stands in for its source construct.
- TARGET discards the outer context.
- Constructs without a selector property still add a nesting level.
- Clause expressions see only enclosing constructs.
- Loop bounds see the leaves that precede the first loop-associated leaf.
- Specification expressions see no construct.
Composite DISTRIBUTE PARALLEL DO [SIMD] now evaluates DISTRIBUTE clauses
before PARALLEL, as in source nesting, and destroys their temporaries
[5 lines not shown]
[flang][OpenMP] Destroy composite num_threads temporaries once
DISTRIBUTE PARALLEL DO and DISTRIBUTE PARALLEL DO SIMD evaluate the
PARALLEL clause operands before creating omp.parallel, using the
statement context of the dispatch. That context is finalized after the
returned omp.distribute, which is inside the parallel region, so every
thread destroyed the same num_threads temporary, for example for
num_threads(size(f())) where f returns an allocatable.
Evaluate these operands in a separate statement context and finalize it
after omp.parallel.
Fixes #230661
Assisted with Copilot and Claude Opus 5.5
VE: Use i32 registers for 32-bit loads in the SjLj dispatch block
Fix verifier errors in exception test. The call site index and the
PIC jump table offset were loaded with ldl.zx, which defines an i32
register, into vregs with the i64 register class.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
VE: Default to SjLj exception handling
The clang driver selects SjLj exceptions for VE (96d4ccf00c8f), but the
triple defaulted to no exception model. Make SjLj the default for the
triple and MCAsmInfo so llc agrees with the driver.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
AMDGPU: Add vgpr32_lo16/vgpr32_hi16 OutPatFrags
Add OutPatFrags for extracting a 16-bit half of a value after copying
it to a VGPR_32, and use them for the existing open-coded instances.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
AMDGPU: Use lo16_to_vgpr32/hi16_to_vgpr32 OutPatFrags (NFC) (#231650)
Replace open-coded REG_SEQUENCE widenings of 16-bit values into a
VGPR_32 with the existing OutPatFrags.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[SelectionDAG] Implement SDTCisVTSmallerThanOp/SDTCisOpSmallerThanOp in SDNodeInfo::verifyNode. (#230679)
I've tried to capture the rules based on what tablegen does:
-Both should be FP or int.
-Both should be vector or scalar.
-If the scalar size is smaller then they contraint is satisified.
-If the scalar size is equal, the total size must be smaller.
We throw an error for comparing the total size of a fixed and
scalable vector. I don't know if we have good examples of what the rules
should be for mixing.
Assisted-by: Claude
[flang][OpenMP] Fix variant selector identity and candidate ranking
Identify user conditions by a profile of the parsed expression instead of
their source text: names by their ultimate symbol and literals by value
and kind. Equivalent spellings of the same condition share an identity,
including across module imports, while distinct variables stay distinct.
Record unknown trait properties with addUnknownTrait instead of an invalid
trait, so their selectors still score under match_any and match_none.
Recognize construct={simd}.
Rank METADIRECTIVE candidates once against the complete candidate set and
keep that order when runtime guards fail, so removing a failed guard does
not restore the raw score of its strict subset. Represent the false
outcome under match_any and match_none with a false condition to preserve
selector identity, and score an implicit NOTHING like an explicit
replacement; explicitness only breaks ties.
Split from #224431.
[2 lines not shown]
Support module importing with differing code models
First, module-level code model information is stored as a module flag.
To support merging of modules with different module flag values, we add
a new module flag behavior, `Ignore`, which ignores differences between
values and takes the flag from the first module.
Second, when a module imports a value from another module with a
dfferent code model, the imported value should retain the code model of
its source module. A small value imported into a large module should
remain small in order to prevent disturbing other references to that
small value in its original module. At the same time, a large value
imported into a small module should remain large as otherwise the value
may not be reachable in the small value.
Co-authored-by: Alan Zhao <ayzhao at google.com>
[Clang][AllocToken] Support TypeFuncHash and TypeFuncHashPointerSplit modes (#228492)
Extend Clang to support the `-falloc-token-mode=typefunchash` and
`-falloc-token-mode=typefunchashpointersplit` modes, which were added
to LLVM in #227686.
In these modes, CodeGen attaches the qualified name of the enclosing
function as the third operand of !alloc_token metadata. Allocations
outside functions use an empty function name. When type inference fails,
metadata is still emitted with an empty type name, so that a
function-based
token can be assigned. This allows distinguishing allocations that would
otherwise all share the fallback token.
Like the Increment and Random modes, `__builtin_infer_alloc_token()`
is not supported in constant expressions under these modes.
[lldb] Forward step plans through BorrowedStackFrame (#231657)
When a frame provider returns an index relative to its input frames,
LLDB wraps that frame in a BorrowedStackFrame. BorrowedStackFrame
forwards most StackFrame methods to the frame it borrows, except for
GetThreadPlanForStepType, so a scripted frame lost its custom step plans
as soon as another provider was stacked on top of it.
Forward it like the methods, and test stepping with a pass-through
provider stacked on top of a provider whose frames have step plans.
Signed-off-by: Med Ismail Bennani <ismail at bennani.ma>
