[AArch64] Add guard for INSvi64lane lane in peephole opt (#240156)
This is unlikely to come up as a problem, but make sure that the lane we
insert
zero into is the top lane.
[OpenMP] Fix context selector matching and scoring (#231668)
OMPContext computes some context selector scores differently from the
OpenMP 5.2 rules in "Matching and Scoring Context Selectors", and
crashes on some selectors. Fix the following; each can change which
variant is selected:
- Device weights. kind, arch, and isa are worth 2^l, 2^(l+1), and
2^(l+2), where l is the depth of the construct context. OMPContext
used the candidate's own construct selector count, so inside two
nested parallel regions kind(cpu) was worth 1 instead of 4 and lost
to vendor(score(2): llvm).
- Scores per selector. A score belongs to a trait selector, but it was
added once per property, so vendor(score(10): llvm, amd) counted 20.
Scores now add once, using arbitrary-width arithmetic instead of
wrapping 64-bit sums.
- Repeated constructs. When a construct selector can match several
enclosing constructs, the highest-valued complete match counts.
construct={parallel} inside three nested parallel regions now adds 2^2
[30 lines not shown]
[LoongArch] Fix misaligned frame-index offset for vldrepl.{h,w,d} (#239516)
eliminateFrameIndex did not check that the final offset is a multiple of
the access size for [X]VLDREPL_{H,W,D}, so a byte-aligned stack object
could produce an unencodable immediate. Materialize the address in a
scratch register in that case, as is already done for [X]VSTELM_*.
Assisted-by: LLM
Fixes #239508
(cherry picked from commit 3ade2bcaa2f31acdf4011e965d5ae39c39796671)
[LV] Add tests for FindLast reductions with nested phis (NFC). (#230640)
Add tests where the data value of a FindLast reduction is itself a phi
that yields the reduction phi on some paths, e.g. via a nested if. Both
are currently miscompiled, as the outer condition is used as the
condition for updating the reduction.
Also enable masked memory ops and regenerate checks with the latest UTC
version to make checks more compact without loss of generality.
(cherry picked from commit 2f272fc322cdc1fe1fbb667cd2cb6abddeb0eb9b)
[VPlan] Bail out on FindLast reductions with extra phi users. (#230676)
handleFindLastReductions uses the mask of the single non-phi incoming
value of the find-last blend as the condition for updating the
reduction. If the data value itself is the reduction phi on some paths,
e.g. via a nested blend for an inner if, the mask is also true for lanes
that keep the old value, and extract-last-active may then pick a stale
lane.
Conservatively bail out if the reduction phi has users other than the
find-last select/blend and the header mask select.
(cherry picked from commit 79caa03343df4da7b8ae0338d932125ef4bcf20b)
[flang][OpenMP] Fix DECLARE_SIMD warning annotations in linear-clause01.f90
release/23.x picked 2cc5b8ca (warn on ignored DECLARE_SIMD) before
94467edd (REF modifier verification), the reverse of their order on main.
2cc5b8ca therefore annotated only Case 1-4, and 94467edd later appended
Case 5-7 without the warning annotation, failing check-flang at lines
76/83/90.
Assisted-by: Claude
[AsmPrinter] Emit null pointer vector splats as zero bytes (#235980)
Emitting a global such as `@x = global <2 x ptr> zeroinitializer` with
`llc -data-sections -filetype=obj` asserts on 64-bit targets.
`ConstantPointerNull` can represent a vector splat since #195486, but
AsmPrinter passes the entire allocation size to
`MCStreamer::emitIntValue`, which accepts at most eight bytes.
Route pointer-vector splats through `emitGlobalConstantVector`, matching
the existing handling of integer and floating-point splats. Its
zero-value path emits the full allocation with `emitZeros`. Keep scalar
pointers on the existing `emitIntValue` path.
Fixes #235977
Assisted-by: OpenAI Codex
(cherry picked from commit fed20e0e68323be32cb0fe8af679b8cb34e72112)
[DAG] LowerShiftByScalarImmediate - early out for shift-by-zero cases (#237365)
Also, add assertions to getGFNICtrlImm that we don't try to shift by
zero
Fixes #235663
(cherry picked from commit 4eff20caf9ca079c6c779f1a0029e7a8f86f4790)
[X86] incDecVectorConstant - use ++/-- prefix operator to increment/decrement (#237176)
Don't just add immediate as it might not promote correctly to types
greater than 64-bits
Fixes #236021
(cherry picked from commit f64f3ea8f0f456790399fb50c7df368eafd26686)
Update llvm/test/CodeGen/AArch64/stack-coloring-alignment.ll
Fix test failure due to release/23.x not containing @tomershafir's change that prints labels in the ScheduleDAG output.
Co-authored-by: Nathan Corbyn <me at nathancorbyn.com>
[LV] Invalidate interleave-groups with gaps if for early-exit loops. (#237409)
If the loop exits from a block other than the latch, the scalar
iteration reserved for the exit may leave the loop before reaching the
grouped accesses.
Invalidate interleave groups in the case, to avoid mis-compile.
Fixes https://github.com/llvm/llvm-project/issues/237192.
(cherry picked from commit ccce555818cd682aff8ae4b9d72eb8ed0044331c)
[lld][LoongArch] Fix crash with TLS IE relocations in extreme code model (#234508)
Restore TLS IE and DESC relocation expressions needed by the extreme
code model fallback in relocateAlloc().
Fixes: #234507
(cherry picked from commit c98ead286790d9ed6d02a10b02129e63c66c965c)
[OpenMP] Fix context selector matching and scoring
OMPContext computes some context selector scores differently from the
OpenMP 5.2 rules in "Matching and Scoring Context Selectors", and
crashes on some selectors. Fix the following; each can change which
variant is selected:
- Device weights. kind, arch, and isa are worth 2^l, 2^(l+1), and
2^(l+2), where l is the depth of the construct context. OMPContext
used the candidate's own construct selector count, so inside two
nested parallel regions kind(cpu) was worth 1 instead of 4 and lost
to vendor(score(2): llvm).
- Scores per selector. A score belongs to a trait selector, but it was
added once per property, so vendor(score(10): llvm, amd) counted 20.
Scores now add once, using arbitrary-width arithmetic instead of
wrapping 64-bit sums.
- Repeated constructs. When a construct selector can match several
enclosing constructs, the highest-valued complete match counts.
construct={parallel} inside three nested parallel regions now adds 2^2
[29 lines not shown]
Test device weights at the TARGET boundary
KIND's weight depends on the depth of the construct context, which
starts at the innermost enclosing TARGET. A PARALLEL outside the TARGET
must not raise it, and a PARALLEL inside must.
[SLP]Fix narrowing of alternate icmp with mismatched extensions
Extend the operands of alternate compares back to the original type
unless both predicates match the operand signedness, as done for
non-alternate compares.
Fixes #240119
Assisted-by: Cursor
Reviewers:
Pull Request: https://github.com/llvm/llvm-project/pull/240141
[SLP] Emit min/max reduction at the root select, not its compare
The min/max matcher accepts select (cmp X, Y), X, Y' with Y' identical
to Y but defined after the compare. Emitting the reduction before the
compare used Y' before its definition. The compare position was needed
only while the reduction replaced the compare uses, removed in D99753.
The root compare now has a single use, so emit at the select, which
all reduced values dominate.
Fixes #240006.
Assisted-by: Cursor
Reviewers:
Pull Request: https://github.com/llvm/llvm-project/pull/240133
