[AMDGPU] Remove write-only private member variables (NFC) (#244910)
This patch removes private member variables in AMDGPU that are
initialized or assigned to but never read, along with their
initializers and assignments.
Assisted-by: Antigravity
[CalledValuePropagation] Fix non-deterministic metadata generation (#244837)
Fixes non-deterministic metadata numbers in CVP outputs.
Non-deterministic numbers in CVP outputs make it difficult to trace when
and how two compilations (with two versions of the compiler) start to
diverge by tracing `-print-after-all` outputs.
To repro: copy sqlite3.c from llvm-test-suite and optimize with `opt -O3
-print-after-all -ir-dump-directory=...` and save the outputs of a few
runs in different directories. The outputs start to diverge after CVP
before this change. With this change we reliably get identical IRs
throughout the pipeline.
[AMDGPU] Keep soft waitcnts unchanged until SIInsertWaitcnts converges
On the first visit to a loop header only the preheader state is known.
A soft waitcnt there (e.g. from a release fence) can have some of its
counters simplified away while another counter keeps the instruction
alive: the fence's own vmcnt(0), a preceding wait.asyncmark, or an
explicit s_waitcnt. The soft waitcnt was then rewritten with the
simplified value and promoted to a hard waitcnt, or merged into a
preceding hard one. When the back edge later brought a pending LDS
store, the original lgkmcnt(0) request was gone, and the s_barrier after
the release fence no longer waited for it.
Treat soft waitcnts as requests that depend on the incoming state: never
merge, promote or rewrite them while the analysis iterates. They are
kept unchanged in DeferredErasures, a hard waitcnt (existing or new)
carries what is needed on each visit, and the soft waitcnts are erased
once the analysis converges. This extends #220357, which did the same
only for soft waitcnts that were simplified away entirely.
[11 lines not shown]
[AMDGPU][NFC] Pre-commit tests for soft waitcnt requests lost at a loop header (#244686)
Two cases where a soft waitcnt at a loop header loses its `lgkmcnt(0)`.
On the first visit to the header no LDS operation is pending yet, so the
soft waitcnt's `lgkmcnt` is simplified away, while a `vmcnt` keeps the
instruction alive. That `vmcnt` comes either from the soft waitcnt
itself or from a preceding hard `vmcnt(0)` that it is merged into. The
DS write at the end of the loop is then not waited for before the
`S_BARRIER` on the next iteration.
The checks show the current, incorrect output (`S_WAITCNT .Vmcnt_0`
before `S_BARRIER`).
Assisted-by: Claude Opus 5.5
---------
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
ExpandVariadics: Invalidate function analyses individually
Avoid discarding analyses that never need to be invalidated, such as
TargetLibraryInfo. Clear the analyses of erased functions.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
AllocToken: Only invalidate analyses of instrumented functions
AllocToken tried to preserve the CFG analyses, but did not preserve the
function analysis manager proxy, so all function analyses were
invalidated.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[mlir] Notify rewriter when replacing SCF, Vector and EmitC bodies (#244448)
Fixes a few places where block manipulation was done without notifying
the rewriter.
[RISCV][Zicfilp] fix KCFI clobbering x7 when cf-protection=branch is enabled (#240347)
When enabling KCFI and cf-protection=branch
(-fcf-branch-label-scheme=unlabeled,
-riscv-landing-pad-label=<some-non-0-value>),
the KCFI check sequence lands between the cf-protection=branch Label
loading (`lui t2, Label`) instruction and the indirect call `jalr a2`.
KCFI may clobber x7/t2 because it is one of its default scratch
registers; at the time of the indirect jump
it may no longer contain the correct value for the landing pad label.
KCFI now avoids x7/t2 when cf-protection=branch is enabled, causing it
to select one of its fallback scratch registers
(x28-x31) instead. In the test example it falls back to x28/t3.
This patch is a prerequisite for the zicfilp func-sig label scheme
implementation https://github.com/llvm/llvm-project/pull/240344 because
the func-sig label scheme working correctly also depends on this bug
being fixed. The patch contains a corresponding regression test.
[CalledValuePropagation] Fix handling of unnamed functions (#244846)
A function without a name gets an empty string as the name, so they were
all treated equal.
Assign each function a unique integer (their position in the module) and
use the integers for ordering them.
Also fixes a source of non-determinism in the pass as we no longer
compare pointer values.
[TableGen] Remove unused member functions (NFC) (#244911)
ClassReference::isInvalid: This function was added on November 22, 2007
in commit f4127dd48e11297ed3c545bb6ee6a70bf91388c7 without any callers.
MultiClassReference::isInvalid: This function was added on April 22,
2009 in commit 753ed8fd9ccd481d668adf30e167d4ae34d5bcdf without any
callers.
RecordsEntry::RecordsEntry: The default constructor was added on June
21, 2018 in commit 7d69e0f37d2829e720662d6398a956c6b2e6048d without any
callers.
Assisted-by: Antigravity
[CalledValuePropagation] Use SmallMapVector instead of SmallDenseMap for deterministic iteration in the fixpoint loop (#244851)
The changed values type `SmallDenseMap<PointerIntPair<Value *, 2, IPOGrouping>,
...>` is iterated non-deterministically because of the pointer key type,
causing non-deterministic debug outputs (`-debug-only=sparseprop`) and compile
times (when counting instructions).
[mlir][tensor] Keep the vector.mask pass-thru when folding extract_slice into transfer_read (#244844)
`TransferReadOfExtractSliceOpFolder` re-masks the folded
`vector.transfer_read` with the mask of the enclosing `vector.mask` but
drops its pass-thru, so the masked-off lanes get the padding instead of
the pass-thru.
Pass the pass-thru to the new `vector.mask` as well.
Found while reviewing #243710, which fixed the same issue in
`TransferReadDropUnitDimsPattern`.
#228140 also touches this function.
Part of #239511.
Assisted-by: Claude
[flang][OpenMP] Require an enclosing SIMD for construct={simd}
A construct={simd} selector without properties was ignored, because SIMD
is a predefined trait selector name (it can take clause properties)
rather than a directive name. A DECLARE VARIANT or METADIRECTIVE
alternative that requires SIMD was therefore selected everywhere,
including outside any SIMD construct.
Record the SIMD construct trait for such a selector. Lowering builds the
construct trait set from the enclosing MLIR operations, so also recognize
omp.simd there; otherwise calls inside SIMD would no longer match.
Fixes #244759
Assisted-by: GitHub Copilot
[flang][OpenMP] Require an enclosing SIMD for construct={simd}
A construct={simd} selector without properties was ignored, because SIMD
is a predefined trait selector name (it can take clause properties)
rather than a directive name. A DECLARE VARIANT or METADIRECTIVE
alternative that requires SIMD was therefore selected everywhere,
including outside any SIMD construct.
Record the SIMD construct trait for such a selector. Lowering builds the
construct trait set from the enclosing MLIR operations, so also recognize
omp.simd there; otherwise calls inside SIMD would no longer match.
Fixes #244759
Assisted-by: GitHub Copilot
[cmake][runtimes] Pass variable type for passthrough CMake options (#227662)
Passing the types affects the handling of `PATH` cache variables.
The motivating case is setting `COMPILER_RT_INSTALL_LIBRARY_DIR` without `CMAKE_BINARY_DIR` getting prepended to it.
Previously reverted because of https://github.com/llvm/llvm-project/pull/198505#issuecomment-4496493320.
However there's insufficient information to replicate or fix the non-upstream build issue so I'll go ahead with reapplying in the hopes that maybe it's magically resolved itself.
Revert-Of: 313ab4dc3b3a5f4c14079c82806c1f28975f724f
Re-Apply-Of: 2e06e008a7b44fab5c0d7fbc4331c696b5e19b06
[mlir][linalg] Do not fold tensor.pad into a read inside vector.mask (#243927)
`PadOpVectorizationWithTransferReadPattern` folds a `tensor.pad` into a
`vector.transfer_read` of its result and replaces the read's padding
with the pad value. It bails out on a mask operand, but not on
`vector.mask`, where the masked-off lanes take the read's padding.
For example, vectorizing a `linalg.pack` (tile 8, padding `-1`) of a
`tensor.pad` (3 to 5 elements, value `7`) gives a masked read of the
pad. After `pad_vectorization`, the packed row is `1 2 3 7 7 7 7 7`
instead of `1 2 3 7 7 -1 -1 -1`.
The pattern now bails out on reads inside `vector.mask` too. Tested by
`@negative_pad_and_transfer_read_in_vector_mask`.
Part of #227821 and #239511.
Assisted-by: Claude