[CodeGen] Fix DetectDeadLanes for same-class copies with mismatched widths (#226556)
isCrossCopy() returned early whenever source and destination shared a
register class, treating the transfer as lane-for-lane. That is wrong
when a REG_SEQUENCE names a subregister slot narrower than the source
operand, e.g. a 32-bit vreg used as the lo16 piece of another 32-bit
vreg. Only take the early exit when the two widths match; otherwise fall
through to findCommonRegClass().
Fixes: ROCM-31212
[AMDGPU] Measure MFMA overwrite hazards at each instruction
Apply previously established processing to:
- VALU overwriting an MFMA result
- VALU overwriting a register an MFMA took as srcC
AI-assisted.
[AMDGPU] Measure MFMA read hazards at each producer
Introduce more sophisticated traversal to avoid the following traps:
- order-dependent traversal and discarding seen BBs despite shorter path
- mis-matching distance and window of different producers
Record the best distance per BB instead of a visited flag and sweep the
arrivals in nondecreasing distance (bucket queue). This pairs producers
with their actual distance to a consumer in one go.
Fixed scenarios:
- MFMA reading an MFMA result as srcA, srcB or srcC
- VALU, memory or export instruction reading an MFMA result
rewrite-vgpr-mfma-to-agpr.ll gains an s_nop 2: a 4-pass XDL write that
partially overlaps the srcC read two slots later requires five wait
states, and none were emitted because the nearest producer wrote the
register in full.
AI-assisted.
[AMDGPU][NFC] Extract the MFMA read-window calculation
Move the wait states a consumer needs before reading an MFMA result out
of checkMAIHazards90A into getMFMAReadWaitStates, taking the producer as
an argument, so a caller can ask about a specific producer. The partial
srcC overlap half moves into getMFMAOverlappedSrcCWaitStates. The caller
passes the producer the walk recorded, so nothing changes.
AI-assisted.
[LV] Honor -force-vector-interleave when generating diff checks (#225768)
We use both IC and UserIC in the checks below. However, once those
checks are done, UserIC will always override IC if it's non-zero. That
means using the max is unnecessary and can result in overly conservative
checks when UserIC < IC.
[orc-rt] Add ORC_RT_LOG_ENABLED macro. (#226500)
ORC_RT_LOG_ENABLED(Level) reports whether log sites at the given level
are compiled in, taking into account both the backend and the
ORC_RT_LOG_LEVEL floor. It takes the same level token as ORC_RT_LOG and
can be used in preprocessor conditionals, allowing code that relies on
logging to surface something important (e.g. a logging error reporter)
to choose an alternative when logging is compiled out.
[mlir][Interfaces] Share more operation interface bodies (NFC) (#226603)
Use raw operation access for methods that do not depend on the concrete
operation type. Keep typed access where operation-specific APIs are
needed.
This save a little bit of code size and improves build time.
Assisted-by: Codex
[flang][openacc] Build full shared-label DO nests in AccNonBlockDoConstruct (#225997)
When an `!$acc loop` is associated with a labeled DO,
AccNonBlockDoConstruct
turned only that outer loop into a DoConstruct and left sibling
LabelDoStmts
that share the same terminator for later CanonicalizeDo. An inner `!$acc
loop`
then swallowed the shared terminator into its own DoConstruct, so
AnalyzeLabels (which runs before CanonicalizeDo) rejected the remaining
LabelDoStmts with "Label 'N' is not in DO loop scope".
Recurse on nested LabelDoStmts while parsing the associated loop so the
whole shared-label nest is a DoConstruct before label analysis. Add a
parser test where only the outer and innermost loops have LOOP
directives.
[libc++][pstl] Implementation of parallel std::swap_ranges() based on parallel __for_each (#224967)
This PR adds an implementation of parallel `std::swap_ranges()` based on
`__parallel_for_each_iter_pair()`.
The implementation is effectively a one-liner, but has to deal with
iterator ranges, thus is located in `cpu_algos`:
```c++
__pstl::__parallel_for_each_iter_pair<_Backend>(
first1, last1, first2,
[](ForwardIterator1 brick_first1, ForwardIterator1 brick_last1, ForwardIterator2 brick_first2) {
std::swap_ranges(std::move(brick_first1), std::move(brick_last1), std::move(brick_first2));
});
```
Part of #99938
[SelectionDAG] Fix result index and vector width in unrollExpandedOp (#225886)
Fixes #224127.
In `DAGTypeLegalizer::WidenVectorResult`, `unrollExpandedOp` computes
the unroll count and widened vector type from the result being legalized
(`ResNo`) rather than unconditionally using result 0. For multi-result
nodes where result types differ (e.g. `ISD::FFREXP`), this prevents
mismatched vector widths and preserves the correct result index from
`DAG.UnrollVectorOp`.
Assisted-by: Claude
---------
Co-authored-by: Demetrios Chiuratto Agourakis <agourakis82 at gmail.com>
[lldb] Add StructuredData::Dictionary::ForEachSorted (NFC) (#226597)
`Dictionary::Serialize`, `Dictionary::GetDescription` and the build
configuration dump in `CommandObjectVersion` each collect the
dictionary's keys and sort them before iterating, so that the output
doesn't depend on StringMap's ordering. Factor that into a new
`ForEachSorted` method.
Signed-off-by: Med Ismail Bennani <ismail at bennani.ma>
[mlir][ODS] Share operation interface methods with raw-op bodies (NFC) (#226317)
Allow an operation interface method to use a shared callback when its
body only needs the raw Operation. Use it for OpenMP block-argument
indexing and slicing methods, avoiding per-operation wrappers while
preserving the existing fallback and external model paths.
Reduces the size of the OpenMP dialect text section by 1.68%.
Assisted-by: Codex
[bazel] Download linux UAPI headers when no local dir is set (#216868)
This PR defines a hermetic source for `@linux_api//:linux_uapi_headers`
that will be used in CI. If no `LINUX_UAPI_INCLUDE_DIR` value is
specified, then Bazel will download the linux-libc-dev debian package
and use its headers.
I originally wanted to pull from https://github.com/torvalds/linux and
build UAPI headers from source as it gives more control, but the repo is
rather large and took ~20s to download and ~10s to build. Using the
debian package's headers is closer to what the LLVM-libc fullbuild CI
[currently
does](https://github.com/llvm/llvm-project/blob/007551bacb899a409218574908d96b0633d52018/.github/workflows/containers/libc/Dockerfile#L38)
and is very fast.
[LLVM] Use complete runtime dependency list for multilibs (#226580)
Summary:
The current handling makes the multilib job depend on the base runtimes
job. This means that we will potentially depend on things that don't
necessarily need to be built to fulfil the multilibs job. Instead, just
depend on the individual targets for that multilib.
[TBAA] Recover !tbaa for a memcpy of struct with same type fields (#214116)
When InstCombine widens a small same-typed aggregate copy into an
integer load/store, it derives no !tbaa, and !tbaa.struct is nulled out,
so the load/store are left untyped. An untyped access may-alias every
typed access and blocks optimization:
struct Coord { int x, y; };
void copy_if_inbounds(Coord *dst, const Coord *src, const long *bound,
int n) {
for (int i = 0; i < n; ++i)
if (i < *bound)
dst[i] = *src; // {int,int} copy -> memcpy + !tbaa.struct
}
compare this to field-by-field copy, which has typed field access
void copy_fields_if_inbounds(Coord *dst, const Coord *src, const long
*bound, int n) {
for (int i = 0; i < n; ++i)
[2 lines not shown]
[AutoUpgrade] Upgrade old-style !tbaa.struct field tags (#225967)
Extend the TBAA auto-upgrade to rewrite old-style scalar field tags
inside !tbaa.struct
to the struct-path aware format. bitcode (MetadataLoader) upgraded to
match the existing top-level !tbaa upgrade.
[Instrumentor] Use CreateAllocationSize to compute size of alloca (#226330)
No significant functional change, but the IR might look slightly
different.
[Clang] Enforce the same size limits for `vector_size` and `ext_vector_type` (#226375)
Fixes #165458
A `bool` vector declared with `ext_vector_type` isn't lowered as an LLVM
vector in memory: `ConvertTypeForMem` packs it into a single integer
with one bit per element. `BuildExtVectorType` only checked that the
element count fits in 32 bits, though, so a vector of 187,553,262 bools
got through Sema and the first consumer that needed its memory type (the
zero initializer of a tentative definition here) asked
`IntegerType::get` for far more than the 2^23 bits it supports. The
count matters for every element type, not just `bool`, because Sema also
forms bool vectors out of other vectors (`c ? true : false` with a
`char` vector condition), and `vector_size` was no better off with its
limits of 2^32 elements and 2^61 bytes.
Both attributes now go through the same two limits when the type is
built, reusing the existing "vector size too large" error: at most
`llvm::IntegerType::MAX_INT_BITS` (2^23) elements, and at most 2^28
[2 lines not shown]
[CIR] Fix fp-contract.c checks for -ffp-contract=off and CIR locations
Bound the trailing CIR-OFF-NOT with a CIR-OFF check on nested_pragmas,
where float_control(precise, on) forms a cir.fmuladd even under
-ffp-contract=off. Allow the loc suffix CIR prints after each op in the
end-of-line anchors.
[llvm] produce a more consistent estimates of bit width needed when parsing an integer (#205947)
A colleague of mine noticed that `"12535824225335233"` parses in MLIR's
integer attribute parser as a 68-bit integer, even though it is a 53-bit
constant. I traced this back to `StringRef::consumeInteger`'s heuristic
estimate of the bit size. This change replaces that heuristic with a
default 64-bit storage, doubling the storage as more digits are parsed.
This produces a potentially larger over-estimate of the total storage
required, but does so in a less arbitrary manner and reduces the
over-approximation for numbers close to the 64-bit boundary.
Nb., A first iteration of this change tightened that estimate to at most
a 1-bit overapproximation with a lookup-table.
Assisted by Gemini
