Fix remaining condition identity collisions and mismatches
Follow up on #233038 by preserving Clang's substituted template bindings
when normalizing condition roots. Profile written array bounds through
TypeLoc, including typedef definitions and adjusted function parameters.
Retain bindings in function types and qualified member references, and
include template-specialization qualifiers and array-trait dimensions.
Use the same type normalization for qualified references and aliases.
Normalize Flang intrinsic declarations and implied-DO indices independently
of their scope-local symbol addresses. Preserve resolved procedure identity
and distinguish implied-DO bindings by nesting and type, allowing identical
conditions to participate in the strict-subset tie-break again.
Correct the Clang runtime symbol checks and declare-variant candidate
order. Exercise fresh matching after PCH loading, and add paired distinct
and identical expression controls. Update the implicit-NOTHING test to
check its guarded selection and barrier fallback under source-selector
subset comparison.
[LV] Check stores to invariant addresses before handling unsafe deps (#233142)
canVectorizeMemory validates stores to loop-invariant addresses only
after handling the case where LAA reports the loop as unsafe. The early
exit checking canVectorizeIndirectUnsafeDependences account for
histograms skipped the invariant address checks.
Reorder checks to make sure such accesses are checked even if there
histograms to fix miscompile.
[OpenMP] Preserve condition identity in subset ties (partial #233038)
Subset comparisons previously treated unrelated user conditions with the
same applicability trait as equal. This could select the wrong directive
when candidate scores tied. Add frontend-produced condition keys and
compare source selectors separately from their effective ranking traits.
The standalone Flang reproducer in #233038 now tests a before b and selects
TASKYIELD when both are true. The shared matcher retains the expected
strict-superset result for identical conditions and preserves precedence
when the first candidate has a higher score. This is a partial fix: known
identity defects and failing regression tests remain.
Implemented:
- Add owned condition keys and optional immutable original-selector
snapshots to VariantMatchInfo. Normalize condition truth bits only for
subset comparison, count condition presence once, and require matching
known keys when a prospective subset contains a condition. Keep the
[51 lines not shown]
[VPlan] Don't look through FOR splices in isUsedByLoadStoreAddress. (#233141)
isUsedByLoadStoreAddress incorrectly looked through recurrence splices.
This lead to makeVPlanMemOpDecision incorrectly replicating loads
(instead of widening), while the address is converted to a single-scalar
by legacy decision.
This fixes a miscompile in the added test.
[flang][OpenMP] Make sure that exec METADIRECTIVE is in execution part
A METADIRECTIVE can be in either the specification part or the execution
part, depending on whether the variant directives contained in it are
declarative or executable. The OmpMetadirectiveDirective AST node can
hold any combination of variants, that is it can represent both kinds.
When the parser builds the specification part and attempts to create
an OmpMetadirectiveDirective, once it gets one it puts it in the
specification part regardless of the contents.
When an executable METADIRECTIVE is the first statement of the execution
part, it will end up in the specification part. This is an identical
situation as that for utility directives.
Recognize executable METADIRECTIVEs at the end of the specification
part and move them over to the beginning of the execution part.
Create an isolated implementation branch from refreshed main and record its
base. Preserve existing local files. Keep B's diff separate from A's TARGET
normalization and construct-subset changes, and from C's scoring changes.
**Implementation approach.** Give the original condition an explicit,
frontend-produced structural identity. Preserve it across candidate copies
and transformations. Compare source selector identity separately from the
effective traits used for applicability and scoring.
Prefer an owned structural key with exact equality, or an interned equivalent
whose owner demonstrably outlives every comparison. LLVM should not depend
on Clang or Flang AST types. A hash can accelerate lookup, but hash equality
alone must never prove expression equality.
The largest design work is producing correct keys. Neither a raw source
string, a folded expression, nor the current generic Clang profile satisfies
the required contract. Complete the capture and equivalence checkpoint below
before treating a proposed key format as ready for integration.
[266 lines not shown]
LLVMgold: Parse -plugin-opt options into LibraryOptionsContext (#233144)
Since #233044, the LLVMContexts that LTO creates read the defaults
instead, so passes ignore options like -vectorize-loops.
Parse them into a cl::LibraryOptionsContext and set it as
lto::Config::LibraryOptions and the PipelineTuningOptions' context, as
lld does.
[VPlan] Only hoist from blocks guaranteed to execute in licm (NFC). (#232512)
When hoisting, licm assumes that all top-level blocks of the vector loop
region execute on every iteration, which holds as long as the region is
linearized. Check it explicitly, by only hoisting from blocks that
dominate the region's exiting block.
Currently NFC, but preparation to run LICM before linearization and
support partially linearized CFG.
PR: https://github.com/llvm/llvm-project/pull/232512
[LV] Add test for miscompiles with histograms and scalarization. (#233135)
Add 2 additional tests for miscompile with stores to invariant addresses
and histograms and incorrectly scalarizing loads feeding a first-order
recurrence.
Also modernize check lines while touching the tests.
[LV] Don't emit duplicate negative stride runtime checks
This speeds up x264 by 2.15% on RISC-V by avoiding a duplicate hoisted runtime check.
When hoisting runtime checks we need to check that any strides are non negative. But for a loop like
```c
for (int i = 0; i < m; i++)
for (int j = 0; j < n; j++)
dst[(i * stride1) + j] = src1[(i * stride2) + j] + src2[(i * stride2) + j];
```
We have the pairs [dst, src1] and [dst, src2]. Both pairs end up checking that stride1 and stride2 are non-negative in IsConflict, and because there's other values between the two checks in the chain of ors InstCombine can't deduplicate it. This PR deduplicates the stride checks.
I'm aware some of this memory check stuff is being moved into VPlan but I'm not sure what the plan is with hoisted runtime checks, so it seemed better to just fix it here first.
This helps all targets but the motivating case is this loop in x264:
```c
[82 lines not shown]
[ExecutionEngine] Set mlir-runner's library options in the LLVMContext (#233131)
Since #233044, the LLVMContext that ExecutionEngine creates reads the
defaults instead, so LowerMatrixIntrinsics ignores
-matrix-default-layout=row-major and
Integration/Dialect/Vector/CPU/*-row.mlir fail.
Parse the options into a cl::LibraryOptionsContext and add
ExecutionEngineOptions::libraryOptions.
[LV] Add test duplicate negative stride checks in hoisted runtime checks. NFC (#233130)
In this test we have [dst, src1], [dst, src2] as pointer pairs, and for
hoisted
runtime checks both pairs need to check that stride1 and stride2 are
non-negative. But we end up emitting duplicate `icmp slt i64
stride{1,2}, 0`
checks across the two pairs when we only need to check them once.
Extracted from pixel_avg in x264.
[Support] Remove the T::Global fallback in LibraryOptionsContext::get<T>() (#233044)
A context's values for T start as a copy of T::Global, which
cl::ParseCommandLineOptions sets, and get<T>() returns T::Global for a
context without values for T. Use a new constant T::Default instead, so
a tool must parse library options with
`cl::parseCommandLineOptions(LibraryOptionsContext &, ...)`.
LLM-aided
[mlir][scf] Test forall promotion with multiple inserts (#232920)
Cover #231826 by checking that disjoint insertions into one shared
output are chained into a single result.
Assisted-by: Codex / GPT-6
[clang][CodeGen] Fix assertion failure for new T[SIZE_MAX] with array cookie (#232570)
Fixes #232250.
`EmitCXXNewExpr` asserts that `allocSize == allocSizeWithoutCookie` if
and only if no array cookie is needed. The "only if" direction does not
always hold.
`EmitCXXNewAllocSize` replaces the allocation size with an all-ones
value when the size computation overflows, so that `operator new` fails.
If the count times the element size is already all-ones and adding the
cookie then overflows, both values are the same uniqued constant even
though a cookie is present. For example, on i386:
struct S { ~S(); };
void f() { new S[0xFFFFFFFFu]; }
Here `sizeof(S) == 1`, the cookie is 4 bytes, and `0xFFFFFFFF + 4`
overflows, so the assertion fired. This can't happen on 64-bit targets
[6 lines not shown]
[DWARFLinker] Emit DW_AT_addr_base for units that had none (#219357)
Fix DWARFv5 units that reach the linker without DW_AT_addr_base, as GCC
emits them at -O2. Both linkers rewrite range and location lists into
base_addressx entries, which resolve through DW_AT_addr_base, but they
only ever patch an attribute copied from the input. With nothing to
patch, the classic linker hits the llvm_unreachable in patchAddrBase(),
and the parallel linker writes indices with no base, so the unit's
ranges resolve from address 0.
Add DW_AT_addr_base to every DWARFv5 unit root that lacks one, whatever
its tag, and emit the unit's .debug_addr contribution even when no
address was pooled, so the attribute always points at a table. Update
mode does not rewrite ranges, so nothing is added there.
The tests link a compile unit root and a partial unit root with both
linkers and check the attribute, the address table, the range list and
--verify. The compile unit test also checks that update mode adds no
attribute, and a third test links a unit that pools no address.
[3 lines not shown]
[DAGCombiner] Fold umin(fp_to_uint(x), 0) to 0 (#232231)
`PerformUMinFpToSatCombine` turns `umin(fp_to_uint(x), (2^n)-1)` into
`fp_to_uint_sat`.
When the clamp width is 0 (2^0-1), `n` is also 0, and `getIntegerVT(0)`
then asserts in `IntegerType::get`, which requires a width of at least
1.
Since an unsigned value is never below 0, the `umin`'s result will
always be 0 as well.
[RISCV][P-ext] Use concat_vectors with 0 in widenPackedVectorWithZeros.
This will be legalized and combined to a vector_shuffle. That
shuffle can then be matched to a pack with x0 after #232892.
This changes the zext.w to a pack, but that's not obviously worse.
If we need to canonicalize we can add a DAG combine later.
[RISCV][P-ext] Remove custom lowering of insert_subvector for v2i16/v4i8 on RV64.
This was for handling inserting into a zero vector. After #232892
we let the insert_subvector legalize to a shuffle_vector which we
can then lower as pack.
This changes the codegen from zext.w to pack, but that's not
necessarily worse. If it turns out to be a problem we can
canonicalize pack with zero with a DAGCombine or isel patterns in
the future.
[RISCV][P-ext] Create zero vector rather than bitcasting from i32 0 in lowerINSERT_SUBVECTOR. (#232852)
Teach LowerBUILD_VECTOR to create SPLAT_VECTOR when all operands are the
same to prevent regression.
[RISCV][P-ext] Call tryWidenMaskForShuffle at the end of P extension shuffle lowering. (#232892)
If the shuffle mask has contiguous elements we can try shuffling them
using a large element type.
