[LV] Model memory runtime checks for inner loops of loop nests in VPlan. (#232794)
Remove the restriction that kept the IR-generated memcheck block for
loops with a parent loop. When hoisting runtime checks, they are
rewritten so they hold across the outer loop and are expanded before the
entry block to the VPlan, using the regular IR expander.
This patch exposes getRuntimeCheckBounds, which does the rewrite, and
marks such rewritten bounds as requiring expansion in IR. VPSCEVExpander
takes a loop as optional argument and uses VPExpandSCEVRecipes for
expressions that are invariant in the loop.
PR: https://github.com/llvm/llvm-project/pull/232794
Merge branch 'main' into users/void/pbqp-inline-asm-error
bd51cc5c5556 (#226523) made RegAllocBase::RegClassInfo a pointer to the
shared RegisterClassInfo, so the call to assignFailedVirtReg() dereferences it.
Assisted-by: Claude Opus 5.5
[MLIR][Python] Fix stale multithreaded_tests.py (#237018)
`multithreaded_tests.py` only runs on free-threaded CPython, so it
rotted unnoticed. On 3.14t it failed every test and then aborted. This
PR:
- Drops calls to live-operation APIs removed in #155114.
- Compares live contexts against a pre-test baseline instead of 0,
because `ir/array_attributes.py` keeps a Context alive.
- Restores the builtin `RankedTensorType` caster after binding
`TestIntegerRankedTensorType`, which hijacked it and caused the abort.
The abort itself is fixed in #237010.
- Only collects test functions defined in each module, so imported
helpers aren't run as tests.
- Skips `testMatchInterfaceEnumReplaceAttributeBuilder`, which replaces
a global attribute builder used by `testMatchInterfaceEnum`.
- Without TSAN, skips the xfail list and tests using the `np_to_memref`
helpers. They hit a ctypes race (python/cpython#127945)
nondeterministically. Under TSAN they still xfail as before.
Assisted by: Claude
[flang] Enable test for all other targets (NFC) (#237224)
x86-registered-target disables the test if the compiler built with
LLVM_TARGETS_TO_BUILD=all or
LLVM_TARGETS_TO_BUILD=...X86..... This patch is to only disable for x86
related targets and enable for all other targets.
[CIR] Make fenv lowering follow the attribute's defaults
An absent fenv parameter now takes its value in the default FP
environment (tonearest, masked, non-strict), so the empty attribute is the
unconstrained default. The lowering reads the parameters through the
OrDefault helpers, as speculation does, and checks strict_except before the
exception mode: strict exceptions keep the FP status exact even when masked.
Clang codegen output and its lowering are unchanged.
Assisted-by: Claude Code (Opus 5.5).
[mlir][ODS] Give AttrDefs an identity constBuilderCall
An attribute defined with AttrDef is its own constant value, so set its
constBuilderCall to "$0" when its returnType is the attribute class. This
lets an AttrDef be used directly in DefaultValuedAttr without a per-def
override, and drops the one LLVM_DIGlobalVariableExpressionAttr had. Defs
with a different returnType keep it unset.
Assisted-by: Claude Code (Opus 5.5).
PowerPC: Remove LiveVariables run and unused LiveIntervals requirement
PPCTLSDynamicCall has not used LiveIntervals since 3ff319c69047, but
still required the analysis. LiveIntervals requires there to be no
unreachable blocks or otherwise it asserts. These blocks were
indirectly removed by adding the extra LiveVariables run, because
LiveVariables has a declared requirement on
UnreachableMachineBasicBlockElim.
Drop the dead analysis requirements along with the LiveVariables run.
This also avoids computing LiveIntervals at -O0 for PIC. This is
still broken if the pipeliner is explicitly enabled on optnone
functions which requires a separate fix.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
RuntimeLibcalls: Fix signatures of fmin/fmax libcalls (#235147)
These used Floating, which always lowered to float regardless of the
libcall type.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[GlobalOpt] Remove malloc promotion (#233797)
This optimization is quite buggy, and it doesn't matter in practice. The code
is already quite complicated and implementing it properly would make the code
even more complicated and make it less applicable and relevant.
Known bugs:
- #233447: the global we optimize needs to be written to once. We assume this
is the case based on static use site count, which is the bug. We'd need to
analyze the use site of the store (e.g. is it in a loop?), the parent
function's callers (transitively) and make sure it'll be called at most once.
This will rule out any uses in loops, recursive functions, and calls from
external functions, ... and make this applicable to even more rarely.
- #215533: the new global gets incorrect alignment. The fix requires analysing
uses of the `malloc` call and the uses of loads from the global. The value of
`malloc` is allowed to be used in GEPs, and the global value is allowed to be
used in PHIs. So we'd need to consider alignments of users of those GEPs and
[31 lines not shown]
[BPF] Fix CO-RE roots for subarray field info and anonymous elements (#235601)
consumeAccessRoot handles an access chain that starts with array
subscripts, such as (*p)[1][2].y. It folds the leading subscripts into
one index, counted in units of the innermost element, and uses that
element type as the relocation root. Four cases went wrong:
- A subarray passed to __builtin_preserve_field_info crashed, e.g.
__builtin_preserve_field_info((*p)[1], 0) with struct S (*p)[2][3],
or __builtin_preserve_field_info(p[1], 0) with struct S (*p)[3].
The terminal field-info call was pushed onto the access stack, so
the walk treated it as another array access.
Fix: keep the field-info call off the access stack. Record elements
now get a relocation such as struct S::[3], and scalar elements reach
the existing "Invalid field access" error.
- An anonymous record behind a typedef became a root with no name, e.g.
&(*p)[1], (*p)[1].x or &p[1] with typedef struct { long x; } Anon.
[39 lines not shown]
[AMDGPU] Materialize WMMA SGPR scales to VGPRs before scheduling
v_wmma_scale scale operands accept an SGPR or a VGPR. On GFX12,
depctr_va_ssrc is a 1-deep counter, so each VALU instruction that reads
an SGPR scale takes the only scalar-source slot. A run of WMMA ops that
all read SGPR scales therefore serializes. One cross-bank copy pays for
that slot once; later WMMA ops then read a VGPR and no longer contend
for va_ssrc.
SIMaterializeWMMAScale runs at the end of the machine SSA optimizations,
after SIShrinkInstructions and before scheduling. It is skipped at -O0
and on targets older than GFX12. GFX1250 is included: the separate VALU
read-after-write hazard no longer applies there, but the dependency
counter still does.
The pass walks every WMMA that has scale_src0 and records virtual SGPR
uses of scale_src0 and scale_src1, grouped by register. Physical
registers and subregister uses are left alone. For each SGPR it inserts
a COPY into a low aligned VGPR (the lo256 class matching the scale's
[21 lines not shown]
[SelectionDAG] Fold scalar_to_vector concatenations
DAGCombiner folds `CONCAT_VECTORS` of `BUILD_VECTOR`/`UNDEF` operands into one
`BUILD_VECTOR`, but skips `SCALAR_TO_VECTOR` operands, so each fragment is
lowered separately.
Fold `SCALAR_TO_VECTOR` operands as `[x, poison, ..., poison]`. Integer scalars
are truncated to the common minimum scalar type, as in the existing
`BUILD_VECTOR` fold; FP scalars are used directly. The fold is limited to
fixed-length vectors and runs only before operation legalization.
[mlir][ODS] Allow a space-only separator for bit enums
Bit enums required a separator containing '|' or ',', so a flag set
could not use the bare keyword syntax of LLVM IR, e.g. `nuw nsw`.
Dialects that want it, such as ClangIR's overflow flags, had to hand-write
a parser and printer for an EnumProp.
Allow `separator = " "` on new-style BitEnums. The generated
FieldParser then reads case keywords until the next token is not one,
symbolize splits on spaces, and the op-format `enum` directive parses
the same way. A literal right after such an enum that is one of its
cases is now rejected as ambiguous. Legacy BitEnumAttr keeps requiring
'|' or ','.
Assisted-by: Claude Code (Claude Opus 5.5).
