CodeGen: Stop running LiveVariables in the optimized pipeline (#234174)
Nothing in the optimized register allocation pipeline requires
LiveVariables any more. The pass will subsequently be deleted. The
only tricky part here is dealing with the AMDGPU pipeline previously
using LiveVariables as an insertion point.
Co-authored-by: Claude claude-opus-4.8 <noreply at anthropic.com>
Co-authored-by: Claude claude-opus-4.8 <noreply at anthropic.com>
SelectionDAG: Mark physreg defs dead on scheduler-cloned nodes (#231154)
When the list scheduler clones a node to break a physreg dependency,
the copies share one SDNode, so InstrEmitter can't tell that a copy's
implicit physreg def (e.g. EFLAGS) lost its users and didn't mark it
dead. LiveVariables recomputes it later, but dead flags are most useful
to earlier optimizations.
Instead, mark the def dead at emission time if the copy has no
scheduler edge for it. This lets convertToThreeAddress form LEAs again,
and is a step toward removing LiveVariables.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[compiler-rt][scudo] Add CMake presets for standalone builds (#221089)
Add CMakePresets.json under compiler-rt to configure and test Scudo
standalone without requiring a full LLVM monorepo build. Includes
scudo-debug, scudo-release, and test presets. Used Gemini to clean up
and make the presets more generalized.
Assisted-by: Gemini
[DFSan] Add tests for reaches function argument callbacks (#234011)
With `-dfsan-reaches-function-callbacks`, a reaches function callback
that is marked uninstrumented in the ABI list and registered with the
runtime overflows the stack whenever a stale non-zero label sits in
`__dfsan_arg_tls`. The runtime calls the `dfsw$` wrapper, which reads
argument shadows from `__dfsan_arg_tls` and reports them, re-entering
the callback through the runtime before the callback's own reentrancy
guard runs. Nothing clears the stale label, so this repeats until the
stack overflows.
[CIR][OpenMP] Add support for 'distribute parallel for' with 'target teams'
Adds support for handling the full '[target] teams distribute parallel for'.
Introduces setting the omp.composite attibute for the 'distribute parallel for'
composite construct.
Assisted-by: Cursor / Claude Sonnet 5 High
[CIR] Implement support for MatrixSingleSubscript expr (#232079)
Implement support for the MatrixSingleSubscript expr in the HLSL target
with column-major support.
Issue https://github.com/llvm/llvm-project/issues/221772
[flang][OpenMP] Don't drop NOTHING with clauses in METADIRECTIVE (#233981)
A METADIRECTIVE replacement of NOTHING was treated as having no
directive, even when it had clauses. A selected `nothing apply(...)` was
therefore lowered as a no-op, silently dropping the loop transformation.
Keep NOTHING as a replacement when it has clauses, and report it as not
yet implemented in lowering.
Assisted-by: GitHub Copilot (Claude Opus 5.5)
[OpenMP] Fix construct context at TARGET and repeated-construct subsets
OMPContext has two bugs in how construct selectors are matched. Both can
select the wrong variant:
- The construct context should start at the innermost enclosing TARGET,
but Clang and Flang add every enclosing construct. Inside a TARGET
region nested in a PARALLEL, construct={parallel} matched the outer
PARALLEL. OMPContext::addTrait now drops the constructs recorded before
a target trait.
- A selector that only repeats a construct, such as
construct={parallel do, parallel} over construct={parallel do}, adds no
property. isStrictSubset required fewer properties, so a score tie
between the two fell back to clause order. A shorter ordered construct
sequence now also makes a strict subset. If the shorter selector has a
user condition, or an ISA name the longer one lacks, the tie still falls
back to clause order, since different conditions or ISA names share one
property.
[9 lines not shown]
[mlir] Reduce contention in parallel pass execution and timing (#232948)
Replace per-operation executor scans with stable per-task worker
indices. Allow default timing callbacks to run concurrently using
thread-local stacks and synchronized parent-scope publication. Existing
instrumentations and custom timing managers retain serialized callbacks
unless they opt in.
Benchmark: 200,000 trivial functions with nested CSE, Release with
assertions disabled. Medians of seven alternating samples against main,
which includes #232520. With timing enabled:
| Metric | Before | After |
|---|---:|---:|
| Pipeline wall | 1.0945 s | 0.1699 s |
| mlir-opt wall | 1.7564 s | 0.8329 s |
| mlir-opt system CPU | 15.4775 s | 2.0195 s |
This is a 6.44x pipeline speedup, 2.11x end-to-end speedup.
Assisted-by: Codex
[flang][OpenMP] Set the target in bbc's semantics context (#233976)
bbc created a target machine but never passed its triple and features to
the semantics context. Without a target triple, semantics treats device
and implementation context selectors as possibly matching, so it checked
METADIRECTIVE replacements that can never be selected for the target.
For example, bbc reported a COLLAPSE error for the SIMD replacement of
when(device={arch(nvptx64)}: simd collapse(2)) on x86-64, while flang
accepts it.
Set the semantics target triple and features from the target machine, as
the flang driver does.
Fixes https://github.com/llvm/llvm-project/issues/233027.
Assisted-by: GitHub Copilot (Claude Opus 5.5)
[libcxx][docker] Update android libc++ builders (#232887)
The base images were consolidated in #225214. Move to the new base
image.
Ensure we run as root, since the new base image runs as `gha`.
To test:
$ sudo docker compose --file \
libcxx/utils/ci/docker/docker-compose.yml build \
libcxx-android-builder
Link: #223854
[SLP] Do not seed store vectorization with struct-typed stores
Vectorized struct cannot be stored as a single wide store, exclude such
stores from the store seeds.
Fixes #234176
Reviewers:
Pull Request: https://github.com/llvm/llvm-project/pull/234249
AMDGPU: Stop adding kill flags in SILoadStoreOptimizer (#234189)
No pre-RA pass should try to create kill flags on virtual registers.
They're unused in any pre/mid-RA pass, and stripped by LiveIntervals.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
[AMDGPU] Count WMMA scale copies against both operands
Copying a shared scale while its partner stays an SGPR adds a va_ssrc
issue without removing one. Choose copies from both scale operands.
[AMDGPU] Materialize WMMA SGPR scales to VGPRs before scheduling
v_wmma_scale scale operands accept an SGPR or a VGPR. On GFX12,
depctr_va_ssrc is a 1-deep counter, so each VALU instruction that reads
an SGPR scale takes the only scalar-source slot. A run of WMMA ops that
all read SGPR scales therefore serializes. One cross-bank copy pays for
that slot once; later WMMA ops then read a VGPR and no longer contend
for va_ssrc.
SIMaterializeWMMAScale runs at the end of the machine SSA optimizations,
after SIShrinkInstructions and before scheduling. It is skipped at -O0
and on targets older than GFX12. GFX1250 is included: the separate VALU
read-after-write hazard no longer applies there, but the dependency
counter still does.
The pass walks every WMMA that has scale_src0 and records virtual SGPR
uses of scale_src0 and scale_src1, grouped by register. Physical
registers and subregister uses are left alone. For each SGPR it inserts
a COPY into a low aligned VGPR (the lo256 class matching the scale's
[26 lines not shown]
[AMDGPU] Materialize subregister WMMA scales at lane width
Copy an SGPR subregister scale into a VGPR of that lane's size so two
lanes of one tuple stay independent.
Co-authored-by: Cursor <cursoragent at cursor.com>
AArch64: Skip physical registers in MIPeepholeOpt visitORR (#234194)
Doubt this appears in any real case, but ran into this when auditing
cases which could possibly need to clear kill flags on physical
registers. Also stop using getUniqueVRegDef in an SSA pass.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
DeadMIElim: Require SSA
Require SSA for DeadMachineInstructionElim. This is only used in the
SSA pipeline. The only test using a non-SSA run is apparently testing
LiveIntervals preservation, which it does not use or preserve.
[mlir][tosa] Fix GCC 7 build failure in TosaProfileCompliance (#234239)
Fixes a GCC 7 build failure introduced by 45b9b97b83db (#233949).
Drop noexcept from the move constructors and move assignment operators
of VersionedTypeInfo and OpComplianceInfo to match their implicit
exception specifications.
Assisted-by: Gemini
[AArch64]Increase the cost of SVE integer compares with lower throughput
SVE integer compares write a predicate and issue on fewer pipelines than
Neon compares on many cores (Olympus: 2/cycle vs 6/cycle, Neoverse V2:
1/cycle vs 4/cycle). LoopVectorize picked vscale x 32 for an any-of
reduction over i64 compares, which runs about 2x slower than the fixed
VF 16 loop on both cores. Double the cost of scalable icmp when the
scheduling model reports lower throughput for the SVE compare than for
the Neon compare, the same way as for fcmp.
Assisted-by: Cursor
Reviewers: sjoerdmeijer, sdesmalen-arm
Pull Request: https://github.com/llvm/llvm-project/pull/233469
[llvm-c][ocaml] Accept Module instead of Context in IRBuilder bindings (#234209)
This deprecates `LLVMCreateBuilderInContext` in favor of
`LLVMCreateBuilderForModule`, which takes `LLVMModuleRef` instead of
`LLVMContextRef`.
In the OCaml bindings, `builder llcontext` is replaced with `builder
llmodule`.
This propagates https://github.com/llvm/llvm-project/pull/229430 to
bindings.
[CIR] Wire AMDGPU into the call-convention lowering pass (#220197)
**Summary:**
- Add an `AMDGPU` target to `CallConvLowering`
(`-cir-call-conv-lowering=target=amdgpu`), driven by the LLVM ABI
library's AMDGPU classifier.
- Add an AMDGPU `CIRABIClassifier` subclass and pass the calling
convention through `CIRABIClassifierCommon`.
- Report NYI for byref (IndirectAliased) and indirect arguments in
non-default address spaces.
- Add cir-opt tests for the supported cases and the NYI errors.
Clang pipeline does not select AMDGPU yet. The triple mapping and clang
tests are in #233822, stacked on this PR.
Related to issue: #220471
Assisted by: Claude Opus 4.8
Attributor: Add only new attributes when manifesting
manifestAttrs collected the attributes to add in an AttrBuilder, then
copied the position's attribute set into another AttrBuilder to merge
them in. Collect them in an array and merge them into the set in one
pass. This requires the attributes passed to manifestAttrs to be sorted
like an attribute set, which all callers already do. The only one
passing more than one attribute is AMDGPUAttributor, whose implicit
argument attributes are sorted.
Instructions spent in amdgpu-attributor:
rocPRIM test_device_scan, gfx942: 770.2M -> 710.8M (-7.7%)
Blender Cycles HIP kernel, gfx1100: 380.0M -> 379.5M (-0.1%)
rocPRIM 16 tests, full LTO, closed world: 3019.8M -> 2855.9M (-5.4%)
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
IR: Add attributes from an array to an attribute set
Add AttributeSet::addAttributes and AttributeList::addAttributesAtIndex
overloads taking an array of attributes sorted like an attribute set,
with distinct kinds. They merge the attributes into the set in one pass,
without first copying the set into an AttrBuilder and adding each
attribute to it.
Co-authored-by: Claude Opus 5.5 <noreply at anthropic.com>
