[NewPM] Port LiveRangeShrink to the new pass manager
Follow the same pattern used for EHContGuardTargets and
CFGuardLongjmp: extract the pass body into a free function, rename
the legacy pass to LiveRangeShrinkLegacy, and add a
LiveRangeShrinkPass for the new pass manager. Register it in
MachinePassRegistry.def, replacing the DUMMY_MACHINE_FUNCTION_PASS
stub, and add the missing include to X86CodeGenPassBuilder.cpp,
which already called addMachineFunctionPass(LiveRangeShrinkPass(),
PMW) in its addPreRegAlloc.
Unlike EHContGuardTargets/CFGuardLongjmp, this pass is skippable
(runOnMachineFunction calls skipFunction), so the new-PM pass uses
OptionalPassInfoMixin rather than RequiredPassInfoMixin, matching
BreakFalseDepsPass.
Update llc-pipeline-npm.ll: the pass now prints its registered
"lrshrink" name instead of falling back to the raw C++ type name.
Add -enable-new-pm RUN lines to the three existing lrshrink tests;
[4 lines not shown]
[Clang] Remove dead code related to atomics (NFC) (#216614)
This PR cleans up dead code in `CGObjC.cpp` related to unaligned
atomics.
Because the synchronization strategy (native vs. objc_copyStruct) is
baked into the ABI for compiled frameworks, it can essentially never be
changed for existing architectures like x86 without breaking backwards
compatibility.
[LoopVectorize] Verify the function once per pass, not once per loop (#216448)
processLoop verifies the whole function once per vectorized loop, making
LoopVectorize quadratic in the number of loops per function on assertion
builds. In an `opt -O3` run the verifier accounts for 59% of the
instructions for a function with 800 vectorizable loops, and 5.6% for
SingleSource/Benchmarks/Linpack/linpack-pc.c.
Move the call to the end of runImpl. Assertion builds still verify what
LoopVectorize produces, but once per function instead of once per
vectorized loop, which costs 0.1-0.7% of the O3 run.
The call was added under DEBUG() in 2012 and became unconditional on
assertion builds in c9f63297e24a. 0aa75fb12faa hit the same problem in
SLPVectorizer and fixed it with EXPENSIVE_CHECKS (#48033).
[NewPM] Port CFGuardLongjmp to the new pass manager
Follow the same pattern used for EHContGuardTargets: extract the pass
body into a free function, rename the legacy pass to
CFGuardLongjmpLegacy, and add a CFGuardLongjmpPass for the new pass
manager. Register it in MachinePassRegistry.def and wire it into
X86's NewPM CodeGenPassBuilder pipeline (AArch64 and ARM do not yet
have a NewPM CodeGenPassBuilder).
The pass only adds post-instruction symbols and never modifies the
CFG, so mark it CFG-preserving in both the legacy getAnalysisUsage
override and the new-PM run() return value, matching the fix applied
to EHContGuardTargets in #217843.
Assisted-by: Claude Sonnet 5
[RISCV] Fix size of PseudoMERGE (#218171)
This can expand to two 4-byte instructions, so mark it as size 8.
Usually it will expand to just one 4-byte instruction.
This was found in #218170.
[NewPM] Port EHContGuardTargets to the new pass manager (#217843)
Adds a newPM pass for EHContGuardTargets (eh-cont-guard-targets).
- Extracts the pass's logic (which has no per-instance state) into a
shared runEHContGuardTargets free function, called by both the legacy
pass and the new pass manager pass.
- Renames the old pass with the "Legacy" suffix, and (matching the same
convention already applied to CFIInstrInserter) renames
createEHContGuardTargetsPass() to createEHContGuardTargetsLegacy() at
all three legacy call sites (X86, AArch64, ARM).
- Adds the new pass manager pass EHContGuardTargetsPass, using
RequiredPassInfoMixin: the legacy pass's runOnMachineFunction never
calls skipFunction, so it always runs unconditionally and should not be
skippable in the new PM either.
- Updates MachinePassRegistry.def and PassBuilder.
- Wires the pass into X86's newPM pipeline, replacing an existing TODO
inside the already-correct TT.isOSWindows() conditional in
X86CodeGenPassBuilder.cpp. AArch64 and ARM have no newPM
[7 lines not shown]
workflows: Merge release-doxygen into release-documenation (#214366)
These two workflows use the same script and have the same structure,
so it's easier just to have one job that builds both.
[BFI] Preserve zero-mass exits in solveIrreducibleMass (#218140)
Fixes #217740.
Keep zero-mass exits in `solveIrreducibleMass`.
Although their mass is zero, these exits are later reused when building
enclosing irreducible SCCs... dropping them can change SCC membership
and eventually trigger the "**unhandled irreducible control flow**"
assertion.
---------
Co-authored-by: Fangrui Song <i at maskray.me>
[RISCV][NFC] Split PreRAExpandPseudo to new file (#218167)
Also update the comments on all the other ExpandPseudo passes, to
clarify where they are run in the pipeline, and that there are many
target-specific pseudo expansion passes, in addition to the generic
passes.
[clang] Trim large(-ish) includes from common headers (#216946)
Several widely included Clang headers pull in whole subsystems to name
types they only use through pointers, references, or declarations.
Because these headers sit underneath SourceManager.h, Sema.h, and
CompilerInstance.h, nearly every translation unit in Clang pays for
them, as well as many down-stream users of Clang.
Translation units across Clang and clang-tools-extra that were relying
on any of the removed transitive includes now include what they use.
Out-of-tree code that was relying on any of these transitive includes
needs the same treatment. The failure is an incomplete type or an
undeclared identifier at the point of use, and the fix is to include the
header that declares it; llvm/Support/VirtualFileSystem.h and AST/Attr.h
account for most of it, since Basic/FileManager.h and AST/GlobalDecl.h
are reached from so much of Clang.
Measured on a random 90 translation unit sample of clang's 1006 library
[4 lines not shown]
[Clang] Fixed an assertion caused by Microsoft integer literals exceeding the maximum value (#212743)
Rewrites the truncation logic for Microsoft integer literals.
Fix #212504
---------
Co-authored-by: A. Jiang <de34 at live.cn>
[ProfileData] Use SortedVectorMap for BodySampleMap, TypeCountMap, and CallsiteTypeMap (#216530)
This patch switches several maps in the sample profile reader/writer
from std::map to SortedVectorMap.
The memory efficiency of SortedVectorMap improves multiple performance
metrics:
Profile merging:
Metric Baseline SortedVectorMap Change
-------------------------------------------------------
Wall Clock 200.22s 157.73s -21.2%
User Time 108.93s 92.79s -14.8%
System Time 70.34s 45.83s -34.8%
Total CPU Time 179.27s 138.62s -22.7%
Peak RSS 58.42 GiB 48.11 GiB -17.6%
Minor Page Faults 28,705,225 17,904,042 -37.6%
Major Page Faults 7 0 -100.0%
[4 lines not shown]
[DomTree] Make addChild/removeChild private, update NewGVN caller (#218164)
NewGVN's value numbering loop converges more quickly when blocks are
processed in a reverse post order. The code additionally ensures a
property the pass does not need (commit 6658cc9ead67 in 2016): in a
preorder of the dominator tree the instructions dominated by a block are
contiguous.
Use the default RPO and drop addChild/removeChild callers, so that the
two members can be made private. `opt -passes=newgvn` slightly
decreases.
[mlir][xegpu] Lower dynamic high-D nd load/store via base-pointer fold (#215711)
Reworks lowering of >2D (batched)
`xegpu.create_nd_tdesc`/`load_nd`/`store_nd`/ `prefetch_nd` (batched
GEMM, rank-4 flash-attention) to keep the full high-D memref as the
descriptor source and carry the leading (batch) offsets as a row offset
into a flattened 2D plane, instead of slicing a per-batch
`memref.subview` during blocking.
For example, the wg-level IR read a 4d vector out of a dynamic shaped
memref.
```mlir
%0 = vector.transfer_read %source[%i, %j, %k, %l], %c0
{in_bounds = [true, true, true, true]} : memref<?x?x8x16xf32>,
vector<2x4x8x16xf32>
```
This lowers with no subview — the full memref is the descriptor source
and all four offsets stay on the load:
[35 lines not shown]
SelectionDAG: Fix widening of vector addrspacecast results (#217898)
WidenVecRes_ADDRSPACECAST unconditionally called GetWidenedVector on the
source operand, which asserts the operand is itself in the
widened-vector
map. When the source vector type does not require widening (e.g. a
<5 x ptr addrspace(3)> cast to <5 x ptr>, where the source is padded
rather than widened), this asserted or crashed. Handle the non-widened
source by padding it up to the widened element count instead.
Co-authored-by: Claude (Claude-Opus-4.8) <noreply at anthropic.com>
[VPlan] Expand sequential/regular UMin SCEVs in VPSCEVExpander. (#209786)
Add support for expanding SequentialUMinExpr SCEV expressions in
VPSCEVExpander.
For regular UMin expressions, the expansion unconditionally expands &
executes all operands, while the semantics of sequential UMin only
require the first operand to be evaluated unconditionally.
For sequential UMin expressions, we need to make sure potentially
UB/poison generating operands must be accounted for. Matching IR SCEV
expander, make sure that divisors of UDiv are poison-free and non-zero
inside sequential UMin. Similarly, freeze all operands other than the
first, to avoid poison from propagating.
PR: https://github.com/llvm/llvm-project/pull/209786
[InstSimplify] Simplify nonzero comparisons involving X urem Y via X u>= Y (#216072)
This teaches ValueTracking that `X` and `X urem Y` are non-equal when a
dominating condition implies `X u>= Y`.
It also handles cases where `X u>= Y` can be proven structurally, such
as when `X` is an `add nuw` of `A` and `Y`.
For a defined `urem`, let `R = X urem Y`. Then:
```text
R == X <=> X u< Y
R != X <=> X u>= Y
```
This allows InstSimplify's existing nonzero reasoning to simplify
comparisons equivalent to `(X - R) != 0`, including:
- `(X - R) u>= 1` and `(X - R) u< 1`
[8 lines not shown]
[MLIR][Python] Add effect and speculatability specifiers for Python-defined ops (#216773)
This PR adds standalone effect and speculatability specifiers for
Python-defined operations.
`NoMemoryEffect` and `AlwaysSpeculatable`, previously nested under
`Pure`, are now public, and `RecursivelySpeculatable` is added. `Pure`
remains a shorthand for attaching `NoMemoryEffect` and
`AlwaysSpeculatable`.
This also exposes `OpTrait::HasRecursiveMemoryEffects` through the C API
and Python bindings as `ir.RecursiveMemoryEffectsTrait`, allowing
region-bearing Python-defined operations to derive their memory effects
from nested operations:
```python
class LeafOp(
TestDialect.Operation,
name="leaf",
[22 lines not shown]
